DEPARTMENT OF HEALTH AND HUMAN SERVICES

\+ + + + +

FOOD AND DRUG ADMINISTRATION

\+ + + + +

RISK MANAGEMENT PUBLIC WORKSHOP

RISK ASSESSMENT IN DRUG AND BIOLOGIC DEVELOPMENT

\+ + + + +

WEDNESDAY

APRIL 9, 2003

\+ + + + +

The Workshop met in the NTSB Board Room and Conference Center at 429
L'Enfant Plaza, S.W., Washington, D.C., at 8:00 a.m., Debbie Henderson,
Moderator, presiding.

[PRESENT]{.underline}

DEBBIE HENDERSON, Moderator

STEVEN GALSON, Deputy Director, CDER

ROBERT J. MEYER, CDER

PAUL SELIGMAN, CDER

ROBERT TEMPLE, CDER

ELLIS UNGER, CBER

[RESPONDENTS]{.underline}

ROBERT CALIFF, M.D., Duke University

CHRISTY CHAUNG-STEIN, Ph.D., Pharmaceia, PhRMA

MICHAEL COHEN, R.PhH, M.S., Sc.D., Institute for

Safe Medication Practices

DAVID FLOCKHART, M.D., Ph.D.

GLENN GORMLEY, M.D., Ph.D., AstraZeneca, PhRMA

RICHARD SIMON, D.Sc., National Cancer Institute

[STEERING COMMITTEE]{.underline}

SUSAN ELLENBERG

SANDRA KWEDER

ROBERT O\'NEILL

[PUBLIC COMMENT]{.underline}

CHRISTY CHAUNG-STEIN, Pharmaceia, PhRMA

MICHAEL COHEN, R.PhH, M.S., Sc.D.,Institute for

Safe Medication Practices

GLENN GORMLEY, M.D., Ph.D., AstraZeneca, PhRMA

JOANNA HAAS, M.D., M.Sc., Genzyme Corporation, BIO

JUDITH JONES, M.D., Ph.D., The Degge Group,

International Society for Pharmacoepidemiology

SIDNEY KAHN, Parmacovigilance & Risk Management,

Inc.

PENELOPE K. MANASCO, M.D., First Genetic Trust

GARY STEIN, American Society of Health-System

Pharmacists

PAUL STOLLEY, Public Citizen Health Research Group

ANSHU VASHISHTHA, M.D., PhD.,Watson Pharmaceuticals

A-G-E-N-D-A

Welcome and Introductions

Debbie Henderson, Moderator 5

Dr. Steven Galson, Deputy Director, CDER 10

Session I - Data Generation

FDA Presentation - Dr. Robert J. Meyer,

Director, Office of Drug Evaluation II,

CDER 11

Oral Presentations - Public Comment (Preregistered)

Judith Jones, M.D., Ph.D., The Degge Group for International Society for
Pharmacoepidemiology 32

Joanna Haas, M.D., MSc., Genzyme Corporation, Biotechnology Industry
Organization 43

Glenn Gormley, M.D., Ph.D., AstraZeneca, PhRMA 53

Panel Discussion and Q&As 64

Session II - Special Considerations

FDA Presentation - Dr. Robert Temple,

Dir. Office of Drug Evaluation I, CDER 98

Oral Presentations - Public Comment

(Preregistered)

Dr. Gary Stein, American Society of

Health-System Pharmacists 111

Michael R. Cohen, RPh, MS, ScD, Institute for Safe Medication Practices
115

Anshu Vashishtha, M.D., Ph.D., Watson

Pharmaceutical 128

Penelope K. Manasco, M.D., First

Genetic Trust 136

Panel Discussion and Q&As 144

Session III - Data Presentation/Analysis

FDA Presentation - Dr. Ellis Unger, Division of Clinical Trial Design
and Analysis, CBER 174

Oral Presentations - Public Comment (Preregistered)

Christy Chaung-Stein, Ph.D., Pharmaceia, PhRMA 185

Dr. Sidney Kahn, Pharmacovigilance & Risk Management, Inc. 193

Dr. Paul Stolley, Public Citizen Health Research Group 206

Panel Discussion and Q&As 212

Closing Remarks - Dr. Robert Meyer 266

P-R-O-C-E-E-D-I-N-G-S

(8:09 a.m.)

MS. HENDERSON: Good morning. Welcome to you all on behalf of CDER and
CBER. This is a joint workshop on risk management. As you probably know,
it goes for the next three days. My name is Debbie Henderson. I\'m the
Director of the Office of Executive Programs at CDER. I\'m certainly
pleased to see such a big audience.

I think it\'s important to note, first of all, that we do not wish for
this to be a day of talking heads. The FDA knows what we think. As I\'m
sure you know, we always know what we think. The purpose of this
workshop is to hear what you think. I want to start the entire three
days off by encouraging you to please let us know what you think.

You can do that by coming to the microphone, the microphones that are on
either side of the room, or by submitting questions during any of the
presentations. We will ask your questions for you if you would prefer.
Please do that. There will be support staff circulating around the rooms
every day to pick up your questions if you have them.

As you all know, this is as the result of PDUFA 3 negotiations. FDA
agreed to write a series of guidances related to risk management. All
the aspects of risk management, risk assessment, and so forth. This is
the result of a great deal of work by a great number of people.

I am going to introduce to you on the next slide \-- I think. Ah, there
we go. This is the steering committee for this whole risk management
effort. They will be sitting as they arrive during the day at this table
over here to my left. You can read the names yourselves. These are
senior management within the Centers for Biologics and the Center for
Drugs.

The real work of the papers that we are going to be discussing has been
done by the working groups who have worked on each of the concept
papers. They will be presented to you and introduced to you as each of
the papers are presented.

There are three groups. Group 1 will discuss risk assessment, "Risk
Assessment in Drug and Biologic Development." That will be today. Group
2 is going to talk about the concept paper on good risk management
practices. Group 3 to occur on Friday will be on pharmacovigilance.

Here is our agenda for the day. We are already a little behind so I\'m
going to be quite succinct. We would all love to finish on time and
we\'ll try for that.

Between now and 8:15 you are going to hear welcome from Dr. Steven
Galson, the Deputy Director of CDER. Session I will go from 8:15 to
10:15 including an FDA presentation, oral presentations from our invited
panel, and then panel discussions and questions and answers from you.

We\'ll have a break from 10:15 to 10:30. Session II will go for the next
two hours at which point you will all be happy to know we will break for
lunch. There are many places to eat. I\'m sure you saw them all as you
were trying to wind through the corridors to find this room. There are
just numerous places for you to eat.

Session III will go until 3:00 at which point we will have another
break. Continue with Session III until 4:00 and then hear closing
remarks. Again, we are really looking for your comments.

All of the concept papers that are going to be presented today are
available on the web and here is the address. As well as the transcript
of this meeting which will be available in about 30 days will also be at
this web address.

Besides looking at the things that we have put together at this address,
you can also submit your comments. Again, please submit your comments to
the docket.

As you have probably discovered, no food or drinks are allowed in this
room. There is a room just as you are coming in and to your right as you
are going out where you are allowed to have food, drinks. I am told that
is called the party room so anybody that needs to have a party during
the day, feel free to go back into that room.

The restrooms are located in the registration lobby area. Please note
the exits from the room.

Lastly, I just want to introduce \-- I guess I won\'t be introducing
them because they\'re not all here but I\'m going to tell you who we
have invited for our external panel. We have Dr. Robert Califf who is
the Professor of Medicine at Duke University. Glenn Gormley who is the
Vice President for Clinical Development at AstraZeneca representing
PhRMA.

We have Richard Simon who is the Chief of Biometric Research Branch,
Head of Molecular Statistics and Bioinformatics Section and the Division
of Cancer Treatment and Diagnostics at the NCI. David Flockhart who is a
Professor of Medicine at Indiana University School of Medicine.

Christy Chaung-Stein who is the Senior Director of Statistics and
Programming of the Biostatistics and Data Management Technical Group for
PhRMA. Lastly, Michael Cohen who is the President of the Institute of
Safe Medication Practices.

Today we have invited an external panel that is unique to today. For the
next two days our format will be just a little bit different. There
won\'t be an external panel but we will be hearing from a few of our
panelists who will make formal presentations during the time slot that
is for them and we hope asking us lots of questions in addition to that
otherwise.

Once again, welcome. If you have any logistics questions, Lee Lemley is
the project manager for this entire event and we probably can\'t thank
her enough for organizing this and making it happen, or any of the other
support folks that are out at the desk or you see around will be happy
to answer your questions.

Next I would like to introduce Dr. Steven Galson of the Deputy Director
of the Center for Drugs.

DR. GALSON: Thank you, Debbie. Welcome to everyone. I want to just try
to help get us on schedule so just spend really a minute here and
emphasize a few points.

The first is that the PDUFA 3 agreement and legislation really reflects
a fundamental change in the way the user fee program operates at the
agency. That is that it lets us for the first time spend user fees to
design risk management programs before approval and monitor the success
of those programs after approval.

We see these guidances as allowing us to ramp up our level of scientific
review and oversight and improve the knowledge of what works and what
doesn\'t work with risk management. We hope that this will then be
reflected in design of better programs as we evaluate what works and
what doesn\'t, and overall lead to an improvement of drug safety.

We see these guidance documents as the first step in moving us in that
direction. We hope that they let us achieve an improved level of
transparency so that the industry is very clear \-- industry and others
outside the agency are very clear about what our expectations are with
regard to how the agency evaluates and analyzes risks.

We can\'t do this by ourselves. We really need to achieve high quality
guidance documents and a high quality program in this area. We need
input from all of you so we really ask for your attentiveness and your
comments either in person today or in writing into the docket. The more
input we get, the better product we\'re going to get at the end and that
will contribute to safer drugs down the road.

Thank you very much and I look forward to the next few days.

MS. HENDERSON: Okay. Moving along, our first speaker is Dr. Robert
Meyer. Dr. Meyer is the Director of the Office of Drug Evaluation II at
CDER and he is going to be making a presentation on \-- um, Data
Generation. Sorry Bob.

DR. MEYER: Good morning. Is this on now? Hello. There we go. Good
morning. In the next few minutes I\'d like to briefly take you verbally
through what amounts to about the first half of the concept paper that
my group produced on premarketing risk assessment.

I think the first thing I would like to do is acknowledge the various
members of this team, because this really was a team effort. Each member
of this team has provided substantial input and thought into the concept
paper and will continue to do so as we work towards the final guidance
over the next year, year and a half range.

What you see here with the asterisks are the leads. I\'ve got a co-lead,
Mary Willy from the Office of Drug Safety, and a co-lead, Ellis Unger
from the Center for Biologics. I would like to especially acknowledge
Barbara Gould who is the project manager on our working group and has
provided just tremendous support in not only getting us to this point
but I\'m sure leading us forward as well.

The color code otherwise, the folks in yellow are from CBER. The folks
in the light sort of teal color, I suppose, are from the Office of Drug
Safety. The rest are from CDER in white. At the bottom are some of the
folks from Office of Training Communication, as well as Office of
Regulatory Policy who have also been invaluable in this effort.

Just to briefly outline the agenda today, we are splitting the concept
paper into three different discussions. I will give the first talk, as I
am sure you have already figured out. Dr. Bob Temple from the Office of
Medical Policy will be talking about special considerations of
generating risk information. Then Dr. Ellis Unger from the Center for
Biologics will talk about analyzing and presenting risk information.

For each of these topics we will start with the FDA presentation. Then
we will turn to some scheduled public comments. Then we hope to have a
good public discussion with our invited panelists, as well as questions
from the audience. As Debbie has already said, these could either be in
written format or we do have some floor mikes.

I would reemphasize something else that Debbie said which is that we are
here to give you our sort of broad concepts that we have considered in
working towards this guidance, but we would very much like to know your
impressions on these matters, particularly what we haven\'t stated to
date. What have we missed.

Starting with the more formal part of the talk, risk assessment is the
process of identifying, estimating, and evaluating the nature and
severity of risk from a product. I will use the word product because we
are covering both biologics and drugs in this consideration.

Clearly risk assessment underlies good risk management and good
pharmacovigilance which are the topics of the next two days in sequence.
But we are not going to be talking so much about the risk management and
pharmacovigilance today. We are going to be focusing on risk assessment.

The concept paper itself in this talk in turn does not cover preclinical
or clinical pharmacology aspects of development. Clearly though these
are exceedingly important parts to good risk assessment.

Having a fundamental basis of what some of the likely risks are from the
preclinical development and a good rigorous clinical pharmacology
program that has properly identified the absorption, distribution,
metabolism, and elimination of the drug, as well as likely matters such
as metabolic interactions are clear important underpinnings or, if you
will, foundations to good clinical risk assessment.

But because this is really focusing on the clinical aspects, I\'m
stating those are important but we are just taking those as a given in a
good risk assessment program.

Furthermore, I\'m not going to say a lot directly about efficacy
considerations, although clearly risk always has to be weighed in some
manner against the benefit and a drug with no benefit, which will always
have some risk, would be an unacceptable situation.

Further, since it really is the purview of the following two days,
we\'re not going to be covering post-marketing risk assessment to any
degree today.

The first matter that we dealt with in our concept paper really had to
do with what the appropriate size would be for a good safety database.
Clearly there\'s a number of factors that go into the determination of
what an ideal size is for any given product.

One of the clear considerations in this would be the novelty of the
product. If you are dealing with a product that has a mechanism of
action that has never been really tempted in a therapeutic before, has
no relationship to prior medications or prior biologics that we have
experience with, you ideally want to have a larger database to better
delineate the risk of that product.

The proposed indications play into it as well. Obviously with a
life-sustaining product, one that has a substantial impact on serious
morbidities and/or mortalities, one might be willing to accept more
level of risk and/or uncertainty than you would be for something that
might be targeted at treating chronic symptoms, for instance.

The intended duration of use is also important if it\'s a chronically
administered drug, and particularly if you are worried about the
possibility of some time related events, then you would want to have not
only a fairly large database but you would want patients to be exposed
over a long period of time.

Finally, the ideal size would be dependent on what sort of information
came from some of the earlier preclinical studies looking at the
toxicology in animals, as well as some of the early clinical findings
from your Phase I and Phase II trials.

Now, the ICH guidance on these matters, which is E-1, does define what
would be in the opinion of the ICH a reasonable database to delineate
safety of long-term treatments. By long-term they are referring to
either chronically used treatments or treatments with intermittent
courses but that are likely to be recurrent. This guidance also refers
specifically to the non-life threatening conditions.

In that guidance the ICH calls for at least 1,500 patients total, with
300 to 600 exposed for six months, or 100 out to a year. Now, with
regard to the 1,500 patients, the ICH E-1 does not tell us who would
constitute those 1,500 patients. Clearly in any drug development program
you have exposures from single-dose trials first in humans, all the way
up to for chronically used drug patients who are exposed to many months
to years.

The working group in putting together this concept paper thought that
for chronically used drugs, the ones that are referred to in ICH E-1, it
really would make sense for this 1,500 patient number to come from
multiple-dose studies. We thought a reasonable duration would be those
exposed for four weeks or more.

Furthermore, we also felt that these 1,500 patients should be exposed to
doses at least equal to the lowest proposed dose or that their systemic
exposure would be comparable to that that is obtained from the lowest
proposed dose.

In other words, we would like these data from these 1,500 patients to be
relevant and informative for the decision at hand as far as
approvability and labeling.

I should say although this didn\'t overtly come up in the paper, that we
didn\'t talk about different routes of administration here but clearly,
I would say, for systemic toxicities if there is a different systemic
route of administration, that may suffice for covering the 1,500
patients. Again, the exposure level would have to be appropriate in our
view.

Now, some may see the 1,500 patients coming from multiple-dose studies
and from relevant doses as being somewhat of a racheting up of what
we\'re asking for. But I just want to remind folks that if you take the
rule of three, that tells us that in a safety database of approximately
1,500 patients, we would have a reasonable chance of identifying (a) one
particular event that occurs at a rate of 1 in 500.

Yet, we also know the drugs commonly are used by thousands to millions
of patients shortly after release. We don\'t think that this level of
assurance is unreasonable by any means.

Now, the ICH document itself turns to situations where these targets
that they have identified may not be enough to support the finding of
safety or the risk assessment of a product.

The circumstances that the ICH document identifies, and this is
paraphrased, are if there is a cause for concern for time related
effects on safety. In other words, if we have a substantial concern that
there is a chronicity of dosing needed before an event shows up, then
you may need more than those 600 patients for six months and more than
100 out to a year.

Clearly, if there was a need to quantify low-frequency events, the
safety database that we\'re talking about, and I think my example of the
rule of three shows this, may not be sufficient to sufficiently do that.

The ICH also mentions that if there\'s limited or unknown efficacy. To
be perfectly honest, I\'m not entirely sure what they mean by unknown
efficacy but I think what I would take as the spirit of this is that if
the efficacy does not seem to be overwhelming, you don\'t want to have
much risk or much uncertainty in those circumstances. In the document
they suggest this would be a reason why you might go beyond these
guidelines for sizes of databases.

Then where there is a concern that a product may add to a background
rate of morbidity or mortality. In other words, if you had a drug, for
instance, a thrombolytic that in some ways actually led to thrombosis or
perhaps some CNS bleeding.

Now, we\'ve added to, or perhaps taken somewhat of a nuance on some of
these in our draft concept paper. We feel that preventive treatment also
falls into a category where we would want to be more certain of the risk
and have less uncertainties at the time of approval and labeling.

If you are entering patients who are basically healthy into a long
course of therapy, you would like to be reasonably sure that you are not
going to be doing more harm than good.

Another situation where we feel that we would not tolerate a lot of
uncertainty about the level of risk is where there is a very safe
alternative that already exist.

In other words, if a product is being developed that doesn\'t seem to be
a major therapeutic advance but merely an alternative to other therapies
already available and those other therapies or therapy are quite safe,
you would not want to tolerate a lot of risk in proving that new
product.

Finally, and I think this is a somewhat difficult one to clearly
delineate as far as when this would occur, but if there is a potential
for a large market and a very fast uptake into the marketplace where you
are going from what you know about the drug from a safety database to
exposures in millions in a short period of time.

Particularly where that is for a non-life threatening condition, you
might wish to have a larger safety database going in. As an example, and
I just use this as an example, something like erectile dysfunction. We
know from recent past experience that that drug, the particular drug
that is approved for that, has had a very rapid uptake in the market.

If subsequent drugs were to be developed for that kind of condition that
is sort of an important but lifestyle indication and will likely lead to
millions of patients being exposed in a short time frame, you might want
to have more certainty of the safety and a better risk assessment going
in.

Now, one of the things that the concept paper poses is a question, and I
would pose as a question to you folks for comment after we\'re done
here, is for acute use therapies and/or those for life-threatening
diseases, neither the ICH nor the FDA has any guidance in existence at
this point.

One could say for life-saving products anyway that are approved on the
basis of mortality trials that perhaps we know enough at that point to
make an acceptable risk benefit determination. If you show an effect on
mortality, I think that you might argue reasonably that the risk, even
if it\'s somewhat undefined, is acceptable because you\'ve shown a
demonstrable effect on the endpoint of interest.

Also, I would note that in accelerated approvals the definitive efficacy
and the full safety may not come until later when you have your
definitive efficacy trials.

Now, what are some of the other characteristics of an ideal safety
database. We\'ve dealt with size. Now let\'s turn to some of the other
things that we have put forward in this concept paper which are, in some
cases, somewhat novel.

The first one is controlled trials performed throughout. Commonly the
late stage trials for safety are open-label trials where patients are
only exposed to the drug of interest and we don\'t either have a placebo
or a positive control or an active control to make any inferences on.

I think that the opinion of the working group is that these trials are
frequently not particularly informative. We would argue that one would
get a much better idea of the risk of a new product if controlled trials
were performed throughout including the long-term safety trials.

We believe that having controlled trials performed throughout is
essential for detecting treatment relation to common outcomes in the
population. It also helps address confounding by indication.

For instance, if one is testing a new asthma medication, inhaled asthma
medication, it\'s very difficult to tell whether asthma exacerbation
seen during an open-label safety trial arcuation itself or due to the
disease.

In other words, they could just be, if you will, almost a treatment
failure, or they could be directly caused by the drug itself. Without a
control group one cannot draw those inferences in a meaningful fashion.
Furthermore, with an active comparator there\'s an opportunity to judge
safety versus an accepted or approved therapy.

Another characteristic of an ideal safety database, in our opinion, is
to enter a diverse population. While I think the general impression is
that we and the sponsor are getting better about obtaining databases
that are from diverse populations. There is still a fairly substantial
way to go in this regard.

We would argue that only patients with obvious contraindications should
be excluded from late-phase trials. Doing so would allow for better
generalized ability of the safety findings and, more importantly, it
would develop better clinical data on potential treatment versus
demographic treatment versus concomitant disease or treatment versus
other treatment interactions with regard to safety.

Another attribute of an ideal safety database in our opinion is the
range of doses should be studied throughout development. There is a
common misperception, I guess would be the best way to term it, that
dose finding is, and should be, complete at the end of Phase II.

Frequently what you know at the end of Phase II is based on
pharmacodynamic endpoints and fairly short exposures. We argue that you
would much better characterize clinical exposure response relationships
not only for efficacy but importantly for safety in continuing to study
a range of doses throughout drug development.

Not only could you generally get a better exposure response data, but in
doing so you might be able to provide important advice for labeling for
dose adjustments in subpopulations. Otherwise you would have
insufficient data to make inferences about whether dose adjustments
would help with a particular adverse event. Or, for that matter, with
efficacy findings.

We feel that doing a range of doses throughout clinical development
would also provide important information on the assessment of efficacy
which, again, you are always balancing efficacy and risk and provide
opportunities to maximize the benefit versus risk determination.

Another thing that we think is useful in developing a good risk
assessment of a new product is for potential interactions that were not
previously identified in the clinical pharmacology studies to be
specifically looked for. Sponsors should be vigilant for drug/drug
interactions.

Particularly for likely concomitant medications. This gets back to the
diversity issue. We are arguing that you should allow a diverse set of
patients in including patients taking likely concomitant drugs.

An example here would be many patients who are on HMG CoA reductase
inhibitors are also on binding resin so I think it\'s very useful for a
sponsor of a new HMG CoA reductase inhibitor, for instance, to do
specific studies or to allow patients with binding resin as a
concomitant medication into the studies and to do specific subgroup
analyses of these.

The other thing that is useful to look for then is product demographic
interactions in these diverse populations. How are the elderly faring?
Is there any gender effect? Are there any racial or ethnic effects? How
are the pediatric patients faring?

We also think that the sponsors should be looking for product disease
interactions. That really means studying a range of diseases, not just
patients with moderate disease or mild disease but a full range of
diseases, and concomitant disease patients. In other words, patients
with diabetes frequently have heart disease. Don\'t eliminate those
patients from the trial unless there\'s a specific contraindication to
doing so.

Product food interactions is another thing that can be and should be
looked for in a database over time. Product dietary supplement
interactions is also an increasingly common concern since many dietary
supplements have potential drug interactions and because the dietary
supplements are becoming increasingly common.

Particularly for some diseases it\'s quite predictable that patients
will be taking these supplements during therapy. We would argue that you
should not eliminate the use of those supplements during the trials to
the extent possible.

I think one way that the working group has thought it would be important
to add to the safety database or to the risk assessment is by providing
data that looks for unanticipated interactions because the clin-pharm
studies, no matter how well done and no matter how well thought out, do
not always predict all the nuances of drug metabolism, particularly
drug/drug interactions or treatment interactions.

We think, therefore, that it is good to perform levels throughout the
drug program including getting some assessments during Phase III trials.
For instance, on this would be doing some population PK. Doing this
could help better define the exposure response relationship for safety
and efficacy and I would argue may help validate biomarkers.

A lot of talk about smarter drug development has revolved around the
efficacy side and efficacy biomarkers, but I think safety biomarkers are
somewhat under-appreciated or perhaps underdeveloped. Having these kind
of data would not only add to perhaps providing biomarkers for efficacy
but importantly may do so for safety.

It also could help define drug level relationship to any unusual adverse
event or, for that matter, important adverse event. If an adverse event
is seen and you have some PK data from the Phase III trials, one could
try to ascertain whether the patients who experienced that adverse event
actually are experiencing higher exposure levels as well.

One other thing that we commonly do not see, and I guess this would go
along with perhaps doing controlled trials throughout the clinical
development would be comparative safety data. There are some instances
where comparative safety data may be particularly useful.

I would note that it is expected with some classes of products. For
instance, the preventive vaccines. Those programs currently are expected
to have comparative safety data or a positive control throughout.

When there is a need to characterize the background rates of events,
particularly as they occur with the products, or in relationship to the
products, we think that a comparative safety data would be useful in
those circumstances.

We also think it would be useful where there is a well-established,
well-characterized, low-toxicity product. Where there is a
well-established related therapy, so if you\'re doing a second in a
class, third in class, or perhaps another treatment for a disease that
is going to be perhaps provided either in addition to or perhaps as an
alternative to an established therapy, having comparative safety data
would be particularly useful in those kind of circumstances.

Clearly where there is an established therapy with an effect on survival
or alternating irreversible mortality, comparative safety data is
important because that\'s really getting at both the efficacy and the
safety. A mortality disadvantage is both an efficacy issue and a safety
issue for those sorts of drugs.

I would state that commonly the ethics of those kind of studies would
dictate a positive control in any case, but we did want to highlight
this even where the ethics might allow for a placebo controlled trial.

I think the last one is fairly obvious, when there is a wish to make a
comparative claim. The reason we put this up here is because a
comparative claim, in our view, should not just be about efficacy, but
it should be about the safety of the product as well. A gain, or
apparent gain, in efficacy that\'s counterbalanced by an increased risk
may not be a gain at all.

Finally, just to conclude my remarks this morning, premarketing risk
assessment as opposed to perhaps tomorrow\'s topic of risk management,
and perhaps even to some degree the pharmacovigilance talk the third
day, is relatively mature but it is clearly still evolving. We have our
thoughts. We would like to hear your thoughts on ways that risk
assessment in the premarketing phase can be better performed.

I think we would all agree that public health industry and FDA would all
benefit by optimizing risk assessment. This would allow for approval of
safe drugs with fully, or as reasonably fully, as possible informative
labeling. Thank you.

MS. HENDERSON: Thank you, Bob. Just to remind the audience and those of
you who have come in late of the order of our agenda. We will be having
for each of the presentations an FDA presentation followed by some
public comments from speakers who were preregistered to speak.

We will then have a panel discussion and questions and answers from our
invited panel, or questions from the steering committee, or even the
working group if there are questions. Then we\'ll have questions from
the audience.

Next on our agenda for this will be Judith Jones from the Degge Group
for International Society for Pharmacoepidemiology. Is she here? Ah,
there you are.

She will be followed then by Joanna Haas. Then lastly in the
preregistered comments from Glenn Gormley.

DR. JONES: Thank you very much. It is a pleasure to be here, panel
members, audience. My name is Judith Jones and I am here representing
the International Society for Pharmacoepidemiology. It\'s a pleasure and
honor to provide comments on the concept paper on premarketing and risk
assessment on behalf of the International Society for
Pharmacoepidemiology, otherwise known as ISPE.

ISPE is a nonprofit international professional membership organization
dedicated to promoting the science of applying epidemiologic approaches
to the use affecting this value and safety of therapeutics.

The society provides an international forum for sharing knowledge and
scientific approaches to foster the science of pharmacoepidemiology.
ISPE has over 700 members representing 45 countries. Our members work in
academic institutions, the pharmaceutical industry, and government
agencies, nonprofit and for-profit private organizations.

Specific backgrounds of the membership include epidemiology,
biostatistics medicine, nursing, pharmacology, pharmacy, law, health,
economics, and journalism.

The following comments are based on the feedback provided by senior
members of the society including the executive committee, the board of
directors, and past presidents.

What I would like to do is make some general comments. I have used a few
slides just to emphasize the major comments. This concept paper is a
very useful summary of the agency\'s views on many possible approaches
to premarketing risk assessment and it will be helpful for guiding the
overall clinical development plan. We think it makes a major step
forward.

Although it is briefly mentioned and alluded to, and also in Dr.
Meyer\'s presentation, it is somewhat surprising that this concept paper
does not link more closely with both the premarketing and clinical
pharmacology activities, as well as the companion documents on risk
management and risk assessment.

The activities in the premarketing period, including some of the key
preclinical activities, and certainly clinical pharmacology studies,
play a major role in identifying and characterizing the priorities in
planning for risk management of a product.

All of these activities should be integrated into a risk management plan
interpreted in the broader sense, not so much as the risk management
outlined in the Concept Paper II over the entire life of the product
beginning prior to entry in clinical trials.

Unfortunately, such a lack of linkage in this may create the danger of
actually consolidating institutional divisions between the premarketing
and post-marketing approval period. Ideally there should be integration
of science in special areas from early development through
post-marketing.

I would like to make just a couple of comments about the risk assessment
concept, drawing on the Food, Drug, and Cosmetic Act, which basically
requires that there is a comprehensive conduct of all tests reasonably
shown to evaluate a drug\'s safety.

It\'s important to remember a couple of things. In the first place, at
any point in time safety is a judgment. It\'s made at a specific point
in time and based on information available. Even the most rigorous
premarketing program cannot identify all risks that may occur when a
product entered the market.

Nevertheless, the concept paper outlines the possibility of expecting
very extensive explorations of risk to support this judgement. Given
that there is a long track record, it may be very important for the
agency to evaluate the basis for its judgement made so far on the data
available to begin to determine just where the level of
comprehensiveness must be.

With regard to Section 3 and the various considerations and generating
risk information, the size of the database as discussed by Dr. Meyer is
very important. I won\'t basically reiterate what I have written and he
has gone over about the rule of three which, in fact, shows that in many
cases despite long-term exposure there will not be sufficient power or
data to determine the risk of interest.

With respect to risk on acute use, other than supervised use within the
hospital, the size of the database might be best informed by
understanding the likely modes of use after marketing by prototypic
indication populations. Even labeling and packaging for short-term use
may be ignored by prescribers and their patients as is the case of
analgesics for acute self-limited pain.

For several other recommended risk assessment activities in this concept
paper, including medication errors, we believe it would be useful to
develop a spectrum of scenarios on how a drug will be utilized in the
real world, including the likelihood of using larger and smaller doses
by the indication population. One can draw on a growing set of
epidemiologic resources to do this.

Also, the paper indicates that a larger database would be useful if
safer alternatives to the investigation product are available. It\'s
important to remind us that it will be necessary that we develop a fair
definition of what an alternative is, but also criteria for safer.

Since many established products have not had the scrutiny or risk
assessment that may result in premarketing risk assessments going
forward that address the concepts of this paper.

With respect to the long-term controlled studies, the notion of having
more controlled data we think is a very important concept. Preferred
comparisons would be from randomized, even blinded studies. Further
studies would benefit from additional review by data safety or drug
safety monitoring boards since in such studies rare events continue to
be difficult to evaluate.

With respect to dose ranging, obviously a better understanding of
exposure response relationships is clearly helpful in assessing benefit.
But it would be very important to define how useful broadening of the
range of doses will be to understand all but clearly common dose related
risk.

There is obviously limitations in power. However, understanding may be
further enhanced by the concomitant use of pharmacokinetic measures in
the trial as recommended elsewhere in this concept document and actually
mentioned by Dr. Meyer.

With respect to drug interactions and the recommended for this, the
possible type of drug interactions listed underline the fact that for
any therapeutic agent there are myriad possibilities for interaction and
it is unlikely that all these possibilities could be explored in a
reasonable clinical program.

That said, certain things could help focus this report. First, conduct a
natural history of the indication population to determine the most
common possible interactions using epidemiologic databases combined with
reasonable pharmacologic pharmacokinetic assessment of the likelihood of
those interactions.

Secondly, it would be important to design trial and adverse reaction
protocols and training of investigators to assure that there is a
sidious collection of data that might reveal an interaction in the event
of an adverse event.

Back to comparative safety data, the need is obviously well recognized
and there is a need to develop their concepts on how the comparisons
will be made. One of the most difficult issues will relate to the
situation where two agents will have very comparable benefits but, in
fact, have very different, or certainly key different adverse events.

For example, in the agent that might have irreversible renal failure as
opposed to irreversible hepatic failure and how those particular
concepts are balanced.

Now, with respect to the special considerations that will be discussed
this afternoon, the recommendations are very very broad. If they were
required for all products, we would be prohibitive. Therefore, the hope
is that the needs for such studies will be directed to clear areas of
public health concern where it can be reasonably assured that the
addition clinical data will provide a clear basis for better decisions.

For example, the large simple safety study is very useful for
understanding the risk and diverse populations. However, it\'s very hard
to maintain simplicity if questions over risk are not well defined and
the results lead to continued uncertainties as is often the case.

We would recommend that if large simple studies are conducted, it should
be with mutual agreement on these possible uncertainties and resulting
actions before launching such trials wherever possible.

With respect to medication errors, since a large part of clinical
development in the premarketing period is conducted under conditions not
analogous to usual use, much of the experience in clinical studies is
not useful to inform of possible medical errors.

To predict medication errors, it may be necessary to develop detailed
time-motion scenarios of how a product is selected, prescribed, ordered,
and used by a patient, although some clues may be derived from the
studies in the indication population sometimes using epidemiologic or
other observational databases and their use of comparable drugs to
determine the potential for medication errors. We have a lot to learn in
this area.

With respect to the data analysis the concept paper provides a very
useful outline for these analyses. However, as noted, an overall risk
management plan that starts at the outside of clinical management can
facilitate data pooling by assuring that all collections of data are
standardized and analyzed using similar terminology and term groupings
throughout the process of development. This is ideally done a priori.

Regarding pooling and use of person time, this recommendation is good
but not always a good one. It would depend on the event of interest and
would not necessarily apply to all idiosyncratic reactions.

For idiosyncratic events that occur uniquely during early exposure to
frequency estimate, we might wish to use number of people as the
denominator.

In conclusion, this paper provides an array of possible ways in which
the risk of a product can be assessed. It\'s a very major contribution.
However, it\'s not really clear how to balance the recommendations in
this document and how much can be done in the premarketing versus
post-marketing period.

For example, what are the tradeoffs for a large safety study during
development versus a more extensive safety program during the
post-marketing phase. There are no easy answers to this question and it
may even require a separate guidance document.

However, this document does describe all possible safety assessments
which might be done during development and provides little guidance on
which circumstance FDA would recommend for applying many of the pieces
described.

In part this can be remedied by better integration of risk assessment
and risk management from the outset of development. In the best of
possible worlds product development with a risk management perspective
is an interactive and informative process with greater experience and
overall therapeutical development and regulation we believe should
improve with time.

The International Society for Pharmacoepidemiology is firmly committed
to providing an unbiased scientific forum to the views of all parties
with interest in the safety of therapeutical and, as such, is deeply
committed to the advance of risk management sciences.

We welcome the opportunity to work with the agency in this area and we
will engage our full membership and the feedback process of this concept
paper as will be demonstrated in the next two days in our comments
submitted.

Our next annual conference will be risk management. Several workshops
and sessions are being planned jointly with FDA staff. I take this
opportunity to invite you all to join us at the combined first
international conference on therapeutical risk management at the 19th
International Conference on Pharmacoepidemiology held in Philadelphia,
August 21st through 24th.

Thank you very much.

MS. HENDERSON: Thank you very much, Dr. Jones. Before we go on, for
anybody that is standing in the back, unless you prefer to stand in the
back, of course, there are plenty of seats up front so feel free to just
come on down and take a seat.

Our next speaker is Dr. Joanna Haas from Genzyme Corporation. Dr. Haas.

DR. HAAS: Good morning. I\'m speaking on behalf of BIO, the
Biotechnology Industry Organization. We are pleased to have the
opportunity to take part in this risk management public workshop.

BIO represents more than actually 1,000 biotechnology companies,
academic institutions, state biotechnology centers, and related
organizations in all 50 U.S. states and also in 33 other nations.

BIO members are involved in the research and development of health care,
agricultural, industrial, and environmental biotechnology products.
Today we would like to contribute our perspectives to the concept paper
on premarketing risk assessment.

BIO member companies largely by the nature of the industry develop
innovative therapeutic products, many of which are for serious disorders
with substantial unmet medical needs. These include ultra and
ultra-orphan diseases.

Given the range of indications in study treatments that BIO member
companies are involved in, we wish to emphasize the importance of an
empirical approach to risk assessment that tailors efforts to the
specifics of each developmental program.

Such a paradigm for risk assessment can best fulfill the shared goal,
shared among industry, Government, and the medical community, of
optimizing the risk benefit relation of new therapies without causing
unnecessary delays in access to effective new medicines.

Each of the individual actions in the series of concept papers has great
value in specific situations. However, if used in settings in which they
are not applicable, they have the potential to increase the uncertainty
and the duration of clinical development without a corresponding benefit
to patients. Thus, the choice of tools for risk assessment in all phases
of development should be carefully crafted and optimized to the specific
circumstances.

Already the crescendo of scientific advances in molecular biology,
immunology, pharmacogenomics and other disciplines is being transmuted
into a myriad of new therapeutic targets and biotherapeutic agents.

Effective risk management strategies should facilitate the development
of novel and effective products. Our obligation is to understand the
risks associated with \-- our obligation to understand the risks
associated with new therapy should not become an unintended obstacle to
weaken the benefits of this new knowledge.

Particularly in the BIO technology industry young and innovative new
companies made special contribution to the translation of new science
and new treatments. Clearly the standards of efficacy and safety,
products must be the same regardless of the characteristics of the
sponsor.

However, risk assessment standards, if they increase the obstacles to
approval without insuring a better product, can be particularly
problematic for such young companies. Fitting the risk assessment
strategy to the product under development addresses this question.

Now, actually before moving on to the questions of what is an ideal
safety database, we also wanted to add that the concept paper,
premarketing risk assessment, includes premarketing developments of both
biological products and drugs. The important safety concerns for drugs
such as QT prolongation are not generally relevant to biologic products.
On the other hand, immunogenicity may need to be addressed for all
biologics.

Given such differences, one must be cautious about developing generic
risk assessment strategies to be universally applied. BIO hopes that
future guidance will further define how differential risk assessment
approaches may be developed for different types of products.

Throughout the documents discussed at the workshop the importance of
evidence-based approaches and the use of validated methods for risk
management programs is emphasized. When designing premarket risk
assessment strategies, this same principle should apply.

Whenever a tool for improved risk assessment is employed, there should
be good reason to believe that it will provide relevant, interpretable,
and pertinent information. Focused differentiated approaches to risk
assessment should be designed to delineate the risks of interest for the
specific product and indication under development.

Risk assessment strategies are a means to an end and should be
continually evaluated from that perspective. They reduce uncertainty
about risks by providing relevant information. Well-designed risk
assessment strategies can provide reassurance about the product safety
profile. Where clinically significant safety issues are identified, risk
assessment provides the basis of designing a risk management strategy.

The premarketing assessment paper poses the question what is an ideal
safety database. It affirms that the composition of appropriate safety
database for a new product would be determined on a case-by-case basis.
This subsequent discussion, however, of the ideal for what \"all
programs\" would include proposes three features of an ideal safety
database that we\'ve heard discussed earlier this morning.

Each feature has the potential to increase the number of patients
included, as well as the duration and complexity of clinical
development. Some of these features may not be applicable to all
programs. It is important that these features not be seen to represent a
de facto minimum standard for a product safety database.

BIO suggest that the concept of an ideal safety database should be
nested in the context of the specific program rather than be developed
as an abstract stand-alone notion. Although this is not the forum for an
extended discussion, three suggestions regarding premarketing risk
assessment deserve comment. Let\'s discuss each point briefly.

Firstly, long-term controlled safety studies are suggested. While the
paradigm of the randomized controlled clinical trial may be the ideal to
address certain concerns, in practice this may not be possible for
several reasons.

Experience with patient recruitment and compliance has shown that such
studies are difficult to conduct. It may not be possible to recruit and
maintain a large enough patient population to address the issue of
concern.

Interpretation of such data is likely to be impaired by dropouts and by
missing data. In situations where there is a major unmet medical need
and where efficacy has been demonstrated, ethical reservations may be
raised.

Should there be data driven concerns about a specific safety endpoint
and appropriate targeted post-marketing risk management plan using, for
example, pharmacoepidemiologic methods of registries may be more
feasible than an effort to generate long-term controlled safety data
marketing despite the theoretical advantages of the RCT.

A second proposed feature of an idea database for all programs is
diversity. A diverse safety database, it is pointed out, is likely to be
more representative and more readily generalizable to the post-marketing
population.

While this is true, generating a diverse safety database has substantial
implications for product development. Inclusion of a more diverse
population in pivotal studies makes it more difficult to demonstrate
efficacy. Such groups have more confounding factors such as concomitant
diseases and medications. They may be less compliant leading to more
steady dropouts and missing data.

The consequence of each of these factors is to increase the number of
patients to be studied in order to demonstrate efficacy or its absence.
A consequence is that patient exposure to an ineffective medication
might under some circumstances be increased.

A diverse safety database, on the other hand, might not always provide a
better basis for assessing the risks in the expanded population. The
numbers of patients in each subgroup of the diverse population may well
be too small to assess subgroup specific risks. Indeed, the risks
detected would have to be high indeed for them to be measurable in the
premarketing setting.

A further potential hazard of this approach is that the heterogeneous
population might generate so much noise that real safety signals could
be obscured. Thus, a diverse safety database is not necessarily a
characteristic of an ideal safety database for all products.

The third suggested characteristic of an ideal safety database is one
that would allow comparison of safety profiles at different doses. Where
the safety database generated by the clinical development program
already includes patients treated at different doses, data analysis
strategies exploiting that information can be very useful.

On the other hand, one would generally not design a clinical program for
the purposes of comparing safety profiles at different doses since the
primary endpoint is efficacy. Phase III studies are neither designed nor
powered to distinguish between the vague concept of levels of risk at
different doses.

Adding additional study arms in Phase III clearly increases the number
of patients and correspondingly increases the duration of the studies
and the required resources. The ability to recruit patients is commonly
the limiting step in clinical development.

In orphan indications this is an inherent constraint. Thus, while having
patients at different dose levels may be useful in a safety database,
one should be very cautious about suggesting that this is an ideal to be
achieved through all products.

The second part of the concept paper, Section G on data analysis and
presentation, offers creative strategies to identify and delineate
specific safety issues. These suggestions open the way to better
understand and utilize data collected during clinical development.

This approach facilitates early identification of safety concerns by
perspectively defining safety issues during clinical development
available and accruing safety data can be organized and coded to cast
light on specific issues.

These are efficient and prudent approaches that are tailored to address
specific safety problems. This approach also facilitates the early
development of a problem specific risk management strategy to be
identified in the post-approval or peri-approval period.

In conclusion, BIO looks forward to additional guidance from the FDA on
risk assessment based on a paradigm emphasizing empiricism,
evidence-based strategies, and the development of approaches that are
crafted to the unique situation that each product development
undertaking represents. This approach will help to ensure that
innovative, effective, and safe biological products continue to be
developed to meet serious unmet medical needs.

Thank you very much for the opportunity to make these comments.

MS. HENDERSON: Thank you very much, Dr. Haas.

The third speaker for the session is Dr. Glenn Gormley. We have to do a
little manipulation of computers here so it will probably be a minute.

DR. GORMLEY: That\'s easier than it usually is when you have to switch a
computer. Terrific.

Well, thanks very much for the opportunity to make some comments on
behalf of PhRMA. Bob has tied up a lot of the issues for today on the
paper. You are going to hear that the collective comments that I\'ll try
and represent this morning from PhRMA are very similar to the ones
we\'ve just heard on this paper from several other groups.

I thought I would start with a few general comments and then evolve into
a few specifics and point out up front that PhRMA has gathered a lot of
comment from its representative companies and from many people
interested in giving us their views. We are electing not to provide a
comprehensive review at this point on the paper.

We are going to point out a few of the issues that are of particular
concern to us and we anticipate that the open dialogue and forum for
today is going to bring out the rest of those and there will be other
opportunities for us to get into more detail.

So starting with a few general comments, I think it\'s important for us
to state right up front that PhRMA collectively as a group very much
embraces the importance of risk management. We share the same interest
as the FDA and the public in bringing safer and more effective drugs to
market as rapidly as possible.

There is a concern, though, that the paper that we\'re dealing with
today has a lot of very reasonable comments, lot of reasonable
suggestions on how you would manage risk. But there is little clarity on
how we would incorporate those points into specific development
programs. In that sense, and we\'ve heard this theme before, it\'s a
good document defining concepts but it\'s not a great document defining
guidance.

We think that is actually intentional on the part of the FDA. They have
left this at a concept phase because they wanted to have an open forum
such as this. They wanted to hear comments before they drill down into
any form of guidance. We are taking it in that spirit and will address
some of the concepts that we think need some further clarification.

Probably the most significant issue for us as we look over the document
as a whole is that there are a number of sections in Concept Paper I
that if they were taken very literally or out of context could
potentially lead to a change in the standard for approval of new drugs
or new indications.

We don\'t think this is the right forum for changing the standard of
approval. We think this is a forum for getting an understanding of what
it would take to support a good risk management program. We think that
is the spirit that the FDA is taking to this as well, but we would push
back some of the points that were taken literally and were intended to
change standards of approval.

As another general comment, and it has been made before as well,
there\'s not a lot of evidence provided in the paper that the strategies
that are being proposed will, in fact, lead to safer drugs reaching the
market compared to current standards. We don\'t want anyone to interpret
that as us not being supportive of the concept because we certainly are.

As we go forward in implementing new ideas and putting together what we
think are better risk management strategies, we need to be very careful
to measure those and make sure that we are doing more benefit than harm.

So with those few general comments, I\'m going to drill into a few
specifics in the paper. For instance, Section 3(a) which deals with an
appropriate size of a premarketing safety database. I referred to
several lines in my presentation in case you want to look those up in
the concept paper as we go along or later in the day.

Somewhere around line 79 there is a discussion about increasing the size
of the required database, potentially above ICH guidelines. Again, I
referred to it before. We are concerned that if that becomes either
arbitrary or expected in all development programs and that it\'s not
directed at some specific goal or objective, that it\'s not going to
provide sufficient reassurance for patient safety or give us good
guidance.

Where there is a specific concern or where there is an issue that will
require an increase sample size in order to have more confidence we are
fully supportive. My colleague Christy Chaung-Stein will talk about that
later today and put some clarity around our concern.

For instance, doubling the size of a database because you have a concern
about an issue won\'t necessarily give you more reassurance unless you
have an understanding of what you are trying to accomplish and the rule
of three is a great setup for that.

Somewhere around line 94 there\'s a discussion requiring substantial
representations of patients exposed above the highest proposed dose. I
think the wording actually refers to at or above the highest proposed
dose.

Our concern there is that if there is going to be an expectation that we
have additional representation in our documents at doses higher than the
proposed dose, we need to be very careful about dose dependent toxicity.
If that is significant, it may well not be appropriate to have
substantial exposure above a proposed dose.

Around line 136 there is a discussion requiring or suggesting larger
databases when there is a potential for rapid exposure to a large
population. Again, if that population \-- if there is a rapid
introduction into a large population, we understand the need to have
more safety.

But the final label will determine a lot of the exposure. Market
activity and economic resources of a company will also determine how
quickly a drug is introduced into the population. It may not be possible
for us as we are developing a program to actually know how rapidly a
drug will enter a marketplace.

Our request and counsel to the agency is that we have an active dialogue
while we are going through the development program if there is an
expectation that it will be introduced into a large patient population.

In III(b) there is a discussion around characteristics of an ideal
database if we look at the language around line 174, a range of doses in
Phase III. Most development programs rely on adequate dose ranging in
Phase II(b) to identify the most likely dose or doses for Phase III.

By using Phase II(b) in that way, we limit exposure to excessive doses
that might provide unacceptable toxicity and it limits exposure to
patients of subtherapeutic or ineffective doses. We think that it is
appropriate to do an adequate dose ranging study in Phase II to identify
the doses that make sense for an approval and study those doses in Phase
III. In some cases that may well be a single dose.

It may be a couple of doses, but we are concerned if the agency is
interested in moving to Phase III programs that require more doses to be
evaluated in those large populations than we actually think will be
registered in the end and be available to patients.

In Section 3(d) of the document there is a discussion around the use of
a comparative safety database. The text around line 238 deals with the
concept of when there is a well established product with minimal
toxicity to treat a condition of interest.

I think we all know today that comparative safety trials are generally
not required for registration. We are concerned that the suggestion in
this document is that standard could change to a relative one requiring
demonstration, here I am quoting from the paper, \"a comparably benign
safety profile.\"

Our concern about this section is that the language doesn\'t take into
account that there may be differential effects that may make a
difference in a safety profile acceptable. If there is a product
entering the market where there is already an effective drug. That
product coming onto the market where there is already an effective drug
and that product coming onto the market has substantially better
efficacy, there may be a difference in that safety profile.

We would hope that we would be evaluating the efficacy and the risk
benefit ratio of each of these products, not simply requiring or
expecting that the safety profile in isolation is equivalent or better
to an existing product.

In Section 3(e) of the document is a discussion around special
considerations for optimal risk assessment. Around line 279 is the
discussion that has been referred to already today about a large simple
safety study, the LSS, conducted prior to approval and designed to
assess few outcomes in a large number of patients.

We think that prior to the conclusion of Phase III it is rarely going to
be possible, and possibly in many cases not ethical, to conduct a large
simple study. In order to do that you need to really understand the dose
or the doses that have demonstrated a favorable risk benefit. You need
to have a fairly good understanding of the safety profile.

If you don\'t understand the safety profile, you can\'t design a very
simple trial. If you don\'t have some understanding of which doses are
actually going to be beneficial and what endpoints have shown efficacy,
again it\'s very difficult to make that a simple trial.

We are remaining convinced that a well-designed Phase III program should
address the issues relevant for approval. Our concern with this language
is that a pre-approval large simple safety study requirement would
appear to add an additional phase beyond Phase III for approval and we
are concerned about that.

We are very supportive that in some cases a large simple safety study
may be appropriate as part of a Phase IV commitment as part of an
ongoing risk management program. We think those are the kinds of
discussions that should go along with the agency as a product being
developed but to introduce that as a step between Phase III and approval
would have significant consequences to our ability to bring those
products forward.

Then finally, we want to raise a couple of issues about the paper that
we just think need additional clarity and maybe we could get some
comment during the course of the day. We\'ve heard Dr. McClellan, our
new commissioner, advocate that Phase IV studies could be used to
accelerate the approval process for many drugs.

When we look at the paper that we are discussing today, that appears to
be advocating more work be done prior to approval and many of those
pieces of work would traditionally be done in Phase IV today.

We would like some understanding of how we should interpret these
messages. When would it be appropriate to move to a Phase IV program as
a way of bringing a product to market quickly and when does the agency
or the public or our industry think it\'s appropriate to spend more time
prior to approval.

Then the last point deals with the question of whether this is, in fact,
more on the concept phase or more on the guidance phase for us. There
are a lot of adjectives used in this document that include \"ideally;
would suggest; case by case; should consider; could be useful; or in
certain circumstances,\" that give a lot of latitude to the
interpretation of the descriptions.

We think that\'s fine in a concept paper. We think that\'s what allows
us to have that open dialogue today. Either we need to bring more
clarity to some of those qualifiers so that we have a sense of when we
should be applying those aspects of a risk management program and when
we shouldn\'t.

Or perhaps we should be discussing these things with the agency on a
case-by-case basis as we go through the development program. It would be
great to get some understanding of what your view is of whether this is
in fact something that we should be taking into account up front or are
we in a case-by-case.

I think I will stop there. Very brief tee-up of some issues for us. We
hope that there will be more robust discussion around each of these. I
know Rob Califf and others will have some thoughts on these points and
could generate some interesting discussion. Thanks.

MS. HENDERSON: Thank you very much, Dr. Gormley. I would now like to
open the floor to questions really from anyone from our steering
committee, from our invited guests, from those members of the working
group if you need clarification or would have comments on the comments
of the speakers, and anyone in the audience. Again, we have staff
circulating.

If you want your questions in writing, write them on cards and pass them
across to the isles and folks from the staff here will collect them and
bring them up and we\'ll read your questions. Or there are, again,
microphones on either side of the room if you have questions or
comments.

Bob, I know you said earlier you have a question.

DR. MEYER: Yes. Actually I had a question for Dr. Jones.

Dr. Jones, during your talk you had mentioned that it would be useful to
develop a spectrum of scenarios of how a drug will be utilized in the
real world. It\'s curious in that suggestion whether you are talking
about us developing these scenarios and then building them into the
studies to better define the safety and perhaps efficacy of such use and
how that might relate to the labeling.

In other words, are you suggesting there are certain senses where we
might even want studies of off-label use that we understand are likely
for such medication?

DR. JONES: Well, I don\'t think I was entirely suggesting off-label
studies. I think that there aren\'t many \-- I mean, there are still
drugs, particularly in the biologics areas that are really making
incursions into brand new therapeutic areas.

A large proportion of the drugs being developed really have populations
that are already being treated in one way or another. I think my
suggestion was that both on the basis of such practical clinical
thinking as well as actually studies of longitudinal behavior of
patients in those indication populations can give some insight into the
potential for the range of use.

That\'s primarily what I was getting at, not really off label.
Practically some safety issues relate to overdosing and those are part
of the label and that might be something that could be informed by,
again, experience in the population with analogous drugs. We have a lot
of data out there that I think could inform this at a much earlier
stage.

DR. MEYER: Thank you.

MS. HENDERSON: The gentleman at the microphone on my right.

MR. BURKHART: Greg Burkhart, industry consultant. I just wanted to ask
Bob a question. He said something I thought was kind of interesting. I
think he said that some of the open label data had been examined and
found to be relatively uninformative and this had lead into the idea of
doing more long-term comparative safety trials. Did I hear that
correctly, Bob?

DR. MEYER: You heard it correctly, although it wasn\'t a rigorous
examination. I think this is the impression of folks who have looked at
many of those studies over time. It wasn\'t a systemic look at the data
so much as the experience of reviewers after seeing this data.

MR. BURKHART: Certainly long-term open-labeled data can certainly be
very informative if you don\'t see anything within that data. Certainly
the size of the database does matter. I think one of the first PhRMA
comments was that it didn\'t really matter. I would find that hard to
believe. I would think the more data you had that was fairly clean and
well collected always is more informative regarding particularly serious
risk and rare events.

I guess I would raise a concern about what long-term safety data
actually would be reminding everyone that we have a difficult time
trying to do long-term comparative studies as it is, particularly with
efficacy and all the concerns that would apply to making a comparative
claim for efficacy are going to apply for interpretation of safety data
whether it\'s naive users, how you would go about enrolling patients.

I would like to see a little bit more thought given to the concept.
Certainly not all data collected in development programs should be
long-term comparative data. Open-label data provides great information
if it\'s well done.

MS. HENDERSON: Dr. Califf.

DR. CALIFF: I am delighted to be here today and I think it\'s great that
we are having this discussion. I am sure the questions will be many more
than the answers that we have.

As many of you know, I did direct the coordinating center for the
Centers for Education and Research in Therapeutics. These documents have
stimulated a very healthy e-mail exchange among the seven academic
centers and members of industry and HRQ and FDA.

Just a couple of points and then a question that may be too broad to
really answer but I hope we can get some more insight into what was just
alluded to.

First, it seems to many of us that we are caught here between the
knowledge that every situation is different, the development of every
drug or device is different. Yet, we need to grasp onto some
generalizable trends. I think we thought it was a good idea to have
those sort of qualifiers in there really prompting people as they
develop drugs and devices to reach an agreement as much as possible
ahead of time about the specifics recognizing that one size doesn\'t fit
all.

Yet, as we already do, we would point out there seems to be a dearth of
empirical research that can really guide us. There was a mention already
made of evidence-based decision making about risk management. We really
lack much in the way of research in this arena. It\'s not very well
funded. It\'s self-serving for us to say but we\'ll say it anyway.

The next thing to say is that this concept of scenario planning, we
think, is an excellent idea. I hope that many people here today, in
fact, I\'m sure they will be, at the pharmacoepidemiology meeting where
this is going to be discussed in a lot more detail.

We all know this is actually what goes on incessantly in drug
development anyway. You are in these meetings but more of the sharing of
what the real scenarios are that may arise. I would just point to one
example that is pertinent to this might have been the drug mebefradil
where scenario planning on what might happen when it got on the market
might have led to a different view of the studies that should have been
done.

The studies might actually have led to a different risk management
program that could have been successful. Who knows but it\'s worth
thinking about.

The next point would be that we would encourage this concept of control
groups whenever possible. They may not be randomized all the time but
without a control group it\'s hard to draw inferences in the active
surveillance. There are really two ways that we think about that.

One is in post-marketing, that we\'ll talk about later, but in
premarketing the difference between an open-ended adverse events
reporting system and a check box where people have to look for it if you
think it might be there.

This really ties us back to scenario planning in many cases. For
example, in cardiovascular disease we only detect about one-tenth the
number of events if it\'s an open-ended question compared to a check box
because a lot of things happen to people and depending on people to
write it down and then try to herd it into some kind of a category is
difficult.

Having said all that, now my question. It seems like a lot of this, a
theme that was important to us was not segmenting this so much but
looking at this as a continuum. The concept was alluded to that
agreements about what would be done post-marketing might influence the
receptivity to the point at which a drug was put on the market, for
example.

We think this is a very important concept because we all know that for
chronically given drugs there is no way in premarketing you can really
know the full extent of the balance between risk and benefits.

So the question is what is the discussion going on now about how if one
designed a program that really would delineate the balance of risk and
benefits over time post-marketing, is there some way to make it a more
fluid effort through some agreements or some other approach than saying
we\'re going to get on the market and it\'s a clear dividing line and
then you start your post-marketing surveillance phase.

DR. MEYER: I am going to actually turn this to either Bob Temple or Bob
O\'Neill because I think that they have been more involved with some of
the discussions with the commissioner and others about what that
proposal from the commissioner might entail. My observation at my level
is that I don\'t think it\'s fully defined at this point but I will turn
it over to Bob and Bob.

DR. TEMPLE: I can\'t speak precisely for the commissioner on that either
and I don\'t imagine Bob can. But I do think it\'s clear we already do
actively think about the post-marketing period to resolve complaints or
problems or concerns that we are not maximally worried about but still
somewhat worried about.

I think that is part of the thinking. That\'s what Phase IV commitments
are. There are specific examples. Ziprasidone was approved with an
agreement. It was actually worked out in Europe but known to us that
there was going to be a very large comparative trial to look for a few
QT related problems.

That was reassuring in light of our considerable reassurance based on
the data we already had but with some residual concerns. I think that
kind of thinking goes on. I don\'t think anybody thinks that we want to
have a lot of drugs yanked from the market because we didn\'t do enough
premarketing stuff. I don\'t think Dr. McClellan is interested in that
kind of tradeoff and I don\'t think anybody else would be either.

Can I offer a couple of other comments? I think every one of us, and the
concept shows this, believes that some degree of case-by-caseness if
unavoidable and we would be troubled if rigid new things emerged.

Having said that, it\'s worth saying that it would be hard to know
whether you have changed the standard for approval with respect to
safety because the standard for approval with respect to safety and the
law is all test reasonably applicable to evaluate safety. If you know
what that means exactly, tell me, but it has a very subjective quality.

I think the thrust of the paper is that there are some things you should
be thinking about along the way. It invites an observant industry to pay
attention to what is accumulating and be adaptive as necessary
incorporating everything you can imagine, animal data, clin-pharm, and
all the rest, but also including the early hints of trouble that might
emerge during Phase III.

Just as an observer, but I don\'t develop drugs, it\'s not always clear
to me that people pay attention to the possibility of changing the line
that has been planned out in light of things that happen. Then I guess I
have to ask other people if they have views about whether you really
know all you need to about dose response at the end of Phase II.

Personally I think that is completely wrong and a terrible error because
you don\'t have any safety information about relatively unusual events
in Phase II. I just want to point out that there are certain drug
classes that are developed entirely with dose response studies
throughout the development course.

All antihypertensives and virtually all antidepressants are developed
that way because they don\'t really believe the Phase II tells them very
much about effectiveness in some cases, certainly for antidepressants,
and safety for any drug because the events you\'re interested in aren\'t
that common. I would be very interested in more discussion of that. I
think that was not our premise going in.

MS. HENDERSON: Dr. Gormley.

DR. GORMLEY: By the way you have articulated your thoughts around this
are really what we were looking to hear and we assumed it all along that
it\'s a very rational, thoughtful, case-by-case idea that you are
looking for. We want to take each drug on its own merits, understand
where the company wants to go with it, what you want to accomplish with
it, and then come up with a rational program that will manage the risk,
which I think is what you just said.

When we read the concept paper and we start to ask ourselves line by
line is this something that the agency is going to expect from all
drugs, then that\'s when we get a little concerned. What you have
described is your vision. If that comes through in the document, it
becomes very supportive of where we\'re going.

Then to your second point around dose response, there is no question
that in many drugs you are going into Phase III understanding that you
are going to need a titration phase. You don\'t know where the patient
is going to land at the end of a regime and you have to test many
different doses in Phase III in order to give good guidance around that
titration.

In other cases the dose dependent toxicities and their mechanism of
incident of that drug drive you to either a single dose or only one or
two doses.

If we can establish that in Phase II, what we prefer to do is not treat
large numbers of people in Phase III at subtherapeutic doses when we
don\'t believe they are likely to be useful or certainly not at toxic
doses. I think we\'re not saying something different. What is in common
between our comments is that collaborative spirit of working through
each drug.

DR. HENDERSON: Dr. Califf and then Dr. Simon.

DR. CALIFF: Let me try to push you a little bit further to be more
specific. I mean, specifically we know that drugs increasingly go to
chronic long-term therapy and people are living forever. People will be
on drugs for 20, 30, 50 years.

Very often at the time of approval you will have a surrogate or
surrogate-like measure. Not the full balance benefit profile at the time
of approval, but you may have a very specific risk. That seems to me to
be where a lot of the action is going to be. How should that be handled?

That seems to be where earlier approval but some risk of being pulled
off the market, as you say, when you get five or 10 years of follow-up.
Sometimes it goes in a very positive direction like statins and ace
inhibitors have gone. Sometimes it goes in a not so positive direction
as you get longer-term follow-up in more patients.

DR. TEMPLE: You could be asking partly about showing whether the
surrogate really predicts the outcome you hope for which isn\'t
particularly what this paper considered. If you just wanted to focus on
safety concerns, there are not too many models but QT prolongation is
one model.

We have evolved informal behavior, which I wouldn\'t try to pretend is
entirely databased. Very small increases lead to approval and then you
sort of watch what happens. You wait for the reports of Torsades to come
in and hope they don\'t.

Where the level is a little higher but you are approving the drug
because you think it makes an important difference, or could, then the
model where you try to do a large post-marketing Phase IV organized
trial, or at least think about that, becomes the other thing you do.

There are other models of that. The long-term beta agonists trials
reflect some of the same concern about what the true outcome is going to
be. Nobody thought that shouldn\'t be approved without those data but
there was a residual concern that could have led to it.

How to give the ultimate answer on whether the surrogate you wish for
leads to a favorable benefit risk ratio, probably the best place to look
is cholesterol lowering agents. We now appear to have outcome data on
every one of them after closing the gap based on my recent [New England
Journal]{.underline} here. It appears that the industry was particularly
interested in being able to say something about outcomes for those
drugs. That was a competitive claim. You might say the capitalist method
worked very well. Product differentiation led to spectacular series of
trials that are benefiting everybody.

I think it is also true that where everybody is already a true believer
about the surrogate, you may not necessarily see that. I don\'t think
you are going to see the same thing with hypertension because everybody
already believes.

DR. CALIFF: One technical point because it\'s been said a couple of
times. We\'re not talking about ratios here, right? We\'re talking about
a balance of some kind because you\'ve got different \-- the risk and
the benefits may be totally different metrics which is \--

DR. TEMPLE: Yeah, I agree with that. I think you are talking about
descriptions. We conveniently call them ratios but you\'ve got to look
at each separately most of the time, I think.

MS. HENDERSON: Dr. Simon.

DR. SIMON: I reacted somewhat to the position paper that at least part
of it was holding up the randomized clinical trial as sort of an ideal
as a way of evaluating safety with regard to known and unknown
endpoints.

I think whereas I would not argue against that it is beneficial to have
data controlled and randomized data for evaluating safety. I think it\'s
actually very problematic what is being proposed here because you are
essentially proposing a randomized clinical trial in which you would do
things that you would normally try not to do in a randomized clinical
trial.

You would have no predefined hypotheses. You would have enumeral
endpoints with regard to \-- you would have enumeral subset analyses
with regard to patients on this or that concomitant medication. And I
think there would be a tremendous risk of false positives. You would
never have the same size necessary for answering these questions in a
reliable way. You could use it as sort of a way of collecting unbiased
data, although it would be very difficult.

As was pointed out, it\'s very difficult to collect good data when you
don\'t know really what your endpoint is because you have so many
endpoints. You would be trying to collect data that would be less biased
then you might have in an uncontrolled situation. Actually, it might be
a very expensive process for generating false positives. That is one
concern of mine.

I think with regard to the multiple dose groups, I think you have to be
a little careful. I\'m sure it varies on a case-by-case basis but I hear
lots of people talking about wanting to optimize the dose and they
don\'t really realize that optimizing the dose is a very different
objective than establishing safety and efficacy of some well-defined
dose strategy. Generally it\'s an order of magnitude more complex and
more expensive.

I think talking about going into Phase III without sort of \-- as a
general thing with multiple dose, whereas in some situations obviously
you would want to, I think in general it\'s not a desirable objective.

Finally, on the issue of broad patient populations for safety, I
certainly understand where that\'s coming from because many drugs are
used in very broad patient populations broader than what they\'ve been
studied at. I think as we learn more about sort of the genetic and
molecular basis of diseases, at least some disease, we are finding
incredible heterogeneity in the pathophysiology of the diseases.

In general it\'s probably \-- in some cases it can have a devastating
impact on power for identifying efficacy to have very broad patient
populations. I think that would be something you would want to use
selectively on a case-by-case basis.

MS. HENDERSON: Thank you, Dr. Simon.

Dr. O\'Neill.

DR. O\'NEIL: Yeah. I just wanted to get back to Bob Meyer\'s comment
about what the commissioner might have meant. I agree with Bob Temple
that he\'s in the early innings of trying to figure out where a lot of
this might go.

What\'s interesting about Dr. McClellan is that he is an outcome
researcher at heart and understands data and how data comes together.
What we are dealing here with is the intersection of you can\'t get the
answer with randomized trials. You also can\'t get the answer with
observational data.

It\'s sort of like everybody is pointing at the other guy saying,
\"It\'s your problem. It\'s your problem.\" The fact of the matter is in
these long-term trials it\'s very hard to keep anybody on therapy for
even 12 months.

I think we have learned a lot just from the 6,000 and 8,000 patient
COX-2 trials which essentially are only 12 months long for somebody who
is on a chronic therapy and essentially loss of follow-up is relatively
high. What we are dealing with here is trials that have predefined
endpoints.

You are also interested along the way of the unintended consequences
that will always come out in long-term trials. You can\'t get off the
hook by essentially complaining about there\'s no predefined endpoints.
We are in this game.

I believe Dr. McClellan recognizes are there other alternative
strategies including the epidemiological strategy, how might you be able
to think about using data, outcome data, and use some other matching
strategies recognizing that you can\'t get around the selection bias
issue.

I think he is at heart a scenario planner so he essentially is saying
you don\'t start to figure this out at the time that the drug gets
approved and then figure out how you are going to start to collect
information.

I think the scenario planning is if you see how this is going to play
out over four or five years how are you going to get access to the data
that you need to get to get examples. We are talking about clinical
trials, using the managed care system, using other facilities, the
physician population.

He\'s very interested in terms of what are other leveraging
organizations that we can help get access to this information and use
other alternative methods. There\'s no answers in yet but I believe he
is a big-picture thinker. I believe this problem can\'t be solved by
RCTs alone and it can\'t be solved by large observational studies alone.

It\'s a combination of any and all of these things because we have a lot
of experience from these large studies. Even well designed RCTs where
lost to follow-up, not being able to \-- a culture of not following
people once they leave trials.

That\'s when things happen to them. I think we need to take a deep
breath and sort of figure out what are some of the strategies that may
need to be applied to some of these areas that we haven\'t thought about
previously.

I\'m not speaking for him. I\'m just saying this is my impression from
how he is opening up the dialogue and bringing out these kinds of issues
from a number of the centers.

Susan has been in those conversations. CDRH has been in those
conversations. CDER has been in those conversations so it\'s just not a
drugs only view of the world.

MS. HENDERSON: Thanks, Bob.

We have a question at the mike and then Dr. Chaung-Stein.

MR. SHRINER: Mark Shriner, Children\'s Hospital, Philadelphia, and
Children\'s Clinical Research Institute.

I was a little disappointed that the document didn\'t say anything about
safety trials in children since 15 percent of the trials that have been
performed under FDAMA and Best Pharmaceuticals for Children Act are
purely safety trials. Many of the trials that are listed as efficacy and
safety have no comparators and, therefore, really aren\'t efficacy
trials.

Those trials, from my perspective, have no hypothesis and any decent IRB
should not approve them since one of the mandates to an IRB is that the
trial be able to answer questions. A trial without a hypothesis is, from
my perspective, not approvable by an IRB.

Secondly, if the trial is approved and it\'s open-label safety \-- I\'m
sorry, not open-label safety but there is a comparator, how do we choose
active comparators in pediatrics when so few drugs are approved.

Lastly, I think that a lot of the chronic use studies, in particular
what I\'m seeing in opiods are scenarios that are being requested by the
agency that reflect adult use of those drugs but are a tiny fraction of
the use of the drugs in pediatrics.

When we do studies for safety purposes for chronic use in children, they
should reflect the way the majority of children use the drugs, or are
prescribed the drugs, rather than the way adults use those drugs.

Lastly, I have a comment which no one has commented on yet about acute
trials. Rapacuronium was approved for single dose use or induction,
muscle relaxant in anesthesia. The label said it had a 3 percent
incidence of bronchospasm. Because there was no comparator with any
other drug, we didn\'t know that it had 10 times the risk.

Without a comparator, even in drug use for single dose at the time of
induction of anesthesia, we didn\'t know we know we had a major safety
problem. I think even in acute drug usage we need to have some
comparison. Otherwise, what does that 3 percent incidence mean? Is that
high or is that low. We had a number of life threatening and we even had
a death or two. Thanks.

DR. MEYER: Thank you.

MS. HENDERSON: Dr. Chaung-Stein.

DR. CHAUNG-STEIN: Yes. I have a comment regarding the size of the
marketing safety database. I will go into a little more detail this
afternoon but I will make a comment right now.

It\'s true that if we have more patients we will be able to estimate the
event rate with higher precision. The question at heart is not to
estimate the event rate for common adverse events. It\'s really it\'s a
rare event.

The issue there is, again, the information is likely to be very small
compared to the increase in the sample size. We might need to observe
thousands and thousands of patients in order to observe one occurrence
of a rare event.

The question then is where is the most appropriate place to observe that
rare event, whether that should be in the premarketing phase. That\'s
the heart of the question.

MS. HENDERSON: Dr. Califf.

DR. CALIFF: The trial one concept, it seems to me, we\'re sort of stuck
between. I think Dr. Simon is right in a way, in that our understanding
about the biologic basis of diseases in the middle of a very rapid
transition now as we get more knowledge about the molecular basis. That
would leave one to believe that you should narrow the population that
you study to the people that really have the biologic problem that you
are addressing with the treatment.

On the other hand, the prescribing world is nowhere near that. That is a
fantasy to the average clinic if you go to it. Drugs that get on the
market are prescribed to a very broad population. That is, in fact, the
financial basis for the support of the industry at this time.

I think we are stuck at sort of a crossroads now, but it seems to me
there\'s a combination of scenario planning and broadness of population
that, at least to me, makes sense. I am particularly thinking about the
elderly population.

We are going to go from 30 to 60 million people over age 75 in the next
20 years. These are going to be the biggest consumers of drugs. Most of
them have \-- they all have impaired renal function by definition
because creatinine clearance.

When we talk about broadness I think much of the concept is not just you
should study everybody who might get the drug, but more that you should
focus on the people where the drug is most likely to be used where the
balance of risk and benefit may be most important.

In other words, where the action is. I would at least submit that in my
experience with premarketing studies it\'s common to purposefully
exclude the people in whom the action is going to occur because it\'s
dangerous. You may find things are really, really good or they are
really, really bad. It inserts an element of risk into the development
program.

I personally believe strongly that the FDA needs to think carefully
about how to incentivize or encourage companies to be more aggressive in
this regard so that knowing that when drugs get on the market they do
get prescribed to these high risk people, that we have more information.

There, in fact, it\'s not a matter of just frequency or narrowing the
confidence intervals on common outcomes. It\'s actually some
concentration of events in a relatively small population where the
action is. I don\'t have a solution to this but I think it\'s a big
issue.

MS. HENDERSON: Dr. Ellenberg.

DR. ELLENBERG: I would like to address Christy\'s issue about the
difficulty of finding rare events. I think we have to consider rare as a
relative term. Something that is rare for a drug that is used in
thousands of people should have a different standard for drugs that are
going to be used in millions of people right away.

I think we get ourselves \-- it\'s not a rare argument to say because we
can\'t detect the one in a million risk we shouldn\'t think about
whether we\'re doing an adequate job in detecting a one in 50 risk or a
one in 100 risk. I think this is where we get into the case-by-case
issues.

Different products may demand a different sort of standard for what
level of risk you need to be able to detect. I don\'t think any of us
involved in doing these documents would think otherwise that there
should be a single standard. That would be crazy.

It\'s not an argument that I would accept that because we can\'t detect
the rarest of risks in a clinical trial, or probably even an
observational study, we shouldn\'t think hard about what the level of
risk is that we should be able to attempt and try and move toward that.

MS. HENDERSON: Thanks, Dr. Ellenberg.

Dr. Gormley.

DR. GORMLEY: I just wanted to reiterate a point I made up on the podium
because I think it is supportive of what you\'re saying. We\'re very
supportive of the concept of increasing the size of a database if there
is a specific issue that we all agree we need to address.

What we\'re advocating \-- I think what you are \-- if there is a
specific goal or objective and you can define it and you can say if we
move the database size from here to here, that will give us reassurances
around something we are looking for, that makes perfect sense.

To say, \"We\'ve got a product with a collection of safety issues that
we\'re not sure of, therefore, let\'s double the database,\" that
doesn\'t make sense because that doesn\'t have a specific goal. I think
you are agreeing with me. It\'s around what you\'re trying to find that
drives what you do with the database.

DR. ELLENBERG: I suspect that there is some agreement and probably some
disagreement if we fine tune this discussion because I think there could
be \-- it\'s not entirely clear what people would consider an
appropriate goal.

You might have a study where you know there is a certain background rate
in a population, for example, and you may not have a particular fear
about the drug causing that problem in the smaller preliminary studies.

Yet, you know when this drug goes out and is used in millions of people,
there are going to be people who are going to experience that after
taking the drug and they are not going to know whether the drug may have
influenced it.

There may be a goal in some circumstances to look at particular events
even if there isn\'t \-- even if there hasn\'t been particular data
brought forward to say this drug may cause this event.

There may be some biological plausibility, for example, that it might,
even if you haven\'t seen it and you might want to design the study. I
think that is where we might get into some discussions and ambiguities
about whether there is a clear goal or not.

MS. HENDERSON: We have a commenter at the mike.

If you would not mind, identify yourself before your comment. Thanks.

MR. KAHN: Sidney Kahn, Pharmacovigilance and Risk Management, Inc. I
would like to continue this discussion because there is one piece that
seems to have been overlooked somewhat. Currently in clinical
development the majority of products do not make it to a filing because
primarily often are safety concerns which are identified during the
development process.

The concept of there being a strong signal during development is one
that is not going to reach the agency typically in the form of a filing.
One thing to look at is the issue of how do we identify in a reasonably
sized database that is practical to assemble with reasonable cost and
reasonable time, those signals that are not egregiously blatant, but
rather are subtle and lead to suggestions going forward remembering that
the current process of development weeds out most of the products with
real safety concerns early.

The mechanism for approval, as we know, has a very good level of
porosity in not allowing many drugs that subsequently have to be pulled
from the market or otherwise will require major interventions. I think
this is the problem that we have between the baby and the bath water. We
have the potential of overreacting to risks that may not be real.

MS. HENDERSON: Thank you, Dr. Kahn.

Dr. Temple.

DR. TEMPLE: I wanted to remind everybody of something that Bob Meyer
said. Having a control group is sometimes highly protective. If you are
looking at a population that has a high background rate of difficulty or
something like that, it\'s not impossible for a cluster of open-label
events to seem to be significant but really that\'s very worrisome.

It should worry the industry, I would think, because you really can\'t
tell. At least sometimes having a control group will give you a better
idea of what the background rate for this population is.

Having said that, Rich is, of course, absolutely right. Safety
evaluations are all free of hypotheses unless there happens to be a
hypothesis you add. That is overwhelmingly the exception.

In the controlled efficacy trials, in the open-label safety things, you
are all behaving expectantly. That is the nature of the thing. That\'s
why it\'s so difficult. That\'s why it\'s hard to know whether you\'re
done.

I guess I would have thought a control group gives you at least somewhat
more assurance than a completely uncontrolled environment. Especially
in, as he said, people who already have trouble because of their
disease.

MS. HENDERSON: Dr. Simon.

DR. SIMON: The only response I would have, Bob, is I also agree with \--
I basically agree with you. It is better to have a good control group
that should help you interpret the data.

It actually does, I think, in a lot of cases lead to misinterpretations
because the fact that you have a control group of people then go on to
analyze their data in inappropriate ways. You do a statistical
significance test on 100 different sort of things. You get a P of .15
and you get worried about it.

MS. HENDERSON: Dr. Califf.

DR. CALIFF: You can shut me up at anytime but if no one else is raising
their hand, I have a lot of questions.

MS. HENDERSON: Thank you, Dr. Califf.

DR. CALIFF: I think Susan raised \-- I\'m trying to interpret what she
said in the context of things that I\'m dealing with right now. I would
be interested in both Glen and Bob\'s view on this. To be provocative
one could take the position that any drug that affects the inflammatory
system is going to have an effect on vascular disease events.

Now, no one would argue that you need to do a 15-year outcome study to
determine that premarketing. Yet, I think it would be hard to argue that
you could be confident that any drug that you gave chronically to a
population over age 50 wouldn\'t affect vascular events one way or the
other.

So in this premarketing/post-marketing thing and sample size, how do you
deal with that? I think Susan was partly getting at that in that you\'ve
got a commonly occurring event in the population anyway if they are not
taking the drug. Now you are giving a drug for another reason that is
going to affect that event rate plausibly and you could look for it.
This could be enormously challenging.

MS. HENDERSON: Dr. Gormley, did you want to comment? Then following that
we will take a morning break.

DR. GORMLEY: Great. Well, let me start with a few thoughts. Part of the
question you\'re raising, I think, is very fundamental to what society
wants in terms of safety and benefit of new drugs. There is no drug that
is risk free. We have an obligation to understand the risks that a drug
has when we bring it to market.

You are raising questions that are very profound about drugs that have a
mechanism of action that is intended for a disease. We demonstrate that
it works in that disease. We look at the safety profile and that looks
very acceptable for that condition. But because of its mechanism it may
have interesting either positive or negative affects on many other
systems.

The cardiovascular system and an inflammatory drug. Would we want to
have drugs evaluated through those very extensive studies necessary to
answer that question before it goes on the market?

Is that something we want to delay the development of a program for in
order to be reassured, or are we accepting that with an adequate Phase
III program that has demonstrated a benefit risk that we think is
acceptable, the company should then commit in the post-marketing phase
to evaluate the sorts of things you\'re raising. That\'s a philosophical
question.

It would have profound implications on the drug development process in
this country, as you know. I don\'t think there is a real answer but it
really needs to be debated. It would change the way we do drug
development if we took one extreme or the other, wouldn\'t it?

MS. HENDERSON: Despite what I said, we have one additional comment at
the mike, Judy Racoosin from the FDA, and we\'ll finish there.

DR. RACOOSIN: Thank you. I just wanted to respond to one of Dr.
Gormley\'s comments about the expectation that patients are exposed to a
certain \-- that we only count patients who have been exposed to a
certain dose.

I think the point that we were trying to project from is that in drug
development while dose finding is going on, patients are going to be
exposed to a range of doses. I think your comment interpreted that we
wanted people who were exposed substantially above what the proposed
dose would be.

I think what our goal was was really to protect against counting too
much experience with patients who are exposed below the lowest intended
dose. We\'ve seen examples where a substantial portion of the safety
database, say a third of the experience is in patients who have been
exposed or treated with doses that are below what has been found to be
the efficacious dose.

Given that we have a limited number we have to work with what we have
and we want to make certain that at least the ICH guidelines are in
patients who have been treated with at least the lowest intended dose so
that we are not counting a substantial portion of our safety database.
We are looking at patients who their experience isn\'t relevant because
they have been exposed to too low a dose.

DR. GORMLEY: A clarification. The reason I focused on the higher end is
because we could agree exactly what the point that was made. Of course
we need to evaluate drugs with a substantial exposure for what we are
proposing to be used so we completely agree with that end of the
spectrum.

MS. HENDERSON: With that we will close this morning\'s session. Thanks
to all of our commenters. The clock that is on the wall in the back says
it\'s 10:17. We will reconvene promptly at 10:35.

(Whereupon, at 10:17 a.m. off the record until 10:38 a.m.)

MS. HENDERSON: Our next speaker is Dr. Bob Temple who many of you in the
audience know. Dr. Temple is the Associate Director for Medical Policy
at the Center for Drug Evaluation and Research. Dr. Temple is going to
talk about premarketing risk assessment special conditions.

Take it away, Bob.

DR. TEMPLE: Okay. The concept paper after giving some general overall
guidance focuses on a few situations that call for special attention.
I\'m going to try to talk about those.

Just to recapitulate briefly, you have already heard that there are some
issues that ought to be paid attention to all the time such as the size
of the safety database, what the particular characteristics and concerns
there are about the drug, what its proposed use is, and the intended
population. All of those things could affect how many people you think
ought to be in the premarketing database.

You heard and we had a good discussion about the characteristics of the
ideal safety database including reasonable duration, diverse population,
don\'t exclude everybody over 75, and more dose response information
than we typically see.

The last seems particularly important for drugs with a high rate of drug
induced adverse effects or biomarkers that might be of some concern such
as CPK elevations, transaminase elevations, uric acid \-- I can explain
that later if anybody wants to know \-- or potassium changes.

One of the continuing themes really of the whole agency is that one
wants to know as much as one can about how to individualize therapy.
There is always worry about drug/drug interactions, drug demographic
relationships, and drug disease relationships.

One suggestion, at least, that routine population pharmacokinetics is
not a bad idea. Then there is the discussion of comparative data. As we
have been saying in part in response to questions, evaluation of safety
has some general principles but I think the paper tries to make the
point that that doesn\'t mean one size fits all or that people should
just have a reflex.

So throughout the planning and monitoring of development there needs to
be constant attention to the results of animal and human data including
some ability to monitor things as they are going and make changes so
that one notes suggestions or problems.

And then attention to population and drug specific features that raise
special concerns either at the outset or as the data accumulate. When
those occur plans need to be modified. The concept paper identifies some
of the things to pay particular attention to.

One consideration is what should the maintenance dose be. It is
extraordinarily common to just have the maintenance dose be the dose
that was just used in the initial short-term studies. At least sometimes
that might not be the best thing. There are two possibilities.

One, there could be pharmacokinetic reasons for thinking about it. For
example, long half-life drugs are sometimes started at high doses, what
you could call loading doses if you were to change the dose, in order to
get a reasonably prompt effect.

Should the maintenance dose, however, be lower; that is, should it be
used in essence as a loading dose. There are a couple of historical
examples, one of which turned out badly, the other of which has not.

Astemizole, an antihistamine, was used at 10 milligrams because you
don\'t want to wait that long for your seasonable allergies to get
better. It turned out that that dose was very close to a dose that could
cause Torsades de Pointes. Probably a factor of less than two.

Now, given the verial and half-life measured in at least a week, the
maintenance dose just on theoretical grounds alone could probably have
been three milligrams which would have taken you a factor of six or
seven away from the dose that causes marked QT prolongation. It seems at
least possible that had that been done initially the drug might still be
there.

Now, that is being smart after the fact, I have to tell you. This is no
crowing here. These are things that you are supposed to learn about and
it\'s not that easy to be smart at the beginning.

Another drug where there is no trouble is fluoxetine. The drug and its
metabolites each have half-lives that last well over a week. Sometimes
two, three, four weeks. It\'s hard to imagine that the maintenance dose
couldn\'t be lower than the current 20 milligram recommended dose, but
that really has never been specifically looked at. As I said, in this
case nothing seems to have come of it.

The other possibility is that a drug might induce pharmacodynamic
changes that would allow lower or intermittent doses or allow drug
holidays. That is a tempting thought when there is known serious
toxicity, but it\'s probably something to think about in other cases,
even if you aren\'t entirely sure.

One example is alosetron. We knew from the registration trials that
constipation was observed in about 30 percent of the population. That is
sort of surprising because the drug was intended to treat diarrhea.

One\'s first question could have been, but we didn\'t ask it either,
could the dose have been backed off as soon as the first signs of
constipation or improvement emerged. That might have avoided the very
severe constipation that was one of the reasons that sales were
suspended. It might have had an effect on its other problems as well.

A second special consideration is drugs that you believe have to be
titrated. Now, that\'s not that common. Most drugs actually don\'t have
to be titrated. But it\'s relatively unusual to see a comparison of this
titration method and that titration method comparing them both to see
whether you can avoid side effects by slowing it down or whether
titrating more rapidly, which we\'ll ask people to get to their dose
more rapidly, actually cost anything in terms of side effects.

An explicit comparison of titration methods is possible and there have
been some labeling changes based on later assessment of titration
methods.

You can\'t help noticing that some kinds of adverse effects are not very
well studied in the ordinary clinical trial. The ones that are usually
cited, we have cited these in the gender document and in other places,
is sexual function. That\'s true for both men and women. And subtle
changes in cognition for drugs that have sedative properties.

It\'s unusual, probably because we haven\'t pressed it that much, to
actually have driving tests be done. Subtle alterations of mood are hard
to capture. You can ask people whether they are feeling a little of this
or that but really looking at it closely is difficult. And these can
become major issues.

The effects of the SSRI antidepressants on sexual functions, while
probably still inadequately represented in labeling, are a major problem
for that class of drugs. There\'s interest in whether isotretinoin
actually causes depression. It might be possible to do studies of those
things and they should be done, one would think, as early as possible,
as soon as there are hints of concern.

Sometimes that is premarketing or post-marketing. It seems at least
possible that these kinds of effects might be dose related or might
respond to drug holidays and so on, but you will never find out if you
don\'t look.

There\'s a brief section in here on pediatric studies. Really the urging
is to pay attention to the need for pediatric studies and how to do them
and to recognize that pediatric use poses special issues of growth and
neurocognitive development.

Having said that, it\'s not easy to know how to do those studies. We do
have a lot of information about impairment of growth from a wide variety
of drugs but really looking at neurocognition is really a challenge. I
guess I threw in the idea that it is better for kids not to have to take
drugs in the middle of the day because it\'s a big problem.

There\'s also a section, just to avoid alarming anybody, not to suggest
that one of these should be part of every database but that the
possibility of doing a large simple safety study should be considered
where that seems like it might be helpful.

Large is the thing everybody notices but simple is an important part of
this. Trials of this kind need to be specifically thought of as having
reduced data collection for the things you are not particularly worried
about. Otherwise, the challenge of doing them becomes overwhelming.

On the other hand, making short of data collection instruments and
thinking about targeted monitoring, not doing the usual every four
weeks, may make these studies more reasonable. It\'s interesting that
they are not as strange as they once were. There have been a number of
these.

There was recently a greater than 10,000 patient study of the new
antibiotic, another of a new COX-2 selective nonsteroidal
anti-inflammatory drug, and the new anti-hypertensive omapaztrilat,
because concern about a higher than typical rate of angioedema.

I have to say I was astonished at how willing the companies were to do
these. It\'s a very impressive thing. There are similar studies underway
post-marketing, ziprasidone versus olanzapine and the gigantic
salmeterol studies.

I don\'t know how much those cost, how difficult they were, how much
trouble they were, but it is obviously something that a company doesn\'t
automatically dismiss as unreasonable. What might make someone want to
do something like that?

Well, an obvious thing is a troublesome safety signal that really raises
questions about whether the drug can be approved at all. A good example
would be increased transaminase elevation to three-fold, five-fold,
whatever, coupled with a few people who accompany that by bilirubin.

Now, ordinarily unless the drug was for something quite important that
would imply the drugs can have great difficulty getting approved but a
larger database might inform you that was a fluke and it\'s not really
true.

Another case where you might think about this is when a drug is going to
be used for a really long time in completely asymptomatic people where
any kind of serious problem would be very troublesome.

In some cases where there are hints that there\'s a potential problem
of, say, QT prolongation but it doesn\'t look terribly bad. If it were
at the margin one might want to know how a completely unselected
population of more like the sort of people who you expect to get the
drug would fair given that problem.

There is also general advice to keep blood or other tissue samples for
potential later pharmacogenomic evaluation of people with serious events
or people you can single out as having better responses. Obviously you
need to pay attention to HIPAA consent issues and all those things and
I\'m not going to address that.

But this is part of a concept that really isn\'t in the paper but did
show up in the QT concept paper we wrote recently that taking a close
look at people who have bad reactions is really critical. Get a blood
level. See if there is something about them that is unusual. Maybe they
are taking too much. Maybe they have some property of interest.

I don\'t think that is typically done. That\'s why the QT document said
find those people. They may be very informative to you. That seems like
a generally important matter.

Then the document finally says there are some things you always need to
think about, one of which is QTC effects may be not for some biologics.
I think that could be true. You certainly have to pursue any potential
evidence of hepatotoxicity which is the leading cause of drug withdrawal
over time.

There is always an interest in polymorphic metabolism when that occurs
because it means a fraction of the patients are going to have a very
large excess of drug. One always needs to look at the relevant known
drug/drug interactions. One needs to keep track of the newly discovered
ones. There are inducers. We now know there are inhibitors of
glucoronidation and there are P lack of protein inhibitors. It\'s
important to keep alert.

I guess I want to put one plug in for population pharmacokinetics.
Mibefradil got in trouble because of its known adverse or known
inhibitory effect on 3A4. But they did a very careful population
pharmacokinetic analysis and were able to satisfy a tough advisory
committee with us that at least except for 3A4 it didn\'t do much else.

The data were very persuasive and very useful. I would say a much
cheaper way to do it than to do 25 drug/drug interaction studies. It\'s
worth thinking about.

For biologics I think the view is that immunogenicity and the presence
of neutralizing antibodies is always of interest. If we are thinking
about live agents, you are always interested in virulence,
transmissibility, genetic stability.

There are people here who can answer questions about those things. When
you are talking about transplantation therapies, you are always
interested in survival obviously, function of the organ, and the status
of host immunoconfidence.

Jerry Phillips is here to answer questions about these and I thank him
for providing these slides. There is a section in the concept paper on
medication error prevention analysis, MEPA, with at least some hope that
the industry would do more of it and make Jerry and his people do less
of it.

So this refers to serious premarket assessment to try to increase the
safety use of the drug products by minimizing medication errors related
to naming labeling and/or packaging.

Proprietary and generic names need to

be \-- I\'m not sure what you do about it if it\'s the generic name \--
need to be looked at for sound-alike and look-alike similarities, at
least until all orders are written on computers or spoken by robots.
This can involve expert analyses and/or focus groups, simulations of
verbal orders, simulation of handwritten orders, again, introducing the
usual carelessness.

There are apparently computer methods that help you do these things.
That should be part of what one is thinking about and considering in
benefit and risk. Packaging, too, can contribute to error so that needs
to be looked at.

Finally, the labeling needs to be looked at for container label
similarities within a product line or class. Similarities with other
products that would lead to confusion and the adequacy of critical
information in a prominent place and whether dosing instructions are
clear and easily understood. As I said, Jerry can expand on those
things.

Anyway, we look forward to further discussion. I have certainly enjoyed
the morning so far and we are sure the experience of this audience can
refine and expand the thoughts in the concept paper. Thanks.

MS. HENDERSON: Thank you, Bob. We have four registered speakers, Dr.
Gary Stein, Dr. Michael Cohen, Dr. Anshu Vashishtha. I apologize if I
have completely maligned that name. And lastly Dr. Penelope K. Manasco.
We will go in that order so our first speaker is Dr. Gary Stein. He is
from the American Society of Health System Pharmacists.

Dr. Stein, are you here?

DR. STEIN: Thank you. My name is Gary Stein and I am the Director of
Federal Regulatory Affairs of the American Society of Health System
Pharmacists. ASHP is a 30,000 member national professional and
scientific association that represents pharmacists who practice in
hospitals, health maintenance organizations, long-term care facilities,
home care agencies and other components of organized health care
systems.

We are grateful to the FDA for calling this public workshop to receive
input on the agency\'s approach to risk assessment of prescription
drugs. ASHP intends to provide more extensive written comments on the
FDA\'s approach through risk assessment by the April 30th deadline.

Today I would like to discuss one major point that we are very satisfied
with in the FDA\'s concept paper. Section 3(f) of the concept paper
discusses how drug sponsors can minimize medication errors.

Specifically this section states, \"Ideally a sponsor would conduct a
risk assessment to ensure that a product\'s proprietary name,
established name, container label, carton labeling, package insert,
and/or packaging do not inadvertently contribute to medication errors.

The concept paper goes on to state the sponsors should assess a
product\'s name, labeling, and packaging by obtaining first-hand
information from physicians, pharmacists, nurses, and consumers. This
sponsor initiated assessment would help to minimize medication errors
and help speed FDA\'s review of these issues.

ASHP welcomes and strongly supports inclusion of this language in any
future guidance document relating to premarket risk assessment that is
issued by the FDA. We urge the agency to quickly implement this concept.

We have been encouraging FDA to do this for a long, long time. In
September 1998 we stated at an FDA health professional organization
meeting that drug naming, packaging, and labeling is a critical issue
that has not been adequately addressed by the agency despite the fact
that there have been abundant evidence that poor product design is a
major contributing factor in medication errors.

The following year we stated that one solution to the problem of
medication errors stemming from poor package design and nomenclature is
to require real-life submission from the pharmaceutical industry prior
to drug approval.

Before the FDA approves any new drug or biological product, it should
require manufacturers to document that it has rigorously tested all
packaging, naming, and labeling for their potential to induce errors.

This testing should be done using proven methods involving practicing
pharmacists, physicians, and nurses in simulated work environments. We
also commented that the FDA has an obligation to quickly review and
revise its procedures to eliminate medication errors that occur due to
look-alike and sound-alike names, similarities in packaging, and other
labeling and packaging problems.

We also noted that patients should be considered the partners of health
professionals in eliminating medication errors and that they should be
involved in providing input into the safety design of drug product
labeling. We are especially pleased that the FDA concept paper includes
a provision for patient consumer input.

In 2002, in comments to the agency on its performance goals for the
reauthorization of the Prescription Drug Marketing Act we stated that
the most consistent message ASHP hears from its members is that the FDA
should be doing more to assure that drugs are safe for patients and that
safety issues must be anticipated through premarket evaluation.

One specific new performance goal that we recommended was for the FDA to
engage pharmacists, physicians, nurses, and human factors experts in
documented failure mode and effect analyses of perspective product
nomenclature and labeling to minimize the opportunities for sound-alike
names and look-alike packaging for causing medication errors.

ASHP believes that the FDA is taking the right approach to this serious
public health issue and appreciates this opportunity to present its
comments relating to the FDA\'s program for risk assessment of
prescription drugs. Thank you.

MS. HENDERSON: Thank you very much, Dr. Stein.

Our next commenter is Dr. Michael Cohen. Dr. Cohen, who is also a member
of our responder panel, represents the Institute for Safe Medication
Practices.

DR. COHEN: Thank you very much. I think you\'re going to hear me echo a
lot of Gary Stein\'s comments, but that\'s okay. I don\'t mind if you
hear this a couple of times. We are also very happy that that
information was included in the document. I\'m also going to focus on
3(f).

You heard that I\'m from the Institute for Safe Medication Practices.
Just by way of background very quickly it\'s a nonprofit organization.
We are located in Huntington Valley, Pennsylvania.

For many years actually this whole program that we operate now with the
United States Pharmacopeia is a medication error reporting program. It
really began in the 1970s as a journal column and has progressed since
then. We have a staff of nurses and pharmacists and physicians.

We receive reports of medication errors from practitioners all over the
United States. We also have an operation in Canada that is very active
and also in Spain. We do an analysis of the errors. We investigate them.
We follow up, speak with people, and try to gather expertise and make
recommendations.

We have some publications as well. I can tell you as Gary did that this
has become a major interest in the United States. It\'s not just the
Institute of Medicine Report that you know about from 1999 that has
generated this interest. It\'s been a long-standing interest for all of
us to prevent patient harm.

We are actually in a situation now where when medical errors happen it
affects things like our accreditation in hospitals or other practice
sites that are accredited by accreditation organizations. It affects
public perceptions, media perception of our organizations and our health
care practitioners.

Obviously it\'s having an incredible effect on the whole malpractice
crisis that you read about almost every day. Medical errors and
medication errors and device related errors are part of that obviously.
It affects licensure in some cases at our state boards and our state
health departments. They are still not over the understanding that when
an error takes place you can\'t focus blame on one individual. There are
many contributing factors that happen and they are not in the control of
one individual. I believe strongly from my experiences over the years
and the obvious.

When changes have been made in products where errors have taken place,
they are eliminated. I believe strongly that the pharmaceutical
industry, and also the device industry, has a major impact on the use of
pharmaceuticals in a safe manner by our practitioners and our consumers.

We have even had situations, by the way, where practitioners who have
made errors have been charged with criminally negligent homicide. This
is a major issue for all of us and obviously we need everyone\'s help.

In the past the help has been mostly having internal individuals
focusing on the labeling, the carton label, the media container label,
etc., the packaging and the drug name internally with internal staff.

Although many of the companies through their trademark attorneys now for
several years have been doing an analysis of the brand names, I do
believe that the nonproprietary name, the established name, should also
be tested.

We\'ve had a lot of look-alike problems, sound-alike problems. Most
physicians and pharmacists and nurses would tell you about the confusion
that exist.

I think probably in most cases the pharmaceutical companies do, in fact,
have an impact on the design of that established name as well. There are
many name pairs that have been confused with the result of patient harm.
I think they should, in fact, be tested as well.

But this testing can\'t be done internally. As it states in the
document, it\'s got to use people that actually work with these products
outside of your organizations. It has to be in the environment in which
it\'s used. We heard from a pediatrician before, for example.

I can tell you the way that medications are used in pediatric patients.
The way that they are prepared and dispensed is far different than adult
medications. You would have to for a pediatric medication obviously
concern yourself with the environment in which that product is going to
be used.

Now FDA is asking companies whether they are brand manufacturers or
generic manufacturers, distributors \-- I\'m sorry, packagers, labelers,
whatever, to analyze these products for safety using real world scenario
testing which is what we just heard. And health care practitioners,
ancillary staff and, in many cases, consumers. I agree with that.

I can tell you that when we do receive reports through the medication
error reporting program, and it\'s activated simply by dialing in a
1-800 number. Mostly we get errors reported through e-mail. These are
done voluntarily. We can call back the reporter.

In fact, I believe that most of the errors that are in the FDA\'s Med
Watch database that have not come directly through practitioner to the
pharmaceutical company actually come through the USPI Medication Errors
Reporting Program.

The beauty of it is that you can, in fact, get back to individuals and
they are very altruistic about sharing information. They will tell you
exactly what happened and you can analyze what actually went wrong. This
program is very valuable for that reason.

I would say that about 40 to 50 percent of the errors that we have
reported to us these days are directly related to medical products in
some way. That may not be an accurate reflection of what is going on out
there. We certainly know there are many practice related errors, but
I\'m talking about the ones that somehow relate to medical products.

I guess the data might be skewed a little when you look at this because
we are known for being able to follow up on product issues. We make sure
that FDA receives a copy of the report in every case. We do try to get
back to the manufacturers and have direct discussions with them.

If we feel something is serious enough we try to advocate for change.
Because the program is known for that we do tend to get more medical
product reports than other types of reports.

It\'s no longer adequate enough for industry to rely on internal
decision making and relying on everyone to read labels properly. I wish
they did. That hasn\'t worked in the past, though. I can tell you we
still have errors because practitioners don\'t read labels.

By assuring that your label is well read, by bringing in human factors
experts that understand how products can so easily be misused, you would
be able to improve the safety of your products enhancing important
information on the label, for example.

It\'s also not enough to ask that practitioners write clearly. I wish
that were the case, too. Unfortunately we have computerized prescribing
on the horizon and many other technologies that will help us reduce
problems.

Even today, even after drug approval, we still get product mix-ups
because of look-alike, sound-alike. We do need to do more to prevent
that as a possibility. We fully support FDA\'s request for this
medication or prevention analyses strategy for safety assessment of each
product\'s proprietary and established name label and packaging.

We do have a publication that basically every hospital gets and they can
make as many copies of this as they want. It list the product errors and
analysis and recommendations that come from experts that we utilize from
outside of our organization. We publish this and many of our
organizations, in fact, do apply this information on a regular basis.
Many folks at FDA have received this as well.

I can tell you also that many of the organizations, whether they are
community pharmacies, independent chains, pharmacy benefits managers,
hospitals, long-term care facilities, those that are accredited in
particular do, in fact, do an analysis of the medications that they are
about to use and incorporate it as part of their formulary analysis
process.

Without a doubt there are times when they feel that a product, an
alternative product might be used instead of one that might be a
liability as far as potential for error because the label is unclear or
the nomenclature is a problem. We certainly have seen evidence of that.

We are recommending and others have recommended, in fact, the Joint
Commission on Accreditation of Health Care Organizations actually
requires of accredited organizations that they do proactive risk
assessment. One way to do that is a process used in industry outside of
the pharmaceutical industry called Failure Mode and Failure Analysis
which is mentioned in the document.

I can tell you that in many cases the formulary committees are actually
doing this now and they look at information that is published not only
by us but also by FDA and the United States Pharmacopeia that relates to
errors to see if product errors have actually happened. If they have, at
times they are able to use an alternative product in order to avoid the
possibility.

What I did was I put together some slides, which are not in the book
actually, but they will be available on the FDA website. It speaks a
little bit about how this process might be conducted. This is certainly
not completely comprehensive, but basically you set up a process flow
diagram.

You understand how the product will be used in the environment, where it
would be used. You analyze the potential for error and how likely it
would be that it would be caught before it would actually be given to
the patient and how severe the outcome might be. That can actually be
given a score.

I can tell you right now that the major university hospitals in this
country are working together at this moment to produce a document that
they will all use that will, in fact, have them run through a failure
mode and effects analysis when they are doing formulary decision.
Obviously this is important.

You have seen already some of the things that Dr. Temple showed that
people can look at to see if there is a similarity. Obviously the more
similar things are, the more likely it is that an error actually
happens. This will be on the slide so you don\'t have to actually go
over that right now.

We not only look at the name but also \-- we should not only be looking
at the name but also the package label. How similar something looks to
something else was mentioned. That is certainly important. Where it is
stored. The concentration expression.

We have certainly had a lot of very serious overdoses where the
concentration was expressed on a per milliliter basis rather than per
container basis. Unfortunately, we\'ve had many fatalities, serious
patient harm reported in the past.

The problem elements such as these. Readability. Again, I don\'t have
time to literally spend with each of these so I\'m just breezing through
them and making this information available on the FDA website.

Also, focusing on specific patient populations is something else that
people are doing. The high risk patient population in particular. Renal
patients, liver patients, patients that are pregnant, patients that are
breast feeding.

Obviously the neonates and pediatric patients, the elderly, oncology
patients for obvious reasons with the types of drugs we use,
chemotherapeutic agents and the background that we\'ve had in this
country with errors in that particular field.

Finally, we are also looking at things like bar coding and
computerization and unidose packaging, whether that is available or not.
ASHP also fully supports the bar code initiative that FDA has just
published recently, the proposed rules for bar coding.

By the way, I\'ve heard a little push back about this already and that
concerns me about the smaller vials and ampules. This is getting off the
track for just a second but most of it has to do with the cost that are
involved with applying the bar codes or the lack of real estate.

I think we can get beyond these issues fairly easily because if you talk
to the hospitals, you will find they are probably spending a lot more
per dose to repackage these things. Many of them, according to surveys
that we\'ve done, will be very happy to pay a reasonable upcharge to get
the bar codes on those products so that we can use bar code scanning.

I think that will dramatically reduce the number of errors which also
relates to the new rules on medication error reporting. Obviously the
better job you do with failure analysis, the better job you do with
helping practitioners avoid medication errors and the less you are going
to have to contend with as far as doing investigations and analysis and
reporting them back to FDA should that rule be finalized.  All of this
is extremely important.

My final recommendations, practitioner input is critical. You cannot do
this internally. You need to test products with practitioners,
consumers, etc., pharmacy technicians, unit clerks on the outside.

You need to work with group purchasing organizations that are doing this
type of work right now and they are hearing it from their customers,
\"We need safe products.\" They are, in fact, eliminating certain
products from the formularies that they make available to the hospitals
and other practice sites. You should be working with them and
understanding their concerns better and responding to them.

And following up on medication error prevention literature being aware
of things that are being published so that you don\'t create the same
types of mistakes that have been made by other companies.

Then finally addressing issues and this is on the post-marketing side,
but just one comment. Many issues are raised by practitioners and I\'m
not sure what the reason for it is, but some of the responses are really
not helpful. Basically that we\'ve added into the database and will
continue to monitor it is very frustrating to practitioners who spent a
lot of time reporting this and hoping that something good comes out of
it.

Certainly we have a lot of companies that do a great job following up
and lots of changes have been made as soon as something is recognized if
it did reach the market without the fix. Unfortunately, we do have some
comments like that that come back to practitioners and nothing is done.
Despite repeated reports coming through we still have the same errors.

I do think we could do a much better job with the practitioner testing
and I\'m very happy to support it in full. Thank you.

MS. HENDERSON: Thank you very much, Dr. Cohen.

Our next speaker is Dr. Anshu Vashishtha from Watson Pharmaceuticals.

DR. VASHISHTHA: Good morning panel members and members of the committees
and the audience. It is definitely an honor to be able to share this
dialogue with the panel and what has been put together.

It seems to me as we are looking at it that we are looking at an
elephant. It\'s a big problem to grapple and we are all looking at our
own side of it and trying to come to grips with it.

As an internist who has been involved in clinical research in the
pharmaceutical industry and in drug safety, what I would like to talk
about today has to do with how we present this elephant of risk in the
investigation of prescription drugs. Some thoughts about how we talk
about it.

If I can go to the first slide, I think the investigative brochure has a
critical role in development presence that is filed for the investigator
focusing on this and also can lay the foundation for the package insert.
If we address it properly it can lead to a better package insert which
can then lead to better risk management and the product after
development as well.

The organization of safety information in the investigative brochure it
seems to me is a critical component that we need to address as we talk
about risk in the premarketing product. The concept that one can start
with, for example, and maybe the panel already talked about it but
perhaps there is some more discussion on the concept of developmental
safety information on DCSI group.

I think it is critical that we are able to present information so that
investigators can understand what the known risks are and also lead to
better data collection related to risk if it is understood better.

I think some limitations, of course, of the investigative brochure was
the first involvement in clinical research and drug safety. They seem to
be a lot informed. One critical point that I think is consistent more or
less is we do not differentiate what is an adverse event from an adverse
reaction.

I know there may be some discussion in sense of an adverse reaction
being related and not just observed in the clinical trial population. It
gets us into the whole realm.

I think some talk by the sponsor in trying to differentiate adverse
information in the investigative brochure to adverse reactions is
considered causal relationship and adverse events where they were just
events reported in the trial since we collect all that but do not
necessarily have a causal relationship with the drug would be helpful.

The frequency of adverse reactions again if we don\'t assess what is a
true causal relationship of the event we cannot give the true frequency
of what they believe to be the risk so we just outline the frequency of
the adverse event which may not really reflect what is happening.

Then there is a show which we may consider whether an adverse event
which is not considered to be drug related by the sponsor should really
be considered labeled for the purposes of expedited reporting, for
instance.

Delving a little bit more then into the concept of the safety
information and some other points beyond that, this was, of course, the
working group 3 and 5 guidelines which is indicated on the slide which
suggest you have to collect all safety related information in one
section which would reflect the safety portion of the anticipated
co-safety information upon approval of the drug so whether it relates to
the population exposed or the contraindications warnings or precautions,
the known adverse reactions.

If this is done in this way which is an attempt to really understand the
risk of the investigational drug, it will help us assess what do we need
to assess in terms of this management on approval.

Again, the issue of separating reactions, there is something I would
like to add to that, that we try to assess causal relationship and drug
advisability, or at least have a discussion around that advisability of
that in this panel, and the impact on those adverse events versus
reactions listed.

Regarding toxicity also we indicate frequencies in the investigative
brochures in different studies are sometimes full studies. It would be
helpful to again characterize them for reactions along the criteria of
the guidelines of very common being one in 10 or more frequent than
that, common being one in 10 to one and 100, uncommon being one in 100
to one in 1,000 estimated frequencies, and rare being one in 10,000 with
very rare being less than one in 10,000.

They may not apply directly to clinical data but I think as we go
towards approval it also helps us think about what sort of studies we
need to really differentiate whether there is a true risk or not.

If you are talking of very rare reaction where the frequency may be one
in 10,000 or less, you\'re talking as was presented in the FDA
presentation earlier a 30,000 sample size to even have a good chance of
detecting that event.

If you want to see these five or 10 events to be able to analyze whether
it\'s really due to the drug or not as opposed to just observed in your
clinical sample, the population goes up that many fold. Consideration of
sample size based on the frequency will help plan this management
approach.

I think a medical review of the safety information and the developmental
co-safety information would look at some of these things and lead to a
better risk management down the road.

Of course, if the product is already approved, then the label approved
and that jurisdiction could form the last part.

I would just like to again comment on a few points where comments were
requested in the paper. The size of databases for acute products should
depend on what factors I think was one of the points when these comments
were asked. It does, of course, depend on the safety part of the
compound and what the anticipated risk is.

The frequency of that risk again based on the DCSI would determine the
size of the database that you want to exclude. I think this again is a
question of dialogue but then what is the risk we need to exclude in a
clinical develop program.

We can\'t exclude all risks but what we can agree on is depending on the
indication and anticipated benefit on the population that will use the
drug, are we comfortable excluding one in 1,000 risk and, therefore,
determine the sample size you would need. Or one in 100 depending on the
population you are studying.

For chronically used products and ICH guidance, I think we have already
talked about the exceptions. I will not add anything more there.

The other portion, the area where comments were asked about the analysis
of missing data. Again, based on just experience in clinical research
and drug safety, I think some of the approaches we can take where safety
applies we are seeing are indicated here.

One is to look for proportionately greater frequency of lost to follow
up in the treated group with placebo which may indicate cost for
evaluating a safety concern. And evaluating the frequency of the adverse
events and abnormalities in the time period following was when placebo
was present.

Another approach might be to look at the adverse event in the subgroup
of the patients who are lost to follow-up because if patients who are
lost to follow-up happen to have high transaminase, that might indicate
something about risk management down the road.

I think we already had some discussion about large simple safety study
commitments. It does seem like these might be most effectively addressed
after approval as opposed to before but that is another discussion.

On line 479 there was a comment to exclude data from pharmacodynamic and
pharmacokinetic studies for safety purposes. That may make sense in
pooling as we discussed, there needs to be a minimal dose to be used and
data from that dose should be considered for pooling safety data.

Some adverse reactions observed in these studies would be relevant
perhaps in the adverse event. Or where hypersensitivity reaction or
where toxicity was seen during those studies they might well be relevant
for safety information or explaining of the product.

With those comments then I would like to end again by thanking the panel
for the opportunity and putting together a very good concept paper to
give us something to chew on and to share thoughts on.

I would, again, emphasize that the investigative brochure not only is a
tool that conveys risk to the investigator during clinical development,
but if we use the DCSI concept, it can lead to better risk management
planning by quantitating the risk and really assessing the relationship
of any adverse events or adverse reactions.

With that I\'ll end my comments.

MS. HENDERSON: Thank you very much for taking the opportunity to address
the panel.

Our last scheduled speaker in this session is Dr. Penelope K. Manasco
from First Genetic Trust.

DR. MANASCO: First of all, thank you for the opportunity to speak today.
I would like to commend the FDA for their efforts to enhance the safety
of medicines in the market place. The briefing documents in this meeting
are testament to the FDA\'s inclusionary approach to evolving the role
of the FDA from that of a strictly regulatory agency to a public health
agency.

I\'m going to start today by just maybe reiterating some numbers that
everyone knows, but I think it\'s important because as we look at how to
respond to the FDA\'s concept paper, I think we have to say that the
system that we have today is not working as effectively as it could
work.

I\'m sorry. I can\'t see my slide. So some of these numbers you may not
be aware of but the morbidity and the mortality for ambulatory patients
in 2000 was estimated to be \$177 billion. Now, that is more than the
cost of drugs for last year. If you add the two together, it is a
significant issue for our economy. And since 1995 that estimate has
doubled. More than doubled, excuse me.

Some additional data is that during the past 25 years one in five
medicines were found to have serious side effects that were not
recognized at the time of marketing. That means the systems that we have
now for clinical trials are not sufficient.

We do need to look at additional approaches. The last point that I think
has been published many times is that the rate of drug withdrawals has
not changed over time.

I\'m here today to talk about not the approaches that have been talked
about in the concept paper which are essentially expanding on
traditional approaches, but actually trying to raise your thinking to
think about using new science, new technology, and new approaches to
look at evaluating risk assessment.

Dr. Temple today spoke briefly about pharmacogenetics and I wanted to
take a minute because many people still believe this is pie in the sky
and not true science. We have the tools now to be able to address the
science of functional or clinical genetics or genomics.

The map of the genome, as everyone knows, has been published. There are
over 2 million \-- or will be by the end of the year \-- over 2 million
single nuclear type polymorphisms mapped that are available to everybody
to use to do their analysis.

Genotyping has dropped from a cost of about a dollar a genotype down to
pennies and 10th of a penny depending on the approach that you take. One
of the pieces that is critical to take advantage of, though, are
patients.

As you think about risk assessment, I think the comment that Dr. Temple
made about collecting patients, collecting DNA on the people who have
serious adverse events can make a huge difference. Instead of just
counting the adverse events and coming up with signals, you can actually
do something about it.

Last but not least, available now are IT infrastructures and molecular
datasets that will allow you to look at or address informed consent,
address security, and address the functional integration of multiple
sources. I\'ll take just a minute to show some data on these for you
today.

This is probably one of the most common examples of pharmacogenetics
that work. This is the TPMT gene which is the thiopurine
methyltransferase gene. The polymorphisms in this gene are already
instructing dosing at a number of institutions, particularly in cancer
therapy, leukemia therapy, but also in transplant therapy now and other
places where azathioprine is being used.

If you have the wild type form of this gene, as you can see at the top,
you actually take a standard dose. If you have \-- if you are
heterozygote, then the dose is actually decreased some. If you are a
homozygote for the mutations, one of the mutations that makes the gene
not work, the dose is much, much smaller.

Now, the heterozygote are about 10 percent of the caucasian population.
The homozygote are about .3 percent. If you do not make those dozing
adjustments, those patients are at significant risk of bone marrow
suppression and fatal adverse events as a result of standard dosing
whether it\'s 6-mercaptopurine or other drugs.

This is another example. This is an example that actually didn\'t make
it to market, but the UDPGT1A1 has in the top box which is part of the
gene there is variability there, as well as there is variability across
the genome.

If you have seven repeats, that\'s the mutation. If you have five or
six, it\'s the wild type. In a study that was done \-- this was a drug
that was in development which actually the efficacy did not come to
fruition. If you were 77, then those were the patients that were
actually \-- that\'s the mutation that is responsible for Gilbert\'s
Syndrome, 77.

That meant that those patients, 92 percent of them either were 77 or 76
repeats. None of the 66, which is the wild type, have the
hyperbilirubinemia. You can see that this is an example of a gene
environment interaction, a gene drug interaction that can be very
instructive in drug development.

This is another example of a study that was done for Abacavir which is a
protease inhibitor looking where DNA was collected in the patients that
had hypersensitivity syndrome and compared to a control group that did
not. This is just looking at the candidate gene work where patients who
were HLA-B57 positive, the cases, as you can see, were far over
represented for this form of HLA where the controls were actually very
low.

You can see that this gene in and of itself would not be sufficient, but
actually additional work shows that a combination of a multiple of
snips, some of which are highly specific and not very sensitive, and
others that are highly sensitive and not very specific can combine to
make a very effective test.

Each of these examples actually have been replicated multiple times in
the population. These are not single studies. These are studies that
show that these data actually can make a difference in therapy. Instead
of just knowing that you have an adverse event, it can actually help you
start to move into preventing those adverse events.

The second piece of this strategy involves IT infrastructure. With the
Internet, with the advent of Internet banking and highly secure other
kinds of transactions that take place over the Internet, it is now
possible to actually have an infrastructure that can provide electronic
consent, that can enable reconsent, that can provide education and
include patients and their participation of their health care.

As you look at the concept paper think about how that could actually
address a number of the issues that have been brought up in the concept
paper. This is actually a schema that was in an accenture document
showing that through a connected IT infrastructure you can actually
enable patients to get more involved. You can enable education. You can
enable data collection. You can actually enable sample collection to
facilitate pharmacogenomic research.

Last but not least, through this kind of IT connectivity, it is possible
to connect groups of researchers that in the past may have been
considered only as a single approach. I am showing here a network called
NetMeDS which is a network for marketed drug surveillance.

Through a common IT infrastructure these different large integrated
health providers can actually support the collection of the kinds of
data that you are looking at to give us better information about
background rates, but can also facilitate the collection of DNA sampling
and consenting so that we could actually do more than just count the
number of adverse events. We can actually do something about them.

So the recommendations we would like to see the FDA consider, instead of
just a paragraph about pharmacogenetics would be to consider stronger
language requiring biosample collection. I\'m focusing on genetics here
but it could be other kinds of biomarker sample collection because I
think that is a very important piece to try to start to intervene in the
adverse event area.

The second, I think, goes to some of the comments that have been made by
a number of people today, and that is to consider implementing test
programs. The concept document and the guidance are going to take
probably a year or year and a half to progress. This would be an
opportune time to consider starting some test programs to actually
provide data so that we can see whether these approaches would work.

Among those could be programs that would instead of looking at voluntary
reporting would be to look at required reporting for adverse events or
medical errors, biosample collection, as I\'ve mentioned before, and use
in broader populations.

Last, but not least, I think we spent a lot of time thinking about
people in the audience doing new drug development, but I don\'t think we
can lose track of the fact that there is a lot of safety data that needs
to be collected and a lot of opportunities for generic drugs and
over-the-counter drugs.

These drugs actually are used by a much broader part of the population.
I think the FDA could take a very strong leadership role in developing
and supporting the development and validation of pharmacogenetic safety
markers in those populations as well.

Thank you very much.

MS. HENDERSON: Thank you very much, Dr. Manasco.

We are now going to open the floor for discussion again, for questions
of clarification, or for discussion with any of the presenters or anyone
from any of our panels, or from the audience.

Before we do that, however, I was asked at the break to re-identify the
groups of people who are here, and particularly to identify our invited
panel in the center since their backs are to you and their nametags face
me. For the benefit of those in the audience that were not here when we
introduced our panels to begin with, to my left is Dr. Steven Galson and
to his left is Dr. Paul Seligman. They are chair and vice chair of the
CBER/CDER working group on risk management.

To my right is Dr. Bob Meyer who is the chair of the working group that
created this particular guidance on risk assessment. The panelists who
are to my left, your right, at the tables are the members of the working
group who developed the concept paper that we are now discussing. The
folks at the tables to the right are members of the joint CBER/CDER
steering committee.

In the center are our invited guests and respondents. To my left, so to
your right, on the far right you have heard from already, is Dr. Michael
Cohen who is from the Institute of Safe Medication Practices.

The chair next to him is vacant. Next to the vacant chair is Dr. Rob
Califf who is from the Duke Clinical Research Institute at Duke
University. Next to him is Dr. Christy Chaung-Stein who is representing
PhRMA today.

Next to her is Dr. Glenn Gormley, also representing PhRMA from
AstraZeneca. And to my far right at that table is Dr. Richard Simon who
is from the National Cancer Institute.

I will try to identify people by name for the purposes of transcription
and for those in the audience that can\'t see who is talking. As best I
can I\'ll ask people who come to the mike to please identify themselves.
Also if anyone from the working group or the steering committee speak if
you would just identify yourself before you speak.

At that I will open up the floor. Who else? Dr. Califf.

DR. CALIFF: Maybe I can start by challenging my colleagues to the left
here. As I said before, I think we all realize we\'re in this transition
phase now in understanding biology. I think Dr. Manasco\'s presentation
laid out some really key issues and even asked for a requirement for
genetic collection of genes.

In my experience in working with PhRMA in particular, there\'s a lot of
schizophrenia about this because it will segment the market. In fact,
very often this is either not done or it\'s not done in a way that is
sort of accessible to the investigators. Do you see this sort of
collecting change as a routine to look for populations at risk for
toxicity or at risk for a nonpositive benefit risk balance being the way
to go?

MS. HENDERSON: Dr. Gormley.

DR. GORMLEY: I\'m going to separate my comments here in two ways. My
main focus being here is to represent PhRMA which is a collective of
groups that have probably a very wide view on the question you\'re
asking. I don\'t think I would be in a position to be able to represent
that completely.

In my dialogue with my colleagues around the pharmaceutical industry, I
think everyone understands that this is going to be necessary for the
future. It is coming whether one likes it or not and it is better to
embrace it and each company is embracing it in their own way.

Some of our colleagues are right at the forefront of that and investing
huge amounts in making sure that they are able to capitalize on it and
understand it. Others have taken a follower mode to try and understand
where this is going to go. It\'s difficult from that perspective.

My own view is that we absolutely have to embrace this. This is going to
be the world of the future. We need to understand if there are specific
toxicities that relate to specific groups of people, understanding who
they are will no longer be acceptable to just homogenize those into
databases.

Having said that, there is a lot of work that needs to be done to
understand exactly how to implement that. If we move too quickly and try
to make too many segments before the technology and our understanding of
that technology fully matured, I think we will create more chaos than
benefit. This is one of those cases where I believe that the technology
is ahead of our understanding of it when they are in need to embrace it.

MS. HENDERSON: Thank you. Dr. Manasco.

DR. MANASCO: I just wanted to make a point, Rob. There is actually a
group called the Pharmacogenomics Working Group that is made up of about
14 different PhRMA companies each of whom is doing pharmacogenetics.
They have actually done some very good work. They have helped plan a
workshop with the FDA.

They have come up with guidance about different terminology and about
issues and consent. I do believe there has been some very good activity
coming out of the pharmaceutical industry and that group was actually
just ground swell from different PhRMA companies.

DR. GORMLEY: Would you agree with my assertion that there is quite a
wide range, though, of engagement on this, or do you think it is more
homogeneous?

DR. CALIFF: I feel compelled to comment on that. Being on a lot of
steering committees designing trials, we know there\'s a wide range. In
fact, the marketing groups and companies almost routinely will try to
fight against us because of concerns that we all know. I think it is a
case of technology being ahead of the field.

Just the construct of how you get doctors to administer a test routinely
when it\'s hard to even take aspirin, as a great example. We can\'t even
give aspirin to more than about 80 percent of people who should get it.
Then you start making it more complicated. There\'s a long way to go.

MS. HENDERSON: Dr. Chaung-Stein.

DR. CHAUNG-STEIN: I know several companies are routinely banking their
samples thinking that we will need to move on with this technology. They
all have to work out how to do the informed consent. I have heard about
different companies taking different approaches so we need to come to
some uniform approach to tackle those issues.

But the companies do realize that we are at a new era and that we need
to move along. If we can help identify individuals at greater risk, we
need to move forward. All the gene banking is going on for a lot of
companies at this point.

MS. HENDERSON: Thank you. We have a commenter at the mike.

DR. GORMLEY: Could I just add something? HIPAA regulations and privacy
issues that we are now dealing with only complicate how to move forward
on that point.

MS. HENDERSON: Thank you. Questioner at the mike.

DR. LEWIS: Ben Lewis, Managing Director of Regulatory Affairs, Rand
Institute. Recently FDA published in the [Federal Register]{.underline}
a listing of various guidance documents that they would potentially
publish this year.

I know in the past the Office of Drug Safety, specifically OPDRA, now
called DMETS, was considering publishing a guidance document for
medication errors. I just wonder if there is any comment on any
forthcoming guidance document on that. Thank you.

MS. HENDERSON: Jerry, did you want to address that?

DR. PHILLIPS: It\'s in progress. We are developing several guidance
documents, one on the process of submitting names to the center which is
probably closer to getting out in final. I\'m more optimistic about that
particular guidance document.

The other guidance documents that we are working on, there is one that
will deal with labeling and packaging issues on how you label products.
And the third, of course that we are talking about today, is about how
you do testing of trade names.

DR. LEWIS: Thank you.

MS. HENDERSON: Another commenter at the mike.

DR. KONDROS: Dr. Kondros from Greenthal USA. I have a question to Dr.
Manasco in terms of pharmacogenomics. We are performing several Phase II
clinical trials in the U.S. and in Europe doing pharmacogenomics for
future testing.

We experienced major difficulties in terms of the constant procedures
for this future testing, not so much from the patients, they are willing
to participate, but from the IRBs of the corresponding bioethics and
these problems are in Europe much more emphasized than here in the
States. HIPAA is not a problem at all because in major European
countries this is already established.

This is our experience that we are facing the discussions with the
bioethics or with the IRBs and the FDA which are not willing to accept
not clearly defined future testing in pharmacogenomics, although we
commit to ensure all the privacy rules and the ethical considerations.

MS. HENDERSON: Dr. Manasco, did you want to comment?

DR. MANASCO: Basically what is starting to happen now in the bioethics
community is that if you ask people to consent for future research, they
are saying that research is not truly informed. You cannot ask somebody
to say, \"Okay, you can do with my sample what you want to in the
future,\" because then they truly are not informed about what will
happen to that.

There are a number of different approaches that can happen but one of
them is to be specific. Most IRBs have actually been able to accept we
will look at drug response, drug metabolism and disease as the areas in
which you will do research. That has actually been something that has
been approved.

The other idea, which is what I brought up electronically, is it is
possible now to use electronic means to actually go back and reconsent
patients and still maintain privacy and confidentiality, but that is a
little bit out on the fringe but you can do that, but the other approach
should work as well.

MS. HENDERSON: Dr. Meyer has a comment.

DR. MEYER: These are obviously very important issues and we very
deliberately sidestep the ethical issues in our concept paper. I think
it will be dealt with more in other fora.

For instance, the agent is working on a pharmacogenetics guidance that
may touch upon these matters. While these are important discussions, I
would ask that we perhaps focus more on some of the basic issues in the
concept paper today. Thank you.

MS. HENDERSON: Dr. Simon.

DR. SIMON: I distinguish different kinds of
pharmacogenetic/pharmacogenomic studies, one focused kind where you have
some hypothesis that some gene or some polymorphism in some known gene
is related to a particular kind of adverse event.

The other is where basically you are going to do a fishing expedition on
a lot of snips. I guess I would caution the FDA from jumping in ahead of
the fact and short of requiring that specimens be collected on all
patients for sort of those kinds of fishing expeditions.

Even though I think in some cases those could wind up to be very useful
studies, they are not really targeted kinds of investigations that are
themselves an intrinsic part of a Phase III trial.

MS. HENDERSON: Thank you, Dr. Simon.

Commenter at the mike.

MR. MAYBERRY: Yes. My name is Peter Mayberry and my question this
morning is on behalf of the health care compliance packaging counsel.
I\'m not positive this is the appropriate forum for this question but
Dr. Cohen raised the bar code issue and proposal out of FDA on bar
coding and vis-a-vis risk management as proposed, the FDA would only
call for a bar code on the drug package allowing manufacturers to still
ship product in bulk so long as there is a single bar code.

You could theoretically with solid oral doses ship 10,000 tablets,
dosage units, with a single bar code. Are there any responses as to how
this would impact risk management or would not risk management be
facilitated if the bar code was on each dosage unit or, at the very
least, a unit of use package?

MS. HENDERSON: Jerry Phillips, should I direct that one back to you? Oh,
you look so sad. Dr. Galson?

DR. GALSON: Yeah. I think this is really out of the scope of this
meeting. First of all, it has more \-- pertains more to maybe the second
or third day of the meeting if it pertains to anything. Then we have
this proposed rule out for comment and I think it\'s appropriate to have
us be collecting comments. It\'s not really the appropriate time for us
to be specifically reacting to comments that may be on the rule. Thanks.

MS. HENDERSON: Thanks, Steven.

Other comments? Dr. Racoosin.

DR. RACOOSIN: Yes. I want to respond to the presenter from Watson
Pharmaceuticals about his emphasis on including drug related adverse
events and labeling and what not. I think in clinical trials
investigators have to make some distinction for themselves as to whether
something is drug related or not because they need to advise their
patient on how to proceed, whether they should continue or not.

I think when companies are analyzing some total data coming out of these
clinical trials, that it is a mistake to rely too heavily on an
investigator attribution because we don\'t want to exclude adverse
events that are potentially just odd that none of us would think might
be drug related, but there are plenty of examples where drugs have
unexpected side effects. If we were to rely exclusively on ones that
have assumed to be treatment related, we could really miss something.

I think in the scheme of things that as we work on how best to do the
clinical trials and look at the safety data, that we are really best
served by looking at all the safety data and not making that distinction
about an investigator attribution.

DR. VASHISHTHA: May I respond?

MS. HENDERSON: You certainly may. We have a response.

DR. VASHISHTHA: Thank you very much for bringing that up. It was a
misunderstanding then and not what I intended to convey at all. I think
investigator attribution has its limitations. The attribution of an
adverse event as an adverse reaction is very much dependent on the
understanding of the full drug from the toxicology preclinical studies,
perhaps the class of drugs, and takes all that into account.

I did not mean to say that adverse reaction would only be something an
investigator considered. Very often in an individual case you cannot
really make that attribution. It really would be something based on the
whole spectrum of safety data that the sponsor when compiling the
investigative brochure would make an attribution to.

Sometimes the investigator may attribute something to a drug which may
go to the other way. It may not really be drug related but they are
taking the cautious approach. Thank you for bringing it up.

MS. HENDERSON: Thank you.

Dr. Temple, you have a comment?

DR. TEMPLE: Yeah. I think Judy was pointing out our general approach to
adverse reactions which is to see anything and have it be submitted if
it\'s important, whatever a person\'s view of it is, whatever their
belief on the basis of animal data or any other data because we\'ve been
wrong.

Drugs have done things that were unpredictable, not seen in animals, and
totally illogical. We just weren\'t smart enough to know how they
managed to do that yet. So, for example, in submitting the results of
data in the safety database, we asked to see details about every person
who left the study prematurely or who died no matter what somebody
thought the reason was because sometimes you learn something about all
those.

It\'s hard to know what you should put in the investigative brochure but
we still expect to see and require under rules continued submission of
anything serious and unexpected whether it\'s attributable or not.

MS. HENDERSON: Dr. Califf.

DR. CALIFF: Temple probably knew this was coming, but just a plea for
the poor investigator out there now. A lot of the ideas that we\'ve
heard this morning are great ideas but they are totally dependent on
having a system in which you can actually get the information you need
from moderate to large numbers of patients.

I\'ve heard a fair amount said about one of the key limitations from the
industry perspective is how hard it is to do studies of an adequate
size. I think it\'s really critical for the FDA to be much more
proactive in streamlining the information collection and helping the
investigator make this happen.

Many of us believe the clinical research in the U.S. is grinding to a
halt at a fairly rapid pace now with HIPAA being sort of the straw that
is breaking the camel\'s back because the reaction that was described
about IRBs is really a fear of risk where it is considered to encumber
so much risk to do clinical research that it may not be worth it for
many people that have been doing it up until now. I think this is really
a major concern.

On top of that, fear of prosecution by OHRP in shutting the institution
down is actually multiple IRBs. I have circulated to Dr. Temple and
others at the FDA and the NIH some of the goings-on here. People are
actually reading these stupid adverse events that are sent from totally
unrelated studies to every IRB in the country taking up to 30 minutes
per event to provide documentation that shows that you did it.

I recently sent Bob an example of a global adverse event that occurred
with a commonly used drug that was shipped to all the IRBs of any center
doing any study with that drug. Thirty days later a report came back and
said never mind so both were sent out. Now you have an hour per adverse
event at every IRB in the United States essentially.

This is really creating an impossible situation to get the data that you
actually would like to have. Passive statements by the FDA are not going
to do the job. It\'s going to take a much more active combined role of
the FDA, the NIH, and other organizations to fix this problem.

MS. HENDERSON: At the microphone, please.

DR. KAHN: Sidney Kahn, Pharmacovigilance and Risk Management. I would
like to follow up that discussion because I recently had the opportunity
to have a conversation with a couple of IRB people at a conference and
they were complaining of precisely this problem of overload and of
receiving large numbers of entirely uninterpretable cases based on the
current reporting obligations.

I would also like to bring to the attention of this group the fact that
the CR6 working group is currently drafting a proposal that may suggest
that communications to investigators and IRBs should focus really only
on those observed events that do appear to represent some change in the
established benefit profile of the investigational product rather than
sending our barrages of essentially uninterpretable data. It might be
worth considering that as part of the risk management tools for clinical
trials going forward.

MS. HENDERSON: Thank you, Dr. Kahn.

Dr. Temple.

DR. TEMPLE: Following one of my many conversations with Rob, I actually
brought up something of this matter at one of our senior management team
meetings. There was suggestion that this document when it finally
emerges might be a useful place to describe the considerable flexibility
there is on reporting and making arrangements for reporting.

For what it\'s worth, right now whatever the specific rules about
reporting that are in regulations, a sponsor who wants to say, \"Okay,
here are some things I am not going to report one by one. They are
serious but they are part of the population so I\'m not going to report
them,\" can reach agreements with the review division to do exactly
that.

They can also reach agreements to not collect certain information. We
have in cardio-renal, for example, in large studies which said we
weren\'t interested in concomitant therapy if it wasn\'t a cardioactive
therapy. It wasn\'t very important. We already knew about the drug.

It\'s also possible not even to collect certain milder side effects if
you already understand the drug or collect them in a subset of the
population. All those things are possible. It does seem like a good idea
for this document as it comes out to describe the considerable
flexibility and the ability to say, \"Okay, these are the things I\'m
not going to report.\"

Why IRBs get trivial adverse reactions in boxcar amounts is not
understandable exactly to me. It\'s not any requirement of ours that I
can see. What I would expect them all to get is any serious unexpected
reaction that has been circulated to every investigator and the
investigator would be likely to turn that over on to the IRB. If those
are being done foolishly, if foolish thing are identified as serious
unexpected reactions, then that is perhaps a problem that we can
address.

MS. HENDERSON: Thank you, Bob.

Dr. Gormley.

DR. GORMLEY: Then just to enter into this discussion from the PhRMA
side, we find ourselves very much in the middle between the dialogue
that is going on right now. Folks like Rob Califf and thousands of
investigators are constantly calling us saying, \"Why are you sending us
all this information? We don\'t need it.\"

But when we read the regulations and try to interpret what is expected
from the FDA, we find ourselves compelled to do many of these things.
Hearing Bob\'s very open-minded perspective that there is a lot of
opportunity here to discuss and manage patient safety but doesn\'t
overwhelm the system is fantastic.

I\'m hoping that his view is held by many different divisions within the
agency because it\'s not the cardiorenal group that we had our biggest
challenges with. They are in oncology and in many other areas. We need
similar kinds of direction in a way that manages this tension and allows
us to do the right thing.

MS. HENDERSON: Dr. Kweder.

DR. KWEDER: I would like to follow up on the comment, the same topic
from the investigator perspective. We certainly have on many occasions
and often do suggest limiting data collection. We already know that
about this product. I don\'t think you need to spend investigator\'s
time collecting it.

Interestingly we do see this tension between the sponsors wanting to be
thorough and the investigators and sponsors worrying that the IRB is
going to come back and say, \"Oh, but no. You need to collect
everything.\"

It is a very complicated equation. I think maybe what we need to do is
go back and think about how to articulate when it\'s reasonable to not
collect certain things.

MS. HENDERSON: Dr. Califf.

DR. CALIFF: Just to continue to be adamant about this, I think people
need to realize that one of the consequences of the current stalemate is
that we are rapidly exporting clinical research outside the United
States. You will see an increasing number of the global clinical trials
having a minimal U.S. enrollment.

If you want data relevant to the U.S., it really has to be fixed. I
would say every global clinical trial we are involved in now, the U.S.
enrollment is less than 40 percent of expected.

An increasing number of patients from Eastern Europe, South America, and
Russia are dominating the enrollment in clinical trials. Nothing against
those countries. I\'m happy to have their data but it may not get you
what you want in terms of real world experience with safety to balance
with effectiveness.

MS. HENDERSON: Dr. Temple.

DR. TEMPLE: It would be helpful if you would identify what you thought
was the specific remediable reason. I also want to make a comment before
you answer. That is the same issue that goes to what data collect goes
to monitoring.

One of the things that I like to point out, courtesy of Rob again, is
that the monitoring in, I guess, GUSTO-1 cost \$7 million or \$10
million. That was the entire cost of doing the ISIS study in Europe
which didn\'t do a lot of local monitoring.

One of the things we also need \-- one of the things that the good
clinical practice guideline, ICH E-6, makes very clear is that
monitoring has to be determined according to the needs of the study,
that the usual industry standard of every four weeks, every site,
probably doesn\'t apply to a 40,000 patient study.

Okay. And that even in some cases there might be no need for on-site
monitoring at all depending on what you can do centrally in making the
case. That is another thing we probably need to emphasize. You can\'t
very well have large simple safety studies if you have to go every four
weeks to every site. It\'s just not going to be feasible.

That\'s probably something else we need to discuss.

Rob, what are the particular things that you think drive all the studies
over that we could change?

DR. CALIFF: I think it\'s very simple that in the U.S. the investigators
are completely beleaguered responding to regulatory \-- what is
perceived as regulatory requirements that make it almost impossible to
stay alive and also do clinical research at the same time.

In the IRBs you can make a list. It is very simple. The biggest single
problem is this mountain of SAEs that they use to disregard that they
now have to respond to. No one can see any useful purpose actually of
informing local practitioners about whether to participate in the
trials.

At the investigator level a lot of it really has to do with complicated
case report forms, concerns about the consequences of auditing the
upcoming HIPAA issue where if you make a mistake you are afraid you will
go to jail or be fined up to \$250,000 per occurrence which puts a
little bit of a damper on your enthusiasm for involving your patients in
the studies.

Probably the majority of it now is around IRBs. We\'ve gone from one to
five IRBs after our disaster two years ago where the institution was
shut down. It\'s taken longer with five IRBs than it did with one to get
protocol started.

It\'s not uncommon for us to be ready to start a protocol in a rapidly
enrolling trial and the trial is over because the patients have been
enrolled in other countries where it is easier to do this.

MS. HENDERSON: Any additional questions or comments? One more.

DR. MANASCO: I think if the FDA could have a dialogue with \-- I don\'t
know. OHRP is not OHRP anymore. I don\'t know what their name is. I
think if there could be a dialogue with whatever the new group is, that
could be very helpful to the trials.

I would like to just echo what Rob said. I think the time it takes to
get clinical trials going is incredible. If there could actually be some
discussion, I think that could be very, very helpful.

DR. CALIFF: I do want to mention people who read the [New England
Journal]{.underline}. Several weeks ago there was a really informative
discussion of the ARTSNET trial in which 20 IRBs, an external data
monitoring committee, all the investigators of 20 institutions, and an
external advisory committee have all been overruled by OHRP and an NIH
study has been on hold for 10 months. There have been multiple
investigations at the site so you begin to look at this and you say,
\"Why bother? You can see patients and do okay.\"

Dr. Franzen.

DR. FRANZEN: Tim Franzen from Lilly. A comment really and a question
about the aggregate of information. Taken individually each of these
sections can certainly enhance one\'s understanding of a study
population and, thereby, convey new information to practitioners.

In aggregate this certainly could represent as it is being implemented a
tremendous amount of new information which may in some cases be
confusing to practitioners and patients when compared to what is now
available.

What are some of the thoughts of the implementers as to how all of this
potentially voluminous information will aid in enlightening rather than
frightening recipients of medicine and information?

MS. HENDERSON: Dr. Temple would like to take a stab at that, Tim.

DR. TEMPLE: I think there isn\'t anything in there that hasn\'t been
done at least sometimes by some people. For example, you could probably
read that and say, \"Holy Cow, the dose response part of my application
is going to grow by 10 fold,\" or something.

But I can tell you antidepressants and anti hypertensives are all worked
up that way now. Their sample sizes and data bases are about the same
size as everybody else\'s. This is mostly, I believe, I would be
interested in what other people think, a matter of anticipating and
planning and building it into the program.

So you can have 2,000 people at one dose, or you can have 1,000 people
at each of two doses and maybe learn a lot more than you would have
otherwise and so on. Keeping blood samples, that\'s work but it doesn\'t
delay or do anything else.

You just have to get permission to do it and you have to plan that ahead
of time. But it isn\'t inherently slowing anything down. I don\'t think
anything in here is at odds with what is at least sometimes done in some
situations.

The question is the need is for an organized think about whether this is
a case where you should do that. If you build it in, it doesn\'t slow
anything down. It\'s not even inconceivable that you can learn something
that will create greater efficiencies.

For example, I know from time to time we have arguments that in
retrospect seem silly about events that showed up in a treatment group
where there was no control. If there had been a control group, maybe
that argument wouldn\'t have happened.

Maybe there would be more of them, too, as Rich pointed out. But there
is at least a possibility that you will learn enough to avoid problems.
Needless to say, our hope also is that you learn enough not to have to
yank drugs off the market or sharply restrict them because you\'ve got
the wrong dose or made a mistake or things like that.

I don\'t think we thought of this as a great expansion of the amount of
work. We thought of it as a more thoughtful approach to what\'s needed
sometimes with more data.

MS. HENDERSON: Anna, go ahead.

DR. SZARFMAN: My name is Anna Szarfman. I work at FDA. I am listening to
the problems with the IRB and how to separate the outliers from the
noise. This is being done under the alias of the Government how to
identify the specific sound, noise that is different to pick up in
something that could be related to an airplane that is being disabled or
a different type of airplane.

Here the problem that we are having is that we are not collecting
medical data using common standards. That is the biggest problem. If we
would be doing that, then IRBs could learn exactly what is expected from
what is not expected using a new statistical items that are very level.

Then pick up what is unexpected, what is frightening. This is being done
in many other areas by EPA, by DOD, by Genomics.

MS. HENDERSON: Thank you, Dr. Szarfman.

Dr. Meyer has a couple of questions he would like to ask our industry
participants before we break for lunch.

DR. MEYER: Yes. I wanted to ask, we\'ve had a couple of public
statements here from various individuals supporting the idea of the
MEPA, the medication error prevention analysis, being done and presented
by industry as part of their risk assessment. I wanted to get any
industry perspective that might be available on that understanding that
some of that may come from the written response to the docket as well.

DR. GOLDHAMMER: Alan Goldhammer, PhRMA. Bob, we are working on that
right now. We actually raised this issue, I guess, a week and a half ago
in combination with regulatory people and trademark lawyers. I think
this is something that we support quite strongly of internally having
the industry do those kinds of assessments and forwarding them onto FDA.
I think it is largely the plea to lighten Jerry\'s workload. There will
be comments coming from us on that.

DR. MEYER: Thank you.

MS. HENDERSON: Thank you, Dr. Goldhammer.

Barring any other questions or comments, we will break for lunch. It\'s
about \-- that clock says 12:18. We will say it\'s 12:20 and so return
to this room at 1:35.

(Whereupon, at 12:15 p.m. off the record for lunch to reconvene at 1:35
p.m.)

A-F-T-E-R-N-O-O-N S-E-S-S-I-O-N

1:39 p.m.

MS. HENDERSON: Thank you very much. We\'ll get started for this
afternoon. We have been asked by several folks if the presentation
materials will be available at all. We will put certainly the slides
that we have access to, all of the FDA slides, will be put up on the
Internet on the address that I gave earlier. If any of the other
presenters would be willing to give us copies of their slides, we would
appreciate it and we will put those on the web as well.

We have a new member of our responder panel who has arrived for this
afternoon and that\'s Dr. David Flockhart. Dr. Flockhart, welcome. We
are happy to have you.

Our first presentation of the day is on data presentation and analysis
and it will be presented by Dr. Ellis Unger from the Division of
Clinical Trial, Design, and Analysis in the Center for Biologics.

DR. UNGER: Good afternoon, everyone. My charge this afternoon is to
discuss the final third of the concept paper which relates to
considerations for data analysis and data presentation.

These topics are addressed in the 1988 guideline for the format and
content of the clinical and statistical sections of an application, as
well as the 1995 ICH E-3 guideline for structure and content of clinical
study reports. It\'s not our intent to supersede these guidances but
it\'s our intent basically to extend them into areas that are not well
covered.

The issues I\'ll talk about now are grouping of adverse events, temporal
relations between adverse events and product exposure, analyses of dose
effects, data pooling, and dealing with missing safety data.

In terms of coding and grouping of adverse events, I was glad to hear
some comment on this this morning. MedDRA or other coding dictionaries
should be used consistently throughout clinical development programs
with adverse events examined as coded. However, specific adverse events
or toxicities, those with a constellation of symptoms and signs, those
with common pathophysiological bases may merit analyses of group terms.

The utility of combining related terms is that one can amplify weak
safety signal. Linking terms by pathophysiologic mechanism or linking
symptom, signs, and laboratory abnormalities that all reflect the same
pathophysiologic situation.

As an example, ischemic events could include arterial thrombosis,
occlusion, embolism, claudication, ischemic necrosis, gangrene,
intestinal ischemia, pulmonary embolism, acute MI, enetc.

The down side of combining related terms is that there is the potential
to mask serious unusual events with less serious more common events. For
example, toxic megacolon could be masked by constipation. That\'s a
problem potentially.

Division of one event into many terms has the potential to decrease the
apparent incidence of an adverse event and we see examples of this
frequently. For example, including pedal edema, generalized edema,
peripheral edema as separate terms decreases the apparent incidence of
fluid retention because they are separated.

Whenever possible we recommend the sponsor respectively group adverse
event terms and develop case definitions in consultation with FDA. We
recognize that some groupings can only be constructed after the safety
data are obtained.

Next I\'ll talk about temporal associations between adverse events and
product exposure. Time to event analyses are not dealt with very much in
the guidelines. We believe that they are appropriate for clinically
important events that occur on a delayed basis.

For example, progression of disability, development of cardiac toxicity,
the need for surgical intervention, and malignancy. It is also
appropriate for adverse events that occur and initiation of treatment
but diminish in frequency over time.

Descriptions of risk is a function of duration of exposure or time since
initial exposure. There is life table analyses for cumulative incidence
of value, assessment of risk within discrete time intervals over the
observation period, a hazard rate curve to illustrate the change in risk
over time.

This is an example that was presented by Abbott at the Arthritis
Advisory Committee Meeting five weeks ago. It shows a TNF blocker at
infliximab timed to first malignancy for rheumatoid arthritis patients.

It also illustrates the value of a control group because one can look at
this and not really know what to do with it. The point is that one can
look at malignancy over time and compare it at a minimum to historic
data and get some sense of the risk.

Another example, this is interferon beta-1B. There are flu-like symptoms
with time. Interferons are known to cause flu-like symptoms at
initiation of treatment. One can hypothesize reasons why the incidence
of flu-like symptoms decreases over time. Maybe they are not reported.
Maybe they go away.

The point is by looking at the relationship over time with the product
in yellow versus placebo, you can look at the time dependency of the
events and you can convince yourself, I think, that basically they are
twice as frequently reported with interferon versus placebo.

The relationship between adverse events and exposure may help determine
whether an event is actually related to the product and, if so, the
magnitude of the risk. In terms of analyzing adverse events by dose,
there are several ways of doing this.

One is to look at adverse events as a function of the administered dose.
This is particularly true for dose ranging studies or simply development
programs that evaluate more than one dose. Adverse events can be
assessed as a function of weight or body surface area adjusted dose.

Adverse events can be assessed as a function of cumulative dose received
prior to the event. I\'ll show you an example of that in a moment.
Adverse events can be assessed as a function of drug plasma
concentration.

This is an example from the Clinical Development Program of Tenecteplase
for Acute MI. Tenecteplase is a thrombolytic agent and the toxicity that
we are concerned about with thrombolytic agents is bleeding,
particularly bleeding into the head because it frequently has
catastrophic consequences.

This slide shows basically sextiles of weight-adjusted dose on the X
axis and percent of subjects with events on the Y axis. You do not see a
dose response here which is good. It is suggested over the dose range
studied. There is no dose dependency of this particular adverse event.

This is an example from actually a very complicated development program,
Darbepoetin alfa versus Epoetin alfa for anemia of chronic renal
failure. Here a couple selected terms. Dyspnea and tachypnea are
assessed by the weight-adjusted dose administered in the four weeks
preceding the event.

Here we are looking for a signal. This is an agent Darbepoetin alfa that
is titrated to effect and it causes erythropoiesis in patients with
chronic renal failure. The issue is if you look at the adverse event by
total dose administered over the study, or the dose administered during
the week of the adverse event, that you may be missing something.

In fact, if you look at the adverse event by quartiles in the four weeks
preceding the event, you see what appears to be a signal and the
presence of an apparent dose response. This is an example of an adverse
event where there doesn\'t appear to be a dose response. This was back
pain.

For step-dosing algorithms there are some particular concerns. By
step-dosing algorithm we mean incremental dosing based on age, weight,
or body surface area. We recognize that there are cut points and the cut
points are necessarily arbitrary. We think it\'s important to make a
specific effort to examine safety and also efficacy at doses just above
and below the cut points.

I\'ll show you a special example of this. This is the dosing algorithm
for Tenecteplase, again, a thrombotic agent that is approved for use in
acute MI. The patient weights are shown on the left, 60 to 70, 70 to 80,
etc. The Tenecteplase doses are on the right.

You notice something interesting about this paradigm which is that if
one has a mass less than 60 they get 30 milligrams, whereas if a mass of
a patient actually is 60, that they are increased to the next dose.

Now, if patients were actually weighed and the weights were well
assessed it wouldn\'t be an issue but, in fact, a lot of people round
weights. for acute MI a lot of people come into an emergency room and
the weights are unknown and they are estimated.

If you look at the distribution of weights for the Phase III study, you
see these spikes that correspond to weights that end with zeros and
weights that end with five. This is particularly concerning for patients
who don\'t have a weight. I\'ll show you these patients here.

There were approximately 20 percent of patients in the development
program had only an estimated weight. In fact, there were many more
weights that ended in zero. The important thing was to look at the
frequency of the most feared adverse event, intracranial hemorrhage, by
the last digit of the estimated weight.

Yes, this is an exploratory analysis but what we found was very
reassuring which is that irrespective of the weight that was estimated,
the incidence of intracranial hemorrhage appear to be about the same.
This was very reassuring.

Next I\'m going to talk about data pooling in risk assessment. Pooling
is actually a meta-analysis of individual patient data. In other words,
retrospectively combining patient level data from different clinical
studies to assess a safety outcome of interest.

Used appropriately pooled analyses can facilitate detection of
relatively rare events, enhance the power to detect a statistical
association, aid interpretation of chance findings in individual
studies, and provide more reliable estimates of the magnitude and
constancy of risk over time.

The characteristics of a valid pooled analysis, first, Phase I
pharmacokinetic and pharmacodynamic studies generally should be
excluded. The risk of safety outcome of interest is expressed in person
years or a time-to-event analysis is conducted.

This is because if you have different studies with different lengths of
exposure, it\'s not appropriate to combine them as though they are the
same, patient versus patient.

Pooled patient populations should be relatively homogeneous and studies
should use similar methods of adverse event assessment and
ascertainment.

A negative result from a pooled analysis does not prove absence of risk
so it may be very reassuring to find no association, but you have to
look at negative results in the context of potential limitations of the
analysis. You have to ensure that the previously mentioned principles
have been appropriately considered in the analysis.

What is the role of subgroup analysis in safety assessment? Subgroup
analyses are critical to the overall risk benefit assessment and should
be addressed in NDA/BLA submissions in detail. These are fairly standard
demographics, baseline disease status, concomitant illness and
concomitant therapy.

Again, it seems fairly self-evident. It is very well covered by the
guidelines. But, on the other hand, we don\'t always see adequate
assessment of subgroup analyses.

The analyses missing safety data is an interesting issue. There is a lot
that has been written on analyses of efficacy in terms of how to deal
with missing data. For missing safety data it\'s not so clear and we
would be very interested in your comments on ways the issue affects risk
assessment. We would be interested in unique methods that might be used
to address this challenge that missing data presents.

My last two slides deal with data presentation. And the bullets on these
slides also seem self-evident but, in fact, frequently what we expected
is not what we get. We are very interested in the relationship of
exposure times to the development of the adverse event.

Summary of adverse event rates using the range of more restrictive to
less restrictive definitions, for example, myocardial infarction versus
myocardial ischemia.

A summary of the distribution of important demographic variables across
the pooled data. Where complete case report forms are called for
hospital reports, autopsy reports, biopsy reports, and radiologic
reports should also be included where applicable.

And assuring that narrative summaries include important supplementary
data, specifically pertinent lab data, ECG data, biopsy data as
previously articulated in guidance.

MS. HENDERSON: Thank you very much, Dr. Unger. Our public comments are
going to be presented by Dr. Christy Chaung-Stein today and she will be
followed by Dr. Sidney Kahn. Dr. Chaung-Stein will go first.

DR. CHAUNG-STEIN: Good afternoon. I am Christy Chaung-Stein of the
Pharmaceia Corporation. I am speaking today on behalf of the
Pharmaceutical Research and Manufacturers of America.

In the next few minutes I will share with you some points to consider in
the areas of data collection and analysis. My comments are a very high
level and many of them have been previously covered. I\'m simply here to
reinforce many of the concepts already covered.

Of course, to start with, PhRMA is delighted to have this opportunity to
provide input and we continue to look forward to our collaboration with
the agency to develop and apply innovative and efficient approaches to
collect and analyze safety data during the premarketing phase of drug
development. We consider this as essential to our ability to evaluate
benefit risk balance of a pharmaceutical product or a product candidate.

In a typical clinical trial situation, we make a rational decision about
the sample size required to address prespecified research objectives. We
believe this philosophy should apply to the sample size required of the
premarketing safety database as well.

We know that it is not the intention or the intent of the premarketing
safety database to capture all rare events, especially the rare ones.
Therefore, we need to be very clear on what type of events we want this
database to have the reasonable chance to observe.

Are we talking about one in 100 or one in 1,000? Only when we can lay
out our research goal can we make a rational decision on the sample size
required.

On the next slide I provide some numbers here. I guarantee this is the
only slide with any numbers on. On this slide I have various event rates
and I have sample size for the second column.

In the third column I have the probability of observing at least one
occurrence of the event. For the fourth column I have the probability
that we will observe at least two occurrences of an event. Sometimes
it\'s not clear to us what qualify a particular event, whether it\'s one
occurrence or two occurrences. That is why I include those two columns
there.

Here I do not take into consideration about exposure so it\'s a very
simple-minded binomial distribution calculation. It\'s a very high-level
kind of a bulk figure here. If we look at the event rate, one in 1,000,
which would be in this particular case \-- does it show up on the
screen?

Okay. If we increase the sample size by 100 percent, we double the
sample size. We increase the chance of observing one event by about less
than 20 percent. The corresponding chance of observing two occurrences
increase as well. The increased chance here diminished as we move to
events with lower event rate.

The calculation can go on with even lower event rate here. One can find
out very quickly that again in probability of observing the occurrence
of event is not proportional to the increase in the sample size.

This brings up the question this morning in certain situations it might
not be \-- the premarketing phase might not be the best place to observe
the adverse events because we might need to observe thousands and
thousands of patients in order to observe one occurrence of adverse
event.

Obviously the decision is dependent on a particular case we are dealing
with. Therefore, there is a need to address a particular goal of a
particular situation what is our objectives of this premarketing safety
database. Then we can make the decision accordingly.

The shadow of multiplicity, and we have heard a lot about different type
of analysis. We all know that when we do multiple analysis, the multiple
analysis will increase the chance of false/positive if we rely on
statistical methods that do not adjust for multiplicity.

Then the question is do we, or should we routinely adjust for
multiplicity. We don\'t think we should do so unless a particular study
is specifically designed from a safety perspective where we are testing
hypothesis on prespecified hypothesis concerning safety endpoint.

Here we like to emphasize that safety analysis are very much exploratory
in nature. The results of such analysis should be interpret in totality
and should be interpreted as such.

Another side of the coin of doing multiple comparison, the slicing and
dicing of the data, is the lack of power as Dr. Simon mentioned earlier.
We need to be realistic as to what the dicing and slicing can tell us.
Simply because we don\'t find anything doesn\'t necessarily mean there
is nothing there. The power is a very important concept to keep in mind.

We also talk about a lumping of terms to look at symptoms. We think
it\'s very important and it\'s necessary to approach certain
symptom-related issues. Our plea here is to make some sort of a uniform
approach which can be applied to multiple drugs.

And also to have a method of study in place to facilitate the
collaboration and dialogue with the agency. We do realize that this
collaboration needs to balance between the need for early planning and
the need for a data-based decision. We hope this collaboration does not
lead to any delay in making that official product to the patient
population.

We talked about pooling and sampling strategy, or the concept papers
talked about it. We agree that pooling needs to be based on some
scientific decisions and pooling is necessary to increase our ability to
estimate event rates. We also need to realize that when we pool data,
the previously observed event rate can go up or down because of the
sampling fluctuation.

Simply because previous observed event rates going down doesn\'t
necessarily mean that we are losing sensitivity, or the pooling is
inappropriate as long as we follow good pooling strategy agreed upon
previously.

Missing data is an issue. We all realize that. There are ongoing
research in this area. The missing data is especially troublesome in the
safety area because the drug exposure, the duration typically has a
positive impact on the occurrence of events.

There are various approaches that various companies have tried. For
example, for quantitative endpoints we can look at the responses at the
time of dropout for those individuals who drop out, and then compare
that to those people who moved beyond a particular point to see whether
there is any pattern among those people who drop out compared to folks
who continued.

Also some sort of imputation probably would be relevant in this
particular arena. This is one area we hope we can continue dialogue and
we can embark some collaborative research with the agency.

There has been discussion of problems earlier about the fact that the
accumulation of safety knowledge is a continuous activity all the way
from the discovery through development to post-marketing phases. We need
to be cognitive of the needs to link all the safety data together.

We feel that a decision tree type of strategy starting with the
preclinical evaluation throughout the product\'s life cycle would be
very helpful to help us make evidence-based decisions.

Basically what this says is if we observe certain signals in
preclinical, what will trigger \-- what type of activities during
clinical development or what type of signals observed during clinical
phase will trigger which type of post-marketing activities. If we can
have the tree type of decision strategy, that will be extremely helpful
to the pharmaceutical industry.

So, in summary, we would like to applaud the agencies who are pulling
together the concept paper and also holding this consultation workshop.
In turning the concept paper into a guidance document, we very much like
to encourage the agency to observe the spirit reflected in the following
bullets.

The objectives should be specific to a particular situation. They need
to be prioritized. Their terminologies and definitions used should be
clear. The proposed approaches should be, or need to be applied
uniformly across drugs.

Most important, we think decisions about a particular pharmaceutical
product, or product candidate, should be based on benefit risk
consideration and it needs to be transparent in nature.

Thank you for the opportunity.

MS. HENDERSON: Thank you very much.

Our next speaker is Dr. Sidney Kahn. We were then scheduled to take a
break but I think because we are ahead of schedule, we will go ahead
with the final public planned presentation which is Dr. Paul Stolley
following this one.

Dr. Stolley, if you will be ready after this one, we\'ll go right to
that and then go for a break.

DR. KAHN: Thank you very much. Good afternoon. I appreciate the
opportunity to provide comments. I am Sidney Kahn. I\'m the President of
Pharmacovigilance and Risk Management, Inc., an independent organization
providing consultation and support to the pharmaceutical industry and
related companies.

Before I get into my general comments, I think there is an overarching
point that I think needs to be made which is often overlooked, and that
is that we are operating in a rather conflicted model of the delivery of
health care in this mode.

On the one hand, the public patients want more, better, risk free, and
cheaper treatments, and they want them now. On the other hand,
pharmaceutical companies in our environment are profit-making
organizations whose priorities are often driven as much by Wall Street
as by Rockville. There is a fundamental conflict in the paradigm between
product development and marketing which the tension abituates
unresolved. I don\'t have the answer to that question today. Maybe
tomorrow.

I also want to point out that a lot of my comments have already been
covered and I will tend to blast over those that other speakers have
made. Finally, I would also like to point out that most of what is here
you have seen before in various documents published by the agency,
presented by the agency in presentations in various fora and published
widely and talked about for the last several years.

I don\'t believe you are going to see anything new and wonderful but I
will still give you what I hope is some additional thoughts that have
not yet been considered.

Obviously, I would like to commend the FDA for its forward-thinking
approach in this area. Particularly for taking the time and activity to
involve stakeholders in the input of these concept papers prior to
formalizing in a regular fora.

There are a few caveats we need to look at. As I mentioned earlier in my
comments from the floor, the risk of medicinal products in the current
system of drug development and marketing is actually quite small.

The figures that have been given recently are just over 1 percent of the
approved products in the last 15 or so years have been withdrawn for
safety concerns. I see a potential pitfall here of a disproportionate
reaction of the agency to what is a very small but highly visible risk.
I think it has to be not a form to the disproportionate reaction to a
relatively not particularly significant area of risk in many areas.

The other pitfall I see is that the adoption of methods that have not
been tested and whose effectiveness is uncertain based on theoretical
considerations, especially if those are embodied in forms of regulation
or guidance is a very dangerous part because it could easily divert
valuable and needed resources from more useful activities that could not
be explored if they were elsewhere.

Of course, there is the problem of a misperception of risk in many cases
that is sometimes fueled by media and other individuals which does
necessarily help the overall process.

I think that the overall goal is well reflected in tomorrow\'s guidance
documents which is buried in the middle there which I have reframed
slightly here. \"Since the ultimate goal is to ensure that efforts of
cost are expended on effective processes that achieve a positive
benefit.\"

I think that should be the overriding goal of this entire initiative. I
would also like to point out that the word achieved should probably be
replaced by maximized or optimized.

I would like to suggest that there are a couple of areas that the
guidance either doesn\'t cover \-- the concept paper. My apologies \--
doesn\'t cover or, perhaps, should emphasize more strongly.

The first one is date of quality. I\'ll talk more about that in due
course. The agency is looking at this and other documents such as the
proposed rule on safety reporting. I think that could be embodied in
approving clinical study risk management based on better risk
assessment.

There are definitions of what constitutes a so-called quality case
currently defined which could possibly be included. In addition, the
data safety monitoring reports as mentioned earlier by Dr. Jones should
also be involved here because they have a role to play in this area as
well.

Trial size. I think we have heard many comments earlier over the day
saying that generalization of this is not possible because the endpoint
is unknown. We all understand that it is possible to do quantitative
analysis of prespecified endpoints as we do for efficacy if we know what
we\'re looking for.

If we don\'t know what we\'re looking for, it\'s very, very difficult
indeed to come up with any kind of quantitative analysis. We\'ve heard
several comments to that effect earlier.

Again, the trial size will be highly dependent also on patient
population, how the trials are designed, and what kind of comparators
are available. The use of differential safety assessment against active
comparators has to be particularly problematic and needs to be addressed
in more detail later on.

So the recommendation that I would make is that for the time being at
least until we can develop better and more robust models of safety
assessment is that we should continue doing business as usual from the
efficacy side, power our studies to detect the appropriate efficacy
endpoints, and continue collecting safety data on an ad hoc basis.

But pay great attention to individual important adverse events, as Dr.
Temple, I believe, mentioned earlier, that really are important because
those are the ones that are going to give you the most bang for the
buck, so to speak, rather than the common and often unimportant assigned
symptoms that are collected in clinical trials.

In the chronic use trials I think it is reasonable to state that in our
experience in the last decade or so most adverse events of real concern
are idiosyncratic and rare using the definition of rare as between 1 and
100 and 1 and 1,000 or other index.

Any common serious adverse events will be show stoppers in the
development process and will probably lead to termination of the
development program at some point well before it gets to a filing
submitted to the agency.

And for low-frequency adverse events by contrast, even if you know they
may occur with that particular product or type of product, it is
unlikely to be detected in any use as we have heard several times before
and the rule of three has been used a couple of times.

All of these limitations of clinical trial numbers apply even more so
when you start talking about slicing and dicing the data as we just
heard when it was well and eloquently explained just now. I\'m not even
going to go into that except to say that if you do have some effects
that are markedly different from the general population, they are likely
to be detected just by normal scrutiny of the collection of data in the
current development process.

Therefore, we should continue to do our studies as currently done for
efficacy and exploratory studies in appropriate patient subgroups such
as the elderly and patients with renal impairment.

One of the questions in the concept paper is how to detect the
unanticipated. How can you anticipate the unanticipated. That is
frustrating, too. In the drug interaction field I can think of several
ways to do this and basically this has been done already but it could
possibly be a little more formalized and perhaps evaluated using data
that the agency already has in its files and archives.

If you know what drug usage patterns are of the potentially directed
products are in the population, and you know the pharmacokinetics and
the pharmacodynamics of those products, you can compare those with your
products in the investigation.

You can do formal interaction studies of the most likely and/or the most
risky combinations. Later on when you get into Phase III studies it may
be reasonable to allow concomitant treatment for some of these products
that are possibly less of immediate concern, or perhaps the same
products, so that you can then do an incidence comparison of the adverse
events of those patients treated with concomitant medications against a
supposedly earlier Phase III clean population for hypothesis generation.

This cannot be used for hypothesis testing. You do not have the powers.
Of course, the transcript used for subsequent studies and monitoring,
for example, in Phase IV studies or other posted group of commitments.

I want to spend a few minutes going over biomarkers. My background is as
a clinical pathologist. I work in the laboratory. I understand the
problems with laboratory analysis extremely well.

Many of the biomarkers that are currently evaluated in clinical trials
are bedeviled by large numbers of false/positive results. Any mild
supposed abnormalities often refer to nothing more than the biological
analytical variability in the population and in the methodology in the
laboratories.

It is also true that the mechanisms that result in common mild
abnormalities like myotransparase elevation two or three times the upper
reference limit are not necessarily the same mechanisms that lead to
severe and idiosyncratic reactions. The transaminase, anything up to 2
percent of control populations, is clearly not necessarily the same
mechanism as that which results in acute hepatic necrosis in the
occasional individual.

Therefore, there are some suggestions that can be made to improve the
utility of biomarkers in clinical trials. The first one is not to rely
only on absolute values, evaluations above three times the upper
reference limit.

It\'s critical to consider what the baseline of the individual was and
to look at magnitude of change from baseline because that may give you a
far better indication of what is going on than just looking at absolute
value, especially if the baseline value is elevated to begin with.

FDA here is uniquely placed to do a useful activity. FDA could develop
criteria for defining the optimal sensitivity and specificity of
biomarkers in its archives if it has hundreds of NDAs and BLAs currently
proprietary that could be used to establish the average different
degrees of abnormality when particular abnormalities occur in both
treated and controlled populations.

Those results could be correlated with subsequent outcomes that are
documented both for clinical trials and subsequent marketed use. I
strongly urge pharmaceutical companies to allow their proprietary data
to be used for this purpose in support of the public health.

Now, I would like to talk about terminology. Terminology is essential. I
believe this particular proposal for optimizing descriptions to signal
protection would be a great advance in the way in which we currently do
business subject to some minor refinements.

Currently data industry standards for clinical trials essentially result
in rigid adherence by the sponsor to the reporters for qualification to
the database by whatever terminology. This leads to a number of
difficulties quite frequently.

Investigators often misclassify events. They call jaundice abnormal
LETs. There was a reported case of acute liver failure in a patient who
has no coagulopathy or jaundice. This is particularly problematic when
interest occurs in a specialty when you talk of a system which falls
outside the investigator\'s area of expertise.

By the same token, even when there is similar expertise, clinical
classifications are often inconsistent because a similar, or even
identical constellations, symptoms and lab findings can be reported by
different reporters as hepatitis, abnormal liver function, elevated
blood transparase and jaundice, and any other set of terms that the
reporter chooses to use.

Currently we are circumscribed in our ability to modify those terms by
the standards of data entry. Therefore, I think it is essential that FDA
work with the sponsors to develop harmonized term definitions which will
then apply consistently when the same observations are found.

It is obviously also important that that consistency be extended across
all sponsors and all FDA offices and that cannot be done on a
project-by-project or symptom-by-symptom basis. If possible, it would be
ideal to have international agreement to this so that the same dossier
and the same approaches can be used for submissions which is what
pharmaceutical companies mostly do.

Lastly, there are a small number of definitions that have been developed
under the auspices of CIOMS and those could be built upon and extended.
There are also some initiatives that are being conducted by the MedDRA
maintenance organization and CIOMS working group on the definition of
the selection of terms for delineating specific syndromes because MedDRA
in itself doesn\'t always allow you to do that.

Finally, there is a working document in development at this moment by
the CIOMS-VI working group on clinical trial data management that
supports a lot of what FDA is proposing in this concept paper.

First is that events of critical importance should be consistently
defined using standard criteria where they exist such as those for acute
liver failure or where they do not exist in consultation with
appropriate experts in that discipline.

Those definitions and the requirements for use of particular terms
should be defined and adequately protocols and in the safety plan for
the product which has not been addressed as part of this but clearly
forms an essential part of any reasonable part in this assessment plan.

When it is necessary and possible for a response of entry data into a
database to overrule the investigator\'s diagnosis of the reporting term
based on what I mentioned earlier about misclassification of the
clinical terms for analysis purposes, not necessarily for reporting.

As long as appropriate order to trail is maintained showing what was
changed and why it was changed, I think that should be acceptable. At
the end of the day, the analyses would be based on the defined terms
assigned by the sponsor rather than on the debated terms assigned by the
investigator if they are wrong. Granted there are a lot more ways to
deal with that but I don\'t have time to go into those right now.

Finally, even though this is contrary to the points to consider in the
document, if a classical constellation of signs and symptoms is reported
without a diagnosis, I believe it should be possible to assign the
diagnosis for analysis purposes.

This obviously could be contentious but, again, subject to the
availability of an adequate order trial, this should not be a serious
problem because one could always debate that at the time for approval on
the labeling.

Thank you very much.

MS. HENDERSON: Thank you, Dr. Khan. We will go right away to Dr. Paul
Stolley who is representing Public Citizen Health Research Group.
Following Dr. Stolley\'s presentation we will take a short break and
return then for discussion.

DR. STOLLEY: My name is Paul Stolley and I\'m a medical epidemiologist
with a special interest in the epidemiology of adverse drug reactions. I
currently work part-time at Public Citizens Health Research Group, a
consumer advocacy organization founded by Ralph Nader in 1971.

My comments today deal exclusively with the concept paper entitled Risk
Assessment of Observational Data, Good Pharmacovigilance Practices and
Pharmacoepidemiologic Assessment distributed by the FDA.

While this paper was to be discussed on the third day, I got scheduled
somehow on the first day so you can think of this as an exciting preview
of day 3 like a movie trailer.

Drugs are approved and reach the market after testing on a relatively
small number of patients and under rigorous scrutiny. Once on the market
the approved drugs may be given to patients with multiple disorders, or
even for unapproved indications.

Unexpected adverse reactions may, therefore, occur or expected reactions
may occur at an increased rate. This argues for an effective
post-marketing surveillance system. The FDA has often asked the
sponsoring drug companies to perform Phase IV marketing studies of
worrisome drugs, but the studies are frequently never completed, not
published, or difficult to interpret due to design problems.

The recent Health and Human Services Office of the Inspector General\'s
Report entitled FDA\'s Drug Review Process for New Drug Applications, A
Management Review, notes that medical officers are \"often uncertain
about what types of post-marketing commitments to require of sponsors.\"

Consequently, we suggest that the FDA\'s Office of Pharmacoepidemiology
and Statistical Science also be involved in the design of Phase IV
studies and other post-marketing programs. The Health Research Group has
documented the failure of drug companies to finish Phase IV studies.

A study we sent to the then commissioner Jane Henney on April 13, 2000,
which is available on our website which will be on one of the papers. I
have 50 copies of this paper to be distributed.

Five to 10 years after making a Phase IV commitment, only 13 percent of
the commitments were completed. What does the FDA plan to do to correct
this poor performance record by industry?

Recently the post-marketing study for the asthma drug Serevent, the
SMART study, was stopped by Glaxo because of excess deaths among the
Serevent users. But the drug was left on the market and this important
study is not available for inspection by the scientific community.

This secrecy may serve the sponsor\'s needs but it leaves physicians,
scientists, and patients in the dark. Another company, or even Glaxo
itself, may now seek to market a drug with the same problems as Serevent
potentially costing more lives.

Post-marketing studies that are made public \-- excuse me.
Post-marketing studies that are not made public, for all intents and
purposes, are post-marketing studies that never happened. Conversely,
were the SMART study to show some benefit for Serevent, we suspect there
would have been a great rush to publish it.

We are also concerned that the promise of post-marketing studies may be
used to approve new drugs inappropriately or delay the removal or
relabeling of dangerous drugs.

In a survey of the FDA\'s reviewing medical officers that the Health
Research Group conducted in 1998, many officers felt pressured to
approve drugs that they might not have approved. The company\'s promises
of Phase IV studies, many of which will presumably never be completed,
tipped the balance for approval.

My next comment concerns the criteria for assessing causality in
post-marketing studies. This section is brief and mentions criteria
without assigning any priority or weight to each criterion.

I think that is probably a wise policy as there is little data to
support such a weighing or score. But the paper should be clearer with
respect to the notion that these are ideal conditions for assessing
causality.

In particular we would not want case reports that involve the use of
drugs in addition to the suspect drug to be summarily dismissed. The
paper does not mention the many epidemiological and statistical
techniques available to deal with confounding by the presence of other
medical conditions or the use of other drugs.

The concept paper is also unnecessarily dismissive of reported rates of
adverse drug reactions. While we agree that wherever possible reported
numbers of cases should be adjusted for prescribing rates, prescribing
data, especially for narrow demographic groups, but these are often
lacking.

Sometimes the signal of reported numbers of cases is so strong that it
alone can suggest a need for FDA action. The current language applies
too strong a standard and may lead to lack of FDA action when the
protection of the public requires such action.

Finally, analysis by so-called race, which is line 308 of the document,
is listed as important in assessing safety signals. We advocate extreme
caution in invoking \"racial\" explanations for observed adverse
reactions. This is because the so-called races have far more genetic
similarities than they have differences.

If analysis by race is used, the racial categories must be listed,
clearly defined, and justified, and the rationale for the use of this
variable detailed. The analysis of the efficacy of a recent AIDS vaccine
using subanalyses by race is an example of misuse of this category, I
believe.

Thank you for your attention.

MS. HENDERSON: Thank you, Dr. Stolley. Okay. It is 2:35. This is about
the only time really that is sensible to take a break even though we
would probably prefer to take one later. Unless everybody would like to
go straight through.

Perhaps we could see a show of hands for those who would like to just go
straight through, have the discussion, and finish the day. Ah. And those
who would prefer to take a break. Ah. Finish the day it shall be. We
will forego a break and we\'ll move right along to the discussion by our
panel. Any questions or discussion either from our internal panelists or
our external panel, or anyone from the audience.

Dr. Simon.

DR. SIMON: I serve on the FDA Drug Advisory Committee, for several
years, and I have been a guest member of several other committees. A lot
of issues of adverse events and puzzling issues have come up. I remember
one issue that did involve temporal associations.

This was an advisory committee about a fat substitute and concern that
that fat substitute might be causing breast cancer. It was interesting
for a number of reasons. One, sort of the concern was based on a P value
calculation. It was sort of borderline significance depending upon
exactly which cases you included or excluded in the analysis.

This was sort of \-- I was on the second meeting of this committee that
had reconvened to sort of address this issue. One of the things that was
sort of striking about the issue was many of these cases of breast
cancer sort of were diagnosed within the first month after the women
went on this study.

From everything that is known about the development of breast cancer,
it\'s completely implausible that the study drug could have caused
breast cancer within that period of time. Most theories of
carcinogenesis believe that it takes years. There is a multi-year
process for the development. But, nevertheless, that did not seem to be
viewed very strongly as sort of evidence that it was against a causative
association.

I think in the document here you have a couple of examples of temporal
associations which would lead you to believe that there could be a
causative association, but I think there are probably also examples
where it would lead you to believe the opposite and you might want to
include one of those.

The other thing that was uncommon between that example and several other
examples that I have been exposed to was essentially the misuse of
statistical significance in assessing adverse events.

The use of statistical significance was even though we say we realize
these are not predefined hypotheses, nevertheless, the calculation of
the P values in finding out that they were maybe in the range of .05
seems to me to be a major factor in people being concerned about adverse
events. There are a number of reasons why for many adverse event issues
statistical significance is not appropriate.

One is, as has been mentioned, the power in general is very poor. A
second is in many \-- well, there may be many tests being done. You may
be looking at many kinds of adverse events so we all know the chance of
finding one significant at the 05 level if you do many tests is very
high.

The third, though, is even if you are doing one test, usually these are
unexpected, unanticipated findings. When you are analyzing unanticipated
events, statistical significance can be very misleading.

I guess my recommendation would be that this somehow be discussed. The
issue of multiplicity be discussed. The issue of lack of power. The
issue of prior probability being low. All of these things really should
be taken into account and probably the use of statistical significance
tests for adverse events, my recommendation would be that the document
recommend against that.

I think the document is \-- well, I guess I\'ll just mention a couple of
other things. One is the issue of pooling. I think there are many
reasons why you might expect that if you have some signal from one study
that leads you then to do some additional study that involved pooling,
the results of multiple studies together, for all the reasons I\'ve
mentioned there\'s going to be a lot of false positive signals.

When you do a follow-up study in which you may be pooling the results of
several cities together, there\'s a lot of reasons to believe why those
false positives then will disappear from the meta-analysis. The document
as it reads now is sort of one-sided.

If the signal persists, then you can believe that it sort of implies.
But if the signal just disappears or gets smaller, then basically you
need to suspect that your meta-analysis is fallacious. I think that is
probably not appropriate.

Finally, the issue of missing data. There\'s lots of sort of ad hoc
rules that people use for missing data. There is really no good method
for handling missing data but my advice would be that the best methods
are methods of multiple imputation where you essentially assume \-- you
try to build a model in which you can predict the outcome but based on
lots of covariats and you build that model based on the complete data
and you see how predictive that model is.

If that model is highly predictive, then you sort of do a multiple
imputation using that model. I think it is actually similar whether
you\'re talking about adverse events or you are talking about efficacy.
There is no great way of handling missing data but to me that kind of a
method is better than the ad hoc rules. Thank you.

MS. HENDERSON: Thank you, Dr. Simon.

Dr. O\'Neil.

DR. O\'NEIL: I have a couple of questions to Christy Chaung-Stein and
Dr. Kahn with regard to a couple of things. Dr. Kahn sort of made the
pitch that we ought to for a few studies do as we are now doing and
don\'t introduce any new ideas.

The presentation on missing data is really about not missing data but
it\'s about when people get withdrawn from a study you no longer can
observe something. There is a culture for safety which there isn\'t for
efficacy of not measuring anything often after individuals have left the
trial.

I guess my question to both of you is for chronic studies the
missingness is about characterizing exposure and outcomes and whether we
as a group in 2003 are doing a decent job on the database of even 3,000.

Because one of the things that Dr. Unger talked about is just in terms
of just characterizing exposure is not really taking time into account
because you have a delayed event but you have no choice but to take time
into account because you have no fixed denominator.

People are not exposed so you don\'t know what to call the denominator.
You\'ve got two things going on. You are interested in the denominator
because it\'s changing. And you are also interested in the denominator
because the drug may be related to the size of the denominator.

The question I have for both of you is do you think that there is
something with regard to lost to follow-up what happens when individuals
withdraw from a study for cause specific reasons, mainly safety.

Following up on Bob Temple\'s comment, if that is where the enrichment
is, what is the new suggestion for what do you do when people withdraw
from a study. Do you essentially follow them to the completion of a
one-year trial even though they pulled out of exposure. Not of the
treatment or the trial, but pulled out of exposure after three and a
half months.

I would be interested in sort of an industry view and a consultant view
on this particular topic because it is relevant to sort of the design
data generation as well as how you even estimate something that you
can\'t observe.

DR. CHAUNG-STEIN: We are playing ping-pong here. I believe currently the
practice is if a particular patient dropped out due to adverse events,
you know, it was reported as such.

When we look at the frequency of individuals with adverse events and
that individual got reported as experiencing adverse events, that
particular dropout was also noted very clearly in the patient
disposition tables.

The question is when we get to specific term summary, if a particular
patient had not experienced a particular event but because that he or
she dropped out gives you another event how to re-record the other event
not experienced.

I don\'t think I have a good answer to that question. One of the
imputation strategies or possibilities might be a way to kind of predict
whether that individual would have experienced that event should that
individual stay on one year or two years. Then, again, that\'s a model
based strategy. It has its own flaws or advantages or disadvantages.

DR. O\'NEIL: That is what Richard is suggesting.

DR. CHAUNG-STEIN: Yes.

DR. KAHN: I\'m not sure I can add all that much but certainly in my
industry experience most protocols require any patient who drops out
because of an adverse event to be followed for a period of time until
the adverse event has at least resolved. At a minimum one should have
information concerning the outcome of the event if it occurs on therapy.

If a patient drops out for other reasons, then that comes back to my
catch 22 question, how do you anticipate the unanticipated or detect the
undetectable. I don\'t think there is any answer to that one.

Obviously it would be ideal if you had concerns for delay toxicity of
some kind to do some kind of post-therapy follow-up for a defined
period. I am familiar with studies that have done that but one has to
draw the line somewhere and you can always say, well, the line was drawn
in the wrong place. This is a very difficult question.

MS. HENDERSON: We\'ll go with you first, Dr. Gormley. Then we have a
question from the audience.

DR. GORMLEY: I wasn\'t particularly invited into this dialogue but let
me put this point forward. I guess as the clinician we are trying to
evaluate the safety database. I might be concerned that the approach of
continuing to follow a patient because they dropped out with an adverse
event might bias against seeing the adverse event in a population.

If it is a dose depending exposure depending when they develop a bump in
their LFTs they drop out. Then I continue to follow them for the next
two years during the trial and they have no LFT bumps and they are not
on the drug, it would be quite inappropriate for me to conclude there
wasn\'t much of an LFT problem or really it was not much of an adverse
event problem because they didn\'t get the exposure.

In the case where there is a delayed toxicity you are looking for, that
makes good sense but I wouldn\'t want us to inappropriately use it. I
wasn\'t sure you were considering that point or whether I had missed
that.

DR. O\'NEIL: I think there is probably a sunset for how long you would
follow someone but I believe Dr. Kahn was also making a point about
biomarkers. Essentially when you withdraw folks on the basis of a
biomarker liver toxicity being a classic example.

What is the reasonable period of time to follow somebody and see whether
that doesn\'t get worse and actually the outcome actually then occurs.
Some hospitalization three weeks subsequent or something like that. This
is more relevant for some types of trials than others.

I have been struck by the large withdrawal rates from symptomatic
clinical trials of anything dealing with chronic exposure drugs even for
a year. The ability to be able to follow individuals for a year is
critical to the assessment of long-term toxicity, particularly if you
want to think about it in the real world where is it due to switching,
somebody just switches to another product, or is there somebody about it
that\'s going on.

There is this issue of the joint probability of being able to stay on
the drug long enough to experience both the benefit and the risk so in a
chronic use study it is very interesting to think about the interplay
between the denominator and the folks who are staying on the drug
because there is a benefit both and they haven\'t had the risk yet.

I\'m just curious about how we start to think about that particular
issue because it is different in the safety arena than the efficacy
arena.

We go through long discussions about following people and different
multiple imputation strategies for the outcome and last observation
carried forward, but we don\'t think about it at all in the safety area,
particularly in terms of where the time comes into play.

That is why I was just curious whether you had any new input because you
did make a plea for the acute studies and let\'s have business as usual.

MS. HENDERSON: Dr. Flockhart.

DR. FLOCKHART: If I could just jump in on that specific note. Of course,
there is a flip side to that, and that is that with the LFTs to continue
to bump up and down after the patient went off the drug, that would be
the opposite. That would argue against that having been specifically due
to the exposure.

But the issue of LFTs does raise the question, and this was brought up
earlier by Dr. Kahn, about the specificity of our biomarkers and how
well they actually reflect the andron toxicity. I must confess that I am
a pharmacogeneticist and a clinical pharmacologist.

I work in a world where I spent the end of last week up at the
proteomics workshop at NIH. I think some of that is actually relevant to
this discussion because if I could just make the rather crass comparison
with the airline crash.

An adverse drug reaction is absolutely not like an airline crash and
probably the most compelling argument for it not being is that we don\'t
have a black box and we do have missing data. Definitely lots of missing
data.

It might be data due to lack of further exposure. It might be data due
to lack of following people after exposure. It might be due to lack of
data recorded at the time because of all the biases around it in current
medical practice.

Really the worlds of genomics and proteomics offer the equivalent of a
way to go into a black box and that is by collecting a sample at the
time, and possibly more samples.

Even just practicing clinical pharmacology for the past 10 or 20 years
it\'s always interesting to know how really the actual drug level gets
noted. I am actually not an advocate of doing that a lot.

I do believe that the collection of appropriately collected serum and
DNA in this context holds tremendous promise for later on if only to
store it even not knowing necessarily what we are going to do with it.
The power of proteomics to dramatically improve the specificity and
sensitivity of our tests in the next few years is really something
remarkable. I think we would be blind. We would be putting our heads in
the sand to ignore that possibility.

MS. HENDERSON: Thank you, Dr. Flockhart. We have a comment from the
audience.

PARTICIPANT: Michael Perriman, Schering-Plough. I would like to ask the
committee and the FDA a rather different question. When designing a
development program one thinks about how many patients one wants to
expose to the drug.

In terms of safety, I think you need to think about what is the question
you\'re really asking.

Do you want to find out with confidence what the frequency of common
adverse events is, something occurring in 10 percent of the population?
Do you want to know if it\'s 10 percent or 12 percent? Or are you
interested in discovering whether infrequent events occur once or twice
in the population? Or do you want to look at subgroups?

Because depending on which question you really want to ask for a risk
management decision, you are going to power your studies in a totally
different way, the totality of your population. You want to look at
different durations of exposure.

Unless you answer that question up front, all the questions about
whether we follow individual adverse events become somewhat academic to
me.

MS. HENDERSON: Dr. Ellenberg.

DR. ELLENBERG: Just to respond, I think that different circumstances
would lead to different questions. I think that is what we are talking
about this morning, that in some cases what you might be interested in
was simply the frequency of certain common or expected adverse events.

I think in other perhaps more limited subset of circumstances you might
be interested in certain types of events that were less common that
might happen and you might want to be able to assess those.

In a different set of circumstances you might be interested in results
in a particular subgroup because you might have reason to believe that
while a product might be used more widely a particular subgroup might be
at greater risk.

I think there is no single answer to that question. I think it is a
case-by-case determination.

MS. HENDERSON: Dr. Gormley.

DR. GORMLEY: I think I would respond to that question by saying when we
go into a development program with a new drug we are interested in all
three of those all the time. How we approach it may be different but we
never know what we are going to get ourselves into and we need to be
able to answer all three of those questions.

I think this whole conference about risk management is about all three
of those points and they are going to require different approaches. Some
of those are on the continuum later and some of those are on the
continuum earlier.

But to say that there is a situation where we don\'t care about one
piece or the other. I think in today\'s world that just won\'t fly. We
just hope that we only don\'t run into some of those situations but
there is some burden on us to prove that.

MS. HENDERSON: Go ahead in the audience.

PARTICIPANT: I agree with that statement but I think the decisions about
today is what order are we doing those things and what confidence do we
have to make that calculation.

For example, if I want to look at the elderly as a subgroup, I have much
less power to understand event rates in that population versus the total
population because I\'m going to have fewer elderly patients in my
studies.

Therefore, if I do want to know about the elderly, I\'m going to have to
do a different study, a different set of studies to get to that answer.
I think what I would like to hear better is how much confidence do I
have to have for those subsets at the time of approval versus in the
early post-approval period versus later on.

MS. HENDERSON: Go ahead Dr. Gormley first and then Dr. Simon.

DR. GORMLEY: Just to follow up quickly so that I\'m not misunderstood.
My earlier comments in the morning were all around the timing of that as
well so I made the point I didn\'t think large simple safety studies
should be done at Phase II and Phase III. You need to know a lot more
about the drug before you enter into those.

I made the comment that I thought those were Phase IV kinds of things. I
think that is directly what the speaker was addressing. It\'s the timing
in which you evaluate them that is most important. That will be
different for different drugs. To expect us all to evaluate them all up
front before approval may not be appropriate but to consider them is
appropriate.

DR. SIMON: I think I\'m probably not going to say something too
different from that but maybe in different words. What I hear the
question is appropriate, saying if you really want a reliable answer to
something, you have to say what the question is.

You cannot get a \-- it is very hard to get a reliable answer to an
unasked question. It is very hard to get reliable answers to all
possible unasked questions, which is what has been laid out in this
document, multiple subsets, multiple kinds of adverse events.

It is really not possible to do that so that sort of gets into what I
was saying about P values. I think if you are going to do a broad-brush
sort of thing like that, I think you have to accept that what you wind
up with is not precise answers and no answers based on very much power.

You are going to have a lot of false positive signals in it. Hopefully
everybody involved should interpret the data in light of that, or
otherwise doing all these subset analyses and everything is just going
to delay everything for a bunch of false positives.

MS. HENDERSON: Thank you, Dr. Simon.

Dr. Temple and then we have two comments from the audience.

DR. TEMPLE: I think we are making something that is not that hard harder
than it looks. We want to know all three of the things that were
requested. We usually have tons of data with respect to the more common
adverse effects. Those are best interpreted in the randomized trials
because you really need a control group for these things to make much
sense out of it.

And when you pool the data you have a fair amount of data on at least
certain demographic subsets. You almost always have enough data to say
something intelligent about women, people over 65, although they are not
very often different from anybody else. Sometimes you can do things with
racial subsets with all the concerns one has about how to define that.

You have very little power to detect the things that are really scary,
the things that make drugs leave the market because if they were common,
the drug would have been withdrawn in Phase II or III.

What you are doing there is, as everyone has said, a tremendous problem.
You are looking for a needle in a haystack and you don\'t know there\'s
a needle. Well, the big question is how do you find out if there is even
something there. Our short answer is you look at all the people who died
and see if anything looks funny about that.

You look at all the people who leave the trial prematurely because if
anything real bad happened, they probably left. We don\'t have a better
answer to that.

You rarely have enough of those events to make really serious
comparisons about the treatment and control group but events like \-- I
mean, think of the things that get drugs off the market they're;
agranulocytosis, liver injury, and Torsades de Pointes. You don\'t need
a whole lot of cases to interpret those things because they very
infrequently happen in the absence of the provocation.

So we need all of those things and it is only when you are particularly
worried about old people that you would set out to design a study with
adequate sample size to specifically look at the elderly. That is pretty
much what our rules and guidance all says. It says you should do an
analysis of demographic subsets to see if anything turns up.

It actually doesn\'t say anything about how much assurance you have to
have as to what you would find because everybody recognizes there is no
very good way to anticipate this if you don\'t know what you\'re looking
for. That is sort of inherent in it.

I just wanted to mention a couple of things really in response to Dr.
Kahn. We actually are working with outside groups to have some
definitions of hepatic injury so people use a common language. That
would apply mostly to situations that are post-marketing where you
don\'t usually have all the information you want.

Premarketing you pretty much have all the lab tests and everything and
I\'m not sure it makes any difference what somebody calls it, you get to
look.

The other thing we are doing, although it\'s slowly because many of the
drugs were a long time ago, we are looking at the ability of various
kinds of signals of liver injury.

For example, transaminase elevations of varying degrees to predict
response so we are looking at drugs that were hepatotoxic and drugs
that, as far as we know, are not hepatotoxic and trying to do some
specificity and sensitivity analyses of candidate markers.

Well, obviously one candidate is an increased frequency of three-fold
elevation of transaminase. Another, my personal favorite, is that
accompanied by a few cases of bilirubin elevation which I\'m ready to
bet will be a very good predictor of troubles.

It has certainly predicted all the drugs we\'ve withdrawn recently and
things like that. We are working on that and strongly believe in it. It
doesn\'t seem out of the question. You could do similar things with CPK
and random myolysis and a few other things.

Of course, we would like to see more biomarkers. When David is all done
we will be able to do this with their genetic typing and there won\'t be
any problems anymore. Until we get there, we have to use crude methods
and do our best.

MS. HENDERSON: Thanks, Bob.

We are going to go next to the two speakers at the microphone.

PARTICIPANT: I just wanted to share with the panel an experience I had
and get some comment. It had to do with dropouts but from a little
different perspective. I had reviewed some data on a GI drug that had no
evidence of any cardiotoxicity. As a matter of fact, the drug was very
well \-- it seemed to be quite efficacious.

What was interesting was that the placebo dropout rate was greater than
the people getting the investigational drug. There was an adverse event
that came up, myocardial infarction, where the incidence was eight in
the investigational group and zero in the placebo group.

After reviewing all the people who had the cases to see if there was
anything that was unique about this, that was a wash. Nothing came out.
Going back trying to figure out why there was such an imbalance I did a
calculation of patient year exposure.

Because the placebo dropout rate was so much greater it was
approximately eight to one investigational drug exposure to that versus
placebo which picked up that \-- what we concluded was that the
imbalance and the incidence of this might very well be due to the fact
that it was background noise and people on the investigational drug were
in the trial longer and, therefore, more likely to report such an event.

To me it seemed like patient year exposure might be even a better
calculation and I was just wanting to hear the group\'s feedback on
that. Thank you.

DR. HENDERSON: Dr. O\'Neil.

DR. O\'NEIL: Not knowing the details, I would say if you control for
duration of exposure in the hazard rate sense and not a person time
sense you get the answer to that. It would not be the reason because it
would come out in the wash.

You would find out whether the event in the first early part \-- let\'s
say if you were to break it up into equal increments of one month for a
six-month study, you would find out by an analysis that looked at folks
that appropriately were exposed for different \-- the hazard rate
essentially where the action was coming from.

One of my concerns is looking at person time and then putting the
observed number of events over the total person time. That\'s an
inappropriate analysis except if you assume that the event rate is
constant throughout the entire study. That is an epidemiological
strategy but it assumes away half of the problem.

The problem that we want to know about right off the bat is whether the
event is time dependent in the first place. I could have it wrong but
that is just a quick response to what I thought was your explanation for
why the event was occurring in the placebo group.

PARTICIPANT: It was just really a comment that if you have an unequal
dropout rate in the two treatment groups, your exposure is impacted.

DR. O\'NEIL: I don\'t want to prolong this but, again, I stick with what
my answer is.

PARTICIPANT: Thank you.

DR. O\'NEIL: But the other problem with that is that the risk groups,
the covariants of the people who stay in and the people who stay out are
changing over time. That may be more indicative of what is going on
because the people stay in the trial because of something.

It becomes real hard real fast because it\'s just not the enumerator
that you are worried about. It is also the risk factors of the
individuals who are able to stay in early and stay in for longer periods
of time. That is something that we don\'t normally look at very well.
Even the factors that are associated with the dropout patterns and
whether they are comparable between treated and control.

In the situation you\'re talking about, if that is a placebo controlled
trial, you probably have different risk factors associated withdrawal
for lack of efficacy versus withdrawal for toxicity purposes. Those risk
factors are likely different between those groups.

PARTICIPANT: Thank you.

MS. HENDERSON: Another comment from the audience.

DR. HAAS: Joanna Haas, Genzyme Corporation. Thinking about safety issues
I think is very much dominated by the way we collect adverse event
information. That is really quite passive in which we collect, record,
and display in a way that is fairly raw. There have been some
suggestions for how we can work with that a bit more. I certainly favor
those personally.

Perhaps a step forward to move between this question of adverse events
information and safety questions would be to look at safety endpoints
and think of things more in terms of safety endpoints. And, to go back
to a point that Judy Jones made this morning, to start thinking about
safety issues from the clinical, or even perhaps entry into development
and think of what our potential concerns are.

What the data that we have about them and that data grows along through
our preclinical development through Phase I through the clinical
pharmacology phase. We have safety information as it goes.

It is diffused and it\'s put into columns, it is put into silos, and
it\'s not often integrated across the safety problems. Then we take the
early adverse event information and instead of parsing it really into
the safety problems, we keep it and display it in the way that we get
it.

I think that if we start generating safety hypotheses early, it allows
us to do a lot of things. It allows us to define a case and that gets
better as you go along. It allows us to have secondary safety endpoints.
And it gives us a lot more certainty about very specific questions.

I think perhaps in Dr. Simon\'s comments and a lot of the comments this
morning this question about how we move from information that is not
structured to information that answers a specific question is something
that we should define better.

MS. HENDERSON: Thank you, Dr. Haas.

Any other questions or comments? Yes.

PARTICIPANT: Yes. I would like to make an explanation to an earlier
comment which suggest that this concept paper is against active funding
from data pooling. When we were in the working group, although we are
very sensitive to the possibility of false positive, we made our
judgement based on our experiences.

At this time the major concerns we have is inappropriate data pooling
which means combine patient population which clearly clinically
heterogeneity. Also combine some study which already suggest some
ascertainment bias.

By combining those studies you will observe some diminished risk or
diminished association. At this time that was the main concern we had
based on our judgement or observation. That is why we put in the paper
this way.

We want to raise these concerns to everybody. When the submission come
in you should look at it. If you find diminished risk or either
association or risk itself, you should at least look at the population
and whether they are similar, whether there is a possibility of case
ascertainment.

It doesn\'t mean we are not sensitive to a false positive just because
we are more concerned at this time the possibility of inappropriate data
pooling. If someone else has any suggestion or any examples otherwise,
you know, false positive is a bigger concern than what we have, we would
very much like to hear that comment, please.

MS. HENDERSON: Dr. Kahn and then Dr. Simon.

DR. KAHN: Thank you. I\'ve had a few more minutes to think about Dr.
O\'Neil\'s question to me concerning why we shouldn\'t change the
current size of the clinical trial database. I think it really gets back
to the key question of how much is enough. Dr. Simon\'s earlier point
about what question are we trying to answer because that is what
determines the size of the clinical trial database.

Until such time as we can get a better handle on the questions we want
to answer, then we can tell you how much is enough and then we can
change the size of the database. But until that time without any
prespecified endpoints, it would be inappropriate to start talking about
changes to the database size.

Clearly if we have alternative data that suggest that we can detect
important adverse events more readily, more quickly with double or
triple the number of patients and the number of product withdrawals from
the market falls from 1.2 percent to .5 percent, that would be a valid
argument for increasing the trial size. I don\'t believe we have that
information at this point.

DR. O\'NEIL: Yeah. Just a comment. I wasn\'t suggesting that as a
justification for increasing sample size. I was saying with what we now
have in hand, same sample size, are we measuring what we are supposed to
measure. That was the issue.

MS. HENDERSON: Dr. Simon.

DR. SIMON: With regard to the pooling, I think the considerations you
raise are very appropriate. I think they are appropriate regardless of
how the result of the meta-analysis comes out. Maybe you are assuming
you wouldn\'t actually wind up seeing the meta-analysis unless it comes
out sort of diminishing the results.

I think the issue is that doing a meta-analysis one needs to address all
of the issues that you pointed out, as well as, I guess, the additional
issue to assure that it is not a selected set of studies, that it covers
the defined universe of studies and that that universe be defined
appropriately.

I guess the issue with regard to the sample size is maybe the thing that
came up this morning with regard to the 1,500 and whether that\'s
reasonable based on a rule of three without really getting more into
what are the objectives you are really trying to achieve.

If the objectives are really to make sure that you have maybe two cases
of something that occurs one in 500, to do that you probably don\'t need
a control group. If those two cases are going to be interpretable at
face value, then you don\'t need a control group.

If you do need a control group, then they probably don\'t occur at one
in 500 or they probably occur at a higher level. I guess the question
just raised is if one is going to talk about 1,500 as the safety
database or something like that, then maybe that should only happen
after there is some more focus on what one expects and doesn\'t expect
from them.

MS. HENDERSON: Comment from the audience.

PARTICIPANT: I want to touch on an area which I did not find in the
concept paper which is the use of formal tools and applications for risk
assessment. I was glad to hear that Dr. Chaung-Stein mentioned decision
tree analysis. We could mention several other tools and application
which can be used to help in risk assessment.

I think that it would be important to endorse this approach as well as
make it clear that any of those tools need to be properly validated for
use. And all the tools used for risk assessment as well. We would be
interested to hear any comments on this.

DR. AVIGAN: Again, as I am listening to this conversation, I would like
to hear more discussion about the imperative to go on multiple tracks.
That is unanticipated scenes which is a long-term goal because some of
these effects could be very rare and picked up at a very late time in
the life span of these studies.

But others could be anticipated because of the class of the drug. It
might be a safety concern that has already been identified with, let\'s
say, another member of the class where there is a particular public
health interest to see where this particular new player is in the
spectrum. There really is a matrix of questions which are parallel.

Again, I haven\'t heard \-- again, if you are talking about strategies
of sequence of studies as you go from the pre to the post-marketing
phases, I haven\'t heard a discussion about how these are really \-- you
know, how these are all connected together, or a convincing discussion
about why they might actually be mutually exclusive.

MS. HENDERSON: Comment from the audience.

MR. SCHIFFENBAUER: It wasn\'t specifically about that. My name is Joel
Schiffenbauer. I work for the FDA. If we are looking for rare events,
the question I have to the panel is why not look at individuals that are
at high risk and stress the situation. You may be able to do smaller
trials and identify rare events using that kind of approach. I just
wonder what people think of that.

MS. HENDERSON: Dr. Temple.

DR. TEMPLE: Yeah. I mean, in many cases for the things that worry us
most, we don\'t know who\'s at high risk. Nobody has ever defined a high
risk hepatotoxicity group and, as is being suggested here, putting
people with really, really long QT intervals into your trial probably
wouldn\'t press most people\'s stomach test.

The fact is for most of the things we look at we really don\'t know what
the predictors for risk are. If we were that smart, we would know a lot
more about the adverse events we\'re talking about.

I guess \-- I mean, you still can\'t go looking for a case of Torsades
that way. It wouldn\'t be right. Whether there are some ways to find
people \-- well, I mean, the fact is one question that comes up all the
time is what to do \-- supposed you have a drug that is metabolized by
P452D6 so you know that 8 percent of caucasian population is going to
have very elevated blood levels. Should you put them in the trial at
all.

A recent experience with strepteria for ADHD actually included them in
the trial, although a value for the \-- I don\'t remember whether it was
genetic or some other test but it was learned whether or not they were
poor metabolizers or not.

They were included in the trial which you might say stresses the system.
It stresses the system in the absence of an available test certainly.
And an analysis was made of whether they had more or fewer adverse
reactions than the rest of the population. It turned out for the most
part they had about the same as everybody else but for a few things.

They were a little bit higher but nothing terrible happened. That might
be an example of what you are talking about, whether to exclude
metabolic abnormalities or not. Now, it depends on what kind of drug it
is and how worried you are obviously. Otherwise, you wouldn\'t feel
responsible doing that.

MS. HENDERSON: Dr. Flockhart is dying to answer that.

DR. FLOCKHART: I can\'t remain silent any longer about this. It seems to
me that there will always be needles. There will always be completely
unanticipated events. My only approach to that is what I said a moment
ago. For God\'s sake, collect a blood sample and some DNA.

But in general that is not helpful most of the time because most of the
time there are actually a defined series of questions that you want to
answer. This is picking up Mark Avigan\'s point I think. We can go after
the things we know take drugs off the market. The agranulocytosis, QT,
and so on that Bob is referring to.

You can also use the known sequence of events that we spent the last
century generating. That is the second messenger pathways of all the
drugs. We have huge amounts of information about drugs now upon approval
about that.

That identifies logical things to go and test for not only in the kind
of fancy analyses that I\'m talking about, Bob, but actually in
extremely simple ones. Class effects of channel blockers, class effects
of any class of drugs. That allows you to define a relatively small set
of questions.

If you had a relatively small set of questions, it is possible to model
incidences. It is possible to model exposure effects. It is possible to
make intelligent guesses not so much to increase sample sizes, which I
would not be an advocate for in this setting, but to assess the
importance of individual observations and the need.

I feel very strongly about this next point. We\'ve got an opportunity
here to recommend something, to recommend some things that we have not
been able to do before to make this whole process work an awful lot
better. It may often be in Phase IV.

Nevertheless, I think it is really worthwhile being able to come out of
this process with processes \-- I think actually decision algorithms are
a very, very useful example of this

\-- processes which allow us to do the safety stuff better in order to
be able to do the kind of intense study that happens after an event is
registered as real as the French do in the yellow card system. We are
the people, the researchers, who get thrown that problem.

In order to pick the ones that are worth going after, I think it is very
possible to come up with decision algorithms, focused questions that
come up with what are the real possible hypotheses because really much
of the time actually it is not a needle, Bob. It\'s something that is
one of a list of relatively short number of things.

MS. HENDERSON: Dr. Szarfman.

DR. SZARFMAN: David Flockhart, you brought up an issue and you have the
most experience. I have only limited experience but with my limited
experience in clinical trials analyses, you expect a half-life for a
drug that is 14 hours and suddenly the patient outliers have a half-life
of 5 days, 7 days. It gives you some level of maybe assurance that
something is going on and we can say maybe these are isolated cases.

We use blood levels to actually assist causality. At certain levels the
patient is having seizures and we are doing that. I think why not to do
it more in clinical trials to help assess causality a little better to
move away and ascertain it better.

DR. FLOCKHART: Personally, I think blood levels are occasionally
valuable in specific cases where there is a tight correlation between
the dynamic effect and the concentration. We don\'t use the afilina
anymore for anything. I think in most random settings where you are able
to take a blood level, it is in general not very valuable unfortunately.

MS. HENDERSON: Dr. Temple.

DR. TEMPLE: David, can you explain a little further what you just said?
We did identify certain things that always ought to be worked up. For
example, the electrocardiograph effects on the QT interval. In fact,
four drugs have disappeared because they had a problem with that.

Another large way drugs have disappeared is because they were
hepatotoxic and so we obviously are very interested in drugs with a real
problem as opposed to a drug that just bounces the transaminase.

Are you saying that there are more things like that that we should be
sure we work up because we know they are a problem? If that\'s what you
mean, say what they are.

DR. FLOCKHART: No. I\'m saying that the list of end results are really
the list that you came up with a moment ago. But our ability to \-- I\'m
really making the biomarker point that there may be much, much better
ways than LFTs. I\'ve made it before. QT interval is so unbelievably
rare that we are going to have to go for some genetic thing there.

It is also the case, and I was picking up on Mark\'s point, that when a
drug goes on the market now, we have a very good idea of its metabolism
if it\'s a small molecule. And we have a very good idea \-- we better
damn well have a good idea of what its receptor is.

Maybe beyond that how the receptor works distally. That to me is a
series of important not just random hypotheses. Those are the genes I
would go after. Those are the polymorphisms I would specifically go for.

DR. TEMPLE: I didn\'t see how that was connected to discovering an
unexpected adverse reaction. Or are you saying it shouldn\'t be so
unexpected if you paid attention?

DR. FLOCKHART: I prefaced my remarks by saying there will always be
needles. There will always be things that come absolutely out of
nowhere.

DR. TEMPLE: So we need to pay attention to the more predictable.

DR. FLOCKHART: Exactly. There are more ways of predicting them.

DR. TEMPLE: You should send of a list, though, because we would really
like that.

MS. HENDERSON: Additional comments or questions. Dr. Gormley.

DR. GORMLEY: I was going to save this perhaps more philosophical
question towards the end but we may be getting there. It\'s a question
transitioning between risk assessment and risk management.

As we begin to \-- the question centers around whether our goal or the
goals of the steering committee that were developing these papers
intended to manage risk in isolation, or optimize the benefit risk of a
product.

As you begin your dose ranging studies for any product, you are
eventually going to get into a range where it has efficacy. As you
increase the dose, you will see more efficacy and more toxicity most
likely.

Eventually you will get to a point where the toxicity or the side
effects no longer justify the efficacy. You are beyond a range that is
reasonable. But you are left usually with a fairly wide range.

I guess that is not always the case but a range where you could either
go for the high efficacy with risk or the low efficacy with less risk.
The documents talk about managing the risk. They don\'t talk about how
you \-- whether you would choose to manage more risk to obtain more
benefit.

I wondered whether optimizing benefit risk is what you had in mind when
you used the word risk or whether that was an isolated concept that we
were dealing with because when we go to talk about measuring it and
managing it we probably will get there.

I would have loved to direct this one to Rob Califf for an interesting
discussion but he\'s not able to join us so maybe, Bob, you could take
it or anyone who has a thought about it.

MS. HENDERSON: I think I\'ll direct it to the chair of the working
group, Bob Meyer.

DR. MEYER: I\'m not sure that I would necessarily use the term
optimizing risk benefit, but I think inherent to all these discussions,
particularly today\'s and tomorrow\'s, there is, I think, the
underpinnings that we do want to take the therapeutic advantage of that
drug and bring it to the level that we can so that risk assessment, I
think, is \-- I hope I said it at the beginning of my talk.

If I didn\'t, I should have \-- really based on having a good knowledge
of the efficacy as well. Although we didn\'t touch upon that in the
paper, I think you are looking for putting what you know about the
efficacy then in balance with a good determination, what the safety
issues are with that drug, and then make it a reasonable balance to get
that to as good a place as we can get.

The reason I just wanted to avoid the word optimization is that I think
as somebody pointed out earlier that implies a whole level of certainty
that is beyond what we probably can reasonably achieve in most clinical
development programs.

MS. HENDERSON: Dr. Temple.

DR. TEMPLE: The ICH dose response document sort of addresses this. There
is a lot of talk about finding the optimal dose, etc., etc. What that
document says is what you need to do properly is characterize the dose
response for the good things and the dose response for the bad things
depending on what the bad things. The good things are different people
will form opinions about what dose to choose.

For example, if the side effect that comes on is something that is
unpleasant but survivable, you might choose to start a high dose and
drop back. If the bad thing that happens is a fatal arrythmia, then you
wouldn\'t do that.

Just to pick an example, the dose chosen of dofetilide is probably not
the most effective dose, but it\'s the dose that is most likely to give
you QT prolongation of a large degree. We made the judgement that dying
is probably undesirable and that you would want to start a dose that was
cautiously chosen in that case.

It is a judgement that you would make every time. What you really need
to do is know about this so that you can label it intelligently, advise
it intelligently, make intelligent judgements.

I think that is all of the things we are talking about whether
effectiveness, as Bob says, or safety you want to understand what these
things are so you can deal with them intelligently; labels, describe
dosing paradigms, who to avoid, all that stuff.

MS. HENDERSON: Dr. Racoosin.

DR. RACOOSIN: I\'m going to pull this back from the philosophical more
back to the practical. I did want to address something, an issue that
Dr. Kahn had raised in his presentation. It had to do with the misnaming
of adverse events by investigators and the idea that the sponsor might
rename those based on the data that is actually there.

I don\'t think that we had really talked about this in the working group
too much, but I think it\'s an idea worth spending some time thinking
about because as someone who spent the last \-- much of the last six
years pouring over NDAs, I am very nervous when I see an adverse event
called acute liver failure that\'s presented not in the serious adverse
event column or doesn\'t have a narrative or a case report form. It
often leads to a lot more questions in trying to sort out, well, in
actuality it was just elevated transaminase.

This gets me to, I think, an even more basic issue that Ellis presented
at the end of his presentation. That is really we are having to make
these determinations of safety based on the information that is
presented to us.

I just want to put in an additional plea that I think will also come out
in the document that our ability to judge safety problems is really
based on the data that we have to consider. I would just stress that in
order to do that and in as thorough as possible way to see do we really
have these events occurring is to provide the documentation.

Unfortunately in many cases that documentation is not included and it
can lead to prolongation of the review process. I think this is very
key. We can plan our trials to include all of the very population, or
keep it more narrow but regardless if we don\'t get the information to
review we are not going to be able to make any determination about
safety.

MS. HENDERSON: Comment from the audience.

PARTICIPANT: Just to follow up on that FDA comment, I think it would be
a very dangerous path to proceed if we end up with safety databases that
primarily contain sponsor-derived terms. That really is a nightmare for
any drug safety physician.

It is our usual practice to upgrade rather than downgrade. I think
safety data should be presented verbatim. We also need to bear in mind
that in Europe the EMEA actually wants AE terms reported at LLT level.
If sponsors get into the situation whereby they start to provide sponsor
derived AE terms, that certainly would be a recipe for problems down the
road. I would strongly argue against such an approach.

Having said that, the analysis of clinical safety data primarily should
be done on investigator terms. It is left to the sponsor to then analyze
that aggregate dataset from their own perspective. These are two
distinct presentations of data that need to be recognized. That goes
into the clinical summary. For the integrity of clinical safety data
reported by investigators ought to be maintained primarily.

Having said that again, one other issue is that our clinical colleagues
who are project champions are more than ready to downgrade events or
change AE terms for all sorts of reasons. Now, the conservative approach
obviously is to maintain the high index of suspicion irrespective of how
AEs may appear during clinical development. Certainly I would argue
against that approach.

MS. HENDERSON: Dr. Racoonsin.

DR. RACOOSIN: I certainly didn\'t mean to imply that was going to be the
standard that adverse event terms are going to be altered. I think,
though, that when an adverse event term like acute liver failure is not
accompanied by any of the qualifications for that meaning an elevated PT
over a certain amount, encephalopathy, jaundice, etc., that it needs to
be addressed in some way.

Now, I believe that the slide described keeping the \-- making certain
that the audit trial is there so I\'m certainly not implying that. And I
think the other thing is that every \-- any thorough review of the NDA
data is going to include a scrutiny of the coding from the verbatim term
to the preferred term so that is going to persist regardless.

MS. HENDERSON: Dr. Unger and then Dr. Kahn.

DR. UNGER: I think I\'ve seen a number of databases where the adverse
events as reported by investigators are, in fact, appropriately cleaned
up and upgraded by the sponsor. But it is also not unusual to find at
the end of the day, for example, if you are looking for heart failure
then, in fact, you find a substantial fraction of subjects who developed
acute pulmonary edema which was classified as a respiratory problems and
not as heart failure. You go back to the sponsor and you say, \"You
either try to explain to me that this is respiratory distress syndrome
or it is heart failure.\" The sponsor will say, \"Well, it\'s heart
failure.\"

The point is that discovery comes to the prepared mind. Often it\'s
possible to have a prepared mind if you have priors. If you don\'t have
priors, sometimes you have to go back retrospectively and go through the
data.

It is very difficult, I think, to come up with standard coding. Well,
the standard coding you can come up with but there are many times where
I think it is necessary to kind of go back and rethink it and
reclassify.

MS. HENDERSON: Dr. Kahn.

DR. KAHN: Sidney Kahn. This discussion of terminology is fascinating
because it really does drive everything that we do because the output
depends on garbage in, garbage out. One area that clearly this risk
management concept paper doesn\'t address that is well addressed in the
concept paper for day three is the issue of aggressive follow-up.

Certainly FDA\'s proposing that sponsors that market the products pursue
aggressive follow-up anytime something is reported to them that is
serious and unexpected. In fact, the same should apply in the clinical
trial environment.

If you have an adverse event that looks odd, it is incumbent on the
sponsor, safety group, or clinical research group to follow up
aggressively with the investigator the issue. What I tried to present in
my talk was not that this would be \-- that you would regularly change
investigative terms.

That was not the intent. The intent was to say that there may be
situations where you go back to the investigator who may be casting
aspersions or any medical discipline here. A psychiatrist was treating a
patient with an antidepressant and this patient developed hepatic
necrosis and you don\'t know what that means.

You go back and you follow-up and the psychiatrist said the patient had
minor transaminase and an ALT of 50. That already happens when the liver
is damaged so it is hepatic necrosis or liver damage or whatever.

Now, in that situation if the investigator does not wish to change the
term, I see it as being entirely appropriate for the sponsor to document
that, have it copied on the case report form query page in the database,
and use the term ALT increased, if it is asymptomatic, for example, as
the term rather than hepatic necrosis which is meaningless.

This is what is being recommended in the CL document. Not wholesale
changes to the database but rather very targeted and focused changes for
coded terms that don\'t make sense as presented because that helps
nobody.

MS. HENDERSON: Yes, sir.

PARTICIPANT: Just to follow up on that, I agree with that comment.
Earlier on it had been said that a lot of the problems we have with data
quality for ICSRs primarily relates to the post-marketing phase. I would
actually disagree with that. During premarketing there is still a lot of
work that needs to be done regarding the data quality of these ICSRs.

One of the tools now that currently is being used is the use of
questionnaires especially where one anticipates designated medical
events. These can be used perspectively and aggressively for follow up.
Certainly the issue of data quality is not an issue that we have to face
only with post-marketing reports.

It still exist in the premarketing phase. Ultimately it is going to
depend on individual company practices that will evolve a standard of
quality for their AE or SAE reports.

For example, there are some companies out there that all in the name of
automation actually data is entered into central safety databases in the
absence of medical input. You now have a situation whereby medical
assessors just review the draft narratives. Certainly that sort of
approach would not facilitate the highest level of data quality for the
individual safety reports.

I think we can try and formulate and evolve guidance based on concept
but individual company practices ultimately is going to determine the
level of quality that will be achieved for individual case safety
reports.

MS. HENDERSON: Thank you.

Yes, sir.

PARTICIPANT: Hello. I just want to make a comment. For our discussion
that we have today, either downgrading or upgrade this code or data
pooling or data analysis, I think all the discussions go back to the
fundamental philosophical question that at this stage of drug
development should we do everything reasonable we can to generate this
positive signal even when we know there might be some false positive.
That is the philosophical question.

Or are we just afraid there might be some false positive to prevent us
to do something might be in our capacity we may be able to do like
downgrading, upgrading given the data pooling. That is just a comment.

MS. HENDERSON: Yes.

DR. AVIGAN: Another variation of the theme that should be considered
that I have seen in some safety databases the transaminase elevations
was mentioned. That has a high background rate both in the placebo group
as well as in the active treatment group invariably.

One of the questions that is a real question is in that dumb number,
which often is the same between the two groups, is there a subset of
patients who have something which includes transaminase but is actually
much more extreme that under the rubric of transaminitis alone would not
be recognized at face value.

One of the real review issues that again in terms of the reporting and
the using of the appropriate terms and given the appropriate narratives
is to dissect out the subsets that are clinically more meaningful of
that subset which actually is a strong signal and distinguishes one
treatment arm from another. I think that is a very important point.

MS. HENDERSON: Yes.

DR. SIMON: I guess my feeling is I would not \-- I mean, I think it\'s
good to encourage getting good data at a preclinical stage, randomized
data. I think that is a great advantage for interpreting it.

I think, as I\'ve said several times, you have to be very careful in
terms of how you interpret that data and you have to interpret it
intelligently with the recognition that there is an opportunity for lots
of false positives.

You have to try to follow the leads, the signals that you see to try to
assess sort of beyond just the statistical subset analysis in terms of
looking at subsets where you might expect for their to be the effect. So
I think ultimately then you may be left with certain uncertainties.

I guess what I\'ve been arguing for \-- I guess I\'m not arguing for not
trying to get good data at the premarketing stage, but rather not
interpreting in sort of a naive sort of way.

MS. HENDERSON: Thank you, Dr. Simon.

We have a commenter at the microphone.

PARTICIPANT: Susanna Perez. It\'s a question to the working group
whether you had the change to discuss the use of observational and
analytical techniques of the clinical trial data sets such as nested
case control analysis or person time analysis using actual exposure
rather than intention to treat exposure.

DR. RACOOSIN: We have used those methods in individual NDAs. I don\'t
think that we really got to a level of discussing that particular \-- at
least nested case control for the purposes of the concept paper.

MS. HENDERSON: Dr. O\'Neil.

DR. O\'NEIL: I think that is a really an interesting line to think
about. There has not been very much attention paid to that. It\'s going
to get harder and harder as we do comparative randomized studies where
individuals cannot stay on the drug. This goes back to my earlier
comment.

I think the biggest problem with long-term safety, and by long-term I
mean after 16 weeks, or even six months. Long-term is really a relative
term. I\'m talking about 15-year outcome studies. The dramatic loss to
follow-up after six months in a lot of studies.

I think most randomized studies turn into observational studies real
fast, particularly when you are not looking at the major efficacy
outcomes when you start to look at all the other outcomes because it is
essentially then an exploratory study that is much closer to looking
like an epidemiologic study.

I think some of the strategies you are talking about, which are relying
much more on matching and propensity scores and some other ways to look
at comparisons and do comparison are really worth thinking about and we
haven\'t spent that much time in this area. It\'s worth pursuing.

MS. HENDERSON: Dr. Temple.

DR. TEMPLE: Another thing that we have done in that arena, although
somewhat uncomfortably, is to look at databases from other related drugs
to compare results with the drug we are looking at. We have done that
with antipsychotic drugs looking at overall and sudden death rates when
we have a potential problem.

Obviously those things are fraught with danger and we are looking to see
whether we can get better access to our own databases. Presumably now
that they are electronic it should be more possible to do that, and even
try to look at baseline characteristics and things like that.

But it is very hard with what we have got now but we have entertained
that and it is one of the things we think about when we think about
electronic data sets. For rare events, of course.

MS. HENDERSON: If there are no further questions, I will turn it back
over to Bob Meyer for closing remarks.

DR. MEYER: Thank you. I have a half-hour scheduled so I am going to
regale you with the heroics of Diane Terassi and the UConn women against
the Tennessee volunteers last night.

Seriously, though, I think this has been a very helpful session. Dr.
Gormley started off saying that they noted that we use fairly
conditional language in this document and perhaps had a lack of
definitive advice. He had expressed the hope that was intentional and,
indeed, it was.

We wanted to engender, I think, the very kind of discussion that we had
today. I think that we have heard some helpful and useful things that
will go into our consideration of the draft guidance document that will
result out of this.

Of course, we will be looking for further public comments to the docket
which I believe is open through the end of the month. We will
incorporate all that into the draft guidance which then again will get
further input from the public at that point.

I would want to note a couple of repeated themes I heard today. There is
a lot of good things said but a couple of things I think came back
repeatedly. I heard a call that I think we share for full integration of
the safety assessment throughout the life cycle of the drug.

From the animals to the clin-pharm to the clinical to the post-approval.
That is all integrated and thought through on a continuum so that the
risk assessment is best informed at every possible point.

Second thing that I heard as a recurring theme was the idea of targeting
the study population and, indeed, perhaps the study designs to obtain
the data that is informed by the understanding of likely use or likely
concern for that drug. We heard the term scenario planning earlier.

I think particularly I would note that the call on our part that they
are not the restriction of the study population, perhaps that is best
put into perspective in that if you get too broad and you have too
erogenous a population, you may not get meaningful data so there may be
times where it does make a lot of sense to in addition to not
restricting needlessly the population to doing specific studies that are
focused on questions or concerns in, say, the elderly or pediatrics
certainly, and other populations as well.

Finally, I think that I heard that FDA should be clear and direct where
we can be when we get to the guidance stage and articulate where things
will remain case by case where there might be evolutionary strategies
throughout the drug development and that we be clear on those
circumstances where we will be looking at things perhaps more on a
case-by-case basis versus having a more firm guidance. I think we intend
to do so. Again, we kept this document quite general with the intent of
stimulating discussion and I think that worked.

Finally, I would like to thank the risk assessment work group
appreciatively known as group one, many of whom you met today. I would
specifically like to again thank B. J. Gould who is our project manager
and has done a wonderful job. Not part of group one but Lee Lemley as
well who overall did the organization for this series of meetings.

I would like to thank our panelists, both the external guests who
participated today as well as the steering committee members and pay a
special thanks to Debbie Henderson for her very fine job of moderating
today.

Lastly, I would like to thank all of you who attended. I do look forward
to further public comment either through the written format or at the
time we go from draft guidance to guidance. Thank you for your interest
and thank you for your input.

MS. HENDERSON: We will start tomorrow and Friday at 8:00 a.m.

(Whereupon, at 3:52 p.m. the meeting was adjourned.)