UNITED STATES OF AMERICA

FOOD AND DRUG ADMINISTRATION

\+ + + + +

CENTER FOR DRUG EVALUATION AND RESEARCH

\+ + + + +

ADVISORY COMMITTEE FOR PHARMACEUTICAL SCIENCE

\+ + + + +

MANUFACTURING SUBCOMMITTEE

\+ + + + +

OPEN PUBLIC HEARING

\+ + + + +

WEDNESDAY,

JULY 21, 2004

\+ + + + +

The above entitled meeting was conducted at 8:30 a.m., in the CDER
Advisory Committee Conference Room, 5630 Fishers Lane, Rockville,
Maryland, Dr. Judy P. Boehlert, Subcommittee Chairperson, presiding.

[PANEL MEMBERS PRESENT:]{.underline}

JUDY P. BOEHLERT, Ph.D., Chair, Manufacturing

Subcommittee

HILDA F. SCHAREN, M.S., Executive Secretary, Advisors

and Consultants Staff, CDER, FDA

PATRICK P. DeLUCA, Ph.D., Professor, Faculty of

Pharmaceutical Science, University of Kentucky

DANIEL GOLD, Ph.D., D.H. Gold Associates

GERALD P. MIGLIACCIO, Vice President, Global Quality

Operations, Pfizer, Inc.

[PANEL MEMBERS PRESENT:]{.underline} (cont.)

KENNETH M. MORRIS, Ph.D., Department of Industrial and

Physical Pharmacy, School of Pharmacy, Purdue

University

GARNET PECK, Ph.D., Industrial and Physical Pharmacy,

Purdue University

JOSEPH PHILLIPS, Regulatory Affairs Advisor,

International Society of Pharmaceutical

Engineers

G.K. RAJU, Ph.D., Executive Director,

MIT/PHARMI, MIT Program on the Pharmaceutical

Industry, Massachusetts Institute of Technology

NOZER SINGPURWALLA, Ph.D., Director, Institute for

Reliability and Risk Analysis, Professor of

Statistics, George Washington University

[FDA STAFF PRESENT:]{.underline}

GARY BUEHLER, R.Ph., Director, Office of Generic

Drugs, OPS, CDER

JON CLARK, M.S., Associate Director for Policy

Development, OPS, CDER

H. GREGG CLAYCAMP, Ph.D., CHP, Director, Scientific

Support Staff, Office of New Animal Drug

Evaluation, Center for Veterinary Medicine, CDER

JOSEPH FAMULARE, Director, Division of Manufacturing

& Product Quality, Office of Compliance, CDER

BRIAN J. HASSELBALCH, Ph.D., Consumer Safety Officer,

Division of Manufacturing and Product Quality,

Office of Compliance, CDER

DAVID HOROWITZ, Esq., Director, Office of Compliance,

CDER

AJAZ HUSSAIN, Ph.D., Deputy Director, Office of

Pharmaceutical Science, CDER

DONALD MARLOWE, FDA Standards Coordinator, Office of

Science and Health Coordination, Office of the

Commissioner

STEPHEN MOORE, Ph.D., Team Leader, Division 2, ONDC,

OPS, CDER

MOHEB NASR, Ph.D., Director, Office of New Drug

Chemistry, OPS, CDER

[FDA STAFF PRESENT:]{.underline}

CHRISTOPHER WATTS, Ph.D., Process Analytical

Technology (PAT) Policy, Office of

Pharmaceutical Science, CDER

HELEN WINKLE, Director, Office of Pharmaceutical

Science, CDER

[ALSO PRESENT:]{.underline}

JEFFREY T. MACHER, Ph.D., Assistant Professor,

Georgetown University

JACKSON A. NICKERSON, Ph.D., Associate Professor,

Washington University in St. Louis

NGA TRAN, Ph.D., Contractor to FDA\'s Office of

Compliance

JOHN BERRIDGE, Ph.D., Vice President, Pharmaceutical

Sciences, Pfizer, Ltd.

PAUL FACKLER, Ph.D., Senior Director, Product and

Biopharmaceutics Strategy Development, Global

Generic Research and Development, Teva

Pharmaceuticals

TOBIAS MASSA, Ph.D., Executive Director, Global

Regulatory Affairs, Operations/Chemistry,

Manufacturing and Controls, Eli Lilly & Co.

FREDERICK RAZZAGHI, Director of Technical Affairs,

Consumer Healthcare Products Association

[C O N T E N T S]{.underline}

[PAGE]{.underline}

Introductions 7

Conflict of Interest Statement 9

Introduction to Pharmaceutical Industry Practices

Research Study, David Horowitz, Esq. 11

Update on Pharmaceutical Industry, Jackson

A. Nickerson, Ph.D. 15

Practices Research Study, Jeffrey T. Macher,

Ph.D. 29

Pilot Model for GMP Inspection, H. Gregg

Claycamp, Ph.D. 65

Presentation of David Horowitz, Esq. 98

Presentation of Nga Tran, Ph.D. 106

Presentation of Brian J. Hasselbalch, Ph.D. 134

eGMPs for the Production of Phase 1 INDs,

Moheb Nasr, Ph.D. 182

Presentation of Joseph Famulare 187

Applying Manufacturing Science and Knowledge,

Ajaz Hussain, Ph.D. 211

Process Understanding and PAT, Chris Watts,

Ph.D. 213

Comparability Protocol Stephen Moore, Ph.D. 245

P-R-O-C-E-E-D-I-N-G-S

(8:41 a.m.)

CHAIRPERSON BOEHLERT: Good morning, everyone. We\'ll start by going
around the room and introducing ourselves. David Horowitz, could you
start by introducing yourself and your affiliation?

MR. HOROWITZ: I\'m David Horowitz. I\'m the Director of CDER\'s Office
of Compliance.

MS. WINKLE: Helen Winkle, Director of Office of Pharmaceutical Science,
CDER.

DR. HUSSAIN: Ajaz Hussain, Deputy Director, Office of Pharmaceutical
Science, CDER.

DR. CLAYCAMP: Gregg Claycamp. I\'m Director of Scientific Support Staff
at CVM.

DR. GOLD: I\'m Dan Gold. I\'m not director of any agency. I\'m with D.H.
Gold Associates.

DR. PECK: Garnet Peck, Purdue University.

MS. SCHAREN: Hilda Scharen. I\'m the Executive Secretary of the Advisory
Committee for Pharmaceutical Science, FDA.

CHAIRPERSON BOEHLERT: Judy Boehlert, Boehlert Associates, LLC.

DR. MORRIS: Ken Morris, Purdue University.

DR. DeLUCA: Pat DeLuca, University of Kentucky.

DR. RAJU: G.K. Raju, MIT Pharmaceutical Manufacturing, NSU.

MR. PHILLIPS: Joe Phillips, International Regulatory Affairs Advisor,
International Society for Pharmaceutical Engineering.

DR. SINGPURWALLA: Nozer Singpurwalla, George Washington University.

MR. MIGLIACCIO: Gerry Migliaccio, Pfizer, representing innovator
companies.

DR. FACKLER: Paul Fackler, Teva Pharmaceuticals, representing the
generic industry.

CHAIRPERSON BOEHLERT: And Joe, do you want to?

MR. FAMULARE: Joe Famulare, Director, Division of Manufacturing and
Product Quality, CDER Office of Compliance.

CHAIRPERSON BOEHLERT: Okay. Thank you, everyone, and once again, welcome
to today\'s session.

Hilda Scharen will now read the conflict of interest statement.

MS. SCHAREN: Good morning. The following announcement addresses the
issue of conflict of interest with respect to this meeting and is made a
part of the record to preclude even the appearance of such at this
meeting.

Based on the agenda, it has been determined that the topics of today\'s
meeting are issues of broad applicability and there are no products
being approved at this meeting. Unlike issues before a committee in
which a particular product is discussed, issues of broader applicability
involve many industrial sponsors and academic institutions.

All special government employees have been screened for their financial
interests as they may apply to the general topics at hand. To determine
if any conflict of interest existed, the agency has reviewed the agenda
and all relevant financial interests reported by the meeting
participants.

The Food and Drug Administration has granted general matters waivers to
the special government employees participating in this meeting who
require a waiver under Title 18, United States Code, Section 208.

A copy of the waiver statements may be obtained by submitting a written
request to the agency\'s Freedom of Information Office, Room 12A-30 of
the Parklawn Building.

Because general topics impact so many entities, it is not prudent to
recite all potential conflicts of interest as they apply to each member
and consultant and guest speaker. FDA acknowledges that there may be
potential conflicts of interest, but because of the general nature of
the discussion before the committee, these potential conflicts are
mitigated.

With respect to FDA\'s invited industry representative, we would like to
disclose that Gerald Migliaccio is participating in this meeting as an
industry representative acting on behalf of regulated industry. Mr.
Migliaccio is employed by Pfizer.

Dr. Paul Fackler is participating in this meeting as an acting industry
representative. Dr. Fackler is employed by Teva Pharmaceuticals.

In the event that the discussion involves any other products or firms
not already on the agenda for which FDA participants have a financial
interest, the participant\'s involvement and their exclusion will be
noted for the record.

With respect to all other participants, we ask in the interest of
fairness that they address any current or previous financial involvement
with any firm whose product they may wish to comment upon.

Thank you.

CHAIRPERSON BOEHLERT: Thank you, Hilda.

We will be addressing two topics this morning, the pharmaceutical
industry practices research study and pilot model for prioritizing
selection of manufacturing sites for GMP inspections. And David Horowitz
is going to introduce us to these topics.

MR. HOROWITZ: Okay. We\'re going to start off with the studies that \--

CHAIRPERSON BOEHLERT: Can you turn on your mic.

MR. HOROWITZ: Okay. We\'re going to start off with the two studies that
Jeffrey Macher and Jackson Nickerson will be presenting.

Jeffrey Macher is a professor at Georgetown\'s Business School, and
Jackson Nickerson is a professor at Washington University in St.
Louis\'s Business School. They both have M.B.A.s and doctorates in
business. In addition to that, I believe Jackson Nickerson has a
Master\'s degree in mechanical engineering, which is also an interesting
complement.

They have both done extensive work prior to focusing on the
pharmaceutical industry on the semiconductor industry and produced a
very highly regarded and participated in a very highly regarded and
successful study of that industry that has been very helpful to that
industry.

And they\'re going to be using some of the same techniques and
approaches in examining the pharmaceutical industry, but also the
regulatory side of this industry that\'s somewhat unique from the
semiconductor and many other industries.

They\'s be discussing two closely related studies of pharmaceutical
manufacturing and regulation, the first of which focuses on FDA\'s
regulatory oversight of drug manufacturing, and they\'ll analyze various
FDA databases using econometric techniques to identify factors that are
predictive of FDA oversight and regulatory outcomes.

Now, this is of interest to us as well sa it is to industry presumably.
We hope that the study will facilitate our ongoing efforts as part of
the GMP initiative to enhance our regulatory oversight, including
aspects of coordination and consistency which we are trying to address
in the GMP initiative and aspects of increasing the risk based focus of
our programs.

The second study will investigate the relationship between FDA\'s
regulatory oversight and the resulting production and regulatory
performance of the drug manufacturers, and it will also look at the
effective of pharmaceutical manufacturers\' organizational variables on
production as well as regulatory performance.

This, of course, is of great interest to the pharmaceutical industry.
It\'s also of interest to us for a wide variety of reasons, one of which
is that we hope to be able to incorporate some of the learning and some
of these results into the model that we\'ll be discussing, which is a
work in progress trying to help us prioritize manufacturing sites for
GMP inspections.

The connection between the two you can probably see, is that factors
associated with strong regulatory performance and production may support
reduced frequency or scope of inspectional oversight.

We also hope generally to gain more insight into how FDA policies and
actions affect industry performance and behavior to better tailor and
adjust our actions to achieve the desired results.

After Professors Macher and Nickerson speak, there will be four
speakers, including myself who will discuss the use of a technique known
as risk ranking and filtering, as we are attempting to apply it to
FDA\'s efforts to prioritize manufacturing sites for GMP inspections.

Starting will be Gregg Claycamp, who will discuss risk ranking and
filtering as a risk management tool and putting it in the context and
comparing it with certain other types of risk management tools.

I\'ll follow by providing some context and a little bit of an
introduction to this first iteration of our site selection model.

After that I\'ll be followed by Nga Tran and Brian Hasselbalch who will
discuss in more detail how FDA went about designing the model, including
many of the data limitations and hurdles that we face in seeking comment
and assistance, and also discussing a technique that we have begun using
called expert elicitation. But it\'s only the beginning, and one of the
reasons we\'re here is because we want more input on that model and we
hope in the future to expand it publicly.

So with that I\'ll ask Professor and Macher to begin.

Thank you.

DR. NICKERSON: Madam Chairperson, committee members, attendees, good
morning. David did such a great job I don\'t think I need to stand up
and give you any presentation. He gave you a very good summary of what
we\'re doing.

Yesterday you heard quite a few words around science, pharmaceutical
manufacturing knowledge, and we also heard some words about management,
organization, incentives. That last category, that latter category of
words falls squarely in the domain of management and business.

It\'s interesting to look around the committee because a necessary
condition to make all of the changes that have been described both on
the FDA side, as well as the industry side, is this notion of management
and change. Yet I don\'t notice anyone from a business school on the
committee. So hopefully the approach that we\'re taking might be new and
different and useful, and we think the FDA is going to find it useful.

We\'ll also tell you about the manufacturing study and the manufacturers
believe it\'s useful because a large number of them have participated.

So what I\'m going to do is tell you about two studies. One we call the
FDA research project, and some of you have heard of these projects
before. Some have not. So we\'re going to walk a fine line between
giving you some introduction and hopefully a little depth, but not too
much depth.

Jeff will stand up and talk about the pharmaceutical manufacturing
research project. Let me just give you a little history about the FDA
project, as we call it.

It began when Jeff and I had a phone call back in the fall of 2001. We
had read some recent press reports that there was an increase in the
number of FDA actions against manufacturers, and this was interesting to
us because we had both participated in a Sloan semiconductor foundation
grant where we studied the semiconductor industry, looking at best
manufacturing practices, and we thought that sort of methodology might
be useful in the pharmaceutical industry. So over the next year and a
half we pursued this topic with the FDA and with manufacturers and
ultimately got the project off the ground.

Let me tell you what the goals of the project are. There are three. We
believe we can develop a risk based assessment of GMP outcomes, that is,
trying to understand why and when we see various outcomes.

In order to do this, we have to identify those attributes that are
correlated with those inspection outcomes, and I\'ll tell you a little
bit about how we\'re going about doing those correlations.

And finally, what we learn we hope to transfer to the FDA. So this is
both in terms of our analysis, some data, and analyzing that data, but
also the methodology or framework that could be used as we move forward
in time.

So let me tell you about the approach. We spent a lot of time
interacting with various people in the FDA in order to identify what
data sets already exist in the FDA. We weren\'t going to create a new
data. We\'re going to leverage off existing databases.

We\'re going to look at and estimate the likelihood of various types of
outcome. You\'re all familiar with the inspectional outcomes and some
other outcomes I\'ll talk about in a few minutes.

Well, in order to estimate the likelihood of these outcomes, we have to
look at a number of factors, and I\'ll review all of those factors. The
factors are about the product of the compound, the plant, the firm, but
also factors about the FDA and the investigators and the amount of
resources allocated and the likelihood of an inspection being chosen.

And out of this, we believe we can allocate or investigate the
allocation of resource and perhaps develop a model to provide some
estimate of what the risk is for either delaying inspection or
accelerating inspection. In other words, how do we optimally allocate
the FDA\'s resources?

And finally, we think we can provide some feedback to the FDA about how
they manage and train their investigator work force and also some
information about the different districts and hopefully some of that
will come through as I talk about the data and the analysis.

Well, we found a number of databases in the FDA. Unfortunately they all
don\'t talk to each other. so part of the big task is once we get all of
this data, we have to combine it and, in essence, clean it so that we
can match it up.

There\'s a database called COMIS, which deals with supplement filings,
DQRS which deals with field alerts. There\'s some outsourcing
information in something called EES. FACTS is the database that is
largely in the ORA and deals with inspections. Product listing, product
recalls, product shortages, those are fairly straightforward.
Registration, which is an annual database. Warning letters, and we
helped construct a training database so that we know at what point in
time the level of the training, the type of course that the different
investigators had before they went out on inspection.

Now, this is all collected, and we\'re trying to integrate all of the
data in order to develop these statistical models.

What are the important outcomes? Well, I already mentioned the
inspection outcomes. On action, voluntary action or ordered action
indicated. Those are the standard outcomes from each investigation.
Beyond that, you might get a warning letter, but there are also other
outcomes, perhaps more real outcomes in terms of field reports, product
recalls, and product availability. So we\'re going to use those outcomes
in our analysis.

Some of the key factors that we\'re looking at that\'s already collected
by the FDA include what type of compound is it. Is it an NDA or ANDA? Is
it prescription versus nonprescription? Some information about the
product class, product subclass, process indicator code. Those are
somewhat rough measures, but measures nonetheless.

We have supplement history, the extent of vertical integration. At least
for certain aspects do you produce the API and formulate? Is your
testing outsourced or done internally?

We can also assemble the history of regulatory outcomes for the product
at least to 1990. It\'s very difficult to go back before 1990. There was
a major computer system change, and it would be rather difficult to
integrate data before 1990.

And then, of course, we can look at the history of regulatory actions
not only for the product, but also for the plant and also for the firm
to see how that affects the likelihood of inspection or the likelihood
of various outcomes.

Other factors. Well, in terms of the facility, we\'d like to know how
old it is, its size, what\'s produced there, the number and the variety
of products. That may impact the quality manufacturing, if you will, or
it may impact the likelihood of the FDA choosing to inspect. Hopefully
we can tease apart those different motivations.

We can look at the change over time in terms of the number of products
or the diversity in products. Importantly, we can look at ownership
changes. That is recorded in the database, and when you have an
ownership change often systems change, and the question is: is that for
the better, for the worse? What are the issues? Does it encourage the
FDA to inspect? We don\'t know, but we\'ll be able to figure that out
from the data.

And, of course, this regulatory history that I mentioned.

Firm level variables. Again, age and size of the firm. There are a
number of manufacturing locations. What\'s the breadth of product that
they produce, both in terms of number and variety? We can look at things
like number of pass introductions because that may affect the amount of
human resources that are allocated to fixing deviations versus
introducing their products.

We can look at the number of past regulatory decisions. So, for
instance, we have heard some stories that if one plant gets a negative
review, then other plants might get reviewed shortly thereafter, and we
can identify if there are these spillovers or reputation effects that
manifest either within a firm or for a particular compound.

If a particular type of compound, let\'s just say aspirin, if something
was found amiss at a plant, then maybe all other aspirin plants are
inspected right away, and we can identify these sort of behavioral
reactions.

Now, so far I have focused on manufacturer variables, but of course, FDA
variables matter also. So we can identify FDA district, not just
domestically but internationally. We have some estimates on the
inspections, the amount of time allocated, the amount of manpower
allocated to these inspections. We have the number of investigators, the
reason for inspection, who\'s on the team, and the time since the last
inspection.

In terms of the investigators, we can look at some very key issues that
the FDA has already moved to try to correct, and you heard that
yesterday, which is, say, in New England one day an FDA inspector might
be out looking at a blueberry packing facility, a fish packing facility
the next day, and the third day they\'re at a biotech firm. How does
that accumulation of experience matter and translate into the outcomes
that we see? We can evaluate that.

Also, there are different stages of training for these investigators,
and we\'ve collected information on who has received what training by
when, and we can ask questions about how that impacts either the
likelihood of a facility being investigated or the likelihood of a given
outcome.

And I\'m using likelihood and probability interchangeably from our talk,
even though they may not be exactly the same.

To preempt that, we do teach Bayesian economics in the business school.

DR. SINGPURWALLA: But maybe you\'re doing it wrong.

(Laughter.)

DR. NICKERSON: Also, we can assess various policy shifts like the SUPACs
when they were introduced and how that impacted not only when firms were
inspected, but also the outcomes of those inspections.

So once we have all of this data and it\'s all integrated together, what
are we going to do with it?

Well, we want to undertake a statistical analysis to estimate the
probability of the various outcomes that we\'ve described. Now, it\'s a
particularly difficult issue because you can\'t use sort of standard
statistical tools, the big workhorses, something called \"ordinary
re-squares\" (phonetic).

It turns out the FDA chooses to inspect for particular reasons and
manufacturers may choose to place certain compounds in particular plants
for certain reasons, and so we have to account for those choices, which
makes the analysis a little bit more difficult, although there are a
number of good techniques to account for these difficulties.

Once we estimate the model we can use it to ask kind of factual
questions, \"what if\" questions. What is the risk of delaying
inspection on this particular compound or this particular facility or
this particular plant.

We can ask questions \"what if we insure that all investigators had the
full complement of training before they went into the facility\" and ask
a wide variety of \"what if\" questions that we believe can help tease
out the risk of either accelerating or providing some backing off of
regulatory scrutiny.

It should also provide some insight in terms of what sort of things
should be monitored as we move forward, what matters, what are the
critical variables and parameters.

So ultimately we think this analysis will improve our understanding,
FDA\'s understanding, and industry\'s understanding of inspection
outcomes and how they relate to the various attributes that we can
measure.

This risk assessment will be used to inform FDA oversight choices. Now,
this is retrospective data,but again, the framework is something that
can be used also moving forward, and fundamentally it tells us something
about particular processes, particular plans, particular manufacturers,
as well as tells us something about particular district offices and
possibly particular investigators, although we don\'t have investigator
names that are all hidden from us with some sort of ID code so that we
can\'t do that matching.

Well, what\'s the status? We\'ve been working on this for a while now.
We completed what we called a pilot study, which involved interviewing
lots of people in the FDA and, as Jeff will tell you in a few minutes, a
lot of people in industry.

We wanted to identify from both sides of the coin what was important,
what was problematic, what the good stories were, what the negative
stories were in order to shape our analysis.

Phase 2 is collecting data. I\'m happy to report that all the data for
CDER at least, all of those data sets, have been assembled, compiled or
sitting in CD-ROMs on a desk somewhere. We\'re waiting for them to be
released to us, and we anticipate that will happen this month.

Once we have it released to us and we\'re still working with CBER, they
have a different set of data sets, and they integrate a little
differently. So we\'re still working there.

Once the data is in our hands, it will probably take a while to go
through and, as I say, clean the data, typos, data entry mismatches, and
resolve as I understand it there are some 13,000 observations, 13,000
plant visits over this time, maybe even more.

In any event, it will take some time to clean that data, and then there
are actually a variety of statistical techniques that we\'re going to be
using depending on what the particular question is. So that might take
anywhere from three to six months once we have the data in our hands.

That\'s the FDA project, and what I\'d like to do is turn the lectern
over to my colleague, Jeff Macher, who is at Georgetown University, and
he\'ll review what we call the pharmaceutical manufacturing research
project.

Thank you.

DR. MACHER: Okay. Thanks, everybody.

This is pretty much the same presentation, just on the manufacturing
side versus the FDA side now.

This research project emerged at the same time when we were discussing
the increase in severity and number of CJ&P violations, but we are
asking another question. We wondered, based upon what we learned in a
study, a Sloan funded study in the semiconductor industry, specifically
on semiconductor manufacturing, whether these violations were related to
managerial, organizational, and technical practices that we found to be
the case in the semiconductor industry.

We learned a lot from the semiconductor industry, and the benefits that
we gave to firms in reshaping their managerial organizational and
technical practices were demonstrable. Most firms improved significantly
their manufacturing performance, and we wondered if we could do the same
thing here based upon a large scale analysis of the number of
pharmaceutical manufacturers that we could get convinced to participate.

So we began interviewing manufacturers in the spring of 2002 and we
literally traveled around the U.S. and to Europe interviewing dozens of
manufacturers. We tried to be as broad as we could. We interviewed many
pharmaceutical manufacturers, biologics, APIs, contracts and generics.
Generics aren\'t listed there.

Really there was two reasons to do that. One, so that we could come up
to speed on this industry. There are some nuances that we didn\'t really
understand and admittedly we\'re still coming up to speed with it.

And then, secondly, we wanted to ask questions that were important to
the participating firms. So there was a good deal of dialogue and
give-and-take in developing a questionnaire that most firms found to be
pretty effective.

We went live with an Internet based questionnaire in the fall of 2003,
in November, and since then I have principally been engaged in marketing
and soliciting participation.

We expect to close the first round of the survey shortly, and shortly
should be in quotes. We don\'t know when that will be, but shortly.

The goals, very similar to the goals that we had in the semiconductor
manufacturing industry. We wanted to develop a standard set of
benchmarks for measuring, manufacturing, and regulatory performance, and
this in itself is an heroic endeavor. We want to identify the
managerial, the organizational, and technical practices that underlie
good and poor manufacturing and regulatory performance and then provide
a confidential score card \-- and this is one of the reasons why we
think it would be beneficial to the firms that participate \-- to
specific manufacturing facilities on how they perform against anonymous
others so that we can compare API manufacturers to API manufacturers.
We\'ll identify who you are against a set of anonymous others, against a
set of peer groups, and I think that\'s beneficial in and of itself.

Our approach, as I mentioned, we developed this focus questionnaire of
potential factors that we thought and based upon input from industry
influenced manufacturing and regulatory performance. We administered
over a secure Web site via the Internet. We assign a unique user name
and password to each participating manufacturing facility. That user
name and password is used by the individuals within each facility to
fill out the data. It\'s completely secure.

We then collect the data. One of the nice things about this is it dumps
the data that\'s collected on the Internet into a relational database.
We can then analyze the data using a variety of econometric techniques
very similar to what Jackson had already presented to you, and then
provide a summary of our findings.

We\'ll write a couple of white papers, make industry presentations such
as this to industry overall, and as well FDA and industry meetings.

The database. We\'ve secured participation from a cross-section of U.S.
and European manufacturers. We\'ve stayed strictly to U.S. and European
manufacturers. Right now 21 firms and 60 manufacturing facilities that
have either finished the completion of the survey or are actively
completing the survey, and it\'s my job to sort of push these people
through.

One of the difficulties obviously is pharmaceutical manufacturing is
crazy enough. We\'re coming into these facilities and asking them, \"Oh,
by the way, can you do a little more work?\"

It has been trying but usually successful to get these people to commit
to it. It\'s just a process that takes some time.

The survey is, as I mentioned, on line, and each manufacturing facility
provides detailed data on between one and five compounds. We ask for all
of the compounds that are manufactured within the facility, but then we
ask each firm to choose or each facility to choose the top five, where
the top five is defined somewhat loose. It can either be in terms of
volume or it can be in terms of the importance of those compounds to the
facility, where importance could be defined in different dimensions.

What we\'re really asking is what are those top five compounds that you
would change your manufacturing, your technical and organizational
practices if we presented data that showed how you can improve? Okay?

The performance outcomes, instead of the semiconductor industry where we
just looked at manufacturing performance, now we\'re looking at both
manufacturing and regulatory performance. In terms of manufacturing
performance, theoretical and actual yields, batches started and failed,
and then a cycle time measure.

Regulatory performance, failed alerts and biologic deviation reports,
and then warning letters, consent decrees, deviations and supplements.
Where we think we\'re going to make one of the biggest impacts is in
deviation and supplement management.

The related key factors that we\'re asking for in the survey, it\'s nine
sections. Actually it\'s 11 sections, but we sneak two extra sections in
by calling them A and B. The company \-- that\'s a joke, by the way.

(Laughter.)

DR. MACHER: The company in the Strategic Business Unit, we asked for
just some simple financial information as well as some demographic
information, things like facility size, facility age, facility location,
things of that nature.

We ask for some brief financial information on each facility that\'s
participating if they have it, revenues, employee sales, R&D expenses,
property, plant, and equipment, some demographic information, number of
employees, age, size, location. I mentioned a few of these already.

Product information, the number of products or compounds manufactured
and their type, and then regulatory inspection information outside of
FDA. So Brazil, EMEA, Japan, things of that nature.

And then questions on the extent of outsourcing within the manufacturing
facilities, development, process development outsourced. Is any part of
manufacturing outsourced? Are APIs done internal to the manufacturing
facility, internal to the firm, or external?

Product and process development. We do pretty big sections here. Most of
my research investigates new process development. It\'s one of the
things that I\'ve gotten into when I was studying semiconductor
manufacturing.

We look at information on where was product and process development done
in terms of its location relative to the manufacturing facility. How was
it organized? Were engineers from the pilot plant collocated with the
manufacturing facility?

This is really a learning before versus learning by doing approach.

And then the timing. How long did it take? How long did process
development take for the specific compound versus other compounds in
this facility, versus other firms, speed and new process development?

Human resource management, another thing that\'s been one of the things
that we learned from the semiconductor industry, was the importance of
incentives related to human resource management. So we\'re looking at
things like employee appraisal, employee promotion, the mobility and
demographics of employees. How much are they trained? What types of
training?

So we\'re asking for data on things as diverse as SPC controls, all the
way up to a variety of different dimensions.

The extent and use of teams within the manufacturing facility. So we\'re
gathering data on whether they employed quality function deployment
teams, cycle time reduction teams. What\'s the team make-up and
composition? Is it just engineers or are there the lowest level
operators involved with technicians, involved with engineers?

Deviation and supplement management. We look at whether the firm employs
an information technology system to track deviations and supplements.
The extent of process analytic technology, that we\'ve taken
information, taken, borrowed, used information from Ajaz in a section of
the survey to look at deviation and supplement management.

And then finally, how is it organized? Who has responsibility for a
deviation correction once it has been in place? How many people have
authority or need a check-off on that? A variety of questions we ask in
deviation and supplement management.

Where are we right now? As I mentioned, Phase 1 was an exploratory pilot
study which was completed in the summer of 2003, which led to the
development of an Internet based questionnaire.

Phase 2, we\'re nearing the end of it, is data collection. We\'ve been
fairly successful with convincing firms to participate, and a multitude
of firms within manufacturing or a multitude of manufacturing facilities
within a given firm.

We\'ll conclude the first round shortly, but we will most likely
continue to market the survey to other pharmaceutical manufacturers, and
then similar to the FDA study, we\'re going to need some time to go over
the data.

So we imagine the analysis will require three to six month of work where
we\'ll do similar, again, to the FDA study some statistical and
econometric analysis and begin writing final reports.

What\'s not included is, depending on our money, Jackson and I have not
taken any money from FDA or industry. So we are funded through grants
from our respective universities and then economic think tanks.

Depending on the amount of money that we have left, we\'ll either visit
a number of the participating firms to make sure that the data that
they\'ve entered and the results that we show are sensible, or we\'ll
hold conferences either at our respective universities or at a location
to be determined. I\'m thinking Hawaii, but that\'s just me.

(Laughter.)

DR. MACHER: I guess that\'s it, and I think now we have questions,
unless you want to end.

CHAIRPERSON BOEHLERT: No, we do. Thank you, Jeff and Jackson.

We have time for questions for either of those speakers. Yes, Gerry.

MR. MIGLIACCIO: Jeff, the regulatory performance, when you\'re talking
field alerts and deviations, are you looking at or are you looking at
the resolution process?

DR. MACHER: Both. We\'re looking at it, for instance, let\'s say for
deviation management, we\'re looking at the number of deviations within
three separate areas: raw materials, process, and equipment. So we\'re
looking at number. We\'re looking at time to deviation correct, and then
we\'re looking at a separate number, whether it\'s a repeat deviation.

MR. MIGLIACCIO: All right. My concern about deviations is deviations can
be cultural. Some of our facilities write very detailed SOPs. So any
deviation from that is a deviation that\'s reported, although at another
site with a much more general write-up, perfectly acceptable write-up
SOP, it wouldn\'t be a deviation. So it\'s cultural.

So we have to normalize for those cultural differences. The same thing
with field alerts. Many field alerts for an OOS will be closed out as
not having been an issue after it\'s fully investigated. So using
numbers, I\'m a little concerned about just using numbers.

DR. NICKERSON: A couple of comments. First of all, deviations is the
trickiest part of the whole survey just because of this. There are
different parameters in the manufacturing processes that will identify
something as a deviation or not.

The way we deal with this, there are a couple of things. One, we look
for whether it\'s recurring deviation by your own definition or a new
deviation.

Second is when we do our analysis across all of the firms or all of the
facilities, we use something called fixed effects, and the idea is to,
in essence, take out the intercept, if you will. That takes out the \--
it adjusts for the different width of these SOPs. What we look for is
the rate of change. Do we see a decline over time in all of these
parameters? And that\'s the key thing we\'re looking for in deviations.

I\'d also point out that in terms of regulatory performance, we also
look at supplements, and we\'re collecting information on how costly it
is to firms to assemble the information, file the supplements, and what
is the success in filing those supplements in terms of timing, but also
approval rates.

So that\'s another dimension of regulatory performance that Jeff hadn\'t
mentioned.

CHAIRPERSON BOEHLERT: Ken and then G.K.

DR. MORRIS: Actually two things. One is that there actually is a
business school person. Granted it\'s not much of a business school.
It\'s Sloan, but you know.

DR. NICKERSON: Who\'s that?

DR. MORRIS: G.K., yeah.

DR. NICKERSON: You teach in the business school? Okay. Well, I didn\'t
see that on your Web site.

(Laughter.)

DR. MORRIS: But you\'re right. It\' snot much of a business school.

DR. NICKERSON: Yeah, right.

(Laughter.)

DR. MORRIS: I just want to make that clear, but the other question is
when you did the API, when you included the API sites in the evaluation,
were these API sites that were always associated with the innovator
company or were these independent API production sites that service more
than just one customer?

DR. MACHER: These would be independent API sites. Now, within the
innovators, they would also have some API compounds, obviously.

DR. MORRIS: Right. No, I guess that\'s my question. Did you both \--

DR. NICKERSON: Yes, the answer is both. So some of the firms have API
collocated with formulation. Some have API distinct, separate,
separately located from formulation, and then there are API firms that
are separate, and so we have all of those in our sample right now.

DR. MORRIS: And do you distinguish between them in your analysis?

DR. MACHER: Yeah, The analysis would then compare API manufacturers,
distinct API manufacturers to API manufacturers, biologic manufacturers
to biologic manufacturers, and then we could even further granulate on
the chemical firms.

We could break up the granularity of the analysis into finer increments,
and it\'s important to know that it\'s not just identifying those types,
but the management processes within those firms that will be able to
identify how they differ also or if they\'re the same.

CHAIRPERSON BOEHLERT: G.K. and then Dan.

DR. RAJU: I had two questions for either of you. One is general and the
other is more specific. So I\'ll ask the general one first.

The history as we got here was that you had experience in the
semiconductor industry and you were going to look at the
pharmaceuticals, and you\'ve reached a point where you\'ve collected the
data and you\'ve begun or you will begin to do analysis and you will
have results shortly.

Yes, we will have some results from it, but you\'ve learned something in
all of your discussions at the sites and the FDA. What was the surprise?
What did you learn qualitatively in terms of your experience at
semiconductors, which is what you\'ve done so far over the last year or
two?

What was the surprise?

DR. NICKERSON: I think what we\'ve learned is that the two projects
should add a lot of value. That\'s what we\'ve learned, and I don\'t
think there\'s one \--

(Laughter.)

DR. NICKERSON: Bayesian analysis is important, but this \--

DR. RAJU: I actually thought your project fits nicely into the Bayesian
framework. I really thought so. I\'m not sure if Jeff does, but \--

DR. NICKERSON: In fact, there are many different techniques for
analysis, and we\'re fortunate at Wash. U. to have one of the world\'s
experts in Bayesian econometrics, Sid Chip (phonetic).

DR. SINGPURWALLA: Yeah, I know him.

DR. NICKERSON: So Sid \--

DR. SINGPURWALLA: (Speaking from an unmicked location.)

(Laughter.)

DR. NICKERSON: I\'ll tell Sid he\'s rather flat and see what he says
about that.

But so there are a number of different techniques we\'re going to be
using in order to analyze the data. It depends on what the particular
question is.

DR. RAJU: Sure, okay. And then I had a second question that\'s more
specific around a couple of things you had here. You asked in the survey
for people who talked about between one to five compounds, and you said
that was somewhat flexible. Is that a good idea for somebody like that
to be flexible if that\'s the basis for you to discriminate and evaluate
performance?

DR. MACHER: I actually don\'t know if I said flexible. If I did say
flexible, I was in error. Okay? So here\'s the idea.

We\'re giving a survey, an Internet-based survey which is going to take
anywhere from two to three weeks to each manufacturing facility. These
manufacturing firms are taxed in terms of what they can provide us. So
we want to make it as easy as we can for them.

For instance, we learned about the generics yesterday. They manufacture
hundreds of compounds. So, in fact, do contract manufacturers. We can\'t
ask them to input information on 100 different compounds. So we have to
be specific in asking them to do their top five, the five that they deem
the most important in the facility.

Almost every facility has given us five compounds per facility. Now,
there are some facilities that don\'t operate. They\'re single compound
focused, but that tends to be the minority.

In terms of flexibility, we\'re asking them to give us those top five
that they deem most important in terms of whether our results would
change the way they go about doing business, whether that would change
the way that they manage organize and implement technology.

So I don\'t know if I\'ve answered your question.

DR. NICKERSON: So let me add on to this. There\'s a sample selection
issue, and that\'s your question.

DR. RAJU: Yeah. It\'s not really that you\'re asking for five, but I
haven\'t heard how you\'re asking them to decide on those.

DR. NICKERSON: So let me tell you what the parameters are. We have a set
of parameters we asked them. We\'re looking for compounds that are at
least two years old, but were introduced in less than ten years.

We asked them for those compounds that are materially significant to
them, where that material significance could be volume or revenue.

We also have a number of characteristics about the processes in terms of
when they\'re introduced, how much total cumulative production has
occurred so that in our analyses we can fully characterize the sample
selection that\'s involved.

So we do have these rather strict guidelines. With just about every
facility we\'ve had a discussion. So we\'re pretty comfortable in
knowing what they\'ve selected versus what they haven\'t selected. So we
have a pretty good idea of the full scope.

Obviously a compound that\'s been out there for 20 years, you\'re not
going to see a lot of improvement in any of the production performance
measures that we\'re looking for, and we\'re not going to look at those
compounds.

It makes no sense to look at a compound that just came out last year
because we don\'t have enough accumulated history. So that\'s the sample
selection that we\'ve decided on, and we do know what the parameters are
pretty well.

DR. GOLD: Is it my turn now?

CHAIRPERSON BOEHLERT: Dan, it\'s your turn.

DR. GOLD: Thank you very much.

I have two questions. Number one \--

DR. NICKERSON: Do you teach at a business school, too?

DR. GOLD: No.

DR. NICKERSON: Okay, sure.

DR. GOLD: No. In fact, I\'ve never even gone to a business school. Is
that beneficial for me?

(Laughter.)

DR. NICKERSON: I don\'t know. We have some programs that I could
interest you in perhaps.

(Laughter.)

DR. GOLD: Deviations are looked at differently by different companies.
Now, you talked about deviations as a general category. Have you defined
deviations for these various companies in a way that enables you to say,
\"Yes, I am going to be able to judge or look at the deviations at A, B,
C, D and E companies in a meaningful way so that I can really understand
how they\'re handling the same deviations differently\"?

DR. NICKERSON: An excellent question. As I mentioned before, deviations
is the toughest part of this, in part because as you mentioned and as
Gerry mentioned firms and even plans within firms will define deviations
differently.

DR. GOLD: Of course.

DR. NICKERSON: Right. So what we\'ve done is we\'ve provided standard
deviation \-- standard definitions on different classes of deviations to
all of the participants, and we\'ve asked them to define their
deviations in accordance with our definitions.

That said, we still expect there to be plant specific differences in
these measures. So the best we can do from the statistical perspective
is to put in what I call a fixed effect. That is, identify that there\'s
a different plant and that, in fact, they may have different definitions
or different thresholds, but then look at the rates of change over time
of the different classes of deviations and the amount of resources
allocated to how you respond to those deviations and compare that to the
way they\'re organized to manage the deviations.

As you probably know, in some facilities the group that identifies the
deviation manages its resolution. In other plants, there\'s a
cross-functional team.

DR. GOLD: Yes.

DR. NICKERSON: In other plants still it gets shoved over to one group
who is supposed to deal with it.

So we believe that we can analyze the different ways in which the firm
is organized to handle deviation and assess the rates of change of the
different parameters we\'re measuring.

DR. GOLD: Yes. There are some firms that include major deviations as
well as minor deviations as part of their deviations list. Are you
segregating these into just the major deviations?

DR. NICKERSON: Largely to the major deviations, yes.

DR. GOLD: Yes, okay. A second item. Another apsect, very significant
aspect of management, facility management, is change control. Now, you
have not mentioned at all the issue of change control and the monitoring
of change control techniques and application of change control and the
drive that change control may have on supplements, on validation and
revalidation and so on.

Are you neglecting that entirely?

DR. NICKERSON: Excellent question. The answer is, no, we\'re not
neglecting it entirely. In the survey, it is hard to give you the full
survey because it\'s so large. In the survey, we pay attention to where
certain decisions are made in the organization. So we know if decisions
are made at the low level, two levels up, three levels up.

And we also look at where conflicts are resolved when there are
conflicts between and among different entities within the manufacturing
facility, and those questions we believe get at basically the issue
you\'re describing.

CHAIRPERSON BOEHLERT: Garnet.

DR. GOLD: Yes, all right. The final question I have is related to, if I
may, API facilities. It is reported \-- I don\'t know whether this is
actually the case \-- but is reported that approximately 80 percent of
the APIs that are used in the U.S. for dosage forms originate from
overseas, and a lot of them are from independent API producers.

What percentage of the API facilities that you\'ve included in your
study are independent API producers and from what range of countries are
you going to be obtaining the data from?

Can you just give us an idea?

DR. NICKERSON: Sure, I can give you an idea. We have maybe three or four
independents in Europe, and we have another four from the United States.
Those are independent API producers.

DR. GOLD: None from Asia?

DR. NICKERSON: In our study we have only focused on Europe and the
United States, in part, because in order to get the study going, we felt
it was important not to take any money from either the FDA or from
industry. The net result is we applied to a number of academic centers
at Georgetown and Washington University.

Well, fortunately we were able to get some money, but not enough to
include either India or China in our study. If we had a larger budget,
we would more than happily include them in the study, but it was just
not economically feasible to do so.

DR. GOLD: But even in Europe there are a very large number of API
producers, independent API producers, including four which seem to me to
be a rather modest number.

DR. MACHER: Well, the participation is voluntary. We have done our best
job of marketing this as best we can, and there are only certain, I
guess \-- so many ways in which we can go forward.

I guess the other alternative is to do nothing and not do the study at
all. And what I\'ll also add is this is just the first phase. The second
phase and subsequent phases will add to the end.

But you know, we can\'t swallow the cow. We need to sort of take a
little bit off as we go.

DR. NICKERSON: The other thing to realize is you asked specifically for
independent API manufacturers. We have a much larger number of API
manufacturers that are in larger firms in Europe. Some of them also sell
out into the market. So, in fact, we may have more apparent API
manufacturers in Europe than the four independents.

DR. GOLD: But the ones you\'re talking about, the larger ones in Europe,
are they affiliated with U.S. or multi-national firms?

DR. NICKERSON: Some are and some aren\'t.

DR. GOLD: Some are and some aren\'t. One of my major concerns are the
ones that are truly independent and not very large and not controlled by
multi-nationals.

DR. NICKERSON: If you can give us a few more names to participate,
we\'ll include them.

DR. MACHER: And actually since I am in charge of marketing right now,
for any of you pharmaceutical manufacturers that aren\'t participating,
please come see me.

DR. GOLD: Yes. Well, thank you very much.

CHAIRPERSON BOEHLERT: Okay. Garnet, your turn.

DR. PECK: Yes. Within the 21 firms, do you have any sampling of the
so-called contract manufacturers, in particular, non-prescription drug
manufacturers?

A lot of these are very large volume operations. I just wonder if there
is a sample.

DR. MACHER: Yes. Yes, we do, but we\'re trying to avoid some firms that,
for instance, make products like skin lotions that are still under some
FDA approval. We\'re looking for products that have a pharmacokinetic
benefit. Things like toothpaste or skin lotion we\'re avoiding.

We do have contract manufacturers in the sample that do prescription and
non-prescription drugs within the U.S. and within Europe.

DR. PECK: It\'s the solid dosage form that I was specifically \--

DR. MACHER: Solid dosage, yes.

DR. PECK: \-- questioning.

DR. MACHER: Yes.CHAIRPERSON BOEHLERT: Others? Nozer?

DR. SINGPURWALLA: Well, as you know, I don\'t teach in a business
school, but some of my weaker students have received positions in
business schools.

(Laughter.)

DR. SINGPURWALLA: Now, I\'m not going to criticize what you have done,
but I\'m going to make a comment. I think the parallel between
semiconductor manufacturing and drug manufacturing is not quite the same
because a semiconductor doesn\'t cause damage to an individual. It may,
but most semiconductors are like little light bulbs. You can throw them
away.

What I would like to suggest is there are some manufacturing functions
which involve great risks, and you may want to look at those. Now, I
don\'t know whether you\'ll have access to them or not, but the Sandia
labs, for example, does manufacture components for nuclear devices. They
carry great risks, and they have come up with a system for manufacturing
under highly risky conditions for risky components.

You may want to look at that, and there may be a better parallel between
drug manufacturing and what they are manufacturing. So what I\'m
suggesting is you may want to look at manufacturing activities that
involve risky elements both in terms of handling the elements and also
in terms of the consequences of bad manufacture.

That\'s just a suggestion, and it\'s not a criticism.

DR. MACHER: I\'m actually going to address your concern. The drug
products that pharmaceutical manufacturers make are safe. They are.
There\'s no question, and I think you\'re misunderstanding what we\'re
doing.

We\'re looking at the process by which drugs are manufactured, given
that there\'s a level of safety that already exceeds any expectation,
all expectations. What we\'re trying to do is improve the efficiency of
the existing manufacturing process. Okay? That\'s what we\'re trying to
do. We\'re trying to make it so firms can improve their yields and their
cycle time, and so that they can solve problems more quickly.

That\'s our objective. That\'s our goal. There are a lot of parallels
between semiconductor manufacturing and pharmaceutical manufacturing,
and you and I maybe can talk on flying about those. I\'ve been in 30
semiconductor manufacturing facilities and about 15 to 20 pharmaceutical
facilities. So I think I have a pretty good idea of the similarities,
and they are there. They are there.

The products that they\'re making, yes, are different. The manufacturing
processes, the way you organize, the way you manage, and the technology
that\'s put in place have corollaries.

DR. SINGPURWALLA: I think you\'re becoming on the defensive, and I\'m
glad you are because that gives me an opportunity to come back.

(Laughter.)

DR. SINGPURWALLA: All I\'m suggesting is look also elsewhere, and I said
I\'m not criticizing what you have done. All I\'m saying is maybe there
are other avenues that may give you more insights and more information
than what you have been doing.

So maybe you misunderstood my intent.

DR. NICKERSON: That\'s fine. Thank you.

CHAIRPERSON BOEHLERT: Any other questions or comments from committee
members?

Ajaz?

DR. HUSSAIN: I think I didn\'t clearly understand the coverage or how
many generic forms would be part of this because my concern is simply
that if we don\'t have, for example, API manufacturers from Asia and so
forth, the survey might not reflect the generic industry, and that\'s a
concern also.

DR. NICKERSON: It certainly is a concern because at this point we don\'t
have any of the Asian manufacturers.

DR. HUSSAIN: But how many generic manufacturers are in the product
manufacturers?

DR. NICKERSON: I don\'t have an exact number for you because there are
some firms that are strictly generic manufacturers, but there are others
that have a little of both, and so I just don\'t have that exact number
for you. Okay?

Clearly, there will be some sample selection issues. No doubt about it.
If we go back to the semiconductor industry, we studied a total of 36
manufacturing plants which if you looked at the number of the firms
involved, the firms represented about 80 percent of the industry. The
plants didn\'t but the firms did.

And I don\'t think we have firms that represent 80 percent of the
industry. We still have firms that represent a substantial share of the
industry.

So there is this tradeoff in terms of getting all of the little firms,
and we\'re certainly under sampling on the little firms mainly because
they\'re the ones that have the fewest resources to contribute.

To fill a survey, just for people to get a sense of this, it takes two
to three person-weeks, which is very costly for the firm, and we\'re
very sensitive to that. We have been ecstatic at the participation we
have received so far.

I\'d love to have more of the smaller firms, but as long as we
understand what the sample selection is, as G.K. was pointing out, then
we can interpret the results accordingly.

CHAIRPERSON BOEHLERT: Ken.

DR. MORRIS: Yeah, just a quick comment. Perhaps the way forward is
because you\'re at the stage of getting the Phase 1 results, maybe after
that it will facilitate expanding it to cover some of these concerns,
but having worked with the same monetary constraints, I know you can\'t
swallow the cow, although certainly we\'ll try.

So it may be the best way forward is to categorize this the same way
we\'re talking about examples that we need. So if we lump this, if you
will, not to do any violence to the study\'s benefits, but if we lump
this in the same category as creating examples, then the first stage may
be just to disseminate the results of Phase 1 and then hopefully resolve
the issues of recruiting as well, some more funding so that you can do
this without having to fly coach.

DR. NICKERSON: That\'s exactly right. We have been flying coach and
staying in coach also. Once we\'re done hopefully the value \-- Howard
Johnson\'s. No \-- once the study is done, hopefully it will demonstrate
the value that we believe is in the study, and as the manufacturers
perceived the value, then perhaps there will be other people signing up,
and perhaps once we have demonstrated our ability to maintain
confidentiality both with the FDA with respect to the FDA data \-- I\'ll
point this way because the industry reps. are over here \-- with respect
to the industry data, then that will also provide a little more
legitimacy, and that may allow us to advance to a second stage.

CHAIRPERSON BOEHLERT: Any other questions or comments from committee
members, FDA?

(No response.)

CHAIRPERSON BOEHLERT: If not, thank you, gentlemen.

DR. NICKERSON: Thank you.

(Applause.)

CHAIRPERSON BOEHLERT: We are slightly ahead of schedule, more than
slightly ahead of schedule. What I propose is we take our break now for
15 \-- well, you don\'t have to break Nozer.

(Laughter.)

DR. SINGPURWALLA: But then you won\'t break when I want to.

CHAIRPERSON BOEHLERT: Well, that is a problem. We\'ll allow you an
individual absence.

DR. HUSSAIN: Madam.

CHAIRPERSON BOEHLERT: Yes, Ajaz.

DR. HUSSAIN: We probably are behind.

CHAIRPERSON BOEHLERT: Oh, we\'re behind?

DR. HUSSAIN: Yes.

CHAIRPERSON BOEHLERT: Oh, we\'ve got one more speaker.

DR. HUSSAIN: Well, the next topic was supposed to have started.

CHAIRPERSON BOEHLERT: Okay. I\'m sorry. We\'re not going to break.
Nozer, you\'re correct. I looked at it rapidly. Yeah, I\'ve been away
too much. I\'m thinking about vacation on Friday.

But okay. Our next speaker is Gregg Claycamp. Sorry about that.

DR. CLAYCAMP: That\'s all right.

Good morning, ladies and gentlemen. My father taught in a business
school, and actually started at the Sloan School, and I mention that in
that \-- let me see if I can keep this started \-- that risk analysis
borrows a lot from many disciplines, including business management,
economics and statistics and engineering, et cetera.

And, indeed, my father is a Ph.D. in economics and had gone on to advise
corporate boards basically in the business strategic management, risk
management area, and even as short as a year ago, we were discussing how
do we advise in my case on risk end points and in his case on market
penetration and percent share and so forth, and suddenly the light bulbs
went off and we realized after all of this time our careers had merged
and we do exactly the same thing. We just had a different lexicon.

And so just setting that, I think my role in these talks here is to set
a philosophical background for what our team has been working on, and so
I just thought I\'d start with that little personal observation.

Risk is an intuitive and familiar concept to everyone. If I polled each
one of you, you would have your own \-- I seem to be on auto pilot here
\-- if I polled each one of you, you would have an idea of what risk
meant to you and what it meant to the organizations you work in, and
they might differ. At least on first blush, they might differ from one
definition to the next, and they\'re probably all correct in that we can
tease out the elements of risk in everyone\'s definition, although they
may seem a bit different.

And the trick is when you have such a conceptual basis, rather than
something that\'s more concrete and exacting to everyone, it ends up
being a difficult challenge for a large and complex organization to
settle on one definition of what risk means to them.

And that has been a large part of this process, is getting everybody at
the table to say, \"Okay. What do we think is risk in these terms?\"

Well, risk assessment, which you\'ll hear about a lot in this process
\-- my show is on auto pilot here, I think. Okay. It\'s still flying on
its own.

Okay. Burt risk assessment is not a single process, but a \-- okay.
Borrowing from the National Research Council, risk assessment is not a
single process itself, but it\'s just really a systematic approach to
organizing and analyzing scientific knowledge and information, and
moreover, this information is directed at supporting a risk decision.

Risk management can be viewed as a systematic process for
identification, assessment, control and communications of risks to life
property or other things of value, including you may actually want to
consider the risk of losing a view if there\'s construction across a bay
from your summer place or something. I mean, anything can be set in that
framework, things of value.

As a broad concept, we have as I\'ve stated many possible meetings,
depending on the individual or the organization or even parts of the
organization. This effort is complex in scope and requires thinking
about risk in many different contextual levels, and I believe that we
can do that without departure from our overall mission to reduce,
manage, and control risk to public health.

So that\'s where I\'m starting from, and now I\'ll try to paint a little
broad brush stroke picture of where these processes are in thinking of
hierarchical levels of risk management.

As used here, we\'ll refer to high level as the broadly based general
and principal driven approaches. These are the ones that are more
qualitative and are based on the principals that are shared among all
fields of risk management.

The low level approaches refer to very specific modeling and discipline
driven approaches. You can view this as a hierarchy in processes and
systems that high levels can generate a number of different low level
approaches and utilize those approaches in an organizational problem of
dealing with many types of risks, many types of hazards, et cetera.

Risk ranking and filtering that we\'ll talk about here is a high level
approach or process, if you wish. So, for example, in looking at the
pharmaceutical area, in particular, I borrowed this from an FDA report
on managing risks for medical product use just showing us that there are
known side effects that come out in the pre-market review of the safety
and efficacy of the drug or the device.

There\'s actual medication or device errors that occur once there\'s
practice so that the missed medication errors in hospital settings, for
example, and device errors, and there\'s this area called product
defects. The product defects are one area tha this particular effort has
been focused on.

There\'s also these unexpected consequences, and that is so that we
can\'t be all knowing, and essentially it has been called Phase 4. We
see things happen when there\'s larger populations using pharmaceutical
products, that they were unanticipated consequences.

Well, the drug quality in one view of this is that drug quality is
really focused on those product defects, and the public health side is
what we\'re trying to link up with and improve that linkage in this
initiative.

So quality systems, one way to view that is that it\'s really focused on
decreasing the likelihood that you\'ll experience probability defects
and also will decrease the chances that given that some would occur
anyway even at a low risk, it reduces the chances that those will ever
make it to the patient.

But there\'s a variety of risk tools that support quality systems
directly, and these are, you know, ongoing and lots of discussions
between the ICH Q8 and Q9 efforts, and these tools that I\'ve listed
here are things such as failure mode and effects analysis, FMEA, and
fault tree analysis, hazard analysis and critical control points,
probabilistic risk assessment, root cause analysis, and many others and
many others that are being invented as we speak that typically are
combinations of processes and models that have already been developed.
They are just new hybrids and slightly changed from the historical
models.

And these tools are very helpful for focusing on assessing and managing
risk, given a specific product or product class. It\'s when you can get
down to the low level detail levels that you want tools that can address
very specifically these issues.

On the other hand, at a high level, the FDA and organizations,
manufacturing organizations, are also faced with dealing with a lot of
different issues and yet hopefully bringing them into some
prioritization in their work planning for their business or regulatory
frame.

So, in other words, you\'re trying to put on the same table all of the
apples and oranges and mix that with the beans and the potatoes and
everything else. We deal with a lot of complex issues and a lot of
issues that have different health endpoints. They have different hazards
and so forth.

So how do we make sense of that at the high level?

And so one way to view this is that you have a series of these on the
pharmaceutical side, a series of these models shown in the previous
slide and the tools that might be used to do the high level
prioritization among many different types of products are things such as
hierarchical holographic modeling, which has been written a lot about by
Yackov Haimes, a systems engineer. It comes from engineering.

Risk ranking and filtering is also one that he spent a lot of time on
and that has a history in aerospace, as well as manufacturing processes.

And risk matrices, and I put the ellipsis at the bottom of that to
indicate that there\'s many high level processes that are being
discussed in the risk management side.

Okay. So questions will change and tools will change with the level of
analysis. At the low level our risk questions might focus on identifying
and characterizing risk to drug quality for a specific product or within
perhaps a specific product glass.

And we can hopefully in many cases start to see quantitative measures
and quantitative analyses, and these analyses will be driven by those.

At the high level risk questions focus on how things compare with each
other. Risk ranking is really you can think of it as a series of
decisions to start to prioritize or rank within a given class and then
across classes as well. And these are essentially tools that are
customized for each application, and so this is a little bit different
and relies on committees willing to be creative and put their best
thinking forward to borrow from every applicable area they can think of
and customize an approach.

And it\'s really driven by principles more than calculational endpoints.
Okay. So just as one low level example, I took a slide that I think many
of you have seen before, and I take a fault tree analysis, and that\'s
kind of a favorite of mine because I come from a radiological health
engineering background, and this was a favorite of getting licensing for
nuclear power, was to do very highly quantitative fault tree analysis,
which is starting with we\'ve got a failure at the top.

If we take a light bulb failing and just for a second think about when
that light bulb fails what goes through your mind. Well, if love
analysis like some of us do, a whole lot of things go off, like, well,
there\'s no electricity. There\'s a thought, and the glass might be
broken. The filament might be broken. There might be a vacuum leak, and
so that first gate just below bulb fails is my PowerPoint representation
of an or gate. It\'s either/or on those first four boxes there.

But you can take no electricity on the left side of the slide. You can
take that back another step and say, well, you might have no electricity
because either the power plant failed or the power line failed or the
connector was corroded, et cetera.

And you can take that even farther down another step. The power line
fails and wind broke the line or a tree breaks the line. Just an old
tree falls on it, et cetera.

Well, this shows how complicated right away a very simple failure can
become, and this is quite minimal to probabilistic modeling. It has been
used, again, in safety analysis many times, and there\'s one challenge,
and that\'s that if you take even a simple manufacturing line and try to
do this, you\'ll quickly find that you\'ve got an enormously complicated
problem at the first glance. You can break down every piece of equipment
into its various faults, and the sources of those faults, and right away
you\'re into a very complicated subject.

And this has been done for things like process chemical manufacturing
where there are significant safety issues in terms of, you know, if you
mix a couple of chemicals you get a very unwanted reaction from toxic
gas release to explosions, et cetera. And so there\'s very elaborate
modeling on the chemical manufacturing side to try to do risk
projections for faults in the manufacturing.

Well, some of these low level tools, they have another hazard that we
always need to think about in these contexts, and that\'s the
philosophical or communication type side of these. When you develop a
highly quantitative risk model which may be built on initial parameter
estimates, whether they\'re flat priors or Jeffrey priors (phonetic) or
whatever, they\'re put together, and they come up with some risk
estimate, and they come up with some uncertainty at the end of that.

That itself may communicate to the audience that the audience may hear
that you have a lot more precision and knowledge about your model than
you actually do. You have to be very careful that on the quantitative
side, it starts to look more impressive than the data that may be
supporting it.

So we\'re very cognizant of that, and we work very carefully to avoid
looking like we know more quantitatively about a system than we actually
do.

Well, that\'s one possible hazard in a fault tree. The other problem is
that you start with that fault, and you may miss the whole picture. You
can go down this fault path, and you miss the whole picture, and the
example I like to use does come from the radiation field, and that\'s
the Brown\'s Ferry nuclear accident in the mid-1970s roughly.

It had, of course, in its licensing process, had very elaborate fault
trees and used a lot of reliability analysis in its history in building.
But what it didn\'t capture is that a couple of plumbers insulating some
duct work would check for a breeze and check that there\'s penetration
of this duct work with the lighted candle, which caught some foam
insulation on fire. The fire spread because there was a breeze going
through the penetration, and it turned out redundant safety system
cabling, and so everything went wrong, and it came very close to
meltdown status.

And you know, that wasn\'t in the fault tree that these would share
penetrations and so forth. So we have to be aware that in any type of
modeling that we do at the low and high level of all sorts of
ramifications of what it\'s communicating, what it can really tell us,
and be very aware of the uncertainty in our modeling itself. What about
other models and other views of the world?

So why use high level systems methods in risk management? Well, as I
mentioned, low level approaches are, indeed, elegant and capture many
details, but they miss interactions and relevance across systems.
Complex quantitative models, as I mentioned, may convey a level of
precision and understanding about a system that\'s unjustified.
Different levels of understanding and quantification may exist for each
subcomponent, but a high level seeks optimal use of diverse kinds of
information to inform risk decisions.

So quantitative risk assessment models are only one thing on the risk
manager\'s tables. There\'s lots of other inputs as we all know going
from the values of the stakeholders, the public, the political issues,
the legal issues, you name it. It\'s all on the table, and these are
only one of the issues.

High level models really have their source and systems approaches in
thinking, and we can have the chicken and the egg discussion on whose
field, business, engineering or whoever started this all, but
nevertheless, it\'s all shared at this point and is useful for our work.

The risk management of complex systems is multi-objective. It has got
multiple decision makers. It\'s hierarchical. There\'s hierarchies and
there\'s lots of overlap, and sometimes there\'s conflicting objectives
and endpoints.

And generally these exceed our human capacity to put everything in a
simple model. So to just go over again kind of the broad brush stroke
philosophy of where we are with this, we look at using the one I
mentioned, hierarchical holographic modeling, which refers to the fact
that it\'s multi-dimensional and it\'s hierarchical.

And basically this slide and the next couple show that it just starts
with an organization of information. Recall I said risk analysis, risk
management is a systematic organization of the information, and so
that\'s kind of the common sense issue. What are the things that we
think are related to risk and given that we can identify the risk
endpoints that are in our interest frame.

And so those may fall within areas of health, compliance, resource,
social, political, geopolitical. You can go on and on and just put
everything on the page.

So how do you make sense of that in high level approaches? We\'ll talk
about one here, which is risk ranking and filtering, and that\'s to
drill down beyond that highest level and start to flesh out a model with
what factors we think may be important in predicting risk.

And those may fall into classes of product and process and whatever that
are at a more detailed level than in our initial chart.

There may be a variety of endpoints where we can start to get closer to
that low level and maybe eve envision having some quantitative models in
form what impact does loss of sterility have on risk, and you know,
that\'s our pipedream thinking for risk analysts is, gee, when can we
get to this and get some real quantitative tools going, and that\'s a
ways off in many of our areas right now.

You systematically develop the low level details. So, for example, you
could break down into what are the things going on by process that might
affect sterility, and actually get into the fault tree analysis and
failure modes and effects analysis that are at the low level.

So low level analysis can be quantitative, relying on these other tools,
but data gaps may need to be filled with estimates from expert
solicitation, and there\'s a lot of intelligence out there that is
accumulated experience of doing this for years, and how can we tap that
information because it might not be existing in a database or in a
quantitative tool? How can we tap that and use it to inform our risk
based decision making, and that\'s where expert elicitation comes in.
It\'s tapping the mental models that are already in existence.

Sometimes only qualitative information is available for specific
processes. So perhaps we might have a qualitative scale such as low,
medium, and high, and I just showed one example of severity scale and a
probability scale because in many of the high level definitions of risk,
risk will be placed in terms of probability of occurrence and the
severity of occurrence. And so that\'s just an example of what that kind
of qualitative scoring might look like.

Now, of course, this can mature over the years, and very low could
eventually defined as one in a million and low as one in ten to the
fifth and whatever. You can think of this as a beginning, and it can
improve as more information comes to the problem.

And this just follows up on it that there is some reciprocity that in
this concept of combinations of severity and probability, that you may
have something that is of high occurrence probability and lower
severity, and that may fall in the same range as something that has the
inverse, the high severity and lower occurrence probability.

Eventually, risk ranking and filtering will take whatever information
that can be identified and looked at as helpful in informing the goal of
ranking our risks and pooling those in some form, usually very simple
mathematical processes to average and weight can be used, and try to
come up with some ranking by combinations of the data that we have and
the expert elicitation data, et cetera.

Now, the question is what is the filter. Well, you know, these are not
classical, empirically driven models which have random sampling and so
forth. We just don\'t have the information and the ability to set that
kind of thing up.

So your best intentions to try to capture models of risk in a given
process or given product and so forth, you may come out that everything
ranks the same at the end, and so filter is a nice way to say you can go
back and say we\'re going to put a policy on that that can, for one
thing, expand the scale and deal with those issues of do we have enough
range to be able to rank in the first place, and it can also be that the
filter is the policy driven aspect, and that\'s \-- in other words, if
we have resources that can only cover some percent of all of the things
that we\'d see as being work that needs to be done, you know, what would
that top n percent look like, or X percent across all organizational
units

And these are very difficult policy issues sometimes because the worst n
could be looked at as across the entire organization or across units of
organizations. Filters may have risk, resource or another basis, and
they may have differential effects on the final ranking. So those may
need to be compared.

So, for example, if you had some kind of risk score and all of these
organizational units just labeled A, B, C through S, you might have a
natural scoring that fell out of that risk ranking, and filtering, and
you might use a risk based filter that says, well, if anybody exceeds
this overall risk score of whatever, then that organizational unit is
prioritized, and so they all did in this case the way I\'ve drawn that
line arbitrarily there.

The other way might be to take a more Perito (phonetic) type approach
and say we\'re going to get the most of the risk score in that top level
A through H, or whatever it is, and have it driven by the resources of
available to do that.

So those are the types of questions that the risk ranking and filtering
leads to once you actually finally get the ranking out of model.

Where does it fit in the overall cycle of risk analysis or risk
management in some writings? Well, you start somewhere, of course, and
our belief is that starting to look at the potential for risk management
models is better than having nothing at all, and it\'s better than
relying on purely historical information locked in people\'s heads. We
want to tease that out into something that\'s workable for now and the
future.

Start with assessments, databases. You know, come up with some
multi-factorial risk model which is on the assessment side, and that
then is information that goes into the risk management side.

And as I mentioned, the not only risk ranking and filtering goes into
prioritizing work, but other factors are always at the risk management
table.

Data sources, including quality systems and manufacturing science, in my
view they really inform the risk modeling at that side and, therefore
inform the risk ranking and filtering, but they are really at the heart
of the detailed information, and this is all as shown as a cycle that
goes on. It\'s iterative and hopefully improves with new information in
each cycle.

Well, I hope I\'ve conveyed that on the high level thinking and the
philosophical thinking, that we\'re at a challenging area where we do
get some real quantitative information here and there, and we have a lot
of qualitative information from experts who have been doing this work,
who have in their head a model that is working perhaps. And it\'s as
Bernstein said, that risk management decision making are about where we
confront probabilities, and it\'s a balance between the measurement and
the gut because risk management is a judgment, and it uses any kind of
information to make the best judgment possible.

Okay. Thanks.

(Applause.)

CHAIRPERSON BOEHLERT: Thank you, Gregg.

And I think we have one more speaker before we take a break. We\'re
going to hold questions until we\'ve had the four speakers on this
topic.

MR. HOROWITZ: Let\'s see if I have more luck with this. Okay. So far so
good.

Okay. What I\'m going to try to do is take up where Gregg left off and
transition to discussing how some of the concepts that Greg discussed
that have been used in other contexts relate to our specific question at
hand, which is: how can we be sure we get the most bang for our buck
with GMP inspections?

Now, that question is even broader than what I\'m going to be focusing
on and what we\'ll be focusing on. We\'re not going to be discussing all
of the different aspects of the GMP program. We\'re not going to be
discussing how to make the GMP program or GMPs themselves more risk
based.

But what we\'re going to be focusing on is putting aside those other
questions now with the program that we have, with GMP regulations and
thinking the way it is currently now. How should we best allocate our
very limited inspectional resources? Where should we go first so that we
don\'t run out of GMP inspectional oversight resources before we get to
some of the most important sites to look at.

So let me go back to the start of the GMP initiative. In almost two
years go, in August of 2002, which I look back at the concept paper
periodically, and I\'m sort of surprised that there are as many things
in there that are sort of predictive of where we ended up because I
think at the time a lot of people viewed those as pipe dreams and just
words that FDA was saying, but I think we have taken some important
strides.

And this model, our effort, we\'re really just getting off the ground on
it, is an effort to pout into practice some of those words that we put
forth in August of 2002.

One of the reasons we said we were undertaking the initiative, and these
were three quotations here is that we wanted to evaluate the currency of
our drug quality programs given that it had been 25 years since anyone
had closely looked at GMPs and drug quality closely as we are now. But
we wanted to, among other things, look at determining whether FDA
resources are being used most effectively and efficiently to address the
most significant public health risks, and we also said that in order to
provide the most effective public health protection, we should match the
level of effort against the magnitude of the risk.

Now, that\'s much broader than where you go for your inspections, of
course, but we also said that resource limitations prevent uniformly
intensive coverage of all pharmaceutical products and production.
Although the agency has been implementing risk-based programs in some
sense, a more systematic, rigorous risk-based approach will be
developed.

Well, what we\'re talking about today, I think is just the first steps
towards that end. This is a slide that amazes me, and it\'s the first
time that I\'ve presented it in public because I just couldn\'t believe
the data, and I\'ve presented the blue line before because there is a
lot of evidence that our resources available to complete systems based
inspections have declined significantly over the years.

Now, some of that decline has to do with resources being put into
pre-approval inspections which have a GMP component to them, but that
partly explains some of the decline, not entirely because it is quite
precipitous, and I think the trend is likely to continue even though
we\'ve tried to stave off some of the decreases in the last few years.

But this green line is quite extraordinary because it shows tremendous
growth in the number of domestic registered firms, and that surprised me
particularly because as this industry is globalized, I though there
would be not such a steep increase in domestic firms. I expected to see
just a steel increase in foreign firms.

And what I think this tells is something else that\'s been going on in
the industry for the few years, and that\'s more use of contract
facilities, moire outsourcing and the phenomenon which is not something
this group typically gets involved in of medical gas repackagers. A lot
of these facilities starting in the \'90s began registering with the
FDA. Many of them were engaged in this activity before, but more and
more started registering.

The more inspections we did, the more registered, and the more problems
we found, the more inspections we did, and it got to the point where
about half of our inspections were devoted to medical gas repackaging,
and this is taking medical gases from larger tanks essentially and
putting it into smaller tanks. It doesn\'t raise many of the quality
issues associated with more complex drug manufacturing.

But anyway, the point of this slide is simply that it became very clear
to us before we started this initiative that what made sense in 1980 and
in 1978 as a strategy for inspection to meet our biennial inspection
requirement no longer makes sense any longer, and we need to think about
where this is going for the future. Every inspection has to count.

So we might not have perfect data. We might not have perfect knowledge,
but we need to at least do the best we can to systematically use the
information we have to prioritize our sites for inspection.

So what I\'m going to do now is try to walk you through how we got from
our sort of vague understandings of risk to try to take some of the
consensus definitions out there of what risk is and how we use that to
develop factors and then try to organize hierarchically as Gregg
described these factors into a model that we could explain to people and
that we would use for thinking about identifying risk factors, weighting
them, and then prioritizing and ranking sites for inspection.

So let me start with risk. As Gregg pointed out, everybody has their own
definition of risk, and they all have certain value to them, and they
are all probably correct in certain contexts, but we wanted to go with a
consensus definition, and ISO and a lot of other consensus definitions
typically include two elements. They typically include the probability
of a harm\'s occurring, and if it does occur, the severity of that harm.

And so I\'m going to look now at the working definitions from Q9 to sort
of figure out what harm is and how to apply these terms. I recognize
there was a spirited discussion on Q9. It\'s still very much a work in
progress. These definitions aren\'t exactly the way we in FDA would have
done it, but I think for our purposes today they\'re illustrative of how
you might go about thinking about these issues.

All right. So if risk is about the probability and severity of harm, of
course, the key is risk to what. In other words, the key is how you
define harm, and the Q9 definitions sort of walk you through several
definitions to actually figure out what harm in the context of
pharmaceutical quality might be.

And they start out by saying harm is damage to health, including the
damage that can occur from the loss of product efficacy, safety,
quality, and availability. Well, that, of course, begs the question,
what is quality.

Yesterday we heard some discussion that Dr. Woodcock has some thoughts
on quality that I want to link to these Q9 definitions. So we\'re going
to focus on quality as the primary harm, that is, the core of the risk
we\'re looking at.

All right. So what is quality? Well, there\'s a lot of literature out
there on quality, nd it has to do with the degree to which a set of
inherent characteristics of a product, system or process fulfills
requirements. Well, that just begs the question of what are the
requirements.

The needs or expectations that are stated, generally applied, or
obligator by the patients or their surrogates, and I think we talked
yesterday about how the regulators sometimes have to stand in for the
patient to determine the needs.

So let me sort of try to combine these terms. My understanding of how
those Q9 definitions and ISO definitions fit together is that risk
quality is the probability and severity that a drug will fail to meet
the needs and expectations of the patients and their surrogates.

Okay. So what are the needs of the patients and expectations of the
surrogates? Well, that\'s what we heard yesterday that Dr. Woodcock has
given some thoughts on, that I think link up nicely to this, and she
talks about clinical performance being the key, and she said recently in
May and before that several months earlier it\'s the delivery of
efficacy and safety as described in the label derived from the clinical
trials.

But I think we all know intuitively that the needs and expectations of
the patients also include the availability of the drug, something we
should consider in our risk matrix, and sometimes price, but that is
something that consumers are more readily able to discern and are less
dependent on FDA for, I think.

Okay. So Dr. Woodcock goes on and talks about how clinical performance
is how the drug performs as described in the approved labeling, and that
it delivers the relevant attributes of the drug and the clinical
database on which the FDA approval decision was based.

So that begs the question which she answers: what are these attributes
that can serve as surrogates for clinical performance? Because these
then become the core to the risks that we\'re going to focus on.

And she identified some of the standard things that people talk about
here, and this is largely true to her slide. We can all disagree about
certain aspects, but I think we all intuitively know that there are
certain areas that are critical quality attributes, that if there is a
chance that one of those things or more of those things could be messed
up, that\'s the kind of risk quality we\'re talking about.

So then risks to pharmaceutical quality can be identified based on the
probability and the severity of an adverse impact on one or more of
those attributes. And you could explicitly include factors that mitigate
the probability and severity of those or the factors that have a
positive impact in your risk model, and we tried to do that.

Okay. So let me try to summarize graphically my conceptual thinking and
our conceptual thinking that underpins the model.

So we have the probability and the severity components here which make
up harm, and ultimately it\'s the probability and severity of the
adverse impact on quality attributes that are that harm. And so the
quality attributes are sort of the linkage between the needs and
expectations of the patient to the harm that we\'re seeking to evaluate
risks or probability of severity of adverse impacts on.

So I know that\'s a lot, but really we tried to sort of go back then and
say, all right, so how do we go about identifying risk factors with that
conceptual framework in mind, and I think this is sort of intuitive to a
lot of people. What hazards can adversely impact drug quality,
attributes, and surrogates; what processes and parameters are critical
for those quality attributes and surrogates; what factors may affect the
identified hazards and the critical parameters and processes; and other
variables that might be predictive of drug products with or without the
identified quality attributes.

And that sort of, I think, goes back to Gregg\'s hierarchical chart.
It\'s just sort of trying to organize our knowledge, thinking, and
intuition about these factors.

Okay. So we start with from the previous chart the probability or
severity of adverse impact on the quality attributes. We identify risk
factors. We, of course, have significant data limitations which prevent
us from including some of those in our model.

We want to build in certain incentives for developing process
understanding, for doing the right thing, and for adopting the kinds of
practices that are believed to be correlated with high quality
manufacturing. You take those risk factors. You quantify them. You
aggregate them. You rank, and then you start all over again.

And that\'s sort of the model that Gregg presented. Okay. Well, I\'m not
going to get into the details of the model during my presentation right
now, but we did that, and we looked at factors, and we fried to organize
them into categories.

Now, there\'s nothing special or unique about these categories. You
could slide it ten different ways, but we felt that some of these
factors are about the product. Some of them are about the process, and
some of them are about the facility.

So what we tried to do is look at the risks associated with each
manufacturing site and aggregate them and rank them against the risk
scores for the other manufacturing site. So our goal is to
systematically incorporate our current knowledge about drug quality
risks in an effort to prioritize sites for periodic systems based GMP
inspections.

Well, not surprisingly, we encountered some very significant data
limitations, and that prevented us from capturing some of the elements
that we hoped to capture this round, and I think this is a challenge
obviously.

But it\'s also a great opportunity for us to go back and look at our
data systems and start thinking about how to better capture data that
will be more useful for this activity.

We also want to create the right incentives for drug manufacturers to
adopt the practices that are correlated and connected with high
performance and high regulatory and high efficiency performance. And I
think this is an opportunity to do that as well.

Okay. So I\'m going to just go through a slide each on each of those
boxes. Remember there\'s product, process, and facility, and I\'m going
to just try to explain why we drew the lines for those three. It could
have been done other ways, but when we were thinking about this category
of factors, the product factors, we were thinking about what are the
intrinsic properties of products such as the deficiencies in quality, if
any, would have a more advertise health impact than others.

And we have some good recall data that\'s potentially useful, and among
other things, it tells how the agency classified those defects
associated with those products or dosage forms.

Another box was about the facilities and what we felt is there\'s a
group of factors that really addresses the question are some
manufacturing facilities or manufacturers in some cases more likely to
produce a product with quality problems.

Well, we think that the effectiveness of the quality systems are
predictive of that, and we believe that there is a connection between
the compliance history or the inspectional record associated with the
firm. Of course, not all violations are the same, but we do believe that
there is some predictive aspects there.

Now, interestingly, one of the elements of risk is exposure, and I think
it relates in part to severity and in part to probability, but if
something goes wrong at a facility, the impact is likely to be much
greater if the drugs are going to every household in the world or in
America than if it\'s just a local facility producing a few drugs for
the community.

So we felt that exposure of the drug products manufactured in a facility
is a risk factor that ought to be considered by the agency in
prioritizing its resources.

We also are very much looking forward to the results, preliminary and
future results, from Professors Macher and Nickerson so that we can
learn from and glean some additional factors that may be predictive of
success that relate to the particular facility.

Okay. Then another category of factors we categorized as the process
factors, and I think this is intended to answer the question are some
manufacturing processes for particular product classes more likely to go
wrong than others? Intuitively we sense that some processes are more
complex and some were simpler, but our data is very limited on this. We
didn\'t have any good quantitative data.

So our risk management experts suggested that we use expert elicitation.
Now, we\'ve started on this process internally within the agency. It\'s
our hope to expand this external experts like yourselves and make sure
that we\'re capturing the best expertise that we can get, but the Office
of Pharmaceutical Science, for example, select, hand pick their best
people to try to assist us in working on that survey. We have
participation from field investigators who have a perspective, from
compliance people, from folks across three different centers.

What we\'re trying to do is to use expert elicitation to identify risk
factors and to assist us in this approach. They\'re going to look at,
among other things, the risk of contamination or mix-ups and the risk of
the loss of the state of control for the process for particular product
classes.

There may be a potential here as well for process capability metrics and
to include other quantitative factors in the future for this model, and
we look forward to your input and others\' on how we could do that.

I think I\'ve been very candid with you that we recognize that this is a
beginning. This cake is not baked yet, but we do believe that there\'s
great opportunity for us to grow and to use this model to be more
rigorous and systematic about our approach to selecting sites for
inspection.

But inevitably the model can only be as good as the scientific or
technical assumptions and the data that are used to develop the risk
scores. We don\'t think there\'s anything magical about the processes
we\'re using.

Multiple iterations and successive revisions will be necessary and we
hope will reflect a growing knowledge base both within the agency, but
more importantly outside the agency, and it will also reflect the
extensive input from our internal, but ultimately we hope from our
external experts.

So your input on prioritizing for improvement we hope will be very
helpful, and we look forward to that.

Thank you very much.

(Applause.)

CHAIRPERSON BOEHLERT: Okay. Thank you, David.

Now I think we\'re ready for break. We will take a 15 minute break and
reconvene at 10:40.

(Whereupon, the foregoing matter went off the record at 10:26 a.m. and
went back on the record at 10:43 a.m.)

CHAIRPERSON BOEHLERT: Okay. We\'re ready to get started with the rest of
our presentations.

Before we have the first presentation, I would just like to note for the
record that we have no participants in the open hearing later this
morning. However, there was one member of the audience that submitted
some written comments. They have been distributed to the committee
members.

Our next speaker is Dr. Tran.

DR. TRAN: Thank you.

Before I get started, I just want to thank David for such a good
presentation about a model that I think Brian and I can just go back to
our desks and continue to work.

However, we\'re supposed to go into the details of this model. Before I
get into the detail, you\'ve got a pretty good overview from Gregg about
the theoretical framework on how we do risk filtering and holographic
modeling and all of that and some of the general nature of a model.

What I\'m going to do before I get into the specific is I\'m going to
talk to you a little bit about some of the applications that have been
out there using the tool risk ranking in regulatory government, U.S.
EPA, California EPA, USDA, and some of the management tools that
Department of Defense had used, as well as industry using the risk
ranking tool.

And the reason I want to talk about it a little bit is as Gregg
mentioned, we borrow and customize the existing protocol model system
out there to make it fit into what we\'re trying to do, and when I first
met David, I was working on a project of risk ranking for DOD and
that\'s how we kind of met, and that\'s how David brought me on board, I
think, to help him with looking into all of this information and put
something together that we just not create out of thin air, but use
existing experience out there with other agencies, other industries.

So this is why this background. I\'m going to go through it very
quickly. I\'m not going to spend too much time.

At the risk of looking very academic, I\'m going to flash through some
very, very busy slides. My background is environmental health risk
assessment. I work a lot with EPA models, a lot with USDA type of
models, and DOD models relating to chemical exposure. So a lot of this
background is chemical oriented, and given that you are in the
pharmaceutical industry, chemical should be something very familiar.

This busy slide is just to let you know that EPA, the European Chemical
Bureau, Health Canada have gone through and developed a variety of risk
ranking tools. These models are used to prioritize chemical substances.
We have thousands and thousands of industrial chemicals out there.

These models are used to prioritize chemicals so that certain ones are
going to be regulated based on potential for harm to the public or
because of the volume that\'s being made up in the general commerce, so
on and so forth.

So there are many, many models out there to rank risk.

This model, I\'m going to flash through some more details, such as this
EPA risk minimization tool. This is a regulatory decision tool, and
before I start talking about these specific models, they have a variety
of complexity, and they typically can range from ranking based on the
pure hazard of a product. They could be based on the ranking of the
potential for exposure for the listed products, or they can be ranked
based on a combination of much of what David and Gregg talked about is
the probability of exposure or the probability of harm, combination of
the public exposure and the severity of the harm.

And this model has tremendous impact on the chemical industry. It\'s a
very basic risk decision tool. It\'s the foundation for their solid
waste management. It\'s called RCRA, Resource Conservation Recovery Act,
and it\'s essentially prioritizing the universe of industrial chemical
out there based on their persistence in the environment and will target
those for specific regulations, an impact on a tremendous amount of
industry out there.

And it is based on the framework of judgment really, and the term that
I\'m going to use a lot is \"surrogate measures.\" Surrogate measure of
exposure, surrogate measure of hazard, and surrogate measure of harm,
and in this framework what they use are chemical emissions and some key
physical chemical parameters to come up with some cutoff to prioritize
chemicals which have tremendous regulatory impact.

And this very busy slide is like an influence diagram, and it looks very
sophisticated, but it really isn\'t. If you look at those boxes \-- and
I\'m going to focus on the human health concern box which is your far
right \-- you see the score three to nine. The reason I want to show
this, you can see the scoring that we\'re going to be using. We talked
about these as weights.

Essentially this system that has been used extensively by EPA is based
on weighting human health concerns associated with chemical on a range
of three to nine, and if you see those boxes that influence those scores
are based on some surrogate of health effects, based on some very
primitive information about cancer/non-cancer health effects, and some
judgment about how to weight those effects on a scale, rankings of one,
two, three.

And on the other side, you have the human exposure potential. This
model, looking very sophisticated in this diagram, if you look really
into the detail, it\'s a very simple expert judgment based on very
limited information, as surrogate measure for exposure and surrogate
measure for hazard and roll those factors up into a score and rank.
Okay?

So this is the kind of concept that has been applied out there. The
reality of it all is they have a lot of issues, a lot of chemicals. How
do you prioritize which to target for regulation to pay attention to, to
do research, to do more testing, so on and so forth.

And these frameworks are expert judgment based with some limited
information, empirical evidence to support those judgments. And for the
most part they are qualitative, high, medium or low ranking system. This
one happens to be a semi-quantitative, ordinal scoring, one, two, three,
four, five, six, seven, eight, nine, ten.

This is another system that EPA has used. They call it facility index
system. This is to identify facility which releases that made up to the
top priority list that they should pay attention to, and they look at
the release information, then use a scoring system. How much is being
emitted into the environment as a volume, as a surrogate for potential
exposure? Those chemicals that are being emitted, what are the potential
human health hazards?

Again, the surrogate measure for those is some weighting system that are
put in, and some of the environmental persistence information, if the
chemical has a long half-life, there\'s a surrogate measure they use to
look at potential exposure.

A combination of those type of risk factors roll up into some scoring
system to prioritize facilities. So that had been done. This was done in
the \'90s, and it\'s still being used by the agency in some fashion.

And very quickly, again, there are many different systems out there, and
the complexity will go from low to high, and in this paper, Pennington
and Yu (phonetic) had summarized all of the systems out there. They\'ve
looked for chemical risk ranking, and from low to high, in Group 1
essentially what I wanted to point out is you go from a very low
complexity or model which is generic emission data to very complex Level
5, which is very complex information, very site specific risk
assessments.

So the parallel is what we\'re doing \-- number three is the scoring and
ranking \-- is middle of the road. It\'s not just volume of the
pharmaceutical products that you make, but it\'s some combination, and
we\'re not talking about a site specific risk assessment with the range
of complex risk assessment that break a point, so on and so forth. We
are about Level 3. Okay?

And, again, DOD has used this kind of approach to compare risk
predeployment. I work on a project for them in looking at some of the
chemical exposure, radiation exposure, physical hazard exposure. The
troops might be exposed if they\'re deployed to certain areas overseas,
and they can be deployed to many, many different areas all over the
world.

So we have come up with a system of prioritizing based on these risk
factors, a combination of some intelligence information and some expert
judgment on how to bend this very qualitative information into high,
medium or low as a framework to prioritize.

These tools are being used by AFMET (phonetic) to look into attachment
data and where they should deploy troops, given what risk constraint
they might have.

So as complex as those deployment situations may be, the data are
limited, and they are forced to deploy under some very quick,
straightforward risk ranking framework, to pull through that information
and come up with some quick decisions. So that\'s been done.

I\'m going to skip this. I think this is very similar to what Gregg
presented earlier. The military model that I\'ve worked with uses a
combination of severity and probability of occurrence to come up with a
ranking scheme to compare very disparate risks from chemical to
radiation, to the bridge being blown up, so on and so forth.

Again, this slide is just meant to say you look at their interpretation
of those very qualitative risk matrix of extremely high risk, from E,
the red boxes, to low, the green boxes, have very critical meaning, and
if you look at this risk level definition of the very last column that
says unit stats, we\'re essentially talking about these qualitative
terms translate to troops deficit. Fifty percent of the troops are going
to be below unit strength. So they\'re talking about translating from
this very qualitative term to something very quantitative, and this is
not based on numerical empirical data. A lot of these are done out in
the field with very limited information.

And, again, this slide is now the military in that context
semi-qualitatively defined the probability of exposure. If you see the
way they did it, they define unlikely as less than ten percent of the
troops are going to be exposed to something, to an agent, to a hazardous
situation.

Again, these scales are set up so that when they are out in the field
with the limited information they may have, they can plug these in and
come up with a ranking. Okay?

Another example that has been used, another example where risk ranking
has been applied as a decision tool is, again, this has to do with
constraint of resources. This is an industry initiative that I helped
with.

It has to do with we have a lot of industrial chemicals that are in
commerce, and there are a lot of chemicals that are used in high volume.
They\'re called high production volume. For instance, they\'re mostly
consumer products, a lot of the aliphatic alcohols, a lot of the
surfactins. We use a lot of those chemicals, and they are very low
toxic, but they have never really been tested for other endpoints, such
as reproductive development toxin, so on and so forth.

So there\'s a pressure to do those kinds of testing, but we have a lot
of those chemicals out there, a lot of products. We can\'t possibly test
for everything. We need priority setting tools. Which of those products
are we going to really actually test?

So this model is to help industry to do just that, and they are using
these. And, again, lack of information. You can\'t really go out there
and measure every single consumer product, every single chemical you
have out there, how much you\'re being exposed to. So we use a very
rough approximation of exposure.

This model is an exposure based risk ranking model to prioritize product
that should be tested for, and this model is based on frequency of how
much of a product you use, amount you use a day, percent that is
retained in the skin. In this preliminary cut of the ranking, there\'s
100 percent absorption, 100 percent retained on skin, so on and so
forth.

And as an example on one of the outputs in this model is for a chemical
type, Chemical A hypothetically. This is a real chemical, but I can\'t
keep the information. This is going to print in a hypothetical Chemical
A. These are the product categories that this chemical goes into.

So based on this scheme, we would test aftershave because given the
approximation of the surrogate of exposure, which of these products the
public are exposed to the most that would have this Chemical A.
Aftershave would be the one.

So that\'s the kind of very simple, straightforward strategy to come up
with what product you\'re going to test. So you can\'t test all of them.

I\'m going to skip the microbiological as the same idea. It\'s using
some information to bend the hazard based on the property of microbes
and score and rank.

The Ross and Sumner is a food microbe ranking system that has been
developed by the Australian authors. This is being used in Australia,
and the point here is this is another risk prioritization tool, and it
asks a series of questions, and I\'m going to just flash through a
couple of questions that this model asks the user to go through.

One is the hazard severity, and again, if you look at this chart, it\'s
again an expert based framework. The question is: how severe is this
hazard?

And the user with this model is asked to put in the weight, and these
are arbitrary weighting factors based on your expert knowledge. Okay?

Again, in these food risk ranking models, you tend to think about
consumer, and are these the acceptable populations that are going to be
exposed, and some of the susceptible populations, infants, AIDS
patients, so on and so forth. So in this model they use again a
weighting system to weight up the population that you should be
concerned about.

And, again, this is based on your knowledge, some empirical knowledge
about what percent of the population you\'re trying to protect, fall
into these categories. So this is some empirical information, plus some
judgment on how you put those weights on those percent of the
population.

And this model is a look at the process. A look at the process is like
to reduce the growth of the microbes and, again, this is arbitrary
weighting based on the expert judgment.

One of the models that is really close to what FDA is doing is the USDA
Food Safety and Inspection Service, inspector optimization system model.
This is the model they use to prioritize the inspector work force.
Again, they also have constraint, limited resources on how many
inspectors they have and how many meat and poultry processing facilities
they have to go and inspect.

And they have written this up in a report to Congress in 2001, and this
model at the time was purely a hazard based risk ranking model. What
they have come up with with this model is a food safety hazard
coefficient that\'s based on the inherent hazard of the food product,
which is meat and poultry, and it has the process of making these food
products, and they use an expert elicitation, but there is no data. If
you are working the food industry particularly, there aren\'t any data
in terms of sampling, very limited sampling data.

So in this FSIS model of prioritizing the facility risk so that they can
deploy inspector resources accordingly, they basically used three
variables. One is a species variable to reflect the inherent biological,
chemical, and physical hazard associated with the meat and poultry that
are arriving at the inspector. The data don\'t exist. Expert elicitation
is used to get at that.

The second variable that\'s a reflection of the inherent hazard is the
process variable, and again, in this process they assume normal process,
normal slaughtering plant, normal packaging plant processes.

And a third variable they put in there is the volume, very similar to
ours. They wanted to have some surrogate that would account for the
potential for the number of consumers that might be exposed should this
product going out they would be exposed to. So they use a volume, the
facilities\' size

And a little bit about the expert elicitation. Again, they don\'t have
any data on the species variable or the process variable. What they went
through is a process of elicit opinions from known experts.

And they have two different elicitations. One is on the hazard itself,
on the product or the species itself. The species are where the cows are
views. And the question that they ask here is: based on your expertise,
rank these; rank order these from one to ten. How hazardous are these?
How likely are these going to be contaminated with microbes going into
the processing plants?

And you can imagine this is a very tough elicitation because where are
these animals coming from, the geography and the season when they\'re
being brought it is going to change the answer.

So this is not an easy elicitation that they had to go through, and they
had to be really careful what expert they\'re going to choose, and they
used a combination of government, academia, and industry expert
elicitation.

And they did the second elicitation on the process, and the process is
the grinding of the beef as an example, the slaughtering process, you
know, different kinds of processes, and again, the same series of
questions were developed, series of experts were selected to elicit and
rank order these.

And so that\'s the process they went through. Their model is hazard
based with a surrogate for exposure which is the volume, and it\'s a
coefficient score at the end to rank the sites.

And their model is also evolving. There\'s also a learning and evolving
and the model is going to be improved over time. This is the latest
presentation by Elsa Murano from SSIS.

Their next step is to put in, to change, to modify to a hazard control
coefficient, and what that does is they can incorporate compliance
history into these coefficients.

So now the first phase is the apparent hazard with surrogate for volume.
The next phase, to put in the compliance history, to improve the
scoring, and to rank the sites to target inspection.

So that\'s what\'s going on out there, and there are many more out
there, and they are evolving, and everyone that is trying to use this
kind of system to work smarter.

Okay. Good. That took me five minutes. I didn\'t want to spend too much
time on that, but if you have any questions, you can ask me later on.

Okay. Now, let\'s go into CDER office compliance process. What do we do?

So having been through all of this risk ranking process with other
agencies, when I met David, I said, \"Please help us with this.\" And as
you know, you works in risk assessment. It\'s easy to talk about concept
in terms. It\'s very hard to operationalize anything. So that\'s the
challenge.

We began all of this a year ago, and David and people at CDER, CVN,
CBER, and ORA have an internal expert working group. I think Gregg and
Brian were all members on that working group, and they have gone through
with their expert in house, gone through and generated a list of what
they think is relevant risk factors that we should consider for site
risk ranking and that we should consider in developing this model.

And they have gone through a process of generating those risk factors
and assign them values, high, medium, and low risk, and this is an
example.

When I first showed up, I was given a paper about five pages long. It\'s
a spreadsheet of factors, a just listing of factors and risk descriptor,
high, medium or low, as you see here. And I looked at it, not having
worked in pharmaceutical, coming from a very different background. I
said, \"I don\'t understand. How do you come up with risk, high, medium,
or low? What\'s the context? Risk to what? Risk to whom, and what is
risk?\"

And I was asking a lot of dumb questions because I just didn\'t know
what all of this was coming from, and by asking some very basic
questions, it became to emerge \-- well, back up to what Gregg said
earlier. As a risk assessor, we like to systematically organize things.
So when I saw these lists of five pages of factors, I wanted to organize
them. I had to put them in context.

So we began a process of coming up for air. We have too many details. We
need to come up for air. We need to get back into the high level
organization, into somehow all of these factors have to fit in certain
categories so that we can systematically organize them, manage them, and
combine them.

And that\'s how the three components are derived. It\'s based on a
process of discussion, of me asking a lot of questions of what are you
thinking. Why do you think this is high risk? High risk to what? High
risk because the product is high risk? This variable, if something goes
wrong, the product is going to potentially impact the users, or if this
variable goes wrong, does it have to do with the process? What does it
have to do with?

And in the end, through a serious discussion, things start to fall into
the natural categories. For instance, some of those factors, I\'m just
showing you some examples here. The dark blue, through membranes,
that\'s a factor that has to do with the product versus cartooning and
packaging has to do with process. So we go through a process of
categorizing that way in the facility.

People talk a lot about approval first time. You know, that falls into
the nature of the facility. What is that facility all about?

And I think David already gave us a pretty good background on this
chart. So essentially we took a bunch of factors, a big list of factors,
organizing them and make them sit on three legs essentially. So now
we\'ve got the three legged stool to work with.

So one of the legs is product. One of the legs is facility, and one is
the process. And the idea of the framework is we\'re going to go back
down, drill down to these boxes, to these legs, and make them walk, and
in the end we can fill it all up and have the site risk potential, and
that can be the score.

So as you can see, this is very similar to some of the other models that
I just flashed through very quickly at the EPA what they\'ve done, the
USDA, what they\'ve done, and what DOD has done. So this is not
different from what\'s been done. It is just a different application.

In the next couple of minutes I\'m going to talk about drilling down to
those three categories. How do we select the factors given the laundry
list of factors that we have categorizing into these categories? Which
of those are workable? Which of those that we can actually work with?
Which of those that we actually have data, empirical?

By the way, of those Bayesian, I\'m a strong believer of having data
before I start. I don\'t have any prior, but that\'s my bias, but then
we also \--

DR. SINGPURWALLA: That\'s a tragedy, too.

DR. TRAN: It\'s a tragedy, but don\'t forget. Once we have the empirical
data, we can put in some judgment. That becomes somewhat of a prior.

DR. SINGPURWALLA: Well, we\'ll talk about this.

DR. TRAN: Yeah, I made a mistake. I told them I\'m a frequentist. Big
mistake.

Okay. And so once we select the factors, it\'s going to be driven by how
feasible are these factors. Do they make sense? Do we have data?

And judgment has to be on some kind of avenues. We can\'t just be
pulling out of thin air, in my opinion, and from that we develop a
logical algorithm to combine and then come up with a final composite
score.

I\'m going to talk first about, again, this. We have three components,
and we\'ll talk about the site product score very quickly. How do we
populate that component?

And we teased that out into two more subcomponents. One is the intrinsic
factors, inherent hazard associated with a product, and these
categories, these factors are the intrinsic factors that David had
talked about earlier, sterility or non-sterile drugs, whether they are
over the counter or prescription drugs.

These are very rough approximation of intrinsic factors. We recognize
that. This is something that in the long run we would add additional
intrinsic factors, true intrinsic factors of potential hazard associated
with a pharmaceutical product, that if something does go wrong, the
consumer will be severely impacted.

So we recognize this is a very rough approximation. This is only the
beginning. What we\'re most comfortable with is recall data. We have
empirical data out there that tells us about the severity of the quality
effect and how frequently that does happen.

So the bottom line is for the moment, the model, we have put a lot of
emphasis on the recall data, and one of the challenges, we\'re using the
recall data is we need to be able to link the recall information to the
site because remember this whole model is to be able to somehow capture
the three components, assign it to a specific facility, come up with
some kind of a score and rank them, rank order them, and then we can
target the right one for inspection.

And our data source for site information is the fear accomplishment
(phonetic) and compliance tracking system, and please don\'t ask me any
more about the database. You have to ask Brian for that. I take the data
from them, and I just use them, and I\'m told this is where all of the
site information are being kept.

And also in this database there are product codes, but these product
codes aren\'t the same as the recall data code. So we have a challenge
of matching data. So that\'s one of our challenges.

And we went through a process of grouping the recall, and I think I have
a slide to talk about that. No, I don\'t.

Essentially what we have to do is since we cannot assign the recall data
to a specific site, we stepped back and we said, okay, let\'s aggregate
the recall data into some fashion that we can link it up to the site,
and one way of doing that is in the FACTS database we kept the data
based on dosage form or profile class. Some of the product
classifications that the earlier presentations, so that\'s how we rolled
the recall data into those product classifications, and then those
product classifications are associated with the sites.

Again, we use the CDER recall database, and we are looking at the recall
data between 1997 and 2004, all of the occurrences that we\'ve had. This
is how we are looking at in terms of putting a weight to the recall
data. This is the recall weight matrix. It looks like that probability
and severity matrix that Greg had showed earlier. Like I said, we borrow
methods from existing literature from other agencies, and this is one of
the ways that we\'re going to weight the recall data, and these are the
weights from one to five that\'s going to be assigned to each dosage
form and that\'s going to be attached to a facility.

And, again, we don\'t have probability. So we are looking at some
surrogate percent of total recall in an HHE class, and five is the
highest hazard. One is the lowest hazard if you want to interpret this
directly.

I already talked about this. I\'m going to skip this because I talked
about the correlation.

Let\'s go to the facility component. So that\'s essentially for the
timing of what the product component factors look like.

The next component, the next category, the next sets of factors are the
facility, and where are the components of the facility box in the site
risk potential score? At the time being we have three basic components
within the site facility score.

The history of inspection. We\'re looking at a scaling, a weight scale
for this factor, and essentially if a site has been recently inspected,
it\'s going to get a very, very low scale, less likely to be picked up
in the next year, so on and so forth, and if that site hadn\'t been
inspected in a long time or never been inspected, it\'s going to have a
higher scale there.

History of compliance and violation. This is the OAI, though no official
action and the OAI category. We\'re going to pull that in here with a
weighting scale, and OAI is going to have a highest score. So the 30
that had a history with OAI would have a higher score there, and the
volume, again, this is a surrogate for potential impact for this
facility should they have something to go around with this facility in
terms of reaching the consumers. This is a really rough approximation.

And, again, for this facility site score, our data came from FACTS,
field accomplishments and compliance tracking system, and we are
downloading the data for the years 2000-2004, and all sites are being
scored in this way. They are all foreign and domestic firms.

Last but not least is the process. I think this is the one that\'s the
most interesting so far, is the process component factor. This is one
that gave us a lot of headache because it was the toughest one.

We didn\'t have any data. The idea here is the factors that should be
fed into the site process score are the relevant inherent process risk
factors. What are those? And the relevant process controls and risk
mitigating factors. What are those?

And we understand that these factors are product and facility specific.
This is when we ask people to kind of come up for air and think broadly.
It always goes down into the very level of detail. A very specific
product, very specific facility. So this was a huge challenge, but I
think the working group was successful in having a lot of discussion on
how to kind of step back up and categorize products, categorize unit of
operations, and come up with a process, an explicitation (phonetic) to
ask people questions, to come up with some information on how we can
come up with this process score.

And I\'m going to turn this over to Brian since he\'s spent a lot of
time with the expert group.

DR. HASSELBALCH: Yes. Well, it\'s a bit strong to say no data. We have
data. It\'s just locked in paper files, and we have no ready way of
getting at it in any time soon.

So we thought it would be nifty to query the experts in the agency. We
could have gone outside the agency, but that involves some other
bureaucratic hurdles we didn\'t feel like we wanted to deal with at the
moment. So to expedite things, we stuck with experts inside the agency.

We began drafting the document with a smaller group of experts among the
various centers involved with regulating medical products, but our
device center, and the key questions we asked in drafting the survey,
which I\'ll show you excerpts from in a little bit, were to ask what are
the relevant process related risk factors. In other words, could we
think of processes in terms of the source of variability.

Naturally, of course, we can because they not only contribute to
variability,b ut when they work well, they contribute to homogeneity or
lack of variability and good quality.

We also asked what, if any unit operations are more reliable to a loss
of control or to risk from either environmental or product to product
contamination?

We drilled down to unit operations you\'ll see shortly, but as you\'ll
also notice, we don\'t actually allow much for the unit operations in a
final aggregation because of limitations of our site identifiers for
information.

Thirdly we asked should the experts or would the experts want to
distinguish among products or product types. Could we categorize all
products into certain groups and expect the experts to reliably
distinguish between those groups of products in their opinions or
judgment about risk to variability, quality and control and
contamination.

Naturally, we felt we could expect that distinction from our experts. So
we set about identifying mutually exclusive categories. We borrowed a
bit, I should say, from ISPE\'s Baseline Guide. I\'ve given the site
here for soderol (phonetic) dosage forms. It\'s at the back. It\'s
intended to be a tool for companies to use in building new sites as to
those areas that may cause them more or less headache or difficulty or
cost in constructing and making operational the new facility.

I\'ve just taken a page out of this. It\'s several pages long, covers
different areas, but very nicely I think it signals us that it\'s
possible to distinguish unit operations by product types when we\'re
talking about GNP issues like variability in terms of process and
contamination.

So a big struggle was in categorizing products to get a number that
wouldn\'t be too burdensome for a panel to ultimately answer on, but on
the other hand to make it fine enough so that we could, going back to
our inventory of sites, identify those sites by those kinds of products.

We code in our agency many things, and one of the things we code in many
different ways for many different purposes are the kinds of products
each site makes, and by \"site\" I mean manufacturing facility.

We found a lot of cross-correlation. I\'ll show you some of that in a
little bit. I know the professors are being challenged by that issue as
well.

Again, we chose to create families of products by their relationship to
similar unit operations, so blending, mixing, tableting or compression
or fill, liquid or solid.

We also distinguished high from low active weights. We felt the experts
might think differently about the influence blending has on a product if
that product ultimately has a lot of active percentage of its total
weight or very little active.

Again, the variety of resources, including experts. Here\'s just a
taste, if you will, of our cross-correlation. The product groups you see
on the left are those groups ultimately that will influence the model.
So that\'s the aggregation. They will \-- I\'m sorry \-- that are in our
expert elicitation survey.

The middle column are those codes that identify those kinds of products
that exist in our data systems, and the description is off to the right.

Here\'s an excerpt from the survey just to give you an indication of the
kinds of products we chose again, and here are the questions we asked
the experts. These are the five questions we asked each expert to answer
on a scale with respect to the various product types you just saw and
the unit operations that you haven\'t seen yet, but that are at a
smaller expert panel associated with those families or categories of
product types.

Three of the questions have to do with and I think get to process
control. The other two have to do with contamination. I think, you know,
our feeling was in crafting the questions this way and including only
these questions, that we were really capturing the essence of the GMP
standard or control requirements.

This is an excerpt, just an example. Again, solid oil drugs, in this
case immediate release, the five questions, the scale that the experts
were asked to answer on, and you\'ll see here the unit operations we
identified as typically occurring or used for this kind of product, and
of course, it would be the same whether it was high active or low active
for the most part.

After asking the experts to go through this ranking exercise for these
different product types by these control contamination questions in unit
operation, we then rolled it all up into a single page questionnaire
about whether they felt essentially whether process control or
contamination was more or less significant for those product types.

So in other words, we took out the unit operations and just asked them
is process control or contamination, if you had to decide, which one
would be more important to you in terms of the quality of the product
being produced from that process.

We, in fact, did not deliver by E-mail. We delivered by paper. Well, we
sent it by E-mail. Everybody printed it out and did it by hand, and then
we consolidated the comments by hand as well.

We got 50 experts to participate from a variety of staff members. We had
a 90 percent response rate. I think that may be because some offices
were really heavy about getting the answers back.

The cooperation was very good, as a matter of fact, and we\'re still
analyzing the results. Now, I don\'t know if you want to go into too
much now, at the risk of some discussion at the moment on how we analyze
or are considering analysis of the expert elicitation data.

Yes, please.

DR. TRAN: I think this is a team effort. I\'m going to need Gregg to
talk about the fuzzy arithmetic. We\'re looking at the data right now,
and we did some exploring and I was just graphing some of the average
answers and see if there\'s anything that looks like an outlier, and for
the most part, the answers are pretty consistent, that there are no real
outliers out there.

And we have the two different ranks. One is the product ranking, the
general big picture ranking. This is a list of product, the using
process, oil contamination. These are the weights as of our last survey.

We did that internal validation. We just want to make sure that the
answer for the unit operation drill-down is not going to be so
different. We wonder if they\'re going to be really different from the
overall ranking and the correlation is pretty good.

And we\'re in the process of developing process weights based on the
unit operation, drill-down survey. That\'s the most comprehensive way of
looking at that, and as a true frequency, I\'m looking at K-Ming
(phonetic) cluster analysis and Gregg as somewhat of a Bayesian, he\'s
looking at fuzzy arithmetic, and the two of us are going to come back
and compare notes and see which way we want to go. I think we\'re going
to go with the fuzzy math as soon as we can get all fuzzy about it.

Do you want to talk about that?

But the K-Ming cluster is just the five questions combined, use cluster
analysis, and the weight is going to be given the highest weight for the
cluster that has the highest center, and that\'s very straightforward.
It may not be suitable for expert data, categorical data. It\'s just
that we think the fuzzy arithmetic might be the better way to go.

Gregg, anything on that on the fuzzy stuff?

DR. CLAYCAMP: I don\'t think it\'s necessary to go into any details now
other than the real objective here is, as Brian mentioned, that we
don\'t want to lose detail in our probing of the expert mental models
that have been out there and been doing this for years, but once it hits
the spread sheet, all of a sudden we have a lot of information before
us, and so we\'re asking questions. Can we collapse this into its key
drivers for the sake of simplicity?

And so it\'s looking at principal components, for example, and you know,
very, very preliminary analysis is they kind of fall into lines that the
experts would have told us in the first place.

So those are the reasons that we\'re looking at those techniques that it
would take, you know, as many as 11 measures down to hopefully a couple
that would be easier to handle as weights in the model.

DR. HASSELBALCH: This is the summary chart. Again, for the model scoring
purpose, we\'ll likely distinguish process controls from contamination
and let both of those contribute to a single site score in addition to
the other categories of product and facility.

Let me just summarize in plain language. At least I can do that.

The model\'s impact on our inspection decisions. It is simply that a
site will tend to be less frequently inspected if it has been inspected
recently and/or has relatively few previous violations of GMPs and/or
smaller volume product. So that all contributes into the facility weight
module.

It will be less frequently selected for inspection if they make
non-sterile OTC drugs and there are other product types that aren\'t
associated with a high frequency of serious recalls; contributes to the
product weight of the model, and the process solicitation data largely
will contribute to the third element, which is that they make products
estimated to be relatively straightforward of manufacture and not
vulnerable to contamination.

Of course, the converse is also therefore true. Sites will be
preferentially selected for inspection on an annual basis if the
opposite holds.

This also summarizes in chart fashion the scoring scheme and the
contributions now into the model. I think this would be a good time for
me to point out that largely we have to communicate. The difficulty or
limiting factor here is largely to communicate this to our field staff.

We have 19 different district offices. Any multiple of that that are
involved in program planning at the district level, we need a any to
communicate to them the center\'s priorities for inspection in a way
that will allow them to strategize or conduct their inspection to take
into account those areas of production or the facility that seem to
matter the most, that seem to influence the most the risk that that
facility has in our marketplace.

This is not a model to predict a violative site, though it\'s going to
have a tendency if we pick bad sites. Historically there\'s a
preference, but it\'s not design for that purpose. It\'s largely
intended to get to those sites, FDA inspectors at those sites, reliably,
at a reliable frequency that seem to matter the most in our marketplace.

Of course there are things we\'d like to include in here for which we
presently lack data or a mechanism to account for them, but these,
again, as David mentioned, we expect that this model will change over
time, and we\'ll have to incorporate additional information as we go
along.

And I think one area where we can easily include some future information
would be in the area of some metric associated with process capability,
whether it\'s a CPK or some measure of yield or success at making
batches. We\'re hopeful that that will have a future impact on the
model, perhaps drive down the score for certain sites.

Okay. There are some questions that I think we\'d like the subcommittee
to ask, and, David, you\'ll facilitate the section?

(Applause.)

MR. HOROWITZ: Okay. People may be getting hungry, and I know there are a
lot of questions and comments that have been building for the hours, and
so I just want to, before we start, reassure people that this is not
your last opportunity to comment on this. This is just the beginning.

In particular, in September when we announce a big announcement on the
GNP initiative, we\'ll be putting forward a small white paper that will
describe some of these things. We\'ll be opening that up for public
comment and whether it\'s a docket or through other forms, and we hope
that you\'ll all bring forward the comments from today, but also other
comments that may occur to you subsequently and other constructive
suggestions on how to make this better.

We\'re hoping to pilot a rough version of this model for the coming
fiscal year, but it won\'t consume all or even a very large portion of
the field\'s resources, but some of the field\'s resources will be
devoted to doing inspections that are derived from this model.

So with the permission of the chair, I could start on these questions
then, and I recognize that you\'ll probably have comments that go beyond
these questions. That\'s okay, too, but if I could, I\'d like to start
on these.

First, can you identify alternative approaches that would systematically
prioritize manufacturing sites for GMP inspections?

I have a feeling that there may be some ideas out there on how we might
do this completely differently, and we\'re all ears. We\'d like to hear
some other ways that we might be able to accomplish the same objective
we have with the limitations that we face in data and other things like
that. So, please.

DR. SINGPURWALLA: Answer to the first question is yes.

MR. HOROWITZ: Okay. Anyone else?

(Laughter.)

MR. HOROWITZ: I want to get that yes.

DR. SINGPURWALLA: Yes.

CHAIRPERSON BOEHLERT: Yeah, I\'ll let you recognize the committee
members.

MR. HOROWITZ: Oh, okay.

DR. RAJU: David, going back to the comment that you made at the start of
your presentation that this is more about inspection rather than the
broader initiative, are you willing to entertain some broader initiative
responses to one that connect back to inspections?

MR. HOROWITZ: Yes, I am, just recognizing that this model is not
intended to go beyond its very narrow purpose, but I\'d be glad to.

DR. RAJU: In the end, safety and efficacy and availability are about a
product that somebody consumes, and he really doesn\'t care or doesn\'t
know what site it\'s made at. So an alternative approach would be about
a violative product and about prioritizing the manufacturing product
rather than the site, given that, of course, the product has to be made
at a site.

I know you\'ve laid the foundation for it. I\'ve seen Brian\'s
presentation, and you\'ve laid the foundation for it, but looking
beyond, could it be about privatizing among products rather than sites
as an alternative approach that your foundation might get to because the
customer really doesn\'t start with the word \"site.\" He starts with
the word \"product.\"

MR. HOROWITZ: Yeah, I\'ll start briefly, and then I\'ll ask the other
speakers, but I think that\'s very plausible. Ultimately though the way
the inspections work is they have to connect the product to a site
because they have to decide where to go, and I think drawing that
connection out would be very valuable, and I hope that the model begins
to do that, but I think there is probably more opportunity for focus and
knowledge to be derived and applied in that area.

Brian, I think you were.

DR. HASSELBALCH: Ditto. Exactly. I think as a start it\'s fine, but I
think the future will have it smarter and make us capable as a
bureaucracy to distinguish not just sites anymore, but processing lines
at sites. Because after all, a site could be very big. It could be
multi-building, huge campus, or it could be one building.

And I think in the future we\'ll be more capable of making those
distinctions, but there are some things that have to happen internally
about how we count the work we do and value that that also have to
change along with that because we\'re now heavily driven by sites,
addresses in terms of budgeting and planning.

But thank you for that comment.

MR. FAMULARE: I just think one fact to think about, G.K., is that a lot
of the work we\'ve done over the last ten years since the generic drug
crisis was product and preapproval inspections, and we\'ve seen the
fault of not covering systems fully, sites fully. So in order to get
back into those sites and systems, proper quality systems at a site
facilitates products, changes, and continuous improvement.

So there is an emphasis back on quality systems which right now
translates somewhat to sites, but as Brian brings up, being able to then
drill that down to product lines\' processes would be the next step.

DR. RAJU: You can go to it both ways. You probably have to do it
simultaneously. The problem with going to the site and all of the paper
work and the quality system and all of the tracking is, given the legal
relationship between regulatory and regulated, there\'s such a big
degree of gray area before you go to the truth with this, the physics,
chemistry, and biology. That\'s the process that goes into somebody\'s
body.

So there\'s the physics, chemistry and biology that depends on a system
to do it right, and the other vocabulary is being put in place, and you
always need both, but I think we probably have overemphasized the
top-down too much.

MR. HOROWITZ: But before I go on to the second question, maybe I\'ll
follow up to Nozer\'s answer to make sure that no one is constrained by
the wording of the question and say that if you have additional or
alternative approaches that you\'d like to recommend and ask us to
consider, now would be a good time.

DR. SINGPURWALLA: Well, I\'m glad you asked because your question says
can you identify, and I said yes. But now you\'re asking me what the
alternative is.

The way I would see it is I would see the problem of inspection, of
choosing a site for inspection, as a problem in making decisions. So I
would draw a decision tree, and I would choose that particular site. I
would prioritize my site according to the expected utilities that I
would get from each decision tree.

So I would draw a decision tree and do it, which is the way one should
choose sampling inspection plans and amount of sampling that needs to be
done.

So I would use the standard recipe for doing it in a more formal way,
and that\'s all I have to say on that one.

But I do have comments on the presentations. So I hope you\'ll give me a
chance.

MR. HOROWITZ: Okay. Can we get through these next few questions?

DR. SINGPURWALLA: Yes, absolutely.

MR. HOROWITZ: Can I just ask Gregg if Gregg wants to respond to that
first on the question of decision trees as an alternative approach?

If you have a comment, please share it with us, and then Paul is next.

DR. CLAYCAMP: Right. At this early stage, that was a little bit
overwhelming overall, but a lot of this does fit right into that type of
process, and that\'s my personal bent, is to set up decision trees.

DR. SINGPURWALLA: So you recognize that.

DR. CLAYCAMP: Absolutely.

DR. SINGPURWALLA: Yeah, thank you.

MR. HOROWITZ: Paul.

DR. FACKLER: I just wanted to say that I\'m guessing you haven\'t
finished this analysis so that these sites haven\'t been identified or
prioritized, but when that has been done, I think it might be useful to
look then at the distribution of the sites, recognizing that more than
half the prescriptions written in the U.S. are written for generic
drugs, it would be useful to look at the distribution of generic versus
PhRMA site and bio versus traditional oral, small molecule sites to see
if the distribution is similar to the distribution of products in the
United States.

Not to say that they necessarily will correlate, but I think it would be
an important thing to look at. I don\'t think you want to make this
simply a scientific assessment or an objective assessment. I think that
there are subjective reasons that might cause you to change you
inspection procedures.

MR. HOROWITZ: Thank you.

Okay. I\'ll go on to the second question then. In what areas would
additional data provide the most value added in prioritizing
manufacturing sites for inspections? I mean, you could all see that our
data is very limited here, and you know, one of the things we need to
think about is prioritizing our efforts to improve this model.

So I\'d like your thought on where we might add data to this model. I\'m
sure there are other improvements people can suggest as well, but for
this question we\'re focusing on where additional data might be most
valuable and improving the model for our purposes of getting the most
bang for our buck.

It looks like Nozer is \-- no, your red light is not.

DR. SINGPURWALLA: No.

MR. HOROWITZ: Okay.

MR. MIGLIACCIO: I\'m having a little liberty with the question.

MR. HOROWITZ: Please.

MR. MIGLIACCIO: Because I\'m not sure. There\'s one data point that I\'m
not sure it is going to have the right value, and that\'s volume. I\'m
very concerned about the volume factor.

First of all, it would imply that GSK and Pfizer would get most
inspections, which if you look at the way some of us run our business,
you will have high volume facilities that make only one or two or three
products, and inherently the risk is lower in running those. There are
fewer changeovers.

And then there\'s the dosage regimen. How much exposure is out there
depending on how many patients there are for that product. Volume in
itself is not a good factor to use. It has to be expanded into other \--
you need to complement that with something else. Pure volume I am very
concerned is going to lead you to low risk facilities when you look at
it.

So I\'m concerned. We have to figure out how to complement volume with
something else because going to your question, you know, you\'re going
to direct it to high sales companies, and that\'s a concern.

MR. HOROWITZ: Okay. Can I just briefly respond to that? And then I see
Ken\'s light is on.

We believe that the model as written now does complement it, as you put
it, with a variety of other factors. If volume were the only factor we
looked at, the model would be absurd on its face, but I think there are
so many other mitigating and other factors.

The weight of volume in determining frequency of inspection is actually
quite low if you take out that factor and the fact that, you know, it\'s
counterbalanced by so many things, some of which you mentioned. If the
high volume site does a good job, for example, you could expect that
they wouldn\'t have a particularly bad compliance history, and I think
that would be something that would be weighted in.

If they do a good job in a high volume site because it\'s easier to
focus on that, they might have fewer recalls associated with that
product, and so forth.

So it\'s definitely something we need to watch for though, and I
understand your concern because you don\'t want to create the wrong
incentives. I mean, obviously we want to encourage firms to adopt those
mitigating and other factors which take advantage of, for example, the
good things associated with high volume manufacturing.

Anyone else from the speakers who wants to address that? Gregg.

DR. CLAYCAMP: Yeah, just to follow that up, you know, at this point if
you try to look too formalistically at the details in this, you\'ll see
things going on that in the modeling sense will look like confounding
and multiple colinearities, et cetera.

So right now, the conclusion you\'d come to is that it is being tempered
by, for example, when we asked the experts in brainstorming what were
the factors to do with processes, making the same thing all of the time
was lower risk than process changes, and so that kind of works against
the volume rating.

So, you know, there are competing factors in the model right now that I
agree with Dave that it probably in the end isn\'t weighing very much.

MR. HOROWITZ: Right.

DR. DeLUCA: David, you need a volume risk index so that when you have
the risk that doesn\'t include the volume, but then that comes in as an
index because if a small firm is a medium risk and a large firm is a
medium risk, then I think the large one plays a role

DR. MORRIS: Yeah, I guess it\'s sort of the same point said slightly
differently, but you know, 100 deaths is worse than 10,000 cases of
diarrhea, for instance.

MR. HOROWITZ: Absolutely.

DR. MORRIS: So even if it\'s local, which is what somebody else had said
earlier, I think even if you have a local effect, it can be much more
detrimental.

the other point I wanted to make in terms of the areas of additional
data, I\'m not sure quite how to do this, but there\'s a bit of a
problem using historical expertise when you factor in where we\'re
going, I guess, because on the face of it \-- I\'m not saying this
can\'t be overcome and within the same system \-- but you\'re bringing
into question issues like, for instance, if you say that your last
inspection, if it was more recent, you\'re at lower risk. Well, if
you\'re controlling your process, monitoring and controlling real time
so that you have gotten the regulatory relief so that you don\'t need as
many inspections, then that ends up making you higher risk even though
it is innately making you lower risk.

Similarly for things like Sterile processes being counted as higher risk
than non-sterile. Historically there have been, you know, some very
elaborate mechanisms for making sure the sterile products manufacturing
is very reliable. So are you penalizing them in the face of being more
reliable?

And finally, the controlling of a process when we\'re talking about the
\-- I\'m referring now to the process of the unit operation ranking of
difficulty in the historical expertise \-- if you\'re talking about
controlling to time as an endpoint, then that will give you in many
cases a very different answer in terms of the reliability or risk of
that unit operation than controlling to the endpoint.

That\'s all I have to say.

MR. HOROWITZ: Thank you.

G.K., I think you have one.

DR. RAJU: In terms of Question No. 2, I think there\'s a systematic \--
if you went back to Janet Woodcock\'s definition of quality and you said
safety, efficacy, and availability you said, but you were the surrogate
of the customer, and then you define surrogate variable, such as
identity purity that you were going to do your regulations around.

But when you made the mapping from the customer to the surrogate
measurements, safety and efficacy, but presumably mapped on, but
availability didn\'t show up in that mapping, and so the system that we
have is predisposed to go after a company that might be making a very,
very difficult product that nobody can ever make, a sterile product, a
vaccine that would never have been on the market, but it\'s available.

So you would go after maybe a sterile product or a very complex process
that they were the most innovative in the world to make. So how do you
eliminate that bias of availability not being in your broader risk,
although it could be outside this model?

MR. HOROWITZ: Yeah, I think that\'s an excellent point. This is why in a
lot of these comments I think one of the themes is we need to be careful
about the incentives we create here because it could have unintended
consequences, and that\'s one of the reasons why we\'re rolling it out
for input, one of the reasons why we\'re going to be phasing it in
slowly.

But I think the particular issue that you raise with regard to
availability, that might be something we could consider as a mitigating
factor or a risk decreasing factor if the product is at risk of loss of
availability. Perhaps that\'s something that we ought to take into
account.

But I want to say though that just because we inspect it doesn\'t mean
it will be taken off the market because there are other ways that we can
take those factors into account.

Now, some would argue a critical lifesaving product that is a single
source product that is really hard to make, we should be inspect them
and working with them in trying to help them make sure they can keep
manufacturing.

DR. RAJU: Right.

MR. HOROWITZ: So it doesn\'t necessarily need to result in reduced
inspectional oversight for this model, but I take the bigger point that
we really need to be very careful about the incentives that we create to
make sure they\'re the right ones to push and encourage the industry to
improve their process of understanding and to adopt the most modern
technologies.

Joe.

MR. FAMULARE: You know, just going off, I second that. Very often when
we\'re in those situations we will inspect more towards working jointly
to resolve those issues and those very complex products, but also to
respond to what Ken said before in terms of depending on the regulatory
paradigm and the advancement of modern technology, PAG, and so froth,
you\'re saying it may result in less inspections or it may result in a
different way of looking at things.

You know, a lot of the discussion yesterday was about reducing
supplements, and therefore, at some point not only will the
investigator, but what we have factored in, the product specialties may
want to look at that. That may be a factor that we bring in to target.
Not only will we look at that at inspection. It may be at an appropriate
frequency, but it will be a way of targeting when we want our product
specialist there because they\'re looking to reduce their supplement
burden, and so forth, and bring that along.

MR. HOROWITZ: Don.

DR. GOLD: There are a couple of points that I wanted to add. One is to
look at or consider hard to fabricate products. I think this was already
mentioned before. There are a number of products in the marketplace that
are quite difficult to fabricate and where controls are very important.

And, secondly, there are some products in the marketplace where control
of uniformity of dosage is extremely important, where the patient has to
be titrated and the product has to be carefully controlled. And I think
that has to be added to the mix as well.

Finally, I\'d like to make another point. Perhaps you\'re getting to
this a little later on in this discussion, but with the absence of a
dedicated pharmaceutical inspector, there is a considerable variability
in the efficiency of inspections that I have seen. I\'ve seen this both
in the United States, and I\'ve seen this at various other parts of the
world.

So when we talk about using the history of the firm or the past
inspection of the firm, whether it\'s a VAI, they get a VAI, I\'m very
concerned that unless we move to a pharmaceutical inspectorate that is
more uniform and better trained in their capabilities, that we may not
be using the proper metric when we talk about previous inspections as
affecting the frequency of the oncoming inspection.

Now, I know, Joe may not agree with this fully, but this is certainly
well within my experience.

MR. HOROWITZ: Yeah, I\'ll yield to Joe in a moment, but I think this
goes back to Ken\'s comments earlier about one of the problems with
getting a model like this off the ground is if you rely on historical
data, but it\'s not static data, the pharmaceutical inspectorate and the
approach to GMP inspections is changing, and I think that we have
created a dedicated pharmaceutical inspectorate that will now be
starting the coming fiscal year be operational.

And I do think that there are a number of aspects of the GMP initiative,
including the creation of the pharmaceutical inspectorate that will
gradually improve the coordination and the consistency of the
observations that come about as a result of GMP inspections.

And what I expect is that over time the data on which we rely, the
historical data on which we rely, will be increasingly reliable and
increasingly valuable to feed back into the model.

But there\'s no doubt that we\'re dealing with some of these challenges
right now.

DR. GOLD: But, Don, if we talk about a pharmaceutical inspectorate
starting some time later this year or next year at the earliest, and
we\'re talking about in implementing this model within a reasonable
period of time I thought you\'re aiming at some time later this year to
start introducing this model. How will we merge the two timetables?

MR. HOROWITZ: Right. Well, that\'s what I\'m saying. The data we\'re
using is based on the old model, and we all understand that there are
certain problems with that, and that\'s why we\'re switching over to a
pharmaceutical inspectorate model, and as a result, our data will not be
as good as it could be and hopefully will be in that area.

I wish that were the only data shortcoming that we were dealing with
right now, but it\'s certainly one of them that we\'ll have to keep an
eye on.

Joe.

MR. FAMULARE: You know, just to speak to your concern about
investigator\'s consistency and how that influences the model, you know,
a lot of this initiative is to address those inconsistencies in not only
the formation of the pharmaceutical inspectorate, but in doing the
expert elicitation, you know, not only were reviewers called on, but
folks in the Office of Compliance of CDER and those investigators that
are predominantly, if not 100 percent, although there are fewer in
number now than we would like, were called upon in terms of their
experience with the expert elicitation.

So we tried to overcome as many of those mitigating factors \-- and
Brian could chime in on that. He\'s most familiar \-- as there could be
to get that consistency in there.

I think what folks have to think about and step back for a while is
we\'re transforming from a system where we inspected or aimed to inspect
every firm every two years that registered, and for years we have not
been able to do that, but we didn\'t have a good working model as to who
we should get to first, and it\'s going to take a while.

We\'ve taken some rough cuts at this. Let\'s do all sterile. Let\'s do
all Rx drugs, and let\'s do all new registrants. But this is taking it
to the next logical step, and when we hear about this in other venues,
probably the most common thing is police work. You know, they\'ve done
computer based policing and so forth.

I recently read an article about a Midwestern city now that just did
this type of work on convenience store robberies, and actually it helped
them to catch crooks because they put a pattern about it as opposed to
just putting old marks that you saw in the old movies on a map where the
crimes occurred.

And even in that same article, that same city, even incorporated an
element of PAT. They put sound detectors to hear gunshots so that you
could go nearest to where the gunshot is and figure out that\'s where
the crime is going on.

So you know, these are not \--

PARTICIPANT: Have you told that to Ajaz?

MR. HOROWITZ: Well, people are probably getting hungry hearing the
reference to convenience stores.

(Laughter.)

MR. HOROWITZ: But you will feel \--

MR. FAMULARE: But I think we have to put it in perspective, that we\'re
now really trying to put together a model of figuring out who we\'re
going to go to first and when, and even to go to the trouble that the
professors had, Jackson and so forth in getting to those overseas
companies. We have to pick and choose our shots overseas even more
stringently because it\'s difficult also.

So this is the first very organized step we\'re going to take in doing
so.

MR. HOROWITZ: The last question or comment on this and then we\'ll have
to hit number three because I know people are eager to move on.

Garnet, please.

DR. PECK: This is for Number 2.

MR. HOROWITZ: Okay.

DR. PECK: You explained and defined various product types, and then you
also comment on unit operations. But there is no explanation about what
was done with the information or the knowledge base that was gathered,
and I think for two it might be interesting to take a look at the
processing and what unit operations are involved and see if there is
some kind of correlation coming out of this, and it may be like the
policeman, you know, spotting something that could be happening with a
particular series of unit ops and analyze those.

So that\'s my thought for Question 2.

MR. HOROWITZ: Okay. Thank you.

Now Brian on the expert elicitation. do you want to respond to what
you\'re planning on doing or have done with that date?

DR. TRAN: Yes. That\'s our plan, is to drill down and analyze the data
at that level, but we haven\'t gotten that far yet. That\'s our intent.

MR. HOROWITZ: Okay. Let\'s look at Number 3 and then depending on the
discretion of the chair, there will be additional time for questions.

But this is just specifically whether there might be some metrics we
ought to consider. Process capabilities come up. SPK is one measure that
is talked about a lot.

If we could build in any more objective data into the system obviously
we want to do it, particularly if it could be widely understood and
accepted. Any thoughts on that? Any metrics of process control, which is
really the heart of what we\'re looking to focus on for the GMP program,
that we might include? Any thoughts on that?

DR. FACKLER: I\'m not sure where you would get this data or if this is
really an answer to this question, but facilities that have a high
turnover in personnel are clearly going to be \-- I shouldn\'t say
\"clearly\" \-- might be more at risk than facilities where you have a
stable set of employees, and I don\'t know how you would necessarily get
that data without going there and asking the question, but to me it
might be a factor.

MR. HOROWITZ: Well, we may hear more also from the Nickerson and Macher
study to identify some objective measures and things like that.

The other thing is some of the data we could go out and determine on
inspections and add to our databases routinely. So one interesting idea
that I heard would be one measure might be look at the percentage of the
root cause investigations that actually get to the root cause versus the
cause is undetermined. That might be an interesting surrogate for a
process understanding.

That\'s not data we currently have in our system, but in theory that
might be something we could collect. You know, there\'s limited
resources, but if we could figure out a few good ones perhaps like that,
perhaps like something else, we could improve our databases.

You know, overall I think it\'s just the process of beginning to think
critically about these things that\'s very valuable for us, perhaps even
more valuable than the actual reordering of the sites. And we\'re eager
to engage in more dialogue like this to get on the same page.

So at the discretion of the chair.

CHAIRPERSON BOEHLERT: I think we could take just a few minutes if there
are some burning questions. I know we probably all had questions as the
speakers presented, the last four speakers. So, Gerry, did you have
something?

MR. MIGLIACCIO: Yeah. I mean, is this going to be transparent? Will
sites understand how they\'re ranked?

MR. HOROWITZ: You know, that\'s one of the hardest questions because,
you know, we want enough transparency to get valuable feedback and
input, and we want to create incentives, of course, and be transparent
enough to do that. In that sense, we\'d like to be able to reward sites
that are doing it right.

But we can\'t obviously make it so transparent so that anyone could run
our model and they\'d know exactly where FDA is going to be at any
moment because there\'s certain regulatory problems associated with
that.

Particularly given our limited resources, there has to be a perception
of greater coverage than we\'re actually able to achieve.

(Laughter.)

MR. MIGLIACCIO: I understand that, but this whole initiative is about
both FDA and industry putting their resources in the highest risk areas.
So if we from a corporate perspective understand what you consider high
risk, that helps us to understand where we need to put our resources.

MR. HOROWITZ: I complete agree.

CHAIRPERSON BOEHLERT: Okay. Ken.

DR. MORRIS: Just a real brief comment. Would that not just be served by
knowing what the criteria are rather than knowing the ranking though,
Gerry?

MR. HOROWITZ: Yeah.

MR. MIGLIACCIO: With the number of facilities that we have that are FDA
approved, I would like to understand how the FDA has ranked them. I
think we may rank them somewhat differently.

MR. HOROWITZ: Yeah, I think the challenge is for us to provide enough
information so that we can be transparent about the things that we think
are the riskiest and the risk factors so that we can have good dialogue
about that, but also so that industry can focus on this.

CHAIRPERSON BOEHLERT: Any additional questions, comments?

Pat.

DR. DeLUCA: Yeah, I\'m kind of old enough to go back about 30 years,
even predate your slide there with inspections in registered firms, but
it seems that some of the questions that are being asked were asked
then, and I don\'t see anything in reference here to a concept that 30
years ago was called self-inspection, and I don\'t see that mentioned at
all in these deliberations.

And I\'m wondering if this isn\'t something that should be incorporated
into this together information that would allow you to prioritized,
where the industry would have actually self-inspection programs.

MR. HOROWITZ: Gerry, do you want to talk about the first party audit
program or address that question?

MR. FAMULARE: Well, you need to address it in two ways. I mean, there
was a major effort to announce a first party audit program some years
ago where we were promoting self-inspections on how the agency could
benefit from those self-inspections to change or mitigate the amount of
inspections we need to do.

It fell on a number of complications, even looking at how some of our
sister agencies wound up, such as OSHA, where they told them they had to
go and do a rulemaking, and we were bound by current regulations and so
forth, where we weren\'t about to be able to offer a definitive no
inspection, no warning letter, no whatever under the act.

It was a little bit easier in EPA\'s case because they could mitigate
certain amounts of fines and so forth. So we went off that path onto the
systems based inspection path to put focus on the proper places in the
inspection.

Further than that, one of the elements in the September announcement
will be a corollary guidance to the GMPs to try and emphasize modern
elements of quality systems, and that for sure will be one of the areas
of emphasis. You know, it\'s an area where we\'ve always not looked
particularly so you could be frank with yourself, but on the other hand,
how could you translate that information to FDA in such a way that you
didn\'t mess up the frankness of your self-audit or prejudice that, but
again, be able to get some benefit from FDA that we need less scrutiny
or less scrutiny in these areas from our self-inspection.

So there\'s certainly been a lot of thought in the various circles
around this particular effort.

MR. HOROWITZ: If I could just follow up on that briefly, we completely
agree that self-inspections are a crucial part of an effective quality
system, and we want to create incentives for firms to do
self-inspections.

We haven\'t been able at this point to capture how you would feed that
directly into the model specifically. For example, if we went out and
asked them did you do a self-inspection, you know, everyone would just
say yes, and really the key is not just whether you did one, but did you
do it right, did you do it well, and we don\'t want to be in the
position of grading their self-inspections because it\'s been our
longstanding policy that we don\'t generally ask to see your internal
audits because we want to encourage you to do them and find whatever is
buried in the closet and to be frank with yourselves about that.

So there\'s a real challenge for how to tap into that, and I hope that
through the quality systems enhancement guidance and perhaps even
through Q10 one day we can create more incentives and guidance to
encourage exactly the kinds of self-inspection activities that we want
to encourage.

CHAIRPERSON BOEHLERT: Any last comments before we break for lunch?

DR. SINGPURWALLA: I do have lots of comments, but I think I don\'t want
to take up lunchtime. I\'m wondering if there\'s a later opportunity.

CHAIRPERSON BOEHLERT: yes.

DR. SINGPURWALLA: My comments are technical.

CHAIRPERSON BOEHLERT: Okay. There may very well be. It seems to me this
was a topic we could have spent the whole morning on. It has elicited a
lot of discussion from the committee, and I\'m sure we\'ll be seeing it
again at a future meeting.

So thank you all for your participation.

MR. HOROWITZ: Thank you all very much.

(Applause.)

CHAIRPERSON BOEHLERT: We will reconvene at one o\'clock.

(Where upon, at 12:07 p.m., the meeting was recessed for lunch, to
reconvene at 1:00 p.m., the same day.)

[AFTERNOON SESSION]{.underline}

(1:02 p.m.)

CHAIRPERSON BOEHLERT: Well, we\'re all here. So I think we can get
started.

One issue that I would like to raise with the committee is we have a
number of presentations this afternoon, and some of them may also elicit
a fair amount of discussion. It\'s your choice if you want to take a
break or not, and just work our way through and perhaps get out 15
minutes early or perhaps, you know, we\'ll use that time for additional
discussion.

Is there any feeling one way or the other on the committee? Raise your
hand if you don\'t want to break.

PARTICIPANT: As long as you can leave at will.

CHAIRPERSON BOEHLERT: You can leave at will. Is that all right if we
don\'t have a break? Skip the break okay?

Skip the break. Okay. We will skip the break, you know, but feel free to
get up if the need arises.

Okay. This afternoon we\'re going to change gears and begin with a
presentation by Moheb Nasr on GMPs for the production of Phase 1 IND
drugs.

DR. NASR: Good afternoon. I hope you enjoyed your lunch and you are
ready for some GMP.

My presentation this afternoon will be very brief. It\'s intended only
\-- and I underline \"only\" \-- to provide a very brief background of
some of the CMC requirement for Phase 1 IND. I will not discuss the
guidance issue. I participated very little in the guidance development.
Joe Famulare will address the guidance, and he will take all of the
questions and all of the blame and some of the credit later on.

Okay. The primary objective of INDs as most of you know, but maybe many
or everyone doesn\'t know everything, in three phases of drug
development, and the focus of IND for Phase 1 is the safety issue. The
focus is on safety.

It\'s basically the first introduction of a new drug into humans. It\'s
intended to conduct some studies and evaluation of pharmacological
action of drugs, potential side effects, predict and evaluate early
evidences of effectiveness and so forth.

Phase 2, it\'s limited work control, and then you expand it into Phase
3.

We have some regulations. Some of you are becoming more familiar with
these numbers, and we\'ll throw more numbers at you today, 21 CFR 312,
and that\'s where many of these issues are outlined in our regulation.

As far as CMC requirement, and that\'s why I\'m speaking this afternoon,
is to indicate the following or share this important message: that the
amount of information needed in the filing depends on the stage of the
drug development. For Phase 1 INDs, the amount of information needed
depends on where we are with the study, the drug itself, some previous
studies, dosage for, route of administration, duration of the study, the
patient population, and if we know of some known risks.

All of these things will determine the amount of CMC information that
needs to be filed at Phase 1.

Talk about the drug substance, there are several attributes and several
quality attributes and information that need to be submitted, and it
varies from drug to drug, from study to study, but in general, we need
some description and some identification of the drug, how it is being
made and prepared, the analytical methods that are used for
characterization and/or assay, and a brief description of a stability
study, if any, at that stage just to assure that the drug would be
stable through that period of clinical trial.

For the drug product, we need to know the components of the drug
product, some quantitative description, the formulation, who\'s making
it, where are they, the method of manufacture, schematic description is
sufficient at this time. We are not asking for extensive batch records
or anything like that, analytical methods, and some information to
assure that the product is stable during the planned clinical study.
Some information about the placebo as well.

What\'s important here and if we\'re talking especially in the new
paradigm where the agency work was sponsors as partners in drug
development, is the degree and frequent communication between the agency
and the sponsors. And as some of you were here yesterday afternoon when
I talked about our efforts to reduce the number of supplements and the
number of our review cycles to save resources, these resources in my
mind should be allocated to facilitate such interaction. That\'s where
we are coming from.

We are not trying to cut the resources from people who are doing the
work now. We are trying to better utilize our resources to focus on
communicating early and more often with the sponsors to address all of
the issues.

This communication and interaction that takes place takes place prior to
the IND. There is a pre-IND meeting, and generally the focus of that
meeting is twofold. One is safety issues, and one is to look at the
potential of any clinical hold issues when I\'m sure that the clinical
study continue on, and if there is any potential that would raise issues
that may end up working the clinical study of hold. We try to identify
these issues early on in order to avoid stoppage of the clinical study.

The end of Phase 2 meeting is very important, and that\'s where more CMC
specific issues are raised. Pre-IND meeting generally focuses on filing
and format issues, and there are follow-up meetings and teleconferences,
fax and so forth.

What I\'m saying is here, even though I\'m just giving a brief
introduction to you, that if you look at this slide, there will be more
communication, but the frequency of communication is not as important as
the quality and the nature of communication, and that will be coming
soon.

Safety concerns. When we say that for Phase 1 IND, the CMC focuses on
safety. Our intention is to make sure through the information we have
there is an assurance of the identity, the strength of the quality and
the purity of the IND drug that\'s being used as related to safety.

For example, how the product is made, what are the impurities that could
be there, that may have been back from safety, the sterility concerns,
stability concerns. Profiles need to be sufficiently refined.

We are not talking here at this stage about setting the specification or
optimizing the preparation of manufacturing and proper characterization
of the drug as well, and that\'s all I have. thank you very much.

CHAIRPERSON BOEHLERT: Thank you, Mobeb, and then, Joe, you\'re next.

MR. FAMULARE: Thank you, Moheb.

And now to get into the issue here with discussing these Phase 1 INDs
that Moheb well introduced. I want to give you a little background as to
why we\'re looking at the Phase 1 of the INDs.

First of all, the Food and Drug Act, 501(a)(2)(B), requires all products
to be manufactured in accordance with current good manufacturing
practice, cGMPs, and in \'78 of course, we published the current version
of for dosage forms the good manufacturing practice regulations, but
they are primarily directed towards the commercial manufacturing of
approved and even drugs without approval, drugs and biologics.

And the preamble said that the cGMP regulations are applicable to the
preparation of any drug product for administration to humans or animals,
and that \"any\" of course is very broad and indicated FDA\'s intent to
public additional regulations specific to investigation of clinical
studies.

Well, we never did publish those specific regulations and over the years
there was a number of questions as to what is particularly applicable
for Phase 1, Phase 2, Phase 3 clinical trials. Methods are invalidated.
A lot of things aren\'t set. You\'re very much learning about the
process, although particularly as Moheb said in Phase 1, what you\'re
particularly learning about is safety is very much the emphasis.

And actually if you look at that quality paradigm that a number of
presenters have gone into here, we\'re really shifting it all on one
side in terms of the safety side, in terms of Phase 1.

At any rate, the agency had come out in 1991 with the guideline for
preparation of new drug products, but it did not adequately cover all of
the various manufacturing situations you might encounter in clinical
trials and really did not fully address the expectation that an
incremental approach to cGMP compliance is acceptable for
investigational products, given where you are in that stage.

And of course, that opened up a lot of questions and concerns. And just
to go back to Ajaz had a presentation on FDA\'s critical path
initiative. In looking at what are the number of new molecular entities
and treatments that are being approved, and it was disturbing that those
numbers were going down.

And, again, if there\'s one take-away from the initiative, the cGMP
initiative or quality initiative, as we like to refer to it as well, is
that we want to be at the forefront of innovating and allowing these
things to occur.

So, therefore, many of the concerns, particularly with Phase 1 INDs, and
what I\'ll be talking about is microdose and screening INDs, these very
early Phase 1 studies, there was inhibition because of the perception of
what part or does all of the cGMPs apply.

So what we have done is \-- we haven\'t done it yet because it hasn\'t
been published, but what we are doing is drafting a guidance about Phase
1 INDs and a complementing regulation to articulate FDA\'s intent to
implement an incremental approach to cGMP compliance for clinical
investigational products, recognizing that some controls and the extent
of controls obviously differ between investigational and commercial
manufacturing, as well as the various phases of clinical studies.

And we\'ve had a cross-agency work group with CDER, CBER, and ORA, and
I\'m just one member of the group. In fact, that group is meeting right
now as we\'re speaking. So I hope they don\'t change too much of what
I\'m saying here today.

But when I say \"cross-agency,\" it\'s not only been the GMP folks that
have been meeting. It has been the review folks on both the CDER and
CBER side, and one of the purposes of having Moheb explain the IND CMC
requirements is that there\'s a lot of complementary work that goes on
here in terms of the folks on the review side see some of these issues
as they come in for the IND and so forth.

And the other thing is to realize that we don\'t have a regular
inspection program for investigating or doing inspections of clinical
studies. Things are looked at on a for cause basis there.

So we wanted to develop a guidance and an approach which would be, of
course, risk based. How could we not be these days? But obviously not to
overuse the term, we wanted to have obviously \-- use the available
knowledge, and we\'ve had a lot of discussion about how knowledge is
transferrable. You know from other studies and other trial batches that
you\'ve done some knowledge. Take that forward, utilize that, and as I
said, in terms of the quality paradigm here the emphasis is very much
here on safety. So it\'s off balance.

And there\'s a number of examples of that quality paradigm. Just think
of all of them except Gary\'s yesterday which was blank.

And we\'re talking about, you know, what are some of the general cGMP
requirements? The thing that I spoke about earlier is in terms of Phase
1 this guidance will apply to investigation of new drug and biological
drug products during Phase 1 clinical studies. So this guidance that
we\'re planning to publish and we hope to publish it for the September
rollout of the GMP initiative will address Phase 1 clinical studies.

Along with this guidance we hope to issue a rulemaking pulling out Phase
1 from 210 and 211 so that there will be no lack of clarity, does it
apply or not, and what we will do is regulate directly off the statute,
501(a)(2)(B), as I mentioned earlier.

Dan can relate to that because that\'s what we do with APIs, but this
guidance will talk about our expectations, and we will specifically
address Phase 1 studies designed to assess tolerability or feasibility
for further drug development work.

Excluded are drug metabolism studies, structure activity relationships
and food interaction studies. The important thing is that we want to
provide direction for special product situations: microdose type
studies, and when you factor in other complicating things, such as
multi-product, multi-lot situations, and specific product types.

And we ran into a lot of these specific product types. We actually were
going to start out doing this draft guidance even less than Phase 1,
just sticking to these microdose type issues, but realizing that many
trials in the biologic realm really start out more broader in the scope
of Phase 1. So, therefore, we took all of these situations into account.

And as I said earlier, this is going to be a companion to other guidance
describing CMC information submitted in Phase 1 INDs, and will
complement what was said in the ICH 17A document about clinical
production of API materials.

We\'re going to discuss in this guidance when it\'s released as a draft
appropriate quality control standards, well defined procedures,
adequately controlled equipment and accurate recording of data
appropriate to this level of production. That\'s the key to remember as
I\'m talking about this.

So take away your thoughts of general cGMPs, 210, 211. We\'re trying to
scale it according to the scope of these operations.

An application that will lead to implementation of cGMPs which is really
consistent with good scientific methods because while some of this takes
place in R&D facilities of established firms, some of this is taking
place very often in laboratory settings. So we\'re trying to make a
correlation between cGMP here and, again, what would be a good
scientific method to do these studies.

It\'s going to talk about the use of available technology and resources
to facilitate product development, cGMP compliance, and lessen cGMP
burdens where it\'s very practical to do so, and it will talk about
disposable equipment and process aids, using prepackaged materials, such
as WFI, and contract manufacturing and testing facilities where it\'s
appropriate.

There will be discussion of the prevention of contamination and
cross-contamination and evaluate potential hazards regarding the
production environment and obviously carry over materials from previous
operation being removed.

So very, very rudimentary issues we want to talk about, and all of this
is very rudimentary material, but again, it focuses on what we see as
essential for a good clinical study, factoring off commercial
manufacturing.

Personnel would have the education, experience and training to do their
assigned functions. In terms of the quality control function, it should
be established for every producer of IND products have responsibilities
documented in writing, including the examination of components,
containers, closures, in-process materials, packaging and labeling
materials, review and approval of production and testing procedures,
acceptance criteria, review of completed production batch records for
release or rejection of each clinical batch.

Talking about the responsibility of staff involved in the production and
in operations with limited staff, QC function may be carried out with
the same person performing production with possibly periodic review by
another qualified person.

Facilities have to have adequate work areas for their tasks, appropriate
source water, and air handling and to cover any possible contamination
or cross-contamination issues.

Very basic information on equipment being in working condition,
calibrated and not additive or absorbative to the test material.

Be able to have control over components, acceptance criteria, use of
certificate of analysis, and enough documentation for trace back of what
that material was by lot number, et cetera, and supplier.

Enough production information so that the laboratory and production data
and equipment used and changes in microbial controls have been covered,
and the theme is to remember so that if you need to go back to this
information you can. Again, good scientific methods. Nothing earth
shaking here.

Laboratory controls such that test are conducted using established
written procedures under controlled conditions and using scientifically
sound analytical procedures, calibrated equipment, and be able to
initiate stability studies to support use of the product during the
length of the investigation similar to what Moheb would say.

Again, we\'re not talking about method validation or anything beyond
here; just very rudimentary information and documentation that\'s
needed.

In terms of the container closure and labeling, to make sure that proper
packaging is used to protect the product from alteration or
contamination throughout storage, handling, and shipping, and of course,
the importance of preventing or precluding label mix-ups.

And distribution should describe the transport of the IND product from
the point of production to obviously eventual use by the patient.

Record keeping should cover these general areas of equipment
maintenance, production, distribution, QC functions, and again,
component records. Really the basic rudimentary things you\'d need to do
to reproduce these issues if this is going to become a viable test
article and go further.

And we propose here a two-year retention period for the records after
approval of the marketing application or if in the case it doesn\'t get
that far at least after shipment and delivery of the last product.

Given those general GMP requirements, we realized that there are special
production situations and actually the screening and microdose INDs
where maybe just one person, one dosage is where we really started this,
because this is where there is a lot of throughput to develop. Where is
there going to be a candidate that will go further?

And, again, with a concern from institutions such as the National Cancer
Institute and so forth, and the concerns of liability under the whole
rubric of 210 and 211, we wanted to set out these clear but important
issues that have to be covered and separate away the issues that need
not be of concern and certainly not be an obstacle to going ahead with
these studies and find the new discoveries that are needed.

And, again, we go beyond that. Like I say, what our initial charge was
with the screening and microdose INDs to cover various situations in
Phase 1, such as multi-product facilities and the need of controls
there, the special situations that biologic and biotech products pose,
and of course, the needs and the importance, the safety aspects
associated with sterile and aseptically processed products.

The application of GMP controls to screening IND in microdose studies
should be proportional to the scale and scope of the operation, and
special provisions for lab scale production are provided in the guidance
with respect to the facility, equipment, and laboratory control.

So it\'s even drilled down a little bit more to more rudimentary
elements for these areas.

In multi-product facilities, the emphasis is that of an area a room is
used for multiple products, that one product at a time is produced in a
given area, and that there be appropriate cleaning and change-over
procedures to prevent carryover of materials, of contamination, or
actual product mix-ups.

For biotech and biological products, additional safeguards are discussed
or planned to be discussed in this draft guidance where some production
systems may warrant that, particularly sometimes to protect even the
personnel involved, pathogenic microorganisms, spore forming
microorganisms, live viral vaccines and gene therapy vectors.

You know, equipment qualification and controls in production should
assure the success of unit operations with safety related functions, and
again, with these type of products, there\'s concern for viral
clearance, virus toxin attenuation and pasteurization. So all of these
issues are touched upon in the guidance for these special situations.

Retain samples, offer an opportunity to go back and look to compare the
assurance of the product throughout the clinical development, and in
process testing and detailed records where necessary insure for Phase 1
products, you know, that you end up producing multiple lots. So this is
where we\'re starting to scale up now. You\'re going to need a little
bit more detail when you start getting into multiple lots.

Of course, for sterile, aseptically produced products, you know, we
thought about actually going to some references, such as USP and so
forth as to there\'s obviously a lot known about that, but on the other
hand, you actually listed some rudimentary bullets in the guidance that
are planned now in terms of having personnel trained in aseptic
techniques, using a proper laminar flow hood and controlling the
environment.

And that\'s pretty much where it ends, and to wrap up on that last
slide, the reason we didn\'t use some of the reference is because many
of them, again, are rooted in commercial manufacture, and we were afraid
we would put folks right back where they were.

So basically, to sum up, this guidance and this technical change to the
regulation to put Phase 1 IND studies under the rubric of 501(a)(2)(B)
and taking it away from the general GMPs should facilitate a lot of the
initiatives and the critical passion initiative where we\'re trying to
go to not be an obstacle to new discoveries; have clear expectations of
FDA of where you need to be at at this type of a study; and provide that
pathway.

Once we get through this process, we\'ll have obviously the draft
guidance will be open for comments. The next thing that we\'ll need to
address is clearer guidance, you know, stepping it up again because we
emphasize the step-wise approach for Phase 2 and Phase 3 studies. So
that will be a later part of our work.

Thank you very much.

(Applause.)

MR. FAMULARE: Questions later?

CHAIRPERSON BOEHLERT: No, we\'ll take questions now.

MR. FAMULARE: Oh.

CHAIRPERSON BOEHLERT: You know, any questions or comments for Joe and
Moheb?

As you heard, the committee is meeting now. So it\'s our opportunity to
have some input.

MR. PHILLIPS: I just have a few comments, observations. I think Moheb
and you have framed the situation every well. I\'m familiar with the
March of \'91 guidance that the agency issued, and it did, in fact, give
a lot of regulatory relief for the production of clinical supplies,
Phase 1, 2, 3.

Now, that\'s 13 years ago, and over that 13 years, I have personally
been involved with many audiences in the States, Europe, Asia and
interacted with groups who are involved in manufacturing clinical
supplies.

I made two observations. Here we are 13 years down the road and there
are still many people in that area who do not understand that that
guidance even exists.

Secondly, for those who do understand that it exists, the R&D people
always raise the issue that \-- and I think Dan alluded to this
yesterday \-- the R&D people always allude to their interaction with
their regulatory affairs counterparts, and the regulatory affairs
counterparts always say, \"Hey, we\'re looking at 210, 211, event though
that guidance exists, let\'s be conservative and ratchet it up a little
bit.

So with that as background, I think that you are making \-- you, the
agency \-- are making a rational approach to taking the Phase 1 study
out from under the 210, 211, and putting it under the legislative piece,
and I defer to David to define this, but 501(a)(2)(D).

The other thing that we have to look at in my opinion is patient safety,
maintain that safety, and I think in your proposal as you spelled it
out, you have dealt with all of those issues. Many of these products are
administered by the clinical pharmacologists as injections. If it\'s
going to be an injection, it should be sterile.

You\'ve dealt with that. Cross-contamination has been a traditional
problem. When you don\'t know too much about the manufacturer perhaps,
you\'ve dealt with that. So I think you made a rational approach in
moving in this direction. I would support it.

That\'s my comment. Thank you.

MR. FAMULARE: Thanks, Joe.

CHAIRPERSON BOEHLERT: Thank you, Joe.

Dan.

DR. GOLD: Yes, Joe, a couple of questions. Number one, if I recall the
guidance that is in effect or has been in effect, it requires written
procedures for the manufacture of the drug product, drug substance and
the drug product, even at Phase 1. Is that correct, Joe?

MR. FAMULARE: You\'re talking about the \'91 guidance?

DR. GOLD: Yes.

MR. FAMULARE: I\'d have to go back and look at that right now.

DR. GOLD: I think it does.

MR. FAMULARE: Basically what we\'re trying to do now going forward is to
have enough documentation to be able to repeat what you did.

DR. GOLD: Okay.

MR. FAMULARE: And that\'s the general direction.

DR. GOLD: This removes it. As I read it, this removes everything.

MR. FAMULARE: This would remove it out from under the rubric of that
guidance.

DR. GOLD: Right.

MR. FAMULARE: That guidance is going.

DR. GOLD: I\'m not objecting to that. I\'m just \-- okay? I just want to
verify it.

MR. FAMULARE: The problem was with that guidance it went across Phases 1
through 3, and there\'s a big difference between Phase 3 and a Phase 1
screening IND.

DR. GOLD: You\'re absolutely correct, and it does not distinguish
properly between the various phases, and that has been one of the
problems.

MR. FAMULARE: Right.

DR. GOLD: One of the real problems.

The other issue that I see is missing here and I want to make certain
it\'s deliberate is that there is no QA review or no quality unit review
of the documentation of the procedures and so on. Is that a very
deliberate approach by your group to remove those restrictions?

MR. FAMULARE: In terms of QA review of documentation and procedures,
even in 210, 211, it\'s under the rubric of QC, and the QC review \--

DR. GOLD: But QC \-- okay, Joe. I equate QC and QA.

MR. FAMULARE: Right, but QC is discussed here and will be discussed in
the guidance as a strong factor that you have to have QC, realizing that
that QC could be very limited in a small lab setting. So we do call for
that element of review. At least we\'re calling for that in the draft
guidance.

DR. GOLD: Well, I saw some of that in here, but I did not see a QC or QA
review of the documentation, and I just wanted to make certain that
that\'s a very deliberate posture on your part.

MR. FAMULARE: No, I believe that is an element in the guidance that
we\'re proposing.

CHAIRPERSON BOEHLERT: On page 7, the top slide in our handout, page 7,
the top slide, under the second solid bullet, the second item, review
and approval of production and testing procedures and acceptance
criteria. Is that what you\'re looking \--

DR. GOLD: Oh, yes, I\'m sorry. The third bullet, review of completed
production records.

CHAIRPERSON BOEHLERT: Yeah, right.

MR. FAMULARE: Right. Yeah, we did keep ‑- that\'s what I was saying,
that we did. That is a factor there, right. Okay.

CHAIRPERSON BOEHLERT: Other questions or comments?

DR. PECK: Yes.

CHAIRPERSON BOEHLERT: Garnet.

DR. PECK: Under the distribution record or distribution section, it
seems rather simple, and there\'s an element here of since it is Phase 1
that there is a group, a person, a clinician or whatever that\'s going
to do this and not necessarily going directly to the patient.

Is there a need to kind of further define this?

MR. FAMULARE: Well, part of it is that this is corollary over the other
300 regs that go to test article accountability. So there was a good bit
of coverage there. Our emphasis here was to make sure, for example if
the product needs to be at a certain temperature that it\'s shipped at
that temperature and maintains its quality from production to the actual
patient in the clinic.

So, again, because of its complementary nature, we didn\'t go into
certain details where we felt from the IND regs themselves. We also had
corollary coverage from some of these issues.

DR. PECK: Thank you.

CHAIRPERSON BOEHLERT: You said you\'re going to look at Phase 2 and
Phase 3 down the road.

MR. FAMULARE: Right.

CHAIRPERSON BOEHLERT: At what point in time are you going to do that
because as soon as this issues, the question is going to be, well, then,
what about Phase 2-3.

MR. FAMULARE: Well, Phase 2 and 3 will remain under 210, 211.

CHAIRPERSON BOEHLERT: Okay.

MR. FAMULARE: With what we would call appropriate discretion. Those
things that don\'t apply do not apply, and so forth, but our subsequent
guidance will clarify those issues, but we really saw this as the
bottleneck in an area to start. The time schedule I won\'t even begin to
discuss until after September.

CHAIRPERSON BOEHLERT: It sounds like it\'s very much later.

MR. FAMULARE: Well, I wouldn\'t say very much later, but you know,
we\'ll get this draft, comments, get this done, and that will be the
next step of the process.

CHAIRPERSON BOEHLERT: Other questions or comments?

Dan.

DR. GOLD: (Speaking from an unmiked location.)

MR. FAMULARE: Thank you, Dan, and when I say \"thank you,\" I mean it\'s
not for me. I\'m only just one member of this group. We don\'t really
have a head to this group, but we have a group of us working together on
it. So myself, Chris Joneckis, Gurag Poocheekian, and there\'s a number
of folks from CBER and one person out in the audience, Chiang, has been
part of the group.

So, yeah, the group has really put their best heads together and
experiences to work on that.

CHAIRPERSON BOEHLERT: Last chance. If not, thank you, Joe and Moheb. It
looks like you have general support from the committee on this guidance.

Okay. Time to change gears again and look at applying manufacturing
science and knowledge in a regulatory horizon when you talk about PAT.
Chris Watts or Ajaz?

DR. HUSSAIN: As Chris comes to the podium, I\'d just like to sort of
give a context and sort of position the discussion we\'ll have with
Chris on comparability protocol and so forth.

One of the aspects I\'ve wanted to sort of point out with these
presentations is that we\'re moving into a new paradigm. We\'re moving
to the desired state, and not only will Chris provide you an update on
what is happening in the PAT initiative itself, but also I requested him
to emphasize a team approach to review and inspection, and that is the
heart of the PAT initiative, is the team approach to doing business, and
to emphasize how we are finding new ways of minimizing, say, the
supplement process or minimizing the need to have a prior approval
supplement as the only means of making decisions.

So I think there are elements of what Chris will talk about which will
highlight this, and the second talk after Chris will be on comparability
protocol, and it\'s a summary of all the comments we have received on
the drug guidance that was discussed before this committee, and our
current thinking. Steve Moore will make that presentation, and Moheb is
working very closely with Steve to sort of move that guidance forward.

The struggle in that is I think we took a guidance which was being
developed before we defined the desired state. That\'s the challenge,
and I think we\'re trying to bring the desired state element into that
guidance, and it has not been easy.

And I think one way, in my concluding remarks I think I would like to
sort of say that, I think. Decisions that I think after this meeting
you\'re making is that we will focus every effort from now on on the
desired state and not really worry about the past.

and I think this is a sort of guidance which we are stuck in the middle
looking at the old state versus the desired state, and we are struggling
to sort of bring that forward, and I think we will come of that approach
to say that we are focusing more on the desired state from now on and so
forth.

So you\'ll see that struggle, and Jon Clark, who co-chairs, changes with
our private approval supplement group with me under the GMP initiative,
will share some thoughts on how we want to proceed.

So that\'s the context of the discussion this afternoon, and I hope that
you\'ll continue the discussion that we had yesterday and keep giving us
ideas and suggestions and so forth on how bet to sort of approach that.

Thanks.

DR. WATTS: Thank you, Ajaz.

I want to thank the committee for allowing me a few minutes of your time
today to talk about what we\'ve done and plan to do with PAT and really
talk about primary this engine that we have at the agency, the way we
refer to it. And I stole that term from Ajaz, \"the engine for
success,\" and I\'m a firm believer that the team we\'ve established
within the agency, the reviewers, compliance officers, and the
investigators from ORA, are really going to be the engine that drives
the success of the PAT initiative within the agency. And that\'s really
going to be the focus of how we manage review and inspection process for
PAT as we move forward.

So just a very brief outline, and a few questions I\'d intend to answer
with my presentation. I do want to focus on the benefits of PAT and how
there may be other approaches aside from supplements into implementing
PAT for the industry.

So with that, a slide that many of you have seen on several occasions,
probably one too many times for some of you. The definition that we came
up with for PAT, and it was discussed at length at the PAT subcommittee
of the Advisory Committee for Pharmaceutical Science, a system for
designing, analyzing and controlling manufacturing through timely
measurements of critical quality and performance attributes of raw and
in process materials and processes, and I think the key here is this
little three-letter word. Frequently that replaced with a two letter
word that creates a lot of confusion. The two letter word is \"or,\" and
a lot of people read PAT as just process monitoring, and the control is
frequently left out.

But I want to emphasize that we\'re really talking about a complete
system for designing, analyzing, and controlling the manufacturing
operation. When we talk about the analytical portion of PAT, process
analytical technology, the focus tends to be on the analytical
chemistry, and albeit that\'s an important part of what we\'re talking
about with PAT, that alone is not the focus. When you see the term
analytical and PAT, I\'d like to have people think more along the lines
of analytical thinking rather than just analytical chemistry. You have
to consider not only the chemical, but the physical, the
microbiological, the mathematical and risk analysis. All of that has to
be considered in an integrated system rather than just focusing alone on
the analytical chemistry.

So with that background and the definition of PAT, how does that link to
what we\'ve been talking about with process understanding? The term is
floating, tossed around quite a bit. The focus is process understanding.
It\'s really what we\'re focusing on with PAT, but what does that mean,
you know, process understanding.

What we allied in the guidance was that a process is that a process is
considered well understood when all critical courses of variability are
identified and explained. That variability is managed by the process and
product quality attributes can be accurately and reliably predicted.

I want to walk through a very quick example later on to give you
specifically what I\'m talking about with those accurate and reliable
predictions, and we really feel the ultimate is that the accurate and
reliable predictions reflect a high degree of process understanding, and
of course, if a process is well understood, we\'ll assume that that then
imposes a lower risk category when it comes to producing a poor quality
product.

So with that, I do want to focus much of the discussion on the team, and
I do want to emphasize that the initiative is cross several centers
within the agency, the field, ORA, CDER and CVM, and you\'ll see the
steering committee. These are the senior managers within the agency who
are really pushing the direction that we\'re going with PAT or setting
the course I should say, and you\'ll see ORA, the Center for Veterinary
Medicine, and CDER, but you know, it\'s not just CDER, Joe. It\'s
obviously from the Office of Compliance, Office of Biotechnology
Products, which is whether Keith Webber is from. Frank is from the
Office of Generic Drugs, and Moheb is, of course, from the Office of New
Drug Chemistry.

So even though there\'s a lot of CDER representation, it is CDER-wide,
biotechnology products, generic products, the new drug products, and of
course, the Office of Compliance.

And I really want to highlight this team that we set in place that
we\'re really going to manage the review and inspection process. These
team members are from the field, from the center, from the Office of
Compliance, from the different review divisions within Generic Drugs and
Office of New Drug Chemistry, and they are what we refer to as the
engine. This is the \-- I think everything is the engine for success
here, but these are the people who are going to be managing the review
and inspection process, the interaction, if you will, with the industry.

And the training program that we went through, we first began with a
team building exercise, and I think that was very important that we
could all get together and just begin to open the communication channels
with one another because it may not be all that often that people from
the field communicate with people in the center, and just to break down
those communication barriers and get more of a personal interaction with
one another I think was very important.

And just briefly, the training session, we had two didactic sessions,
one that began at the agency where we focused on several different
technical aspects that we went through, that we felt were important
background information for people who were going to be responsible for
review and inspecting these facilities and these applications, and of
course, we went through three practicus at the University of Washington,
Purdue, and the University of Tennessee.

And there we actually focused hands on, if you will, on training to see
what the industry may be looking at or what the industry is actually
looking at in terms of implementing PAT.

So as far as the training program, we have completed the initial
training program. We\'re currently doing a lessons learned, and I do
want to emphasize that we have every intention and, quite frankly, we
are moving forward with the continuing education effort because although
in many aspects the initial training program was very successful, to
think that we have covered all of the bases that we need to cover in
terms of being sure this team is well prepared and stage prepared for
what may come to us in the future, continuing education is going to
continue to play an important role there.

So along those lines, we want to involve this team that we have in place
right now in the next training for the people that we have coming around
for the next round of training with the PAT team, and they were also
heavily involved in the guidance finalization process, finalizing the
PAT guidance, the team from ORA, you know, again, the Center for
Veterinary Medicine, Center for Drugs, were heavily involved in
reviewing the draft guidance, the comments that came in, the public
comments that were submitted to the docket, and the process as far as
finalizing the draft guidance that we\'re going to issue.

What I really want to focus on is this team approach to review and
inspection, and I can\'t emphasize enough that it really is a two-way
street. A lot of people see it, and they think that the people who are
in the center and review the applications are going to have some input
into the inspection process.

While that is very true, there\'s also the other direction of the
Street. The people who are responsible for the inspection process will
also have some input as to what is said about the review of an
application or a supplement, if you will, that may come into the agency.

So we\'ve all heard about the 1,700 some odd supplements that the Office
of New Drug Chemistry gets on an annual bassi, and this is, indeed, one
route for implementing PAT within your company,b ut I want to highlight
two other options or alternatives, if you will, for going forward with
PAT implementation, and these are in the draft guidance, and one of
these is that you can implement under the facilities or the company\'s
own quality system, and following implementation within the company\'s
own quality system, an inspection by the PAT team or the PAT certified
investigator may follow if the team deems it\'s necessary.

Another option following na inspection, the FDA certified or the PAT
train and certify an investigator, can approve this process or the team
as a whole can approve this process.

And I really want to highlight that outside of supplements or
submissions such as a comparability protocol, there are other avenues
for implementing PAT within a specific company or organization, and
these are only a couple that we chose to highlight within the guidance.
There are many other options that a company may have if they want to
come forward and say that this is the approach that we think is
appropriate for what we\'re trying to do here. We want to just stick it
in our annual report. You can inspect it when you get here if you feel
it\'s necessary.

There are many other options that a company can consider rather than
coming forward with the supplement or comparability protocol, and I
really just wanted to get that point across because the team as a unit
will manage this when the inspection is taking place or when the review
of a supplement or application is taking place. It will be the entire
team that\'s responsible there. So it\'s not just a submission that has
to be made to get approval to implement PAT within your organization.

So a very quick example. I want to walk through a quick example of how
regulatory relief may come.

This is an existing title production process, if you will, the typical
raw material dispensing, blending. You\'re going to mail after blending.
I\'ll blend it again. Typically you\'re going to include your lubricant
there and then go straight to compression. This is a direct compression
process, and typically of the tests that are done, the dissolution and
content uniformity tests are done at the compression stage.

And we\'ve heard many times this tends to be in product focused or the
testing to document quality phase, if you will. So if we think in terms
of the PAT approach, if you think about that example of the process that
I gave you, the PAT approach, if we want to focus again, the emphasis
there is focus on the process understanding. What parameters are
critical to the quality of this product? How do they affect quality or
why do they affect the quality of this product?

That begins to get us down the road of answering those questions. We
begin to understand how and why this impacts our process. So we get that
understanding. This can be done, just one example, experimental design,
and then how do we analyze these parameters. We\'re talking about on
line analysis with PAT. How do we analyze these parameters? Remember the
definition for PAT, design, analysis, and control. Once we pick what we
feel is the simplest \-- and I always emphasis to keep it simple \-- the
simplest technology, not necessarily the most expensive or newest out
there \-- the simplest form that\'s going to allow me to analyze and
control the same parameters and design analysis and control. We
implement our control strategy.

That\'s it. If we\'re focusing on process understanding and we think
about the definition of PAT, design analysis and control, how do we
control this process?

So the example that I gave you, and again, hypothetical example, if we
do an experimental design and we see that the level of disintegrate and
the particle size of the active are the critical attribute when it comes
to meeting my desired product quality attributes that I\'m looking for
in the table that I produce.

For example, if it\'s you know, a pain reliever, you want your relief
right away. You don\'t want to have to wait, you know, an hour or two
hours to get relief from your headache. You want the product quality
attribute there. Us as consumers would say I want my relief immediately.
I don\'t want to have to wait two hours for my headache to go away, for
example.

So the critical attributes here are the disintegrant level and the
particle size. So if we move forward to an example of a PAT approach, if
particle size is critical, in order to analyze it and control it within
the manufacturing process, we first have to begin to understand, well,
what\'s going into the process.

If we understand the particle size distribution of our active is before
we go into the process, then we can begin to tailor our process to
control that particle size distribution.

So one example of this comes from AstroZeneca is as they\'re dispensing
the material into their blender, for example, they\'re analyzing this
material as they\'re feeding it into their blender. So they know what
the particle size distribution is of this material before we even begin
to blend.

So with that in mind, be can begin to control the blending operation. So
if we have, for example, an analyzer on our blending operation, that\'s
not only going to tell us when we reach a homogeneous mix because
remember the other critical variable that we had was that we needed an
even distribution of our disintegrant. It\'s going to cause our tablet
to explode, if you will, when we take it, and we get the active
ingredient available for absorption and relief right away.

So not only can we control the disintegrant mix, but we can also be
looking at the particle size distribution as we\'re going through, and
this will allow us to begin to build some of those predictive models
that will allow us to feed forward into this is the particle size coming
in. This is my particle size while I\'m blending.

So if you think of the initial process that we had, the raw material
operation, blending, milling, and blending, if I know my particle size
distribution coming in, I\'m blending. I know what my particle size
distribution is coming out of my blender. I may not need to blend every
single time. I may have the particle size distribution that I\'m looking
for at this stage.

And we don\'t want this process to be frozen in time, if you will. If
you don\'t need to mill, you already have the particle size distribution
that you\'re looking for. Skip that milling stage. go directly to
blending your lubricant and move forward to compression because you\'ve
already met your desired particle size distribution. That milling stage
adds no value whatsoever when it comes to meeting the desired product
quality attributes of your product quality attributes of your product.

So if you think about the PAT process that we have now versus what we
had with the original tablet production, we\'re beginning to understand
what the distribution is, the particle size distribution of our
material, the attributes of our raw material coming into the process.

We control as we\'re moving forward in this operation. We can begin to
build predictive models. If we know what the particle size distribution
is coming in and we know, for example, if we\'re right on the edge of
the distribution that we need, that\'s critical for us to meet our
desired product quality attributes, we may be able to blend for just a
little bit longer and meet that particle size distribution so that we
don\'t have to go forward with the milling step. We can skip that
milling step altogether and improve our efficiency, right?

So these predictive models will tell us, all right, if I have this given
particle size distribution, I can predict that I\'m going to stop my
blender at Time X. And while I\'m doing my blending operation, my
control strategy actually shuts down my blender at the time that I
predicted. What is that? That\'s the process understanding. Remember the
accurate and reliable predictions? That reflects a high degree of
process understanding.

So if we can convey that in some way to the agency and say, you know, I
understand my process. I know what particle size distribution I need,
and this is how I control it with my process. If I need to mill it, I\'m
going to mill it. If I don\'t need to mill it, I\'m not going to mill
it, and I\'m not going to send the supplement to you to tell you why
I\'m not milling it because you already know.

We do away with some of those 1,700 supplements that Moheb has to deal
with on an annual basis.

So thinking about that example, how is PAT benefitting us here? We no
longer have this laboratory determination of blend homogeneity if that
is done or the particle size distribution. We\'re doing it. We\'re
actually controlling it while we\'re manufacturing our product. We\'re
blending it to an end point rather than to a specific time that we
validated when we did our three validation batches.

We\'re milling only if we need to. If we don\'t need to mill it, skip
it. I\'m not going to do it this time. And This begins to open the door
for us to real time release because we\'re assuming we\'re building in
quality as we\'re manufacturing the product. We don\'t need to test it
at the end whenever we get our tablets out of the compression or out of
the tablet press. We don\'t need to test those every single time.

But when we do, if and when we do, we\'re actually validating that our
process is under control, that the control strategy that we have in
place is, indeed, functioning as it should.

Optimization, this allows us to optimize the blend time. If you think
back, if we\'re only going for a specific period of time rather than
till an endpoint, there\'s not really a lot of flexibility in that time
point. So you can begin to optimize your blending operation to meet not
only homogeneity, but maybe to meet that particle size distribution that
you\'re looking for so you can avoid going through that non-value added
milling step.

And, again, this would begin to build in these feet forward models for
blend characterization because we have to begin thinking of the blending
operation. What we have is not only an output. It\'s actually an input
into the next unit operation that we have.

The material that we get from our blending can go into our milling
operation or it may, indeed, be sufficient enough to go straight into
our next blending stage and straight to the tablet press.

So how does this reduce the regulatory burden? Questions that we get all
the time. The process is no longer, borrowing a phrase from the [Wall
Street Journal]{.underline}, it\'s no longer frozen in time. We actually
have free rein to avoid that milling step if we have to.

No supplement for a process change. I don\'t need to mill. I\'m not
going to send a supplement to you that tells you I\'m not going to mill.
I need to blend for a little bit longer this time. I\'m not going to
send a supplement to you that tells you I need to blend for a little bit
longer. You already have demonstrated that process understanding.

And a team approach. I really can\'t emphasize this enough. It\'s a team
approach through review and inspection. So when the inspector shows up,
they\'re on the same page was the reviewer who looked at your
supplement, if one came in, or they have a resource that they can use
while they\'re on site. They know people who may be on the team, who may
be able to answer a technical question that they have about the process
that you have in place.

And during that inspection that\'s your summary basis for approval. So
with that, I hope I gave you really what we\'re talking about with
process understanding and PAT. The inverse relationship between the
level of process understanding and the risk of producing a poor quality
product, if the process is well understood, there are obviously less
restricted approaches to \-- less restrictive regulatory approaches to
manage change, and if we focus on process understanding, we can
facilitate risk managed regulatory decisions and innovations.

And this can really lead to the several options for implementing. We no
longer need to go through the submission or supplement process when it
comes to making a change to our process. We\'ve demonstrated that it\'s
well understood. We know what the impact are and any changes that we
make. So we can go ahead and move forward with those changes.

So I hope that was a good example to really emphasize what we\'re
talking about with process understanding and PAT and how it may be a
benefit to the industry.

Very briefly, where we\'re going with PAT, we are finalizing the
guidance. I spoke to you very briefly about how the entire team was
involved in that process \-- Ajaz mentioned this at the last advisory
committee meeting \-- expanding the scope of PAT to include the Office
of Biotechnology Products, and quite frankly, the reason OBP wasn\'t
included int he draft guidance is OBP didn\'t exist when we were coming
up with the draft guidance.

Continuing education and training of FDA staff, that\'s going to be, I
think, the oil change, if you will, to the engine that\'s driving the
success within the agency.

ASTN technical committee, Del Marlowe, the agency standards coordinator,
spoke to you very briefly about that yesterday, and of course, research
continues to play an important role with what we\'re doing in terms of
developing the sound scientific basis to the policy that we develop and
the training that we conduct within the agency.

So with that, I\'m not going to take any more of your time, and I guess
I\'ll turn it over to Steve or Judy.

CHAIRPERSON BOEHLERT: I would just ask if there are any committee
members that have specific comments on the PAT presentation. Yes, G.K.

DR. RAJU: So, Chris, you\'re saying if you ‑-

DR. GOLD: May I ask a question?

CHAIRPERSON BOEHLERT: G.K. is first and then you can.

DR. GOLD: I\'m sorry?

CHAIRPERSON BOEHLERT: G.K. got first and you\'re second.

DR. GOLD: Okay. I\'ll wait second.

DR. RAJU: So, Chris, you gave a really nice example. So if somebody
actually independent of any bioequivalence and despite the SUPACK
guidances and their categorization, I mean, exactly that submission to
you without any connectivity back to the patient in terms of
bioequivalence, that would be within your mandate to say it\'s okay
without any supplements, within the mandate of the PAT group and the
guidance?

DR. WATTS: Well, I don\'t want to say that it\'s \--

DR. HUSSAIN: No, I think the context of the no supplement, the changes
for the existing product right now, the changes in the specification,
you have no option but to have a supplement process.

DR. RAJU: But if there is no change in specification; only the process.

DR. HUSSAIN: The way it is a quality submission commitment, it is a
change. It is a change today. So what we\'re saying is that the team
approach to review and inspection opens up new avenues for allowing some
of this to happen, but that is only in the context of process
understanding.

When that has been shared, and that goes to the design space that we
discussed yesterday. So what it means is the design of experiment mark
is actually based on our own lab data. If the design of experiment that
Chris showed, the chart, we actually had the questions you\'re asking. I
mean, those were the critical factors that affected resolution and so
forth.

That\'s the knowledge base under which we can start moving in that
direction.

DR. RAJU: So you still have to bring that in.

DR. HUSSAIN: Oh, yes, absolutely.

DR. RAJU: But you don\'t have to bring that in from a patient, inside a
patient point of view. You can do that totally from the [in
vitro]{.underline} information.

DR. HUSSAIN: It will depend on exactly what your process understanding
is, what is critical what is not critical. If it is critical enough for
the patient, then the biostudies could be part of that.

CHAIRPERSON BOEHLERT: Okay. Dan.

DR. GOLD: How does your work related to the requirement for stratified
sampling?

DR. WATTS: I think that\'s just an example, if you will, of assuring
blend uniformity.

DR. GOLD: I\'m sorry. I didn\'t hear you. Say again.

DR. WATTS: That\'s just an example of how you can assure blend
uniformity. That\'s not the only way. There are many other options for
assuring blend uniformity. That just happens to be one that was
discussed and came forward with the PQRI.

DR. GOLD: So does this mean that if a firm goes this route they will not
have to justify what would happen during interruptions, refilling, or
change in hopper, for example, or taking samples during the changing of
a hopper? Is that what I\'m hearing?

DR. HUSSAIN: No, I think you\'re missing the point completely.

DR. GOLD: No, I don\'t think I\'m missing the point. I\'m trying to
clarify the point.

DR. HUSSAIN: No, no, you are because you requested the stratified
sampling, which is testing ten tablets in a stratified way. I think the
risk of that is much higher than the risk what you\'re talking here
because no in-process controls you. No controls on your incoming raw
materials. You\'re making a decision on ten tablets, although in a
stratified way

DR. WATTS: If you look at the definition of PAT, a system for designing,
analyzing, and controlling. If you\'re just looking at tablets, there\'s
no opportunity to control. It\'s too late. You\'ve already made them.
All right?

DR. GOLD: No, I fully appreciate the difference in technology. What I\'m
asking is from a compliance point of view, if we proceed this way, does
this mean that a stratified sampling is not a requirement, a compliance
requirement?

MR. FAMULARE: You know, we\'re talking here about a whole control system
in real time release. So any sampling and testing that\'s done could
only, as Chris described, validate the process. You\'ve already done
what you have had to do before you even get to stratified sampling.

So they\'re two completely different things. You know, it\'s apples \--

DR. GOLD: So you mean we still would need to verify stratified; you are
introducing a new product?

MR. FAMULARE: No. You could. You could. Let\'s say you came in with a
brand new PAT application or you supplemented an existing one for your
product specifications. Your release criteria could be based on the PAT
controls, the fact that through these controls you\'ve come out with the
product that\'s meeting its desired quality specifications.

DR. HUSSAIN: The key here is this in the sense I think, for example, if
you have a scenario where there is a risk factor of changing a hopper
and potential segregation after that, in that case there\'s a different
application. It could be an on-line assessment on every table. So
instead of doing ten tablets, let you might be assessing thousands of
tables.

I mean, so the sample size goes up dramatically of what you evaluate
here. So the decision is not based on ten tables.

DR. MORRIS: Just a comment, and I guess the way I think of it is that
you\'d be doing the establishing of the criteria during development. so
by the time you got to the level of implementing the process of
understanding base to monitor and control, you would already know that
the release specs based on the PAT approach would have been
substantiated.

So if you have segregation in a hopper, you might need another sensor if
you have a model that tells you that that is a critical control point to
monitor, is the way I think about it. I don\'t know.

DR. GOLD: So that are you saying that when we introduce this we would
still have to do those evaluations initially, for example, on changing
hoppers.

DR. HUSSAIN: Well, I mean it\'s pure and simple product development
studies. You have to do what you have to do.

DR. WATTS: You can\'t do a DOE without defining the extremes.

DR. HUSSAIN: Exactly.

DR. GOLD: All right.

CHAIRPERSON BOEHLERT: Any other questions or comments?

DR. SINGPURWALLA: Yeah. How did stratified sampling get into this
picture?

DR. HUSSAIN: Don\'t bring that up. That\'s not the topic.

DR. SINGPURWALLA: No, no. Dan asked the question, and you know, I feel
obliged to, you know, think about it. So how does stratified sampling
get in this? Did you mention the word stratified sampling?

DR. WATTS: No.

(Laughter.)

DR. GOLD: No. I am bringing up stratified sampling because currently
it\'s a requirement in the absence of PAT, is it not?

MR. FAMULARE: It\'s not a requirement.

DR. HUSSAIN: It\'s just one way of doing things. It\'s not a
requirement.

MR. FAMULARE: It\'s a guidance. In fact, that guidance even borrows some
of the language from the PAT guidance that this is just one way to go.
You don\'t have to go this way.

DR. GOLD: Well, you can offer an alternative, but you still have to be
able to prove that you have uniformity through the various changes that
occur through the processing, correct, Joe?

MR. FAMULARE: You don\'t even have to go as far as that last statement.

DR. GOLD: Okay.

MR. FAMULARE: You want to have uniformity, period.

DR. GOLD: Yes.

MR. FAMULARE: In terms of changes, you know, it\'s one thing that you
identify your critical control or weak points. It\'s another thing to
have a deviation that was unexpected. So, I mean, the whole point of the
blend uniformity, the stratified sampling or one of the main points was
to take care of sampling bias. I mean, that wasn\'t focused on if you go
back to that guidance, what are your weak points. It was really focused
more on sampling bias and the limitations of that.

DR. SINGPURWALLA: Can I articulate on this? I think I see the point that
Dan is raising and the presentation that you made. I hope I\'m correct
in articulating it.

I think what you are talking about is continuous monitoring and control,
as done by control theorists.

DR. WATTS: Right.

DR. SINGPURWALLA: What Dan is talking about is when you do not have
continuous monitoring and you do not have continuous coupling. You do
sampling and to account for the biases, you may want to stratify.

And I think he is monitoring continuously. So from one point of view I
would look at his presentation as something in control theory; is that
correct?

DR. WATTS: Absolutely.

DR. SINGPURWALLA: It\'s process controlled, control theory, and somehow
you threw in design of experiments because most chemists and chemical
engineers and pharmacists like design of experiments. So somehow it\'s
kept in.

(Laugher.)

DR. WATTS: This is the point, but just because you can\'t control
something doesn\'t mean you have to. Moisture, for example, if it
doesn\'t matter if I have between two and 20 percent, it doesn\'t affect
the performance of this granulation in this process or the stability of
the product. Why do I need to control it to 2.5 percent, for example?

DR. SINGPURWALLA: (Speaking from an unmiked location.)

DR. WATTS: To determine what\'s critical.

DR. SINGPURWALLA: Right to determine the critical points. Yeah, that\'s
fair.

CHAIRPERSON BOEHLERT: Okay. Are we ready to move on?

DR. HUSSAIN: I think so, but I think this is an interesting challenge.
You always keep going back to the past. I\'m not looking to the past
anymore for that. We need to come and talk about the new stuff before we
let this \--

DR. GOLD: Well, I\'m very happy to talk about the new stuff. I\'m just
afraid that we may also be looking at some of the old stuff during the
way, on the way.

CHAIRPERSON BOEHLERT: Next up will be Stephen Moore to talk about
comparability of protocols.

DR. MOORE: Thank you. I\'d like to give you an update on the
comparability of protocols and an update on the progress of the
guidances and the revisions of those guidances.

And just to cover today the general topics, definition and general
aspects of the probability protocol, regulations that we have published
on comparability protocols, the draft guidances that are in the works,
and also talk about the public comments and give you some highlights
there that we received in the docket, and spend most of the time on our
current thinking.

A definition of a comparability protocol, it\'s a comprehensive detailed
plan that describes the specific type of proposed change, the tests and
studies that will be performed, analytical procedures that will be used,
and the acceptance criteria that will be achieved for the purpose of
demonstrating that a change \-- that there is a lack of an adverse
effect on the product quality for that change as it may relate to the
safety and effectiveness of the drug product.

And I\'d like to say that this is a basic definition of the
comparability protocol that stems from the regulation, and a
comparability protocol can be much more, as you\'ll see later.

A comparability protocol, some of the general aspects that should be
well planned in advance. It should be scientifically and technically
sound, that is, that is based upon knowledge and understanding, And I
will discuss that in more detail in further slides, and it should be
adequate and kept current to implement the change and comparability
protocols are drug process controls and change specific.

This is the regulations that have been published on comparability
protocols. Actually the regulation first came into effect in 1997 for
biotechnology and biological products, and most recently in April is now
in effect for a chemical entities.

And the regulations state that what must be in comparability protocol
and in accordance with that definition that I just gave you, and it also
says that a comparability protocol can be submitted in an original
marketing application or it can be submitted as a prior approval
supplement.

And it says that changes to the protocol have to be submitted as a prior
approval supplement, and that FDA will review this protocol and if
justified, can designate a reduced reporting category for that change
under the protocol.

These are the draft guidances that are up on the Web. There\'s two of
them. They are companion guidances, and the first one applies to the
chemical entities, drugs and includes synthetic peptides drug products,
and that one was put up in February of 2003.

The other one covers biological and biotechnology products, which went
up a few months later.

The public comments are under review now in the comparability protocol
working groups and for final publication of these guidances.

And I just wanted to give you some of the highlights of these guidances,
and what I\'ve done is excerpt this and paraphrased this for brevity to
give you more or less what is the message we\'re hearing from the public
comments.

And these I\'ll read off: the efficient use of comparability protocols
should provide regulatory relief by expediting review and approval of
post approval changes. And I think we all agree with that.

And many changes are not anticipated at the time of filing a marketing
application. We are seeing mostly changes are comparability protocols
filed in prior approval supplements. There have been some submissions in
the original marketing applications.

And the commenters in the public documents say that the level of
specificity requested, and they\'re talking about what was in the
guidances, may define the protocol so narrowly as to diminish its future
usefulness.

And here what we are taking this and what we\'re hearing is that
protocols need to be made more flexible in order to be made more useful,
and that the key to the use of a comparability protocol is the
availability of sufficient manufacturing science data to demonstrate an
adequate understanding of the control process controls and we can\'t
agree more with that.

Continuation on the comments, they wanted us to clarify what we meant by
a comparability protocol for changes of a repetitive nature. What we
meant was that comparability protocol was for repetitive use or could be
used repeatedly, and I think that\'s very important because this kind of
protocol is very valuable. Once we approve it, a company can use it to
make changes, and that regulatory relief that\'s granted initially can
apply to changes into the future, and we won\'t have to go back and
review their plans again.

And they asked us to provide examples for reduction in a reporting
category from a prior approval supplement down to annual reporting. This
we are working on, and I\'ll show you some more details later.

They also asked for modifications to a comparability protocol. Can we
find ways to lower that into categories other than prior approval. As
the regulations stated that those modifications would be for prior
approval, but 31470 and others, the companion one for biologics also
says that we can do this through guidances.

And another point, the cGMP aspects of post approval changes should be
addressed and we are doing that.

Also, finally, we applaud the FDA for its efforts, and we do appreciate
that feedback from the commenters to the public document.

And now I\'m going to turn to the current thinking on comparability
protocols. Essentially we see it as two basic kinds of protocols and
this is from also built upon our experience of the kinds of protocols
that have been submitted.

One kind is a single use comparability protocol, and these are designed
to make a specific one time change. Usually these are for rather
complicated changes.

And another type of protocols that I was talking about, the repetitive
use comparability protocol, that is designed so it can be used to make a
specified type of change and changes within that specified type can be
made repeatedly and over time.

Some more aspects, details about single use comparability protocols that
could cover a single change or multiple related changes, and we have
seen examples of both.

And for multiple related changes, what we are finding is that there is
not always a distinct discrimination about how they are going to
evaluate those individual changes. So we in the guidance are going to
make that clear, and that each of the individual changes should be
clearly defied how they\'re going to assess them, and also the combined
effect of all the changes if they\'re making multiple changes should be
assessed.

And there are many, many examples of what single use comparability
protocols could be used for. I mean, essentially they soul be for any
changes in the drug substances, drug product manufacturing process.

And there are some exceptions, and I\'ll get into that later, about what
might not be appropriate in a comparability protocol, and they can be
for changes in scale and multiple related changes that are related to
changes to scale, and this may also common occur at different
facilities.

Aspects of a repetitive use comparability protocol. Generally these are
more narrowly defined, and the concept here is these are modular in
nature, and we find that boundaries need to be established so that we
are certain that the comparability protocol remains valid over the type
of change that is defined.

For example, if you had a change for differences in scale, you might
want to set a boundary of half X to ten X. Well, inside that range you
could be able to freely make those scale changes. Outside the protocol
may not be valid, and we need to know that during the review process so
that we\'ll be sure that we\'re looking at all that needs to be looked
at.

And in general these multiple changes are usually comprised only of
subcategories of the specified type of change, and I could explain that
better by examples.

The classic case of a repetitive use protocol, and these have been used
for a long time, are container closer system changes in which we have
show equivalency of various container closure components.

And also we want to expand this idea to changes within a unit operation,
and you may be able to change the conditions or the parameters of that
step, and once that is approved during the protocol, you may have free
use, the ability to change that without regulatory oversight.

And just briefly going over what the advantages are and disadvantages
are, I think many of these are already apparent, and to industry the
main advantage and the original intent of developing regulations and
guidances for the comparability protocol is that that would help shorten
the time length for distribution of product and reduce the filing burden
for commonly made changes.

And so while you\'re waiting for FDA to approve, and now it\'s four
months for a prior approval supplement, if we can get the plans approved
ahead of time, you can make the change under a greatly reduced reporting
category and burden.

And the disadvantages, of course, I mean in all cases the risk of an
adverse effect is not eliminated, but we intend to say that the
comparability protocol should be constructed in such a manner that if
during the implementation of a change is found that there is an adverse
effect, the protocol would be strong enough, rigorous enough to catch
that and would stop the implementation.

The advantages or disadvantages to FDA. We\'re seeing, hopefully as
being responsive, in finding ways to reduces manufacturer\'s down times
is why they\'re waiting for a prior approval, and we are hoping that
this many reduced the overall number of post approval supplements.

One advantage is that unless the protocols were remained in the original
application, this is going to increase our work load of supplements
because not all cases would we be able to downgrade the change to annual
report, and I\'ll get into that later. It would be related to complexity
of change and how much information is provided with the protocol.

So it\'s possible that I could increase our work unless those things are
considered.

And what might be appropriate and what might be not appropriate under
comparability protocol. We think it\'s appropriate under a comparability
protocol that the lack of an adverse effect can be demonstrated by
analysis of the product quality characteristics. We\'re talking about
CMC here.

And not considered appropriate, nonspecific plan for CMC changes. We
have had some protocols that were written apparently too far in advance
that they did not know the details of that change or how that change was
going to be evaluated.

Also not considered appropriate, if the comparability protocol would
require pharm. tox studies, biopharmaceutic studies, other clinical
safety or effectiveness studies to be done. And in those cases we would
not be able to offer a downgrade, I am afraid.

And continuing with our current thinking on comparability protocols and
some of the principles and recommendations we\'re trying to articulate
in the guidance, that comparability protocol should be based on and
provide evidence of scientific knowledge and technological knowledge and
understanding of the drug. That includes the drug substances, the drug
\[product\] and all of the materials that are used in its manufacturer,
the manufacturing process, the controls, the proposed change itself, and
what is the potential effect of that change on the product quality; and
that this knowledge and understanding could have been gained through
pharmaceutical development information pertaining to the drug and its
manufacturing process.

And adding to that, commercial scale production experience would
contribute, and one may be also able to cite scientific and technical
and technical literature.

These are continuing with the principles and recommendations. In
developing your comparability protocol, all of the potential effects of
the change should be identified and not just the obvious. this is a Q5E
concept that was rolled into this guidance.

And the pre and post change drugs should be compared for all changes.
I\'m speaking of the changes with a drug substance, then the comparison
mainly resides there.

And for all the changes this has been a longstanding policy that we
normally see in our supplemental applications.

And the combination of routine product quality control testing,
supplemented with characterization studies as needed would be utilized,
and the analytical procedures that are utilized should be sufficiently
discriminatory due to potential differences in the pre and post change
products.

And then an integrated analysis of all the available data surrounding
the development of change and implementation of the change should be
performed prior to concluding a lack of adverse effect of that change
and perhaps implementing the change.

And then just a few words, and I won\'t belabor this. Demonstration of a
lack of adverse effect because this is what the protocol was designed to
do. This should, of course, be based upon such knowledge and
understanding that we have been discussing.

And the product quality characteristics of the pre and post changed
products should conform, of course, to their specifications, and the
specifications would apply to all the materials, including drug
substance, drug product that constitute the drug.

And not only that, but that such conformance of the acceptance criteria
should also be made for the characterization studies, and that these
data should be comparable with respect to the mean and deviation of
previous product made by the current process and also applied to those
types of characteristics that are expressed qualitatively.

And also we should consider the effect of the change on the
manufacturing process and the process controls. Of course, the process
controls will be met. In some cases you may even have to change the
process controls, but essentially that would be the bottom line.

And the effect on the process controls as they relate to the product
quality would be considered.

And now turning to how do we propose and how does the company propose
and how does FDA justify designated a reduced reporting category, given
the submission of a comparability protocol, and there are several
factors that would be considered, and one factor, the foremost factor,
the degree of the demonstrated knowledge and understanding of the
product, the process, et cetera, et cetera that is provided with the
protocol.

And of course, you need to consider what is the normal reporting
category for that change, and that can be found in the regulations and
our guidances, and that would be the starting point for the downgrade.

And also we considered the specific aspects of the drug, the process
controls, the change would also be considered, for example, complexity
of that process, complexity of the product as well. So it would be input
into that.

But also I mean this can be tempered with knowledge and understanding in
a complex product if it\'s well understood.

And then also the validity of the comparability protocol and some of the
things associated with the validity is is it scientifically and
technically sound.

And now getting into the plans on our current thinking, how do we get
there, and these are the various categories of changes. Prior approval,
CBE, CBE-30, and annual report that are specified in our guidances,
specified in our regulations and our guidances, and so those are the
starting points.

So you would have to know how your changes fit into this hierarchy
originally, and then how can we get from prior approval down to annual
report, and we believe that would be capable if a substantial knowledge
and understanding is presented, that that is demonstrated with the
comparability protocol submission.

And it could be in the submission. It could be referenced or
cross-referenced off to the original NDA or other submissions to your
marketing application that would allow us to go there and look.

And the use of the comparability protocol would substantially reduce the
potential of an adverse effect on the product quality in that case, and
this first category is beyond really what the regulations, I think, the
original writers had intended. They had talked about a reduced reporting
category, not talked about how do we get to prior approval. They leave
it to us in guidances to figure this out. And with our current paradigm,
this is what we believe.

The current state of affairs is more or less the second bullet, an
intermediate or moderate reduction, and where an adequate knowledge and
understanding would be provided in the protocol, but that would be
differentiated from such substantial knowledge and understanding.

And the third category, we have not seen many of these kinds of
protocols submitted where they\'re downgrading, asking for a downgrade
to CBE-30 of CBE down to annual report because the comparability
protocol itself takes a prior approval supplement.

I mean, this could be overcome if they were combined in a same
submission. We have seen that in some occasions.

And now I want to talk in more detail about how to get from prior
approval down to annual report and what is our current ideas where and
preliminary comments on how do we get there.

Of course, I just talked about the substantial knowledge and the
understanding of the drug, the process controls, the change and the
potential effects of that change, and the relevance and the adequacy of
the test studies and the analytical procedures to assess the effects of
that change and may need to include preliminary data to support a lack
of adverse effect.

And of course, the bottom line, FDA will look at this information and
then determine whether it was sufficient to downgrade to annual report.

And more specific examples of ways in which we think you can get there,
provided with the comparability protocol is data from pharmaceutical
development studies, for example in a pharmaceutical development report.
That would be included in the protocol. That will help in defining the
change, identifying the critical process steps, parameters, variables,
controls and interactions of variables, and if needed, data from pilot
scale batches, and we know that this is typically done on the road to
making a change; that we don\'t think that companies generally jump
directly from the lab to full scale manufacturing. We\'re not trying
this out first on pilot scale and then optimizing the situation.

And data from full scale production batches \-- these might be initial
batches \-- if available, but not necessarily required.

There\'s other ways to get there. You might have data from a previous
change made to a similar product or the same change made \-- sorry \--
similar changes to the product or the same change to a similar product.

There\'s other ways to comparability protocol. It might involve a two
tiered downgrading, and I won\'t talk about that much.

There are some exceptions that are perceived that might get in the way,
in our ability to down grade to annual report., the change may be too
complex. Of course, I talked about very \-- complex changes, changes
that require pharm. tox input, biopharm, or clinical input.

There may be changes in which the impurity profile is changed, and that
will also translate to a change in the need for specifications. These
may be possible impediments on the road to annual reports, and we are
still discussing that within the OPS.

The commoners in the docket asked us how can we modify comparability
protocol in ways that are other than prior approval, and we\'re thinking
about that, and I wanted to give you some specific examples.

We see the need for that, that they may need to modify the acceptance
criteria. They may have actually missed the mark in determining what
those are in implementing change, and they may need to modify the change
itself in order to get it back within the desired target. Changing the
change.

And, of course, over time, a comparability protocol could become
obsolete. There may be new scientific advances. There may be safety
issues that arise, and the comparability protocol needs to be kept
current and valid. So we don\'t want to impede manufacturers in keeping
their comparability protocols current.

And we\'re trying to identify examples, specific examples in which
modifications could occur to a comparability protocol in all of the
different categories of the FDAMA categories.

And I just want to summarize up. The comparability protocol can be
useful to industry to shorten the time line for distribution of drug
products, and FDA is exploring ways to make protocols more useful and
flexible, and we believe that substantial regulatory relief can be
granted through this road or avenue of using a comparability protocol,
provided that an applicant demonstrates a substantial understanding of
their product and their process.

CHAIRPERSON BOEHLERT: Thank you, Dr. Moore. Any questions or comments?
Moheb?

DR. NASR: If you\'ll allow me, I would like to make some general
comments and statements. First, I would like to thank Steve and the
working group. You have been working very, very hard, and very
diligently, trying to get this document out. Because they understand the
need of such a document, and its potential ability to facilitate
submissions and so forth. The document is not out yet, and it\'s not
because of Steve. I am the one to blame. So if you have any problem or
an issue about the document not being out, please don\'t put the blame
on Steve and his working group, because they are working very hard.

I am holding the document for a variety of reasons, and I would like to
share with you, and I would like to seek your input. The main \-- the
original focus of this document was to create a guidance along the same
lines of a guidance for large molecules. And it is very much embedded in
the regulations, and regulatory policies, and so forth.

When I came to the Office about a year ago and started stirring things
up a little bit. And I started asking many questions. I was troubled by
many things, such as the original draft, if you recall, would have meant
in many cases of increasing, or to be more quantitative, duplicating the
number of supplements. So rather than having a supplement to make a
change, now you submit a supplement that we are calling comparability
protocol, to be followed by another supplement to make the change. The
main advantage could have been that you can implement the change without
waiting for the approval for the second supplement. But you cannot get
the change going until we approve the first supplement. That\'s the
problem I have. Another problem I have, it would have very much doubled
the workload that we have for our staff.

Number three, which is the major issue, the first two we can handle. And
Steve has been working very hard to address these two issues. But the
main problem I have, the way the draft has been, and the comments we
have received, do not really articulate our current thinking. And if you
look at what the guidance is, a guidance is not a regulation. It\'s a
way for us to share with you our current thinking, and suggest ways for
you to provide the information for us, for proper assessment in order
for you to continue to manufacture products. I don\'t think of a
guidance the way it was, before I came to the Office \-- so again,
don\'t blame Steve, blame me \-- does not really share our current
thinking.

What\'s our current thinking? I think Ajaz has tried for years, for a
couple of years at least, to articulate that, and we are still debating
and trying to define the desired state.

DR. HUSSAIN: It\'s define, Moheb.

(Laughter.)

DR. NASR: Right. Explain what it means for different scenarios, and so
forth. What we are saying is if you understand your process, if you
understand your product, and you have built enough data, generated data,
because of the design of experiments and other experimental protocols,
and statistical methodology used, and you have defined the space that
you have seen in John Berridge yesterday, and Ajaz and others as well,
where we are comfortable that within that defined space the quality of
the product will not be compromised.

In our current thinking, in the new paradigm if you wish, it is up to
you to make and implement these changes. You don\'t have to come to us
and say \`I\'m going to make that change. Is it okay? Do I need your
stamp of approval? How am I going to deal with our inspectors?\' What we
are telling you is since you have done your work, you understand your
process, you understand your product, go ahead and make such a change.
And it doesn\'t have to be a change from prior approval supplement to
CBE-30 or CBE-0. And that\'s where we are struggling with this.

A few other points I would like to make, and after I make my points I
will appreciate for you, Judy, and your colleagues to provide us with
comments about how can we make this document as useful to you as
possible to facilitate the process. Not necessarily to \-- not only to
reduce the filing categories. I have a problem with my eyes, that\'s why
I have to take my glasses on and off. I\'ll fix it tomorrow. I mean it.

What we are trying to do with this guidance now is very much to bridge
between the existing system, or the existing paradigm, and our future
thinking. And that\'s the reason for struggle. I think in our future,
the new paradigm, the idea is not to reduce regulatory requirements, or
filing categories. It is to look at ways to possibly eliminate
supplements altogether. And that\'s some new things. And you know, we
need to hear from you how we go about that. And I think hopefully the
comparability protocol in the final draft after I\'m done with it, may
provide some ways to facilitate this.

Because we received a lot of comments on this guidance, Steve and his
working group have been working very diligently trying to do two things:
to expand the guidance to address all the issues raised by the public.
That\'s number one. Number two, to provide more details and examples of
when to use it, and when not to use it, and so forth. I think this is
very good and noble, but it resulted in increasing the volume of the
guidance to become quite a bit. Useful, but more descriptive than I
like. So we are working on a compromise, and Steve and I have been
working very closely with this, along with people in this immediate
office, in making the guidance brief but useful. I think we would like
to make it useful, but at the same time there is no reason to make it
extremely detailed because I can assure you, no matter how many issues
we cover in the examples we illustrate, it will never cover everything.
So why not even try. Why should we try.

And I think at last I would like to hear from you, and I hope you focus
your comments on what you like to see in the final draft of this
guidance. We are working very hard, but we have some internal struggle
of how to make the guidance useful, and to bridge between our current
regulatory policy and our future paradigm, and facilitate the transition
from the existing system into the future regulatory process. Thank you.

CHAIRPERSON BOEHLERT: Okay. Moheb has asked us some questions on how FDA
may make this guidance more useful. And I\'d be happy to listen to
committee comments. Any comments? Gerry?

MR. MIGLIACCIO: First, Moheb, I very much liked what you just said. I
guess you expected that.

DR. NASR: I\'m surprised, Gerry.

(Laughter.)

MR. MIGLIACCIO: Clearly, a single-use comparability protocol is going to
have limited utility. The firm is going to have to prepare two
supplements basically, and you\'re going to have to review two
supplements for single-use. Much more utility for repetitive changes.
And the concern has always been the specificity may limit
repetitive-change use. So, that\'s certainly one thing that we do see a
very good use of comparability protocol for repetitive changes, but how
specific does it have to be defined, and how broad can the applicability
be. So that\'s one.

But I think you hit it. You know, John Berridge talked about the design
space, the variable space yesterday. We have to figure out a way to
continue \-- what\'s the process for first defining it in the original
NDA, and then continuing to build it. And as it builds, to continue then
to build in the flexibility to make changes without any supplements.
That\'s the process we have to nail down. And it would be ideal if that
could come out. But I think you will see firms who choose to do this,
and to continue to build that design space, will need some way to get
that in to the NDA and reviewed so that they can expand the design space
and make those changes. So that is something that we\'d be looking to
discuss, the mechanism for doing that.

CHAIRPERSON BOEHLERT: Dan?

DR. GOLD: I am very much in favor of the vision that I think you are
trying to put forward. And I must say I frankly did not understand why
\-- if a fully thought out comparability protocol, fully defined, with
all the parameters clearly specified, all the data be gathered, fully
specified, the acceptance criteria completely defined, if the firm
achieves what they say they will achieve if they do the study, I could
not understand why I would then have to put in another document such as
a CBE-30 or a CBE. I did not understand why I would not be able to go to
an AR immediately. Because if I have clearly defined all the
requirements that I will meet, and then I do meet those requirements,
and your staff has accepted all that in advance, why not be able to go
all the way? So I am very much in favor of the vision that you are
trying to move toward.

CHAIRPERSON BOEHLERT: G.K.?

DR. RAJU: I agree with the comments that were made before. I just wanted
to raise two points. You can choose to make them irrelevant if you
don\'t agree, and don\'t want to think about it further.

If we allow a rapid transformation of the manufacturing system over the
next two years, and we greatly enhance the capability, and in doing so
increase the amount of supplements rather than decrease it, is that a
bad thing? I move on.

Number two, is the right body of unit the number of supplements, or the
quality of the supplements? And isn\'t that \-- once you make it
consistent with the vision, shouldn\'t the focus be on quality per
supplemented \-- time per supplement, rather than number of supplements.
I agree with everything, but those are the two points.

DR. NASR: I think you are raising a very good question, and I want to
make that very clear. I\'m not saying that time will come where we will
eliminate all supplements. I think what we are trying to work on is to
justify the need of supplements for considerable changes that cannot be
evaluated at the manufacturing site. I mean, if you make some minor
changes that will not impact the quality of the product, the process
remain under control within that defined space, why do you have to come
to NDC? I don\'t want to see you. Basically go ahead and implement the
change, since you have laid out early on your experimental design and
how you are going to control the process, and the parameters are well
defined within that space. There is no reason for supplement.

However, if you elected to make a major change that may impact for a
change in the specification, or may require evaluative study. Where we
are getting to potential clinical impact, this may be a time where you
can propose the change and bring your experimental design to us for an
assessment to make sure, because we have a responsibility to the public
that the change you are making, the major change you are making, will
not adversely impact the quality of the product as it is related to
safety and efficacy. That would be the only time, in my mind, where a
supplement is needed. If you are changing a lubricant on a seal on a
filling machine, I don\'t think you need to come to us with a hundred
supplements to do that.

DR. RAJU: So we won\'t get to a place where there\'s zero supplements,
but getting there means first increasing it before it goes down. How are
we going to find out?

DR. NASR: I think our role will be to facilitate continuous improvement.
And some of this continuous improvement can be done without any
regulatory oversight, and some may still need some regulatory oversight
in the form of scientific dialogue to have an assurance what you do is
scientifically sound.

MR. FAMULARE: A question I might raise to Moheb and Steve. If the change
is bringing you closer to the specification, or closer to the design
space, as opposed to you\'re further away from it, then could we \-- is
that an area of no supplement? Is that how you\'re looking at it?

DR. NASR: I think, if I hear you correctly Joe, you want to change the
space. And you are saying \`Are you willing to expand that space?\' I
think that will be something that we need to look at.

MR. FAMULARE: Well \--

DR. NASR: But, but \-- just let me finish, please. But, if we agreed on
that space, and that\'s the data, and this is the scientific model you
have, you can go ahead and make the changes within that space. If you
come and say, \`Well, I\'m going to expand the space, and instead of
having that oval-shaped, I\'m going to have some points scattered around
and generate another geometry, if you wish,\' this will be a time where
we need to sit together and see the impact of such a change on the
space, on the quality as it relates to safety and efficacy.

MR. FAMULARE: Right, what I was thinking of is if you\'re going beyond
the space, your process is drifting beyond the space and then the change
brings it back in, is that something that you want to see?

DR. NASR: No.

MR. FAMULARE: Right. And I think that would make a good corollary to the
Q10 and how \-- the Quality Systems, and bringing things towards
continuous improvements. And I think eventually this will correlate with
that.

DR. NASR: Some people, however \-- I know you don\'t \-- but some
people, however, think of the concept of continuous improvement, that
there will be no regulatory oversight whatsoever. I think we need to
minimize regulatory oversight to facilitate continuous improvement, but
there will be some key elements that must be integrated, must be
presented in a coherent manner. And these are elements that may require
evaluation assessment, good Quality Systems to manage the process of the
plant, a good GMP inspection, and defined space regulatory processes.
All these things need to be together.

MR. HOROWITZ: I don\'t disagree with anything Moheb or Joe said. I agree
completely, and I just wanted to echo a couple of the sentiments.

Continuous improvement doesn\'t require the absence of all regulatory
oversight. I think we all agree with that. Our system intentionally has
redundancies built into it. And that\'s a good thing in terms of
protecting the public health. Sometimes it can get in the way of
continuous improvement to the extent those redundancies become
burdensome. And it\'s partly our job to identify areas where we could do
without some of those redundancies. And I think there\'s often overlap
between the safety oversight and the benefits on the review side, and
the safety net that we have with Quality Systems and with GMP oversight.
And there are certain instances where we could take the chance, if you
will, as regulators, to give more flexibility to the regulated industry
to make changes, knowing that if something goes wrong, there are other
safety nets. There\'s a Quality System in place. And if we get more
assurance that the Quality System is effective not just to prevent
errors, through change control and other things, but also to be able to
detect them, to detect them in a timely fashion.

And I think that\'s what Q10 is really about. It\'s about giving the
regulators more confidence in the ability of the Quality System to serve
as that safety net, to give us greater confidence and greater ability to
remove some of the redundant oversight that may have been in place on
the review side.

One last point. It all comes back to specifications, though. We could
have all the Quality Systems in the world, but once the specifications,
as part of the QA process, become more rational, more clinically based,
I think we can ultimately have greater confidence in the ability of
enhanced Quality Systems to catch real problems that affect the clinical
\-- of clinical significance that would affect the patient. And I think
that\'s all part of the desired state. It\'s going to take awhile to get
there because there are a lot of pieces that need to be put in place.
And things like Q10 and other aspects of this require a bit of a leap of
faith for all of us, to be willing to say \`We can\'t be sure whether
this is ultimately going to have the payoff we\'re expecting, but we\'ve
got to build a foundation if that might happen.\' It might not be a
sufficient condition, but many of these things are necessary conditions
to move forward to the desired state. Thank you.

CHAIRPERSON BOEHLERT: Any other questions or comments? Gerry.

MR. MIGLIACCIO: David, the way you\'ve described Q10, obviously we agree
with. The question is if we don\'t get the support in ICH for Q10, it
has to happen here. So we need a contingency plan, as we\'re still not
assured that it will move through. It\'s not approved yet to move
forward.

DR. HUSSAIN: It has been accepted. I mean, the timing of that is going
to be just \--

MR. MIGLIACCIO: The timing.

DR. HUSSAIN: A step of when Q8 and Q9 goes to Step 2. That\'s the
timing. It\'s a timing issue. I think we supported it throughout the
process, and we leave it to our regulatory colleagues from Europe and
Japan because of their resource issues. So I think the steering
committee has accepted it.

CHAIRPERSON BOEHLERT: Are we ready to move on?

MR. FAMULARE: I just had one short comment, that I mentioned over the
course I think of yesterday, that we have this Quality Systems guidance
coming forward, and it\'s more broad than Q10, but certainly comments to
that guidance when it issues in September can certainly latch on those
things here, and get it moving. And it may spark movement also in ICH.

DR. NASR: I just want to add one thing in response to Gerry\'s question
about Q10 implementation and timing. I think it\'s a good thing it will
have a global agreement of the goals of Q10 and how to get there, but I
think we internally here at the Agency have decided to move on. So we
are making some drastic changes now, both on the review side and the
inspection side to facilitate continuous improvement. And we are very
serious about that.

CHAIRPERSON BOEHLERT: Okay, I think we\'re ready to move on. Thank you,
Stephen. And the next speaker is Jon Clark, who\'s going to talk about
changes without prior approval.

MR. CLARK: If I could have someone come up here who knows this computer
and get my talk up. I\'ve had experiences, bad experiences, with this
before. I don\'t care to repeat them. Thanks.

One of the things that\'s striking to me while listening to all this
conversation is that it largely steals much of the thunder from what I
wanted to say here.

(Laughter.)

MR. CLARK: But I do want to bring \-- I will be able to speed up this
talk considerably, because I don\'t think \-- much of what I thought
might have caused conversation probably won\'t, now that we\'ve had the
conversation.

But one of the things I hear people talk about, and I have a long
experience with review work. I\'ve done more reviews than perhaps
anybody should. And one of the things that we consistently confuse, and
I have confused in the past, is the difference between a specification
and a process control. And I want to articulate that by how I got to
work today, how I came here today. And I used a car like so many other
people do. Mine happens to have the shape of a pickup truck, which gives
me a lot of advantages.

But one of the things is the process control is the speedometer, the
temperature gauge, tells me everything\'s working all right. The map
that I have on the seat next to me, that\'s a process control. The
specification\'s about where I have to go. The specification doesn\'t
come out of the process that I\'ve done. It doesn\'t come out of me
looking in the back mirror. The specification has to do with where I
want to go. That all comes out of the front window. So, keep in mind
that when we talk about specification, we need to clean up a little bit
our terminology, because we\'re being a little sloppy here in places.
And if you think about, a specification comes from the next step, not
from the one I just completed.

And the way we apply that to pharmaceutical process is that we need to
be thinking about the spec for the LOD, or the spec for the moisture in
the granulation shouldn\'t be set by how well my granulation is working.
It should be set by what my tabulating machine can tolerate, by what the
degradation profile of what the raw material, the API, is. So keep that
as a thought. Go into that, and I\'ll give my formal talk, the one that
my supervisors actually approved, and we\'ll go from there. Thank you.

So, changes without prior approval. How do we get from where we are now
to where we want to go. And I hope at the end to talk a little bit about
the desired state. But I want to point out that you have to be very
careful because I remember a previous great American who once said that
the most feared words in the land are, \`Hello, I\'m from the
Government, and I\'m here to help you.\' So, let\'s move from that,
hopefully get to another quote later on, and see where we go.

An overview of the traditional system. We\'ve gone through it ad nauseum
today. But the traditional system of approval and change control does
seem burdensome. There should be a way to protect public health without
slowing innovation. And the methods and standards for this are already
available, and part of this talk will go into some things that weren\'t
brought up. But we\'ll see if they contributed or not.

We need to train ourselves into new ways of thinking, but we do have
shared concerns. One of the concerns is that the pharmaceutical industry
is one of the most technologically advanced discovery organizations, but
remains more conservative when it comes to using cutting edge technology
in manufacturing. Concern over how regulatory agencies will react to
using knowledge and technology is a big problem. Agency focus on changes
that have inconsequential impact on product quality, and can result in
delay, is a very big concern. And that\'s part of what this talk is all
about.

There is, from looking from where I have been standing for so long,
looking out, there is a complex interaction between the industry\'s
commitment to high quality products, and their commitment to most rapid
introduction to the market. There are some inherent interactions there
that concern us as reviewers and approvers.

Optimization before approval has certain good points. One is that it
provides the greatest immediate benefit to the patient. That\'s the last
bullet under that subtopic. But the greatest cost is in time and
developing all the optimization information. There also is, when you
start production in that paradigm, there is no baseline from which to
measure improvement. You\'re kind of thrown into a situation, and you
don\'t really know after that whether or not you\'re optimized or not.
So optimization has a funny definition when you\'re talking about before
approval.

In a continuous improvement environment, the time element is minimized
because you can get to the market with an adequate product and with an
adequate process. Also, it enables measurement of the improvement
because you do have that baseline. And the feed forward data in scope
\-- protocols, can all be designed around a continuous improvement
paradigm, and that helps us from our end.

And I would like to point out, the inclusion of development data helps
in the initial review, but it can not equal the knowledge that is
obtained during routine production. And yes, even reviewers see this in
the applications. We see that in a large way in the number of
supplements we get. And we can see that there are improvements being
made most often.

I want to steer our way through a few points. Raw materials process. The
term \"measurement.\" Steering the process. And last is variability.
When it comes to raw materials, it\'s pretty well demonstrated. The
pharmaceutical raw materials are variable. It doesn\'t mean that there
isn\'t a company out there that hasn\'t learned how to pressure their
suppliers into keeping the raw material variables down to a minimum.
That is done very often. The point is that it\'s very expensive to do.
So we cannot also assume that holding inputs constant will always
produce a constant product, and that is because you do have variables in
the raw materials. So the conclusion: attempting process control through
raw material control is really futile. And futile does not mean
impossible. It means expensive, and it means inefficient.

Let\'s talk about the process. Discovery and design suggests a process
model, if you will. The model should be designed so that the parameters
for that model. This is a sort of a very soft, high-level model. Those
parameters that are suggested by the model need to be able to be
measured in the real world. So if you say that, well, this outcome is
dependent on some nuclear magnetic resonance, it\'s not going to be
measurable. So you have to make sure that you have a measurable
parameter. And as the model evolves, the measurement strategy should
evolve with it. And the effect of change can be better predicted when
you have realistic models.

And I\'ll also point out, the last point is that there is a dearth of
process models in applications. We don\'t see that. What we see are very
specific demonstrations of actually manufacturing the product.

Let\'s talk about measurement. Measurement is most effective when used
to control the process in real time. We heard Chris talk about that. And
Chris is gone now. But with PAT, that\'s all about PAT. But it goes
beyond PAT. It\'s just inherently a fact of nature that measurements are
more effective when you\'re looking at using it to control a process.
And yet, in spite of that, the traditional approach, and probably
because of the age of the art of chemistry and how long the Agency\'s
been involved, the traditional approach has been to sample a product
pretty much after it\'s been processed or some intermediate product, and
then test that for compliance with a criterion via a laboratory
determination. And that\'s the term actually used in the CFR.

And we talk about steering the process. We talk about changing time,
speeds, and temperatures, based on measurement to achieve a target value
for a product parameter. And we also want to point out that discarding
batches, or discarding portions of batches, in a hope to get some
recoverable material that\'s marketable out of them, is a sign of a
failure to properly steer a process.

Variability reduction always adds value. It increases the process
capability. It also minimizes the risk of out-of-specification results.
And it\'s also a prerequisite for any kind of a successful
investigation. Because if you have a lot of variability, you\'re not
going to be able to figure out what\'s going on. And for the sake of
G.K., I\'m referring mainly to common variability and not special.

So we have a situation spectrum that I drew up. I presented it before.
And basically it\'s a spectrum to try to demonstrate a world where you
have extensive product testing with little process understanding is not
as desirable as a world where you have high process understanding, high
process understanding to the point of obviating end product testing.
Now, I gave this slide at an Arden House conference 10 months ago or so,
and it was something of a shattering thing to have an FDA\'er say. But
today obviously we have everybody saying something very close to this.
So it\'s very good.

And then we have a little \"therefore\" at the end. The FDA focus on
laboratory testing is not ideal for controlling processes. We need to
encourage process understanding and engineering. We need to focus on the
resources, on manufacturing process instead of lab tests and criteria.
And we need to avoid this trap of measure it because you can. There are
\-- often we\'ve seen, many times, where someone will say, \`Well, we
know that you can get this value out of your process, so we insist that
you get that value every day,\' when no one has ever bothered to go back
and look and see whether that parameter mattered at all. And if it
doesn\'t matter, then why are we measuring it to begin with.

Also, zero tolerance limits. There is sometimes a need for zero
tolerance limits. But I\'ll make the submission that a zero tolerance
limit is mainly a sign of a lack of knowledge. And as you get to a
higher level of knowledge, and in this graphic I have up here now
increasing process understanding and control, the need for zero
tolerance limits goes down. And although in this graph it goes down to a
minimum value, I would submit that an edit of this graph would have it
go down to zero, because that really is where we want to go.

I also want to point out that post approval regulation, and knowledge,
and process understanding are related in this graphic. Of course, the
more knowledge you have, the less post approval regulation we would
need.

And the current paradigm is described in this graphic. We have raw
material going into a manufacturing process. It has locked process
variables. And coming out of that we have a product. And any variability
in a raw material in this particular schematic, the variabilities pass
through the manufacturing process, and because it is so locked, that
variability goes right through to the product.

I submit a dynamic system, where you have a raw material going into a
manufacturing process. You have measurement-dependent process variables.
For whatever purpose that might be, you are actually measuring what\'s
going on, and you might change your process variables according to that
measurement in real time. You would have some kind of an input response
to that. You would have an endpoint response, and then eventually you
would get out the product. You give these terms new names, and you just
have PAT. It\'s raw material manufacturing process. You go feed forward,
feed back, critical process parameters, critical quality attributes. The
product name still stays the same.

And we are not alone. It\'s just a series of things that have derived
from a military standard that has since become an ANSI standard. It\'s
numbered here for the sake if you want to go look it up. It\'s not
currently used because the military actually references the ANSI
standard in this case. It was done in 1996. And their points ring very
true today for us. And these are mainly out of the introduction, not the
sampling procedures which they also describe, which I\'m sure that Dr.
Singpurwalla would probably have a problem with. But I don\'t know.

So leave that where it is, and let\'s look at the philosophy in their
introduction pages. In a process control, the statistical control
methods are the preferable means of preventing non-conformances,
controlling quality, and generating information for improvement.
Sampling inspection by itself is an inefficient industrial practice for
demonstrating conformance to the requirements of a contract and its
technical data package. That contract in this case is of course CNDA. To
the extent that such practices are employed and are effective, risk is
controlled, and consequently inspection and testing can be reduced. Now,
when I first had this slide, we were talking about prioritizing our
inspections in such a way. But as you saw today, we\'re talking about
that with David\'s efforts earlier today.

The objective is to create an atmosphere where every noncompliance is an
opportunity for corrective action and improvement, rather than one where
acceptable quality levels are the goals. In other words, throwing away
parts of a batch in order to get it within criteria is not a correct
methodology. The goal is to support the movement away from an inspection
strategy into effective prevention-based strategies, including a
comprehensive Quality System, continuous improvement, and a partnership
with government. You may have trouble with the word \"partnership.\"
It\'s up for debate, but the point is that we are all after improving
the public health, protecting the public health. Use the terms you wish.

And more. Process should be the focus of the Quality System,
consistently producing conforming product, controlled as far upstream as
possible, robust variation, operated to constantly reduce variation,
utilization of equipment in a way that minimizes variability around
target values, managed for continuous improvement, designed and
controlled using a combination of practices and methods, in order to
ensure defect prevention and process improvement. That\'s the end of the
military standard stuff.

And I bring up William Edwards Deming. Can I have an effective
presentation without quoting William Edwards Deming? I think not. Not in
this area. And this was quoted yesterday in a couple of presentations,
at least in part. \"Cease dependence on inspection to achieve quality.
Eliminate the need for inspection on a mass basis by building quality
into the product in the first place.\" Depending on inspection is like
treating a symptom while the disease is killing you. The need for
inspection results from excessive variability in the process. The
disease is variability.

Ceasing dependence on inspection means that you must understand your
processes so well that you can predict the quality of their output from
upstream activities. Upstream activities and measurements. Does anybody
need a definition of \"upstream\"? I hope not. That means before the
product\'s made.

Here we have I try to capture some of that in the one single slide. On
the left-hand side you\'ll see a box that says \"Range of raw materials
in facility attributes.\" Now, we could have a long list of things I\'m
talking about. It\'s a range of things that could be variable. It could
be long enough to not fit in that box. What I have there is pretty full
anyway. And the ideal situation is that you have a process that\'s
designed to limit the product variability in spite of these other
variabilities.

Variation control is also part of Anna Thornton\'s Variation and Risk
Management book, which is something of a how-to book on how to create a
Quality System that is designed around controlling not just any
variation, but the variation that\'s important to the parameters of your
product that you think are important. And she talks about identification
of key characteristics. Those are to assure achieving critical quality
attributes. That\'s what the CQA stands for. And she talks about a
variation flowdown, where you look at a variation that you\'re seeing in
one place, and you look upstream until you find out where that variation
is really being triggered, and control it there.

It talks about assessment, and which variations put the critical quality
attribute at risk. It talks about mitigation. You can either eliminate
the source of the variation, or try to reduce its impact, or a little
bit of both. And she talks about setting up whole organizational
structures on these ideas.

These are examples of evidence that came out of the military standard
that I was talking about earlier. I\'m going to try to get through them
by just flipping through them because it\'s simply a list of pieces of
evidence that one could supply to a third entity to demonstrate that you
have control of your process. It\'s about flow charts, and identifying
what essentially are operating procedures and plans for variation. But
due to the time on the clock I\'m going to run through them.

I submit that the contribution \-- the institutionalization of knowledge
in your organization is a quality concern. We need to apply solutions
wherever they will provide improvement. And a prior regulatory approval
for every improvement does in fact defeat this goal.

An application without supplements, what are we talking about? What do
we need to see in that application? What are the critical quality
attributes and the means of monitoring and controlling them? What are
the fundamental scientific mechanisms of the physical changes in the
process? Can you describe them? Can you articulate what those are and
tell us how you\'re controlling them?

How do formulation and process factors affect product performance?
Control and operation using mechanistic scientific principles directly
while you\'re manufacturing the material. Demonstrate a range of
operating ranges, controls, and principles. That creates your space. A
history of manufacturing success with similar drugs, or similar
operating principles, or similar site operations. All those things
contribute to this history. And they should be used to create the space.

Significance of the site location and environment on the quality of the
finished product, more of the same. Drug product specification, based on
attributes critical to product performance experienced by the patient or
the health care provider. Process control relationships to finished
product quality. These are all the kinds of things we\'d like to see.

Another thing on this list that we do not see now today are models. We
don\'t see models about how to control \-- what your control strategies
are. And it became a little bit extensive. Didn\'t find its way on the
slide, but I did write it down and would like to take the time to read
that to you once I locate it in here. And what I wrote down here was
model, model, model. Batch records, batch control cards. There\'s little
value in batch records or batch control cards, or equipment settings or
controls, when it comes to process understanding. We\'re talking about
being able to bring the reviewer up to a certain level of confidence
that you have. Not bring the reviewer a total amount of process
understanding, but bring that person\'s confidence level up that you
have an understanding of the process with a model. And that is what your
specification in the application could be.

Operational freedom. Once you\'ve done that, this process understanding
knowledge leads to greater freedom from narrow operating procedures,
which we often see today because, in place of models, we see batch
sheets. Greater freedom from narrow operating procedures and allow focus
on drug product quality. We need to provide for use of alternatives to
any application requirement. And that includes components,
manufacturing, and packaging procedures, in-process controls, analytical
procedures. And anyone who thinks this is a surprise needs to read the
regulations, because those things are listed, as they are in this bullet
point, at 21 CFR 314.50 (d)(1)(ii).

Focus on process science understanding. The FDA wishes to avoid allowing
the submission of great operating procedure in the application \-- great
operating procedure in detail with equipment specifications to create
something of a safe harbor. And I have that in quotes because safe
harbor is a quick way for me to get you an understanding, but I\'m not a
100 percent confident it is a perfect term. But it creates something of
a safe harbor for a process. We want to avoid creating that safe harbor
for processes that do not consistently result in quality of product that
is suitable for use. In other words, the model is more powerful.

Batch records should not be used as manufacturing process control
specifications, or change control restrictions. Stability analysis is
more valuable than raw data. Understanding degradation mechanisms helps
us predict, helps you predict the impact of change.

Agency acknowledges concern about commercial research data. And it has a
lot to do with when you do research on production batches, on commercial
batches. What is the effect of doing that. And there is some concern
about the data coming out of those batches for both commercial
production and for research data. And we\'ve had in several guidances
some language. And I bring that language to you today for comment. And
that language is the FDA acknowledges concern that process research data
may indicate a problem when a product still meets its approved release
methods. The FDA began the research data exemption concept in several
guidance documents. That exemption does not protect a person that
knowingly does harm without attempting corrective action. It also is
designed to place this information outside the scope of a normal
inspection. That\'s the term used in the guidance paragraphs.

It shouldn\'t impact on the ability to release products that meet all
the aspects of the company\'s currently registered quality control
strategy. And that would include all the terms we\'ve talked about
earlier.

And I\'d just like to close with the situation spectrum, again. And that
is that of course extensive product testing with little process
understanding is less desirable than a high process understanding. And
even though you have obviated the need for end product testing. And I
think that might mean a little bit different thing the second time I say
it than it did on the first.

And with that I thank you, and if anybody cares to have any questions or
tell us that we\'re barking up the wrong tree, we\'d love to hear it.

CHAIRPERSON BOEHLERT: Thank you, Jon.

(Applause.)

CHAIRPERSON BOEHLERT: Are there any committee questions or comments for
Jon? Yes, Paul.

DR. FACKLER: I have one question and one comment. The comment has to do
with one of your slides where you said FDA focus on lab testing is not
ideal for controlling a process, and asking for data just because it can
be obtained is a problem. I fully support that comment, but don\'t know
how you\'re going to implement it across the Agency. I can\'t tell you
how many times we get asked for information on a product that is, I
think, completely meaningless to the quality of the product. But
somebody knows that you can make the measurement, and wants to see the
measurement, and set a specification on it.

MR. CLARK: I ask you in return have you included in your application the
kinds of process, knowledge, and understanding, the kind of models that
I\'ve described in this presentation?

DR. FACKLER: Absolutely not. No. So the other thing I was going to say
is when you say obviate the need for end product testing, is it possible
that we\'re going to be able to manufacture a product and just ship it?
We\'ll have enough process controls that there won\'t be any
measurements done. We\'ll just drop it in containers and send it on its
way.

MR. CLARK: There is a 21 CFR 165, that requires two tests: strength and
appearance in the laboratory determination. Now, I have not been put in
a position of playing with the term \"laboratory determination.\" I
don\'t know if that\'s being planned or not. That\'s the only roadblock
I see.

DR. HUSSAIN: The way we have defined real time release, you\'re not
eliminating any tests. You\'re using a different test method. It\'s an
online test method. That\'s about it.

MR. CLARK: Hence the term \"obviate.\"

MR. FAMULARE: The emphasis is on the word \"test.\" You know, there\'s a
lot of things that can meet the criteria for \"test.\"

DR. NASR: I\'d like to add one comment. I think you raise a very good
question about \-- we ask for data, and you go and generate the data
just because you can. And how we handle that. And Jon tried to explain
what he meant by his slide. But let me ask you a question. What do you
do when we ask for data just because you can? Do you generate the data?

DR. FACKLER: Well, there\'s two scenarios. One is that we need approval
for the product as fast as we can, so we give you the data, meaningless
as it is. The other scenario is we take the time to communicate back to
you and say \`Do you really want this? Is it really pertinent to this
kind of a product?\' But that sets us back, and time is money.

DR. NASR: Well, I see more of the first scenario. I see very little of
the second scenario. Where really I think you are pressing for time and
we are pressing of time as well. But if we don\'t deal with this, what
we are ending up with is we are in a vicious cycle. We ask for data,
generate the data, and the data may require more questions, and so
forth.

CHAIRPERSON BOEHLERT: I used to think in those situations, well, we\'ll
give you what you want just to get approval, and then after approval
we\'ll file a supplement. But you never have time to do that then
either, so it never does get done. And that does happen.

MR. CLARK: Ken, yes.

DR. MORRIS: Yes, one of the things I think that \-- and we\'ve talked
about this internally, I know, is the idea of using models to be able to
give you enough confidence so that you can, in a relatively short order,
be able to make a case. Which is not always based on the specific data
that are being requested. But what happens is, and this happens during
consulting all the time, is that when somebody comes and says I have a
problem, well they do have a problem, but the problem that they have
isn\'t the one that\'s presented. That\'s the symptom. The problem came
somewhere upstream. And if you have to take the time to find the problem
that was manifested as that symptom, then of course you\'re completely
correct, you just can\'t do it economically. If on the other hand
you\'ve already demonstrated understanding the process to the level
where you see where it deviates from what you\'d expect, or more to the
point that you\'re raising, when it doesn\'t deviate, irrespective of
the test that\'s being requested, then I think it\'s a fairly quick
process.

There\'s a lead time, of course, but it\'s a transferable lead time I
think. And I think particularly for generics where you have just tons of
data, historical data I mean, for giving tablets. For instance, I think
we were talking about yesterday where you have just hundreds and
hundreds of examples of tablets where the formulations aren\'t
dramatically different. Those data pooled would seem to me to be a very
powerful set of data for making the argument. But that\'s just my
opinion.

MR. CLARK: Thank you. Anyone else?

CHAIRPERSON BOEHLERT: Any other questions or comments? Joe.

MR. FAMULARE: Just to go back to your slide about the ideal application,
and then the need for no supplements based on that. A lot of that is
built on the new paradigm, having process understanding and so forth.
That\'s all right. Don\'t touch it, Jon.

(Laughter.)

MR. FAMULARE: I think another scenario, and Moheb and I already kind of
discussed it on the side of the table here, is when you don\'t have that
process understanding. The application file is reviewed, it\'s approved.
And you end up learning tings over the processing of many batches. And
you realize that over time what you thought would be an optimum process
is really going way off to one side of the space. It\'s going to fall
off, and you want to get it back to the middle again. Those are the
types of changes that I think can be made by the company as well under
that, to get things back on center. You\'re not changing the specs. You
need to do that. And I was saying to Moheb, that\'s where I see the
conflict and conflagration and inspections. You\'re damned if you do,
and you\'re damned if you don\'t. You\'re either cited for not following
your application, or you\'re cited for being way off to the side here.

MR. CLARK: I\'d like to build on that a little bit, if you don\'t mind,
Joe.

MR. FAMULARE: Sure.

MR. CLARK: And that is that we\'ve seen \-- we talk to companies that
come to us. And the bigger disappointment for me now, after doing all
that review work I\'ve done, is that I think that a lot of the
information we\'re talking about to build that space is already there.
We\'ve talked to companies. They show us what they\'ve done. And then
for some reason they feel inclined to reduce this model to a batch
sheet, and then they submit that thing. And I\'m not sure that we have
to worry about them doing a lot of work that they don\'t already do.
You\'re just asking them to build that model, build the space, give us
some confidence in it, and make that your specification.

MR. FAMULARE: Yes, well, that\'s \-- yes, that could bring up another
point, whether, you know, I\'m talking about you\'re good with the spec.
If it\'s going to be that you\'re changing the spec, obviously that\'s
going to come in.

MR. CLARK: The model is the spec.

MR. FAMULARE: Yes. And the spec defines the space. Now, there are other
instances where you want to change the space, but that\'s another story.

MR. CLARK: Well, that\'s a different story. I\'m talking about not
necessarily having to change the space. You have a space. You\'re
comfortable with the space, but you need to operate within it instead of
worrying about getting permission to operate within it.

MR. FAMULARE: Yes, I guess in my scenario they may have developed that
knowledge over time, but they didn\'t have it when the application was
approved.

MR. CLARK: That happens, but \--

DR. HUSSAIN: Joe, let me give you a specific example. Let me just create
an example. I think we have talked about it.

MR. FAMULARE: Right.

DR. HUSSAIN: An example might illustrate that better to the committee.

MR. FAMULARE: Okay. An example may be a suspension product where the
company will realize that they\'re throwing away the last third of the
batch. They can\'t maintain the suspendability over the filling time.
And what they will do is work to change that. In this scenario, they
actually got it to where they had a consistent suspension through the
filling process. And the observation was on the 483, you did not follow
your file process.

MR. CLARK: I would love to answer that now, if you don\'t mind.

MR. FAMULARE: That\'s fine.

MR. CLARK: What was the control parameter that caused them to stop
filling at the 30 percent level and abandon the batch? What was that
control parameter?

MR. FAMULARE: That was testing. It was testing for, you know, the \--

MR. CLARK: What they need is a real-time monitor that tells them
they\'ve lost suspension. And then that\'s the model, that\'s the metric
\--

MR. FAMULARE: But actually they improve the process so that they can
keep it through the whole time consistent, and not, you know, you had
the example of not steering when you\'re throwing out part of the batch
all the time. You\'re throwing out a third of the batch.

MR. CLARK: Well, I\'m not sure that the sample \-- you couldn\'t use
that same idea in the sampling paradigm. Because if they\'re pulling the
sample to see when they lose suspension, you get away from making 30
percent your mark, or time your mark. You get back into \`Did I lose
suspension?\' as your mark. You still solve some of the problem.

MR. FAMULARE: But I\'m saying that change was in the fringe purview, and
they resolved it because they got back to closer to their mark. I mean,
just as an example. I think it was a good thing that they did. But the
confusion, or the need, or whatever, to file all that, and to have that
happen \-- and this was a product that had to keep producing. It was
medically important. It wasn\'t something that they could just say, all
right, we\'ll stop for a half a year. I mean, it\'s important to the
firm not only medically but financially too. So I mean it\'s not
something they want to stop. A lot of the discussion here was about
throughput and efficiency, and keep optimizing that.

MR. CLARK: Right.

MR. FAMULARE: So it\'s just a matter of the timing of all this as well.

DR. MORRIS: Can I just ask, Joe, are you saying that even given the fact
that they were able to improve it and demonstrate their improvement,
they still got \-- they were still cited for it?

MR. FAMULARE: That\'s correct. The opposite example is when the firm
continues to make something in a non-optimal way because they want to
make sure that they have completed all the filing requirements before
they make the changes. So that\'s the flip side of the example.

MR. CLARK: I just caution people, when you make your filing, and you
have a parameter that\'s causing a problem in the batch, it\'s the
parameter that should be the control, not the 30 percent mark. I think
you said 30 percent. You were throwing away.

MR. FAMULARE: Throwing away 30 percent of the batch, right.

CHAIRPERSON BOEHLERT: Any other questions or comments? Okay. Thank you,
Jon. Ajaz, I think we\'re ready for summary and wrap-up, if you\'re
ready.

DR. HUSSAIN: Thank you, I\'m ready. I think Madam Chairperson, members
of the subcommittee, I wrote formally. The invited guests and staff, I
really enjoyed this meeting. It was a very productive meeting, and
thanks to all for your recommendations, comments, and for challenging
our assumptions. I think that that is always good to have.

Just to sort of summarize what I was able to gather, and I think
summarize this also for you. We started the discussion with respect to
looking at what we have done with \-- in a very summary way the
pharmaceutical quality, the quality initiative for the 21st century. We
received updates on what is happening in ICH Q8, Q9, and the proposed
Q10. And we also talked about the ASTME 55.

The key learning from the discussions of the subcommittee at least for
me was I think there was a strong agreement among the committee members
that these current activities are important and are helping us to move
towards the right direction. And by providing more detailed information
and what is needed in the desired state. I think these are all helping.

There was a caution that we need to keep these activities as synergistic
as possible, especially ASTM and ICH activity. And the committee
suggested that I think there needs to be some communication of what we
are doing at least in ASTM to our European regulatory counterparts. And
I think we will take that advice, and in November seek to update them on
this.

I think the scientific principles and principles of risk management that
we are embarking on are helping us move in the right direction. But I
think this theme came again and again. And this was that there is an
urgent need for a concrete example of case studies, both for generic
drugs and for innovator drugs, to help us clearly put a strong
foundation of what the desired state looks like with that concrete
example. And I think that is an important aspect that kept coming back
again and again.

After that discussion, I think we also had some specific questions with
respect to are Q8, Q9, and the proposed Q10 helping us move in the right
direction. And we also asked about quality by design, and how do you
sort of consider and link that two failure mode effect analysis and so
forth. But the key, I think, answer to that was that I think failure
mode effect analysis is a tool, but it has to be used within the broad
context of the scientific principles and so forth that cannot be
separated. And that was a key message.

And with respect to the second question, I think we really asked for
some help in helping to clarify what is minimal requirements, what is
optional requirements, and so forth. And I think one of the suggestions,
especially from Garnet Peck, was the preamble, at least. How we
introduce that question I think has more valuable information, and we
probably need to retain that, is how we are providing incentives and so
forth.

And in some ways I think that was important, more from \-- not from a
scientific challenge perspective but from a communication perspective.
Because that was the topic for discussion at ICH again and again, and
will be so when we go to Japan, especially because I think the European
system already has development pharmaceutics, already has some of these
elements that we are talking about. The disconnect and the difference I
think that we have right now is we did look at the development
pharmaceutics, those reports. We didn\'t find those very useful. So it
was not that we wanted to simply adopt that. They\'re not very useful.
They don\'t give you any process understanding. So what would surprise
to all of us is \-- not surprise. I think the design state is \-- I
think we are talking about a different level of sophistication here. And
I think that\'s the challenge to maintain that. And I think that will be
a challenge in Yokohama, Japan, as we go towards that. But in many ways
I think the committee\'s discussion was very useful even for that aspect
of that.

There was another question that I was hoping to ask, and then hoping to
seek committee input directly. But I think I did get that indirectly.
It\'s help in defining the design space that we are talking about. And
much of the discussion led to that, and I think Jon actually nicely
summarized some of the bullet points that leads to the design space. And
I think that was very useful.

We then had an introduction to Bayesian approaches. I really thank
Professor Singpurwalla for doing that. Recently, I\'m forgetting the
date now, FDA and Johns Hopkins University had the joint collaborative
workshop on this very topic. In your background packet we included a web
link to all the presentations. I think the first two presentations on
the introduction are very useful, if you care to look at that site.

But that workshop is a strong signal with all of us inter-directors and
our deputy commissioners that are sort of supporting that is that FDA
really would like to move in this direction. All of FDA, especially
CDRH, is already utilizing some of these principles. And I think we have
a strong interest in this aspect, and we will pursuer that. The
challenge is, I think many of us, most of us, are not well versed with
this. There is a learning curve for all of us. What I like about it, and
what I gathered from the presentation of Dr. Singpurwalla was, I think
from my perspective, the confidence level of decisions made under
Bayesian are better than when we don\'t make it without the prior. The
decision quality improves under Bayesian thinking and approach because
you don\'t just rely on a P value, you bring a prior likelihood
measurement.

That\'s from a strength perspective. But from a personal perspective,
you really need a statistician to work with the engineer or a scientist
to do that. You just \-- to make a statistical decision, the most
scientific decision. So, hopefully I was correct in my understanding.

DR. SINGPURWALLA: On the dot.

DR. HUSSAIN: On the dot. Well, I think that is the strength. And I think
personally, before coming to FDA my work was in modeling, and was in
neuro \-- molecular biological intelligence. There\'s a direct
connection to that. So I was always fascinated and excited about that
possibility.

I used the time after the Bayesian presentation to just update on the
critical part in issue there. I just touched upon the industrialization
dimension of that. But that is a significant initiative. And we hope to
issue a list of research projects abroad, or just projects that Agency
can be working on. You can contribute to that list. I don\'t have a
docket number handy, but I think there is a docket number on that.

In terms of industrialization, I sort of presented some of the
challenges I see, especially in research and education. Clearly, I think
I suggested to the advisory committee that I think we need to move
towards a more support for pharmaceutical engineering program, possibly
a national center for pharmaceutical engineering, or multiple centers
for pharmaceutical engineering.

The point Dr. Peck made was a good one, that I think we really have to
be careful how we define \"pharmaceutical engineering\" because you have
to bring a systems thinking, to bring biology, pharmacy, chemistry, and
engineering, all together. It\'s not just engineering, and I think
that\'s important.

FDA, especially OPS, will be working with a number of schools who have
expressed interest in moving in this direction. And we are meeting with
some soon. And you will see possibly a collaboration emerging between
FDA and these schools, hopefully to support the move in this direction.

Following this I think we had very extensive and very exciting
discussion on quality by design, and what it means for specifications.
And I think this is important. Specifications, Jon is right. I think you
have to be careful how you define \"specification.\" Specifications
under the ICH umbrella is defined as an attribute, best method, and
acceptance criteria. So three elements go together to define what we
mean by \"specification.\"

I shared with you some thoughts on the dissolution test. And the message
that I was trying to give, that was I think the challenges we face today
is not the dissolution of the drug. That\'s not important. That was not
the message. The message I was trying to give you is the methods that we
have might not be the right methods. And even though dissolution is
important, when you have a calibrator tablet that keeps shifting, and
when you have a calibration standard that is three times the size of
what would be accepted under an F2, what are we doing? And we have been
using this for years. Isn\'t it time to put this on the table and start
addressing some of this? Industry\'s very happy with F2 metrics. That\'s
the way I look at it. Right, Gerry? So they haven\'t complained. So why
should FDA complain? So I think it\'s time to really discuss these
issues which have been lingering on for years. And if you really look at
the measurement systems that we have, most of our measurement systems
where we have problems are physical measurement systems. We still don\'t
have a good means of comparing particle-sized distribution. Hopefully
PQRI in one of these years will come up with a solution. But we
haven\'t.

So if we really look at it, the message I was trying to give was when it
comes to physics, we do not have to do this. When it comes to chemistry,
we are doing extremely well. In chemistry, we actually have done an
extremely good job on identification and other things that Moheb
described. But when it comes to physics, it\'s not.

So the future is dominated with physics. If you really look at it, at
least with respect to nanotechnology and drug device combinations, say
drug-eluding stents, these are all physical problems that are being
confronted. And you\'re not really ready for that. In many ways, when
Dr. Singpurwalla asked me to redefine the desired state, it\'s today we
are using all of this to improve our efficiency today, but five years or
ten years from now our systems may not be adequate to control the
quality of the futuristic product. So we really have to move in that
direction anyway. So why not do it in a pro-efficiency now and be ready
in a proactive way to address those challenges we\'ll face of the
complex nanotechnology-based drug device combinations. So I think
that\'s the way forward.

Sorry. I learned so much so I have to share this back with the \-- but I
think the key aspect was, I think you saw already impressive
presentation by G.K., as usual, on how we sort of move towards a
manufacturing science and knowledge. I tried to cover the specifications
and then took it to the next step and said, all right, the root cause
investigations when you do it right, and how do you do it right, and how
do you sort of communicate that knowledge. And then you had very
excellent presentations by Moheb Nasr and Gary Buehler sharing with you
some of the activity, some of the programs, how they are planning in a
step-by-step fashion to move towards the desired state while managing
the current workload and then moving towards that.

And I think clearly the focus today has been on Office of New Drug
Chemistry. And because they had a wonderful opportunity with the
pharmaceutical development and reinventing themselves quite rapidly.
Office of Generic Drug has such a high workload right now, I think they
will have some challenges, and the points made are well taken, and I
think we\'ll have to work very closely on that.

And so we wrapped up yesterday with I invited Ken Morris to come back
and talk to you, because I think he has been working with our CMC
leadership, both to generate and from New Drug division to start
brainstorming. And the whole message comes back as unless we come up
with very concrete questions, set of examples and so forth, we will have
a difficult articulating what the desired state is. I\'m not sure Q8 in
its full version reached that. I think we need these studies.

And at that point I raised the question and invited John Berridge, and
really raised the question. I think we need a working group under this
committee. And the committee agreed that that\'s a good thing to move
forward. And as a next step to this activity, I will contact Madam
Chairperson, and we will put a working group together, possibly a
working group to address all of the challenges we face with respect to
pharmaceutical development knowledge, design space, and so forth. So
requesting industry reps to consider suggesting names who would be on
this working group. At this point I think what I would suggest is people
with very broad knowledge base and talent would be the right people,
because then we could task out each work to more technical folks. And I
think it\'s important to do that. So we would like to move on that very
quickly. Maybe within a week I\'ll contact \-- later this week I\'ll
contact Judy, try to assemble a team.

CHAIRPERSON BOEHLERT: We\'ll talk later today. I\'m leaving on Friday.

DR. HUSSAIN: Okay.

CHAIRPERSON BOEHLERT: Okay.

DR. HUSSAIN: And we will put a group together that will help with our
knowing the training programs needed, the workshops needed, and so
forth, but also creating some case studies and so forth.

But what I also propose now I think, listening to all the discussion, I
think one of the most important, critical project, research project,
that we need is creating the case study. And I think we need to sort of
put together a program. I know Monsoor is here, and it\'s a very
opportune time that we are trying to meet with one of the major
pharmaceutical companies on a research proposal, a creator, and maybe
this could be another creator that that company might pick up. So
that\'s one of the things that we can pick up and create that case study
with that company.

So we have many opportunities with academia. We can work on creating a
case study. But we also have companies coming with a research proposal
on very similar grounds, so we might create another case study out of
that too. But then we also work with the working group to create case
studies from that perspective also. So that discussion was very, very
valuable to us, and the importance of case studies is clearly paramount.

I think that the question we had asked is one of the current activities
and planned activities in NDC, OGD, that you would suggest, I think. We
didn\'t get many concrete suggestions, but I think what you saw in
Moheb\'s presentation you liked the direction Moheb is moving. And I
think you supported that strongly. And I think we will support that
strongly. I think some concerns of the workload in generics was raised,
and how we will manage moving towards the desired state, and how we will
manage the supplement load, which is twice that of ONDC, 3,400
supplements. And the new number of new drug applicants, AND has 566.
It\'s a humongous workload. So we\'ll have to be very careful how we
manage that.

And I think that\'s not the only two offices. We have Office of
Biotechnology Products, which was not discussed today. At some future
point we will \-- especially I asked Chris to mention to you that we \--
they will be part of the PAT, so one of the \-- we\'ll bring Office of
Biotechnology for discussion with you next time when we meet.

So that was Day One. If I have missed any important aspects, please
committee members, let me know. I think I\'ll stop for a minute for Day
One.

I think before I talk about Day Two here, I had a brief conversation
with Helen before she had to run and so forth, because one of the things
we wanted to share with you today is that all of our activities in OPS
will be focused on moving towards the desired state. I think that\'s one
of the decisions I think we wanted to make after this meeting. This
meeting was an opportunity to read, debate, discuss, and so forth. So
all the guidances that we have coming out, and which are planned, will
have an element. And I think you saw the discussion, the comparability
protocol, that illustrates that point. It will be focused on moving
towards the desired state.

There are many outstanding guidances, many guidances \-- all the
guidances like suit pack, we\'ll have to revisit those. And I think so
all of our activities we\'ve planned will be firmly grounded in making
sure it is consistent with the desired state that we want to move
towards. So that was the message I wanted to tell everybody.

But the challenge is going to be very great because it\'s not that we
\-- just tomorrow it will be decided. It\'s a long process. There\'s a
lot of work to be done, a lot of education, a lot of interim training
and so forth. But the opportunity is for companies that understand the
processes, that do their good research and good science, and that share
information. The desired state is not that great for companies that want
to do the bare minimum. So the advantages are \-- and the good part is
most companies do that today. And it\'s a communication and sharing of
all that information is what it is. Because the quality of drugs today
is good. And I think it\'s an efficiency question, but tomorrow we\'ll
be ready for the challenges.

I mean, today was an important discussion. We started with I think the
study done in the collaboration with us \-- not in direct collaboration
\-- by the two management professors I think will be very useful. And
you got an update on that. There were a number of questions that will be
useful to them to improve that model.

And then our colleagues from Compliance presented their pilot model for
site selection. I think that was a wonderful discussion. At last, after
my \-- David can share any comments if he has any. And I think the
discussion was very, very useful. The three questions that were asked we
did get some input, and they did comment on that.

Well, let me wrap up my parts. The discussion that followed on Phase I
investigation of new drugs, I was just sort of observing and listening.
It is actually quite a big deal. It is a wonderful step in the right
direction. So I hope you understand the magnitude of that impact. And I
think Joe, others, have been working on it for quite some time. And
that\'s a significant step in the right direction, I hope.

The afternoon session, we wanted to sort of give you more of update,
rather than pose questions to you. But we wanted to show you with the
PAT process that the guidance will be final. We have been innovative in
ways of finding ways that do not require prior approval supplement.
Again, clearly I think the regulations require when you have a change in
specification, you have no option but to have that. But when you bring
alternate methodologies where you don\'t need a change in specification,
you have ways of getting the supplement. And through communication and
team approach, especially product reviewers and inspectors working
together creates more opportunities.

And Steve talked to you about his challenges, his group\'s challenges,
on moving the comparability protocol guidance to be more useful. And I
think the feedback that was received was very valuable again. And I
think Moheb and others are working with that group now to make sure that
it remains focused on the desired state also. And I thank Steve for all
of his efforts.

With that I think Jon, I think, summarized some of the thoughts quite
well. Very well done. And I think you can see the level of understanding
Jon shared with you. And in many ways I think the bullets that he has,
especially of what\'s to be the submission that gets you literally no
supplement from a change perspective, I think is a good start, and will
be very useful for Q8 and so forth.

With that I\'ll stop and thank you, and invite David and Helen to say a
few words.

MS. WINKLE: Well, I just want to echo what Ajaz has said. I think that
this was actually an excellent discussion. In fact, it was probably some
of the best discussion I\'ve heard at any of the advisory committees
since I\'ve been here. Your all\'s contributions were very, very helpful
to us, I think, in moving ahead.

I think I may need to be really clear. It\'s going to take us all a
while to get where we need to go. As far as I\'m concerned, I guess
we\'ve crossed the Rubicon, and we\'re on the other side, but finding
our way now that we\'re on the other side is going to take time.

And I really feel that there\'s a lot of contributions that this
subcommittee can make to helping us. And I think the idea of having a
working group to look at some of the specifics of the framework of where
we\'re going, helping us design that, and helping us address things that
are important to industry as we do design that framework is going to be
really the crucial part of us finding the direction and moving ahead.

So again, I think it\'s really been good. I think the people in our
review area, as you can see from what Moheb and Gary both had to talk
about yesterday, we do understand the need to change. We do understand
that we need regulatory flexibility, not only for ourselves but for
industry as well. And we\'ve got to find the appropriate ways to do that
so that the quality of the product remains at the high level it\'s at
today. So we don\'t want to just make change for change\'s sake, but I
think that there\'s a lot to be gained from that.

So again, I want to thank you. I want to again thank Ajaz for putting
this together. I think it was a very good agenda. I think it helped
stimulate the conversation, and I want to thank David and the people in
compliance too for coming and talking about some of the issues on that
part of the whole product quality. I think this is a big continuum, from
review through the compliance, through the whole life cycle of the
product, and working with Compliance has been very valuable to us as we
move forward. Thank you.

MR. HOROWITZ: I don\'t have much to add other than to echo in expressing
my gratitude to the committee for the comments that we got. And I hope
that you\'ll consider submitting written comments, or even calling me up
informally to give me your views that you weren\'t able to express
during this forum. And in particular, in the September announcement,
there will be a brief white paper that expresses some of these same
ideas. And that will be another opportunity to solicit comments. So I
hope you\'ll take advantage of that. Thank you very much.

DR. HUSSAIN: I have to thank and recognize Bob King. I mean, he \-- this
was the first meeting he took on fully himself, and I was three weeks on
vacation. So I think without Bob King\'s help, we really could not have
put it together.

(Applause.)

CHAIRPERSON BOEHLERT: Okay. Thank you for that excellent summary, Ajaz,
and for your kind comments on the committee\'s deliberations. I\'d also
like to thank all the committee members for very active participation. I
also think it was a good meeting, and look forward to further discussion
on many of these same topics as we go down the road. So just in closing,
I\'d like to wish you all good travel to wherever your destination may
be, and we\'ll see you all next time. Enjoy your summers.

(Whereupon, the foregoing matter went off the record at 4:01 p.m.)