# Presentation: 675396

- Experiences with Single Trials Leading to Section 115 of FDAMA and May 1998 Evidence Document.
- Robert Temple, M.D.
- Associate Director for Medical Policy
- Center for Drug Evaluation and Research

## FDAMA

- Section 115 modifies 505(d) with an _addition_.  The section now reads (new part capitalized):
- “the term ‘substantial evidence’ means evidence consisting of adequate and well-controlled investigations, including clinical investigations, by experts..., on the basis of which it could fairly and responsibly be concluded...that the drug will have the effect it purports or is represented to have...IF THE SECRETARY DETERMINES, BASED ON RELEVANT SCIENCE, THAT DATA FROM ONE ADEQUATE AND WELL-CONTROLLED CLINCIAL INVESTIGATION AND CONFIRMATORY EVIDENCE (OBTAINED PRIOR TO OR AFTER SUCH INVESTIGATION) ARE SUFFICIENT TO ESTABLISH EFFECTIVENESS, THE SECRETARY MAY CONSIDER SUCH DATA AND EVIDENCE TO CONSTITUTE SUBSTANTIAL EVIDENCE...”

## What Does the New Language Mean?What are relevant science and confirmatory evidence?

- 1.	Usual standard of at least two studies remains, but we _can_ rely on a single study if _relevant science_ supports that conclusion.  What is _relevant science_?
- 2.	It seems to say that when we do rely on a single well-controlled study, there _must_ be _confirmatory evidence_.
- 	Does that mean reliance, as in the past, on a single persuasive study, _without_ identified confirmatory evidence, is no longer legal?  Probably _not_ what was intended.

- 3.	What does “prior to or after” signify; the _order_ of studies is of no consequence.  As this is a condition for marketing, it cannot refer to later confirmatory evidence (i.e., after marketing).  The phrase remains mysterious.
- 4.	What is “confirmatory evidence?”  There is _no_ pertinent legislative history on this point nor was it, to my best knowledge, ever discussed.  [Dr. Peck, to whom this is sometimes attributed, declines credit for it and does not know.]
- 	“Confirmatory” sounds strong; but context suggests it is _less_ strong than evidence from another study would be.

## The Evidence Document

- Providing Clinical Evidence of Effectiveness for Human Drug and Biological Products (May 1998)
- •	Response to FDAMA but under development 		much earlier
- •	It at least partly answers questions raised by 		section 115
- •	Illustrations cited in guidance and others that 		indicate what could be considered confirmatory 	evidence and/or relevant science
- •	Describes some of the reasons for a 2-study 		requirement

## Another FDAMA Change

- Section 403 - Supplement Guidance:
- 	Without changing the approval standard for 				supplements, FDAMA required FDA [403(b)(2)] to 		“specify data requirements that will avoid duplication 	of previously submitted data by recognizing the 		availability of data previously submitted in support 		of an original application”
- 	The evidence document was actually initially 			developed to describe data needed for supplements, 	not for initial submissions but its implications 			proved broader.  The previously submitted data 		are one kind of “confirmatory evidence”

## Substantial Evidence

- What is it?
- Substantial evidence based only on controlled studies as the only basis for approval was a radical concept, surely the most important _scientific_ component of the FD and C Act (Pre-clearance was a comparably radical change in procedure)
- From the beginning, however, there was debate and argument about how much evidence was needed and what the standard for adequate and well-controlled studies was
- Definition of well-controlled study slow in coming:  1970

## History of Plural(limited review)

- 1.	The Act at 505(d) uses the plural “investigations” (but wrapped in the mystery phrase “including clinical investigations” - what, does that mean there are _other_ well-controlled studies?)
- 2.	The legislative history (Senate Report) suggested that the phrase referred to not only the _quality_ but also the “quantum” of required evidence

## History of Plural (cont’d)

- 3.	Long-standing FDA interpretation, used repeatedly in the DESI review, that Congress generally intended to require at least two adequate and well-controlled studies, each convincing on its own, to establish effectiveness.  Accepted by Court of Appeals (Warner-Lambert v Heckler 1986).  There is great concern that if the standard is changed, settled issues might be reopened

## History of Plural (cont’d)

- 4.	Form 356H (about 1970 - used until 1985)
- 	“Ordinarily, the reports of clinical studies will not be regarded as adequate unless they include reports from more than one independent, competent investigator who maintains adequate case histories of an adequate number of subjects [both treated and controls]”
- 	Also says that, except where impractical “some of the investigations should be performed by competent investigators within the U.S.”

## History of Plural

- 	Form 356H (cont’d)
- 	This was certainly read as referring to independent studies (multi-center studies uncommon) but one _could_ read it as referring to multiple investigators within a study

## Basis for Legal Standard of“Independent Substantiation”

- Why “independent substantiation” (substantiation distinguished from replication, which could imply exact repetition, not necessarily most persuasive)
- •	Potential unidentified biases
- •	Chance - quantified by statistical evaluation.  Could control 	by using a lower p-value for single study (0.025 x 0.025 = 	0.00125)
- •	Note large universe of trials, perhaps hundreds - 				unrecognized multiplicity
- •	Single site peculiarities
- •	Fraud

- These factors can be addressed but independent substantiation greatly reduces “the possibility that a biased, chance, site-specific, or fraudulent result will lead to an erroneous conclusion that a drug is effective.”

## History of Plural

- Other reasons that could be grounds for concern
- 1.	Often no protocol
- 2.	Endpoints not always specified; often changed without documentation
- 3.	Interim looks unsupervised and common (cimetidine anecdote)
- 4.	Analytic plan not specified
- 5.	No clear rules on analytic set, ITT, missing patients (Anturane Reinfaction Trial)
- 6.	Methods of blinding primitive (whole code in envelope, peel off label)

## History of Plural

- What is Different Now
- 1.	Always a protocol
- 2.	Endpoints are specified, primary identified; analysis specified; corrections for multiple endpoints.  Caution on cause-specific outcomes
- 3.	All patients accounted for; analysis of all patients with data now standard
- 4.	Interim looks described, blinded to investigators, use standard group sequential methods
- 5.	Multicenter
- 6.	Studies of closely related claims, populations

## History of Flexibility

- Even before we began writing the Evidence Document, we did rely on single studies in some cases.  Often these were:
- 1.	Large, independently conducted multicenter mortality trials
- 	•	Timolol post-infarction
- 	•	BHAT
- 	•	Other post-infarction
- 	•	CONSENSUS
- 	Often extreme p-values, internal consistency
- 	Independent DMC
- 	Ethical barrier to repeating the study
- 	(Pharmacologically related drugs)

## Flexibility (cont’d)

- 2.	Related controlled trial information
- 	•	Different stages, doses, etc.
- 	•	Related diseases (Harter proposal for NSAIDS:  1 	RA, 1 OA, 1 AS, but not two of each; rejected by 	A.C., but correct)
- 	•	DESI was a 2-study review _but_ studies often in 	different populations, of different doses, etc.  All 	considered pertinent
- 	•	Working on document early 90’s

## Evidence Document

- Caveats
- 1.	Assume that the single study is “bulletproof.”
- 2.	“A single favorable study among several similar attempts that failed to support...effectiveness would not constitute persuasive support...unless there were a strong argument for discounting the [failed studies]”

## The Quantity of Evidence

- Developed pre-FDAMA , the Evidence document nonetheless addressed the question of how much clinical evidence was needed to support effectiveness, or, more to the point, when would evidence _other than_ two A+WC studies suffice to support a particular claim or product.  The document describes 3 situations when that could be appropriate:
- 1.	Extrapolation from existing studies - the _no __	__	__study_ case
- 2.	A single A+WC of a specific new use is 			supported by information from other A+WC 		studies
- 3.	Reliance on a single multicenter study, without 		supporting information

## 1.	Extrapolation from existing studies

- In general, this case means that you can extrapolate results of well-controlled studies to a new population, or different dose, regimen or dosage form.  Examples:
- •	Any bioequivalence approval - same drug; 		different salt or ester; identical blood levels 		are the validated surrogate (although we do 		not describe it that way)
- •	Different populations:  pediatric uses 			(judgment), age groups, genders (don’t 		usually need separate studies)

- •	Modified-release dosage forms and different doses or regimens:  depends on how close PK profiles are and what is known about C/R relationships, time course of response, etc. (may need a single study)
- •	Historically:  antihypertensives, antianginals, generally need a study; recent Bupropion CR did not (despite negative clinical trials).  Propranolol post-infarction anecdote
- •	Prodrug/metabolite:  deesterified ACE inhibitors as IV drugs.  Approval of most metabolites has required studies (fexofenidine, certirizine)

## 2.	Single study of a particular  claim with independent substantiation from related study data, generally A+WC studies in other settings, populations, etc.

- •	Whether these are really cases of 2 A+WC studies or one study with “confirmatory evidence” (an obviously good kind) is perhaps moot.  This is the “easy” case of confirmatory evidence because it is the kind of data we generally consider evidence of effectiveness.  Any given situation needs judgment but it provokes little conceptual discomfort

- •	Could describe the “other” data as “strong priors,” allowing the single study to be given greater weight.  We decided against an explicitly Bayesian formulation
- •	Idea goes back a long way; I have notes from almost 10 years ago; GC was very nervous because of 2 study court cases arising from DESI (counting is easy; explaining is hard)

## One Study Plus Related Studies:  Examples

- A.	Straightforward Cases of “confirmatory evidence”
- 	1.	Studies of different doses, regimens, dosage forms (may 	need no new study; if needed, generally only one).  		Anecdote:  DESI history, entirely “proof of principle” 		(different doses, products, dosage forms, regimens, all 		examined together)
- 	2.	Studies in other phases of the same disease.  Generally, 		expect similar direction of response in all stages, though 		magnitude and B/R may differ (typical in oncology, for 		same tumor; severities of heart failure)
- 	3.	Studies in other populations (if additional studies needed)

- 4.	Combination and Monotherapy; each supports the other (typical in oncology, antihypertensives) - NB - not “automatic;” in one recent case, we did not conclude that an AED effective in combination was shown effective as monotherapy by a single favorable study:  the effect was small and needed a larger dose; a second larger and longer study showed no effect.
- 5.	Studies in a closely-related diseases or in pathophysiologically-related conditions:  e.g., one study in each of two inflammatory conditions; one study in each of two pain models; anti-platelet drugs in acute coronary syndrome and post-PTCA

## One Study Plus Related Studies:Examples

- B.	More difficult cases
- 	6.	Less closely related diseases, similar 		purpose of therapy.  Effectiveness in one 	tumor might suggest reliance on a single 	study in a second tumor (possibly 			depends on tumor types); effectiveness 		of antibiotic at one site might support 		another setting with similar pathogens, 		at least in some sites

- 7.	Studies with 2 different, but related clinical endpoints.  Enalapril for CHF supported by one (of 2) exercise tolerance studies and one (dramatic) survival study; given both symptomatic and survival claims.  Other examples could include different (but related) tests of depression or cognitive function, effects on survival and recurrent infarction in different studies.
- 	Issues:  Suppose one endpoint is a surrogate; does it support an outcome claim (e.g., lipid-lowering drug with one outcome study and one study showing decreased coronary obstruction).  This would seem to depend on amount of support for surrogate and existing outcome data

## One Study Plus Related Examples

- C.	Most Difficult Case
- 	8.	Support by pharmacologic/pathophysiologic effect
- 		NB:  a) this is _not_ the case of whether an accepted surrogate 		(these lead to ordinary approval) or a “reasonable” surrogate 		(these lead to accelerated approval), can be used as evidence.  	They can, although in both cases they generally do not lead to 		approval of an outcome claim.  Could a surrogate be used to 		support a single study of outcomes?
- 		b)  few examples given because this is a treacherous area - 		there is always _some_ pharmacologic effect; when is it 			confirmatory?
- 		c)  This is not the case where a single persuasive study is 		sufficient
- 	Principle:  “When the pathophysiology of a disease and the mechanism of action of a therapy are well understood, it may be possible to link specific pharmacologic effects to a strong likelihood of clinical effectiveness”

## Pharmacologic Effect (cont’d)

- Examples cited include:
- •	Replacement therapy, such as coagulation factor - clear evidence that deficiency leads to disease.  Evidence of restoration of the missing physiologic activity provides support
- •	Correction of inborn effort of metabolism
- •	Vaccines:  one clinical study plus animal challenge protection models, human serological data
- •	Caveats:  Pharmacologic effects have misled (arrhythmia suppression, increased cardiac output by PDE inhibitors)

## Pharmacologic Effect (cont’d)

- Probably most sensitive case, because of potential broad applicability.  Raises critical questions:  1) how much reliance do you place on clinical results with pharmacologically-related drugs; i.e., are the results with those other drugs “confirmatory evidence?”  Do we have a “de facto” 1-study standard in this case?  2) how much weight does belief in mechanism carry; i.e., to what extent is that “relevant science” or “confirmatory evidence?”
- Cases:
- 1.	Mortality/hospitalization in CHF.  ACEI’s (several) are 		effective.  Other mechanism adverse
- 	Is one not-overwhelming (but statistically significant) 		study with ACEI sufficient?  Is one study of an angiotensin 	II inhibitor (probably same mechanism) sufficient?

- 2.	Effect of lipid-lowering drugs
- 	Members of an effective class (HMGCoA RI’s) vs. others.  Do you need less data with a new member of the class that has worked or does one class effect make the surrogate adequate support for all classes?
- 3.	Reduced events in acute coronary syndrome or post-angioplasty with anti-platelet drugs; how much does pharmacologic evidence of platelet inhibition count; are different classes the same or different?

- 4.	Beta-blocker outcome data; is a single study generally enough, given results (we seem to act as if it is)
- Issues:
- 1.	Suppose there are no data with the class but epidemiologic evidence; i.e., is one lipid-lowering outcome study always enough, given epidemiology and given results to date?
- 2.	Is one osteoporosis outcome study enough for drugs related to elandronate, to any class that alters BMD?
- 3.	What’s the level of pharmacologic/pathophysiologic data that is sufficient?  There are always hypothesized mechanisms; where is line drawn?  When mechanism is supported by outcome data?

## 3.	Evidence from Single Study

- Note, law does not _seem_ to include this, but we will continue anyway
- We identify 5 characteristics that support
- •	Large multicenter
- •	Consistency across subsets (is this just the 		same as low p-value?)
- •	Multiple studies (strata, factorial) within a study
- •	Multiple endpoints, different events (death, MI; 	in MS, exacerbations and MRI data)
- •	Statistically very persuasive (the most common 	basis), Clopidogral example - one case of use of 	other data on control

## Caveats

- Matter of judgment
- “Generally limited to situations in which a trial has demonstrated a clinically meaningful effect on mortality, irreversible morbidity, or prevention of a disease with potentially serious outcome and confirmation...in a second trial would be practically or ethically impossible.  Repetition of...trials showing only symptomatic benefit would generally not present the same ethical concern”
- In fact, we have almost never asked for repeat of an outcome study that was persuasive

## Caveats

- 1.	Even strong result not a guarantee (vesnarinone; effect of aspirin on AMI in Physician’s Health Study)
- 2.	Look at other available data (vesnarinone, inverse dose-response, no effect on symptoms or hemodynamics)
- 3.	Unexplained failure of a second controlled trial often a good reason not to rely on a single trial

## Conclusions

- Quantity of data on effectiveness is (as it has long been) one of the two most controversial issues (how safe is sufficient is the other), and is potentially troublesome
- 1.	Safety is obviously judgmental - for effectiveness it is 		easier to just “count” studies and cite precedents than to 	explain exceptions
- 2.	Reasoning on 1 vs. 2 often subtle, not always clear.  Hard 	to keep track of all decisions across all divisions, but 		people observe differences and will challenge us
- 3.	Decisions often evolve over long time with complex 		influences (EOP2, pre-NDA, Ad Comm, new guidelines, 		new data) and with many parties influencing, some 		parties are more worried about precedent than others

- 4.	What can we do?
- 	•	Pay close attention to any situation in which there is 		reliance on a single study
- 	•	Know the reason and discuss it in reviews and decision 		documents/minutes, etc.
- 	•	Collect cases, examples; a decision file; review and share 	(e.g., rounds) close cases
- 	•	Internal discussions
- 	•	Develop materials for Advisory Committees
- 	•	Continue to consider “confirmatory evidence” (role of 		epidemiology, concentration/response relationships, 		persistent subgroups effects)