# Presentation: 338366

## Active Control Non-Inferiority Studies:Theory, Assay Sensitivity, Choice of Margin

- Robert J. Temple, M.D.
- Associate Director for Medical Policy
- Center for Drug Evaluation and Research

- February 19, 2002

## History of FDA Interest in Non-Inferiority Trials

- Personal interest since early 1980’s
- Hints in regulation on adequate and well-controlled studies (1985) that equivalence trials present problems
- Various guidances on analyzing trials
- Temple and Ellenberg [Annals Int Med Sept 2000]
- ICH E-10:  Choice of Control Group and Related Issues

## ICH

- ICH = International Conference on Harmonization
- Three region (U.S., Europe, Japan) effort to harmonize the technical requirements for the marketing of new drugs:
- Quality (manufacturing and controls)
- Safety (pharm/tox)
- Efficacy (human efficacy and safety)
- Participants are the 3 regulatory authorities and their respective manufacturer organizations (e.g., PhRMA)
- Develop guidance documents in the 3 scientific areas; these are then adopted, more or less uniformly in the 3 regions.  The guidances are controlled in final stage by the 3 regulatory bodies.
- Does NOT attempt to harmonize decisions

## Guidance of Interest

- ICH E-10:  Choice of Control Group and Related Issues in Clinical Trials
- General discussion of all types of control groups, including historical controls, ethics of placebos and a wide range of other matters, and devotes particular attention to use of active control equivalence/non-inferiority designs, not to demonize them, as has been alleged, but to describe their logic and inferential difficulties and to emphasize the need for evidence of assay sensitivity
- Much of what follows is considered in ICH E-10, but that document discusses the M1/M2 (1, 2) distinction only minimally

## Evidence of Efficacy

- Two distinct approaches:
- 1.	Difference-showing
- 	Superiority of test drug to control (placebo, active, lower dose) demonstrates drug effect
- 2.	Equivalence or non-inferiority in an active control study
- 	Showing similarity to a known effective therapy (active control) and attributing the efficacy of the active control drug to the new drug, thereby demonstrating drug effect.
- 	Non-inferiority trials show that the new drug is not worse than the control by a defined amount, the margin (M) or delta, that amount being no larger than the effect the active control would be expected to have in the study.
- 	NB:  In subsequent slides M and delta are interchangeable terms.

## Equivalence/Non-Inferiority

- The “naïve” approach in active control trials is:
- Compare new and control drug.  If there is no significant difference, declare the new and old drugs “equivalent” and the new drug effective.  One problem is that increasing variance alone (e.g., making the study too small) will create “success.”
- A non-inferiority design specifies as a null hypothesis that the new drug is inferior by some margin M, and tests this statistically.  If the 95% C.I. upper bound for inferiority of the new drug is < M, the null hypothesis of inferiority is rejected.  Note that if the C.I. is very wide because sample size is small, the study will not declare non-inferiority.
- The non-inferiority trial design solves the size/variance problem, but not the assay sensitivity problem.

## Evidence of Efficacy

- Equivalence or non-inferiority trials always raise a question:
- Did the active control drug have an effect of the size expected _in the trial carried out_?  If it did not, equivalence or non-inferiority by the expected effect is meaningless.  The equivalent or non-inferior drug could have no effect at all.
- Put another way, if no difference > M is seen, does that mean both drugs worked or that neither drug worked?

## Assay Sensitivity

- Assay sensitivity is a property of a clinical trial:  the ability of the trial to distinguish effective from ineffective drugs.  Assay sensitivity depends on the effect size you need to detect.  A trial may have assay sensitivity for an effect of 10 but not an effect of 5.  You, therefore, need to know the effect of the control drug.

## Active Control Trials

- Thus, there is an inherent, usually unstated, assumption in a non-inferiority trial, namely that the active control was effective in the particular study in question, i.e., that the trial had assay sensitivity
- The assumption:
- 	•	is not necessarily true for all effective 					drugs
- 	•	is not directly testable in the data collected, 				because there is no placebo group to serve as 			an internal standard
- 	•	in essence, causes an active control study to 			have elements of a historically controlled 				study

## Fundamental Logic of Trials

- Superiority = Efficacy
- 		 			    (if control _>_ placebo)
- Non-inferiority  Efficacy
-                       (unless assay 										sensitivity is present)

## Assuring Assay Sensitivityin Non-Inferiority Trials

- In a non-inferiority trial, assay sensitivity _is not measured_ in the trial.  That is, the trial itself does not show the study’s ability to distinguish active from inactive therapy.  Assay sensitivity must, therefore, be deduced or assumed, based on historical experience showing sensitivity to drug effects, a close evaluation of study quality and, particularly important, the similarity of the current trial to trials that were able to distinguish the active control drug from placebo.
-  Assay sensitivity can be measured in an active control trial if there is an “internal standard,” a control vs placebo comparison as well as the control vs test drug comparison (i.e., a three-arm study).

## Sensitivity to Drug Effects/Assay Sensitivity

- Sensitivity to drug effects
- A historically based conclusion that appropriately designed, sized, and conducted trials in a particular disease, with a specific active drug (or group of related drugs) reliably show an effect of at least some defined size on a particular endpoint.  Usually established by showing that appropriately sized (powered) and well-conducted trials in a specified population regularly distinguish the active drug(s) from placebo for particular endpoints.
- Sensitivity to drug effects is an abstract conclusion about well-designed trials of a drug in a particular disease.  Assay Sensitivity is a conclusion about a particular trial.

## Historical Evidence of Sensitivity to Drug Effects

- To people's surprise, there are many effective drugs that cannot be said to have HESDE, i.e., that are not regularly superior to placebo in well-done studies of adequate size
- To illustrate, about 46% of well-done trials of effective antidepressants cannot distinguish drug from placebo.  No one knows how to choose a population, sample size, or design that will alter this state

## Historical Evidence of Sensitivity to Drug Effects: a Precondition for Assay Sensitivity

- Examples of situations in which studies of current drugs _lack_ sensitivity to drug effects:
- •	Depression
- •	Anxiety
- •	Dementia
- •	Symptomatic congestive heart failure
- •	Seasonal allergies
- •	Gastroesophageal reflux disease
- •	Post-infarction beta blockade
- •	Post-infarction aspirin

## Assay Sensitivity and Study Quality

- If sensitivity to drug effects exists for a therapeutic class, assay sensitivity in a particular study can still be undermined by a variety of study conduct factors that “bias toward the null,” i.e., obscure true differences between treatments.
- These factors include:

## Assay Sensitivity and Study Quality

- Poor compliance
- Non-protocol crossovers
- Spontaneous improvement in the population
- A poorly responsive population
- Use of concomitant medication that interferes with test and control treatment or that reduces extent of potential response
- Poor diagnostic criteria (patients lack the disease to be studied)
- Inappropriate (insensitive) measures of drug effect
- Poor quality of measurements
- Mixing up the treatments

## Historical Evidence of Sensitivity to Drug Effects

- Conclusion of HESDE applies only to trials of a particular design (patient population, selection criteria, endpoints, dose, use of washout periods) .  Changes in these can affect the effect size of the active control and, therefore, the appropriate margin, or completely undermine assay sensitivity

## Choosing a Non-Inferiority Margin

- The margin M is thus the degree of inferiority of test drug to the control drug  that the trial will exclude statistically; that is:
- The upper bound of the 95% CI for C-T must be _<_ M, where C-T is the difference between control and test drug.
- M must be chosen to be no larger than the effect the control drug would be reliably expected to have (based on history) compared to placebo in the trial. Call that value M1 (or 1).  If M1 is the entire effect the control can be presumed to have had in the study, if C-T were > M1, the new drug might have no effect at all.

## Null and Alternative Hypotheses

- In a trial intending to show that there is not a difference greater than M1 between the control (C) and test (T) treatments, the null hypothesis is:
- 		Ho=C-T_>_M1
- 		Ha=C-T<M1
- The null hypothesis would generally be tested as a one-sided hypothesis with the 97.5% upper bound of the CI for C-T needing to be <M1.  If the upper bound is _>_M1, all of the effect of the control drug could be lost and T might have no effect at all

## Choice of M1 is Crucial

- If you say (allege) the control agent has an effect M1 and find that C-T<M1, then T is “effective”
- But if, in _this_ trial, C had an effect of only 0.5 M1, T will not really have been shown effective, IT WILL ONLY LOOK THAT WAY
- Therein lies the regulator’s anxiety.  You must really know (without measuring it) what the effect of the control in this study was to assure assay sensitivity

## Choosing a Non-Inferiority Margin

- The margin used in a trial, M, could be the entire effect of the control drug (M1).  Assuring that C-T is < M1 will then show that the test drug is better than placebo. The M chosen could be smaller, however, M2 (or 2) if there were need to assure preservation of more than just _any_ of the control drug effect, e.g., preservation of some fraction of the effect of the control drug.  Choosing an M2 smaller than the whole effect of the control may be important when the effect is clinically critical, e.g., mortality.  M might then be 50% or 25% of the entire effect of the control agent.

## Slide 22

## Confusion of M1 and M2

- There has been a tendency to consider M1 and M2 separately.  That is all right if M1>>M2 (e.g., many antibiotic treatments, treatment of acute leukemia) but not if M2 is almost equal to M1 (or larger).  In the past, for example, it was common in cancer trials to declare equivalence if survival inferiority of 20% was excluded.  But the control agent in many studies did not have a known effect as large as 20% more than no treatment (that’s a 2 month survival advantage if the control is 10 months) so that successfully excluding a more than 20% difference could nonetheless represent loss of all effect or even harm.  In many cases this approach was used even if no survival effect of the control was documented
- There is a certain logic to that approach regarding clinical value, but it cannot show _effectiveness_

## M1 vs. M2

- The non-inferiority margin of interest in many trials of many anti-infectives is M2.  That is, there may be no doubt at all that an active agent can be distinguished from an inactive one every time, and the real interest is in assuring that a new agent is not inferior by too much.  Examples:
- 	Non-anti-infective
- 		Treatment of ALL
- 		Treatment of testicular cancer
- 		Acute response to bronchodilators
- 		Anesthetic agents
- 		Thrombolytics
- 	Infectious disease
- 		UTI
- 		Meningitis
- 		Many others

## Choice of Margin (M2)

- 	How much effect needs to be retained in such situations?
- 	The very reason you can’t do a placebo-controlled trial is a reason for assuring preservation of a good part of the control drug effect.  So for:
- 			50% thrombolytics
- 			50% capecitabine
- 			75% adjuvant breast Ca
- 	That is not all one hopes to retain, but sample sizes become rapidly unrealistic if greater retention is sought
- 	You will hear a great deal about M2/2 for antibiotics

## M1 for Anti-infectives

- There are at least a few situations where the effect of active agents is very small and hard to demonstrate, and you will also hear about these.  When that is so, a non-inferiority design becomes problematic, for all the reasons described above, and defining M1, and thus establishing efficacy, may be very difficult using a non-inferiority design.  Placebo controls therefore need to be considered.
- Will those be ethical?

## ICH E10 - Ethics of Placebo Control

- Severity of Illness
- Clearly distinguishes between available drugs that prevent serious harm and those that treat symptoms

## Use of Placebo Control

- Principal issue is the ethical one.  There is no issue 	when no effective therapy exists.  The question is when is it acceptable not to give existing therapy and randomize to drug or placebo.
- “In cases where an _available_ treatment is known to prevent serious harm, such as death or irreversible morbidity in the study population, it is generally inappropriate to use a placebo control.”  (Generally, because if treatment is so toxic that people refuse it, a placebo may still be possible.)

## Use of Placebo Control

- “In other situations, where there is no serious harm, it is generally considered ethical to ask patients to participate in a placebo-controlled trial, even if they may experience discomfort as a result, provided the setting is non-coercive and patients are fully informed about available therapies and the consequences of delaying treatment... Whether a particular placebo-controlled trial of a new agent will be acceptable to subjects and investigators when there is known effective therapy is a matter of investigator, patient, and IRB judgement, and acceptability may differ among ICH regions.  Acceptability could depend on the specific trial design and population chosen.”

## Placebo Controls in Infectious Diseases

- It may be possible to design trials, where M1 cannot be specified, that randomize to drug and placebo (active control as well, preferably), that allow early escape for anyone not doing well, failing to respond by time X, etc. (i.e., early escape and rescue designs).  Again, you will hear much more about this.