# Presentation: 062442

## Statistical Issues in Specification of 

**Daphne Lin, Ph.D.**
**Erica Brittain, Ph.D.**
**Division of Biometrics III**

## Outline

**Intro: Non-inferiority trials**
**What is ********?**
**History of ******** Selection**
**Principles for determining ******
**Difficulties in practice**
**Alternate designs**
**Summary**

## The Problem

**How can we show the new drug has ****identical**** efficacy to the standard drug (active control drug)?**
**Answer: WE CAN’T!**
**SOLUTION:  We must allow some potential difference in efficacy: ******

## What is ?

**The largest clinically acceptable difference**
**Should be smaller than differences observed in superiority trials of active comparator**

## Non-inferiority Trials

**Non-inferiority trial: to show the new drug is no worse than the standard drug by some margin ********

## Estimation of treatment effect & Confidence Interval

**Define treatment effect as the absolute difference of percent cure rates (e.g.; 85%-75% = 10%)**
**A confidence interval around this estimate of treatment effect is used as the primary analysis for non-inferiority trials**
**Interpretation of the 95% confidence interval: 95% confident that the true difference in efficacy between these two drugs is contained in the confidence interval**.

- 

## Example 1: Non-inferiority demonstrated

- 95% confidence interval

**4.6%**

**- 8.6%**

**Observed success rate** **between test and control drug **
**e.g.; - 2%**

**200 pts/arm, test drug: 88%, control drug: 90%, CI: (-8.6% 4.6%)**** **

****

## Example 2: Non-inferiority not demonstrated

- 95% confidence interval

**1.1%**

**- 13%**

**Observed success rate** **between test and control drug **
**e.g.; - 6%**

**200 pts/arm, test drug: 84%, control drug: 90%, CI: (-13%, 1.1%)**

****

## Goals of non-inferiority trials:

**Indirectly determine if the test drug is better than placebo**
**Directly determine if the test drug is similar to the active control drug**

## History of  Selection

**in FDA’s Division of**
** ****Anti-infective Drug Products**

## 1992 PTC Step Function Approach

**1992 PTC Step Function Approach**

## Concerns about 1992 PTC approach

**Seriousness of disease and consequence of treatment failure were not taken into account**
**whether ******** was small enough that a drug with no efficacy could meet the standard was not considered**
**Has undesirable statistical properties ( e.g.; ********  jumps from .15 to .2 when success rate changes from .8 to .79)**

## Concern of Potential “Bio-Creep”

**Trials over time used progressively less effective control arms**
**Example: if ********of 20% is used**** **

**100%**

**60%**

**70%**

**Drug 2**

**Drug 1**

- 0%

## July 1998 AC Meeting

**The FDA stated ******** should reflect:**
**historical cure rate (with and without therapy)**
**risk associated with treatment failure**
**advantages and disadvantages of study drug**

## July 1998 AC Meeting: Proposal

**Clinically relevant delta**
**Indication specific**
**Special situations**
**Discuss with Medical Division during protocol development**
**Sponsor should provide rationale for selection of control arm**

## Committee for Proprietary Medicinal Products (CPMP)

**Published “Guidance on evaluation of new anti-bacterial medicinal products” in 1997**
******=10% for “common non-serious infections”**
**Smaller for very high cure rates**
**Based on “minimum clinically relevant difference”**
**Justified in protocol**

## Transition

**For past 1-2 years, have worked with sponsors on case-by-case basis to specify ******
**In February 2001, a disclaimer added to PTC document stating that PTC approach phased out**
**There is a need to establish standards (ICH-E10 principles)**

## Now what?

## Road Map

**Principles for determining ******
**Based on ICH-E10**
**Difficulties in practice**
** ****This is the hard part: we need advice**
**Alternate designs**
**Summary**
**GOAL: Choice of ******** is not a technical matter**

## To demonstrate efficacy:

**Experimental drug should be better than placebo**
**In some settings, experimental drug should also have similar efficacy to existing therapy**
**Choose ******** to assure that both of these goals are met**

## ICH-E10 says

**Non-inferiority design “is appropriate and reliable only when the historical estimate of drug effect size can be well supported by reference to results of previous studies of the control drug”**

## Translation

**We must KNOW with good precision the ****magnitude**** of advantage of the active control drug over placebo in setting of clinical trial**
**In practice: if advantage is large, precision of estimate probably won’t matter**
**If potentially modest, precision is critical**
**If only a single trial of AC w/ borderline efficacy -- poor info about magnitude (Statistical significance not enough)**

## Principles from E10

** ******** “****based on both statistical reasoning and clinical judgment”**
** ******** ****cannot be larger than the advantage the “active drug would be reliably expected to have compared with placebo in the setting of the planned trial”**
**Usually choose ******** to be even smaller to ensure some clinically acceptable treatment benefit maintained**

## Possible Choices for 

**70        72       74      76****************80****************84    85**
**True**** ****% Success Rate**

**True**** ****Success Rate of Placebo**

**True**** Success rate of Active Control**

****

****

****

## General approach to 

**Determine two values:**
**Conservative estimate of advantage of AC over placebo (********1****)**
**    ****DATA-BASED**
**Largest clinically acceptable difference between AC and experimental drug (********2****).**
** ****JUDGMENT-BASED**
** ******** ****= ****smaller**** of these two values: Value used in upcoming NI trial**

## Estimation of benefit of AC over placebo (1)

**The true success rate of active control minus the true success rate of placebo in setting of clinical trial**
**By ****how much**** is AC better than placebo in the NI trial, if placebo were present?**
**Based on historical data -- NOT A CHOICE**
**Not important when benefit is large**

## Estimating benefit of AC (1):  Be Conservative

**E10: ******** “should reflect uncertainties in evidence on which choice is based and should be suitably conservative”**
**If ********1**** is over estimated, chance of concluding efficacy when new drug is no better than placebo is too high**
**Err on side of underestimating benefit.  IF poor historical info: don’t use “best” guess, use smallest of reasonable values**

## Estimating benefit of AC(1 ):What is best information?

**Best information for determining ********1**** **

**No information for determining ********1**

**Multiple PCTs w/ same design as NI**
**Single PCT with same design as NI**
**Multiple PCTs with different design**
**Single PCTs with different design**
**Observational studies**
**Anecdotal / Case Series**

## Estimating benefit of AC(1):Placebo Control Trial Data

**What if placebo controlled data exist, but:**
**Trials are old: antibiotic resistance, changes in clinical care management**
** ****Proposed active control not studied**
**Few in number / consistency?**
**Differences in entry criteria, assessment criteria, timing, populations**
**Take data with big grain of salt --Then what?**

## Estimating 1: Bottom Line

**Use historic data -- preferably from placebo controlled trials with same designs as upcoming NI trial**
**Bad news: historic data is often poor because of ethical constraints**
**conservative estimate not straightforward**
**Good news: precision is probably irrelevant if benefit is KNOWN to be very large**

## : Step 2

**Recall two step process**
**Step 1: Determine estimate of advantage of AC over placebo (********1****)**
**Step 2: Select acceptable loss from AC (********2****)**
** ******** ****= smaller of ********1**** and ********2**
**Selection of ********2**** is primary concern for most Anti-infective indications**

## Selection of clinically acceptable loss (2)

**NOT STATISTICAL DECISION**
**CLINICAL**** judgement of largest acceptable difference between AC and the new drug**
**a difference **_**>**_** ********2**** is so important clinically that it must be ruled out -- no tolerance for any difference **_**>**_** ********2**
** ********2**** is borderline value between just barely acceptable and NOT acceptable**

## Selection of clinically acceptable loss (2)

**Consequence of treatment failure**** is important factor in determining ********2**
**What proportion of study failures are deaths or other very serious morbidity?**
**Can treatment failure be easily reversed/addressed?**

## Clinically Acceptable Loss (2):Consider consequence to patients

## Clinically Acceptable Loss (2):Consider Clinical Trial Realities

**Some patients do not have disease**
**Trt diff :Pts with bacterial infection: 12%**
**Trt diff: Pts without bacterial infection: 0%**
**Measured trt diff: If 50/50 mix: 6%**
**If ********=10%, may conclude New is efficacious**
**BUT key population difference>10%**
**BE CAREFUL: Other factors dilute observed treatment difference too**

## Selection of clinically acceptable loss (2)

**SUMMARY:**
**Consequence of treatment failure**
**Potentially large impact on patient care **
**Be careful about clinical trial realities**
**CLINICAL judgement**

## General Approach:  for each indication

**Provides regulatory consistency**
**But, requires vigilance**
**New AC may be less efficacious than original AC (is ******** smaller than advantage of AC over placebo?)**
**Emerging resistance and other temporal changes may have diminished the efficacy of any AC**

## Consequence to Sample Size

****** ****cut in half, sample size quadruples**
**But**** if new drug is slightly better than AC, sample size can be sharply reduced**
**80% Power, ********=10%**
**Assume cure rates both 80%, SS/grp=252**
**Assume new drug rate is 82%, SS/grp=168**
**Sample Size cut by 1/3**

## BIGGEST CHALLENGES

**Biggest challenges in selection of ******
**Indications where treatment effect is potentially modest but not precisely known**
**Serious indications with low incidence**
**Superiority designs may offer an important alternative to NI design: stronger evidence and potentially smaller sample size**
**Can they be done ethically?**

## Early Escape: Ethical?

**Applicable to few indications (less serious)****	**
**Two arms: Experimental vs. placebo**
**KEY: Pts seen several days after baseline**
**If blinded assessment no improvement**
**Failure in analysis, therapy switched**
**Ethically consistent with “wait and see”**
**Variant: Early Escape with 3 arms**

## Other Superiority Designs

**Placebo Add-on:                                                      ****	****Existing + New   vs.   Existing + Placebo**
**Does New have benefit in presence of Existing therapy; labeling implications**
**Dose-response (High vs Low dose)**
**Superiority to some comparator**
**Combination**

## Selection of : BIG PICURE

**Take Home Message**

## Choice of  impacts patients

**If ******** is incorrectly chosen so that greater than advantage of AC over placebo**
**Patients may get drugs with no benefit, while exposed to toxicity and potential for development of resistance**
**Potential**** benefits of using smaller ********even where no concern about placebo rate******
**More pts cured overall, higher survival**
**Subtle but important differences detected**
**Smaller ********: larger and longer studies**

## Take Home Message

**Absolute: ******** MUST be smaller than conservative estimate of the advantage of Comparator over placebo**
**Challenge: insufficient historic data**
**Clinical judgement: may decrease ******** further**
**To rule out important loss in efficacy**
**Superiority designs can play important role in some settings **