# Presentation: 162601

## Some Phase 23 Clinical Trial Designs

**Peter F. Thall**
**Biostatistics & Applied Mathematics Dept**
**M.D. Anderson Cancer Center**

***Statistical Design Issues in the Development of Type 1 Diabetes TrialNet Clinical Trials****** ***
***NIDDK, Bethesda, Maryland ***
***March 7, 2005***

## Outline

** ****Problems With  Conventional Methods**
** ****Two-Stage Select-and-Test Designs**
**- Binary Outcomes**
**- Time-to-Event Outcomes **
** ****A Bayesian Phase 23 Design**

## CRITICISMS

- 	**E-vs-S** treatment effect with between-trial effects
- - Data from phase 2 are discarded when phase 3 starts
- - There is a delay between phase 2 and phase 3

**CRITICISMS**

**CONVENTIONAL PHASE 2 CLINICAL TRIAL**

-  Small study of one experimental treatment (**E**)
-  Often a single-arm trial of **E **alone, without randomization
- - Efficacy and safety are evaluated using an early outcome (**Y**)
- - Data on **E** are compared to historical data on standard rx (**S**)
- If **E** is “promising”  Organize a randomized phase III trial of
**   ****E**-**vs**-**S** based on a harder, time-to-event outcome (**T**)

##  Assumes Y is a reasonable surrogate for assessing  E-vs-S treatment effects on survival time T

- - If one really believes **Y **predicts **T**, then ignoring **Y** data observed in phase 3 wastes information
- - While phase 3 is being organized, time is lost

**CONVENTIONAL PHASE 3 CLINICAL TRIAL**

-  Use phase 2 data to decide what to test in phase 3
-  Randomize between **E** and **S**, usually multi-center
-  Typically based on **T**= survival time or DFS time
-  The scientific standard for deciding if **E** is “effective”

**CRITICISMS**

## The “Select  and  Compare” Problem

**- K new experimental treatments,**** E****1 ****, . . . , E****K ****,**
**	****are available at the same time**
**Q****1****: Is *****any***** ****E****j**** ****substantively better than ****S**** ?**
**Q****2****: If two or more ****E****j****’s**** are substantively better than ****S****, which is the best ?**
**- Example:**** ****For rx of acute leukemia or MDS**
**	****S = idarubicin + ara-C (IA)**
**	****E****1 ****= IA + Mylotarg**
**	****E****2 ****= IA + Topotecan**
**	****E****3 ****= IA + Fludarabine**

## A Deeply Religious Approach

** ****Conduct a K+1 arm randomized phase 3 trial including ****E****1 ****, . . . , E****K ****and S**
** ****Perform all pairwise ****E****j ****versus S**** comparisons, controlling the overall false positive rate**
** ****Use a sample size that achieves a given power**

** “****Power” may be defined in many ways, since there are numerous desirable non-null cases.**
** ****If *****no***** ****E****j ****>> S****, this is extremely wasteful.**
** ****Even if *****one***** ****E**** i ****>> S****, this is still very inefficient.**

**Some Problems With This Approach**

## A Two-Stage Select-and-Test Design

**Thall, Simon and Ellenberg****  *****Biometrika *****75:823-831, 1988**** **

**Stage 1****: Randomize (K+1)n****1**** patients among ****E****1****,...,E****K****, S ****. **
**If the best ****E****j ****= ****E***** ****is minimally promising compared to S, then proceed to stage 2. Otherwise, ****stop**** and conclude that no ****E****j**** is better than S.**
**Stage 2****: Randomize 2n****2**** additional patients between **
**E***** ****and ****S****. Based on the data on ****E***** ****and ****S**** from both stages, decide whether ****E***** ****>> ****S****. **
**Control overall false positive rate and “power”**

## S

**S**

**E****1**

**E****2**

**E****3**

**S**

**E*******

**If E*********>S go to stage 2**
**If not,  ****stop ****and accept the null**

**Using all data on E********* and S, decide whether **
**or not E*********>>S**** **

**(K+1)n****1**** patients**

**2n****2**** patients**

## 	T1 and T2  are the test statistics at stages 1 and 2

**Stage 1**: T1 _<_ y1    Stop and accept Ho: ******1** =... = ******K** = ******0**
- 	    T1 > y1   Proceed to stage 2
**Stage 2**: T2 _<_ y2    Accept Ho: ******1** =... = ******K** = ******0**
- 	    T2 > y2   Conclude ********* > ******0**

******0**

******0 ****+ ********1 **

******0 ****+ ********2 **

**Marginal **
**improvement**
** ****over S**

**Clinically significant  **
**improvement**
** ****over S**

**Null **
**response rate **
**with S**

## Slide 10

| Outcome | Total # Patients | Error if |
| --- | --- | --- |
| T1 < y1    
Stop early and accept Ho | 
(K+1) n1 | Some 
Ej >> S |
| T1 > y1,T2 < y2    Accept Ho at stage 2 | 
(K+1) n1 + 2 n2 | Some 
Ej >> S |
| T1 > y1,T2 > y2   Conclude  E*>>S | 
(K+1) n1 + 2 n2 | No
Ej >> S |

## Optimization of Design Parameters

- The “least favorable configuration” (LFC) is
- 1 = = K-1 = 0 +  1     and   K = 0  +  2
- Given
-  = Overall type I error
-  = Generalized power = Pr(Conclude E* >> S | LFC)
- 1 , 2  and  0  :
- Find {n1 , n2 , y1 , y2 }  to minimize
- 		1⁄2 E(N | Ho)  +  1⁄2 E(N | LFC)

## Numerical Illustration

- K = 3****0 = .40,  = .75,  = .05, 1 = .05,  2  = .20 
- n1 = 47, n2 = 63, y1 = .55,  y2 = 1.944
- 	188 patients in stage 1
-  	126 patients in stage 2
-          314 maximum sample size
- 	Pr(T1 _<_ y1  | Ho) = .47    Eo(N) = 255
-  	T2 > 1.944  Choose  E* ; if not, retain S

## Advantages of the Phase 23 Design

**2) The null success probability ********0**** ****with S must be known **
**3) Frequentist ******** Underestimates variability**

**1) Randomize throughout ******** Stage 1 data on  E********* and**** ****S may be combined with the stage 2 data for the final test **
**2) Avoids the bias of an uncontrolled pre-test selection**
**3) Controls overall error rates **
**4) Simple to implement**

**Advantages of the Phase 23 Design**

**Drawbacks**

## Some Additional Practical Issues

- A binary outcome usually is inadequate in phase 3
- Patient prognostic covariate effects are often larger than any E-vs-S treatment effect
- 3) If, say, E1,E2,E3  come from three different drug companies, the company reps are very unlikely to want the drugs compared to each other.  Each company typically wants its drug compared only to S (or to nothing at all).
- 4) If E2 shows promise in phase 2 while a phase III trial of
- 	E1-vs-S is ongoing, should one :
-    a) start a separate trial of E2-vs-S ?
-    b) add E2 to the ongoing phase 3 trial ?
-    c) run the E1-vs-S trial be to completion and then compare E2 to the winner ?

## A Select-and-Test Design With Event Times

**Schaid, Wieand and Therneau,****  *****Biometrika *****77:507-513, 1990**** **

**Stage 1****: Randomize (K+1)n****1**** patients among ****E****1****,...,E****K**** ****,S**** during initial accrual period [0, t****1****].  **
**T****j****(t) = log rank statistic for E****j****-vs-S at time t**
**1) If T*********(t****1****) = max{T****1****(t****1****),.., T****K****(t****1****)} < c****1**** ******** ****Stop**** and conclude that no ****E****j**** is better than ****S****.**
**2)  If T*********(t****1****) = max{T****1****(t****1****),.., T****K****(t****1****)} > c****2**** ******** ****Stop**** and conclude that ****E********* is better than ****S****.**
**3) Otherwise, move all E****j ****with c****1**** **_**< **_**T****j****(t****1****) **_**<**_** c****2**** to Stage 2 **

## Stage 2: Randomize (K2+1)n2 patients among S and all K2 experimental treatments surviving stage 1. 

**Based on **
**T****j****(t****2****) = log rank statistic comparing ****E****j**** to ****S**** based on all data through time t****2****,     for  j=1,..., K****2 ****, **
**compare each (surviving) E****j**** to S using cut-off c****3**
**Control the overall type I error and “power”**

## Choose n1 , n2 , t1 , ta , t2 , c1 , c2 , c3  to control the overall type I error while achieving a targeted generalized pairwise power

**Choose n****1**** , n****2**** , t****1 ****, t****a ****, t****2 ****, c****1 ****, c****2 ****, c****3  ****to control the overall type I error while achieving a targeted generalized pairwise power**

**t****1**

**t****2 **

**Stage 2 Accrual **
**Terminated**

**Final **
**Tests**

**Stage 1**
**Decisions**

****

**t****a**

## Advantages of the Schaid-Wieand-Therneau Design

**2) Allowing multiple E****j****’s  to go to stage 2 may produce a very large trial**
**3) Frequentist ******** Underestimates variability**

**1) Randomize throughout ******** Stage 1 data may be combined with the stage 2 data for the final tests **
**2) Avoids the bias of an uncontrolled pre-test selection**
**3) Controls overall error rates **
**4) A computer program is available from Dan Schaid**

**Advantages of the Schaid-Wieand-Therneau Design**

**Drawbacks**

## A “Seamless” Phase 2 3 Design

**Standard Rx, S =** **CHEMO + RADIATION**
- Median survival with **S** = 15.5 months

**Experimental Rx, E =** **S** + **Ad-p53**
**Ad-p53** is an adenovirus injected into patient’s tumor
**Ad-p53** may restore apoptosis (programmed cell death) while sensitizing cancer cells to chemo-radiation

**A “Seamless” Phase 2******** 3 Design**
**Inoue, Thall, and Berry,****  *****Biometrics *****58:823-831, 2002**

## NSCLC trial

**Patient outcome is bivariate**** **
**- Survival time, T **
**- Local control, Y = Indicator that aspiration biopsy of the primary tumor at 5 mos. is negative, a “phase II” outcome (in general, Y may be multinomial)**

**Prognostic variable**
**Stage III (85% patients) vs. Stage II (15% patients)**

**Accrual & Sample Size**
**- Accrual rate during phase 2 = 20 patients/month**
**- Accrual rate during phase 3 = 30 patients/month **
**	****(add an additional 10 institutions) **
**- Limitations: 900 patients, follow-up 6 years**

##  Patients with LC are expected to have 

- better survival than those without LC (****)

-  Ad-p53 may improve the probability of achieving LC  (****)

- Ad-p53 may improve survival without mediation through LC (****)

## A MIXTURE MODEL

- Specify p(T | Y=y,Z) and p(Y=y|Z)
- Z = covariates, Y = category-valued variable
- Overall survival distribution is a mixture:
- p(T | Z) = y p(T | Y=y,Z) p(Y=y | Z)
- p(T) accounts for Y not observed if T< t0
- Z includes treatment and covariate effects

## Illustration of the Mixture Model

- RX

- LC

- No LC

- 5 months

**T**=Survival Time,       **LC**=Local Control
**P[T>24] = P[T>24|LC]* P[LC] +  P[T>24|no LC]* P[No LC]**
**If Prob(2-year survival) = .70 with LC and .40 w/o LC**
**TRIPLING ****P[LC]**** from .16 to .48  only increases ****Prob[T>24]**** from .45 to .54, that is, by 21%**

## Summary of the Design’s Main Features

- 1) The probability model and decision rules **account for BOTH local control and survival time**
**2) Randomize** patients between E & S **from the start**
**3) “Phase 2” data are included in “Phase 3”** decisions
**4) Decide whether to proceed from “Phase 2” to “Phase 3” repeatedly** during the interim period [8 - 12] mos, not at only one point in time
**5) Continue accrual** in “Phase 2” **while “Phase 3” is being organized**  No “down time”
**6) Decisions** (go to “Phase 3”, stop for futility or utility) **are** **based on** **predictive probabilities**

## A BAYESIAN PHASE 23 DESIGN

-  	[Mean Survival time without Ad-p53]

- [ > 0]=[Ad-p53 improves survival]

******1****(t)** = **Predictive probability** >0 if accrual is closed at time t & patients are followed 12 more months
******2****(t)**= **Predictive probability** >0 if accrual continues to 900 patients & follow-up continues to 72 months

**A BAYESIAN PHASE 23 DESIGN**

##  Deciding Whether to Go to Phase 3 

-  Large 2(t) likely  **Organize phase 3**
-  Large 2(t) unlikely  **Stop**,** conclude E<S**
-  Otherwise  **Continue phase II**
- Time to organize Phase 3 ~ 9 months

** ****Deciding Whether to Go to Phase 3**** **

##  Decision Rules Used in Both Phases 

-  Large 1(t) and large 2(t) both likely 
**	****Stop and conclude E>S**
-  Neither large 1(t) nor large 2(t) likely 
**	****Stop & conclude E<S**
** **Neither large 1(72) nor large 2(72) likely 
**	****Stop & conclude E<S**
-  Otherwise  **Continue**

** ****Decision Rules Used in Both Phases**** **

## HYPOTHESES

- H0

**HYPOTHESES**

- Median survival: 15.5 months

- Improvement of 25%
- over null median survival

- Ad-p53

- LC

- SURVIVAL

- Ad-p53

- LC

- H1

##  Simulation Study 

**THE BAYESIAN PHASE 23 DESIGN :**
- - 9 months to organize phase 3
- - Accrual increases from 20 to 30 patients per    	month when phase 3 begins
**10,000 Simulation Repetitions per scenario**

## “CONVENTIONAL” DESIGNS

**Phase 2** = Simon Optimal 2-stage (1989)

-   Early stopping rule for futility, size =.05,
- 	power =.90 to detect p1=.16 vs. p2 = .36
-      n1= 21 at stage 1 and n2=30 at stage 2

**Phase 3 = **Conventional group-sequential design

- -  O’Brien-Fleming bounds
- -  Early stopping for both efficacy and futility
- -  Overall Size =.05, Power =.80
- -  Maximum # tests = either 4 or 18

## Slide 31

## Slide 32

## A More Realistic Comparison:Duration of the Entire Phase 23 Process

- Conventional 2 + 3

- Bayesian  23

-  9.4

-  30.7

-  45.4

-  46.7

-  20.4

- 21.6

- H1

- H0*

- H0

- SURVIVAL

- Ad-p53

- LC

- H1

- Ad-p53

- SURVIVAL

- LC

## Advantages of the Bayesian Phase 23 Design

**1) All current data are included in all decisions**
**2) Accrual is not suspended to evaluate early outcomes**
**3) The trial continues and data accumulate while phase III is being organized**
**3) The mixture model may yield a much quicker trial **
**4) Bayesian ******** Accounts honestly for variability**

**Drawbacks **

**1) Very complicated ****(****a simpler version is in development****)**
**2) No usable computer code currently available ****(****see above) **

## Some Additional References

- Follman DA, Proschan MA, Geller NL. Monitoring pairwise comparisons in multi-arm clinical trials. *Biometrics* 50:325-336, 1994.
- Simon R, Thall PF, Ellenberg SS. New designs for the selection of treatments to be tested in randomized clinical trials (Disc: p447-451). *Statistics in Medicine* 13**:**417-429, 1994.
- Strauss N, Simon R. Investigating a sequence of randomized phase II trials to discover promising treatments. *Statistics in Medicine* 14: 1479-1489, 1995.
- Stallard N, Todd S. Sequential designs for phase III clinical trials incorporating treatment selection. *Statistics in Medicine* 22:689-703, 2003.
- Maitournam A, Simon R. On the efficiency of targeted clinical trials. *Statistics in Medicine* 24:329-339, 2005.