# Presentation: 071712

## A Regulatory Perspective on Adaptive Randomization*

-                         H.M. James Hung
-      Division of Biometrics I, OB, CDER, FDA
***Presented in 41******st****** DIA Annual Meeting, Washington, DC, June 27, 2005***
*******The views presented are not necessarily of the U.S. Food and **
**Drug Administration**

- James Hung, 2005 DIA Meeting

##          Acknowledgment

-     Thanks are due to Charles Anello for his comments and suggestions.

- James Hung, 2005 DIA Meeting

##                                Outline

- Fundamental of Clinical Trial
- Fisher’s ‘Analyze as You Randomize’
- Simple Randomization
- Baseline Adaptive Randomization (BAR)
- Guidance from Literature on BAR
- Some Regulatory Concerns
- Summary

- James Hung, 2005 DIA Meeting

##      Fundamental of Clinical Trial 

**Randomization (random allocation)**
-    to achieve balance in all known and unknown,
-    observed and unobserved covariates (e.g.,
-    prognostic factors) at baseline
- Baseline balance is “expected” to be achieved in the long run (in  average). Apparent or accidental imbalance may occur in any individual trial.
**Blinding**
-    to minimize operational bias and selection bias

- James Hung, 2005 DIA Meeting

## Fisher’s ‘Analyze as You Randomize’

-     Base statistical inference on design (use design to generate p-value of statistical test).
- The frame of statistical inference is based on the _probabilistic treatment assignment in all possible ways_ that the design can generate, conditional on the observed outcome and covariates of the patients in the trial.
- This usually cannot be approximated by simulating trial outcome according to some population model.

- James Hung, 2005 DIA Meeting

##               Simple Randomization

- Method 1:  Sample sizes in each treatment group
-   are known exactly a priori
-      each group size is fixed exactly at  *m** *
- Method 2: Target sample size is established but
-   final sample size is not known with certainty
-   (most common in large clinical trials)
-      target sample size is fixed at *2m *but the final
-      sample sizes are  (*n**1*, *n**2*)  where *n**1* and *n**2** *
-      often differ from  *m*  and  *n**1*+*n**2*  *2m*

- James Hung, 2005 DIA Meeting

##      Simple Randomization (Cont’d)

**Under this design, ANOVA approximates randomization test properly.**
- For large trials, simple randomization is
- satisfactory in balancing [Lachin (1988)] on
- baseline covariates.

- James Hung, 2005 DIA Meeting

##      Simple Randomization (Cont’d)

- For small or moderate trials, baseline imbalance
- may appear (more often) and is certainly of
- concern in practice, even when it is due to chance
- alone.
- Adjusting for covariates that appear to be
- unbalanced from post hoc examination of data is
- not good statistical practice.
- - prespecify adjustment for all known prognostic covariates in statistical analysis plan

- James Hung, 2005 DIA Meeting

##      Simple Randomization (Cont’d)

- To improve balancing, stratification can be used with simple randomization (e.g., patients can be stratified by age and/or gender) and simple randomization is carried out within each stratum.
-  restriction in # of covariates to stratify on
- Random permuted blocking with a small block size can also be used to improve balancing at the local level, e.g., block size of four: (ABBA), (ABAB), (BAAB), ...
- - high predictability if cell totals are known

- James Hung, 2005 DIA Meeting

##  Baseline Adaptive Randomization

** ****(BAR)**
- Treatment assignment of the next patient, or the
- probability of the assignment, determined to
- minimize a measure (need pre-specify?) of
- overall covariate imbalance when that patient’s
- covariate values are considered
- - probabilistic  (e.g., Pocock-Simon)
- - deterministic (e.g. Taves minimization)

- James Hung, 2005 DIA Meeting

## Baseline Adaptive Randomization

- Ex.  To improve balance on gender distribution,
- _probabilistic BAR_ ( p = prob. of assigning A)

- _deterministic BAR_

| Pt  |   M |   M |    F |   M |   F |
| --- | --- | --- | --- | --- | --- |
| p |  0.5 |   0.3 |   0.5 |   0.4 |   0.8 |
| trt |   A |    B |    B |    A |    A |

| Pt  |    M |    M |    F |    M |    F |
| --- | --- | --- | --- | --- | --- |
| p |  0.5 |   0.0 |   0.5 |   0.0 |   0.5 |
| trt |    A |    B |    B |    B |    A |

- James Hung, 2005 DIA Meeting

## Baseline Adaptive Randomization

- _Probabilistic procedure_
-  do better balancing than simple randomization
-  random treatment allocation
-  design-based inference is available
-     e.g., probability distribution of test generated
-     by re-randomizing patients conditional on the
-     order of enrollment of the patients
- - statistical analysis can be based on a
-    population model that the patients in the trial
-    come from a homogeneous population (
-    unverifiable assumption)

- James Hung, 2005 DIA Meeting

## Baseline Adaptive Randomization

- _Deterministic procedure_
- - do better balancing than probabilistic BAR
-  largely non-random treatment allocation (except
-    for 1st few patients or for ties)
-  permutation test might not be available unless
-    patient entry into the trial is a random process
-    (unverifiable assumption)
- - statistical analysis is mostly based on a
-    population model that patients in the trial are
-    from a homogeneous population (unverifiable
-    assumption)

- James Hung, 2005 DIA Meeting

##   Guidance from Literature on BAR

- Balancing on covariates tends to decrease variance of estimates.
- Unadjusted standard test has conservative type I error with BAR.
- Adjusted test (adjusting for covariates for balancing) has appropriate significance level.
- Randomization methods do not allow tests of
- alternatives or generation of confidence intervals (model-based methods still needed)

- James Hung, 2005 DIA Meeting

##   Guidance from Literature on BAR

- BAR can be difficult if interactions between factors for balancing can be predicted.
- Prefer to incorporate a random vector to further reduce predictability and to allow calculation of a randomization test (if needed).
- If there are time trends in the patient baseline characteristics or strong correlation between outcome and patient entry order, then the standard tests w/o proper adjustment are conservative (**??**).

- James Hung, 2005 DIA Meeting

##   Guidance from Literature on BAR

- Balance on unknown covariates with BAR cannot be worse than with simple randomization.
- Balancing with BAR can improve efficiency of treatment effect estimate.
- For deterministic BAR, predictability is high with knowledge of marginal totals and algorithm

- James Hung, 2005 DIA Meeting

##     Some Regulatory Concerns (1)

**     **
- With BAR, analysis must be adjusted for the
-  covariates employed for balancing in order to
-  yield tests of proper size.
- However, unlike quantitative response variable,
- other types of response variables (e.g., binary, time
- to event, categorical) often rely on a nonlinear
- model for covariate adjustment (e.g., logistic, PH
- regression).

- James Hung, 2005 DIA Meeting

## Some Regulatory Concerns (1, Cont’d)

- Even under simple randomization, the covariate adjustment using nonlinear model can result in a smaller p-value but a larger variance of the effect estimate than unadjusted analysis in an individual trial. That is, the treatment effect estimate is larger than that produced by unadjusted analysis.
**Will BAR have additional adverse impact on p-value or type I error rate of standard test?**
**- In practice, need to compare the results of covariate adjusted analysis with the results of re-randomization test**

- James Hung, 2005 DIA Meeting

##      Some Regulatory Concerns (2)

- In the presence of a strong time trend in patient’s baseline characteristics, the literature suggests that standard tests tend to have the type I errors distorted toward conservative side under BAR  all based on simulation studies
**Will the distortion never be anti-conservative?**
**This question is related to Question 1).**
**- In practice, need to compare the results of covariate adjusted analysis with the results of re-randomization test**

- James Hung, 2005 DIA Meeting

##      Some Regulatory Concerns (3)

- Under BAR, are there any additional difficulties in handling dropouts in analysis?  probably yes,  because analyses need adjustment for covariates for balancing in the model.
-  not even sure of how to handle missing values
-   due to MCAR

- James Hung, 2005 DIA Meeting

##          Some Regulatory Concerns (4)

- In many situations, interim analysis and data monitoring are necessary. So is some design modification (e.g., sample size re-estimation).
- If BAR is used, **what will the potential impact be on interpretation of trial results?**
-  statistical validity
-      Are the common group-sequential methods
-      still applicable?
-  logistics and trial conduct issues

- James Hung, 2005 DIA Meeting

## Some Regulatory Concerns (5)

- With deterministic BAR, designed-based inference might not be possible (sample space might be too small?)
-  conditional on patients’ entry order?
-  conditional on covariates considered?
-  conditional on patients (assuming patient entry is
-    random)?
- High predictability
-  how to verify that this is not an issue for a
-   practical application

- James Hung, 2005 DIA Meeting

##                         Summary

- Probabilistic BAR may help balancing on baseline covariates in small trials.
-  need to prespecify allocation probability rule
-    select allocation probability  low predictability
-  need to check validity of standard asymptotic test
-    compare with re-randomization test
-  be aware of potential compromise on blinding
-    need SOP and document trial management
-  need a pre-specified plan to deal with unexpected
-    strong time trends in covariates in analysis

- James Hung, 2005 DIA Meeting

##                                               Summary

- Logistics needs to be carefully considered in use of
- adaptive randomization.
-  need to anticipate and deal with logistical
-    problems at the design stage
-  SOP may be needed
-  have 3rd independent party execute allocation?
-  how to avoid operational errors due to practical
-   complexity

- James Hung, 2005 DIA Meeting

##                              Summary

- Deterministic BAR is discouraged because of uncertainty about analysis and interpretation, though some argue that it is mostly harmless.
-  no ability to do design-based inference unless
-   patient entry is random (unverifiable assumption)
**Why taking risk of possible compromise on false positive rate to seek a complete balance?**
- - treatment allocation may be predictable.
**Why taking risk of selection or operational bias?**

- James Hung, 2005 DIA Meeting

## Selected References

**Selected References**
- Busye and McEntegart (2004, Applied Clinical Trials)
- CPMP Points to Consider document on Adjustment for Baseline
-    Covariates (2003)
- Forsythe (1987, Computational Statistics & Data Analysis)
- Frane (1998, Drug Information Journal)
- Halperin and Brown (1986, Statistics in Medicine)
- ICH E-9 guidance on statistical principles for clinical trials
- Kalish and Begg (1987, Controlled Clinical Trials)
- Lachin (1988, Controlled Clinical Trials)
- Lachin, Matts, Wei (1988, Controlled Clinical Trials)
- Ohasi (1990, Environmental Health Perspectives)
- Permutt and Grosser (2004, SOCT talk)
- Pocock and Simon (1975, Biometrics)
- Scott, McPherson, Ramsay, Campbell (2002, Controlled Clinical
-    Trials)

- James Hung, 2005 DIA Meeting

## Selected References

**Selected References**
- Taves (1974, Clinical Pharm. Ther.)
- Weir and Lees (2003, Statistics in Medicine)
- Green (2004, presentation in DIA-Japan workshop)
- Hagino, Hamada,Yoshimura, Ohashi (2004, presentation in DIA-
-   Japan workshop)
- Ohashi (2004, presentation in DIA-Japan workshop)
- Rosenberger and Lachin (2002, book)

- James Hung, 2005 DIA Meeting