# Presentation: 551500

## THE ROLE OF COVARIATES IN CLINICAL TRIALS ANALYSES

- Ralph B. D’Agostino, Sr., PhD
- Boston University
- FDA ODAC
- March 13, 2006

## OUTLINE

- Setting of Randomized Clinical Trials
- General issues for the use of covariates in primary analyses
- Clinical trial scenarios for covariate analysis (including secondary and third level analyses)
- From good to not-so-good practices
- Closing Comments

## SETTING OF RANDOMIZED CLINICAL TRIAL

- Two treatments to be compared
- Subjects randomized to treatments
- Desire to test differences of treatments on a primary end point (e.g., time to death)
- Baseline data (covariates) collected on subject
- Age, Gender, Severity, Location of Cancer, Clinic Site

## GENERAL ISSUES FOR USE OF COVARIATES IN PRIMARY ANALYSIS

- Randomization is assumed adequate to balance treatment groups and covariates not used in primary analysis
- Covariates used to reduce variability (to increase precision)
- Randomization may not balance treatment groups. Imbalances identified and these covariates put into primary analyses.

## GOOD PRACTICES

- The following scenarios have the feature of a careful analysis plan stated clearly in the protocol or statistical analysis plan (SAP) that was developed before data set is locked

## CLINICAL TRIAL SCENARIOSINCLUDING SECONDARY &THIRD LEVEL

**PRIMARY ANALYSIS:** RANDOMIZATION IS ASSUMED ADEQUATE TO BALANCE TREATMENT GROUPS AND COVARIATES NOT USED IN PRIMARY ANALYSIS
- A statistical test is performed comparing directly the two treatments with a statistical test that does not include covariate (e,g., log rank test on time to death (survival) analysis). This is the primary analysis.

## GOOD: T 1 better than T 2 Events Plots over Study Time

- 0.0

- .25

- . 50

- 30.0

- .75

- 1.00

- 0

- 0.5

- 1.0

- 1.5

- 2.0

- 2.5

- 3.0

- TREATMENT 1

- TREATMENT 2

- -

**Time from randomization in years**

## 1. NO COVARIATES USED IN PRIMARY ANALYSIS 

**SECONDARY LEVEL ANALYSIS** examines covariates and subgroups to see consistency and sensitivity of overall results
- Covariate analysis. In survival analysis extend log rank test to Cox regression that adds one or more covariates at a time (e.g., treatment and age to examine age effect in presence of treatment)

## 0.1

**0.1**

**1**

**10**

**Overall**

**Gender**

- Males
- females

**Age**

- < 65

- >65

**Years with **
**Condition**

**Hazard Ratio**

- yes

- no

**Location X**

- NO

- YES

**1 better     2 better**

**Special subgroup**
**Treatment really works**

## 1. NO COVARIATES USED IN PRIMARY ANALYSIS

**THIRD LEVEL ANALYSIS**
- Build a multivariate model that includes all important covariates and treatment
- This allows us to demonstrate consistency across variables and build a multivariate model that includes effect of treatment. It also can identify important subsets.

## CLINICAL TRIAL SCENARIOSINCLUDING SECONDARY &THIRD LEVEL

**2**. **PRIMARY ANALYSIS:** COVARIATES USED TO REDUCE VARIABILITY (TO INCREASE PRECISION)
- A statistical test is performed comparing directly the two treatments with a statistical test that adjusts for preselected covariates (e,g., Cox regression on time to death (survival) analysis that adjusts for age or severity). This is the primary analysis.

## 2. PRE-SELECTED COVARIATES USED TO REDUCE VARIABILITY

- Example: Weight reduction study use baseline weight
- Example: Age, disease severity and location of cancer may be a priori selected for primary analysis. These variables may also have been used in randomization for stratification

## 2. PRE-SELECTED COVARIATES USED TO REDUCE VARIABILITY

**SECONDARY AND THIRD LEVEL ANALYSES **ARE THE SAME AS BEFORE EXCEPT A-PRIORI SELECTED COVARIATES NEED TO BE CONSIDERED** **

## CLINICAL TRIAL SCENARIOSINCLUDING SECONDARY &THIRD LEVEL

**3.PRIMARY ANALYSIS** RANDOMIZATION MAY NOT BALANCE TREATMENT GROUPS. IMBALANCES IDENTIFIED AND THESE COVARIATES PUT INTO PRIMARY ANALYSES
- Potential covariates on which imbalances may exist are identified in protocol and tested for significance and then put into primary analysis (e.g., severity, gender, age) if significant
- Control of alpha error rates should be considered

## 3. COVARIATES FOR WHICH IMBALANCES IDENTIFIED FIRST

**SECONDARY AND THIRD LEVEL ANALYSES **ARE THE SAME AS BEFORE EXCEPT IDENTIFIED IMBALANCED COVARIATES NEED TO BE CONSIDERED** **

## BAD PRACTICE

- The following has the feature that the PRIMARY ANALYSIS FAILED and then exploratory analyses (stated as planned or not) undertaken to identify covariates and determine treatment effects

## PROBLEMS

**Once primary analysis fails the alpha error has been used. There is no way of retrieving it with a secondary or exploratory analysis **
**The variability often associated with this practice is often so large that it adds nothing beyond original failed primary analysis**

## OVERALL TEST IS NOT SIGNIFICANT

- TECHNICALLY YOU CANNOT GO BEYOND THIS WITH ANY STATISTICAL STATEMENTS
- USEFUL TO LOOK AT COVARIATES AND SUBSETS AS EXPLORATORY ANALYSIS (NOT EVEN APPROPRIATE TO CALL IT A SECONDARY ANALYSIS)

## PROBLEM: T1 not better  Events Plots over Study Time

- 0.0

- .25

- . 50

- 30.0

- .75

- 1.00

- 0

- 0.5

- 1.0

- 1.5

- 2.0

- 2.5

- 3.0

- TREATMENT 1

- TREATMENT 2

- -

**Time from randomization in years**

## SEARCH FOR COVARIATES

- Age ( > = 65 years vs < 65 years)
- ECOG Performance (1 or 2 vs 0)
- Tumor size (> 18.7 cm2 or not)
- Disease state

## ECOG Status EFFECT 

- 0.0

- .25

- . 50

- 30.0

- .75

- 1.00

- 0

- 0.5

- 1.0

- 1.5

- 2.0

- 2.5

- 3.0

- Status = 0

- 1 or 2

- -

**Time from randomization in years**

## PROCEDURE

- Can look at covariates one at a time related to outcome variable or in conjunction with treatment
- In the end a multivariate analysis is done with covariates and treatment in model

## TYPICAL RESULTS

- PRIMARY ANALYSIS
- HR = 0.98, .95 CI (0.78 TO 1.24)
- COVARIATE ANALYSIS AFTER FAILED PRIMARY ANALYSIS
- HR = 0.86, .95 CI (0.67 TO 1.10)
- TWO HRs NOT SIGNIFICANTLY DIFFERENT
- TWO CONFIDENCE INTERVALS OVERLAP (NO DIFFERENCE)

## ANOTHER TYPICAL RESULT

- Primary analysis
-  HR = 0.98 (.78 to 1.24)
- One set of covariates
- HR = 0.68 (.50 to .94)
- Another set of covariates
- HR = 0.89 (.63 to 1.05)
- WHAT DO WE BELIEVE?

## CONCLUSION: STATISTICAL PROPERTIES OF ANALYSES

- If Primary analysis is significance
- Then as secondary analyses we can examine covariates and control error rates (that is, can control chance of identifying falsely significant covariates). Looking for joint relation of treatment and covariates to end point
- If number of covariates is unspecified, then analysis is exploratory even here

## Conclusion: Statistical properties of analyses (continue)

- If Primary analysis is not significant, then we cannot control error rate (falsely identifying significant covariates and “saving” or improving treatment differences). **We have used “alpha” on overall test **
- Often do not get clarity from later analyses

## Closing Comments

- Careful analysis plans can be stated for dealing with covariates. Error rates (level of significance) can be controlled even for considering covariates if structure of statistical approach is clearly stated.
- We must not confuse statistical procedures with covariates stated in a pre-specified manner from covariate analysis that follow after failed primary analysis and have no way of controlling statistical error rates