# Presentation: 670824

## Statistical Review

- Chul H. Ahn, Ph.D.
- Biostatistician
- CDRH/FDA

## Population and Sample

- Population - a collection of objects of interest
- Sample – a subset of the population
- Want to know about the population using      sample.
- Should be representative of the population.

## BTT Study (original submission)

- Two-arm, non-randomized study (225 pts)
-      Either the patient received the Novacor LVAS (Treatment) or was treated with medical management (Control group)
- 190 LVAS patients, and
-      35 Controls (28 retrospective, 7 concurrent)
- Primary endpoint - survival time, with transplant considered as censoring

## Using the subset of data from BTT study
    - Seek to expand the indication to include the  
      patients with “relative contraindication (RC).”

Of the 225 patients, the sponsor found 87 patients who are considered to have “RC.”

Of the 87 patients, 75 are LVAS, and 12, Controls.

The sponsor claims device’s effectiveness based on these 87 patients.

-     - Seek to expand the indication to include the
-       patients with “relative contraindication (RC).”
- Of the 225 patients, the sponsor found 87 patients who are considered to have “RC.”
- Of the 87 patients, 75 are LVAS, and 12, Controls.
- The sponsor claims device’s effectiveness based on these 87 patients.

- Current Study

**Notes:**
The current study is using the subset of data from their original submission. It is seeking to expand the indication to include the patients with so called “relative contraindication.” The sponsor used six criteria to choose the patients with “relative contraindication.”  //

They found 87 such patients out of 225. It is almost 40%.  I am wondering why there were so many “RC” patients in this BTT study.  
  - Did they become worse after listed?
  - Or were they listed by mistake by the centers?  //

Among 87 patients, there were 75 LVAS patients and 12 Control patients.//

And, they claimed the efficacy of device based on these 87 patients.//   

## BTT 

**190**
**LVAS**

**35 ****Con**

**BTT **
**Population**

**Population**
**With**
**“****Relative Contraindication”**

**75 ****RC**

**12 ****RC**

**87 “RC”**

**Notes:**
This red circle shows the 225 patients from the BTT study.//

The sponsor’s intention was to make this sample a representative of the BTT population. When they conducted the BTT study, they never thought about the population with so called “RC.”  //

After the BTT study had been completed, the sponsor tried to seek to expand the indication to include the patients with “relative contraindication” //  

The sponsor found the subgroup with “RC” from BTT sample. //

There were 87 patients, 75 from LVAS  group and 12 from control group. //

These 87 patients may be a subset of the population with “relative contraindication.” However, it may not be a representative of that population. //

This population with “RC” is different from the BTT population, and moreover, we have little information about this population in terms of their size and characteristics(?). //  

## BTT 

**190**
**LVAS**

**35 ****Con**

**BTT **
**Population**

**Population**
**With**
**“****Relative Contraindication”**

**75 ****RC**

**12 ****RC**

**87 “RC”**

**Notes:**
This red circle shows the 225 patients from the BTT study.//

The sponsor’s intention was to make this sample a representative of the BTT population. When they conducted the BTT study, they never thought about the population with so called “RC.”  //

After the BTT study had been completed, the sponsor tried to seek to expand the indication to include the patients with “relative contraindication” //  

The sponsor found the subgroup with “RC” from BTT sample. //

There were 87 patients, 75 from LVAS  group and 12 from control group. //

These 87 patients may be a subset of the population with “relative contraindication.” However, it may not be a representative of that population. //

This population with “RC” is different from the BTT population, and moreover, we have little information about this population in terms of their size and characteristics(?). //  

## Retrospective Subgroup Analysis

- Can the sponsor extend the result based on the  87 patients to the entire population with “relative contraindications”?
-  Problematic, because this subgroup may
-      not be representative of the population
-      with “relative contraindications”

**Notes:**
Now, the question is whether the result based on this subgroup analysis can be extended to the entire population with “relative contraindication” //

It may be problematic because this subgroup may not represent such population //

## 75 LVAS vs. 12 CONTROLs

-    Are they comparable?
**The year of implant**
**Baseline covariates**
**Propensity scores**

**Notes:**
After the sponsor identified 87 patients as those with “RC”, they compared the primary endpoint between 75 LVAS and 12 Controls. //
However, the question is whether these two groups are comparable. //
We will look at three items to examine this.  They are the year of implant, baseline covariates, and propensity scores. //

If we go through all these three items, we will arrive at the following conclusion that these two groups are not comparable. Or, any direct treatment comparisons on primary endpoints are inappropriate. First, we will look at the year of implant. //

## Year of Implants

## Differences in Baseline Covariates

_*Factor               Control      LVAS      p-value*__* *_
- Milrinone               0.0           0.3        0.000
- Pre. Antihyper.  83.3%      33.3%       0.001
- Age                     41.8         49.9         0.038
- Prev. TIA          16.7%        1.3%        0.047
- Dobutamine        12.7           9.1         0.065
- Creatinine             2.5           1.9         0.076
- Bleeding           16.7%        2.7%        0.084

**Notes:**
Now, I would like to show you that there are large difference in the baseline covariates between two groups. This is the list of covariates which shows statistically significant difference between two groups with p-value < 0.1, the cutoff value for marginal significance. They are ordered in the magnitude of p-value. They are Milrinone, Pre- Antihyper., age, history of TIA (transient ischemic attack), dosages of Dobutamine, Creatinine level,  bleeding. (You may notice that age is in favor of Control.) //  

Dobutamine and Milrinone are medications for pulmonary congestion, hypotension, or right ventricular infarction. 

## Two treatment groups are not comparable

Imbalance of the year of implant
Imbalance in multiple baseline covariates

Any direct treatment comparisons on effectiveness endpoints are problematic

So, all p-values from direct treatment comparisons are uninterpretable

**Imbalance of the year of implant**
**Imbalance in multiple baseline covariates**
- Any direct treatment comparisons on effectiveness endpoints are problematic
- So, all p-values from direct treatment comparisons are uninterpretable

- Treatment Comparisons

## 
What about treatment comparisons adjusting for imbalanced covariates?

Traditional covariate analysis 
Propensity score analysis

Example of adjustment for one covariate, e.g., health condition 

- What about treatment comparisons adjusting for imbalanced covariates?
**Traditional covariate analysis **
**Propensity score analysis**
- Example of adjustment for one covariate, e.g., health condition

- Treatment Comparisons (cont.)

## One Covariate: Health Condition 

- Suppose the event rate is higher in the control group where there are sicker patients.
- Then, the lower event rate in the treatment group may not be due to the treatment, but simply _because there are healthier patients in the treatment group_
-  Should Compare patients with similar
-      health condition!

**Notes:**
Before we consider the propensity scores, I would like to talk   a little bit about covariates. 
We say that health condition is an important covariate when the event rate depends on health condition .  //

Suppose the event rate is higher in the control group where there are sicker patients.//

Then, the lower event rate in the treatment group may not be due to the treatment, but simply because there are healthier patients in the treatment group.// 

In this case, we need to compare patients with the similar health condition! And, we want to see some overlap in health condition between the two treatment groups. Otherwise, two treatment groups won’t be comparable.

##    One solution: Replace the collection of covariates with one single number (propensity score)

       Age, Gender,                                           Propensity
      Health condition, ...                               Score (PS) 

PS: The conditional probability of receiving the LVAS, given a patient’s observed baseline covariate values, e.g., age, gender, prior cardiac surgery, ...

-    One solution: Replace the collection of covariates with one single number (propensity score)
**       ****Age, Gender,****                                           ****Propensity**
**      ****Health condition, ...****                               ****Score (PS) **
- PS: The conditional probability of receiving the LVAS, given a patient’s observed baseline covariate values, e.g., age, gender, prior cardiac surgery, ...

- What If There Are Many Covariates?

**Notes:**
What if there are many covariates? //

One solution is to replace the collection of covariates with one single number, called, propensity score.  //

Like age in the previous slide, we compare patients with similar PS,//

 and we want to see enough overlap in PS between the two treatment groups.
 Otherwise, two treatment groups will not be comparable.

## When the propensity scores are balanced across the treatment and control groups, the distribution of all the covariates are balanced in expectation across the two groups

So, we can use the propensity scores as a diagnostic tool to measure treatment group comparability

The two treatment groups will be comparable if there is enough overlap between them.

- So, we can use the propensity scores as a diagnostic tool to measure treatment group comparability
- The two treatment groups will be comparable if there is enough overlap between them.

- Propensity Scores

**Notes:**
Here are more comments about the propensity scores.
When the propensity scores are balanced across the treatment and control groups, the distribution of all the covariates are balanced in expectation across the two groups //

So, we can use the propensity scores as a diagnostic tool to measure treatment group comparability //

However, the two treatment groups did not overlap enough to allow a sensible treatment comparison //

Hence, any treatment comparisons adjusting for imbalanced covariates are problematic

## Propensity Score Analysis

- Adjusted for all imbalanced and/or clinically important baseline covariates
- The propensity score model accurately predicted the treatment group membership
- However, the two treatment groups did not overlap enough to allow a sensible treatment comparison

## Propensity Scores

- 6

- 3

- 3

- 57

- 6

- 5

- 7

- (76%)

- (9%)

**Notes:**
This graph shows two box plots. A box plot provides an excellent visual summary of many important aspects of a distribution. The left box-plot shows the distribution of propensity scores for the control group. 

The box stretches from the lower hinge (defined as the 25th percentile) to the upper hinge (the 75th percentile) and therefore contains the middle half of the scores in the distribution.
The median is shown as a line across the box. The "H-spread" is defined as the difference between the hinges and a "step" is defined as 1.5 times the H-spread. Inner fences are 1 step beyond the hinges.

In the left, 6 observations corresponding to 50% of the data are inside box, and in the right, 57 observations corresponding to 76% of the data are inside the box. And, these two boxes do not overlap at all. As you can see there isn’t enough overlap to compare two treatment groups. // 
(( So, as I said earlier, we come to a conclusion that two groups are not comparable.))

Outer fences are 2 steps beyond the hinges.There are two adjacent values: the largest value below the upper inner fence and the smallest value above the lower inner fence. As shown in the figure, a line is drawn from the upper hinge to the upper adjacent value and from the lower hinge to the lower adjacent value.

((( 25% of the distribution is between median line and the top of the box and 25% of the distribution is between median line and the bottom of the box.)))

## Distributions of Propensity Scores

**8**

**4**

**10**

**13**

**17**

**17**

**18**

- <.749

- <.922

- <.983

- <.996

- <1.00

## Propensity Score Analysis

- So, any treatment comparisons adjusting for imbalanced covariates are problematic

## Sponsor’s Survival Curves for BTT

## Sponsor’s Survival Curves for “RC”

## Concerns with Sponsor’s Survival Curves

- Two treatment groups are not comparable
- Survival time is not independent of censoring
-  p-values may not be interpretable
- If we assume that the two treatment groups were comparable, then we may also find a subgroup with significant difference in survival curves between two treatments

## Survival Curves for Subgroup X

## Population

**“****RC”**

**X**

**Y**

- With “RC”

**“****RC”**

- BTT Population

## Statistical Summary

- Subgroup with “RC” may not be  representative of intended patient population
- Two groups (75 LVAS and 12 Controls) are not comparable so that any direct or covariate adjusted treatment comparison is problematic