# Presentation: 668170

## Statistical Methods in the Evaluation of Red Blood Cell Products(In vivo study)

**Jessica Kim, Ph.D.**
**Mathematical Statistician **
**FDA/CBER/OBE/DB**
**Blood Products Advisory Committee**
**91st Meeting, May 1, 2008 **
**Rockville Pike, Rockville, MD**

## Outline

**Background**
**Current recommendations for the RBC products: Statistical methods **
**Emphasis on the individual viability**
**Statistical power of a study**
**BEST (Biomedical Excellence for Safer Transfusion) Collaborative study on RBC recovery**
**FDA’s analysis of BEST/FDA data**
**Summary**

- May 2008 BPAC

## Background

| Period | FDA acceptance criteria for RBC products 
in vivo study |
| --- | --- |
| ~1997 | 75% RBC survival |
| 1998~
2003 | Mean RBC survival ≥ 75%
Standard deviation ≤ 9%
At least 20 units at 2 sites |
| 2004~ | One sided lower 95% CI for the population proportion of successes > 70%
(A success: RBC survival ≥75%) |

**CI: Confidence Interval**

- May 2008 BPAC

## FDA Current Acceptance Criteria:   in vivo 24-hour RBC survival study

| A one-sided 95% lower confidence limit for the population proportion of successes is 
	 > 70% (95-70 Rule)
“Success” : in vivo RBC survival ≥ 75%  |
| --- |
| The sample mean survival is ≥ 75%
The sample standard deviation is ≤ 9%
At least 20 units in total/At least 2 sites |

- May 2008 BPAC

## In Vivo Study:Study Hypothesis 

**A one-sided ****95%**** lower confidence limit for the**** population proportion of successes ****is**** > 70% ****(“****Success****” : *****in vivo***** RBC survival ≥ ****75%**** )**
**Null hypothesis: **
**Population proportion of successes**
**   ******** ****70%**
**Alternative hypothesis: **
**Population proportion of successes**
**   ****>  ****70%**

- May 2008 BPAC

**Notes:**

Note that the way statisticians think of  hypotheses is that the alternative is what we want to show, and the null is everything else.
Because of the single cut-off value under a testing hypothesis approach, we can also state the issue as “Determining the maximum unacceptable proportion of success”.
 We also want to have “large” minimum acceptable proportion of successes 

## Study Hypothesis (cont.)Based on the BEST data

**In vivo RBC Survival %**

**Proportion of **
**successes**

**BEST: Biomedical Excellence for Safer Transfusion**

- May 2008 BPAC

## Sample Size to Meet Minimum Acceptable Proportion of Successes

| Observed proportion of successes | # of allowable failures | One-sided lower 95% CI for the population proportion of successes  |
| --- | --- | --- |
| 21/24 (87.5%) | 3 | 70.8% |
| 24/28 (85.7%) | 4 | 70.2% |
| 28/33 (84.9%) | 5 | 70.8% |
| 31/37 (83.8%) | 6 | 70.5% |
| 35/42 (83.3%) | 7 | 71.0% |

- May 2008 BPAC

## Sample Size and Statistical Power

**Statistical Power**
**The ****likelihood**** of achieving a statistically significant result if your research hypothesis is actually true. **
**If the population proportion of successes is greater than 70%, what is the ****probability**** that a RBC recovery study will meet the acceptance criteria?**
**Sample Size and Statistical Power**
**If the ****likelihood**** is good (e.g., at least 80%), then your sample size would be considered adequate.**

- May 2008 BPAC

## Power Consideration

| True Rate | Sample Size (# of allowable failures) |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | 9 (0) | 14 (1)  | 19 (2) | 24 (3)  | 28 (4) | 33 (5) |
| 0.75 | 0.075 | 0.101 | 0.111 | 0.115 | 0.135 | 0.132 |
| 0.8 | 0.134 | 0.198 | 0.237 | 0.264 | 0.315 | 0.329 |
| 0.85 | 0.232 | 0.357 | 0.441 | 0.505 | 0.587 | 0.626 |
| 0.9 | 0.387 | 0.585 | 0.705 | 0.786 | 0.858 | 0.894 |
| 0.95 | 0.630 | 0.847 | 0.933 | 0.970 | 0.989 | 0.995 |

- May 2008 BPAC

## Power Consideration (cont.)

**P: true rate**

**N=24**

**N=28**

**N=33**

- May 2008 BPAC

## BEST Study Results of RBC RecoveryRBC Type: 42d Liquid Stored

| RBC
Type | N | RBC Recovery (%) |  |  | Frequency 
of RBC 
Recovery
 75% | Frequency 
of RBC 
Recovery
 70% | Frequency 
of RBC 
Recovery
 67% | Mean 
(%) | SD
(%) |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  |  | Mini
mum | Lower 
5%
Quantile | Med-
ian |  |  |  |  |  |
| 42d
Liquid
Stored | 641 | 36 | 70.7 | 82.4 | 88.3% | 95.5% | 98.1% | 82.1 | 6.71 |

**Source: This table is quoted from Dr. Dumont’s paper**

- May 2008 BPAC

## BEST Study Results of RBC RecoveryRBC Type: 42d Liquid Stored (cont.)

| Recovery Success
Threshold | Estimated “true” rate from the BEST data | Probability1 meet FDA current criteria (95-70 rule): n=24 |
| --- | --- | --- |
|  75% | 88.3% | 0.693 |
|  70% | 95.5% | 0.979 |
|  67% |              98.1% | 0.999 |

**1**
**  ****Binomial probability with at most 3 failures out of 24**
**  ****Source: This table is quoted from Dr. Dumont’s paper**

- May 2008 BPAC

## BEST Data# of studies meeting old criteria vs. current criteria

| Year | old criteria | current criteria |
| --- | --- | --- |
| 1990 | 1 | 0 |
| 1991 | 1 | 0 |
| 1992 | 1 | 0 |
| 1994 | 1 | 1 |
| 1995 | 2 | 1 |
| 1996 | 2 | 2 |
| 1999 | 3 | 2 |

| Year | old criteria | current criteria |
| --- | --- | --- |
| 2000 | 2 | 2 |
| 2001 | 2 | 1 |
| 2002 | 2 | 2 |
| 2003 | 2 | 2 |
| 2004 | 1 | 1 |
| 2005 | 2 | 2 |
| 2006 | 1 | 1 |

**Note:  Four additional ****approved studies**** (2004~2007) will be added in the FDA’s analysis**

- May 2008 BPAC

## BEST and FDA Data RBC Survivals by Year-Study

**FDA data: 2004.83, 2007.57, 2007.61, and 2007.74**

**Horizontal line indicates cutting value for an individual unit to be considered as **
**a success:75%**

- May 2008 BPAC

## BEST and FDA DataProportion of Successes

**Studies didn’t meet the current criteria**

- May 2008 BPAC

## Statistical Power Considerationn=735 (BEST:641 + FDA:94)combined BEST & FDA data set analyzed by FDA

| True Rate
Using BEST  /FDA data | Sample Size (# of allowable failures) |  |  |  |
| --- | --- | --- | --- | --- |
|  | 20 (2) | 24 (3)  | 28 (4) | 33 (5) |
| 0.888 (≥ 75%) |    0.608 |    0.721 | 0.801 |  0.842 |
| 0.901 (≥ 74%) |    0.683 |    0.791 |  0.862 |  0.898 |
| 0.927 (≥ 73%) |    0.824 |    0.906 |  0.950 |  0.970 |

- May 2008 BPAC

## BEST and FDA Data combined data set analyzed in different time periods

|  | 1990~1997
(n=140) |  | 1998~2007
(n=549) |  |
| --- | --- | --- | --- | --- |
|  | success  rate  | power         n=24 | success  rate | power        n=24 |
| RBC%≥75% | 0.836 | 0.43 | 0.898 | 0.78 |
| RBC%≥74% | 0.864 | 0.58 | 0.907 | 0.82 |
| RBC%≥73% | 0.886 | 0.71 | 0.933 | 0.93 |
| # of studies meeting current criteria | 4 out of 8 
 |  | 17 out of 19 |  |

**BEST and FDA Data ****combined data set analyzed in different time periods**

**Note: study 40 and 41 were not included (46 data points) due to unknown approval year**

- May 2008 BPAC

## BEST and FDA Data combined data set analyzed in different time periods (cont.)

- May 2008 BPAC

## BEST and FDA Datacombined data set analyzed in different time periods (cont.)

|  | 1990~2003
(n=516) |  | 2004~2007
(n=173) |  |
| --- | --- | --- | --- | --- |
|  | success
rate | power n=24 | success
rate | power n=24 |
| RBC%≥75% | 0.870 | 0.62 | 0.931 | 0.92 |
| RBC%≥74% | 0.884 | 0.70 | 0.942 | 0.95 |
| RBC%≥73% | 0.915 | 0.86 | 0.948 | 0.97 |
| # of studies meeting current criteria | 13 out of 19 
 |  | 8 out of 8 
 |  |

**BEST and FDA Data****combined data set analyzed in ****different time periods (cont.)**

**Note: study 40 and 41 were not included (46 data points) due to unknown approval year**

- May 2008 BPAC

## BEST and FDA Data combined data set analyzed in different time periods (cont.)

- May 2008 BPAC

## Statistical Power Considerationn=689 (595 + 94) from 1990 to 2007

| Threshold value of RBC recovery | 1990~1997 
(n = 140) |  | 1998~2003 
(n = 376) |  | 2004~2007
(n = 173) |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | Success
Rate  | Power
N = 24 | Success
Rate  | Power
N = 24 | Success
Rate  | Power
N = 24 |
| ≥ 75% | 0.836 | 0.43 | 0.883 | 0.69 | 0.931 | 0.92 |
| ≥ 74% | 0.864 | 0.58 | 0.891 | 0.74 | 0.942 | 0.95 |
| ≥ 73% | 0.886 | 0.71 | 0.926 | 0.90 | 0.948 | 0.97 |
| # of studies meeting
current criteria | 4 of 8  |  | 9 of 11 |  | 8 of 8 |  |

**Statistical Power Consideration****n=689 (595 + 94) from 1990 to 2007**
**combined data set analyzed in different time periods**

**Note: study 40 and 41 were not included (46 data points) due to unknown approval year**

- May 2008 BPAC

## BEST and FDA Data combined data set analyzed in different time periods (cont.)

- May 2008 BPAC

## Summary

**Clinical significance to ensure that **_**each**_** recipient receives  a high viable RBC product should be emphasized **
**The power to meet FDA’s “95-70 rule” depends on the study size as well as an estimate for the population proportion**
**Increase study size**
**Estimate “true” rate under **_**current acceptance criteria**_** /or under **_**more recent time period**_
**80% power has been used for a typical clinical trial**

- May 2008 BPAC

**Notes:**
Current acceptance criteria serve well for the purpose of regulating high viable RBC products

## Summary (cont.)

**The data show that manufacturers are able to produce better products over time**
**Current acceptance criteria serve well for the purpose of regulating high viable RBC products**

- May 2008 BPAC