# Presentation: 681011

## The Orthopedic & Rehabilitation Device Advisory Panel  Meeting(November 21, 2002)

**Statistical Issues for PMA P000054**
**(Recombinant human bone morphogenetic protein-2/ Absorbable collagen sponge device  for orthopedic trauma, long -bone fractures)**
**Chang S. Lao, Ph.D.**
**Division of Biostatistics, OSB/CDRH/FDA**

## Outline (Statistical Review)

**1.****	****Pooling of multi-clinic data**
**	****	****(Meta-Analysis)**
**2.****	****Intra and inter-observer agreement (Reproducibility)**
**3.****	****Survival Analysis (time-specific) versus crude event ****	****(SI) probability**

## Important considerations (pooling)

**% No SI**
**(Success)**

**Center 2**

**Ideal**
**(Parallel)**

**Acceptable**
**(Quantitative**
**  ****center by   tr.interaction)**

**Questionable**
**(Qualitative int.)**

**Center 1**

**Center 2**

**Control**

**High**
**dose**

**High**
**dose**

**Control**

**Center 1**

**Control**

**High dose**

**Control**

**High**
**dose**

**Center 1**

**Center 2**

**Control**

**High dose**

**Control**

**High dose**

## Hypothetical example of wrong pooling

- 		                             **Site 1****         ****	****          ****Site 2****	****    ****	**** ****Pooled**
- 			**High   Control  Total    High  Control    Total    High  Control  Total **
**Outcome****	****dose****	****         ****	****         dose****	****    ****	****     dose**
**Success**                 **10          40      50           60         40      100        70        80  ****	**** 150**
**Failure**		   **20          80    100          30          20        50        50      100       150**
**Total**                       **30        120    150          90          60      150      120      180       300**
**Chi-square**		**0****	****	****           0 ****	****	****	****5.56**
**p-value**	         	         	**1 (NS)****	****	****           1****	****(NS)****	****	****0.0186 (Sig.)**
**% Success**	**33****	****33****	****         67      67****	****     58****	****   45**
**Invalid pooling by summing all observed cell numbers! **

##  Estimated Differences (high dose - control) in % Success (Free of SI) and the exact 95% CI by regrouped investigator sites

##  Estimated Differences (high dose - control) in % Success (Free of SI) and the exact 95% confidence intervals (CI), CCRE patients

## Statistical methods (Meta-analysis)

**Model**
**Fixed effect**
**Random effect**

**Assumption**
**Each center has the **_**same**_** clinical effect; consider within-center variability only**
**Diverse nature in study design; methods among sites, heterogeneity; consider both within-center and between-center variability**

## Meta Analysis on difference of two proportions of success (high dose - SOC)

**30 regrouped investigator sites **
**FDA: excluded 3 regrouped sites with zero variances (for both regrouped investigator sites and CCRE study; use method of moments, DerSimonian & Laird, Stat. in Med., 1986)**

## FDA results: Difference of two success (No SI) proportions, high dose – control,regrouped investigator sites

**Model****	****mean diff****	**** se****	****p-value****	****95% CL**
**Fixed ****	****	****0.1168 ****	****0.04****	****   <0.01     (0.039, 0.194)**
**Random****	****0.1207****	****0.0818     NS****	****      (-0.04,  0.28)**
- 2 = 95.7 (26 d.f.), p<0.005, rejects the null hypothesis (between-center variance = 0)
-  				random effect model is valid

## FDA results: Difference of two successproportions, high dose – control, CCRE

**Model****	****mean diff****	**** se****	****p-value****	****95% CL**
**Fixed ****	****	****0.097****	****	****0.042  <0.05       (0.014, 0.18)**
**Random****	****0.11****	****	****0.079     NS****	****      (- 0.05,  0.26)**
- 2 = 80 (26 d.f.), p<0.005, rejects the null hypothesis (between-center variance = 0)
-  		**random effect model is valid**

## Reproducibility Study (intra-and-inter-observer agreement) 

**n = 60 patients from 10 US****	****Trauma Centers**
**(different from PMA study; protocol #:C9612-11)**
- 	t**wo teams (multiple raters), fracture union**
- 					**Kappa ****	****95% CL****	****n**
**Inter-observer****	****	****0.87       (0.7, 1.0)****	****20**
**Intra-observer (1)****	****0.50****	****     (0.27, 0.74)****	****20**
**Intra-observer (2)****	****0.49        (0.25, 0.73)****	****20**

## Kappa Index for Agreement

**Prof. Gary Koch (Biometrics 1977)**
**	****Kappa****	****	****	****clinical interpretation**
**	****< 0****	****	****	****poor**
**0 - 0.2****	****	****	****slight**
**0.21 - 0.4****	****	****	****fair**
**0.41 - 0.6****	****	****	****moderate**
**0.61 - 0.8****	****	****	****substantial**
**0.81 - 1.0****	****	****	****nearly perfect**

## Problems of PMA (Kappa)

**Sample patient selection**
**Random sample ?  Representative?**
**Masking?**
**Time comparability?**
**Multiple raters/Team **

## Survival Analysis vs. crude SI event probability

**Life-table analysis - Patient follow-up data **
**recently received **
**Fracture healing assessed by Investigator**
**Radiographic assessment of fracture union by independent radiology panel**
**CCRE study**

## Survival Analysis in PMA

**Without seeing the patient follow-up data and survival analyses, patients censored, lost, missing, and at risk of SI cannot be evaluated at time t**
**	****(Assumption: Censoring independent of treatment)**
**Crude SI event probability < time-specific cumulative SI event probability by survival analysis, unless ALL patients had completed the entire follow-up study, or ALL patients with SI**

## Conclusion

- 	**Study Design**
**Heterogeneity among centers – random effect for combined analysis**
**Direct adding up all numbers and the corresponding analyses are not valid**
**Survival analysis is required**** ****due to patients censored, lost to follow-up**
**Questionable reproducibility studies**