# Presentation: 830266

## Statistical Review of the Observational Studies of Aprotinin Safety Part I:Methods, Mangano and Karkouti Studies

- CRDAC and DSaRM Meeting
- September 12, 2007
- Mark Levenson, Ph.D.
- Quantitative Safety & Pharmacoepidemiology Group
- Office of Biostatistics

- v.: 9/7/2007 AC Submit

## Acknowledgements to Dr. Mangano and Dr. Karkouti

## Outline

- Review Objectives
- Statistical Methods
- Mangano Review
- Karkouti Review
- Summary

## 1. Review Objectives

- To confirm the reported findings based on investigators’ methods
- To evaluate the statistical robustness of the findings
- FDA analyzed the 3 studies
- Same methods applied to all three studies
- Robust
- Diagnostics

## Outline

- Review Objectives
- Statistical Methods
- Mangano Review
- Karkouti Review
- Summary

## Propensity Scores (PS)

- Adjust for differences in baseline risk factors between two treatment groups (Treatment A and Treatment B)
**Definition**: Probability of assignment for a patient to Treatment A versus Treatment B based on measured risk factors
**Intuition:**
- Suppose Treatment A patient and Treatment B patient have the same PS.
- Comparisons between these two patients are fair

## Propensity Scores Practice

- Balance: similarity of distributions of a risk factor between treatment groups
- PS methods cannot account for unmeasured confounders
- The PS are estimated based on statistical modeling
- Diagnostics important

## Propensity Scores Practice (Cont.)

- Estimating treatment effects using PS
- Matching (Karkouti)
- Stratification (FDA)
- Multivariate regression (Mangano, i3)

## FDA Analysis Methods

- Pre-specified
- Propensity scores with stratification was used to adjust for baseline risk factors
- Medical, epidemiological, and statistical expertise was used to choose risk factors
- Diagnostics (analytical and graphical) were used to evaluate balance and explore findings

## Outline

- Review Objectives
- Statistical Methods
- Mangano Review
- Karkouti Review
- Summary

## Mangano Study: Key Points

- Prospectively specified
- Inclusion criteria
- Outcome definitions
- Subgroups
- Analysis methods
- In-hospital outcomes (NEJM)
- Long-term mortality follow-up outcomes (JAMA)
- 7 of 69 centers did not participate in the long-term follow-up

## Mangano Study: Key Points Analysis Methods

- Multivariate regression with and without propensity score as a covariate
- Logistic for in-hospital outcomes
- Cox PH for long-term mortality
- Propensity score for any active agent versus no agent for full analysis group
- No adjustment for geographical differences

## Mangano Study: Patients by Geographical Region and Treatment Group

| Region | No Agent
N=1374
% | Aprotinin
N=1295
% | Amino-caproic
N=883
% | Tran-examic
N=822
% |
| --- | --- | --- | --- | --- |
| Europe | 57 | 69 | 0 | 49 |
| North America | 24 | 29 | 96 | 29 |
| Other | 19 | 1 | 4 | 22 |

## Mangano Study: Long-Term Follow-up

|  | No Agent
N=1374
% | Aprotinin
N=1295
% |
| --- | --- | --- |
| Completed 5-year follow-up or died | 73 | 83 |
| No post-hospital follow-up | 10 | 1 |
| Lost to follow-up in the post-hospital period | 17 | 16 |

## Mangano Study: Demographic Factors

| Characteristic  | No Agent
N=1374 | Aprotinin
N=1295 | P-Value |
| --- | --- | --- | --- |
| Age (mean ± sd) | 63 ± 10 | 65 ± 9 | <.001 |
| Male (%) | 81 | 79 | 0.135 |
| African American or Hispanic (%) | 4 | 4 | 0.542 |

## Mangano Study: Selected Baseline Risk Factors

| Characteristic  | No Agent
N=1374
(%) | Aprotinin
N=1295
(%) | P-Value |
| --- | --- | --- | --- |
| Surgical: CABG + Other | 11 | 19 | <.001 |
| Surgical: Non-Elective | 21 | 15 | <.001 |
| History of liver disease | 8 | 12 | <.001 |
| History of renal disease | 13 | 19 | <.001 |
| Previous sternotomy | 3 | 13 | <.001 |
| Preop: Creatinine >1.3 mg/dL | 14 | 15 | 0.338 |
| Preop: Ejection fraction ≤ 44% | 18 | 15 | 0.071 |
| Preop: MI | 15 | 16 | 0.863 |

## Mangano Study:Study Reported Findings and Methods

- Primary findings of NEJM and JAMA based on investigators’ methods reproduced
- Imbalances in baseline risk factors and geographical regions between aprotinin and no-agent groups after PS adjustment
- Lack of overlap in propensity score distributions between aprotinin and no-agent groups

## FDA Analysis Results

## Mangano Study: FDA AnalysisBaseline Risk FactorsBefore and After PS Adjustment

|  | Before PS Adjustment |  |  | After PS Adjustment |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | No
Agent
% | Aprotinin
% | P-Value | No
Agent
% | Aprotinin
% | P-Value |
| Surgical: CABG + Other | 11 | 19 | <.001 | 15 | 16 | 0.543 |
| Surgical: Non-Elective | 21 | 15 | <.001 | 18 | 19 | 0.651 |
| History of liver disease | 8 | 12 | <.001 | 10 | 10 | 0.913 |
| History of renal disease | 13 | 19 | <.001 | 16 | 16 | 0.849 |
| Previous sternotomy | 3 | 13 | <.001 | 6 | 8 | 0.029 |
| Preop: Elev. Creatinine  | 14 | 15 | 0.338 | 14 | 14 | 0.943 |
| Preop: Eject. Fr.  ≤ 44% | 18 | 15 | 0.071 | 16 | 16 | 0.878 |
| Preop: MI | 15 | 16 | 0.863 | 15 | 15 | 0.745 |

## Mangano Study: FDA AnalysisIn-Hospital Outcome Adjusted EstimatesAprotinin vs. No Agent*

| Outcome | No Agent
(%) | Aprotinin
(%) | Risk Ratio
Aprotinin/No Agent
(95% CI) |
| --- | --- | --- | --- |
| Renal Composite | 4.8 | 7.8 | 1.63 (1.03, 2.60) |
|      Renal Failure | 2.5 | 5.1 | 2.05 (1.05, 3.99) |
|      Renal Dysfunction | 4.3 | 5.4 | 1.26 (0.76, 2.11) |
| Cardiovascular Composite | 19.5 | 22.2 | 1.14 (0.94, 1.38) |
|      Myocardial Infarction | 14.8 | 16.4 | 1.10 (0.88, 1.39) |
|      Congestive Heart Failure | 8.4 | 8.8 | 1.05 (0.75, 1.47) |
| Stroke | 2.0 | 2.8 | 1.36 (0.70, 2.64) |
| Death (in-hospital) | 4.6 | 4.2 | 0.91 (0.54, 1.53) |

- *Analysis based on 1307 no agent patients and 1222 aprotinin patients

## Mangano Study: FDA AnalysisRenal Composite

## Mangano Study: FDA AnalysisLong-Term Mortality Adjusted EstimatesAprotinin vs. No Agent*

| Outcome | No Agent 
(%) | Aprotinin
(%) | Risk Ratio
Aprotinin/No Agent
(95% CI) |
| --- | --- | --- | --- |
| 6 Weeks | 5.2 | 4.8 | 0.93 (0.57, 1.51) |
| 6 Months | 6.4 | 7.0 | 1.10 (0.73, 1.68) |
| 1 Year | 7.1 | 8.2 | 1.16 (0.79, 1.71) |
| 2 Years | 8.4 | 10.7 | 1.27 (0.90, 1.79) |
| 3 Years | 9.7 | 13.0 | 1.34 (0.98, 1.83) |
| 4 Years | 11.4 | 15.9 | 1.39 (1.05, 1.84) |
| 5 Years | 14.3 | 18.1 | 1.26 (0.98, 1.62) |

- *Analysis based on 1307 no agent patients and 1222 aprotinin patients

## Mangano Study: FDA AnalysisLong-Term Mortality Adjusted Estimates

## North American Subgroup

## Mangano Study: FDA Analysis by Region and Treatment Group

## Mangano Study: FDA AnalysisNorth America In-Hospital Outcome Adjusted EstimatesAprotinin vs. Aminocaproic*

| Outcome | Amino.
(%) | Aprotinin (%) | Risk Ratio
Aprotinin/Amino.
(95% CI) |
| --- | --- | --- | --- |
| Renal Composite | 2.4 | 9.4 | 3.90 (1.98, 7.70) |
|      Renal Failure | 0.6 | 5.3 | 9.17 (3.10, 27.15) |
|      Renal Dysfunction | 2.0 | 6.7 | 3.39 (1.54, 7.44) |
| Cardiovascular Composite | 17.7 | 22.5 | 1.28 (0.94, 1.74) |
|      Myocardial Infarction | 12.1 | 14.9 | 1.23 (0.83, 1.81) |
|      Congestive Heart Failure | 7.5 | 10.5 | 1.40 (0.84, 2.35) |
| Stroke | 1.5 | 3.2 | 2.10 (0.76, 5.81) |
| Death (in-hospital) | 2.1 | 4.7 | 2.21 (0.88, 5.52) |

- *Analysis based on 789 aminocaproic patients and 342 aprotinin patients

## Mangano Study: FDA AnalysisNorth America:Long-Term Mortality Adjusted Estimates Aprotinin vs. Aminocaproic*

| Outcome | Amino. 
(%) | Aprotinin
(%) | Risk Ratio
Aprotinin/Amino.
(95% CI) |
| --- | --- | --- | --- |
| 6 Weeks | 2.4 | 4.7 | 1.91 (0.82, 4.47) |
| 6 Months | 3.5 | 7.3 | 2.11 (1.04, 4.28) |
| 1 Year | 5.1 | 9.9 | 1.96 (1.10, 3.47) |
| 2 Years | 7.3 | 13.7 | 1.89 (1.19, 3.00) |
| 3 Years | 10.0 | 18.4 | 1.84 (1.23, 2.73) |
| 4 Years | 15.1 | 21.9 | 1.45 (1.03, 2.04) |
| 5 Years | 19.8 | 24.6 | 1.24 (0.92, 1.67) |

- *Analysis based on 789 aminocaproic patients and 342 aprotinin patients

## Mangano Study: Review Summary

- Renal outcomes (particularly renal failure) effect in a range of patients and in North American region subgroup
- Effects for cardiovascular, cerebrovascular, and in-hospital death outcomes not statistically demonstrated
- Long-term mortality effects in a range of  patients and in North American region subgroup

## Outline

- Statistical Review Objectives
- Statistical Methods
- Mangano Review
- Karkouti Review
- Summary

## Karkouti Study: Key Points

- Retrospective study of 5 years of patient data from a single center
- Patient population: Cardiac surgery (CABG and non-CABG) with cardio-pulmonary bypass
- Aprotinin used for high risk patients, tranexamic acid used for other patients
- Used propensity scores with 1-1 matching
- 449 of 586 aprotinin patients matched

## Karkouti Study: Demographic Factors

| Characteristic  | Tranexamic
Acid
N=10251 | Aprotinin
N=586 | P-Value |
| --- | --- | --- | --- |
| Age (mean ± sd) | 63 ± 12 | 55 ± 17 | <.001 |
| Male (%) | 75 | 65 | <.001 |

## Karkouti Study: Selected Baseline Risk Factors

| Characteristic  | Tranexamic
Acid
N=10251
(%) | Aprotinin
N=586
(%) | P-Value |
| --- | --- | --- | --- |
| Surgical: Not CABG only | 33 | 89 | <.001 |
| Surgical: Non-Elective | 8 | 19 | <.001 |
| Previous sternotomy | 5 | 61 | <.001 |
| Preop: Creatinine abnormal | 19 | 26 | <.001 |
| Preop: Ejection fraction <40% | 20 | 23 | 0.148 |
| Preop: MI | 17 | 7 | <.001 |

## Karkouti Study: Study Reported Findings and Methods

- Primary findings using investigators’ methods reproduced
- Observed risk factors balanced with propensity score matching approach

## Karkouti Study: FDA Analysis

- A subgroup of patients with overlap in propensity scores was defined to enable treatment comparison
- Subgroup contained
- 553/586 (94%) of the aprotinin patients
- 3759/10251 (37%) of tranexamic patients
- Baseline risk factors more similar between treatment groups in subgroup than full group

## Karkouti: FDA AnalysisAnalysis Subgroup, Baseline Risk FactorsBefore and After PS Adjustment

|  | Before PS Adjustment |  |  | After PS Adjustment |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | Tran.
% | Aprotinin
% | P-Value | Tran.
% | Aprotinin
% | P-Value |
| Surgical: Not CABG only | 83 | 92 | <.001 | 83 | 83 | 0.873 |
| Surgical: Non-Elective | 16 | 18 | 0.137 | 16 | 19 | 0.366 |
| Previous sternotomy | 14 | 64 | <.001 | 19 | 22 | 0.126 |
| Preop: Elev. Creatinine  | 3 | 5 | 0.010 | 3 | 3 | 0.950 |
| Preop: Eject. Fr. <40% | 21 | 22 | 0.426 | 21 | 19 | 0.693 |
| Preop: MI | 11 | 7 | 0.003 | 10 | 16 | 0.036 |

## Karkouti Study: FDA AnalysisTreatment Effect EstimatesAprotinin vs. Tranexamic Acid*

| Outcome | FDA Analysis
Risk Ratio
Aprotinin/Tran. | Matched-Pair
Odds Ratio
Aprotinin/Tran. |
| --- | --- | --- |
| Renal dysfunction |  | 1.53 (1.11, 2.12) |
| Renal failure | 1.38 (0.86, 2.23) | 1.85 (0.94, 3.63) |
| Myocardial Infarction | 1.42 (0.71, 2.83) | 1.22 (0.51, 2.95) |
| Stroke | 1.72 (0.93, 3.19) | 1.15 (0.55, 2.43) |
| Death (in-hospital) | 1.18 (0.79, 1.76) | 0.90 (0.54, 1.51) |

- *Analysis based on 3759 tran. patients and 553 aprotinin patients

## Karkouti Study: FDA AnalysisRenal Failure

## Karkouti Study: Review Summary

- Renal dysfunction effect statistically significant
- Some evidence for renal failure effect
- Effects for myocardial infarction, stroke, and in-hospital death outcomes not statistically demonstrated

## Outline

- Review Objectives
- Statistical Methods
- Mangano Review
- Karkouti Review
- Summary

## Summary

- Evidence for renal effect, including renal failure consistent
- Effects for cardiovascular, cerebrovascular, and in-hospital death outcomes not statistically demonstrated
- Evidence for long-term mortality effect
- Potential for unadjusted confounders between the treatment groups which may bias the treatment effect estimates