# Presentation: 633651

## Computerized Thermal Imaging  Breast Cancer System 2100 (CTI BCS2100)

- Radiological Devices Advisory Panel
- December 10, 2002
- Statistical Presentation
- Harry F. Bushar, PhD

- Harry Bushar

- 12/10/02

## Outline of Statistical Presentation

- Clinical Study Protocol
- Objective
- Design
- Population
- Demographics
- Evaluation (Effectiveness + Safety)
- PMA Clinical Study
- Effectiveness + Safety
- Amendment 4 Clinical Study
- Effectiveness
- Amendment 5 Clinical Study
- Effectiveness
- Amendment 7 Adjustment
- Effectiveness
- Statistical Conclusions

- Harry Bushar

- 12/10/02

## Clinical Study Protocol

- Study Objective
- “The objective of the study is to determine if the CTI System, when used in conjunction with clinical examination and/or diagnostic mammography, increases the ability of physicians to differentiate benign from malignant, or suspicious, breast abnormalities.”

- Harry Bushar

- 12/10/02

## Clinical Study Protocol

- Study Design
- prospective
- blinded-to-histology
- multi-center
- intended to compare
- Level of Suspicion (LOS) (0-5) of malignancy of suspicious breast lesions for clinical examination or diagnostic mammography before BCS to
- LOS + (BCS Index of Suspicion (IOS) (0.00 – 100.00) of malignancy of suspicious breast lesions)
-  biopsy as the “gold standard” for pathology.

- Harry Bushar

- 12/10/02

## Clinical Study Protocol

- Study Population
- Original study population = 600 patients with biopsy.
- Actual study population = 2,407 patients with biopsy.

- Harry Bushar

- 12/10/02

## Clinical Study Protocol

- Study Demographics
- Gender
- 2,364 Female
- 15 Male
- 28 Unknown
- Ethnicity
- 53% Caucasian
- 30% African American
- 13% Latino
- 2% Asian
- 1% Other
- 1% Unknown
- Age
- 12% (< 40)
- 55% (40 – 60)
- 32% (> 60)
- 1% Unknown

- Harry Bushar

- 12/10/02

## Clinical Study Protocol

- Primary Effectiveness on Overall Population
- Evaluation
- Area Under the ROC* Curve (AUC) to compare results of diagnostic mammography (LOS) without BCS (IOS) and diagnostic mammography with BCS (LOS + IOS).
- Sensitivity
- Specificity
- “The CTI System will be considered effective if its performance in conjunction with diagnostic mammography and/or clinical examination is clinically better than mammography and/or clinical examination alone.”
- *ROC = Receiver Operating Characteristic

- Harry Bushar

- 12/10/02

## Clinical Study Protocol

- Secondary Effectiveness on Subpopulations
- Mammographic Lesion Type
- Calcifications
- Masses
- Distortions
- Mammographic Lesion Size
- < 0.5 cm
- 0.5 – 1.0 cm
- > 1.0 cm
- Mammographic Lesion Depth
- As available

- Harry Bushar

- 12/10/02

## Clinical Study Protocol

- Safety Evaluation
- occurrence of adverse events.

- Harry Bushar

- 12/10/02

## PMA Clinical Study Population

- The sponsor acquired BCS images from 2,407 patients at 6 US clinical sites from 12/20/96 through 4/30/01.
- The sponsor actually analyzed only those patients with both
- mammography, not those with just clinical examination, and
- biopsy within 60 days.

- Harry Bushar

- 12/10/02

## PMA Training Clinical Study 

- 700 patients, consisting of the first 220 patients + 480 patients randomly selected from among the next 1,912 patients, were used to set the following BCS IOS cut-off:
-  ≥ 20.59 implies a recommendation for biopsy of a given lesion and
-  < 20.59 implies a recommendation for short-interval follow-up of a given lesion.

- Harry Bushar

- 12/10/02

## PMA Testing Clinical Study

- 1,432 patients, enrolled from 12/20/96 through 10/30/00, were initially available to test the effectiveness of the BCS both in the original PMA and in Amendment 4.
- 769 patients, with 187 malignant lesions + 688 benign lesions, were actually included in the effectiveness evaluation both in the original PMA and in Amendment 4.
- Note that each patient had 1 to 4 lesions.
- Note also that the sponsor assumes that lesions within patient are independent.

- Harry Bushar

- 12/10/02

## PMA Clinical Study Results

- Primary Effectiveness by ROC AUC
- The sponsor found, after excluding calcifications alone, a statistically significantly greater ROC AUC for (IOS + LOS1) than for mammography LOS1 scores (1-4) alone at significance level = 0.05.
- The sponsor found, after again excluding calcifications alone, but expanding mammography LOS1 scores beyond (1-4) to include 2 additional intermediate categories (3.50 & 3.75), no statistically significant difference between ROC AUC for (IOS + LOS2) & (LOS2 alone) at significance level = 0.05.

- Harry Bushar

- 12/10/02

## Slide 14

- Harry Bushar

- 12/10/02

## Slide 15

- Harry Bushar

- 12/10/02

## PMA Clinical Study Results

- Safety
- The following 4 adverse events occurred out of 2,407 subjects (12/20/96 through 4/30/01):
- 2 “mild”, possibly-related, resolved adverse events were both associated with patient discomfort during positioning.
- 1 “serious” and 1 “mild”, not-likely-related, resolved adverse events were hospitalization for treatment of a pre-existing metabolic disorder and dizziness when sitting up after thermal imaging, respectively.

- Harry Bushar

- 12/10/02

## Amendment 4 Clinical Results

- Overall (187 malignant + 688 benign)
- Sensitivity = 97.1% with
- 	95% CI* = (94.1%, 98.8%)
- Specificity = 14.3% with
- 	95% CI  = (12.1%, 16.6%)
- *CI = Confidence Interval

- Harry Bushar

- 12/10/02

## Amendment 4 Clinical Results

- Calcifications* (105 malignant + 368 benign)
- Sensitivity = 94.8% with 95% CI = (89.6%, 97.8%)
- Specificity =   9.2% with 95% CI = (  6.9%, 12.1%)
- Masses (90 malignant + 322 benign)
- Sensitivity = 100% with 95%  CI = (96.7%, 100%)
- Specificity = 18.0% with 95% CI = (14.6%, 21.9%)
- Distortions (16 malignant + 15 benign)
- Sensitivity = 100% with 95%  CI = (82.9%, 100%)
- Specificity = 16.7% with 95% CI = ( 3.9%, 40.2%)
- *The sponsor interpreted these effectiveness clinical results by lesion type to specifically exclude calcifications alone.

- Harry Bushar

- 12/10/02

## Amendment 4 Interpretation

- The sponsor’s initial rejection of overall effectiveness, followed by the sponsor’s differential findings among the 3 lesion type sub-populations clearly indicates exploration:
- 	which does require confirmation,
- 	which must be based on new data.

- Harry Bushar

- 12/10/02

## PPMA* Amendment 5 Population

- Study population = 275 additional patients.
- Gender
- 274 female
- 1 male
- Ethnicity
- 65% Caucasian
- 31% African American
- 3% Latino
- 1% Asian or Unknown
- Age
- 13% (< 40)
- 60% (40 – 60)
- 27% (> 60)
- *PPMA = Post-PMA

- Harry Bushar

- 12/10/02

## PPMA Amendment 5 Study

- 275 additional patients, enrolled from 11/1/00 through 4/30/01 at 3 of the 6 original US clinical sites, were initially available for confirmation of the effectiveness of the BCS in Amendment 5.
- 173 patients, with 43 malignant + 159 benign lesions, were actually included in the Amendment 5 effectiveness evaluation.
- Note that each patient had 1 to 3 lesions.
- Note also that the sponsor assumes that lesions within patient are independent.

- Harry Bushar

- 12/10/02

## PPMA Amendment 5 Results

- Overall (43 malignant + 159 benign)
- Sensitivity = 93.8% with
- 	95% CI = (84.0%, 98.5%)
- Specificity = 20.0% with
- 	95% CI = (14.9%, 26.0%)

- Harry Bushar

- 12/10/02

## PPMA Amendment 5 Results

- Calcifications (24 malignant + 87 benign)
- Sensitivity = 93.1% with 95% CI = (77.9%, 99.0%)
- Specificity = 17.4% with 95% CI = (11.1%, 25.5%)
- Masses* (15 malignant + 63 benign)
- Sensitivity = 93.3% with 95% CI = (72.1%, 99.7%)
- Specificity = 25.4% with 95% CI = (16.6%, 36.0%)
- Distortions (8 malignant + 4 benign)
- Sensitivity = 75.0% with 95% CI = (40.0%, 95.4%)
- Specificity = 25.0% with 95% CI = (  1.3%, 75.1%)
- *The sponsor interpreted these effectiveness results by lesion type to specifically include only masses.

- Harry Bushar

- 12/10/02

## Amendment 7 Adjustment

- The sponsor’s attempted Bonferroni adjustment, in response to FDA deficiency 1.a), by
- widening sensitivity/specificity CI estimates, which are
- based on a simple direct combination of exploratory + confirmatory clinical data (PMA/Amendment 4 + PPMA Amendment 5), to
- test a theoretically possible hypothetical set of hypotheses for
- 7 (lesion types) or
- 63 (lesion types, sizes, & depths),
- which hypotheses are not explicitly included in the protocol,
- Is not statistically acceptable, because
- the sponsor simply estimates sensitivity/specificity CI’s for various sub-populations,
- without actually statistically testing any hypotheses.
- Therefore, there are no multiple comparisons requiring adjustment.

- Harry Bushar

- 12/10/02

## Statistical Conclusions

- Diagnostic mammography, not just clinical examination, is required for use of BCS.
- The sponsor’s primary effectiveness demonstration, using ROC AUC, loses statistical significance, when mammography LOS (1-4) is expanded by just 2 additional intermediate categories (3.50 & 3.75), after excluding calcifications alone.
- The sponsor’s initial rejection of overall sensitivity, followed by rejection of calcification-alone sensitivity, indicates exploration:
- which requires confirmation,
- which requires new data.
- The sponsor’s attempted Bonferroni adjustment, by widening sensitivity/specificity CI estimates, is not statistically acceptable.

- Harry Bushar

- 12/10/02