# Presentation: 894002

## EVALUATING SURROGATE ENDPOINTS USING SENSITIVITY AND SPECIFICITY ANALYSES

- R. Sridhara,  G.Y.H. Chi, R.T. O’Neill
- Division of Biometrics 1
- OB, OPaSS, CDER, FDA

- BASS 2002

- 12/11/2002

## Disclaimer

- This talk is not an official FDA guidance or policy statement.  No official support or endorsement by the FDA is intended or should be inferred.

- BASS 2002

- 12/11/2002

## Outline

- Examples
- FDA Regulation
- Definition
- Illustration
- Other Validation methods
- Future Applications
- Summary
- References

- BASS 2002

- 12/11/2002

## Examples

- Tumor response as a surrogate endpoint for survival
- RNA viral load and CD4 counts as surrogate endpoints for AIDS infection
- Cholesterol levels as a surrogate endpoint for coronary artery disease
- Bone density measurements as a surrogate endpoint for incidence of fractures
- Arterial pressure as a surrogate endpoint for incidence of stroke

- BASS 2002

- 12/11/2002

## FDA Regulation: Subpart H - Section 314.510 

- “FDA may grant marketing approval for a new drug product on the basis of adequate and well-controlled clinical trials establishing that the drug product has an effect on a surrogate endpoint that is **reasonably likely**, based on epidemiologic, therapeutic, patho-physiologic, or other evidence, **to predict clinical benefit** or on the basis of an effect on a clinical endpoint other than survival or irreversible morbidity”.

- BASS 2002

- 12/11/2002

## Definition1

**Biomarker** (Biological Marker):  A characteristic that is objectively measured and evaluated as an indicator of normal biologic processes, pathogenic processes, or pharmacologic responses to a therapeutic intervention.
**Clinical Endpoint**:  A characteristic or variable that reflects how a patient feels or functions, or how long a patient survives.
**Surrogate Endpoint**: A biomarker intended to substitute for a clinical endpoint, i.e., a biomarker that is expected to predict clinical benefit, harm, or lack of benefit or harm.

- BASS 2002

- 12/11/2002

## Conceptual Model2

- Biomarkers (for Efficacy)

- Biomarkers (for Toxicity)

- Surrogate Endpoints (for Efficacy)

- Evidence that it is Likely to Predict

- Surrogate Endpoints (for Toxicity)

- Provisional Evaluation of Efficacy and Toxicity

- Clinical Endpoints (for Efficacy)

- Clinical Endpoints (for Toxicity)

- Continuing Assessment of Benefits to Risk Ratio

- BASS 2002

- 12/11/2002

## Effect of Therapeutic Interventions2

**Therapeutic Intervention**

**Clinical Endpoint**

**Biomarker**

**Affects**

**Measured to**
** ****Substitute for**

**Beneficial or Harmful Effects**
**Not Measured by a Biomarker**

- BASS 2002

- 12/11/2002

## Prentice’s3 Definition

- Surrogate endpoint is a response variable for which a test of null hypothesis of no relationship to the treatment groups under comparison is also a valid test of the corresponding null hypothesis based on the true endpoint

- BASS 2002

- 12/11/2002

## ILLUSTRATION4,5

- _Phase I Study of Suramin in Hormone Refractory Prostate Cancer Patients_:  (Single Arm Study)
- 109 Patients were treated using adaptive control with feedback dosing to maintain plasma suramin concentration within a target range at the University of Maryland Cancer Center between 1990-1993

- BASS 2002

- 12/11/2002

## Primary Objective: To assess Toxicity

Secondary Objective: To assess Efficacy 
Survival
Tumor Response - Assessment difficult and Subjective

- Secondary Objective: To assess Efficacy
- Survival
- Tumor Response - Assessment difficult and Subjective

- BASS 2002

- 12/11/2002

## Suramin:  Polysulphonated naphthylurea; Preliminary studies suggested anti-tumor activity in Prostate Cancer
Prostate Specific Antigen (PSA)
Enzyme produced by prostate glands
Normal range: 0 - 4 ng/ml
Elevated in most prostate cancer patients
Decrease in PSA levels seems to correlate with survival

- Prostate Specific Antigen (PSA)
- Enzyme produced by prostate glands
- Normal range: 0 - 4 ng/ml
- Elevated in most prostate cancer patients
- Decrease in PSA levels seems to correlate with survival

- BASS 2002

- 12/11/2002

## The Problem

- Is PSA a Surrogate Marker of Survival?
- Criteria for Response: % Decrease in PSA from baseline measurement to week 4 measurement
- PSA = 100 x _(Baseline PSA - Week 4 PSA)_
- 					Baseline PSA

- BASS 2002

- 12/11/2002

## Challenges

- Is 50% decrease in PSA a RESPONSE?
- Is 75% decrease in PSA a RESPONSE?
- Can survival for at least 1 year be considered gold standard for efficacy?

- BASS 2002

- 12/11/2002

## Survival Curves

- BASS 2002

- 12/11/2002

## Exploratory Analyses

- Tree-Based Model6
- Sensitivity Analyses7,8
- ROC curves
- Likelihood Ratios9
- Logistic Regression

- BASS 2002

- 12/11/2002

## Tree-Based Model

- An exploratory technique for uncovering structure in data
- Generally used in:
- Screening variables
- Devising prediction rules that can be further evaluated
- Summarizing large multivariate datasets
- Assessing adequacy of linear models

- BASS 2002

- 12/11/2002

## Tree-based modeling was used to identify a threshold value of percentage decrease in PSA
Survival time of 1 year was used as the the decision criterion
Model fitted by binary, recursive partitioning

- Survival time of 1 year was used as the the decision criterion
- Model fitted by binary, recursive partitioning

- BASS 2002

- 12/11/2002

## The Tree

- BASS 2002

- 12/11/2002

## Survival Analysis with Classification of the Tree

- BASS 2002

- 12/11/2002

## Interpretation of Tree Results

- Poor prognosis group: more than 15% increase of PSA (n = 7; probability of < 1 year survival = 1.0)
- Good prognosis group: > 70% decrease of PSA (n =31; probability of < 1 year survival = 0.1)
- Unknown prognosis group (n = 60; probability of < 1 year survival = 0.4)
- Results not helpful in identifying the threshold

- BASS 2002

- 12/11/2002

## Sensitivity Analyses

**Survival**
**    ****< 1 year****	****	******** 1 year**
**          ****+****	****	****	****       -**

******PSA**

- < X%
-    +

-  X%
-    -

- a

- c

- b

- d

- a + c

- b + d

- a + b

- c + d

- BASS 2002

- 12/11/2002

## Sensitivity Analyses

- Sensitivity = a/(a + c) = **S**
- 			  =  ___PSA < 50%, Survival < 1 yr_
- 			    # of patients with survival < 1 yr
- Specificity = d/(b + d)
- 			  =  ___PSA ____ 50%, Survival ____ 1 yr_
- 			    # of patients with survival __ 1 yr
- False Negative Rate = c/(c + d)
- False Positive Rate = b/(a + b)
- Relative Risk = {a/(a + b)}/{c/(c + d)} = **R**
- Attributable Proportion = **S** (1 - 1/**R**) = **AP**

- BASS 2002

- 12/11/2002

## Results of Sensitivity Analyses

- BASS 2002

- 12/11/2002

## ROC Curves

- BASS 2002

- 12/11/2002

## ROC Curves

- BASS 2002

- 12/11/2002

## Likelihood Ratios

- Likelihood Ratio (LR) =
- [{# of patients having PSA in a range R, and survival of < 1 year}/
- {# of patients having PSA in the same range R, and survival of  1 year}]

- BASS 2002

- 12/11/2002

## Likelihood Ratios

- BASS 2002

- 12/11/2002

## Logistic Regression

- BASS 2002

- 12/11/2002

## Other Validation Methods

- Freedman et.al.10
- Proportion of Treatment Effect explained (PE) by an intermediate endpoint in the setting of a binary outcome variable
- PE = 1 - s / 
- Conceptual problem: Not a true proportion (can be > 1 and  can be + or -)

- BASS 2002

- 12/11/2002

## Other Validation Methods

- Buyse et.al.11
- Relative Effect (RE): Effect of intervention on true endpoint relative to that of its effect on surrogate endpoint
- RE =  / 
- Adjusted RE = z  / 
- Binomial and normally distributed endpoints

- BASS 2002

- 12/11/2002

## Other Validation Methods

- Buyse et.al.12,13
- Relative Effect (RE): ratio of the treatment effect on survival over the treatment effect on tumor response
- RE = log HR / log OR
- Fitted a weighted regression to predict treatment effects on survival (log HR) from treatment effects on response (log OR): R2 gave the proportion of variability in log HR that could be explained by the variability in log OR

- BASS 2002

- 12/11/2002

## Other Validation Methods

- Molenberghs et.al.14
- Li et.al.15
- Pepe et.al.16
- Others

- BASS 2002

- 12/11/2002

## Points to Consider

- Unlike the other validation methods listed in previous slides, the sensitivity/specificity analyses and the attributable proportions used in the illustration are within one treatment arm - not compared to control.
- Measure of surrogacy in treated vs. control with respect to, for example, attributable proportion needs to be examined. Potentially this will enable to discriminate among multiple biomarkers.

- BASS 2002

- 12/11/2002

## Future Applications

- Generally illustrations and examples considered here and in literature are in the evaluation of efficacy
- Extremely useful in evaluating safety
- Early detection
- Sensitivity and Specificity Analyses originated from Diagnostic tests

- BASS 2002

- 12/11/2002

## Safety Evaluation

- Example:
- Hepatic Toxicity (True Endpoint)
- 	Toxicity = 0 (absent) or 1 (present)
- Bilirubin or SGPT (Surrogate Endpoint)
- 	Normal = 0 or Abnormal = 1

- BASS 2002

- 12/11/2002

## Safety Evaluation – Future Work

- Combination of two or more surrogate markers may better predict true endpoint
- Develop methods to combine attributable proportions or relative effects of two surrogate markers for the same true clinical endpoint

- BASS 2002

- 12/11/2002

## Challenges

- Surrogate independent of treatment
- Surrogate independent of baseline risk (demographics, behavior, pharmaco-genomics, etc.)
- Accuracy, Precision and Timing of measurement of surrogate

- BASS 2002

- 12/11/2002

## Summary

- Biomarkers serve as early indicators of benefit or risk
- Surrogate markers are biomarkers that can be substituted for a true clinical endpoint
- Validity of surrogate endpoint must be established
- Sensitivity/Specificity analyses are useful tools in evaluating surrogate endpoints

- BASS 2002

- 12/11/2002

## References

- 1. DeGruttola VG, et.al. (2001) Cont. Clin. Trials 22: 485-502
- 2. Biomarkers Definitions Working Group (2001) Clin. Pharm.Ther. 69:89-95
- 3. Prentice RL (1989) Stat. Med. 8:431-440
- 4. Sridhara R, et.al. (1995) J Clin. Oncol. 13: 2944-2953
- 5. Sridhara R, et.al.  (1998) Cancer Epi., Biomarkers & Prevention 7:631-34
- 6. Breiman L, et.al. (1984)
- 7. Steele G Jr, et al. (1982) Ann. Surg. 196: 162-169
- 8. Schatzkin A, et.al. (1990) J Natl. Cancer Inst. 82: 1746-1752
- 10. Giard RWM ,et.al. (1996) Eur J. Cancer 32A: 2042-2048
- 11. Freedman LS, et.al. (1992) Stat. Med. 11: 167-178
- 12. Buyse M, et.al. (1998) Biometrics 54: 1014-1029
- 13. Buyse M, et.al. (2000) Biostatistics 1: 49-68
- 14. Buyse M, et.al. (2000) The Lancet 356:373-378
- 15. Molenberghs G, et.al. (2001) Stat. Med. 20: 3023-3038
- 16. Li Z, et.al. (2001) Stat. Med. 20: 3175-3188
- 17. Pepe MS, et.al. (2000) Biostatistics 1: 123-140

- BASS 2002

- 12/11/2002

## References

- Temple RJ (1995) in Clinical Measurement in Drug Evaluation pp:3-22
- Fleming TR, et.al. (1996) Ann Intern Med 125: 605-613
- Herson J (1989) Stat Med 8: 403-404
- Ellenberg SS, et.al. (1989) Stat Med 8: 405-413
- Wittes J, et.al. (1989) Stat Med 8:415-425
- Hillis A, et.al. (1989) Stat Med 8:427-430
- Lin DY, et.al. (1997) Stat Med 16: 1515-1527
- Daniels MJ (1997) Stat Med 16: 1965-1982
- Gail MH, et.al. (2000) Biostatistics 1: 231-246
- Huang CC, et.al. (1999) N Engl J Med 341: 328-335
- The heart outcomes prevention evaluation study investigators (2000)     N Engl J Med 342: 145-153

- BASS 2002

- 12/11/2002