# Presentation: 502675

## “Clinical Benefits, Decision Rules, and Multiple Inferences” 

**George Chi**
**Director**
**DBI/OB/OPaSS/CDER**
**             ****CDER Scientific Rounds**
**                             ****February 12, 2003**

## Acknowdgement

**Bob O’Neill**
**Peiling Yang**
**John Lawrence**

## Outline of Talk

**Rationale**
**Definition of Endpoints**
**Primary, Secondary, and Composite Endpoints**
**Clinical Decision Rules**
**Definition and Examples**
** ****Multiple Inferences**
**Clinical Decision Rules and Multiple Inferences**
**Control of the Probability of Making An Erroneous Inference**
**Endpoint Specificity/Non-specificity, Strong and Weak Control**
**Conclusions and Recommendations**

## Rationale

**Conceptualize current practices**
**Describe the link between the clinical and statistical roles **
**Avoid or minimize unnecessary problems**
**Permit clearer understanding of some complex issues**
**Clear up ambiguity and confusion**
**Avoid inappropriate applications of multiple testing procedure**
**Improve consistency in regulatory practices**

## Primary Endpoints

**Definition 1: In a clinical trial, **
**                  *****primary endpoints*****,               **
**    ****is a set of clinical endpoints based on which clinical benefits are assessed.**
**Primary endpoints usually provide characterization**
**        ****of various aspects of a disease**
**These primary endpoints are used to describe**
**    ****clinical benefits**

## Examples of Primary Endpoints

**Example 1:**
**Anti-hypertension trial**
**Supine diastolic blood pressure**
**Example 2:**
**Congestive heart failure trial**
**All cause mortality**
**stroke**
**myocardial infarction **

## Example of Primary Endpoints

**Example 3:**
** **
**Alzheimer trial**
**Alzheimer Disease Assessment Scale - Cognitive** **Subscale **
**Clinician’s Interview Based Impression of Change**

## Example of Primary Endpoints

**Example 4:**
**Epilepsy trial**
**Percent reduction in seizure rate**
**Percent reduction in drop attack rate**
**Parental global evaluation of seizure severity**

## Example of Primary Endpoints

**Example 5:**
**Acne trial**
**Clinical global**
**Inflammatory lesion counts**
**Non-inflammatory lesion counts**
**Total lesion counts**

## Secondary Endpoints

**Definition 2: In a clinical trial, *****secondary endpoints***** is a set of clinical endpoints that are intended for possible inclusion in the label, after efficacy has been demonstrated**
**Secondary endpoints are generally not intended for making primary efficacy claim**
**Clinical decision rule and statistical inferential structure supporting the decision rule do not cover the secondary endpoints**
**Multiplicity adjustment needed**

## Composite Endpoints

**Definition 3: A *****composite endpoint***** is an endpoint that is defined in terms of two or more primary clinical endpoints (at the patient level)**

## Composite Endpoints

**Rationale for Composite Endpoints**
**Disease needs multidimensional characterization**
**Low event rates in component endpoints**
**Mortality needs to be accounted for**
**Treatment effect on individual primary endpoints may be small**
**Increase the power of demonstrating overall treatment effect**
**Not certain which components will win**

## Example of Composite Endpoint

**Example 6: ****Rheumatoid Arthritis trial**
**Primary endpoints:**
**tender joint count **
**swollen joint count**
**patient pain assessment**
**patient global assessment**
**physician global assessment**
**patient self-addressed disability**
**acute-phase reactant (ESR or CRP)**

## Rheumatoid Arthritis (ACR20)

**Define a binary composite endpoint as follows:**
** ****A patient has a response, if he/she shows at least a**
**20% improvement in tender joint count ****and**** **
**20% improvement in swollen joint count ****and**
**at least a 20% improvement in 3 out of 5 following endpoints:  **
**patient pain assessment**
**patient global assessment**
**physician global assessment**
**patient self-addressed disability**
**acute-phase reactant (ESR or CRP)**

## The LIFE Study

**Example 7: This is a study of Losartan (COZAAR) vs Atenolol in 9193 hypertensive patients**
**There are three primary endpoints,**
** ****{Cardiovascular death, stroke, myocardial infarction}**
**Composite(CV death, stroke, MI) = The time to the first occurrence of either CV death, stroke, or MI **

## The LIFE Study

**Example 7: Losartan (COZAAR) vs Atenolol (Cont.)**
**Trial Outcomes**
**Primary Composite Endpoint (508 vs 588, p = 0.021)**
**Components of Primary Endpoint (as a first event)**
**Stroke, fatal/non-fatal (209 vs 286)**
**Myocardial infarction, fatal/non-fatal (174 vs 168)**
**Cardiovascular death (125 vs 134)**

## The LIFE Study

**Example 7: Losartan (COZAAR) vs. Atenolol (Cont.)**
**    ****Individual Component Endpoints:**
**Stroke, fatal/non-fatal (232 vs. 309, p = 0.001)**
**Myocardial infarction, fatal/non-fatal **
**    ****(198 vs. 188, p = 0.491)**
**Cardiovascular death (204 vs. 234, p = 0.206)**
**Due to CHD (125 vs. 124, p = 0.839)**
**Due to Stroke (40 vs. 62, p = 0.032)**
**Others (39 vs. 48, p = 0.411)**

## The LIFE Study

**Composite(CV death, MI, Stroke)**

**p = 0.021**

**Composite(CV death, Stroke**)

**p = 0.0027**

**Composite (CV death, MI)**

**p = 0.45**

**Composite (MI, Stroke)**

**p = 0.046**

**Stroke**

**P = 0.001**

**CV death,p = 0.21**

**MI, p=0.49**

## Composite Endpoints

**Some Questions for Composite Endpoints:**
**A positive outcome based on a composite endpoint only demonstrates that the drug has some “overall” clinical benefit contributed by one or more of the component endpoints**
**Clearly, composite endpoints vary from one to another.  The following are some general questions that may not be appropriate for any specific composite endpoints **

## Composite Endpoints

**Some Questions for Composite Endpoints:**
**Do we give the composite claim?**
**If so, how should/do we describe the clinical benefit claims based on a composite outcome? **
**How do we determine if a component contribute to the composite?**
**What is the basis for excluding a component from the claim?**

## Composite Endpoints

**Some Questions for Composite Endpoints:**
**Should we require inference on individual component endpoints?**
**Is it more advantageous to the sponsor to have a claim based on the composite or on an individual component - a paradox?**
**What is the best strategy for analyzing a composite endpoint?  Closed testing procedure for exclusion?**

## Clinical Decision Rules

**Definition 4:  A *****clinical decision rule***** is a decision rule describing the *****desired***** clinical benefits (winning scenarios) in terms of the outcomes on the primary endpoints**
**To actually assess a win (or wins), require statistical (multiple) testing procedure **
**In statistical parlance, a winning scenario corresponds to the part of the parameter space defined by the alternative hypothesis**

## Diagram

**Primary Endpoints**

**Clinical Decision Rule + Statistical Testing Procedure**

**Positive Decision**

**Negative**

**Claims of Clinical Benefits**

**Secondary Endpoints**

## Clinical Decision Rules

**Example 8: In a Congestive Heart Failure trial,**
**The primary endpoints: {all cause mortality}**
**The clinical decision rule is**
**             ****< win on all cause mortality >**
**Secondary endpoints **
**                         ****{stroke, MI} **

## Clinical Decision Rules

**Example 9: In a Congestive Heart Failure trial,**
**The primary endpoints:  **
**           ****{all cause mortality, stroke, MI}**
**The clinical decision rule is**
**  ****< win on all cause mortality, stroke, ****or**** MI >**
**Secondary endpoints**
**           ****{CV mortality, CV hospitalization} **

## Clinical Decision Rules

**Example 10: In an Alzheimer trial,**
**  **
**The primary endpoints:**
**          ****{ADAS-Cog.S., CIBIC }***
**The clinical decision rule is**
**   ****< win on ADAS-Cog.S. ****and**** win on CIBIC >**
*** ****ADAS-Cog.S. = Alzheimer Disease Assessment Scale - Cognitive Subscale**
**    ****CIBIC = Clinician’s Interview Based Impression of Change**

## Clinical Decision Rules

**Example 11: In an Epilepsy trial,  **
**The primary endpoints:**
**       ****{% reduction in seizure rate, % reduction in**
**         ****drop attack rate, parental global evaluation**
**         ****of seizure severity}**
**The clinical decision rule is**
**     ****< win on seizure rate ****or**** (win on drop**
**        ****attack rate ****and**** win on seizure severity) >**

## Clinical Decision Rules

**Example 12: In a Rheumatoid Arthritis trial,  **
**the primary endpoint is the binary composite**
**endpoint described in Example 6.  **** **
**The clinical decision rule is,**
** ****<**** win on the binary composite endpoint****) >**

## Clinical Decision Rules

**Example 13: In an Acne trial,  **
**The primary endpoints:  **
**      ****{clinical global, inflammatory lesion**
**       ****counts, non-inflammatory lesion counts, **
**       ****total lesion counts}         **
**The clinical decision rule is**
**    ****< win on clinical global ****and**** (at least 2 out**
**        ****of the 3 lesion counts endpoints)** **> **
**    **

## Single Inference

**In a clinical decision rule,**
** **
**       ****A “****win****” means *****making a positive***
***       ******inference***** favoring the treatment based on**
**       ****the statistical test and the trial data **

## Single Inference

**In Example 8, the clinical decision rule, <win on all cause mortality>, involves only one win.  This win corresponds to rejecting the null and accepting the alternative (positive inference).**
-                **H****0                                                                          ****H****a**
**************   ********1**** ******** 0                                      ****************   ********1**** > 0****                                                                                                                              **
**                                                                                         ********1**
**Probability of Erroneous Inference = P(rejecting H****0 ****| H****0**** is true)**

- 0

## Single Inference

**In Example 10, the clinical decision rule,**
** **
**    ****<win on ADAS-Cog. S. ****and**** win on CIBIC>,  **
**   ****corresponds to making a positive *****joint***** inference in favor of the treatment, albeit it is a complex inference**

## Example 10: Alzheimer

**ADAS-CogS******

**CIBIC ******

- Ho:

- Ha:

**Example 10: Alzheimer**

**ADAS-CogS******

**CIBIC ******

- Winning Scenario

## Single Inference

**In Example 12, the Rheumatoid Arthritis trial, the clinical decision rule,**
** **
**      ****< Win on the binary composite endpoint >,  **
**   ****corresponds to making a positive inference in**
**   ****favor of the treatment based on this binary**
**   ****outcome endpoint.**
**However, this decision rule does not require component specificity**

## Single Inference

**In Example 7, the LIFE Study, the clinical decision rule,**
** **
**  ****<Win on the Composite (CV death, stroke, MI) >,  **
**   ****corresponds to making a positive inference in**
**   ****favor of the treatment based on the composite.**
**However, this positive inference is not component specific.**

## Multiple Inferences

**In a clinical decision rule,**** **
**   ****multiple “wins”**** means *****making multiple positive inferences***** favoring the treatment based on the multiple testing procedure and the trial data****             **

## Multiple Inferences

**In Example 9, the clinical decision rule,**
**     ****<Win on all cause mortality, stroke, ****or**** MI>,**
**    ****corresponds to making one, two or three positive inferences favoring the treatment.**
**This rule wishes to make positive inferences on the **
**    ****individual endpoints**

## Multiple Inferences

**In Example 11, there are two wins:**
**Win on seizure rate, ****or**
**Win on both drop attack rate ****and**** seizure severity**
**   ****Each win corresponds to making a positive**
**   ****inference favoring the treatment**
**The second win defines a positive *****joint***** inference**

## Multiple Inferences

**In Example 12, there are ten possible wins:**
**  ****Wins on (******** 20% improvement in tender**
**      ****joint count ****and**** in swelling joint count) ****and**** **
**      ****(3 out of the 5 remaining primary endpoints)**
**Each win corresponds to making a positive *****joint***** inference favoring the treatment **

## Multiple Inferences

**In Example 13, there are three wins:**
** ****Win on (clinical global and both inflammatory,  **
**     ****and non-inflammatory lesion counts),**
**Win on (clinical global and both inflammatory,**
**     ****and total lesion counts), ****or**
**Win on (clinical global and both non-inflammatory,**
**     ****and total lesion counts)**
**Each win corresponds to a positive *****joint***** inference**

## Multiple Inferences

**The challenge in multiple inferences is to identify multiple testing procedure that is optimal to the problem at hand**
**Make as many positive inferences as possible corresponding to the specified clinical decision rule with proper type I error rate control**
**So what kind of type I error rate control do we need? **

## Multiple Inferences

**The basic problem with multiple inferences can be seen as follows**
** **
**   ****Consider the clinical decision rule :**
** ****<Win on any one of k endpoints:E****1****, E****2****, ..., E****k****>**** **

## Multiple Inferences

**Can we test each endpoint E****i**** at ******** = 0.05 and declare a win on those endpoints E****i**** with a       p****i**** < 0.05?**
**The answer is ****No! **
**Table 1 shows that when the E****i****’s are assumed independent, and the treatment has no effect on the endpoints, the probability of finding at least** **one significant result increases as k ********. **
**The same trend can be observed even when the endpoints are correlated(see Dr.Huque)**

## Table 1:  Probability of Making At Least One Erroneous Inference*

*** ****Under the assumption that the treatment has no effect on any of the endpoints considered**

## Weak Control of Type I Error

**Definition 5: A multiple testing procedure is said to *****control the type I error rate weakly*****, if the probability of making at least one erroneous inference (wrong decision) **_**when the treatment has no effect on any of the given endpoints**_** is maintained at a given significance level ******** **
**The preceding multiple testing procedure does not provide weak control (Table 1). **

## Strong Control of Type I Error

**Definition 6: A multiple testing procedure is said to *****control the type I error rate strongly*****, if the probability of making at least one erroneous inference (wrong decision) (**_**under all possible parameter configurations, i.e., the drug may have no effects on all endpoints or may have effects on some endpoints but not others**_**) is maintained at a given significance level ******

## Weak and Strong Control

- 

- 

- 

- 

- 

- 

- 

- 

**Weak Control     Type I Error Rate**

**Strong Control   Type I Error Rate**** **

**Weak and Strong Control**

## Endpoint Non-Specific Inferences

**A multiple testing procedure that provides ****only**** weak control may not be able to identify the specific endpoint that wins**
**That is, it generally does not permit endpoint specific inferences, i.e., make positive inferences on the individual hypothesis**
**The reason is that it does not control all potential sources that may give rise to an erroneous inference**
**There are procedures where weak and strong   control are the same**

## For Given Multiple Testing Procedure

**Endpoint Non-specific**

**Weak Control of Type I Error**

**Endpoint Specific**

**    ****Stronger      Requirement**

**Strong Control of Type I Error**

## Endpoint Non-Specific Inference

**Example 14: The Simes procedure first orders the p-values p****1****, p****2****, ..., p****k**** obtained from testing the individual endpoints E****1****, E****2****, ..., E****k**** respectively as follows: p****(1)**** ******** p****(2)**** ******** ...******** p****(k)****.**
**If p****(j)**** ********  j ********/k, for any j=1,2,...,k, then one can conclude that the treatment has effect on one of the endpoints, but one cannot specify which one  **
**The Simes procedure is valid under multivariate normality assumption with positive correlation, and controls type I error rate weakly**

## Endpoint Specific Inference

**Example 15: Hommel extended the Simes procedure for making inferences on individual hypotheses (endpoint specific inferences) by applying the closure principle to Simes procedure.**
**It follows from the closure principle, that Hommel’s procedure strongly controls the type I error rate.**
**(Dr.Huque will present an example****)**

## Endpoint Specificity

**From a regulatory perspective, most of the time, endpoint specific inferences are needed, since they provide information on what specific clinical benefits can be expected of the treatment**
**Composite endpoints are not component endpoint specific by definition.  Does this present a problem in some situations?**

## Conclusions and Recommendations

**Specify appropriately the set of primary, secondary, and composite endpoints (if applicable) based on clinical and regulatory considerations**
**Define the clinical decision rule based on the set of primary endpoints and/or composite endpoint**

## Conclusions and Recommendations

**Provide proper statistical inferential structure to support the clinical decision rule**
**The complex null and alternative hypotheses need to **
**    ****be correctly stated reflecting the clinical decision rule**
**Choose the optimal multiple testing procedure with**
**    ****proper allocation of ********and control of type I error rate relative to endpoint specificity needed**
**Sufficient sample size/power**
**Desired level of strength of evidence should be built into the trial design**

## Conclusions and Recommendations

**Specify appropriate multiple adjustment method for the set of secondary endpoints**
**Primary efficacy claims in the label should accurately reflect the trial outcomes based on the clinical decision rule**
**Secondary labeling information based on the set of secondary endpoints should reflect the result after adjustment for multiplicity**