# Presentation: 124578

## Meta-Analysis: Does it have a role in Drug Development?

- Charles Anello,Sc.D.
- Deputy Director
- Office of Biostatistics,
- CDER FDA
- anello@cder.fda.gov
- American Association for the Advancement of Science, 14-19 February  2002, Boston Mass.

## Disclaimer

- The views expressed in this talk are those of the  author and do not necessarily represent those of the Food and Drug Administration

## Topics

- Regulatory environment
- Examples of Meta-analysis seen in the Regulatory context
- Efficacy versus Safety
- Potential roles
- Conclusions

## FDA\CDER’s Mission

- To promote and protect public health by assuring that safe and effective drugs are available
- Regulatory decisions arise from reviewing the evidence supporting claims of safety and effectiveness obtained during the  drug development process

## Drug Development Process

- Pre Approval( pre-clinical (toxicology, carcinogenicity,reproductive) and Clinical(pk\pd, phases II and III)
- Post Approval( post-marketing clinical and non-clinical studies and post-marketing surveillance

## Regulatory Environment

- The 1962  Kefauver-Harris Amendments:
- “ Substantial evidence” means evidence consisting of adequate and well controlled investigations, including clinical investigations, by experts qualified by scientific training and experience to evaluate the effectiveness of the drug involved, on the basis of which it could be fairly and responsibly be concluded by such experts that the drug will have the effect it purports or is represented to have under the conditions of use prescribed, recommended or suggested in the labeling...”

## An Adequate and Well Controlled Study

- A clear statement of objectives
- Design that permits a valid comparison with a control and provides a quantitative assessment of the drug effect
- Method of selection of subjects to assure they have the disease
- Minimizes assignment bias

## An Adequate and Well Controlled Study

- Minimized measurement, ascertainment and analysis bias
- Well defined and reliable methods of assessment of response(outcome measures)

## Regulatory Environment

- For three decades this has been interpreted to mean at least two(2) adequate and well controlled studies related to the same claim.

## Regulatory environment

- The 1997 Food and Drug Modernization Act(FDAMA) :
- Amends section 505(d) by adding “If the Secretary determines, based on relevant science, that the data from one adequate and well controlled clinical investigation and confirmatory evidence(obtained prior to or after such investigation) are sufficient to establish effectiveness the Secretary may consider such data and evidence to constitute substantial evidence for the purpose of the preceding sentence”

##  Regulatory environment

- Although at least two(2) adequate and well controlled studies is still the usual standard, in some situations one adequate and well controlled study and “confirmatory evidence” may be sufficient

##  Regulatory environment

- During a regulatory review one usually has access to all the relevant studies whether they have been published or not.
- During a regulatory review one usually has access to the protocols and the individual patient records.
- This is very different from the situation surrounding a literature based meta-analysis

## Efficacy versus Safety

- Standards are not the same
- Approaches to data collection and analysis are different
- Extent of the evidence is not equivalent

## Efficacy

- ..there is a lack of substantial evidence consisting of adequate and well controlled investigations, as defined in section 314.126 that a drug product will have the effect it purports.. Under the conditions of use prescribed.. In the proposed labeling(21 CFR3145.125(B)(5)

## Efficacy

- Randomized studies
- Endpoints pre-specified
- Study power to detect a specified effect
- Individual studies planned to stand on their own
- Generally two or more studies
- Consistency of effects

## Safety

- ..there is sufficient information about the drug to determine whether the product is safe for use under the conditions recommended or suggested in its proposed labeling(21CFR314.125(B)(4).

## Safety

- Randomized and Observational Studies
- Usually do not know endpoints in advance(use broad screening and  careful observation) multiplicity of outcomes
- Effect sizes unknown but of low incidience
- Focus on association and estimation
- Exploratory
- May require combining data across studies

## Differences in Approaches to Assessment of Safety and Efficacy

- Critical efficacy endpoints identified during design of protocol
- Generally safety endpoints not know prior to conduct of trial
- Adverse events can occur at low frequency
- Sophisticated statistical analysis often restricted to evaluation of efficacy
- There is a tendency to more crudely combine safety data from multiple trials

## Possible Roles of Meta-analysis in Drug Development

- Estimating a common effect more precisely
- Subgroup Analysis
- Hypothesis Generating
- Establishing the margin in non-inferiority trials
- Evaluating heterogeneity

## Estimating a common effect more precisely

## Anihistamine

- Estimated the effectiveness Antihistamines to reduce runny nose and sneezing
- Used all RCTs
- Recorded outcome and entry criteria
- Obtained overall estimate of effect
- (Weintraub, D’Agostino,et.al.1995)

## Triazolam

- A meta-analysis of 25 RCTs of triazolam,fluazepam and placebo
- Looked for differences in dropout rates associated with various sysmptoms

## Hypothesis Generating

## Examples of Meta-analysis seen in the Regulatory context(H. Pylori and Ulcer Recurrence)

- Reviewed 12 literature studies and found that H.  Plyori eradication should be considered an acceptable primary efficacy or surrogate for studying ulcer treatments provided at least two endoscopic tests are used to define H.Plyori eradication.
- Hopkins, Girardi and Turney 1996

## Subgroup Analysis

## Survival Effects of ZDV, ddI and ddC

- Seven clinical trials for treatment of HIV-infected individuals
- Trials partitioned in two subgroups based on baseline CD4cell counts CD4> 50 cells/mm or ,50 cell mm
- Survival measured effect
- Led to conclusion that “ exclusion of patients with CD4<50 cells/mm may result in prolong and or longer clinical trials
- (Kazempour ,Kammerman and Farr,1995)

## Using Epidemiological Data

- Comparing the risk of angioedema during exposure to angiotensin-converting enzyme inhibitors with that of exposure to calcium channel blockers in different races
- Data from different Medicaid populations( Michigan, Ohio and /Tennessee ) who had angtiodema listed on their claim forms(
- (Burkhart et.al., 94)

## Examples of Meta-analysis seen in the Regulatory context( Cumadin and CVD)

- A Meta-analysis for Coumadin(warfarin sodium) related to prevention of death, recurrent  MI,etc.
- Meta- analysis was considered supportive evidence but concerns were expressed about changes in clinical practice over the studies

## Establishing the Margin in Non-inferiority Trials

## Establishing the Margin in Non-inferiority Trials

- A non-inferiority trial is conducted without a placebo.
- There is a need to establish show that the active control can consistently shows superiority over a placebo
- The amount that the active control beats the placebo is called the margin.

## Establishing the delta in non-inferiority trials

- A meta-analysis was used to establish the margin for the  Thrombolytic tPA.
- There were placebo controlled trials of Streptokinase but not of tPA.
- There were trials of tPA versus Streptokinase
- (Lachenbruch, Walton,Gupta, CBER, 1997)

## Evaluating heterogeneity

## Advisory Committee Presentation on Vioxx (Rofecoxib)

- Discussion on the meta analyses for cardiovascular risk assessment
- Qain Li,Sc.D.

**Notes:**
Good morning! My name is Qian Li, statistical reviewer from Office of Biostatistics, CDER/FDA. I am going to discuss the meta analysis for cardiovascular risk assessment. 

## Slide 34

## Slide 35

## Slide 36

## Slide 37

## Confirmed Serious CV Thrombotic Events Observed in VIGOR

- 50 mg

- p-value=0.002

**Notes:**
To begin with, let’s look at the cumulative incidence curves of thrombotic cardiovascular serious events for rofecoxib 50 mg and naproxen observed in the VIGOR trial which has been mentioned by Dr. Targum. 
The difference of the CV events between the two treatment groups was statistically significant. On average, rofecoxib 50 mg doubled the risk of a cv event in naproxen treatment group. Notice that the two curves start to diverge 6 weeks after the trial and further separate at later stage of the trial at around 8 month. This suggested non-constant risk ratio over time.

## Key Features of the Meta Analysis

- Meta analysis of 25 studies and more than 28,000 patients
- different dose levels of rofecoxib were pooled (12.5, 25, and 50 mg)
- studies of different duration were pooled (4 weeks to one year)
- different indications were combined (RA, OA, Alzheimer and back pain)

**Notes:**
To further understand the risk of CV events associated with rofecoxib 50 mg observed in the VIGOR trial, a meta analysis of 25 studies involved more than 28,000 patients was conducted.  The VIGOR trial was part of the meta analysis. 
The key features of the meta analysis are that different dose levels of rofecoxib were pooled together, the dose were from 12.5 mg to 50 mg; studies of different duration were pooled together, the studies from 4 weeks to more than one year; different indications were combined together by stratified analyses, these indications including RA, OA, Alzheimer and back pain.

## Issue of the Meta Analysis

- Does the meta analysis adequately address the role of rofecoxib 50 mg in relation to CV events?

**Notes:**
The issue we have about the meta analysis focus on rofecoxib 50 mg since it was 50 mg of rofecoxib that showed statistically significantly increased risk of CV events compared to naproxen in the VIGOR trial. The question we have is whether the meta analysis can  adequately address the role of rofecoxib 50 mg in relation to CV events.

## Rofecoxib 50 mg Patients in the Meta Analysis Data Sets

- Of the 28,349 patients
-     			_patients in rofecoxib 50 mg_
- VIGOR  			   4047
- Non-VIGOR  		   1881
-     (>6 month)		  (~900)

**Notes:**
Now let us first look at the numbers of patients that were assigned to rofecoxib 50 mg in the meta analysis data set. Of the 28,000 patients in the meta analysis data set, a total of  less than 6000 patients had exposed to rofecoxib 50 mg. In these 6000 rofecoxib 50 mg patients,  4047 patients were patients from the VIGOR trials which was longer than 6 month. Outside the VIGOR trial, only 1881 patients had rofecoxib 50 mg and of these patients only 927 were involved studies that were longer than 6 month. Obviously, there is not enough patients for rofecoxib 50 mg outside the VIGOR trial in this meta analysis data set.

## Concerns of the Meta Analysis

- 1. The risk ratios between rofecoxib & comparator may not be constant over time,
- short term studies may not be able to demonstrate the treatment difference
- need long term exposure data

**Notes:**
 In addition, we have some concerns about the meta analysis. One is that the risk ratios between rofecoxib and comparator may not be constant over time. As observed in the VIGOR study, the treatment difference started to occur after 6 weeks of the trial. So short term studies may not be able to demonstrate any treatment difference. We need studies that had longer duration with sufficient number of patients. 

## Concerns of the Meta Analysis

- 2. The risks may not be the same for different dose levels of rofecoxib, pooling may obscure the risk associated with the high dose group.

**Notes:**
Another concern is that the risk may not be the same for different dose levels of rofecoxib. It is common sense that pooling may obscure the risk associated with the high dose group since we will observe average risk of the different doses of rofecoxib. The interpretation of the average risk becomes complicated if different dose levels have different sample size and different dose level have different duration. The risk of rofecoxib 50 mg will be buried under these complications. This is not a conceptual concern.

## For Studies with Duration Longer Than 6 Months

- - This assessment suggested a trend to increase risk with rofecoxib 50 mg.
- - Data provided by the sponsor upon request by the agency.

**Notes:**
As a matter fact,  there are data to suggest a trend towards increased risk with rofecoxib 50 mg. For example, based on data provided by the sponsor upon request by the agency, for studies with duration longer than 6 months, the relative risk of rofecoxib 50 mg appears to be higher in both comparisons with naproxen and other NSAIDs including ibuprofen and diclofenac as they were highlighted.

## Heterogeneity Issues

- Pooling different dose levels is problematic for an evaluation of rofecoxib 50 mg
- Inadequate number of patients outside VIGOR for rofecoxib 50 mg, especially for duration over 6 month

**Notes:**
To summarize the major limitations of the meta analysis: Pooling different dose levels of rofecoxib is problematic for an evaluation of rofecoxib 50 mg. It make the meta analysis invalid for assessing the risk of CV events for rofecoxib 50 mg. 
Furthermore, there is not enough patients for rofecoxib 50 mg outside the VIGOR trial, especially for duration over 6 month. There were only about 900 patient assigned for rofecoxib 50 mg in studies over 6 month.

## Meta-Analysis: Does it have a role in Drug Development?

## Possible Roles of Meta-analysis in Drug Development

- Estimating a common effect more precisely
- Subgroup Analysis
- Hypothesis Generating
- Non-inferiority Trials
- Evaluating heterogeneity

## Are there valid Uses of Meta-analysis in the Drug Development Process?

- Could a Meta-analysis provide a evidence equivalent to an “adequate and well controlled study” ?
- Could a Meta-analysis provide confirmatory evidence?

## Could a Meta-analysis provide confirmatory evidence?

- If pre-planned
- If meets standard of Adequate and Well Controlled
- If individual patient records are available for re-analysis

## Could a Meta-analysis provide  evidence equivalent to an “adequate and well controlled study” ?

- Yes, provided it meets the definition:
- A clear statement of objectives
- Design that permits a valid comparison with a control and provides a quantitative assessment of the drug effect
- Method of selection of subjects to assure they have the disease
- Minimizes assignment bias
- Minimized measurement, ascertainment and analysis bias
- Well defined and reliable methods of assessment of response(outcome measures)

## Conclusion

- Meta-Analysis: Does it have a role in Drug Development? Yes:
- The useful roles include: more precise estimates of a common effect, hypothesis generating,subgroup analysis,finding a margin, investigating heterogeneity.
- But a Meta-analysis will provide substantial evidence only rarely.