# Presentation: 161249

## Rapid Cycle Analysis for EarlyDetection of Vaccine Adverse Events

**Richard Platt, MD, MSc****for the ****CDC Vaccine Safety Datalink Investigators**
- Harvard Pilgrim Health Care and Harvard Medical School

## Why Do We Need Early Detection Systems?

- Rare adverse events may be impossible to detect in pre-licensure studies
- Spontaneous reports to passive surveillance systems, e.g. VAERS, often need rapid follow-up
- Designing follow-up studies can take months to years using traditional approaches

## Rapid Cycle Analysis

- A modern approach to surveillance that takes advantage of VSD’s strengths
- Update data on all vaccines and all subsequent outcomes every week
- Choose vaccines and potential adverse events to monitor
- Conduct weekly analysis

## Sequential Analysis Methods

- Early data contribute to every subsequent analysis
- Repeated statistical testing of the same data requires special methods
- New method:  Maximized SPRT (Kulldorff et al., 2004)
- A refinement of sequential probability ratio testing (Wald, 1955)

## Example:  Rotavirus vaccine and intussusception

**1999**

**Vaccine licensed Aug 98**
**15 VAERs reports through Jul 99**

**Vaccine**
**suspended**

**Withdrawn**

## Example:  Rotavirus vaccine and intussusception

**Log **
**likelihood **
**ratio**

**Critical value **
**of LLR = 3.3**

**Example:  Rotavirus vaccine and intussusception**

**Vaccine**
**suspended**

**Vaccine**
**withdrawn**

## Example:  Rotavirus vaccine and intussusception

**Example:  Rotavirus vaccine and intussusception**

**Vaccine licensed Aug 98**
**15 VAERs reports through Jul 99**

**Vaccine**
**suspended**

**Vaccine**
**withdrawn**

## Rapid Cycle Analysis – Ongoing Surveillance via VSD

- Meningococcal conjugate vaccine and Guillain-Barre syndrome
- Rotavirus vaccine and intussusception, gastrointestinal bleeding, and other events
- HPV, Tdap, MMRV, influenza vaccines – being implemented

## Implementing Rapid Cycle Analysis

- For each vaccine, choose the outcomes of interest
- Choose the comparison method –concurrent controls, historical rates, or both
- Create programs to generate aggregate data from the 8 VSD sites
- Program analysis and run weekly

## Choosing Outcomes

- Select outcomes that are:
- Clearly defined
- e.g., Guillain-Barre syndrome or seizures rather than “neurologic problems”
- Acute-onset
- Relatively uncommon
- Plausible

## Concurrent Comparison Analysis

- Uses matched controls, e.g., patients making preventive visits
- Advantage:  Avoids false signaling or missed signals due to secular trends
- Limitations:
- Need to define appropriate control groups – not simple!
- Vaccines may be adopted rapidly, leaving few controls

## Results of Concurrent AnalysisOutcomes within 42 days after index event, VSD population, May 2005-March 2007

| Cumulative Ns | Menactra vaccination | Concurrent controls |
| --- | --- | --- |
| Index events* | 107,321 | 107,240 |
| GBS | 0 | 1 |
| Facial paralysis | 4 | 6 |
| Thrombocytopenia | 2 | 1 |
| Seizures | 10 | 18 |

**The index event in concurrent controls was a preventive visit (N=262,102); the analysis selects 1 control per vaccination visit.**

## Historical Comparison Analysis

- Uses incidence rates from existing data
- Advantage:  Knowing the historical rate of rare events allows earlier recognition that a small number of cases among vaccine recipients is unusual
- Example:  4 cases of Guillain-Barre syndrome occur in vaccinees, 0 in controls
- Limitation:  Secular trends

## Results of Historical AnalysisThrombocytopenia within 42 days after meningococcal vaccination, May 2005-March 2007

|  | Cumulative # of cases in 42 day window: |  |  |  |
| --- | --- | --- | --- | --- |
| Week | Menactra group | Expected based on historical rate | RR | LLR* |
| 12 | 1 | 0.083 | 12.1 | 1.58 |
| 25 | 1 | 0.168 | 5.9 | 0.95 |
| 51 | 1 | 0.249 | 4.0 | 0.64 |
| 95 | 2 | 0.650 | 3.0 | 0.90 |
|  |  |  |  |  |

**Results of Historical Analysis****Thrombocytopenia within 42 days after meningococcal vaccination, May 2005-March 2007**

## What if:  Historical maxSPRT analysis of low platelet countCritical value B = 2.87 for =.05 with upper bound=1

***  LLR for one-sided test**
****  P<.05****	****	*******P<.001**
**   **

|  | Cumulative # of cases in 42 day window: |  |  |  |
| --- | --- | --- | --- | --- |
| Week | Menactra group | Expected based on historical rate | RR | LLR* |
| 12 | 1 | 0.083 | 12.1 | 1.58 |
|  |  |  |  |  |
| What if: |  |  |  |  |
| 12 | 2 | 0.083 |  | 4.45** |
| 12 | 3 | 0.083 |  | 7.86*** |

## Limitationsof Rapid Cycle Analysis using maxSPRT

- Signals are not definitive, and follow-up may require
- Alternative analyses
- Evaluation of temporal clustering
- Chart review
- Unanticipated adverse events are better evaluated using data mining methods, e.g., temporal scan statistic

## Next Steps

- VSD plans to implement surveillance rapidly whenever a new vaccine is introduced
- Refine methods of analysis for new situations, e.g. zoster
- Add new populations

## Coming soon: CERTs Health Plan Consortium for Public Health

- Goal: Improve the safety and safe use of marketed vaccines and prescription drugs by studying their use in health plan members
- Target population: 100 million
- A planned activity of the Centers for Education and Research on Therapeutics (CERTs)
- Created under Congressional mandate to be a trusted national resource in therapeutics
- Administered by AHRQ in consultation with FDA
- Accepted processes for administering public-private partnerships

## CERTs Health Plan Consortium for Public Health – Aims

- Timely risk identification and quantification
- Prospective evaluation of new therapeutics captured by health plan data
-  focus on pre-defined list of potential problems
- Detailed followup of selected problems
- Identification of potentially unsafe use of preventive and therapeutic agents
- Other topics, subject to Board approval

## CERTs Health Plan Consortium for Public Health

- Structure: Public-private partnership
- health plans, federal agencies, industry, professional societies, public, foundations, academic community
- Data: health plans’ automated data (claims+) with access to full text medical records
- Transparency:
- Proposed protocols available in advance for public comment
- Final protocols publicly available
- Final results publicly available
- Confidentiality and privacy:
- Individuals: HIPAA/IRB compliant
- Health plans: identity and proprietary data protected

## CERTs Health Plan Consortium for Public Health – Funding

- Infrastructure requires core funding
- Individual projects will require separate funding

## CERTs Health Plan Consortium for Public Health

- Existing health plan data allow substantial enhancement of timeliness, power, and efficiency of post-marketing studies of therapeutics
- This information can/should complement other sources – Medicare/Medicaid, VA, Vaccine Safety Datalink

## Collaborators – partial list

- James Baggs, CDC
- Jeff Brown, Harvard
- Arnold Chan, CERT
- Bob Davis, CDC
- Inna Dashevsky, CERT
- Rich Fox, Harvard
- David Graham, FDA
- Margarette Kolczak, CDC
- Martin Kulldorff, Harvard

- Ned Lewis, Kaiser
- Kim Lane, CERT
- Renny Li, Harvard
- Tracy Lieu, Harvard
- Parker Pettus, CERT
- Irene Shui, Harvard
- Eric Weintraub, CDC
- Katherine Yih, Harvard
- Ruihua Yin, Harvard
- Health plan-based VSD and CERT teams