**Public Comment**

**Concept Paper: Risk Assessment of Observational Data: Good
Pharmacovigilance Practices and Pharmacoepidemiological Assessment**

Dockets Management Branch (HFA-305)

Food & Drug Administration

5630 Fishers Lane rm 1061

Rockville, MD 052

e-mail: <fdadockets@oc.fda.gov>

internet:
[www.accessdata.fda.gov/scripts/oc/dockets/edockethome.cfm](http://www.accessdata.fda.gov/scripts/oc/dockets/edockethome.cfm)

# Executive Summary:

Below is the text of the slide that I presented orally at the workshop.
They were also submitted as a PowerPoint presentation. In this summary I
will focus on four main points.

**One:** A Public Health Focused Philosophy needs to be applied to
spontaneous reporting. That is, the presumption that the health
professional (does not apply to consumer or lawyer reports) reporter's
observations are true unless shown otherwise. This position "Encourages"
companies to obtain full data to assess alternate explanations.
Currently, they are inclined to say the cases are "confounded" and
therefore not causal. This is not only fallacious reasoning but one that
may be to the detriment of public health. The burden of evidence for
public health protection needs to be low and conservative.

**Two:** Reporting Rates are still being calculated from spontaneous
data sources by the FDA and industry with some regularity. This is an
unfortunate and unscientific occurrence. The statistical assumptions in
Finney (1971) no longer valid in U.S. scheme mostly because of non-HP
reporters, a preponderance of nonserious, as well as solicited
reporting.

Because the Sampling scheme is unknown cases are not true numerators
therefore no rates can be calculated that can be interpreted with
confidence. Spontaneous reporting schemes are qualitative not
quantitative data sources. Extrapolations (1%, 10% etc.) are voodoo
science. Reporting rates can provide contributory evidence for
regulatory decision-making only when the outcome reaction is exceedingly
rare.

## Three: There is NO Under-Reporting of spontaneous report in the US!

Voluntary spontaneous reporting schemes were designed to signal new,
rare, unusual serious ADRs and **NOT** designed to collect all
reactions. This is a characteristic **NOT** a limitation or flaw! In
U.S. there is extensive over-reporting since common, known and
nonserious reactions make up the vast majority of the reports. The only
true underreporting relate to missing data in medically significant
cases, a quality issue.

**Four:** Secondary (Medical record/Billing) Data sources do not collect
ADR data. Billing codes based on ICD9-CM **NOT** ADRs, but subsequent
diagnose. This results in gross Misclassification that blunts risk
estimates toward unity. Therefore, cannot be used as quantitative data
sources!

They have a limited value in true risk management!

**Text of the slides:**

**1\] Definitions:**

•**Postmarketing Surveillance (PMS)**

--Monitoring of Safety, Quality, Advertising After Market Approval

•**Pharmacovigilance (PV)**

--Review and Assessment of Case (Spontaneous) Reports

•**Pharmacoepidemiology (PE)**

--Use of Non-experimental Methods to Assess the Therapeutic Value of
Drugs in Clinical Practice Environments

•**Risk Assessment (RA)**

--PV + PE + All Needed Knowledge to Make a Global Evaluation of a
Specific Risk

•**Risk Management (RM)**

--Strategy that Uses All of the above, and a

--Toolkit of Regulatory Interventions, and

--Variety of Risk Communication Devices

**2\] FDA Question #1\
**How can the quality of spontaneously reported case reports be improved

•Public Health Focused Philosophy: True unless shown otherwise

--"Encourages" companies to obtain full data to assess alternate
explanations

--"Confounding" does not mean lack of causal relationship!

•Focus on "important'

--Active query at point of initial contact

•FDA compliance on quality standards \> timeliness

•Implement new proposed regulations ASAP after comment period closes

**3\] FDA Question #2\
**What are possible advantages or disadvantages of applying datamining
techniques to spontaneous reporting databases for the purpose of
identifying safety signals?

•Datamining is of value for

--secondary "alerting"

--Identifying previously unrecognized data relationships

•Most valid: Internal (denominator) based proportional analyses and
comparisons within pharmacological group

•"Alerts" then require clinical judgment to declare a "signal" requiring
subsequent epidemiological analyses

**4\] FDA Question #3\
**What are possible advantages or disadvantages of performing causality
assessments at the individual case level?

•Useful: Sorts high volume by quality of evidence

•Without some "probable" cases there is little evidence from the
remaining majority of "possible" spontaneous reports

•All included cases =\> Case series assessment

•"Definite" case = Myth!

**5\] FDA Question #4**\
Under what circumstances would a registry be useful as a surveillance
tool and when would it cease to be useful?

•Require valid comparator group(s)

•SADR data needs to be collected using the same methods and instruments
as used in pre-approval RCTs (i.e., "Systematic collection")

•Prospective cohort studies are **NOT** powered to find new ADRs

•Best in assessing risk factors for [known]{.underline} reactions for
subsequent prevention, i.e. WHY (root cause) they occur.

**6\] FDA Question #5**\
Under what circumstances would active surveillance strategies prove
useful to identify as yet unreported adverse events?

•Active case-control surveillance methodology (a la Shapiro) should be
re-explored

•MedSun strategy extended to drugs

•Use of Poison Control/Drug Information Centers as dedicated data
acquisition sentinels

--Drug-Drug interactions

--Medication errors

**7\] FDA Question #6**\
Under what circumstances would additional pharmacoepidemiologic studies
be useful?

•When they can be;

--Replicated

--Validated

--Provide Quantitative data

•Measure risk

•Measure change in risk

# The Power of Myth

## 8\] Confounding Myth

•Spontaneous case reports seldom contain all history and data on
concomitant conditions or drugs

•Absence of these data may be due to:

--True absence or Poor data acquisition

•Presence of these confounders does not "clear' the drug

•Dismissing 'confounded cases' *en masse* in complex medical conditions
is an error because "unconfounded' cases may be due to poor data
acquisition

•Apply Public Health Focused Philosophy: True unless shown otherwise

•Popperian Logic: relative likelihood of alternate explanations

## 9\] Reporting Rate Myth

•The statistical assumptions in Finney (1971) no longer valid in U.S.
scheme

--Non-HP reporters, nonserious, solicited, etc.

•Sampling scheme is unknown

--Qualitative not quantitative data

--Cases are therefore not true numerators

•Extrapolations (1%,10% etc.) are voodoo science

•Reporting rates can provide contributory evidence for regulatory
decision-making only when the outcome reaction is exceedingly rare.

•Otherwise, reporting rates can only serve as a confirmation bridge
between the cases and the observational research

•More valid: Internal (denominator) based proportional analyses and
comparisons within pharmacological group

## 10\] The Under-Reporting Myth

•Voluntary spontaneous reporting schemes were designed to signal new,
rare, unusual serious ADRs

•**NOT** designed to collect all reactions

•This is a characteristic **NOT** a limitation or flaw!

•In U.S.: Over-reporting!

•True underreporting: missing data in medically significant cases

**11\] Secondary Data Myth**

•Secondary (Medical record/Billing) Data sources

--ICD9-CM **NOT** ADRs

--Misclassification blunts risk estimates

•Therefore it cannot be used as quantitative data sources!

•Limited value in true risk management!