Docket FDA-2008-N-0234

Developing Guidance on Conducting Scientifically Sound
Pharmacoepidemiologic Safety Studies Using Large Electronic Healthcare
Datasets.

Agency: FDA

Action: request for comments

The GPRD group, based within the UK Medicine and Devices Regulator
(MHRA) submits this response on the basis that:

1)  The GPRD database has been and is used for many major drug safety
    issues, by regulators, academics and pharma companies.

2)  It was specifically set up to enable research and takes data from a
    universal healthcare, cradle to grave system where primary care is
    the gatekeeper to practically all care.

3)  The FDA have a contract with the GPRD and use the data to conduct
    drug safety research.

4)  Staff of the GPRD have extensive and wide experience of the issues
    being addressed both from working within the GPRD group and from
    prior appointments and existing external, mostly academic links.

Drs Tjeerd van Staa, Head of Research and Dr John Parkinson, Director of
GPRD are the authors of this document and are happy to discuss any
aspects raised. (john.parkinson@gprd.com)

 

1.  What information and what level of detail are needed for FDA to
    ensure the appropriateness of the data source to address the product
    safety questions being asked? How does this differ by type of data
    source (electronic medical records (EMR) vs. claims)?

a\) Quality control criteria for the database and conformance to such.
This should include details of:

\*checks of internal and external validity

\*published studies showing peer review of data quality issues

\*published coding validation studies

There should be NO use of statements such as \"we are like other
database XXX\". This is not proof. Each database, even database from the
same IT system should have to prove their own quality. There are many
reasons why it might differ- significantly.

Provide details of \"missing data\" defined as observations that could
have been recorded but were not. The GPRD recording guidelines and
payment for quality data relate directly to this issue. GPRD has missing
data but it is in the range 0.5-10% and getting lower all the time; peer
reviewed publications detailing the validated percentages.

Full disclosure of conflicts of interest related to any publications
used in the above should be given by the database owner where they were
not fully declared in a publication. e.g- Research group X acts as an
agent for this database and they pay no costs or charges for use of the
data\"- ie they have an interest in showing it is \"good\".

Details should also be provided of whether the dataset is time frozen or
is the subject of updates that might affect earlier data- affect in the
sense of improve the quality/validity of prior data- eg the removal of a
disease code that subsequently turns out to be incorrect.

b\) The exact details of how total "n" for the database is constructed
(does it include temporary patients and those not up to standard for
instance). A full \"n\" plan should be provided showing how the n is
derived. This should essentially detail the stable study population with
long-term follow-up

c\) Declaration of how coding is done- by whom and that \"computer\"
income maximisation or any other non clinically related changes do not
take place.

d\) Show what the effect of drug exposure not recorded in the database
might be e.g.web or other such drug purchases? This may be related to
degree of payment for drugs by persons and or OTC use.

e\) Not to use information about a database that has not been verified
by peer review or by third party means.

2.  What are the challenges of using enrolment data for defining study
    populations in claims databases? Describe effective strategies for
    addressing the absence of formal enrolment data in some EMR systems

This question is only partially relevant to GPRD data as this data
contains dates of birth, death, enrolment to a GP practice and date of
leaving the practice.

3.   Under what circumstances should FDA consider studies using non-U.S.
    electronic data sources in its assessment of product safety
    questions?   

Pharmacoepidemiology ideally requires access to data that:

 

a\) is high quality

b\) is longitudinal (left and right censor dates)

c\) is life long, if possoble

d\) is across the widest spectrum of care

e\) can be validated

f\) can provide access to the full record including text as well as
coding

g\) that has peer reviewed publications in support of the above

If this cannot be provided within the USA for specific safety issues
then non-USA sources must be sort- The FDA currently uses GPRD data.

4.  How can FDA assure that the study design accurately captures the
    clinical events, exposures of interest, and confounding factors
    needed to answer the product safety question under investigation?

By the use of external scientific review that provides input to the
study at many level- coding, data recording, the practice in the
clinical area, the healthcare delivery system. GPRD operates this via an
Independent Scientific review board- ISAC.

5.  What are effective strategies to address confounding by indication
    and the effect of measured and unmeasured confounders?

High-quality data are the most important aspect in minimising bias.
  Some statistical techniques can be used (such as propensity scores),
but these techniques are not perfect and can if improperly used lead to
greater bias- what will be will be. Acceptance that channelling bias
with a newly introduced medication is difficult to manage without
resorting to "randomisation" in everyday clinical care. Such
randomisation is now possible within the GPRD system.

 

6.  What are other challenges to internal and external validity in
    studies using EMR and claims databases? What are the best practices
    for addressing them?

 

 A key challenge is that different studies within a single database may
produce variable results. The best practice of dealing with this is to
allow broad scientific access to datasets (i.e., not limited to single
investigator groups), so we can learn about the methodology of analysing
complex datasets and start to understand why such might happen. More
work needs to be done in the area of discrepant results between case
control studies where the cases are heavily defined/limited and cohort
studies. There are also issues related to results from RCTs (low
external validity) and observational studies on real world data. The Cox
2 situation is a good example where the clinical trial day in day out
exposure over a long period of time is rarely found in the real world.
One way forward is to model CT data with real world data and to start
thinking of clinical trial and observational data as different and
exploit the differences rather than to think gold standard or not.

Randomisation into everyday clinical care, cost-effectively facilitated
by using the clinical database for tracking the data is also an option
now available within GPRD. Recruitment to such studies is aided by the
IT system, the generalisability of the result confirmed by comparing
those in the trial on relevant drugs with those not in the trial.

\*\*End\*\*