# INTEGRATED SURVEILLANCE SEMINAR SERIES

CENTERS FOR DISEASE CONTROL

COORDINATING CENTER FOR HEALTH INFORMATION AND SERVICE

NATIONAL CENTER FOR PUBLIC HEALTH INFORMATICS

DIVISION OF INTEGRATED SURVEILLANCE SYSTEMS AND SERVICES

TITLE: Automated detection and reporting of notifiable diseases in near
real-time using electronic medical records

POWERPOINT PRESENTATION: Electronic medical record support for public
health

SPEAKER: Micahel Klompas

MODERATOR: Dan Sosin

December 12, 2007

JENNIFER WARD, EPIDEMIOLOGIST, DIVISION OF INTEGRATED SURVEILLANCE
SYSTEMS AND SERVICES, CDC: I think we will go ahead and get started.
It's 11 o'clock. I'm delighted to begin this session of the Integrated
Surveillance Seminar Series. I want to welcome all of those who are here
in person, as well as those who are joining us by webinar and Envision.
The seminar series is hosted by the National Center for Public Health
Informatics, Division of Integrated Surveillance Systems and Services.
It's part of a larger Integrated Surveillance Community of Practice,
which is intended to facilitate interactions between stakeholders in the
field of public health surveillance. The seminar series strives to
establish an interactive forum for discussing the vision of integrated
surveillance, identifying issues and approaches to its achievement, and
providing a mechanism for discussing best practices in the area of
integrated surveillance. Following the presentation by our speaker, we
will have a question and answer and discussion period. For the benefits
of our participants via webinar and Envision, we are going to try and
get a microphone here in the back, and we would ask that you use that
for asking your questions so that those on the phone can hear.

We are pleased to have Dr. Dan Sosin moderating our discussion today.
Dr. Sosin is the Associate Director of Science for the Coordinating
Office for Terrorism Preparedness and Emergency Response at CDC. Please
join me in welcoming Dr. Sosin

DAN SOSIN, ASSOCIATE DIRECTOR OF SCIENCE, COORDINATING OFFICE FOR
TERRORISM PERPAREDNESS AND EMERGENCY RESPONSE, CDC: Thanks, Jennifer. I
very much appreciate the invitation to moderate this seminar. I have a
ratty sticker - sticky note on my desk which says, "Things to do for
myself" -- which is, to attend more seminars and conferences." It's
ratty because it's been there a long time. I don't get to go very often,
so having an iron-clad excuse like this is saving me from attending
other meetings and being able to be here today and I appreciate that.

Today, we will be hearing about automated detection and reporting of
notifiable diseases in near-real time using electronic medical records
presented by Dr. Michael Klompas. Dr. Klompas is a researcher at the
Harvard School of Medicine where he practices infectious diseases and
epidemiology. Harvard is one of those rare institutions where a close
linkage has been developed with local public health, and where there is
an appreciation of the importance of close collaboration between
clinical medicine and public health practice. And again, you need to
have an epidemiologist and a clinical practitioner in this environment.

Dr. Klompas has been focusing his research interests on leveraging
electronic data streams from the clinical sector to improve the
timeliness and quality of public health surveillance. Dr. Klompas did
his undergraduate training at the University of North Carolina in Chapel
Hill, has a Masters of Public Health from Harvard, and received his
medical degree from the University of Toronto. He currently serves as
the project director and co-investigator for the electronic support of
public health or ESP project at Harvard School of Medicine that we will
hear about today.

Recent calls from the White House with respect to the Homeland Security
Presidential Director, Tony Wan, and from Congress last year with the
Pandemic and All Hazards and Preparedness Act, have reinforced what has
been our hope and our burden over the last 10 plus years, that
information technology advances in the healthcare sector, if handled
properly, and that's a big challenge, could vastly improve the
timeliness and quality of public health surveillance and serve as a
vanguard for health security in our country. The expectation from our
national leaders, at least our national leaders and certainly others, is
that the government at the federal, state, local levels should provide
real-time, or near-real-time visibility of the national health threats
by thoughtfully leveraging the best systems and methods available today.

The objectives for Dr. Klompas' talk today are to describe a portable
architecture for extracting, analyzing, and communicating information
from electronic medical record systems, to illustrate algorithms for
analyzing electronic medical record data, to identify notifiable disease
conditions, and to delineate possible strategies to facilitate
widespread implementation of automated disease detection and reporting
from electronic health data. This group forum today gives us a chance to
dig deeper into the model that Harvard provides and discuss challenges
of our national surveillance and our national at CDC national
responsibilities - to ask questions such as how affordable is this
system, and what do we need to know to assure that it is a reproducible
model, to ask questions about the role CDC might play in creating the
environment or the substrate for replication of this model, and to have
a better understanding that if we were to build this system of these
systems, what would we be missing in our national surveillance needs.
So, the challenge to the audience today is to go beyond your personal
interests as you listen and participate today, to direct the discussion
in ways that we can leverage this experience and knowledge towards what
is called for in the Pandemic and All Hazards Preparedness Act; in
real-time, electronic, nationwide public health situational awareness
capability; through an interoperable network of systems to share data
and information to enhance early detection of, rapid response to, and
management of potentially catastrophic infectious diseases and other
public health emergencies that originate domestically or abroad. So,
with that, challenge the audience to review and hear this presentation
and engage in conversation. Please welcome Dr. Michael Klompas.

DR. MICAHEL KLOMPAS, EPIDEMIOLOGIST AND INFECTIOUS DISEASE PHYSICIAN,
HARVARD SCHOOL OF MEDICINE: Good morning ladies and gentleman. Thank you
very much for the introduction. And, thank you very much to Dr. McNabb,
Dr. Lenert, and NCPHI for giving me this opportunity to come and address
you. I very much appreciate it. Our work is funded by CDC and we are
very much interested in aligning with CDC's objective and mission.

As alluded to, \[section omitted\] I'd like to talk to you about our
project on Electronic Medical Record Support for Public Health, which is
an effort to find notifiable diseases from electronic medical record
data. This work is funded under the rubric of the CDC Center of
Excellence in public health informatics based out of Boston, which is
from a NCPHI grant. And, our Center of Excellence is a collaboration of
the following entities: it's Harvard Medical School, Children's
Hospital, the Massachusetts Department of Public Health, the Harvard
Vanguard Medical Associates, is a large medical group in Boston, and the
Brigham Women's Channing Laboratory.

So, here's the quotation that was printed in *Public Health Reports*,
generated by CDC's precursor from 1913 -- 1951: "No health department,
state or local, can effectively prevent or control disease without
knowledge of when, where, and under what conditions cases are
occurring." And, I don't think there's much debate that it is as true
today as it was about a century ago. Unfortunately, however, the manner
in which this information is acquired today still is quite similar to
the way it was acquired a hundred years ago, which is, by and large,
public health awareness of notifiable disease based upon paper reports
submitted by clinicians upon their own initiative, when they make a
decision to go report to public health. While this system has its uses
in so far as it can provide perfect detailed information from a person
who knows the patient best, as we know, it's often slow, it's
incomplete, it's labor intensive, and requires a person to take the
initiative to report. In a number of states that have taken the next
logical step which is to shift this paper-based system onto the web, and
this of course this gives great improvement in speed and accessibility
of data and puts the data into electronic form which makes it easier to
analyze and to pass on, but still depends upon the clinicians to take
the initiative to report that information.

The other big boon that has come across us in the past 10 years or so
has been electronic lab reporting, which is lovely. It gives you fast,
accurate, often digital information with no need for clinician
initiative, so I think that's been a tremendous increase in the
authority of public health to know what's going on. That being said,
however, electronic lab reporting systems do have their limitations,
which includes some of the following: Number one is that you're often
missing detailed demographic information and patient and clinician
contact details. Secondly, you don't get any information about the
patients' symptomotology, and if the patient's pregnant or not and what
kind of treatment was prescribed. Thirdly, even though in theory of
electronic lab reporting system could do this, they typically don't -
integrate multiple lab tests to render a more sophisticated diagnosis.
My example of it here is the detection of acute HIV in a patient who has
a negative ELISA but a very high viral load. Likewise, you're not going
to get a clue that a lab test might potentially be a false positive. And
my example of that is a patient in whom you could get a positive IgM to
hepatitis A, but no liver function tests are ordered. Suggesting that
this test was not ordered because of acute presentation of hepatitis,
but rather because the patient is being screened for some other reason.
This is probably a false positive.

Likewise, an electronic lab reporting system cannot tell you anything
about a purely clinical diagnosis, for example, pelvic inflammatory
disease, or an irritated migrans rash of early Lyme disease. And then
finally, if the patient has a chronic condition like hepatitis B; the
electronic lab report system is going to kick out another report on the
condition on the same patient every single time the patient gets
rechecked for their current status. And so, based upon these
limitations, we said, "Can we do something? Can we put together the
clinical richness of manual clinician-based reporting that is been
traditionally captured by paper-based reports with some of the speed,
efficiency and automation inherent in electronic lab reporting systems?"
Can we fuse these two pathways together with automated disease detection
and reporting from electronic medical records, and thereby give us fast,
accurate, digital reports that are also clinically detailed and
meaningful.

In this process of trying to create such a system, we pursued some
allied goals in our charter, which was to try to do this in a way that
could be generalizable, in a way that could have application outside of
just our local operation in Massachusetts, that could potentially serve
as a model for the nation as to how to extract information from
electronic medical records, and at the same time, to try to fit in with
CDC's effort through the NEDSS-based system, which is giving electronic
reports from the state to federal level, can we feed in electronic
reports from the local level to the state, and in turn, could we move on
to NEDSS up to CDC? Could this be a very nice complement to the existing
state to federal system?

So, in order to do so we created a something that we called Electronic
Support for Public Health, or abbreviated ESP, which is software and
architecture to automate the detection and recording of notifiable
diseases. This system's survey is codified electronic medical record
data for patients and notifiable conditions, and when it finds one, it
generates and sends a secure HL7 report to the appropriate health
department.

Here's an overview of how it works: We begin with a practice electronic
medical record where the physician enters information as well as pulling
information from the laboratory. And on a monthly basis, we have the
central EMR create an export file containing information on every single
patient encountered in the previous 24 hours. It captures all the
diagnoses that were rendered that day, all the laboratory orders, all
the laboratory test results, the medications that were prescribed, the
patient's vital signs, their immunizations, the patient's demographics,
the physician demographics -- it takes all that information on every
single patient, on every encounter from the prior 24 hours, and it ships
it over to our ESP server for analysis. The ESP server lives within the
data center of the medical practice. All that highly private information
remains on the facility of the host medical practice as an insurance
towards security. But once it's on our server, that information now is
available for us to analyze and to try to find these notifiable
conditions. When we do find notifiable conditions, then we generate a
HL7 electronic report, and we electronically pass it on to the health
department, in this case Massachusetts Department of Public Health.

I'd like to emphasize that fact that the architecture we selected for
this separates the ESP analysis system from the host's electronic
medical records. That was a very intentional part of the design, because
it gave us the following advantages: First of all, it makes the system
agnostic to the source electronic medical record system. It doesn't tie
you into one particular proprietary piece of technology. All the source
electronic medical record system has to be able to do is to export
information. I'll tell you more later, that information can be more or
less in any form so long as it retains the required data elements. And
so, that's the first step towards making the system more universal, more
generalizable. Secondly, by putting the information on our own server
rather than leaving in on the host's medical practices' server, it makes
it easier for us to go in there to change or to add disease
definitions - it gives us more control over the analytical side of
things. We retain security, as mentioned, by keeping that server on the
host's medical practice's premises, behind their firewall under their
physical and logical control, and it makes our system unobtrusive. We
offload computing burden from the host's electronic medical records
system, and we're out of the face of the clinician. There's no barriers
put into their day-to-day routine practice in seeing patients.

The information that does come over is all cross-fed into universal
nomenclatures in order to assure uniformity so that if we do link into
different electronic medical record systems with different proprietary
coding structures or nomenclatures that we need only apply new disease
definitions once on the universal nomenclature data set. So, diagnostic
codes would come through as ICD-9s, lab test and orders, in our shop,
they begin as CPT codes and they translate over to LOINC; prescriptions
are represented as NDC codes; and diagnosis and organisms in SNOMED
terminology.

In our particular application we do have a case management interface
which presents to an infection control person or researcher or whoever's
controlling the system, an opportunity to review the cases that are
found by the system prior to transmission to the health department. This
is an optional functionality that's built into the system, which we did
at the request of our clinical colleagues, because this being a pilot
and a research-based system, I think there was some discomfort with
allowing electronic systems to automatically and entirely take over the
reporting of confidential health information to health authorities. They
wanted at least a human being to put an eye on this information. We felt
torn about this, on the one hand I think that the sensitivity to the
privilege of this information in assuring that it would only send out
accurate patient information was well heard and therefore we did this.
At the same time, we can imagine in a system that is trying to automate
things and try to get information flowing as quickly as possible, this
is a roadblock along the way.

Let me show you what this looks like. So, on a daily basis, there's a
web interface, and I apologize if you can't see this too well. But, it
gives a listing of patients who upon the system believes has found cases
of notifiable conditions - Over here, two cases of Chlamydia and one of
gonorrhea, and if you click onto one of these conditions it'll bring up
a screen like this which will tell you who this person is, whether
they're pregnant or not, what the laboratory conditions pertinent to
that condition are and the results, what medications were prescribed,
and if the patient has symptomotology as can be discerned from ICD-9
coding, like arthritis or abdominal pain or fever, would present that as
well. Once that information over here is presented to the physician
reviewer, they can go ahead and authorize the reporting of that case to
public health.

The report that does go to public health contains the following
information: there's patient demographics, there's the responsible
condition, there's cyclic contact information, why is it that ESP
believes the patient has the reported condition, but also, and these are
the elements that go above and beyond the electronic lab reporting
system which would give you, the treatment that was prescribed to that
patient, the patient's symptoms as can be discerned from ICD-9 coding,
and the patient's pregnancy status if that's pertinent to the condition.

This system has now been operational in Atrius Health in Boston. Atrius
is 27 multi-specialty, multi-site-practice based in eastern
Massachusetts. We have about 600,000 patients and over 500 doctors in
the system, and we cover about 10% of the population of Massachusetts. A
single ESP server covers the entire population. The ESP server itself is
located in the central data processing center of the medical practice.

So, to date we've reported over 1,100 cases of Chlamydia, 150 cases of
Gonorrhea, 25 cases of pelvic inflammatory disease, 6 cases of acute
hepatitis A. We have definitions programmed into the system, operating
in test mode for acute and chronic hepatitis B, acute hepatitis C, and
tuberculosis, as well. And, I'll share with you some of the performance
information on some of those conditions, as well, as we move through the
talk.

How can identify these cases? So, we use logical combinations of lab
tests results, lab test orders, results, diagnostic codes, vital signs,
medication prescriptions, to try to define these conditions. And, this
is novel work that we always hoped that we could just take from CDC or
some other authority. But of course the existing case definitions still
exist very much in the clinical or epidemiological surveillance realm
and have yet been to be translated into the kinds of terminology that
are appropriate for an electronic-based system. And so, as we try to
develop these conditions based on our operating thinking as clinicians
and as epidemiologists and what they should be, we go ahead and we test
them. And we have the benefit of a natural laboratory of our health
system's or our clinical system's electronic medical records spanning
back to about 18 years worth of data and covering many hundreds of
thousands of patients. So, once we develop an algorithm, we apply the
algorithm to a pool of historical information, we pull the charts of
every single patient identified by the algorithm in order to assess the
accuracy of the identification, we then compare that list with the
Massachusetts Department of Public Health list of patients who ought to
have that condition in the state in the given time period, and that's in
order to -- twofold -- in order to identify the patients we might have
missed, so we can refine the algorithms further, and secondly, in order
to get some sense of the performance of the electronic algorithms
compared to traditional paper-based reporting. And, then we repeatedly
refine the algorithms in order to maximize their accuracy. We
collaborate closely with epidemiologist in the Massachusetts Department
of Public Health on a program by program basis in order to refine those
definitions and make them as useful to the programmatic interests of the
relevant divisions.

So, let me give you some examples over here. This is a case
identification logic for Chlamydia. Chlamydia is actually quite straight
forward, because the disease is defined by a positive Chlamydia test.
So, our challenge is to identify all the tests for Chlamydia that exist
in our healthcare system and define anybody that gets a positive result
from one of those tests as being a case. What's interesting about this
list is the number of tests that are out there for this -- for Chlamydia
-- and I'll tell you this is only a partial list, it's only about half
the relevant conditions. And you'll notice that there's a lot of
duplication over here where there appear to be similar tests with
variant codes, which is, I think, one of the unfortunate practical
realities of the ways that coding is taking place in our medical system.
One of the interesting things to us as being, is by coincidence, over
the past year when we've had our system active, at Harvard Vanguard,
they've supported strategic alliances with other practices in the area
and integrated the medical records to give -- and as each new practice
is integrated into the common electronic medical record. We suddenly get
all their information as well and low and behold we see that we have to
update our disease definitions to capture all the new codes that each
new practice has for the very same tests. So, we're still looking for
the identical test, but there are new codes to describe that test from
every new medical practice that is added to the system. You need some
sense of just the idiosyncrasy and variance of the way that coding is
done on the ground medical practices.

Here's something a little bit more sophisticated -- and, this is the
logic I'm developing for acute hepatitis B, and -- a couple of different
approaches to looking for it -- one would be to combine a patient who
has elevated liver function tests and IgM to core antigen, as well as
pulling in additional information from the electronic medical record and
ICD-9 coding. So, my first example has where the ICD-9 code for jaundice
in addition to liver function tests, another way to get at the same
issue of hepatitis, and combining that with a very specific test for
acute disease IgM to core antigen. A lot of patients don't get a lot of
IgM to core antigen when they present with acute hepatitis, so another
definition over here looks for the combination of elevated liver
function tests, elevated bilirubin, hepatitis B surface antigen or 'e'
antigen present -- but then also says, look back in the patient's
record, make sure it's not chronic hepatitis B by making sure they've
never had a positive hepatitis B test before, and likewise, if there's
no ICD-9 code over there for chronic hepatitis B. Through this
increasingly complicated definition, we've been able to get pretty good
identifications of the acute cases. But, to give you the sense of power
of using an integrated electronic medical record system, where you can
take multiple streams of data in addition to what comes out of the lab,
you can also look at the historical information, look at the diagnostic
information -- you can look at prescriptions and what have you as well
to try to get the algorithm to be as sophisticated as possible to
maximize accuracy. And then to give you another sample of the same
challenge, here's our definition for active tuberculosis, you'll see
over here for tuberculosis we've incorporated some medication
prescriptions into the definition on the basis that pyrazinamide is a
very specific drug for the treatment of active TB. You'll also notice
over here that we're using order for an AFB smear or culture followed by
ICD-9 code for TB, and I think it's interesting that the definition is
based on an *order* for an AFB smear as opposed to a positive result for
an AFB smear, and that's intentional in order to capture cases of
culture-negative disease. By combining the ICD-9 code with an order for
2 or more anti-tuberculosis meds (medications), we retain our
specificity while increasing our sensitivity for capturing those
culture-negative cases.

Here's some information, showing you a year's worth of comparisons
between manual reporting and electronic reporting. The manual reporting
is done in our system by dedicated infection control people, and we had
them continue their practice as usual when we implemented ESP over the
past year and a half in order to be able to give us a comparison; how
does ESP compare to traditional systems? Let it be said that, in our
practice where we do have a dedicated infection control person who can
do reporting is way ahead of the curve; most medical practices do not
have a dedicated person in order to do reporting -- it's left to
clinician initiative. And so, from this sense we're comparing ourselves
against a very high standard -- an above state-of-the-art, in terms of
standard. Nonetheless, we were still able to show about a 40% increase
in the number cases of Chlamydia that we were able to find, a 50%
increase in the number of cases of Gonorrhea, a substantial increase in
the number of cases of pelvic inflammatory disease and acute hepatitis
A. The big increase in cases of pelvic inflammatory disease is
interesting because that's a clinical diagnosis -- you can imagine in a
system, a practice where infection control practitioners are doing the
reporting, who's not necessarily privy to clinical diagnoses, but only
privy to laboratory information; obviously, that's going to be something
difficult for that person to report, but by generating a case definition
in ESP that includes ICD-9 coding as well as some other factors, we were
able to capture some of those cases.

What I'd like to really emphasize is that, this is not purely a numbers
game. That the benefits of ESP is not purely about increasing the number
of case reports that are found, but also I think some of the quality and
richness of the information about those cases, as well. So, looking at
pregnancy, for example, which I think is a high priority thing to know
about, to prioritize public interventions for patients with certain
conditions. On our paper reports, pregnancy was only commented upon in
only 5% of cases. But, ESP commented upon pregnancy in 100% of cases.
And so, therefore it's not surprising that under the ESP system we
substantially increased the number of patients we found that were both
pregnant and had a case of either Chlamydia or gonorrhea or public
inflammatory disease, a twelve-fold increase or eighty patients in
addition that were found this way.

Likewise, in terms of treatment information that was reported, we were
also able to increase the number of patients that treatment details were
shared from 86% to 100%. Again, take that 86% with a grain of salt, that
86% again is in a system that has a dedicated reporter. I think routine
public health reports rarely include the treatment information. The true
number for the system at large might be more in a 40-50% kind of a
range. We also found that we were able to pick up on, or correct some
transcription errors. So, looking at just the name field, of the
patients, in comparing what was on the paper reports versus what was on
the electronic reports, we found discrepancies 6% of the time -- and
those include things like: reversing the first and last names, basic
spelling errors, an incorrect first name -- again, you can imagine how
that can put the roadblocks into public health.

What about the accuracy of these conditions? Over here, I'm including
some of our test conditions as well -- as I mentioned, every single case
that's identified by ESP at this point -- let's look at the
full-condition review in order to show the accurate side -- I can tell
you what the false positive right here is. For Chlamydia and gonorrhea,
despite having reported over, I think 1,300 cases now, there have been
no false positives - think that's simply a function of just how simple
the definition is; you just have to have a positive lab test -- it's not
that tough. Pelvic inflammatory disease, there was one false positive,
and I'll tell you about that in a moment. Likewise, with acute hepatitis
A - one. Amazingly, none for acute hepatitis B, which I think is a very
complicated condition to detect electronically -- we've done really
nicely with that. And, TB -- two cases out of eleven.

Let me tell you what those false positives are. I think it'll give you a
flavor for both how the system can go wrong and also, for how to
integrate the thinking of this sort of a system into public health. So
the false positive for pelvic inflammatory disease was a patient who had
pelvic pain and ended up having a positive culture for *Herpes simplex*
and Chlamydia. And, when I reviewed that case, I think pain was a
function of primary herpetic outbreak as opposed to her associated
Chlamydia infection. But, you can see how they are pretty darn close,
nonetheless. The second patient is the patient who had acute hepatitis
A. It was a young woman with 10 days of pharyngitis and fatigue, showed
negative monospot, a positive IgM for hepatitis A, and Epstein-Barr
virus -- viral capsule antigen IgM which is positive. Looking at that
case, my clinical nuance is that this is actually a case of mono, not
acute hepatitis A, in that the hepatitis A IgM was a false positive.
And, in theory we could refine the algorithm for detecting this disease
in order to eliminate these kinds of false positives. We could do that
by making the requirement for the LFT elevation to be much more
dramatic, people who have mono classically -- it's not always true, but
-- have a lower elevation of LFTs. Look for other positive tests and use
that as exclusion criteria. The reason I didn't do this, however, was
that the specific request of our public health -- of the qualified
health partners is that -- we really want to gear these algorithms
towards sensitivity. It's much more important to us that we capture
every potential case, rather than to take a chance of missing any cases
by the algorithms that get a little too smart for themselves. In
addition, it's worth noting, that this patient would be reportable under
traditional systems, via through ELR systems, which would report this
patient because she got a positive IgM on hepatitis A, so she would be
reported as a case of hepatitis A. And, also the clinicians could report
this a suspected case of hepatitis A. Because, I guess without a biopsy,
if you don't know what the true cause is, clinically, it would be very
appropriate to report this case. And so, allowing ESP to report these
sorts of false positives, in a sense we're mimicking clinical practice
and gearing ourselves. I think the reality of public health practice in
terms of what our partners said they want. Likewise, the false positive
cases of tuberculosis again, I think, are well within the ballpark even
thought they're not true cases. One is a patient exposed to MDR TB that
had no active disease. And the patient was caught by the algorithm
because he was prescribed pyrazinamide as treatment for a potential case
of latent tuberculosis infection -- so, you get a sense of how the
algorithm went wrong. But, again this something that -- arguably --
public health would want to know about in that, a clinician under the
manual-based system would be very, very valid but would be quite right
to report to public health nonetheless. The other patient was somebody
who had a prior history of TB and presented with hemoptysis and nodules
on their chest x-ray. Yet again, still the same theme of the false
positives are not read in events from that field -- their conditions are
closely associated with their entire condition, that in and of
themselves are valid information that can and should be reported to
public health and is hooked through, as is the current practice with
manual systems.

Looking at this hepatitis B example, but more to get a better -- more of
a flavor of what ESP can add over an ELR system. So, in our year and a
half worth of data we have about 600 positive test results for hepatitis
B, which under an ELR system would all be reported to public health. You
take a closer look at those 600 positive tests however, you'll see the
following. It only captures 138 distinct patients, so a lot of
re-reporting of the same individual. The ELR might fail to actually tell
you that. It might be able to de-duplicate those cases for you. What
it's not going to tell you is that 5 of those cases are acute and 133
are chronic -- to do that sorting for you in order to prioritize public
health intervention. And again, from our public health partners, for
them to be able to the leap start on which of those 600 cases -- what is
the needle in the haystack of 600 cases that actually represent acute
cases that require acute immediate investigation for contacts and other
risky scenarios? -- to be able to give them that head start on
information, I would argue, is very valuable.

There were some cases that were missed by our system. So, 5 cases that
were known to the department of public health were missed by ESP. Just a
point of comparison -- that's versus 266 cases found by ESP that were
missed by DPH, so we're not doing too badly. Taking a look all those
missed cases, there were tests that were edited off the placement of the
EMR -- in other words, they were initial negatives results that got
turned into positives. That's a systems message for us, about a way in
which we need to refine our architecture in order to capture edited
results. Of interest, there were also 11 cases that were missed during
the upgrade of our EMR from one version to another, and in the transient
one-day disruption of the flow of data, so we missed those cases that
day. We subsequently found that we missed those cases in a day, but
before then it's another hint at some of the operational challenges in
actually making systems like this a reality. The things that can
interfere day-to-day functioning of your EMR is going to interfere with
your public health awareness.

So, where are we moving next? We're adding additional diseases to the
system. We're working right now on Lyme disease and some of the vaccine
preventable diseases, and -- let me give you a flavor of some of the
protocol that's under development here for measles, mumps, and rubella.
There are two derivations in this disease, again using the guidance from
our epidemiologists in Massachusetts. And one is, not simply to report a
patient who has a positive result, but rather to report those on whom
there's a clinical suspicion, as gauged by either an ICD-9 code or just
a lab order for an IgM to measles, mumps, or rubella. And, that's in
order to kick start public health investigations in this very, very
sensitive area. We also, in contrast to other conditions, will include
patient's complete immunization history in order to help with the
interpretation of that case. And it gives accurate clinical clinician
contact information to facilitate the investigation. I think what's most
interesting about this definition is that we are thinking about -- we
have not done this, but -- thinking about including a simultaneous
message for that clinician who ordered the IgM to measles, to ask them
some questions about their case immediately -- what are their patient's
symptoms, who are their patient's potential contacts, is their patient a
childcare worker -- whatever pertinent piece of information we need, and
to immediately send that information along with the lab order report to
public health, again to kick start the investigation.

There's another system as well -- which is taking a step back for a
moment and taking a look at the breadth of information that's been
captured by ESP --all these diagnoses, and prescriptions, and lab
orders, and so on and so forth and say -- well, are there other public
health applications beyond notifiable condition reporting that could
leverage the breadth of information that we have, as well as the fact
that we have a system that can do analysis, and a system that can
communicate. And, so we actually have a grant from AHRQ, to try to
incorporate vaccine adverse event surveillance and reporting into the
system. In a nutshell, the way that this is going to work, is we will
prospectively survey every patient who gets a vaccine for the next 30
days and looks to see do they pop up with novel diagnoses, or suggested
by chemical changes, or new vaccine allergies that might be concluded in
this patient to have an adverse effect from that vaccine. And if so,
we'll send a message up to that doc who ordered that vaccine, and say,
"Hey, your patient suddenly popped up with hepatitis after that vaccine.
Do you think this could be related to that vaccine?" And, if the doc
says, "Yes," then we'll offer to submit an electronic report directly to
VAERS on their behalf.

I think that thinking beyond vaccine adverse events and taking into
account the breadth of information captured by ESP and its analytical
and communication ability, there might be even further applications that
could be -- serve additional public health priorities -- and that
includes patient safety initiatives, such as ensuring that there's been
adequate follow-up of critical test results, drug interactions, renal
dose adjustments, medication adverse effect, making sure patients got
missing health -- identifying those patients who have missing health
maintenance activities, vaccine registries -- I think there's a vast
array of other potential clinical applications that could use this
particular platform. We could also incorporate syndromic surveillance,
asthma surveillance...There's also thought of, "Can we make the system
even more useful by adding insurance claims, in order to capture what
happens what happens outside of a medical practice, inside of a
hospital, in order to, I think, comment more actively on true exposure
of the patient's different medications by not looking at prescriptions
but actually looking at the medications dispensed in order to increase
the sophistication and utility of the system?"

We're also working right now, making good progress on getting the system
implemented in North Adams, Massachusetts. And North Adams is part of a
collaborative, under the rubric of the Massachusetts e-health
collaborative, which is an initiative in Massachusetts to give
electronic medical records to every single physician in 3 pilot
communities across the state. And, North Adams has got to the point
where there is a computer in every doctor's office. There in the process
of creating a regional health information exchange to allow for exchange
of information across the region. We are working closely with them to
put an ESP server inside their health information exchange in order to
see if we can prove the portability of the system, to adapt if for the
health information exchange environment, and to see if our adage that
this system has been designed for portability and generalizability in
fact is true or not. This is going to be our test bed. And right now --
there'll be no promises -- but, our hope is that could come live as
early as this spring -- February-March kind of area.

What I'd like to think about with this audience is: What are the steps
toward disseminating ESP beyond Massachusetts? The software itself is
freely available under a lesser general public license, but I think we
can all foresee that installation and maintenance of new ESP systems is
not trivial and does require some IT, epidemiologic, and administrative
expertise and resources. Is there a role for CDC or other national
leaders to provide guidance and push and resources in order to allow
that to happen? In order to help us conceptualize this challenge, I'd
like to lay out some of the barriers to broader implementation to ESP.
And the first one is -- I think the most obvious one -- which is that,
only that 35% of multi-physician practices actually have EMRs. Clearly,
the first step over here is to put more EMRs in more offices in order to
be able to leverage the advantages of electronic information. The second
big challenge we found is that a lot of the -- to simply have an EMR is
not enough -- the EMR has to capture the breadth of information that
we're using for our disease definitions and that are ultimately of use
to public health. And, one of our big challenges in finding a partner to
reproduce ESP has been finding somebody who enjoys the same breadth of
information we have. There are other health information exchanges out
there. There are lots of systems with electronic medical record systems.
Few of them include the full breadth of medication prescriptions, and
lab orders, and lab results, and so on and so forth; they are usually
much, much more limited. We'll have to contend with different coding
nomenclatures and cultures in different EMRs. The fact that there is a
constant influx of new lab tests, new diagnosis, new medications, and
their associated codes, the absence of standardized disease definitions
tailored to electronic data, the absence of standardized reporting
elements for most diseases -- although it's changing - the paucity of
resources to support the implementation support of ESP-backed systems
and general public wariness about electronic surveillance and health
reporting.

We're going to take a few of these and look at them a little more
closely. First of all, the problem of heterogenous EMR systems -- that
there are different proprietary standards out there. And, there's a vast
array of different email systems and each has different capabilities,
different operating protocols - our solution for that has been to
decouple ESP from the host EMR, and therefore, all the host EMR needs to
be able to do is to export plain text files or HL7 files with the recent
encounter data.

That brings up the problem of heterogenous codings. Are we allowed to
get that information onto ESP, but if it is coded in different ways for
different systems, what are you going to do about that? Our solution has
been to map those proprietary codes into universal nomenclatures, and
let it be said though, which -- that can seem to be a daunting task, I
think that the current LOINC list has, I think 48,000 entries on it, and
nobody wants to map 48,000 conditions - but, you really only need to map
those lab tests that are pertinent to the notifiable diseases or to the
case identification logic that we've created to date. So, there are
actually only about 30 different tests that are code mapped within ESP,
which all of a sudden becomes a very manageable number -- that's half an
afternoon's work.

To give you a bit more of a flavor of some of the challenges of code
mapping over here, here's a partial list of some of the Chlamydia tests
that are from our system. If you'll take a close look at them you'll
notice a few interesting things. First of all, there's this thing,
LA0219 -- that turns out to be a proprietary code, our system just made
that up, so nobody else has that. There are multiple codes over here for
the same tests; this is all identical tests with the three or four
representations of it. This one over here is an incorrect code that's
actually a gonorrhea code that's being used for surveillance of
Chlamydia. And, this one over here is an obsolete code, which doesn't
appear in the current CPT terminology, yet it's still in use by our
health system. I'm telling you this to give you a flavor, of first of
all, the "wild west" of coding that exists in different medical
practices, and the importance -- the essential importance -- of mapping
the crazy combination, that quilt work of codes that exist on the ground
into some common terminology, which in this case happens to be LOINC
coding.

There's this problem of constant new appearance of lab and drug codes;
how do you keep your mapping up to date? Our solution is to build into
ESP itself a natural language process that constantly scans every single
incoming code for those that might be pertinent to the existing case
definition. And what you see at the bottom of the screen is an email
automatically generated by the systems saying, "Hey! There's a new test
over here for *Neiserria gonorrhea*. Add it into your case definition."

There's the challenge over here that currently there are no standardized
definition for identification of notifiable diseases from EMR data, and
we all know that standardization is essential for data compatibility
across sites and states. And, the challenge -- the validation of
definitions when you do create them requires very large populations in
order to assure the algorithms accuracy for rare diseases. In that, the
solutions for this are less clear to me. I would think that it's
something -- a challenge that's larger than our operation, because even
in our shop of 500,000 patients and 20 years worth of data, we'll still
only get a handful of patients who have tularemia or anthrax. If we
create a definition for that, how can we tell you any thing about its
performance characteristics or its accuracy? So this seems to be a place
for collaborative work between different entities, under the rubric or
umbrella of CDC or CSTE. And, John Abellera has been doing beautiful
work on the electronic reporting elements -- what should be reported
once you find that disease? -- and, perhaps there room to translate that
work to take it to the next step into, how do I find those diseases in
the first place?

And, then finally, how do you disseminate ESP or ESP-like systems across
the country? Where should disease detection and reporting be integrated
into the health care system? And a few different ideas -- one would be
to integrate the ESP logic that case identification into EMR software
itself to make notifiable disease reporting a HITSP standard for EMR
certification, just as EMR now has to also has to report on drug taking
and drug prescriptions, what have you -- this should be another level of
functionality to support public health. Another model might be of that
which we are pursuing in North Adams, which is to install ESP-like
systems in regional health information exchanges. If so, who is going to
lead this effort? Who is going support this effort? Who is going to
define the standards for this effort? Who is going to collect the
results of this effort? Another possibility might be to use ESP case
identification definitions on data collected by BioSense; although, I
speak from a position of complete ignorance as to the actual breadth of
data that are captured by BioSense.

So, those are some thoughts and some challenges that I had with this
organization. Here are the people who worked together on this large
collaborative effort. We are always looking for collaborators, people
who would like to take on the mantle of either testing the
reproducibility in new sites, in helping us to deliver these case
definitions, or from CDC's perspectives in helping us with
standardization -- on how to make this model something that does more
than simply an ivory tower effort in Boston, MA. Thank you very much

DAN SOSIN: By protocol, we'll take questions from afar first. Starting
with the telephone, then going to -- there is no Envision, right?

JENNIFER WARD: There is Envision. But we can't see them

DAN SOSIN: I usually see them on TV. Envision...and then we'll take them
here. I guess we also have some questions coming in electronically and
we'll be sure to ask those questions, as well. Can we go to the phone
first - any questions?

JOE FRED GONZALES: This is Joe Fred Gonzales at NCHS. Do we have any
idea of the statistic of 35% multi-physician practices have EMR, only
35%, right?

Michael Klompas: That's the number from the health information 2006.
Their number is as current as the end of 2004, which is the most recent
statistic information available.

JOE FRED GONZALES: That's pretty staggering, to think that 65% of the
physicians - multi-physician practices don't have electronic -- an EMR
system. They are not using the EMR system. I guess I would be concerned
about that. How are the 65% distributed all over the country? Are they
concentrated pretty much in part of the country where we could be having
something going on and not -- obviously, it couldn't be reported fast
enough because there's not any electronic way of doing it. I don't know
-- I guess I'm just concerned about the 65%. There's no chance of
getting timely information, real-time information, or as close to
real-time as possible.

Michael Klompas: I think you're point is very well taken. And, I think
this speaks to the mandate that came from President Bush 3 years ago to
try to radically increase the drops and uptake of electronic medical
records systems across the nation. I think there is a kick in census
from the policy point of view that has to happen, and the challenge is
means -- is how to operationalize that and part of that challenge is
means to -- how do you support doctors who do want to dock electronic
medical record systems? Right now, their financial incentives are not
quite right. The doctor has to pay, but the benefits go to the health
insurance company or to the patient. They also need models to facilitate
the adoption of electronic medical records that shift some of the
financial incentives to make them more attractive, which include funding
opportunity from insurers to doctors to try and leverage those systems
for them. But, I think your point is very well taken so far as this is
obviously step one.

DAN SOSIN: So, certainly representativeness is an issue today, but we're
all counting on a time period when all those barriers to uptake of EMRs
are removed, and we do anticipate a time where this will become standard
practice and we need to be ready to implement. Here's a good approach as
a model for how we might support that in public health. Any other
question from folks on the phone?

JUNE (OR): This is June in Oregon. I just had one question and that's
about the laboratory reporting. It seems like the medical records have
to have the lab data already involved there. Is this from one major
laboratory or is it from multiple micro, clinical lab growths? -- The
Boston area.

MICHAEL KLOMPAS: So, it's the breadth of laboratory information that's
captured by the electronic medical record, which in our shop, and I
think in most places with fairly sophisticated electronic systems, will
integrate lab reports from both large national laboratories, in-house
laboratories, and specialty laboratories. Because that information is
all ultimately needed by the treating clinician, that information
ideally all flows back into the electronic medical record system. Is
that the ideal home for it? Your point I think is important, because
there are other systems where the electronic medical records are not
quite so sophisticated, and might only capture information from one
large public -- one large national laboratory -- and might not have the
ability to insert into the electronic medical record specialty lab
information.

DAN SOSIN: Thank you. Any other questions from folks on the phone? We'll
come back towards the end. How about envision? Alright, how about in the
audience? We'll take -- I've got a couple questions here, too. \[Section
omitted -- not relevant\] How about Jerry?

JERRY: You mentioned the capability to look back in time and see what
previous lab tests were. I'm just wondering -- the ESP server -- how
much data can it hold? How does it handle that retrospective look? I'm
assuming that you -- and maybe this is actually the first part of this
-- how much data is actually held in its memory at any one time, and how
does it accomplish that look back?

MICHAEL KLOMPAS: So, the question is -- how much information is actually
held on the ESP server and what are the implications of that for doing
historical back queries when you want to get a sense of whether a
condition is acute or chronic? Is that correct?

JERRY: Yes.

MICHAEL KLOMPAS: So, the -- we've been very greedy. We've kept every
single piece of information that's come our way. We feel okay about
again that because that information still is under the health practices'
premises under their control and with their agreement. Because the
implementation in Harvard Vanguard only uses flat files which actually
are very small, the total amount of information is actually pretty
manageable. We have 45 million records on the system right now. But,
only actually takes up a few megabytes of RAM. And, we -- so a server
that is no bigger than your standard desktop computer is well able to
handle the load. I think your point, your question also has important
implication for when you want to implement this elsewhere. What if your
electronic medical records system is brand new and doesn't have the
patient's historical information, or what if you're only able to bargain
with your health information exchange to give you a prospective flow of
information and no prior information? Is that going to compromise your
algorithms that look for -- that try to sort out acute from chronic
information -- and, I think the answer is almost certainly going to be
yes, although there is some effort to try to get around that by that by
incorporating the ICD-9 coding. So the patient monitored - You might not
be able to note that the patient had a positive hepatitis B test back in
1987, they could very well still have current, chronic hepatitis B ICD-9
code on the record. So, but I think that's an important lesson that
there might actually have to be a lead in period where you accumulate a
certain amount of information before you actually turn these algorithms
on.

DAN SOSIN: Can I follow up on that question? Also, as we think about
what we miss in a system - in a model like this. That lack of historical
information would also come with patient populations who are transient.
So, you would have them in and out of your system. Can you tell us a
little bit about how much that happens in your systems and if there are
any solutions you've been thinking about regarding that population --
how you might have the historical information?

MICHAEL KLOMPAS: Right. So, we get to enjoy a bit of a cloud-9 existence
up in our system, because we're based out of what was formerly an HMO.
And, so we have largely a captured population that has remained more or
less consistent over time. So, in some sense we've managed to avoid some
of the transientness and flow of people. Again, you might be able to get
complete information on that patient to be able to tell you the public
health at any one moment; whereas the breadth of disease that they've
had over the year it should want the most current disease it is where
you should be able to get information. The other approach, maybe, to
deal with this problem though about patients who jump from practice to
practice would be to base ESP in the health information exchange for a
community or for a region where it's capturing information from multiple
different medical practices and therefore can do the integration work
for you.

DAN SOSIN: Lisa.

LISA: Thank you for that, first of all, a really intriguing project.
I've a couple of comments and a question. \[Section omitted -- not
relevant\] Obviously, this whole thing hinges on ICD-9 coding. And,
there's a large literature that talks about inaccuracies in coding and
all kinds of medical record type things -- so, I wonder how you dealt
with that. And, then other question is -- you had mentioned that each
office or each system has its own quirks about their coding and I wonder
if you can just talk a little bit about the effort that it takes to make
sure these algorithms are accurate for each office you bring on board --
what that looks like as you're thinking about spreading this wide.

MICHAEL KLOMPAS: Great. So, the first question has to do with the
accuracy of coding. It's well known that the ICD-9 coding done by docs
is only maybe about 70% accurate. And, does it have implication for the
accuracy of disease detection given that a lot of our algorithms do
incorporate ICD-9 coding? The answer is going to be, "Absolutely. It
certainly must have implications." The way we try to deal with that,
however, is to never build a disease based purely on an ICD-9 code. You
have to always take in additional pieces of information, such as the
medication prescriptions, which can be another guide to the way the
doctors think about the cases, what they order, what the laboratory test
results are in order to refine the algorithm. The other important thing
is that each of these algorithms as described are tested and so that we
ultimately can describe precisely the sensitivity and specificity --
probably the sensitivity, maybe the specificity -- of these algorithms
in practice. I think what's going to be intriguing is when you shift
from, say our group, which is HS healthcare, and you go to North Adams
where docs might think about coding in a different way -- is that going
to change things? On the one hand, maybe yes because it's a completely
different part of state, different culture; on the other hand, possibly
no, in so far as, we're already tracking the coding of over 500
different doctors and there is no particular reason why the doctor one
part of town ought to be coding the same way as the guy in the suburb 20
miles away. But, I think the bottom line is testing, testing, testing --
being able to fully describe to you the points and characteristics. I
can tell for example, the positively predicted value of acute hepatitis
B algorithm is 97%.

DAN SOSIN: We'll go to the back of the room first and then we'll come
up, get you guys up front.

AUDIENCE MEMBER: \[Indistinguishable speech\]

MICHAEL KLOMPAS: I'm going to address your first question first, because
I understood that clearly. Your second question unfortunately was
interrupted by some of the beeps. I'm going to ask you to repeat it. The
first question has to do with the, I think, the challenge of
generalizability. How do you spread a system that right now is based
upon one single server that looks after one large medical practice, is
that really an extrapolatable model when there are so many different
medical practices out there? And, the answer is that really has to do
with my last slide -- looking to you, this group, for guidance on how we
ought to address that problem. I think that at the end of the day there
could be different models suitable for different kinds of practices, as
to how to do this. So, for example, a large multi-specialty, multisite
practice like our one, HHS healthcare, where one single server can
capture 500,000 people, that are pretty good. But, obviously, that model
is not realistic at all for a single doc in a box who has a solid
practice who certainly is not going to want to install an entire server,
and, so that person -- I think the model ought to be either that you
integrate the rule of right onto his electronic medical record module
software as a separate module in the software network could sit on the
docs desk computer, and it's also running his EMR. Or, the other model
is to put this into the health information exchange where you serve an
entire community at once. So those are possible approaches.

DAN SOSIN: Let me, for the sake of time - and there are number of
questions -- let me to take a couple more in the room, then I'm going to
the ones that have been coming in by computer. Up here first, come over
here, and then I'll come back to you.

AUDIENCE MEMBER: Your system sounds wonderful. It has a lot of attribute
including completeness, and timeliness, and richness. And, I wondered,
as a follow-up question, one is that it seems to me that some of these
functions can exist apart from some settings, so that for instance, if
-- since it's such a complex system, you don't need to use the complete
system just for the prompting part of it, you can just can just kind of
prompt the health department about a condition that is relatively
statistically declined, they can always call back in order to pursue
that, I would assume. Similarly, I would imagine you can use this for
feedback to providers in order take some action based on a laboratory
test or something like that. Any comments about that?

MICHAEL KLOMPAS: So, the question has to do with, does the system need
to be as complicated as it is and can we decouple some of the functions
of the system, for example, this idea of querying back to clinicians
about information about cases and can that be done separately from the
system. Is that correct?

AUDIENCE MEMBER: Just for more flexibility.

MICHAEL KLOMPAS: For more flexibility. I think this is very much a work
in process. To address this specific idea of sending information back to
the doctor or asking the doctor questions about the case, or following
up to see -- did this doctor prescribe a treatment for the known
Chlamydia of if they didn't or did even they know they ought to? Those
sorts of functionalities require messaging from ESP back to the doc.
We're still trying to work out how to do that best, because it's -- in
terms of the aim to couple the different kinds of proprietary EMR
systems - it's fairly easy to get them to spit out common information;
it's a lot tougher for them all to receive information. So, we have a
couple of solutions that we worked on in our own shop. We can design
something that's tailored to the EMR systems that being used by Atrius,
by - which is EpiCare. That is something that we are actually creating.
You could have also had conventional email that invites all the problems
about confidentiality. You could have email with a web link into a
secure server, which would be another option. Although, then you'd have
to ??? do doctors read their email? I think that that's really much a
point for open debate and discussion, and I don't really know what the
final answer ought to be for that.

AUDIENCE MEMBER: \[Indistinguishable speech\]

MICHAEL KLOMPAS: The question has to do with how we define conditions in
particular, different coding structures, and our current coding
structure is ICD-9, and the question is, "Should we be using
meta-coding?" Again, I think it's a question that's unanswered. We're
using the coding that's currently available to us, and which I think is
still the dominant standard for the United States, which is ICD-9 based.
Could another coding system come to replace that? Absolutely. And, if
so, we would have to do the translation work. That could either mean
redefining our algorithms using another coding structure for diagnoses,
or it could mean another mapping interface. We'll switch from, say ICD-9
into whatever the case may be. I think it's matter of simply being
responsive to what's prevalent in the community.

DAN SOSIN: Let me take two questions that came in electronically.
Initial question from Margaret Filos. Excuse me, if I mispronounce your
name. "Do your demographic variables include occupation?" And second,
"Will ESP include reporting of chronic diseases that may be reportable
in Massachusetts based on state laws?"

MICHAEL KLOMPAS: So, moving on to the first question of demographic
variables -- do they include occupation, which I would imagine people
would want to know. Is this person with acute hepatitis A a food
handler? The answer is that the capture of variables like occupation is
only as good as the source electronic medical record itself. And, while
there is a feel that most electronic medical records force something
like occupation, as something that's typically filled out the first time
a patient joins a practice and is rarely updated as a person might shift
their jobs. So, the answer to that would be -- I would personally be
very distrustful of what the EMR claims the occupation would be. And,
that might be -- I think the implication is -- either you have to go
down the step where you do ask the doc -- query back to the doc. Or, you
need to accept that electronic reporting systems are not going to
replace gumshoe epidemiology -- that somebody's still going to have to
help them further contact this clinician or patient, as the case may be,
and get the very, very rich detailed information that you need to do
effective public health work. But, that the importance of ESP is really
to jumpstart that as much as possible, to give you as much of a head
start as you can. The second question about reporting, "Will ESP include
reporting of chronic diseases that are reportable in Massachusetts based
on state law?" -- So, for example hepatitis C - and the answer is, to
the extent that is requested by the Massachusetts department of public
health, yes. So, there are 74 reportable conditions in Massachusetts,
each development of the algorithms for each condition is a substantial
body of work that can last anywhere from 3 to 12 months. So, we try to
prioritize our condition development work quite carefully. And, we seek
guidance from the department as to what they want. And, their guidance
has been to either seek the very high frequency conditions, where
there's a lot of numbers so it's helpful to deal with them
electronically rather than on paper, or to seek the very high acuity or
high public health impact diseases, such as the vaccine preventable
diseases or influenza. So, that's really where we've been going. When
they say to us, "Work on chronic hepatitis C," we'll do so.

DAN SOSIN: Great. Thanks. We have a question from Tom Safranek, "Your
model places an ESP computer behind the firewall on site where the
electronic health record is located. Have you considered dumping an
extract of the electronic health record onto a common ESP server that is
outside or beyond the," -- I'm not sure if I have this right -- "LIS
firewall where all the data from the various EHR systems would be
processed in a single location?"

MICHAEL KLOMPAS: So, a very important question is, I think, as we think
about models for expanding beyond one city into a national model, is how
to capture this information. Is there potentially an efficiency to be
gained from having a sort of a monster ESP server that shows an entire
state or even an entire nation, and we immediately run into the problem
of people's willingness to share confidential information. Particularly,
when you say, "I'm going to send this ESP server every single piece of
information about every single patient that comes through the door,
whether or not they have a notifiable condition to process it." Our
experience with other projects, and I'm assuming it upon this project,
is that medical practices are very very wary, very very loathe to open
their doors and give out every single piece of information that they
have. So, I think from an informatics point of view, that's a beautiful
solution. From a public policy or a public acceptability point of view,
I think it's going to be very challenging to do that.

DAN SOSIN: Thanks. I'll come back into the room. There.

AUDIENCE MEMBER: I was wondering, what are the possible solutions to the
considerable burden it places on hospitals to maintain mappings and
things, is to think of making ESP indispensible to the hospital. And, as
much as we'd like to think public health reporting is indispensible, the
physicians would probably disagree. So, I was wondering if you had any
discussions with physicians within the hospitals to look for areas
that - which they can leverage themselves, which is sort of outside the
curfew of routine reporting that might make this into sort of a killer
app\[lication\] that people are looking for that would lead to people
wanting to support it.

MICHAEL KLOMPAS: Great. So the question is: Let's recognize that public
health reporting might not be a priority of many medical practices or
hospitals -- Is there a way to sneak ESP public health reporting health
systems into hospitals by making it attractive to them for other
reasons? I completely agree with your premise and your solution. I think
that our response to that is to try to start thinking about some of the
clinical decision support modules that I briefly alluded to at the end
of the talk. And, can we sell the system to hospitals on the basis of
patient safety or quality improvement, saying "We'll start looking for
drug interactions for you. We'll take your patient who is on Worfrin and
monitor them so that when they prescribe them Amoxicillin, their INRs
are about to shoot the roof, we'll let the doc know ahead of time, we'll
monitor the INR, and when it goes through the roof, we'll make sure the
doc does something about it -- change the dosing on the ??? and etc. --
we'll monitor your patient that's on a fixed dose of Digoxin or some
cramping for some reason so they can adjust the dose of the Dig\[oxin\].
We're most certainly thinking along those lines and are just opening a
conversation right now with the medical directors of our own group and
work out what their priorities are so we can put their suggestions in
our system -- so, 100%.

DAN SOSIN: Great. Thanks. Daniel?

DANIEL: Updated early. So, we have the ESP server on the inside of
medical practice's firewall, and sending as only the data necessary for
public health reporting. Now, in the case where codes are changed and
updated, new lab tests, new algorithms, or even new diseases coming in
that haven't been transmitted before, is there also an inbound
connection for updatability or would there have to be manual
intervention as well, so you get everything to the latest build?

MICHAEL KLOMPAS: So, the question concerns the maintenance of the ESP
system itself once it's been put into the community HIE, the medical
practice -- what happens if we update the case identification algorithms
or the analytical software? How are those updates going to travel back
to the ESP system? That's a question that, I think we thought a little
bit, but haven't had to contend with because we only have the one system
so far. I think the ideal would be to create a mechanism where you have
patches that automatically flow at a certain time of day or of the week
where the host ESP knows it has to open itself up to us so you can patch
it at the right time. I'm not an informatician, so I'm not going to blow
my ignorance any further on the question, but suffice to say that that's
very loose outline of the model that we have would be interim patches.

DAN SOSIN: Let me go back to the phone or to Envision. Do we have
additional questions from folks there? Any more in the room?

AUDIENCE MEMBER (NY): Hey Dan, can you hear?

DAN SOSIN: Yes, we can.

GREG SMITH (NY): This is Greg Smith in New York. I had a quick comment
on a question. I really appreciate the presentation. I think it gives a
view of the future of public health surveillance and I really liked the
suggestion that we should be working towards trying to make these kind
of standards part of HITSP for the long term, but I think the thing that
held us back is: we don't know what the -- we haven't really decided in
public health all of our own standards of what we want. My question is:
Can you give an idea of how many resources we would need? For instance,
if New York were to identify a participant -- how much effort, how much
work, how many resources would be involved if we wanted to try to join
you in bringing on a site practice or reel into this kind of a system?

MICHAEL KLOMPAS: So, thank you. The question concerning about the amount
of resources required in order to implement ESP in a new location and I
can tell you what I experienced during our initial implementation and
our ongoing work at North Adams. The major steps are as follows: First
of all, you need buy in of the host medical practice or community. So,
that's a lot of talking, that's a lot of rubbing shoulders, that's a lot
of diplomacy. In terms of actual resources, the informatics itself, the
system is not expensive. The ESP server itself is worth about \$5,000.
The most important thing about it is the programming effort in order to
export that information onto the ESP server itself the first time. And,
our experience in North Adams and Atrius health has been about \$10,000
worth of work for about 40 hours worth of programmer time. And, then
there's the work of code mapping and testing, which is a substantial
body of work. I think its initial implementation is probably about a
month to two months of worth of work. The ongoing maintenance is
actually pretty small; it's an hour or two per week. So, take those
numbers with a tremendous grain of salt. They're extrapolations based on
past experience of only two places that are very specific with a lot of
idiosyncrasies, but I think the message is that resources are needed,
but the resources that are needed are not enormous.

DAN SOSIN: Thank you. Scott?

SCOTT MCNABB: Michael, thank you very much on behalf of our division and
national center for coming down to give this great talk. And thanks to
you Dan for moderating. We're starting to see the data with
completeness, the improvement of completeness for reporting. I know that
you didn't show timeliness of ??? The potential for quality improvement
measurement of improvement of life quality of people that have been
diagnosed identified quickly of diseases and urgencies. Is that part of
your plan to incorporate those variables? And, would you comment on
timeliness.

MICHAEL KLOMPAS: Sure. So, Dr. McNabb has asked about the - for the some
of the metrics for the measurement of the impact of the ESP system, such
as timeliness and cost-saving and implications in terms of quality of
life for patients who might get diagnosed earlier or get treated earlier
than they otherwise would have. So, with regard to the timeliness
question -- I think we all -- reason a priori to a system like this --
ought to substantially increase timeliness. We haven't measured it
formally, thus far, for the following reasons: Number one is that we
have this case mapping interface roadblock inside of the system which
automatically adds days onto the measure, so we don't feel it'd actually
be an accurate reflection of what the system will be. The other
roadblock is -- I alluded to the fact that our system is, our host
medical practice is in some flux in bringing new practices on board at a
fairly rapid clip. Each time they bring a new practice on board, we've
had a dramatic influx of historical information going back to June of
2006, which is when the ESP project started. We go ahead and we report
all those cases and they're counted in our completeness estimates, but
there's a six month lag in 40 of those cases, which is not reflective of
the live performance of the system on a day to day basis. We'd love to
be able to report the information. We don't have it yet. What we are
doing is we've got to the point where we feel that Chlamydia and
gonorrhea, at least, that we are convinced of the accuracy of the
algorithm, and we don't feel a need for the human review step anymore.
And, we've just got the agreement of our medical practice to turn those
two conditions fully automatic. Once we do that, we'll go ahead and
we'll measure the timeliness factor on a prospective basis. So, I hope
to be able to tell you the answer to that question in the future. As to
the other questions regarding cost savings and impact on quality of
life, that's the million dollar question. Do these public health systems
actually make people healthier, happier, safer if they'd like. I think
we'd all want to get the point of being able to measure that and
demonstrate that -- I have no idea how to measure that. We also want to
be able to say that the system is more than simply a numbers game in
terms of reporting but can we actually improve peoples' lives. We
haven't thought of a way to do that in a reliable fashion as of yet. If
you have any ideas, we'd love to hear them. One idea was to say - track
the incidence of cases of pelvic inflammatory disease in a community
that has ESP versus not, but you can just imagine the size and duration
of a study that would need to be designed in order to capture that kind
of an outcome. On the table for consideration are a couple good ideas.

AUDIENCE MEMBER: Thank for your presentation. A key piece of public
health reporting is where and where that patient was. Across the system,
are you capturing just the facility location or the patient address of
where they say they lived, and is that information transferred and
propagated throughout the ESP system? Are there any spatial analytics
that are planned or thought of in the future? And, how are we
determining what a community is? Are they health service areas, practice
areas? And, how are we trying to get a population to ??? impacting the
ESP system as a whole?

MICHAEL KLOMPAS: So, a question about the geography associated with ESP.
In particular, what do we know about the location of the patient,
location of the clinician, and has it opened an opportunity for any kind
for spatial cluster detection? And, the answer is: we can have the
patients address as captured by the electronic medical records, we have
the clinician or facility address as captured by the record, and that is
reported unto public health. It does indeed open up the opportunity for
spatial cluster detection. We have a number -- some expertise -- in our
department on that. We have done surveillance work. We have people using
space time scan statistics, which actually was developed in our
department. So, we certainly would like to go that direction. I think
the questions going to be ask, "What the best disease model in which to
try to do that work? What's going to be the most interesting?" The
proposal on the table right now adds more cluster detection. And, we
have early tentative discussions with the department of public health as
to whether they want to do that and with our statisticians. Potential
absolutely exists. We've done nothing concrete thus far on that. As for
the population capture area, it's the people associated with the medical
practice and whatever their geography might be relative to the location
of the medical practice. I don't have a clear definition on that.

DAN SOSIN: Thanks. We have another question that came in electronically
that I think is really a request for a link or where more information
can be received regarding ESP software you had mentioned as being
publically available. Is there a place where folks can go to gain access
to the software and technical information that would help them with
reviewing it and possibly implementing it?

MICHAEL KLOMPAS: Yes. So, there is a website which has the code
available and some of the documentation. It's ESPhealth.org. Let me warn
you though -- the website needs a little bit of maintenance work to
bring it up to date. I'm hoping it'll be in better form some time in the
early part of the year. You can actually go there right now and get the
ESP code, although, the code that's available on the website is a couple
months out of date. So, that's what needs to be put up to date. In
addition, in terms of further discussion of the program, there's a
published manuscript on the program in the journal *Advance Disease
Surveillance* in volume 3 issue 3 which you can take a look at. Or, if
you contact me, I'll be happy to forward it to you directly. And then,
we're just in the process right now of generating two manuscripts, one
targeted towards JAMIA describing the informatics and the architecture
of the system and one targeted towards the MMWR describing some of the
clinical performances of the system. Those are on the cusp of
submission.

DAN SOSIN: If I can be so bold as to take the last question. You
referred to the case management as a roadblock - it's interesting to me
that there are clearly going to be some conditions that will require
some case management, that will not be so automated, and that in some
ways many of the new approaches to surveillance we've approached where
we have lost of specificity for increased sensitivity. That, we're
looking at these systems in some way to flag public health to get more
engaged. Can you speak a little bit as to how you would set the bar for,
and what you anticipate in terms of your overall system being someday
fully automated versus some aspects requiring some, you know, interface
that's going to require public health and medical judgment to act on?

MICHAEL KLOMPAS: Thank you. The question -- we've approached this
problem by seeking guidance again from our state health department
epidemiologists as to what is important to them. Their bias also leans
towards sensitivity, towards completeness of case capture at the cost of
specificity, and the algorithms have been designed accordingly. What I
think is very informative is the slide that I showed on what the false
positives are to see where the system goes wrong. My argument was that
those are all cases that a clinician under current practice would report
anyway and that health problem assessment investigation would reveal as
false positives. And, to my mind, if we can continue to show that the
algorithms are highly sensitive and have real specificity, because the
false positives themselves are not random events but are reasonable
things to report that would allow for the opportunity for the fully
automated system. I think the challenge is that when you do want to
throw in a case management or physician review step, you have to work
out on a broad scale, who's that person going to be? So, if you have a
large practice where there is a dedicated infection control staff, it's
okay, but then the infection control practitioner can do that work. If
you're a doc in solo practice or you're a community of docs in small
practices without some sense of a centralized infrastructure, it's
totally unclear as to who that person should be. Should you send that
information to the physicians themselves to review? It certainly is a
potential model; although, the feedback we've gotten from people is that
docs don't want to do this. Is that true or not? We can argue that docs
are currently pulling out pieces of paper and so if anything this should
be easier for them. My response is that docs are not pulling out pieces
of paper, that's our problem -- the good docs are, but not everyone. So,
maybe some sort of intermediate model where they do offer it to the
doctor to review - if they don't review it after a certain period of
time, then you automatically message it with the associated message of,
"Not reviewed by doctor," -- that could be a potential answer to that
question. But, I don't know. I think the take home message of it here --
this is novel work. There's a lot of unanswered questions, I think that
are fertile for thoughtful minds to approach. To the extent that if you
collaborate or get guidance from the experts, which I think is CDC and
state health departments and CSTE, on how to do this best, I think is in
all of our interests.

DAN SOSIN: Thank you very much. Let me offer my thanks to Michael for
coming, spending time with us, and sharing this project; to Harvard for
the novel and animated work that you do; to NCPHI for sponsoring these
Centers of Excellence. I'll turn it over to Jennifer to close out the
session

JENNIFER WARD: Okay. I would like to echo the thanks to both our speaker
and moderator for this excellent session, and, to thank the participants
to contributing to our session today. I hope it was beneficial and that
it will help spark further collaboration to move this work forward. Our
next seminar series will be January 9^th^. The topic will be: An
overview of national health information technology standards and how
they apply to public health practice. Standards are an essential
building block of moving this work forward and it's kind of a 101 for
public health practitioners, what they need to know about the landscape
of standards. It will be presented by Dr. Steve Steindell from the
National Center for Public Health Informatics. We invite your
suggestions for future topics that would be helpful in advancing
integrated surveillance efforts. You may do this via our email at
<integratedsurveillance@cdc.gov>, and I'll put that up in a moment. You
may also join the integrated surveillance community of practice online
workspace through the seminar series website or by emailing us. On
there, you can post suggestions for future topics, further discuss the
topic that was presented today, and collaborate online with others who
are working in the area of integrated surveillance. The online workspace
can also serve as a repository for some work that you are doing in this
area, if there are practices that you would like to share. And, we thank
you again for your participation and we look forward to seeing you again
at our next session.