American Health Information Community

Quality Workgroup Meeting #9

Friday, June 22, 2007

**Disclaimer**

*The views expressed in written conference materials or publications and
by speakers and moderators at HHS-sponsored conferences do not
necessarily reflect the official policies of HHS; nor does mention of
trade names, commercial practices, or organizations imply endorsement by
the U.S. Government.*

**[\>\> Judy Sparrow:]{.underline}**

Welcome everyone to the 9th meeting of the Quality Workgroup. Just a
reminder, this is a FACA committee and we\'re operating within the
public sphere. Also, the materials and agenda are available on the ONC
Website for this meeting as well as for all of the other Workgroup
meetings. And there will be an opportunity at the end of the meeting for
the public to make comment.

Let me just remind Workgroup members to please identify yourself before
you do speak at the meeting. And also, let\'s begin with introducing the
members who are here at ONC and then we\'ll \-- Matt, you can introduce
the people on the telephone. And we have \--

**[\>\> Michelle Murray:]{.underline}**

Michelle Murray, ONC.

[**\>\> Carolyn Clancy:**]{.underline}

Carolyn Clancy, AHRQ.

**[\>\> Judy Sparrow:]{.underline}**

Okay, and \--

**[\>\> Cinyon Reed:]{.underline}**

Cinyon Reed, ONC, BearingPoint.

**[\>\> Judy Sparrow:]{.underline}**

And Matt, if you could let us know who is on the telephone.

[**\>\> Matt McCoy:**]{.underline}

Yes. From Boeing Company we have Rick Stephens, and Pam French is on the
phone as well. Jerry Osheroff from Thomson Healthcare. Atif Zafar from
Indiana University is here as a designee for William Tierney today.
Kimberly Suggs from HCA is on as a designee for Susan Postal. Reed
Tuckson from United Health Group, and Jonathan Teich from Harvard
University. Is there anybody who I have missed?

**[\>\> Judy Sparrow:]{.underline}**

Great, and thank you. Let me just also remind those on the phone to
please keep your telephone on mute when you\'re not speaking. And with
that I\'ll turn it over to the co-chairs, Dr. Clancy and Mr. Stephens.

[**\>\> Carolyn Clancy:**]{.underline}

Good afternoon, everyone. A lovely day here in Washington, and I hope
that wherever you are, you are having the same kind of beautiful
weather. It's a little bit rare for us here in the summer. I think our
first official call before sort of setting the stage for our discussions
and presentations today, is to ask for someone to accept the minutes.
Does anyone want to make a motion to that effect or raise any issues?

**[\>\> Rick Stephens:]{.underline}**

This is Rick Stephens. No issues. So moved.

**[\>\>]{.underline}**

Second.

[**\>\> Carolyn Clancy:**]{.underline}

Thank you. We will declare the minutes for our meeting summary for May
3rd accepted.

What I wanted to do was just launch a discussion, and if you can put up
the slide of the diagram, that would be really terrific. Workgroup
members, you should have gotten a copy of this with the materials that
were sent out two days ago. And it will be appearing just in a moment.

This is the diagram that\'s labeled draft, use of HIT in quality
measurement. So just to remind you where we have been, we started off
with a very specific charge that called for us to identify a core set of
measures derived from those recommended by AQA and HQA, and
systematically identify the next steps for identifying the data elements
required to build those measures into electronic health records, and
then hand that set of data elements over to HITSP, the HIT Standards
Panel and ultimately to the Certification Commission for Health IT. So
that was kind of the sort of easy stuff. Not so easy, as you\'ll hear
from Paul Tang a little this afternoon. But as we\'ve discussed in
recent meetings, while we\'re continuing to make progress on that
specific charge, we\'re also moving ahead with the broad charge for this
Workgroup. And we have framed this as a problem statement. What health
information technology requirements, competencies, and skill sets are
needed to collect, aggregate, and analyze longitudinal measures that
look at patient care quality across entire episodes of care? Please
notice how that rolls right off my tongue. It sounds great, doesn\'t it?

What we have right now is a growing amount of quality information
available for the public, as well as for clinicians and healthcare
organizations to use to improve, that actually is more like little
snapshots. We have not yet built in continuity, although we are starting
to take on some of the tough work in the quality world of harmonizing
the measures, so at least the measure specifications are the same. But
we don\'t yet have information that gives patients and others a clear
view of what happens in an episode of care. And our charge in this
Workgroup isn\'t to redesign the quality enterprise, but it\'s to
anticipate where the quality enterprise is heading, and to make sure
that the health IT components are there in place to support that
enterprise.

So if you just take a look at this diagram, you\'ll see at the top that
there is two big boxes, and this is kind of rough and by the way, any
feedback would be immensely appreciated. One box talks about national
goals, and the other talks about an evidence base, because what we\'re
trying to get to is as much consensus as possible about valid, reliable
measures. First step of course is to develop and maintain measures which
are then given to the NQF for endorsement for use for public reporting
and other purposes. Those measures are then implemented and ultimately
we try to see what kind of impact this has had, and some of that
implementation activity also feeds into ongoing maintenance of measures
and so forth.

Now, for that little cycle to proceed as seamlessly as possible of
course where we want to get to is that collecting the requisite data
isn\'t a burden, but you heard it here, actually adds value to the daily
work of providing care. I think it\'s fair to say that right now we are
so far from that, I don\'t actually have language to describe it. But
certainly speaking to physicians and other clinicians, at best they
think this is a burden that might some day might payoff. And at worst
they have much stronger language to share but we won\'t share that here.
Now, in order to get to a place where collecting data is indeed
seamless, we have to get our heads around a couple of issues.

Now, if you look at the bottom box, not quite at the bottom, that says
public reporting, accreditation, and quality improvement, those boxes
are established and built out with fine work from the Joint Commission,
from CMS payers, in a few short months value exchanges, and so forth.
The National Committee on Quality Assurance and so forth. What we\'re
missing, though, as we slowly migrate to a world where quality
measurement, data for quality measurement will be coming from multiple
sources, is some sense of data stewardship. And who collects that data,
who sets the standards, that make it very, very clear how the
measurement specifications are translated into the requirements for the
data to be collected.

Now, just getting your head around data coming from multiple sources, I
think, is actually a pretty tall leap. I think most people think of
quality assessment as sort of a thoughtful person with very careful
rules and specifications, walking through a patient chart. And indeed
today that is mostly how it takes place. However, as we\'ve discussed in
the, previously, what we\'re trying to do is take advantage of other
data streams so in fact this is much easier to collect and our goal is
not just to get smarter and be able to drive faster through the rearview
mirror but actually to get to a place where the information is provided
to clinicians at the point in time when they need to make the right
decision. So in that way, our goal is not just about better, faster
measurement, and measurement implementation. It\'s about better and
faster improvement as well.

So down at the bottom, and let me just put in a footnote here that right
now the AQA Alliance has a, what\'s called an RFI in Federal land, or
request for information, out on what are the requisite characteristics
of a data steward. I will say this noting that there is not a clear
consensus right now in terms of how we\'re going to do this. Clearly, in
some areas we\'re going to take advantage as much as possible of
entities that are set up to exchange health information like RHIOs,
health information exchanges and so forth. The hospitals right now use a
model where patient-level data is sent to a central repository. In the
AQA, there\'s been more discussion about the construct of a data steward
that sets standards for how measure specifications are translated into
the requisite data streams, how that information is collected, and that
there\'s more likely going to be multiple aggregators. This is not
legislated, it is not set in stone, and one of the reasons we put out an
RFI was simply to be able to get as much input as possible on that. So
I\'m just making a footnote of that because it\'s likely to come back to
this Workgroup for fuller discussion and to let you know that you still
have an opportunity to weigh in. You can find that at AQA alliance dot
org. And I\'m betting it\'s on the AHRQ Website, AHRQ dot gov, but I
can\'t really point you to the precise place so I might try AQA Alliance
first. The RFI was actually released by AHRQ.

So down at the very bottom of this diagram, then, you see two boxes that
talk about Health IT, and then about policy, and next to those boxes are
the groups that make \-- that contribute to making that policy.

Again, these are all very careful pieces of the puzzle at something of a
macro level. Let me also just say that we will be sharing with members
of the Workgroup prior to our next meeting a somewhat more detailed
diagram, that we\'re still working on and didn\'t think was quite ready
for you, that actually describes and tries to depict how the data flow.
Because again, I think getting your head around, particularly if you
come from the clinical care side, the fact that data will be gathered
from multiple sources, and that we need rules and clear policies for how
to do that effectively and accurately and reliably, is going to be a big
piece.

So I just wanted to set a context for where we are today. We\'re making
progress on the specific charge, and you\'re going to hear a report from
a meeting that was held May 31st at the National Quality Forum, to take
the next big step forward in that specific charge. I was not able to
attend the meeting, but I no longer allow people too much time to tell
me all about it because I heard it was spectacular and they\'ve all
needed to tell me at great length what a fabulous job Paul Tang, Helen
Burstin, and others from the Quality Forum did with this work.

But we\'re also going to be hearing before that an overview of what is
more meant by episodes of care. This builds on and describes some work
that\'s current currently ongoing at the National Quality Forum and then
we\'ll hear from presenters that talk about longitudinal data strategies
in action.

Now, if all of you are beginning to think that this sounds like a whole
bunch of snapshots but just arrayed in a different way than we currently
array snapshots of quality of care, I think you\'d be right. If there\'s
one person who reminds us, every time we talk to him, that we have to
begin to think about systems and how these pieces fit together in a way
that\'s functional and ultimately helps us get to the place where
quality assessment is value added to the whole enterprise, it\'s Rick
Stephens. So with that I\'m going to hand it off to him.

**[\>\> Rick Stephens:]{.underline}**

Carolyn, thanks very much and that was a great discussion about what we
see on the screen right now on the use of HIT in quality measurements.
This whole notion we know that we are a system, our challenge is we just
use different words or language, allow different incentives or
motivations to drive our behavior, yet we all want a common outcome,
great quality, and, in terms of healthcare, at a cost and good
measurements so we can all have the right discussions aimed at aligning
patients, payers, vendors, employers, providers, hospitals, the
government, the associations, and everyone who is actually involved in
this system.

And to me, what this begins to do from a quality perspective is to begin
to show the relationships that will allow us to help, you know, to go,
as Carolyn talked about, going the next level down, what is the real
language, what motivates us in each of these acts? I'll use under the
evidence base, in terms of endorse measures, what motivates us to
endorse the particular measures so we can actually all get the same
data, same view, the same perspective and begin to drive change, and
that\'s what the systems model is all about.

So I want to echo Carolyn\'s comment as we all look at this, we ought to
think about it, what does it mean from our perspective in each of those
boxes. Not with a lot of specific detail but just start thinking about
that\'s where we need to come together because as we start flowing out
this in more specificity, we\'re really going to need that so we can
have the discussion about getting alignment, we\'re bringing some system
tools to the table. So lots of work to do, but Carolyn, thanks very
much, and back to you.

**[\>\> Charlene Underwood:]{.underline}**

This is Charlene Underwood. Which presentation am I looking at? I can\'t
get on the Webcast.

**[\>\> Carolyn Clancy:]{.underline}**

Charlene, this would have been sent to you. This is the diagram entitled
draft, use of health IT, HIT, in quality measurements.

**[\>\> Charlene Underwood:]{.underline}**

Yep. Got it. Okay.

**[\>\> Reed Tuckson:]{.underline}**

Carolyn, are you taking questions at this point?

[**\>\> Carolyn Clancy:**]{.underline}

Reed, you read my mind. Take it away.

**[\>\> Reed Tuckson:]{.underline}**

No, first of all, again, let me echo, I think this is a very useful, the
presentation was terrific and the chart is great. Would you just say one
more word about the relationship between AQA and its efforts now with
all of its data aggregation initiatives and data aggregation pilots and
how you would sort of see the data aggregation, data stewardship on this
chart and the RFI sort of synching up, how do they connect?

[**\>\> Carolyn Clancy:**]{.underline}

Sure. So right now there are six pilots, initiatives, sometimes known as
BQI, for Better Quality Information. Sometimes known as BQIMB, Better
Quality Information for Medicare Beneficiaries, the B would be silent,
or BQIMs. And that is because the funding source is actually the \-- is
CMS. And by legal requirements for the QIO program which is supporting
these six initiatives, the focus has to be very, very explicitly on
information that\'s useful to Medicare beneficiaries. So that imposes
some limitations on which measures can be used.

Right now they are doing something fairly unprecedented. They are
working under a subcontract to one quality improvement organization, and
are combining data from Medicare with data from multiple private sources
to come up with a single report on physician performance. And they will
be reporting on that either later in \'07 or early 2008. Now, that
sounds like a lot of fun if you like to play with data. You can\'t even
begin to think about it for more than a few minutes, though, without
realizing that there\'s going to be a lot of problems.

At our last meeting we heard at least a couple of the presenters talk
about how they deal with such, with such challenges as what happens if I
have the same information from two or three different data sources but
it\'s not precisely identical. So the woman from Blue Cross Blue Shield
in Illinois said well, we have an internal rule that says we give the
most credit to the doctor when we\'re looking at quality. Now, I\'m not
saying that\'s the right rule, but that\'s their rule. When you start
getting into issues of coding and working with claims data, it turns out
that virtually every insurer has a lot of internal rules, and I\'ll bet
some of them are even written down. But when you try to aggregate and
combine data from those multiple sources, it gets pretty tricky. So
we\'re going to be learning a lot from those initiatives, and then from
the value exchanges when they\'re launched, about how to do this well.

Another way to say this, Reed, is just because you have a lot of data in
a computer doesn\'t mean you can actually analyze it well or reliably or
fairly. So the RFI will actually give us some very solid conceptual
qualitative information about what are we thinking of the
characteristics of this kind of identity, who would this organization
need to be aligned with to set the standards that we talked about
earlier. The pilots in value exchanges will give us some very clear
empirical experience about how detailed and precise are the standards
that that entity would have to set. Does that help?

**[\>\> Reed Tuckson:]{.underline}**

Terrific, thank you.

[**\>\> Carolyn Clancy:**]{.underline}

Other questions or comments on this diagram?

**[\>\>]{.underline}**

I think it\'s great, to try to get, start to get the pictures. Like what
we wrote in the vision but it starts to visualize it. I think it\'s
great.

**[\>\> Jonathan Teich:]{.underline}**

Yeah, this is Jonathan, I think it\'s very nice. It reminds me eerily of
the Krebs cycle, so I'm not sure if that's important to our development.

\[laughter\]

[**\>\> Carolyn Clancy:**]{.underline}

I\'m sorry to say I share that impression. Do we still have the Krebs
cycle?

**[\>\> Jonathan Teich:]{.underline}**

I think our bodies do. The stars, the enabler opportunities, the
beautiful eight-pointed stars, is this trying to say all of those stars
represent places where HIT and policy can be applied?

[**\>\> Carolyn Clancy:**]{.underline}

Yes. And thank you. That\'s one of those things that tended to jump off
at me from the screen a little bit easier when I was looking at it
earlier but I\'m literally looking at a black and white one here.

**[\>\> Jonathan Teich:]{.underline}**

Yeah, it\'s \-- I mean, whatever we can do to make that as clear as
possible is great. It looks very good, I think it\'s very nice
organization.

[**\>\> Carolyn Clancy:**]{.underline}

Thank you. I wish I could take credit for it. I\'m just a consumer here.

**[\>\> Jerry Osheroff:]{.underline}**

This is Jerry Osheroff. The IHE quality domain, Integrating the
Healthcare Enterprise, they have a significant activity cooking in this
area and just on \-- I\'m not seeing them listed here, so that\'s
probably something that might be worth adding into this diagram and
making sure that all of those synergies there are cultivated as we think
about pulling this together going forward.

[**\>\> Carolyn Clancy:**]{.underline}

You\'re right about their activity and actually if you look at the
bottom next to the two bottom boxes \--

**[\>\> Jerry Osheroff:]{.underline}**

Yep, there it is, okay.

[**\>\> Carolyn Clancy:**]{.underline}

Let me say I had a meeting with some folks who were lauding their
efforts.

**[\>\>]{.underline}**

That\'s good.

[**\>\> Carolyn Clancy:**]{.underline}

Any other comments? Well, good. Let me just also say that at our next
meeting our expectation is that we are going to be getting into these
systems issues much more clearly. I said to Rick the other day, because
he\'s from Boeing, that in a funny way, I thought our job was much
harder than building airplanes. I was more respectful than that, of
course, but I said basically I envision they got to make a plane by
themselves. But our job was \-- but he pointed out quite accurately that
yes, they get to make a plane by themselves but once you use the plane,
then you have to intersect with air traffic control, all kinds of
airport rules, weather, and all the stuff that makes flying so much fun
these days. They\'re going to be actually lending some expertise in
helping us think through some of the systems issues. So we\'ve noted
throughout our discussions that the work flow issues around collecting
data and so forth are pretty paramount.

If there are no other comments, then let\'s turn to \-- is Mark Rattray
on the phone?

**[\>\> Mark Rattray:]{.underline}**

Yes, I am.

[**\>\> Carolyn Clancy:**]{.underline}

Terrific. Can I ask you to give your presentation describing what\'s
been happening at NQF?

**[\>\> Mark Rattray:]{.underline}**

Sure. NQF has established a workgroup that addressing episodes of care
from a priorities and goal setting standpoint. We have a fairly clear
pilot project scope that\'s in slide 1. And that is to develop a
comprehensive measurement framework for chronic care episodes. We looked
at a subset of priority conditions to begin to flesh out a framework and
concept as to how we might measure, report, and improve care across a
specific condition type of episode of illness. Also, to take a good look
at what is the current state of what might be broadly termed efficiency
within these priority conditions. Looking hard for what are the measures
that are currently available, what are the data streams and data capture
requirements. And then take those specific target areas and try to drive
national performance goals, measurable targets for the initial two
priority conditions over the next three to five years. And try to
develop a more comprehensive view of what episodes of care would look
like in a future vision research agenda in terms of evolving a
measurement and monitoring system that\'s more comprehensive than our
previous approaches to episodes of care.

On slide 2, you see the members of the committee and on slide 3 is just
a very brief update as to what\'s going on at this point. We are still
in the workgroup mode and have yet to report back up to the NQF steering
committee and so we don\'t have a lot of specifics to put in front of
you today. But that we have meeting, the workgroup is meeting next week
in Washington, D.C. and we have a further workshop scheduled in August
to begin to bring even further representation into the discussion with
respect to content.

We have spoken to the fact that we\'ve chosen acute myocardial
infarction and low back pain as our initial priority conditions. At this
point most of the work has been around acute myocardial infarction. Just
to let you know that we are taking a comprehensive view what have an
episode of care would look like for this condition, going back in time
to risk factors, to prevention, moving forward to secondary prevention,
and looking at broad indicators of what might be termed efficiency, but
also other elements of quality and IOM aims that would apply to these
particular conditions as we build out episodes of care.

So, rather general depiction of what\'s going on now. A lot of work
behind the scenes, especially in trying to develop a reusable framework
as we move beyond this, the initial condition of AMI into low back pain
and to further priority conditions as would be warranted by the
prevalence of conditions as well as the expenditures related to
conditions.

That is all I have at this point.

**[\>\> Carolyn Clancy:]{.underline}**

Pardon me?

**[\>\> Michelle Murray:]{.underline}**

Do you want to start at the beginning of your other presentation,
overview of the episodes of care?

**[\>\> Mark Rattray:]{.underline}**

Sure. Just to briefly review for the group episode of care from the
construction standpoint. What\'s meant by resource use measurement
variation, what are some of the known implementation issues, some
research that has been occurring in the very recent past, some remaining
questions around the current use of episodes of care, and some potential
future episode of care approaches.

The slide, on slide 2, where you see a graph of showing a calendar year,
for representative purposes, we\'ve chosen kind of an idealized episode
of care where the concept of episode of care really relates to trying to
capture, longitudinally, resource deployment to treat a particular
condition for a particular patient. This is a break-away from
informatics approach historically used where you tend to measure various
elements, types, and quantities of services, and the prices of those
services, as opposed \-- and look at them at specific points or
quantities over, across patients and over a broad number of conditions.
And with episodes of care, the goal is to try to create a currency which
we can talk about condition- and patient-specific treatment.

In this particular example we see a patient who has sought physician
care in the month of February, underwent some pharmacological treatment
that intensified through March and April, did not improve, underwent
sophisticated diagnostic imaging and subsequent, a facility-based
procedure with associated physician procedure costs and additional
pharmacy costs. As that particular episode wanes, you see the fall-off
in use of services at the end of the year, should be actually that the
episode resolves in November with no further treatment. But this is kind
of the comprehensive longitudinal view that episodes of care concepts
look at. And again trying to identify specific conditions at the patient
level to be able to look at relative variation in resource use to treat
conditions.

On slide 3, what is typically done is you begin to then drill down
further a bucket cost into categories for subsequent drill-down and to
be able to identify where the variation is actually \-- where the
variation is actually occurring so you can provide feedback to the
physicians such that they can gain insight as to how their practice
model, how their practice behavior, compares to other physicians with
similar types of episodes of care.

And in slide 4, you see just a graphical depiction of a comparison
between a physician\'s peer average episode cost, or I\'m sorry,
physician\'s episode average cost, compared to peers, and what this
does, then, is allows further drilldown into the actual underlying
constituent claims, services delivered so that a physician may gain
insight into the specific areas of variation compared to peer
performance, both positive and negative for quality improvement,
purposes.

On slide 5, we see a general depiction of how the processes work when
analysts work, analysts approach episodes of care. Generally there's
three categorical steps involved, easiest to represent is the initial
data acquisition and preparation phase, where claims extract or
administrative data extract is obtained for analysis. The data is
prepared for processing within the grouper. That moves on to a phase
where there, within the grouper, groupers themselves, there are specific
settings that the measuring entity would apply and then processing the
administrative datasets through the grouper, to generate grouper summary
data. In the analytics and reporting phase you\'re really talking about
non-grouper-specific functionality and really more about the orientation
of the measuring entity and the intentions of the measurement as far as
the nature and types of reports that would be generated to drive quality
improvement, to drive transparency, to drive further drill-down in
understanding variation.

There are known issues for each of these steps and I\'ve listed some of
the key ones associated with each of the steps and clearly continue to
have data aggregation challenges, data from disparate sources, trying to
not only prepare the data for processing through the grouper but also
trying to standardize data from multiple sources so that the data field
mean the same thing as the data enters the grouper.

In addition, as you might imagine, when we look at episodes of care,
we\'re looking at trying to track the entirety of resource use to treat
a condition so that we can have good comparable, or comparative data at
the other end. And in many instances we are challenged, we work with
groupers to try to get complete datasets of the services delivered. And
most problematic historically has been obtaining mental health data,
pharmacy data, and there are often challenges, especially at the data
aggregation level around identifying providers as the same providers
across datasets. And then one needs to include eligibility files to
apply the various algorithms within the groupers appropriately.

We know that there are multiple entities, including AQA and NQF and CQA,
that are addressing known issues around attribution methods, sample
sizes, how to handle outliers, how to determine a peer comparison group.
Some progress has been made there, but there clearly is not a defined
standard at this point in time, which continues to create a bit of angst
as physicians are measured against different standards and different
methods.

And finally, we know that while their grouper vendors have done internal
studies as they\'ve developed these groupers, there have not been a
high, a large number of clinical validation studies as to whether or not
the underlying clinical logic will pass intense physician scrutiny as
one drills deeply down into how all the claims are parsed across various
episodes and how those episodes are subsequently attributed to
physicians. And what has heated up the landscape quite a bit is the move
towards consumer transparency and using grouper tools to generate
performance information around cost of care episodic cost efficiency.
There are increasing number of studies, which is encouraging, that are
looking at groupers and physician resource use, and some of the
challenges in the fact that we are dealing with a very proprietary rule
with two dominant groupers on the market. MedPAC has begun to tackle
these issues starting initially in June of 2006, with a five percent
sample across all MSAs, looking at the two groupers, identifying the
differences and similarities. And there are definitely differences and
similarities and what MedPAC did in 2006, in their 2006 report is to
begin to characterize some of those differences and similarities.

There was a very interesting finding that came out in the June 2006
report which was counterintuitive initially, where we saw a finding
where in Miami there are high per capita costs for coronary artery
disease versus Minneapolis and yet per episode costs in Miami were
substantially less expensive than in Minneapolis. And in further
drill-down one found that in \-- or what was found was that in Miami
there were many more episodes being created, which had a dilutive
effect, but there was also a significant difference in the type of
episodes that were being created, being predominantly diagnostic
episodes in the Miami area, and predominantly \-- and heavier weighting
of therapeutic or treatment episodes within, with a greater degree of
in-patient costs associated in Minneapolis. So the initial worry around
this was that, do episodes of care really reflect true efficiency, or
are we seeing other factors operative? And it\'s pointed out to us that
we need to look when we roll up episodes of care into categories such as
coronary artery disease, we begin to blunt some of the true underlying
variation, which was, in this case, a greater number of episodes in
Miami, a dilutive effect, and a much bigger, much larger distribution on
the diagnostic episodes and less on the therapeutic episodes. And one
can, separate discussion as to why that happens in Miami versus
elsewhere. Many are familiar with different practice patterns that exist
in Miami, and also different behavior of Medicare beneficiaries that
seems to occur in the Miami area.

There are exciting things going on in other areas. GAO recently released
a report that didn\'t look at episode of care variation but did look at
the per capita variation and how that fit into physician variation and
practice patterns. And in slide 7 one sees a distribution of the CBSAs,
the Core Base Statistical Areas that GAO looked at. What they
hypothesized was that in any given market one would only expect
approximately a one percent outlier physician group, and in all of the
market areas they found high percentage in excess of that from
approximately .13 or 1.3 percent, up to .28 percent in Miami. And if you
look at slides 8 and 9, you can see the distribution of the outlier
behavior that occurred in Des Moines, Iowa, which had a very tight
distribution with the outlier threshold being that point at which
physicians are beyond the one percent outlier threshold for that
geographic area.

So you see a much higher outlier threshold in Miami in slide 9, and you
see a much broader tail of physicians that have practices with
significantly more expensive beneficiaries, as expected. And for \-- one
finds that for those that, for those physicians that were out in outlier
status, beneficiaries were 15 percent more likely to have been
hospitalized, 57 percent to have been hospitalized multiple times, 51
percent more likely to use home health and 10 percent less likely to
have been admitted to a skilled nursing facility.

So questions remain. The efforts to aggregate data are still very early.
And one of the questions is are there compromises that are required to
bring data together for aggregation purposes, actually provide the
increased and more reliable measurement that we\'re seeking through a
greater sample size, where will it introduce compromises along the way?
Will the underlying clinical and numerical logic that occurs in the
process of episode measurement withstand close provider scrutiny,
especially those physicians that fall into an outlier status? How will
we best translate this variation for consumers? Will an episode of care
base clinical improvement or transparency effort actually succeed in
reducing adverse episode resource use variation, and of course the
desire is for that occur while improving quality, and which requires a
whole set of discussions around close linkage of quality to the episode
of care measurement to make sure that we\'re looking at both aspects of
quality and efficiency.

Currently the focus, on slide 11, the focus is on condition-specific
aggregation of resource use for physician performance measurement. And
again predominantly using administrative and \-- administrative data
usually claims data. And in the future and we\'re seeing some hints of
this as we begin to work with the NQF efficiency workgroup, is not only
looking at resource use, but beginning to contemplate such things as
optimal resource types and timing of those resources. The phrase that
we\'ve heard so often before of the right care at the right time, and I
think at the right, in the right sequence may also be a future
consideration. We have condition-specific episode stages that are being
conceptualized now in pre-diagnosis prevention, acute and, if
applicable, chronic and/or resolution phases. We clearly, as we build
out the episode of care concept further, we need to develop appropriate
corollary measures, quality measures for each condition phase, and we
definitely want to move towards a broader definition of episodes to
include the remainder of the IOM aim to the extent possible. And we are
dependent upon intellectual capital investments to create a broad but, a
broad framework that allows a more comprehensive view of illness and
episode of care. But also to begin to look at how we can build in
improved data capture, improved leveraging of electronic health data,
especially through electronic medical records. We have a lot to learn
around data aggregation and the creation of large warehouses and we need
to, as we develop these new constructs, to develop new episode analytic
capabilities to provide the users with the appropriate information that
is needed to move forward.

At this point I\'ll stop and answer any questions that might be there.

[**\>\> Carolyn Clancy:**]{.underline}

Yes, well, let me just say that was terrific. And thank you very much,
and I don\'t think I\'ve had quite as clear a view or presentation
certainly of how what folks who are doing focused on groupers meshes and
how much the NQF approach actually enhances and expands and complements
that. Let me say while you\'ve been presenting, we\'ve been joined by
Margaret VanAmringe and Janet Corrigan. Janet, I don\'t know how much
you\'ve heard of Paul\'s presentation, I was focused on what he was
saying and you were so graceful sliding in. And I didn\'t know if you
wanted to add anything from NQF.

**[\>\> Janet Corrigan:]{.underline}**

Not really. I think it\'s been covered very well. Terrific presentation,
Mark, thank you.

**[\>\> Mark Rattray:]{.underline}**

Thank you.

[**\>\> Carolyn Clancy:**]{.underline}

Questions or comments from the Workgroup members?

**[\>\> Charlene Underwood:]{.underline}**

Mark, this is probably obvious. This is Charlene Underwood. You can tell
you live and breathe this. It was a great presentation. But as you look
at the implications, in your Workgroup, do you think of the implications
of trying to define this, from those of us like vendors who are trying
to do billing for our customers? The reason I ask is we must spend
hundreds of hours here trying to figure out what an episode is so that
we can link clinical episode to cost and quality too. It's nowhere near
the level of sophistication you\'re talking about. But is any of that
embedded in the work that you do?

**[\>\> Mark Rattray:]{.underline}**

I guess I didn\'t quite understand the question.

**[\>\> Charlene Underwood:]{.underline}**

As you think about how this will ultimately be captured from the systems
that are, you know, we\'ve got these disparate systems out there.

**[\>\> Mark Rattray:]{.underline}**

Yes.

**[\>\> Charlene Underwood:]{.underline}**

So like right now, you know, there\'s no way for us to link episodes
together, plus many customers have co-morbidity, so you don\'t know what
to link, what episode of care to link it to anyway. Do you think of any
of the mapping or how that information can be gathered at all?

**[\>\> Mark Rattray:]{.underline}**

Well, certainly the NQF workgroup on this is clearly looking at the fact
that we have serious challenges of bringing in the right data into the
episode of care construct. So that is one of the big challenges. I think
that as a result this work is not going to happen overnight. However, it
is great to begin to keep the episode of care concept at front of mind,
as we tackle many of the current frustrations we have with electronic
data capture with digitalization of healthcare that actually occurs. So
without \-- I don\'t think I can be more specific than we all recognize
the challenge and as we begin to create the future generations of
episode concepts, that they be built hand-in-hand with the availability
and capability to capture data electronically. And there\'s obviously no
way that we could do this in a manual fashion and be able to widely
deploy these tools.

**[\>\> Janet Corrigan:]{.underline}**

Carolyn?

[**\>\> Carolyn Clancy:**]{.underline}

Yes.

**[\>\> Janet Corrigan:]{.underline}**

This is Janet Corrigan. I think in some ways the project you\'re going
to hear about a little bit later on the agenda from Paul Tang, a small
NQF project that took a look at many of the measures that are currently
being used by AQA and HQA and CMS, and thinking through what the data
implications are for those from the EHR. In many ways that\'s the first
step to answering the question. And I think sort of the next project
that we need to do on the horizon is to take the leap to episodes of
care and the full set of measures that will be encompassed and likely to
come out of the Mark\'s workgroup and others in this project and then
re-look at what the implications are for data capture in the electronic
health record. The project that was just completed I think does, has
made some progress in thinking about how best to identify the
denominator populations for these different measures, whether it\'s
individuals that are going to be looked at in an episode of care, or
individuals that you need for the nominator of a discrete quality
measure.

[**\>\> Carolyn Clancy:**]{.underline}

Margaret.

**[\>\> Margaret VanAmringe:]{.underline}**

This is Margaret VanAmringe. And Mark, it was a great presentation. You
were very clear. I was kind of interested in having you talk a little
bit more about the first analytic step where you\'re actually trying to
find the same patient across different sets of claims data. Because I\'m
assuming one has to look across commercial claims data, Medicare claims
data and make sure that you\'ve got the same patient before you can
group into the episode. And what kind of challenges have you been
finding there?

**[\>\> Mark Rattray:]{.underline}**

Well, most of the work that\'s been done historically has been limited
to commercial data, and as we \-- and only recently have these groupers
been applied to Medicare data to a great extent. And it\'s not been as
much of a challenge because as you might imagine there is not, there is
some overlap between the commercial and Medicare patients, Medicare
beneficiaries, in other words some Medicare beneficiaries may have
commercial insurance and \-- but what typically happens is these, the
analyses that have been done to date have excluded those over 65 when
looking at commercial data and they \-- it\'s actually fairly important
that one looks at commercial data versus Medicare data separately,
because they have such, much different, often much different benefit
structures. So we haven\'t had had the challenge historically, however,
now as we begin to aggregate data across Medicare with commercial data,
those types of issues are beginning to be front of mind. Historically,
the people that have used these tools have really looked at these
populations separately.

**[\>\> Margaret VanAmringe:]{.underline}**

Okay. I was afraid of that.

**[\>\> Mark Rattray:]{.underline}**

Yeah.

[**\>\> Carolyn Clancy:**]{.underline}

Other questions? While you\'re thinking, I have one or two. You alluded
to difficulties capturing mental health and pharmacy data. I would
presume that that is not inherent to the data but actually to the
structure of health benefits. Is that correct?

**[\>\> Mark Rattray:]{.underline}**

Yes, on mental health, the issue has been entities just wishing to avoid
the additional privacy protections around mental health data, to try to
incorporate that. But you can also consider the mental health piece
similar to the pharmacy piece in that. So many times these benefits are
managed by a carved out \-- or an entity that produces or provides these
services on their own and it\'s often challenging to receive those, the
data extracts back from those entities in a way that we can incorporate
into the overall episode of care dataset. So it\'s, on the mental health
side, it\'s many of the privacy and limitations of use of that data. On
mental health and pharmacy, it\'s often an issue that these services are
provided by separate entities other than the health plans or at least
claims are managed by separate entities and then trying to get those
datasets merged into the dataset used for grouping has been challenging.

[**\>\> Carolyn Clancy:**]{.underline}

That is helpful. And I just wanted to clarify one point since a whole
big part of the systems issue that we are going to be tackling at our
next meeting and is so inherent to our work gets into consistency of
coding and so forth. So I hadn\'t actually heard the phrase until
yesterday, a greater number of episodes. But just to clarify, in essence
what was happening in an area that appeared to be providing more
efficient care, or more resource \-- less resource-intensive, was that
there was differential use of some codes. Is that correct? For example,
I believe, if I recall this, that varicose veins were coded as
cardiovascular disease?

**[\>\> Mark Rattray:]{.underline}**

Yes, much of the work that remains to be done is to really go back
upstream into the data from the actual grouper output. And this has been
something I\'ve always \-- it\'s been one of my dream projects is to
move, is to go ahead and move through and group the data, but then move
back in time and say okay, you know, how were these episodes generated,
now that we have the episode defined, go back to the claims data and
say, was the, was the underlying logic appropriate in pulling these
types of claims into this episode? I think when we do that, and
especially if we, at least on a sampling of those claims, can get back
to the actual medical record itself we will find that there is
significantly more noise in the system related to those simple acts of
checking off various diagnoses boxes or even procedure code boxes that
occurs as the patient is leaving the office or occurs in the billing
office afterwards. There are probably, there is probably a much more
underlying noise in the system than anybody has looked at yet related
to, to the simple act of marking codes down on a piece of paper and
sending it off to the billing department. I think we may find that it\'s
not as dramatic as one might worry about, but I think that until we do
those studies, those look-back studies, we don\'t have a good handle on
the quantification of how much noise is introduced by the circling of
diagnoses and procedures at the time of the patient\'s visit.

[**\>\> Carolyn Clancy:**]{.underline}

Sort of reminds me when I first started seeing patients beyond
residency, the job of actually checking off the code was taken out of
the doctor\'s hands. Professionals took care of that. Not us. Any other
comments or questions before we move on? Janet, yeah.

**[\>\> Janet Corrigan:]{.underline}**

On that same line of thinking. When you say that there\'s noise in the
system, I mean right now these, the data by episode are not being used
for pay for performance or public reporting so. So some of that noise
you would expect to be random noise, sloppiness, I guess. But it sounds
like we\'re going to be \-- as this gets used more at the episode level,
pay for performance, we\'re going to have noise that is intentional or
up-coding or miscoding or differential coding, correct?

**[\>\> Mark Rattray:]{.underline}**

Correct, but there are definitely systems and entities out there that
are using episode of care data for transparency and for pay for
performance currently. So that is going on now. I think much of what\'s
happening is essentially catching up with some of these underlying
studies to give a comfort level with how much of that is going on.

**[\>\> Janet Corrigan:]{.underline}**

Is it different in Miami than in Minneapolis?

**[\>\> Mark Rattray:]{.underline}**

Well, we certainly know there are greater number of services provided in
Miami. And there are certainly \-- we know that there are more
physicians involved in the care of beneficiaries in Miami versus
Minneapolis. So \--

**[\>\> Janet Corrigan:]{.underline}**

But I meant, is there more P for P in reporting in episodes in one
versus the other?

**[\>\> Mark Rattray;]{.underline}**

Oh. Not at this point in the Medicare, in the Medicare beneficiary
world.

[**\>\> Carolyn Clancy:**]{.underline}

I think a differentiation may be that Reed Tuckson is in Minneapolis.

\[laughter\]

Well, thank you very, very much for a very clear presentation. We\'re
going to move on now. Now, you\'ve heard a little bit about, from the
wild west of claims data and trying to figure out \-- make coherent
inferences about use of resources. And sometimes it\'s tempting to think
that tightly integrated delivery systems have all the answers to these
questions. So to test that premise, we\'re going to hear from two
presenters, Dr. Jon Einbinder \-- and if I\'m not saying that correctly,
please correct me \-- from Partners HealthCare System and followed by
Dr. Scott Young.

So, Jon \--

**[\>\> Jon Einbinder:]{.underline}**

Thank you. Can everybody hear me okay?

[**\>\> Carolyn Clancy:**]{.underline}

Yes.

**[\>\> Jon Einbinder:]{.underline}**

Okay. So I\'m going to try to give you a whirlwind tour of some of the
work that's being done at Partners. Thank you very much for calling us a
tightly integrated delivery system.

[**\>\> Carolyn Clancy:**]{.underline}

It\'s all relative.

**[\>\> Jon Einbinder:]{.underline}**

If you could advance to slide 2. And the perspective I\'m going to take
is a little different, I'm going to be focusing a bit more on the use of
clinical data for quality measurement and quality reporting and that\'s
really my responsibility at Partners. And you know you can sort of see
the outline of my intended presentation here. And the areas I\'m going
to try to touch on from the sort of study questions or guide questions
that I had are really business functions, financial models, data
aggregation methods and issues, and a bit about longitudinal
measurement, both as it relates to some of the episodes we\'ve heard
about, but also other ways of looking at longitudinal care. Next slide,
please.

Actually, we can advance through these pretty quickly. These are just in
there if you want to read about Partners, we\'re big, have a lot of
hospitals, next, please.

\[laughter\]

Two particularly big hospitals that make our lives very difficult in
terms of being integrated. And we can just kind of blow through to maybe
about slide 8. Actually stop, there please. Thank you.

The key points I wanted to make in the introductory slides are a couple
of things. First, Partners is a complex environment, it will be very
interesting for me to hear about Kaiser, which I think of as a much
simpler environment in some ways. You know, we\'re a federated system,
our budgeting is decentralized and it\'s hard to get institutions like
Brigham and Mass General to agree on measurement and sort of central
data strategies. The driver for a lot of the measurement work that is
going on for quality data management, which is my team, has really been
driven from the top, from Dr. Jim Mongan, who is the CEO of Partners and
kind of a desire to show that Partners as a system delivers better care,
adds value, and frankly can charge more than some of our competitors for
equivalent care, or what we think of as superior care.

This slide, slide 8 that we\'re on right now summarizes a little bit
about the mission of my group, of quality data management. It's really
that we\'re trying to provide the infrastructure and systemic approach
to manage data for reporting and analysis, really with an eye towards
clinical operations and quality. And there\'s a big assumption here,
which I always have to remind folks about here, at Partners, that our
data has secondary value. There\'s strategic value for using the data we
capture, taking care of patients, for understanding and improving
practice, assessing performance and driving improvement.

And then speaking a bit to some of the drivers, or some of the financial
and business drivers for actually paying for the infrastructure for
measurement. It\'s the usual suspects that I\'m sure you\'re all very
familiar with. We do have a fair amount of pay for performance here
that\'s driven quite a bit of our IS infrastructure. We care very much
about public reporting of quality measures and even care even more about
what shows up on the front page of the Boston Globe. There\'s the high
performance medicine initiatives which are basically Jim Mongan\'s
projects to try to move Partners forward in a number of ways. Then
there\'s some very practical considerations, folks that are asking for
data and not having a good place to go. So there\'s a lot of ad hoc data
and measurement requests. There was the desire for us to have on
measurement functionality within our electronic medical record, which is
called LMR, and that was required for, among other reasons, to get the
CCHIT certification which we\'ve now done. And then also an emerging or
increasing emphasis on registries and population management tools. I put
in parentheses research because it\'s not my purview but I will point
out that from a data aggregation point of view, research is a big driver
at a place like Partners. And that\'s been really a very good use case
or business case for infrastructure. Next slide, please.

This is just a screen shot of a summary page of our electronic medical
record called LMR. And this system has formed really the basis of a lot
of the measurement activities that I\'m responsible for. Next, please.

So the strategy we\'ve taken from an aggregation point of view is to
build a data warehouse, and basically this is a fairly conventional
approach to data warehousing, but we take data sources like the LMR, and
it turns out of course that data by itself is not terribly useful. You
need to combine it with lots of other things like patients and providers
and locations and lab data and allergies and vital status, et cetera. We
bring that into a central data warehouse that we call the quality data
warehouse and make that data available to other systems and to end users
in a variety of ways through ad hoc queries, reports, dashboards, and
disease registries. Next slide, please.

The next few slides, I believe, are just some examples \-- actually,
before I get to that. And probably preaching to the choir but the real
reasons in my view to externalize that data in a data warehouse, are
basically that we can\'t do the kind of measurement we need to be able
to do in the transaction or actual clinical system. Those systems are
poorly suited to analysis. We don\'t want to compromise system
performance. It\'s hard to run aggregate queries and it can take hours
to do that. You require dedicated programmers. The data hasn\'t been
cleaned up yet so it may be incomplete or un-coded or miscoded.
Documentation may be erratic at best, so lack of clear definitions and
business rules. And you usually can\'t get what you need out of one
system, so data integration becomes very important. Next slide, please.

And the next few screens are just examples of actual reports that we
make available through our electronic medical record. These are \-- and
there are various kinds of reports, but typically these would be used or
viewed by any physician, nurse, or other user of our electronic medical
record. That\'s at the provider level, or at the clinic or institution
level by medical directors and administrative staff. There are
descriptive reports. This is just a simple what medications have I
prescribed, and you can start to drill into that a little bit more. Next
slide, please.

This is \-- it turns out that much to my surprise, and I wouldn\'t have
expected it, that measures like this, many permutations of visits
without notes, time to starting a note, time to finishing a note, number
of notes completed within 120 hours, et cetera, many variations of those
wind up being some of the most frequent requests that we get. And
actually many of these, or some of these note measures are being used
internally for provider incentive plans both at MGH, and Brigham. One of
the reasons I included this report in this presentation, is that
something that sounds pretty simple, which is did a note exist for a
visit, requires that we can actually link data to a visit. And that is
sort of well short of building episodes of care, but I show this example
because that\'s a big challenge in and of itself. This report is built
by linking schedule data with note data, as well as including provider
information. Next, please.

This is a slightly more complicated report, but it illustrates again
some of what really for us is our current state of the art. This is a
pediatric body mass index report. And a provider or clinic would look at
this to basically get an idea of who their obese children are. And this
can be used for various interventions and programs. Again, something
that seems fairly simple to do, but actually again required linking
quite a bit of internal and external information, in this case, to
calculate some of those BMI percentiles. Next, please.

We\'re not really involved in episodes yet in any of this. So move on.
This is a screen shot of the front of our coronary artery disease
quality dashboard. I\'m getting a little more sophisticated than what
you\'ve seen so far and this is now summarizing, this is sort of a
cross-sectional performance but a provider\'s performance for taking
care of patients with CAD, compared to their clinic, compared to
external benchmarks, and again there are facilities for drilling into
this in other regards. This involves longitudinal data but the data, the
measures that we\'re showing here are certainly cross-sectional, at the
moment you\'re viewing the dashboard, what are your measures. Next,
please.

Then just talk a little about where we are starting to do some work with
longitudinal data, and to give you an idea of, at least from my
perspective at Partners, what\'s going on compared with say what happens
in the claims world in the presentation we heard earlier. We have a
goal, what sounds like a fairly simple goal, we want to follow our
cardiac surgery and cardiac intervention patients and just understand
what happens to them. So did patients who had a particular kind of
procedure or particular kind of pump protocol, did they do better,
worse, et cetera. And you know, the fact is we really don\'t know what
happens to these patients after they leave the hospital. So there\'s a
project that we\'ve been working on to longitudinally follow these
patients, and you know, we started out talking about all the different
data sources we could combine. Do we want clinical data, EMR data,
administrative data, claims data, et cetera and ultimately we settled on
something that sounds, again pretty minimal, but it\'s proving to have
high value. We are taking patient-level data from STS, from the Society
of Thoracic Surgery, and the American Society of Cardiology, linking
that patient-level data with National Death Index data. And the next
step will be to implement periodic follow-up by nurses and who will call
or contact those patients and then enter updates into the system. And so
there was a question earlier about linking patient-level data and
patient identifiers across claims systems. It turns out for us that
linking patients across various data sources is probably one of the big
challenges right now, and that was a challenge for this project. And
also of course with the national death data. Next, please.

A word about disease registries. You know, again, this is something else
that everybody is quite interested in at the moment. A lot of this is
driven by our pay for performance initiatives, and from my perspective,
the registry involves compiling lists of patients who may have diabetes
or heart failure, et cetera, and then we collect data about those
patients and pull it into a database and there are various user
interfaces that you stick in front of these databases. As I sort of
pointed out in the coronary artery disease dashboard slide, the data
itself is longitudinal but we do not group that data into episodes.
It\'s really aggregated at the patient level and the provider level.
Next, please.

In the next few slides, I\'m just going to review for a moment from a
clinical perspective what the idea of episodes of care or longitudinal
measurement, what\'s the vision that we\'d really like to have. Consider
that this is sort of just a hypothetical example but if we wanted to
better understand utilization and cost for hip replacement, the
challenge we currently would face is that no single data source provides
a complete view of orthopedic care at Partners. The state of the art
today is that we\'re able to look really at what happens in the
hospital. So we can look at cost and utilization for the inpatient
portion of the episode, but we can\'t say a lot for those patients about
what happens before they go into the hospital and after they leave the
hospital. Next slide, please.

An ideal world we would be able to view patient data from the entire
episode across settings and across entities. So we\'d be able to not
know about what happened in the hospital but know about pre-hospital
testing and pre-hospital visits, we would know about what happens after
discharge, including again radiology, lab work, complications,
readmissions, skilled nursing facilities. Next slide, please.

To actually start to realize this vision requires that we integrate data
from a number of different sources across multiple entities, and the
current state of the art at Partners, anyway, and at most health
systems, is that these data sources are not linked in any analytic way.
So on the bottom, these are acronyms that mean something here at
Partners, may mean something to you, but we need our data from EMR, CDR
is our clinical data repository, so it's lab results, TSI, which is our
cost accounting system, and other sources, professional billing,
pharmacy, order entry, claims, patients experience of care, et cetera.
And you need to pull all of that together, or many of those, if you want
to try to get at understanding episodes. Next slide, please.

The challenge \-- actually I would say there are several challenges that
kind of precede or are prerequisites of dealing with episodes from a
clinical perspective. And if you think about it from a database or a
data point of view, there are a number of relationships that we're not
very good at establishing across our data. Patients have data, so
patients have lab results, they have notes, they have blood pressures,
et cetera, they also have providers and they may have a primary care
physician, specialists, nurses, case managers, et cetera. They have
visits, encounters and they also have episodes, which would be
longitudinal across visits. Providers, doctors, nurses, other, have
patients. Visits have data and they also have a location to them. And
episodes are comprised of visits. The problem is that we have a hard
time, or the questions we\'re really working on are the ones on the
bottom of this slide. What is a visit? You can\'t build episodes if you
don\'t know what a visit is. And that\'s something, there are multiple
definitions for visits and it\'s something we\'re spending some time on
currently. How do you link patients and providers? So this is really
talking about patient panels. Which patients go in which providers'
denominators or patient panels. And how do you combine data sources like
the ones I showed in hip replacement side and you need to be able to
join on patients and providers and visits and locations. And then once
you\'ve done all that, how do you combine visits to build episodes?
That\'s sort of my attempt to characterize the data challenge. Next
slide, please.

I\'ll try to be brief about this, but just to step back for a moment and
remind everybody of course that data can be used for different purposes.
And, you know, we tend to focus on these uses differently. Data can be
\-- or measurement can be focused externally, what Brent James might
call measurement of judgment. These are things you get paid for, hired,
fired, promoted for, these are things that are publicly reported. Today
these tend to have an inpatient focus, we tend to use administrative
data for them. Where we need clinical detail we tend to do manual
abstraction. We are starting to do more of an ambulatory focus. A lot is
driven in our pay for performance contracts. And there we\'re really
using claims data, usually with the addition of some lab results and,
increasingly, some electronic health record information. Measurement
that\'s internally focused for learning tends to be focused on quality
improvement, on understanding, on efficiency, this is both inpatient and
outpatient, a lot of it is automated because you don\'t really want to
put the resources into manual for this and it tends to use more clinical
data. Next slide, please.

State of the art today is that most measurement of measures of both
judgment and learning are aggregated at the patient, provider, or visit
level, but not at the episode level. Within the work I\'m doing with
clinical data we\'re not ready to build clinical episodes. We\'re
spending our time getting data; joining patients, providers, and
locations; defining panels or patient-provider relationships; defining
visits; figuring out how to join claims with clinical data; and then
improving the coding and completeness of clinical data. Our customers
are not asking for episodes, in fact I say the top thing they\'re asking
for is actionable reports, how do we take the reports that we\'re giving
them and help them to be more actionable. And just a word about episodes
are used within the groups at Partners that work with claims data, with
things like ETGs, and one example that I thought would be worth
highlighting is that we do have a project, another one of Dr. Mongan\'s
signature initiatives, to do predictive modeling. This is led by Dr. Tim
Ferris. They\'re working with a vendor called IHCIS to use claims data
and lab data to identify high-risk patients. And this is using episode
groupers. Next, please.

I\'ll just finish up with a couple, these are probably going to be more
questions than answers, but start asking the question, what is the value
of the clinical data or the EHR data? Do reports and dashboards make a
difference? Who will look at those reports? And, you know, that\'s the
work that I spend a lot of time doing, and then the question of course
is, what is the added value of grouping that data into episodes? Next,
please.

Just a word about the trouble of getting data, or the issues with
aggregating data, since that was one of the topical questions. How do we
get data from transaction systems? We do that lots of different ways.
There are custom programs or tools. We work through interface or
integration engines, we use Web services, we take whatever we can get,
form flat files, XML feeds, HL7, and just to point out that for data
warehousing, or data aggregation in general, there\'s a unique set of
challenges compared with data integration for clinical applications. Map
and normalize disparate data sources and if there\'s one thing that sort
of I\'d like you to take away from my presentation today, it\'s this
trio or triad of patients, providers, and locations. Those are really
the three key dimensions that we\'re dealing with. If I had to add a
fourth it would be visits, and episodes might be the fifth. Next,
please.

Last but not least, just a quick word about data quality, which is when
you are working with clinical data, it\'s different from claims in terms
of its completeness and its quality. Clinical data is drawn from
multiple sources, and in the case of Partners, it\'s data that\'s been
collected over many years. Some of the data is converted from many
legacy systems. Concept management and knowledge management has in many
cases been very diverse or heterogeneous and laissez faire, and the use
of clinical systems is by and large voluntary. So whether or not a
problem is listed on the problem list, or a vital sign on the flow
sheet, is a voluntary thing, unlike the capture of an ICD9 code or a CPT
code for a visit. I think that may be my last slide. I\'ll finish up and
be happy to answer any questions.

[**\>\> Carolyn Clancy:**]{.underline}

Let me just say that that was terrific. What I\'m thinking, for purposes
of a discussion, is that it might be more sensible to have Scott Young
give his presentation and then have a discussion. So I\'m hoping you can
stay around that long to join us. Is that okay?

**[\>\> Jon Einbinder:]{.underline}**

That\'s fine.

[**\>\> Carolyn Clancy:**]{.underline}

Okay. So Scott, are you here?

**[\>\> Scott Young:]{.underline}**

I am.

[**\>\> Carolyn Clancy:**]{.underline}

Terrific. Take it away.

**[\>\> Scott Young:]{.underline}**

Great, thank you. Let\'s go \-- well, good afternoon, my name is Scott
Young, and I\'m from Kaiser Permanente from the care management
institute.

[**\>\> Carolyn Clancy:**]{.underline}

A recovering Fed, I might note.

**[\>\> Scott Young:]{.underline}**

A recovering Fed, that\'s right. Let\'s go to the second slide. We\'re
going to have just a little bit of background on Kaiser Permanente, how
we use data, our data sources, those sorts of things, future directions,
and try to answer a few of your questions. Interspersed in this, I'm
going to have a couple of examples from data use. Third slide. Next
slide.

You know, Kaiser Permanente has, as Jon was saying, you know, we're big,
we\'re integrated care, 8.6 million members, 13,000 docs, we're in 8
regions, we span 6 time zones. And data comes from each and every one of
these instances, sometimes multiple feeds of data come from each of
these instances that we have to make sense of and translate into care
delivery, and communicate that to you. Next slide.

Some of our big key drivers are \-- this is no surprise. The growing
chronically ill population and demographic shift, baby boomers and
elderly. Advancing medical science and technology, and all the
implications that\' come with that. How do we logically bring that in
and make sense of it? You know, our increasing need for performance
information, both internal to Kaiser and external to that is a huge
driver. And finally one that\'s emerging more and more is transitions
between care settings. Both transition into Kaiser Permanente and
transitions external to Kaiser Permanente. People coming into our system
or going out of it. This is just a huge area of focus for us. Next
slide.

This gives you a sense what have those chronic illnesses look like in
Kaiser. Six percent of our population accounts for about a third of our
cost, with individuals with multiple chronic conditions. Twenty-one
percent of the people with one chronic condition, that\'s about a third
of our cost. That's no surprise for anyone. Next slide.

Numerical counts on that. One out of four Kaiser Permanente members is
obese. We have about a million individuals who suffer from chronic pain,
25,000 new cancer cases each year, a half million members with diabetes.
These sorts of things. These are the kinds of numbers that we look at on
a day-by-day basis and try to make some sense of and make impacts upon.
Next slide.

How do we measure quality, what kind of data to we use to do that?
Kaiser Permanente designs and maintains a robust analytic capacity, if
nothing else we value data and build systems at both the regional and
national level and even down to the medical center level around data
capture, analysis, and feeding back. We use common sets of metrics, you
know, within KP, HEDIS, JCAHO, et cetera. It\'s interesting where we get
data from. It\'s generated, immense amounts are generated internally,
within the care system, both from business applications and clinical
applications. Members directly provide data for us. And that\'s becoming
more and more common with our personal health record, KP.org. We even
envision a time when people will send us photos or things like that over
the Web. People directly make inputs to us now via e-mail, secure
e-mail. External entities and providers provide data and all that has to
be amalgamated, prioritized, and somehow standardized within our system.
Quite a challenge. Next slide.

Now, we also have to present this data in such a way that it makes sense
to our providers and we\'ve designed a series of what we call big Q
metrics, common measures, and you can see the eight of them aligned out.
Clinical effectiveness, safety, HEDIS index, JCAHO, safety and risk
management \-- those are separate \-- service, how are we servicing,
provide services to our members, and finally, resource utilization or
efficiency of care. That\'s what we at the leadership level and the
leadership level at the regions are looking at on a day-by-day basis.
And we actually have those metrics which are live. You can actually
drill down into them. Core value metrics, underneath that, these are
additional metrics that we provide that are actionable at both the
operational and functional leader level. I mean, those are things like,
was information provided for a care handoff, clinical outcome,
effectiveness of those, were screening preventive services provided, and
in what way. Next slide.

I\'m going to go a little deeper into our internal data sources. The
first and primary one is our Epic installation, which we call KP
HealthConnect. We have both inpatient and outpatient Epic installed
across Kaiser Permanente, the last installations are coming up end of
this year and into \'08. The integrated health record is really what we
count on as our most powerful tool for data collection and aggregation.
Within that, though, we have legacy data systems which predated KP
HealthConnect and the conversion from those into KP HealthConnect is
essential, ongoing, and I would say nontrivial. Those are a mix of
business, financial, underwriting, and clinical systems. And you know,
we have to standardize the information within our data, within that, KP
HealthConnect, and any kind of enterprise data capacity we have. We have
a population care information system and multiple registries as well.
That\'s for panel management or care management, again I want to take
you back to our initial slides when we talked about the burden of
chronic illness, which really is beset for many of us who are in the
healthcare industry and you certainly saw that numerically for us. This
is an essential component to manage critically ill patients.
Interestingly enough, many electronic health records do not come with a
population care system that is functional. Also within that we are
bringing in the health risk assessment, the HRA, as part of it. Next
slide.

This is just a \-- slide number 10 \-- is just to let you have some look
at what the HealthConnect application actually looks like. The
integrated EMR, clinical decision support, remote access, Internet
access to our members, automated reminders, and charting and
documentation, and you can see the degree of comprehensive data and
integration that goes across those. And as well on the right-hand side
you can see the kinds of measurements and tools that we drive out of
those different entities from KP HealthConnect. Really each of those
layers is the different layer of data aggregation, data storage, data
query that goes with it.

Slide number 11 gives you some sense for KP.org, our member Web portal
and how that really interconnects with the care delivery system, all the
way from scheduling, to billing, to pharmacy, clinical aspects both in
inpatient and outpatient arena. Members are directly, you know, have
direct relationship with the record through KP.org, I mean I can dial up
and actually refill a prescription. That\'s a piece of data that\'s
provided by the member that goes into the system that we count.

Slide 12, the first of our examples, I\'m sure each of you have \-- are
familiar with this article. The Risk of Acute Myocardial Infarction and
Sudden Cardiac Death. This was the Vioxx study that came out many years
ago, and you\'ll see Dr. Graham from the FDA as well as our Dr. Campen
from Kaiser Permanente as well as others on that article. Next slide.

How did our Cox-2 study come about? It really started with data usage
and data looks from across Kaiser Permanente that led our pharmaceutical
and therapeutics process, you know, to really kind of limit the exposure
to Cox-2 drugs, Vioxx, out of some concerns of the efficacy, safety, of
that entire class of drugs.

Slide 14. And if you look across that, Kaiser Permanente at that time
had about a 4 percent usage of Cox-2, the Vioxx class of drugs, versus
about 35 percent for the rest of the community.

Slide 15. This study came about because of our pharmacy outcomes
research group really bringing data from across millions of members and
really looking at this, the epidemiological study, and determining an
increased risk for patients taking these drugs. It really was a
phenomenal piece of work.

Slide 16. Now, in 2001, one of our physicians contacted a medical
officer at the FDA, Dr. David Graham, and with their cooperation, got a
small grant from the FDA to do a phone survey and that\'s what actually
led to the Lancet article on the increased risk afforded by Vioxx for a
coronary artery disease or myocardial infarction, pardon me.

Slide 17. We think that had an impact. It stimulated a number of other
studies on Cox-2 agents. It really illustrated a gap between, as you
know, marketing promises and real world outcomes, and we think it
contributed to the FDA\'s scrutiny and the company\'s decision to
withdraw these drugs from the marketplace. That\'s really an example of
taking data from across a system, from across an integrated delivery
system, and turning that in to an actual impactful change, we would say.

Slide 18. I want to talk a little about data aggregation from multiple
sources. Some of what we consider critical issues and concerns. We
talked about this before, the need to standardize. And you know, the
decision around using HL7, SNOMED CT, including the CDA and CCD data
transmission standards, we believe is critical. We must be able, just as
you saw, to be able to take data from across systems, from across
disparate systems, and look at them together, aligned, to be able to
bring out that kind of powerful, you know, interventions that we saw
with the Vioxx study.

You know, for us we need to expand the currently collected data to
include health risk and health status data. You know, a previous speaker
talked about predictive modeling. We think that's very, very important,
but to do that we have to do to get past our current clutch of clinical
data to get into that. Got to be able to incorporate data from chart
notes. That\'s really difficult to achieve at this point. The natural
language processing procedures we have right now are really inadequate
to do that. So to do this we either need to teach humans to speak
machine language or teach machines to speech human language. That\'s
going to be difficult in the short-term.

Finally on the multiple sources we need to resolve issues of
attribution. I mean, who cares for a diabetic? Is that a care team, is
that a provider, is that the system? You know and we really need to make
some decisions, we would say nationally around that. It's certainly
something we struggle with inside Kaiser Permanente. Next slide.

Let me take you quickly through another example that we use that was one
that was internal to the Kaiser Permanente. In the mid \'90s we started
looking at our members with diabetes much more critically. We noticed we
were able to identify members with diabetes, an increase of between 4.4
percent and 10.5 percent between 1997 and 2004. We saw some acceleration
in the use of diabetics on ACE inhibitors. Next slide.

We also saw an acceleration in the number of individuals using LDL in
control, and the number of individuals concomitant with that using lipid
lowering agents, mainly Lovastatin, for this, all the way from 38.6 to
50 percent. Individuals with diabetes were being more aggressively
identified, and currently being on the right drugs, lipid lowering
agents as well as ACE inhibitors. Next slide, 21.

Within some of our regions we started to notice that some combinations
of drugs were leading to improvements in our diabetic members, as well
as members at increased risk for coronary artery disease. We needed to
identify the critical components to reduce cardiac events in these
members and we did identify a combination of aspirin, an ACE inhibitor,
and a lipid-lowering agent that has the secret, you know, triad of
drugs. You know, this was tested on a pilot within a KP region and then
we validated this taking Kaiser Permanente data and injecting it into an
artificial modeling capacity called Archimedes. Archimedes actually grew
a population inside a computer, actually modeled using ALL, and
identified a significant reduction in cardiovascular disease and
subsequent cost in this population. You know, we actually subsequently
rolled that out into all eight Kaiser Permanente regions just based on
that computer model. So that is another very powerful use for data built
within our system. Next slide, 22.

Now that data aggregation I\'d like to talk about some interests to
consider. In one of the questions I noticed in the Federal Register
notice, what medical conditions would this be applicable to? We would
say we really wouldn\'t limit that at all. We think it\'s applicable to
all medical conditions but particularly data aggregation is applicable
to chronic and complex or multiple chronic conditions.

A critical barrier regarding the source and quality of data. You know,
information is everywhere. It\'s widely dispersed and it\'s in paper
format, some of it is proprietary, non-clinical and billing systems and
underwriting systems, those sorts of things. And for us we need to align
what that information looks like, where it comes from, and standardize
and prioritize it to bring it into our data warehouses and right now
that\'s not there. And you can imagine, if we\'re struggling with that
inside Kaiser Permanente, what it\'s like in a nonintegrated system or a
nonaligned system.

And finally, the ability to organize data into care episodes remains
difficult for us. Technologies exist but some of them remain proprietary
and make comparability a difficulty, between one technology and the
other.

Slide 23. Specifically within episodes of care, let me just talk about
methodology and mechanisms. A key issue of what outcomes and processes
are not supported by the episode methodology, not necessarily what is
but what is not. And we would say that, you know, doing care management
or population management is very difficult. The population denominator
is sometimes missing, and if we\'re \-- many times utilizing persons or
populations is more appropriate than the episode itself. The episode
approach in our minds sometimes fails to assess the appropriateness or
the volume of the episode, and if you look a little different
denominator that might change that.

Slide 24. Data feedback mechanisms, and this is really how do we use
data. Data is obviously a key attribute of the KP quality and system
improvement approach. We use it all the time. I mean, as I said earlier,
if nothing else, core data. Currently virtually all information for
performance improvement is retrospective in nature. Sometimes that data
is too old to be or feel actionable, at the level of operations. You
know, in our world, as we\'re making more and more data performance
oriented down to the medical center, you know, unit-based team, provider
level, we\'re having to change what the data looks like to be
appropriate at that level. And of course as you can see we can roll it
up at the national level as well.

The performance improvement domain requires the most currency and
accuracy of the data and to that level, you know, you almost want data
to be quasi-instantaneous, you know, at the provider level. You know,
what happened in the last two weeks with my patients, what happened in
the last month with my patients, what happened in the last week, you
know, my patients with diabetes or coronary artery disease and how will
that affect my decisions. So that\'s a constant struggle and what we\'re
really talking about at the current state is what does it look like in
the last quarter or sometimes what did it look like in the last year.
But for us, we believe making that performance improvement information
almost instantaneous is kind of the Holy Grail.

Slide 25. The future. You know, we think, you know, widespread adoption
of the EHR is going to enhance the availability of actionable data, and
I think that\'s something that your Workgroup and your committee has
been championing for a long time, and we applaud you on that. Data
collection and aggregation will increasingly become a byproduct of
routine care. That\'s what we\'re seeing right now with KP
HealthConnect. Integrated systems we think will have to learn from
population outcomes. And again, I go back to what we would say are
instrumental changes in those two examples I gave you. Data-driven
feedback, of course become more and more real-time. And I talked about,
I spoke about that earlier, but that\'s our hope.

Last slide, 26. Finally, conclusions. Currently, needed data is not
widely or completely available, both inside integrated systems and
certainly not outside them. We are currently aligning and utilizing data
from multiple sources both inside and outside, problematic from a
standardization point. And EHRs we believe remain the key tool for data
collection and provision of evidence-based care. I thank you.

[**\>\> Carolyn Clancy:**]{.underline}

Wow, let me just thank you, as well, Scott. That was wonderful. And I
think you alluded to this, but just to be very clear. I found both
presentations both invigorating and humbling at the same time. So I
guess you both shattered our illusions that your systems had solved all
problems, but I think actually your presentations are also very helpful
focusing our attention on some future challenges that we very much need
to anticipate. So let me also say that my favorite phrase of the day is
the management of concepts has been laissez-faire. Anyway, let me open
this up to the Workgroup for questions, comments, observations.
Margaret.

**[\>\> Margaret VanAmringe:]{.underline}**

I would just also agree with Carolyn. This is very humbling. I think the
number of hurdles are fairly Herculean, at the minimum. But also I like
the fact that some of the positive outcomes that could come from looking
across the episode of care and populations are very concrete, very real
world, and you can see the benefit of them right away. And I also agree
that one of the things that we probably do need to look at is the cost
benefit of doing these types of episode of care, because obviously it\'s
going to be very expensive to do these. But you pointed to some of the
benefits that can radiate well beyond just a small population of
patients or small cohort when they\'re done correctly or well.

[**\>\> Carolyn Clancy:**]{.underline}

Others?

**[\>\> George Isham:]{.underline}**

This is George Isham. Can you hear me?

[**\>\> Carolyn Clancy:**]{.underline}

Yes.

**[\>\> George Isham:]{.underline}**

I think that the point about population use of the data and its
relationship to episodes is critical. Particularly the point was made
about inability to judge appropriateness and the number of episodes. And
I just think those examples that were given where the population data
was used to modify care, are just very critical.

[**\>\> Carolyn Clancy:**]{.underline}

So while you\'re continuing to gather your thoughts, I have a question
for Jon. I was very intrigued by your slide that talked about
registries. And I think if I heard you correctly, you are getting data
from registries developed for STS and the American College of
Cardiology. Is that correct?

**[\>\> Jon Einbinder:]{.underline}**

Correct, yes. It\'s actually, it's probably the data we submit to STS
and ACC are essentially being diverted into the longitudinal registry.
But it\'s essentially \-- it is the same data.

[**\>\> Carolyn Clancy:**]{.underline}

Let me back up for a moment and say when we did our future visioning
exercise, we had lots of problems and again both of your presentations I
think are helping us get some more clarity here about what was kind of
near-future and what was way out there and we\'re going to get there
some day but it\'s a way off. You are in effect drawing from your LMR to
submit data for the clinicians for these registries.

**[\>\> Jon Einbinder:]{.underline}**

Not exactly. The LMR is one of our systems, and it\'s the ambulatory
electronic medical record. The STS data, which is coming from us from
three hospitals, from Mass General, Brigham and Women\'s, and North Star
General, are being collected in a number of ways. And in some instances,
manual ways. In one instance, noticeably Mass General, a fairly
automated way tied to an electronic medical record at Mass General which
is called OnCall. So at MGH, a lot of the data is collected in an EMR,
not LMR, drawn into the cardiac data warehouse, and then being
aggregated for submission to STS. The ACC data is coming from a variety
of cath lab systems and the usual sorts of administrative sources.

[**\>\> Carolyn Clancy:**]{.underline}

Okay, that\'s very, very helpful and the reason I was quite intrigued by
that is putting aside episodes just to think about longitudinal
assessments in some fashion, and leaving the E word behind for a moment.
You know, a number of clinician groups I think have displayed incredible
leadership in creating these registries. At the same time I personally
would like to think that at some point in the future it would be
possible to pre-populate those from a clinical information system as
opposed to having these, just a growing array of standard-alone systems.
Now, clearly there\'s going to be a need to link to either a data
warehouse or something like it for the longitudinal follow-up,
particularly for people who are mobile and so forth, but that\'s why I
was very curious about that point.

Let me ask you another question. Scott, you talked a lot about instant
feedback. What kinds of feedback are you giving clinicians now?

**[\>\> Scott Young:]{.underline}**

Well, you know, it goes all the way from, you know, performance around,
you know, screening, cervical cancer screening or breast cancer
screening or those sorts of things, down to control of hemoglobin A1c
and that sort of thing, for their population. That occurs at varying
intervals, kind of all depending upon the level of the team, individual
clinician team, monthly, quarterly that sort of thing, to the regions or
to medical centers, which may be quarterly. That\'s something that I
think what we\'re trying to figure out is kind of what\'s the right
interval to give people feedback on. At what point do you kind of
overwhelm people with feedback and it becomes background noise and at
what point does it become actionable? So this is still a script being
written across Kaiser.

[**\>\> Carolyn Clancy:**]{.underline}

That\'s helpful. One term I didn\'t hear either one of you use
explicitly was clinical decision support. I wonder if you could talk
about that a little bit.

**[\>\> Scott Young:]{.underline}**

That\'s a big deal for us. I mean, and it\'s been a big deal as we\'re
taking our national guidelines and actually converting those into smart
sets or best practice alerts, which is the mechanism we use within Epic
or KP HealthConnect. Again, the magic in that is where does that become
problematic or intrusive and where does it become useful? You know, and
push or pull. And the accounting, I mean, you say, gosh, you ignored a
smart set. That, you know, is either a \-- that\'s a performance
indicator or not a performance indicator. Those are things that we\'re
still struggling with.

**[\>\> Jon Einbinder:]{.underline}**

I would say sort of a very similar answer but maybe a couple of
additions. We have, as you probably know, a lot of decision support in
our clinical systems. What we started to do from a population management
point of view and with a reporting point of view is to take the very
same logic, the very same rules that are firing in the clinical systems
and run them against populations and turn those into reports. So we have
a \-- if there\'s an overdue for A1c we have a reminder in LMR, there\'s
an overdue for A1c reminder report in report central. There are
different workflows and we think they\'re probably complementary, you
know, hopefully complementary workflows to actually improve performance.

The notion of feedback is something that we do some of around the use of
decision support, and hope to do more. A lot of the feedback to date on
decision support isn\'t so much about the use of decision support but
it\'s on the things decision support relate to. So if we have an
enterprise initiative, as we do, to improve documentation of smoking
status, we have a report that\'s about the documentation of smoking
status. It\'s not about whether you ignored the reminder for that, but
partly that\'s because we don\'t have the reminder data incorporated
into the warehouse yet.

**[\>\> Scott Young:]{.underline}**

I think also, we\'re all running into this in integrated other systems.
When we take guidelines, which were paper guidelines before, and were
comprehensive, you know, and had these algorithms in there, that the
human brain can actually kind of go through and sift through quickly,
you put that into an electronic system, if you put it in just like to
occurred in the paper environment, it fires off alerts over and over
again. It just multiplies because it\'s getting data from multiple
sources. It\'s really a different animal in electronic versus paper.

[**\>\> Carolyn Clancy:**]{.underline}

Paul, did you want to say something?

**[\>\> Paul Tang:]{.underline}**

I was just going to give another data point. We report quality on the
same clinical measures quarterly and the other piece is unblinded so I
can dial up any documented place and see their quarterly report for
this. And we\'re actually in a side project to look at if you provide
incentives reports for clinical measures, quarterly versus once a year,
does that make a difference.

[**\>\> Carolyn Clancy:**]{.underline}

That was Paul Tang from the Palo Alto clinic who will be our next
presenter in just a couple of minutes.

Anyone else, other questions? Margaret?

**[\>\> Margaret VanAmringe:]{.underline}**

This goes back actually to a point that came out in Mark\'s presentation
about the underlying clinical logic to these groupers. And I was
wondering, either Jon or Scott, what are you doing, if anything, to look
at and validate that the episode of care after it\'s been grouped, has
not actually omitted certain things which clinically might not be
readily apparent in a grouper methodology, but that the two visits are
really related. They\'re just different, maybe different specialists are
looking at different sides of the same problem but they didn\'t relate
that in the record. So I\'m concerned about that kind of maybe failure
in the grouper methodologies and I wonder if you are going back and
looking to see whether or not the \-- taking a sample and seeing whether
or not the clinical logic was actually met that you would expect to have
met. If I made that clear.

**[*\>\>* Jon Einbinder:]{.underline}**

We\'re still trying to define what a visit is, so the answer would be
no.

**[\>\> Margaret VanAmringe:]{.underline}**

Okay.

**[\>\> Scott Young:]{.underline}**

You know, and for us I think that we\'re early at looking at this. If we
had that instance occur here, we would say it would probably be a
failure of our system in general. But I think we\'re looking at it,
we\'re really looking at that.

**[\>\> Rick Stephens:]{.underline}**

This is Rick Stephens. Question for you, Scott. Your last chart to me is
one of the most important ones because it talks about the importance of
integrated delivery systems and aligning the utilization data, about
standards, and of course focusing on electronic health records remaining
the key tool. We\'ve talked a lot about the technology side. When you
look at Kaiser and the multitude of locations and hospitals you\'re
involved in, can you talk a little bit about what you\'ve done to get
the doctors and the clinicians aligned about using this, because
everyone has a different language and coming from a different direction.
What have been the secrets about getting some alignment about these key
issues across your enterprise?

**[\>\> Scott Young:]{.underline}**

I\'ll tell you, we \-- a couple of things. I need to acknowledge Andy
Wiesenthal and Louise Liang have been a couple of secret weapons here in
doing that. But it was \-- we had two approaches, whether we did what we
call the big bang approach, roll everything out kind of everywhere
within a region at the same time, versus the incremental approach and
kind of roll them out medical center at a time. Both have their
advantages. Medical center at a time is really taking one group, kind of
finding champions, kind of hand-rearing a cadre of clinicians and other
providers and staff to using the system, and then having that medical
center or those champions kind of roll that out across a region. You
know, kind of teaching others what they know.

I also think having standardized toolkits has been just incredibly
important. You know, taking lessons learned from across Kaiser, from the
first instance that we did this, which, you know, obviously probably
didn\'t go as smooth as we wanted to, to the last one, which is going
much smoother. And having those lessons learned be available to all.

And lastly, I would say it\'s listening to clinicians and frontline
providers and what they say works and doesn\'t work in the system. And
having the ability to modify the system based on that input. In other
words, it needs to work for them, not have them work for it.

**[\>\> Rick Stephens:]{.underline}**

I appreciate that. I think those are going to be important things for us
to remember as we think about this broader system, as we try again
impact behaviors and I think, Carolyn, as we think about it, that\'s the
soft side we\'re going to have to think about, with our recommendations
going forward.

[**\>\> Carolyn Clancy:**]{.underline}

And Jon, if I am recalling correctly, one of the more than one grants
that we\'re funding at Partners, actually involves testing some
different types of drop-down menus?

**[\>\> Jon Einbinder:]{.underline}**

Yes, I think this is probably one of the, we call them advanced
reminders, but there\'s, they're essentially actionable reminders. On
the top of the chart if you get a reminder saying patient has been on
lithium for more than 365 days without a lithium level, in the old days
it would have said suggest ordering lithium level, but you would have
had to go do it. Now there\'s a drop-down, you just click order lithium
level or you select something else that says done elsewhere, remind me
later, whatever. That\'s I believe the study you\'re referring to.

[**\>\> Carolyn Clancy:**]{.underline}

That\'s helpful. As we get into this, I think one of the issues about
the quality of the data is going to be precisely how clinicians enter
it. Migrating from the illegible scribble to actually collecting
something that might one day be translated as a data element is an
interesting journey to contemplate.

Jon, just before we transition, I had a question for both of you, but it
was really prompted by your slide about the uses of the information,
external versus internal. And it strikes me that there\'s kind of an
interesting tension between our specific charge and our broad charge.
Our specific charge is really focused on making the reporting for
judgment easier. And you know, to be honest that\'s very tactical and I
would argue sort of a practical thing to have four cornerstones of
value-driven healthcare that don\'t connect is not a state of affairs
we\'d like to have for a real long time. On the other hand, to actually
get to where it\'s value-added, it does seem to me that sooner or later
we bump into your observations about knowledge management and management
of concepts.

So a question for both of you, and you can reflect on this if you\'d
prefer and we\'ll bring it back, but I think it feeds into our new work
on systems and so forth, is to what extent would additional work, which
could be convening, it could be some case studies or whatever, sponsored
by this Workgroup, might actually help to elevate the enterprise for
all. I mean, what we\'ve been doing is hearing from the two of you and
some other presenters and hearing some, what would have to be described
as really cool stuff in terms of people taking advantage of their
internal data streams, really getting a, some view that these data
streams are a strategic asset that they\'ve never actually had before,
and using that in selected instances to transform care from being sort
of reactive to proactive and that\'s very exciting. But I\'m also
getting this very clear sense of how early days it is. So I guess one
question for both of you is would it help if we were to kind of elevate
that conversation and bring in more folks to have that kind of
discussion?

**[\>\> Scott Young:]{.underline}**

I mean, I would say yes. I think it\'s critically important to bring in
others. And I think we really have to understand not only what happens
inside integrated systems, but what happens outside integrated systems
into dis-integrated care, and what happens at the merge. Again, when we
talked earlier about transitions in care, transitions between entities
or even between entities within a system, really understanding that. We
think that is such a big black hole that we see quality and safety and
cost dropping into. So yes, I mean, I would heartily endorse that.

**[\>\> Jon Einbinder:]{.underline}**

I would agree with that, and I think it, obviously have to think about
sort of which settings we would want to focus on. And for example, if
you\'re obviously not part of a large integrated delivery network but if
you\'re part of a midsize group practice without an EMR or doing an
implementation of a commercial EMR, how does this work apply to you. Or
if you\'re a community hospital, et cetera. So I think that\'s, you
know, very important. Before I came to Partners, I was at a place where
we didn\'t have a lot of this stuff and I used to get really frustrated
hearing presentations from David Bates and others, and I'd say, well,
that's great for you, but what about us? There's a definite tension
there. And I think it\'s very important. I know here in Massachusetts
with the Mass eHealth project there are a set of three implementations,
community implementations, of vendor EMRs, and that quality measurement
and a set of standard measures from those EMRs is part of the projects.
I think there\'s a lot of that kind of work that if you aren\'t already
hearing about, it would certainly be useful to look at.

[**\>\> Carolyn Clancy:**]{.underline}

That\'s helpful. I\'m just thinking about the fact that it\'s not really
in our, even in our broad charge to inspire so much, but if people began
to have a view of the future, I think we could actually have a clearer
path to getting to what I think of as value added in this enterprise. In
other words, moving beyond the place where we are now where at its best
this is not too much of a burden, but it really isn\'t about what I as a
doctor am doing every day or what I think is important. Obviously, that
is one huge challenge, but I found your observation, Jon, so striking
that it just kind of jumped off the page at me.

**[\>\> Jon Einbinder:]{.underline}**

One last comment on that, if I could. I really believe that, I believe
that using the EMR data for quality measurement is important and an
incredible resource, but I also think that data is generally very poorly
suited for measurement for judgment, for all the reasons of it is
heterogeneous, it is incomplete, there's a lot of missing data, et
cetera. It\'s extraordinarily useful for understanding, and the places
where we are using it for judgment like the provider incentives around
note completion, for example, takes a huge amount of work and you really
learn a lot about the limitations of the data.

[**\>\> Carolyn Clancy:**]{.underline}

Well, thank you, both, again, this was really incredibly from my
perspective and I\'m guessing the lack of other comments reflects Friday
afternoon or a similar sort of amazement, as well as humility about the
challenges going forward.

We\'re going to go back now to the specific charge, and hear from Paul
Tang. And before you came, Paul, let me just say that I was bragging on
the, what I\'ve been hearing from the meeting. So we\'re really pleased
that you could join us today.

**[\>\> Paul Tang:]{.underline}**

Well, thank you, Carolyn, for giving us the opportunity to present the
results of the NQF expert panel on how to make EHR data more useful in
supporting quality measures. Just a heads up to the person flipping the
slides, I tend to go pretty quickly, so we're ready for the next slide,
please.

[**\>\> Carolyn Clancy:**]{.underline}

It\'s not his issue, it\'s the technology.

\[laughter\]

**[\>\>]{.underline}**

It lags a little bit.

[**\>\> Carolyn Clancy:**]{.underline}

He\'s fast enough.

**[\>\> Paul Tang:]{.underline}**

Okay. I want to start out with just a brief discussion about the
accuracy of claims-based quality measures, just to frame the problem and
give the motivation for why would we want to look elsewhere. Then talk
about the process we used in the expert panel to elucidate the EHR data
elements and the quality. And conclude with recommendations we\'re
making to HITSP, to CCHIT, and to the quality measure development
process through NQF. Next slide, please.

As we all know, historically the quality measures, performance measures,
were built on, were derived from, claims data. Why? Because that\'s
where the data was. It\'s sort of like the drunk under the lamp post,
that\'s where the light was, that\'s where we\'ll go. So if you start
there and say well, in this general illuminated area, what\'s
standardized, what's combinable, what questions could we conjure up
based on that data, and what quality goals could we set based on those
questions. And it appears that you really should bring in a metal
detector to see if you really are in that general direction. Next slide,
please.

That brings to us asking the question, what is the impact of using
administrative data and calculating the performance measures and quality
measures when you compare claims-based methods with EHR-based methods.
Next slide, please.

So the method here, the goal is to compare the calculation for some
well-known quality measures, using administrative data versus clinical
data from an EHR. And we did these for 125 diabetics that were
identified through gold standard chart review. We really knew that these
were diabetics. In the first instance claims-based definition and the
one using at the time was the CMS DOQ, doctor office quality project in
which they defined a diabetic by saying those people who in 18 months
had 2 visits where an encounter diagnosis included diabetes. And the
other method we used was to query coded information, coded information
only, in the EHR. Next slide, please.

And so the good news is that using only coded information, not having to
deal with the progress note text, we could identify 97 percent of those
gold standard defined diabetics in this population. And furthermore, we
could identify 94 percent of them if we considered only diagnoses listed
on the problem list. So that\'s the good news. The bad news is if you
use the claims-based definition, you would miss 25 percent of our
diabetics. In another disease, coronary artery disease, you\'d miss 30
percent of those with coronary artery disease. A huge problem of those
missing from the denominator. As you might worry, that affects the
calculations. So half of the diabetic measures were significantly
statistically different if you used the claims-based definitions of
those identified that way versus those missed that way.

[**\>\> Carolyn Clancy:**]{.underline}

Did you look \-- you\'re checking sensitivity by doing it this way. Did
you look at specificity in \--

**[\>\> Paul Tang:]{.underline}**

Yes, those were very good. I don\'t know that I have it in the slides
yet.

[**\>\> Carolyn Clancy:**]{.underline}

Okay.

**[\>\> Paul Tang:]{.underline}**

In the next slide, please, there\'s some examples. So the first one
looked at hemaglobin A1c measurement, and we would have scored 97
percent, which might be worth a pat on the back but probably humanly
impossible, considering most physicians are that way. And for the people
you missed you would have scored only 68 percent. And what I think that
reveals is you would probably have a different view of yourself,
receiving one score or another, and you might allocate resources
differently based on one score versus the other. And that\'s the
problem. So blood pressure measurement it was 61 percent, versus 45
percent. You get the picture. It can be very different. Next slide,
please.

So the concern is that you would underestimate, using the definition
that we used in the DOQ project with two encounter diagnoses, you would
underestimate the target population, which would bias it toward a higher
score. The way that works, it\'s self-fulfilling prophecy. If you went
to the trouble to see them a couple times during the measurement year
and you coded the fact that you were working on the diabetes, more than
likely you were working on the diabetes and doing better than the people
that you either ignored or didn\'t pick up.

And that also, because it\'s based on claims submissions, it\'s subject
to gaming. And the reason is because you can create your own systemic
bias, Janet was mentioning potential \-- one way that's created is by a
billing paper sheet that only has a certain diagnosis because that\'s
what you get paid the most for. That\'s a pretty systemic bias. And
there is no clinical downside to doing that. There\'s a financial upside
but no clinical downside because it doesn\'t come back to haunt you.

On the EHR side, if you use diagnoses on the problem list, it is likely
to be more accurate because you\'re depending on that for your clinical
care. You are less likely to do gaming because it would come back to
haunt you if you don\'t code this person correctly. And ironically, if
you use these more accurate data, you may get a lower score. So you
would have a disincentive to report more accurate data. Next slide,
please.

So as we are considering the chance of, you know, the country is
vigorously pursuing adoption of EHRs, well, wouldn\'t this be a nice
time to flip the question, which is why it\'s upside down.

\[laughter\]

Okay. When it\'s animated, it makes more sense.

\[laughter\]

Wouldn\'t this be a nice time to flip the question and say, golly, what
quality goals is the country interested in, what data would we like to
ask providers to answer those quality questions, can we standardize and
combine those data and can we make those exist in the EHR in a high
quality format? To also comment on the internal/external measures that
were talked about in the preceding talk, as I said, we do measure
quarterly and report that out to everybody, basically, in the
organization. And \-- but we report on internal measures that we have
created ourselves and the reason is because it makes clinical sense.

So at the same time we lecture the doc to say we want your hemoglobin
A1c to be less than 7, it would probably be counterproductive and
perhaps illogical to say, and we\'re going to report what percent of
your diabetics have hemoglobin A1cs greater than 9. So to make our
clinical guidelines consistent with the report, we generate internal
report and we do external reports simply as a waste of time to comply
with (inaudible). Next slide, please.

So you ask the NQF to put together an expert panel that would look at
the situation, try to figure out are there high quality data elements in
the EHR that we could capture and use in quality measurements as a
byproduct of taking care of these folks. Next slide, please.

So NQF put together a list, a member of committees, that included
clinician, medical informatics professionals, people who develop
measures, people who accredit, and vendors of EHR systems to contribute
to this effort. Next slide, please.

We had active and significant contributions from the Federal liaisons to
the committee and of course we couldn\'t have done this without the
outstanding staff they have at the NQF. Next slide, please.

So the framework for prioritizing the conditions and the data from that,
we used these three dimensions. One is what\'s the clinical importance
of the health condition. That sort of gives us a high-low kind of
assessment of the clinical condition. Second, and this is probably the
new part, is let\'s assess the quality of the data you get out of the
EHRs. That\'s alluded to by the previous two speakers. And let\'s look
at the diversity of clinical data available in the EHR. Next slide,
please.

So what we did was first go through and for all of the conditions that
were covered by AQA and HQA measures, we labeled them as either one of
the IOM priority conditions or not. And then from the ones that were
part of the priority conditions, we chose four conditions as the initial
sort of set to work on. And those were diabetes, heart failure, coronary
artery disease, and medications management. So for those measures, and
we extracted all the data elements \-- thank you very much \-- extracted
all the data elements that went into the calculations of those measures.
And then formulated an EHR data quality figure of merit \-- and I\'ll
explain that in just a little bit \-- so that we could go through and
analyze the quality of the data elements that go into each of the
quality measures, and based on that, say, you got high quality data
going in, or not, and what can we do about it? Next slide, please.

So first let me talk about the figure of merit we developed for
assessing the quality of data in the EHR. So this is basically trying to
say; this data accurate, reliable, and standardized? Three things that
would make it combinable and useful in aggregate form. So we came up
with five attributes you could assign to an EHR data.

One is, does it use data standards? And we thought, you can see the
weights on the right. That\'s a very, very important attribute of data
to make sure that you can use it in the quality measures.

Secondly, is the data that\'s coming from the EHR coming from an
authoritative source? If it\'s coming from a clin lab and directly
traced into the EHR, probably the answer is yes. There are other sources
of data that may not be as highly accurate and we would want to know
that. So that\'s a high weight.

Feasible workflow. Somebody used the term I think the soft side or
something. If it takes ten clicks to get the data in, it won\'t be in,
or if it does get in, I don\'t know how well you can rely on that. In
fact, the truth of the matter is, if it takes two clicks, it\'s one too
many. So that turns out to be very important to get high quality data
there reliably.

Version control is there because if these are going to be used by
quality groups and performance incentives and accreditation, you've got
to know what really happened at the time that care was rendered. And so
there has to be a way to know what was true at that time. So that\'s
what version control means.

And finally is it traditionally, is it typically in EHRs? And when we
look at EHRs, the committee considered more comprehensive EHRs. So you
had to pick your site somewhere, and it\'s not a future to be EHR. It\'s
really one you could buy today but it's of the comprehensive sort. So
that\'s sort of the attributes we looked at with respect to each data.
Next slide, please.

So let\'s look at a cluster of quality measures having to do with
diabetes. On the right, on the columns you\'ll see the measures that
deal with diabetes. On the left side, the rows are data elements, such
as vital signs or clinical lab result. And this is sorted by the data
quality weighted score, the score from highly accurate and reliable and
standardized to not so high. Okay? So let\'s just use an arbitrary
cutoff of let\'s say 70 and you can see the grayed rows are those where
\-- and they cover a lot of the bulk of the data elements for the
diabetes measures. So that\'s the good news. You can see at the very
bottom we have something with a failing grade of 47, and unfortunately
it\'s involved in each one of these measures. So it could actually pull
down all of these measures. And that is, we code name that the billing
code or that\'s the things you would find on billing systems or billing
sheets and not the EHR problem diagnosis. So that, we found to be a real
liability to the entire measurement system approach. So that in fact
caused us to create one of our most important recommendations. Next
slide, please.

And that is basically to ditch this billing code algorithm. Sometimes
people use, well, just see if there\'s one encounter diagnosis with some
condition, let\'s consider them a denominator. That also leads to a high
false positive, because you \-- a lot of people will write, let's say,
if you check for glucose you may think to rule out diabetes, but you
code it as diabetes, mistakenly, that\'s just the way things are done
and lo and behold you have a lot of people who were coded with diabetes
who have really good A1cs, or glucose. So the expert panel recommended
that instead of using billing diagnosis, that we would use the
interdisciplinary problem list in the EHR and that the diagnoses there
be used to identify the people in the denominator, the people in the
target population. Next slide, please.

In the heart failure cluster, you see the \-- and by the way, there\'s
an error on these charts. For all the AQA measures, the billing
diagnoses are important and those should have an X there. But you see
that liability again and the next group up is basically having to deal
with diagnostic test results. Those are the tests where you need to have
an interpreted result rather than just a clin lab number. The clin lab
numbers are there, reliably, standardized, et cetera, the test results
are, diagnostic tests results are typically dictated, so we're into that
free text. But there\'s very few actually coded results we need. In
fact, in heart failure and CAD it\'s the LV ejection faction, so if we
could just have that and maybe even 20 more, we would in good shape but
that does not exist now. So you can anticipate that\'s going to be one
of our recommendations. The other one that happens in heart failure, is
you\'re supposed to have discharge instructions that talk about what you
should do after your hospitalization to prevent having your heart
failure be exacerbated. And that is something that\'s not captured well
in the EHRs. Next slide, please.

For coronary artery disease, a lot more measures. And you see the same
bad actors, that is the text report of diagnostic tests, such as echo,
where you might get an ejection fraction or a cath or an (inaudible)
angiogram. You also see something new which is the duration someone is
on a medication, and that happens because you\'re supposed to be on
beta-blockers after an MI for at least six months. That\'s a really hard
thing to figure out. And so that is a different animal, challenge. The
other thing you\'re seeing is allergies. Allergies is there primarily as
an exclusionary criteria. And the difficulty in EHRs is there\'s, in
most cases there\'s only one place to stick drugs that have caused
mischief in a patient. Most of the time the mischief is, actually most
of the time it\'s an upset stomach. So they\'re basically drug
intolerance or side effects, rather than a hypersensitivity reaction.
Because there\'s only one place to indicate that so that it will trigger
off the drug interaction or the drug allergy checking, that\'s a
limitation of the system and a limitation of standards. Because there\'s
multiple ways to code meds, there may be half a dozen commercial
systems, medication knowledge bases, and they all have different
identifications. NLM, for example, is working on RxNorm. That\'s an
example of a standard to come. Next slide, please.

So looking at those, looking at that analysis where you take high
priority conditions, look at the data elements that are included in
their calculation of the scores, and look at the quality of the data
from EHRs, we had the following recommendations.

One is if, with that problem list diagnosis recommendation that means we
would ask HITSP to identify standardized code sets that would help us
accurately represent patient diagnoses that would appear in a problem
list. Example, SNOMED is certainly one that is now available free of
charge and has been designed to capture problems and clinical conditions
better than the statistical classification system of ICD. ICD10
eventually will be here, but that does a better job, for example than
ICD9. One of the examples we used in asthma, thinking about the
interplay between clinical decision support and measurement, is you\'re
supposed to treat people with mild persistent asthma or above, but not
mild intermittent asthma. By golly, those codes don\'t exist in ICD9.
They do exist, for example, in ICD10. So that causes us to all to have
our, create dummy diagnoses, and that all of a sudden then makes these
systems and the data from them not interoperable. So that is a key
recommendation.

The second thing is this help with the medication allergies, and side
effects. How do we code the differences between the two? And how do we
find out uniquely what is medication across EHR systems? Also, some of
the systems only accept or at least convert a drug to a drug class, yet
you actually want to say something about a drug.

Next is this whole results of diagnostic tests and particularly the
LVEF, as an example. We need help in identifying standards that would
help us encode those results.

And finally, there are some results of other things like a consult,
which is sort of like a diagnostic test, and that would apply to
diabetic retinal eye exam. Next slide, please.

The other group we would like to address is really the EHR vendors or
CCHIT as one of our conduits to them. And one is to make it easy for us,
I explained the problem with allergies, make it easy for the clinician
to capture the distinction between a true hypersensitivity reaction
versus a drug interaction, a drug intolerance. Make it easy for us to do
that. Make it easy for us to code that difference. That will go a long
way to helping us with exclusions.

Second, make it easy \-- we can\'t just say, well, you must code LVEF,
cardiologists or radiologists, we need to make it easy to do so in those
departmental systems. That\'s again the software, the workflow.

Third, if you give us the tool to compose discharge instructions, which
may include information about heart failure and the care at home, then
don\'t make us separately record the fact that we did do that, make it a
byproduct of doing that. Just like it\'s a byproduct of ordering a med
to put it on the med list.

And finally this notion of if we want to tackle things like six months
duration of a medication, it would be better when we get to direct
linkages with pharmacy, to use that pharmacy data and the refills in
order to derive that duration. Next slide, please.

And through the course of this analysis, we find that there are ways to
up the yield from the quality development measurement process that would
produce measures that are, one, easy to obtain, two, are accurate and
reliable and three, give us the answers to questions we\'re interested
in. So one is to use problem lists as the means of identifying the
target population for (inaudible) data. Second is to perhaps share this
EHR data quality metric with the development organizations so they, too,
can assess what data elements should be in their measure to get high
quality results when you use the EHR. And then three, let\'s say it may
be time to reevaluate some measures that may produce low quality data
elements and hence give you low quality data anyway, or information
anyway. And maybe instead of requiring six, verified six months of
continuous prescription, maybe you say at three and six months as a
slice. There are ways that maybe give you actually better data if you
think about what data you have access to in the EHR. So next slide.

The final steps is what can we do different, what can we do next? So
we\'ve given you a set of analysis for high priority conditions, and
that accounts for about 40 percent of the measures actually. And we
could apply the same method and apply the same data quality metric to
the rest of the AQA, HQA measures. Another step would be to formulate a
quality measure strategy that would encourage the development of quality
measures that would consider the EHR as an origin for the data element.
And finally, we need to develop a strategy to migrate from
administrative claims-based definitions to ones that are derived from
EHR. Happy to answer questions.

[**\>\> Carolyn Clancy:**]{.underline}

Thank you, Paul. This is not the first time I\'ve heard you present that
you make an immense amount of work just sound logical and we all could
have done it, too. I have a specific question and will then turn to the
Workgroup. Functionally, I\'m stuck on why it is important to
differentiate between side effects and classic or true hypersensitivity.
I have a lot of patients who tell me they\'re allergic to codeine. What
they really mean is that they get nauseous, which is a known side
effect. In any case, they\'re not getting codeine.

**[\>\> Paul Tang:]{.underline}**

One of the implications is once you put codeine in there, or once you
put \-- once you put a codeine-containing drug, typically they will
start generalizing to a drug class and in the codeine example may even
go to narcotics. And all of a sudden you\'re faced with getting a red,
this person is allergic to narcotics or codeine or anything. And you
have to deal emotionally and with the accountability, your overriding of
that decision.

[**\>\> Carolyn Clancy:**]{.underline}

Okay.

**[\>\> Paul Tang:]{.underline}**

Many of the EHRs haven\'t even progressed to the points where you can
even indicate why you\'re overriding. They just see that okay now when
you get your next quality report, performance report, you\'re going to
see you ignore \-- actually, it was at Partners they said that most
people blow through 87 percent of their drug interaction alerts. Does
that look good on your record or not? And that kind of thing. So if we
can capture the true reason you are prescribing this anyway, we\'ll all
be better off.

[**\>\> Carolyn Clancy:**]{.underline}

This may be one example where paper is better because you can just write
allergic in quotes.

\[laughter\]

**[\>\> Paul Tang:]{.underline}**

Or nausea. Without the quotes.

[**\>\> Kristine Martin Anderson:**]{.underline}

Terrific work. Very impressed. It's Kristine Martin Anderson, Booz
Allen. I have a question about \-- I agree with your ultimate conclusion
which is that EHR data should produce better results for quality
measures, but it's in the adoption curve and we\'re not there and
quality measurement plowing forward, I wondered if \-- I think maybe the
DOQ-IT definition is so crude, have you looked or are you familiar with
some of the, there are some more extensive algorithms around, say, a
diabetic, that also looks at what tests were done, what drugs they\'re
on, and it\'s not just diagnosis-based and I wonder if there\'s work
been done or still work to be done around the questions where can a
claims stream get you relative to where we need to be.

**[\>\> Paul Tang:]{.underline}**

Okay. That\'s a good question. And in this study that I cited when we
looked at the different calculations, we used those algorithms based on
coded data and found that there was a highly sensitive and highly
specific measure, diagnosis on problem list, and chose to use that
rather than incurring other costs as you heard. Sometimes people can\'t
get all the lab tests, et cetera. That would be fine, but you still
require clinical systems actually to get a lot of those \-- the input to
that information. And you also have the disconnect between those who
enter the bill, and the diagnosis on the bill, from those who need, rely
on it for clinical care of a patient and basically it\'s the inaccuracy
of this separate parallel space that causes the errors. And even though
you can correct for some of the errors, why not actually start with the
clinical \--

**[\>\> Kristine Martin Anderson:]{.underline}**

I agree with that, if you have it.

**[\>\> Paul Tang:]{.underline}**

Correct. So age old question, okay, only 9 percent have it right now.
Well, so how do we change that? We can encourage by holding out the
carrot, but we also want to remove all the sticks that prevent you from
even getting to the carrot. And what I try to point out is, for those
who have it, you actually get penalized if you use the clinical data.
And so we have to actually overcome that disincentive for using your
clinical measures. So one possibility is \-- and first, we can\'t create
the clinical measures overnight \-- is to start creating this parallel
set of clinically derived measures. So that when the world is there, we
have something to turn to because it could take years. And the other is
actually to offer premiums for those who are reporting
clinically-derived scores. So that one, you can overcount this bias
against the clinical measure, and you can reward people for going that
direction sooner. So it\'s a combination of why do it now? It's to offer
an incentive to do it faster and take away the disincentive for even
doing it in the first place.

**[\>\> Kristine Martin Anderson:]{.underline}**

Okay, I definitely agree that we should do it now. I also am thinking
about how we keep it comprehensive. So your comments are very well
taken. Thank you.

And one other question. On your recommendations to NQF, I wonder if
those are really recommendations to measure developers. Is that \--

**[\>\> Paul Tang:]{.underline}**

That\'s a surrogate. It was really to measure \--

**[\>\>]{.underline}**

We're going to convene the major measure developers on a monthly basis
to talk \--

**[\>\> Kristine Martin Anderson:]{.underline}**

That makes perfect sense to me. Thank you.

[**\>\> Carolyn Clancy:**]{.underline}

Margaret.

**[\>\> Margaret VanAmringe:]{.underline}**

I had a couple questions. I know you chose just a few conditions. But if
these recommendations to HITSP and NQF were actually implemented and we
had some solutions to this, what \-- was there any discussion about what
percentage of the current performance measurement requirements out there
that this might help solve? I mean, in terms of the EHR.

**[\>\> Paul Tang:]{.underline}**

Tried to get this calculation right before coming here, but where we got
is the data elements cover about 43 percent of the total data elements
for all of the measures. I don\'t have \-- it covers about, it\'s
actually about 40 percent of the measures of the total set of AQA being
measured.

**[\>\> Margaret VanAmringe:]{.underline}**

Okay, and I had a couple other things. Was there any discussion of
format, of standardizing some of the formatting for some of these things
that might get done, so it\'s not just the element but how it\'s
actually encrypted, coded, whatever.

**[\>\> Paul Tang:]{.underline}**

That\'s a good question. That would be something HITSP would be more
equipped to handle. We were looking at the data elements.

**[\>\> Margaret VanAmringe:]{.underline}**

Okay. And I guess the other was from just an observation. Looking at the
problem list issue versus the coding, it was making me think of some who
have for years suggested that what we really need is a mechanism to be
able to scan physician notes at the intake, or the problem list, and
have some kind of free text recognition of the most common terms used by
physicians or other clinicians when writing down problem lists and if we
had that and could build that into the front end of the EHR, that that
might be valuable and I was just wondering what your comment might be on
that.

**[\>\> Paul Tang:]{.underline}**

Natural language processing is still a to be. Or like \-- it's probably
further than voice recognition, actually. So voice recognition is always
five years away, so this is probably ten years away, at any given time.

The other thing is you do want \-- once you teach them, by essentially
rewarding them with \-- I mean, physicians actually like the reports.
When they believe in them and get them in a timely way. So they can see
\-- what do you as part of this workflow and this soft part of EHRs, is
you reward them for doing the right thing. One of the ways to reward
them for putting accurate diagnoses on the problem list is to keep
leveraging that so it makes other things more efficient. Same thing like
in medication. So a medication list has got to be up to date for it to
be useful. How can we reward them? One of the ways we do it is we will
automatically process all the rules, when you get a prescription
renewal, if you have accurate med lists, we\'ll do most of that
processing for you. In fact, we will by protocol and nurse, substitute,
literally take work out of your hand and actually do a better job
because we\'ve looked at everything. Essentially it keeps feeding on
itself. If you make it a rewarding experience, and an efficient
experience, to enter the high value data elements, like problems, meds,
allergies, and then lab comes in for free, then it feeds on itself. It's
a constant feedback loop. Then we get to where we want to go. If we
never start it, and in fact jumpstart it, it\'s hard to get to.

**[\>\> Margaret VanAmringe:]{.underline}**

One last comment. I mean, I was struck by, under your NQF
recommendations, the functionality to capture the fact that there was an
issuance of discharge instructions and to just comment that there is a
greater opportunity to hear that at some point probably needs to be
considered and that is kind of what are the minimum data elements that
are needed in a discharge note when one patient is moving to another
part of the system, so that the capturing is not just that the discharge
instruction were issued but that these minimum items such as the
allergies and various things were captured as part of that discharge
note. So I think that\'s something eventually I would like to see in the
EHR, that minimum set.

**[\>\> Paul Tang:]{.underline}**

Well, I would take it a step further. So right now we\'re looking at a
process of did you write something down or did you hand something out.
It has nothing to do with was it there at the time you needed it. You
can go to the quote PHR side, and say, and look at what elements were
there or used or read, and say have we improved the overall care and
coordination of care through transition. And that would be \--

**[\>\> Margaret VanAmringe:]{.underline}**

That\'s a few more years away.

**[\>\> Paul Tang:]{.underline}**

Well, actually \--

**[\>\> Margaret VanAmringe:]{.underline}**

But it\'s a great thing to aspire to.

**[\>\> Paul Tang:]{.underline}**

Well, again, it\'s only as far as you want to make it. People can do
this right now.

**[\>\> Margaret VanAmringe:]{.underline}**

No, I realize. I would like to just have them make sure they have the
minimum amount of information when they hand over, it would be nice to
have, too. So thank you. Very interesting.

[**\>\> Carolyn Clancy:**]{.underline}

Other questions or comments? Thank you again.

**[\>\> Paul Tang:]{.underline}**

Thank you.

[**\>\> Carolyn Clancy:**]{.underline}

We may have to call on to you present to the full AHIC in the near
future. But this is really, really, really clear. So it\'s really
terrific.

We have one and a half more items left on the agenda before turning this
over for public comment. And so I\'m going to turn to Kelly Cronin to
talk about clinical decision support.

**[\>\> Kelly Cronin:]{.underline}**

I think we did talk about this in our last Workgroup meeting and I just
wanted to clarify to see if anybody had any questions about the e-mail
that went out the other day, which is asking for the Workgroup\'s input
on what we would be most interested in advancing in terms of target
areas or priority areas in CDS.

[**\>\> Carolyn Clancy:**]{.underline}

I think you all got an e-mail looking for your suggestions, inputs, and
comments.

**[\>\> Michelle Murray:]{.underline}**

It was from Lauren Kim who works here at ONC.

**[\>\>]{.underline}**

Oh, yesterday. I\'m sorry.

[**\>\> Carolyn Clancy:**]{.underline}

How could that be?

\[laughter\]

**[\>\> Kelly Cronin:]{.underline}**

I think essentially to recap on what we talked about in our last
meeting, we recognize that there\'s probably at least four or five
Workgroups that are interested in clinical decision support, and I think
our Workgroup clearly has it in our broad change to advance this area.
But it\'s also related to a lot of population health, or public health
issues, personalized medicine, to a certain extent how consumers are
going to be involved in shared decision making or the decision making
process. And certainly the EHR Workgroup is interested in this.

So we wanted to have sort of a coordinated way to look at this across
Workgroups, building off of the work that Amy has already done and put
forth in the CDS road map. And in the last couple months, in convening a
planning group, which is comprised of a lot of the Workgroup co-chairs
and some of the people who are most involved in clinical decision
support across the Workgroups, we\'ve decided to try to come up with a
framework for approaching this, but getting everybody\'s input across
the Workgroups on what they think is the most important to focus on.

We also in the interim are trying to get an update on exactly where the
market is, so where are electronic health records and other kinds of
health information technologies that are being used for clinical
decision support and get a sense on how sophisticated or how far we
still have to go, given that there is lot of concern about how well
integrated these are and to care delivery and how obtrusive or much \--
or how sensitive they really are to workflow. So we have Gartner doing
an analysis of exactly what\'s the state of the industry in this area.
We also are moving forward with a scan of where are all the agencies
across HHS, and really beyond HHS, across the government, where are they
in terms of supporting various activities for CDS. So whether it\'s
related to research projects or demonstration programs, or different
activities that CMS may have going on related to CDS. We\'re trying to
get an overview, really, of what are all the possibilities we have to be
building off of and coordinating this moving forward. So while AHIC is
probably not the long-term sort of coordinating mechanism for this area,
it\'s been recognized for the last couple of years that we need to do
more of this both across the agencies but also really across a lot of
people and industry and academia that are interested in this.

So anyway, the purpose for today was really just to make sure everybody
was sort of clear on what the ask was, and if we want to spend a few
minutes talking about, it I know that Jerry Osheroff in particular, if
he\'s on the phone, has thought a lot about this and how we might try to
focus or prioritize our activities in such a way that we could leverage
a lot of the things that are already happening at CMS, and elsewhere. So
that if we do take sort of a condition-by-condition approach, we might
actually leverage some more, or we might be able to achieve some more
progress.

**[\>\> Janet Corrigan:]{.underline}**

Kelly, that\'s helpful. I wonder if it wouldn\'t make lot of sense to
coordinate it the same way we\'re selecting the measures or conditions
for (inaudible) care, which is really starting from the IOM priority
list of 29, what, 16 of those are major conditions and then four
cross-cutting areas that aren't condition-specific, and you might want
to also go immediately after the two conditions that the NQF project in
priority areas is looking at because it would be nice if we actually had
an approach where we\'re pushing on the decision support side at the
same time we\'re looking at those two areas as pilots for, potentially,
episodes of care measurement.

**[\>\> Jonathan Teich:]{.underline}**

Janet, this is Jonathan Teich. I think that makes a lot of sense. We\'re
certainly looking for priority areas and have been both out of the road
map and out of the first meeting of the CDS group. It seems clear we
would settle in on one of the IOM's priority areas so I would certainly
support that.

[**\>\> Carolyn Clancy:**]{.underline}

And yeah, because I can brag a little more than Janet would ever be
inclined to. This actually came out of Crossing the Quality Chasm report
where I think there was a majority, although not overwhelming unanimity,
that one way to think about crossing the quality chasm was to identify
some areas where we knew what to do but there was a clear gap between
what was happening and evidence-based care. And so when Janet was still
at the Institute of Medicine, she led a group to identify I think 20
priority areas. And it\'s really great not only because of the list,
which of course is what everyone paid attention to, but also because
they put an amazing amount of thought into how one goes about setting
and updating priorities. So I do think it\'s incredible resource. I\'m
virtually \-- besides the Web, I know that we have multiple copies, I
have them all locked up. We have copies to share of the report.

**[\>\> Kelly Cronin:]{.underline}**

That would be fabulous. In particular there was some discussion sort of
offline about maybe focusing on one or two episodes or one or two
condition areas that might be, where we really could see some
incremental progress. And I think if we are going to be looking at
potential projects in the next couple years like funding a
public-private repository where we might have a set of executable
clinical practice guidelines or rules that would allow for best
practices to become computable, then it would be probably more feasible
to start with a fairly narrow focus. And test that out, get some
experience and then build on that. That\'s not to say that CCHIT
couldn\'t take on probably a little bit more, and I think that they\'ve
expressed some interest in trying figure out, you know, what over the
next few years should they be prioritizing in terms of functionality
related to clinical decision support that could be tied to maybe more
than one condition area.

**[\>\> Rick Stephens:]{.underline}**

This is Rick Stephens. One of the questions I want to raise and I\'m not
sure if having other clinical decision support, if it fits in these
areas for future use cases, but I want to continue to bring up this soft
issue, and that\'s motivating people to go off and do the changes we
think are necessary if they\'re going to be aligned into common systems.
My experience in change management when we\'re doing large-scale systems
is doing the technology is one thing. Getting the hearts and minds of
people to follow it is entirely different, particularly again when they
have different things that are driving their view of how to get things
done. So I would hope that we would think about how to go think about
that, so we\'re better informed about what is really driving each of the
major constituents to do what they do and that play a part in these use
case studies. And so is that \-- is this the right place to bring that
up or not or do we need to do that somewhere else?

[**\>\> Carolyn Clancy:**]{.underline}

No, I think it\'s exactly the right place. Just to build on Janet\'s
point about the utility of building on work that the NQF has already
started, I mean the New York Times had a story a couple months ago about
what a spectacular job we do with invasive treatments for cardiology.
We\'re getting door-to-balloon time down and the mortality rates you can
look at from 10, 20 years now look like ancient history. They should be
in a museum somewhere because we\'re so much better at it. Yet when
those same patients are discharged, 50 percent compliance is a real high
water mark. So it\'s exactly that kind of thinking about where are the
problems, where are the disconnects, particularly outside the either
integrated or federated worlds that we heard about earlier today.

**[\>\> Rick Stephens]{.underline}**

Yep. I agree \-- it\'s the federated world, the integrated world, it's
not just \-- how do you drive people\'s behaviors?

[**\>\> Carolyn Clancy:**]{.underline}

Yes.

**[\>\> Jerry Osheroff:]{.underline}**

This is Jerry Osheroff. Can I jump in with a comment?

**[\>\> Carolyn Clancy:]{.underline}**

Yes.

**[\>\> Jerry Osheroff:]{.underline}**

I think we have this incredible opportunity to focus on the outcome. If
you pick a disease, whether we look at MI, or CHF, or whatever. We\'re
hearing a lot of details about what great organizations are doing
wrestling with these problems, Paul made this terrific presentation
about getting the data together, and I think we have this wonderful
opportunity to focus on the ultimate outcomes here, things that
everybody cares about, mortality, hospitalizations, morbidity, all that
and sort of ask the question, how can we put all these pieces together?
The workflow pieces, the decision support pieces, the measurement
pieces. So that in the end we\'re driving those numbers from where we
know they are now and we know they aren't where we want them to be, and
asking the questions, what are all the things that we need to do to get
us from where we are now to where we need to be? And then all these
specific things like the electronic guidelines, the specific techniques
for measuring things, we can make those things better by doing some of
the kinds of things that I think this group has done well so far and
that we can build on. Namely, hearing from organizations that have done
best practice things in terms of driving things forward and then
disseminating those best practices, identifying what the gaps are with
the creation and dissemination of the knowledge and the measurement, and
then closing the gaps there. I think if we say that we\'re going \--
that our goal is going to be to get us from a specific place where we
are now to a specific improvement goal that we want to achieve
nationally, and then figure out how we can line up all the forces to
make sure we drive that advancement, then hopefully we\'ll be able to
touch all the bases and do all the things that are needed to drive these
measurable improvements that everybody is going to care about.

[**\>\> Carolyn Clancy:**]{.underline}

Great. Other comments? Janet?

**[\>\> Janet Corrigan:]{.underline}**

I think that\'s a terrific comment. And just to share with you a little
bit, the group that we have working at NQF on the priority and goal
setting pilot project that's focusing on two conditions in terms of a
very longitudinal episode of care, they\'re looking at low back pain and
heart disease, with long time windows. The latter one, HD, is really
AMI, but then looking about 12 months before, 12 months after so it ends
up looking at the full episode, is heavily, heavily oriented on the
outcome metrics, not only the readmission, mortality, but health
functioning.

And it strikes me that if we were to take those two conditions \-- and
that work would be completed this fall. We're really getting to the end
of it, fairly soon here. If we were to take even those two, the other
thing that is a real opportunity here is to broaden the concept of the
clinical decision support and be thinking, and maybe you already are,
don\'t know what your terminology includes, but to also be thinking
about patient decision support because so much of that piece really has
to do with not only the health behavioral change up front but the
compliance with the treatment plan and understanding of it. So it would
be terrific if we could tie all of these together, the two episodes that
we\'re making a lot of progress on that will be rolled out as pilots
accompanied by the decision support both for the clinician and for the
patient.

**[\>\> Kelly Cronin:]{.underline}**

Yeah, we have talked to, in the planning group internally here at ONC
about the scope of what we should be taking on and realistically and I
think a lot of people are in agreement ultimately we absolutely want to
be advancing decision support for patients, but we\'re fairly far away
from that, just in terms of a reality check. But it\'s not to say that
we couldn\'t be thinking about it at this point and doing some initial
planning.

**[\>\> Helen Darling:]{.underline}**

Yeah, this is Helen Darling, if I may make a comment on a couple of
these last points.

[**\>\> Carolyn Clancy:**]{.underline}

Glad to hear your voice, Helen. Welcome back from France.

**[\>\> Helen Darling:]{.underline}**

Nice to hear yours. I\'m very struck by the importance of this last
exchange and what\'s been said and would just add that if some of
what\'s happening, some of the information that\'s coming out, like some
of the wonderful reports we\'ve heard so far, has acceptability in the
professional community, so you have a lot of very knowledgeable people
who do this work, say, well, it\'s not perfect, but it rings true and we
know that this, these data do reflect underlying problems that also are
reasonably characterized. And that information then is publicized
through a variety of vehicles, including just people speaking and op ed
pieces and consumers guides and articles like the patients' inserts in
JAMA, and things like that, so that the word begins to get out that
there are issues, there are alternatives, there are ways to do these
things better. And the more this is known, it will keep driving change.
Because otherwise the more we keep it kind of back in the minds and
hearts of people who do this for a full-time living, and talk to each
other as opposed to try to get some of the word out to clinicians and
patients and consumers, then we won\'t make the kind of rapid progress
we need to make and we won\'t get the resources to support these kinds
of activities. So I just hope that we will kind of keep pushing for both
ever improvement on the technical side, but enough exposure to the world
that there continues to be momentum and eagerness to make progress, and
to invest in what we need to do to make the progress.

[**\>\> Carolyn Clancy:**]{.underline}

Helen, thank you so much for that and I have to say that I really share
your enthusiasm for Janet\'s suggestion about thinking now about how do
we link this to I guess what could be called patient decision support.
It has not been a prominent theme of our discussions at all and if it\'s
not, we are missing a huge opportunity, particularly as a lot of these
companies make investments in personal health records and so forth.

**[\>\> Helen Darling:]{.underline}**

And they\'re also making a lot in patient decision support, I mean
that\'s way up there in the tools that we consider essential to
improving care and supporting patients to make the best decisions for
themselves.

**[\>\> Jonathan Teich:]{.underline}**

Right. Helen, this is Jonathan.

**[\>\>]{.underline}**

May make sense to go after some of the patient decision support because
as soon as you shift your paradigm sort of to focusing on outcomes, it
may be they\'re more important than getting to the \--

**[\>\> Jonathan Teich:]{.underline}**

Yeah.

**[\>\>]{.underline}**

\-- clinician. So even though it may be hard, we may get a bigger bang
for our buck.

**[\>\>]{.underline}**

Be more malleable.

**[\>\> Jonathan Teich:]{.underline}**

It may be less hard than we think. We have, the patient decision support
is not absent, it just looks different from clinical decision support.
It\'s not based on drug interaction alerts and things that happen during
a CPOE system. There\'s actually quite a bit of patient decision
support. What I think it behooves us to do is think about it all in had
one unit we\'re trying to work out the standard approaches. Just the
same way that Paul is talking about trying to figure out what the
standard ways of representing quality metrics are, we\'re trying to
figure out the standard ways to represent decision support interventions
and I would very much include patient processes in this because it\'s
just a different type of the same thing.

**[\>\> George Isham:]{.underline}**

This is George Isham. On the issue of decision support, which clinical
conditions or topic areas we\'re going to pursue, the IOM list is a good
list for reducing burden disease and for areas of focus. But one list
that we\'re short on is a list of overused services. And that I think is
underrepresented in the approach used to deliver the, develop the IOM
list. I\'m thinking particularly with respect to decision support for
MRCT use, which we now have a community process in Minnesota for doing
which has it embedded in electronic medical records of a number of the
systems there. And is the contributing, I think, significantly to more
appropriate use of those tests. So I hope we\'re not going to forget the
very practical kind of list that I think we need to generate of high
priority areas that attack waste and overuse of care in \-- along with
the other suggestions that have been made so far.

[**\>\> Carolyn Clancy:**]{.underline}

Thanks, George, I think that\'s an important point. Recognizing our time
constraints, Janet, let me just thank you again. I know that you have to
go but this work was really fabulous.

**[\>\> Janet Corrigan:]{.underline}**

The committee did a wonderful job.

**[\>\> Kelly Cronin:]{.underline}**

I have to say for anyone who wasn\'t able to participate, Paul did an
amazing job leading this group and it was a very complex amount of work
that was done in a very compressed time frame and between I think the
conceptual framework that they work with and a lot of the staff-level
work that went into this, it made it very efficient and it was just a
really good exchange among a lot of experts. So \--

[**\>\> Carolyn Clancy:**]{.underline}

Didn\'t agree with each all the time.

\[laughter\]

A half an item and then we\'ll go to public comment. I\'m calling this
half an item because it\'s more in the spirit of a public service
announcement. And it will be by no means done today. I think many of you
are aware that Secretary Leavitt, who created and established the
American Health Information Community, has a very strong interest in
identifying a strategy, I think the phrase he\'s often used is to dock
it in the private sector. Now, this brings up all kinds of interesting
challenges. How do you continue the momentum that the current Community
has established? How \-- which functions of the Community are inherently
governmental and need to stay in government? If it is established in the
private sector, what is the business model for a sustainable enterprise?
How does one assure the ongoing involvement of senior Federal officials
who can make decisions and commitments to the enterprise? And so forth.
So needless to say, this has been a fairly lively topic of discussion
and in fact it\'s actually a fairly lively topic of discussion right now
on Capitol Hill. I simply wanted to tell that you this is very much a
work in progress and part of the reason you haven\'t heard Kelly
speaking earlier today is she\'s going to be submerged, giving this her
full attention for the next couple of months. Which only means that I
have to find her after hours for help with this Workgroup. But this is a
process that will play out. At the last Community meeting on June 12th,
there was a first discussion of it, but there will be more discussions
and we will be bringing that back to you. So it\'s not as if you\'re
going to be reading in your e-mail or in the newspaper, it\'s all done.
There will be opportunities for public comments, and can you remind us
when that is?

**[\>\> Kelly Cronin:]{.underline}**

Yeah, we\'re figuring out exact dates. It\'s changed from July to
August, because we really have a lot of work still yet to do in July,
and we want to have something that\'s really been vetted more
extensively by AHIC members through the month of July, prior to
releasing it publicly for comment. So it will be for at least three
weeks in August and we can get back to you with the exact dates.

[**\>\> Carolyn Clancy:**]{.underline}

And you will be notified by e-mail and so forth, but I just wanted to
give you a heads up that A, it\'s very much a work in progress, and B,
your input is very much desired. So the show is not over. The movie is
still playing out. And that was really the only purpose of my making
this announcement, which is why I called it half an item.

Public comment. Anyone, Matt?

**[\>\> Jennifer Macellaro:]{.underline}**

This is Jennifer. You\'ll see a slide up in just a second that's got a
phone number for people to call in who have been listening over the Web.
People who've been listening on the phone just need to press star-1 to
alert the operator. And there\'s an e-mail address there if anyone wants
to write in comments after the meeting. I\'ll check back in a minute or
two.

[**\>\> Carolyn Clancy:**]{.underline}

Thank you.

**[\>\>]{.underline}**

The Partnership for Health Improvement.

**[\>\> Kelly Cronin:]{.underline}**

Yes, that's what's in one of the bills in the Senate, but we actually
haven't yet come up with a name. We've been calling it the successor
organization, but yeah, we really don\'t have a name.

**[\>\> Jennifer Macellaro:]{.underline}**

This is Jennifer, I do have one comment. Could we get Lou Diamond\'s
line open, please?

**[\>\> Louis Diamond:]{.underline}**

Can you hear me?

[**\>\> Carolyn Clancy:**]{.underline}

Yes, Lou.

**[\>\> Louis Diamond:]{.underline}**

Hi, Carolyn. I firstly, I hope you guys are sitting in a room with
windows. It\'s beautiful sunny day. Last time I was at the meeting
physically it was a little cramped.

[**\>\> Carolyn Clancy:**]{.underline}

Sorry about that, Lou.

**[\>\> Louis Diamond:]{.underline}**

It wasn\'t for me. I don\'t attend the meeting regularly, I just hope
you have some windows there. Just two quick comments, if I may, Carolyn.
The presentations today were excellent. I would just suggest to you for
your consideration that revisiting the question of data linkages and how
\-- and some of the challenges in regards to data linkages and kind of
thinking through a framework such as how do we deal with the lack of
patient IDs, et cetera, could be very helpful.

My second comment has to do with the use of the problem list for case
finding for quality measures. And it makes imminent sense from the point
of view of the inaccuracies of ICD9, but I haven\'t thought through what
the implications are going to be for linking the quality measures with
the cost of care and eventually efficiency measures when the latter will
be using probably claims data and the current ICD9. So I think that\'s
going to be a very interesting issue for us to deal with going forward.

[**\>\> Carolyn Clancy:**]{.underline}

Thank you very much for that. I appreciate it. And I hope the next time
you come visit us, it will be a little less cramped.

**[\>\> Louis Diamond:]{.underline}**

It wasn\'t my problem. That\'s okay. Thanks.

\[laughter\]

[**\>\> Carolyn Clancy:**]{.underline}

Thanks. Okay, well, thank you all very much. This was really a terrific
meeting. And if the presenters are still online, thanks very much for
the work you put into this as well and have a terrific weekend.