**Transcript of Cyberseminar**

**VIReC Database and Methods Cyber Seminar**

**Measuring Laboratory Use and Results using the VA DSS National Lab Data**

**Presenters: Ann O’Hare, MA MD; Adam Batten, BA; Jeff Todd-Stenberg, BA; Daniel Bertenthal, MPH**

**April 7, 2014**

*This is an unedited transcript of this session. As such, it may contain omissions or errors due to sound quality or misinterpretation. For clarification or verification of any points in the transcript, please refer to the audio version posted at www.hsrd.research.va.gov/cyberseminars/catalog-archive.cfm or contact**virec@va.gov*

*Moderator:*Welcome to VIReC Database and Methods Cyber Seminar entitled “Measuring Laboratory Use and Results using the VA DSS National Lab Data.”  Thank you to CIDER for providing technical and promotional support for this series.  Today’s speakers include Dr. Ann O’Hare, Jeffrey Todd-Stenberg, Adam Batten, and Daniel Bertenthal.  Ann O’Hare is a staff physician, HS R&D investigator and associate professor of medicine at the University of Washington.  Jeffrey Todd-Stenberg is a research health science specialist at the VA Puget Sound healthcare system.  Adam Batten is a statistical programmer at the VA Puget Sound healthcare system, and Daniel Bertenthal is a statistician at the San Francisco VA Medical Center.

Questions will be monitored during the talk and will be presented to the speakers at the end of the session.  A brief evaluation questionnaire will pop up when we close the session.  If possible, please stay until the very end and take a few moments to complete it.  Without further ado, I am pleased to welcome today’s speakers.

*Ann O’Hare:*Thank you very much, Arika, for the introduction.  My name’s Ann O’Hare, and I’m a nephrologist here at, as Arika indicated, at the VA Puget Sound healthcare system.  Joining me in the presentation today are four of, three of my colleagues who really have traveled part of the journey with VA data use with me.  Our goals today are to—sorry.  We’re just trying to advance the first slides.  Not having such luck—

*Moderator:*You need to click on that arrow.  Yup, you’ve got it.

*Ann O’Hare:*Got it, okay.  Forgive us.  This is our first cyber seminar.  Our goals today are to really try to describe to you our experience using VA lab data for research.  We’re going to give you a very specific example from our own work of how these lab data can be used.  In this process, we’re hoping to give you an overview of different ways of working with national VA lab data.  Mostly what we’re going to be talking about is the DSS system, so we’ll lay out some basics about DSS and then describe specific projects that we’ve done using these data with the hope of giving you a sense of what the potential can be for these data sources, at least our experience of it.

Then we’re going to turn to talk a little bit about the corporate data warehouse and lab data available in there, which is an emerging, I think, parallel source of data.  Obviously both of these data sources originate from the same Vista lab package at individual medical centers, but there’s some differences in terms of the end user, in terms of what it takes to work with these data sources.  Then we will explain to you our own experience, which is really just a brief foray, so far, into using CDW for ascertaining lab data.  With that, I’m going to hand over the talk to Jeff Todd-Stenberg, who’s going to talk in more detail about DSS.

*Jeff Todd-Stenberg:*Yes.  We have a question here at the outset in terms of user participation.  People with experience or not having experience with DSS lab data, particularly for research.  There’s a poll going out on your screens that we would like you to record responses to.  Specifically, have you worked with lab data in DSS, and also more specifically, have you worked with serum creatinine data in DSS?  Let’s just take a minute to have a poll close on that before we proceed.

**Okay.  Actually, we’re going to go ahead and proceed now with the—I assume we’ll get the results of the poll shortly—

*Moderator:*I’m sorry.  Give me just a second and I will pull those back up.

*Jeff Todd-Stenberg:*Sure.  Okay.  It looks like about 38 percent said no on the first question and—excuse me, 38 said yes, and 61 percent said no, roughly.  We do have some representative users out there.  Creatinine itself, serum creatinine, it looks like a smaller population said yes, about 17.6 percent versus 82.3 percent saying now.  It’s an interesting mix of, for the group.  We’ll just go ahead and proceed, then, in our presentation.

**Okay, so let’s start out just basically describing what DSS is.  It’s, for the Department of Veteran Affairs, its central managerial cost accounting and executive information system.  Its primary purpose is to provide managerially useful information, for example, productivity measures, costs per unit of work, quality assessment.  There are different kinds of products that are created for the system, and those address different needs within the VA.  They’re very useful to lots of folks, including healthcare managers, Undersecretary for Health, Secretary of the VA and Congress, in varying degrees and based on what’s needed at the time.

DSS source data comes out of the core Vista systems at each medical center.  The Vista systems are, of course, the VA’s hospital information system.  Those are decentralized systems at each site.  They’re managed regionally within four regions within the VA, so the actual physical systems are groups together for an information technology management purpose at four central places within the VA.  Each of them contains information recorded at a specific site or their respective site.  They use standardized packages; in most cases, there’s some flexibility, but not particularly in DSS.

DSS collects a lot of source type of data through its particular method of capture within these Vista systems.  It captures a lot of financial data, it captures workload data, patient information certainly, and it’s eventually processed through different kind of algorithms that DSS uses.  It goes into a centralized database nationally.  At each site, it’s collected into the Vista systems.  There’s kind of a miniature version of the DSS data there, and then it gets ported to the national database.

Let’s go ahead and go to the next slide.  We have the core DSS database system, and then out of that system there are, for a variety of purposes and a variety of users including research and operational quality improvement, quality assurance purposes, there are a variety of national data extracts that are created out of the DSS data.  As you see in the chart here, they highlight some of the core ones used.  These, again, are along the lines of either financial, in terms of cost, accounting, or charge-based cost; workload statistics and records; and patient information, demographics, eligibility and characteristics.

Those get ported out to these national data extracts of DSS data.  Like on the right-hand side there, you see the DIS.  That’s the discharge file, which is the set of files associated with inpatient discharge activity for hospital stay.  There are outpatient files that capture outpatient workload records, and a pharmacy extract, which of course captures the types of prescriptions recorded, both inpatient and outpatient.  The RAI is, I believe that one is radiology; I think it’s really RAD, it’s just kind of hidden behind the LAB.  That is a brief, it’s not a GUI type of interface, but it does record certain kind of procedure codes associated with radiation treatment.

Then on the right, we’re focusing more on the LAB and the LAR.  The LAB is essentially the laboratory test information and the LAR is the laboratory result information.  Of course, they’re related, but sometimes people are more interested in the result one, and you can always tie back to what kind it was by a code number.

Okay.  That’s kind of an overview of that.  Hello, we must have hit the wrong thing.  Share my screen—

*Moderator:*No, let me—I’ll put it back up here.  Give me just a second—

*Jeff Todd-Stenberg:*Okay?  All right.

*Moderator:*There we go.

*Jeff Todd-Stenberg:*The national data extract for LAB, these are clinically oriented and, I guess, utilization oriented in a different way, workload and costs associated with laboratory tests ordered and completed in the VA.  The laboratory result files contain results for a defined list of tests.  Currently there are 91 types of test whose results are in this file.  Now, it started out, there were like 16, or something like that.  Then they added around 30 or so.  It’s grown over time from the early 2000s to present day.  They’re continuing to add additional tests that are of benefit to, again, policy makers, researchers, quality improvement folks and so on within the VA.

**They schedule these DSS national data extracts out of the DSS system on a monthly basis, sometimes quarterly basis, and they also maintain cumulative files that are year-to-date while the year is building.  Right now, we have lab data in result form from the fiscal year 2000 and the lab test data from 2002 forward.

Lab tests used in our work here specifically, we focused a lot on—for the type of work that Ann O’Hare is doing—we focused a lot on these kinds of labs, for example serum creatinine, urine albumin to creatinine ratio, in some cases hepatitis C antibody, whether it was present.  You see on the right these DSSLAR number, laboratory result number.  31, 56, 89.  These are essentially the numbers associated with the lab tests.  The list of 91 that we referred to has a number for each type of test.

Those are standardized, and it’s very handy the way they’ve done that.  It makes it very easy for the researcher to work with the data, because even though the results may have little difficulty in terms of the types of information—not all numeric or not all character—at least you know what test it is.  For some other systems, which we’ll talk about in a while, test results or even test names vary broadly from Vista system to Vista system.

The DSS national data extract formats—okay, so we’re, we have different kinds of, some historical references and some current references.  The reports and data cubes at the VHA Support Service Center, also known as VSSC, does have the ability to have drill-down type reports or aggregate reports and what they call fact tables or data cubes so that the user, appropriately authorized, can generate, based on underlying connection to various types of DSS national data extract data.

The SAS datasets were traditionally on the—stored and made available in SAS format on the Austin Information Technology mainframe, but due to policy changes and cost considerations, they were removed and discontinued from being created in that environment in SAS format after the end of fiscal year 2012.  I believe the last files were removed essentially, or access to the files, in March 2013.

Now, all of those historical files and ones built since then are—the historical sets, data sets, are in the VINCI environment, which is kind of related to CDW but a little bit different.  That’s for the 2000 to 2012.  In additional, all of those older data sets plus any new data sets created for DSS, all are on the corporate data warehouse SQL environment.  There’s an advantage to that, too, in so far as the naming conventions have been more standardized for those files.

For a user today to make a request for data that’s current, really, DSS data extracts in the, from the SQL system and CDW would be the way to go.  For more historical references, you can still get access to the—well, you can get, download extracts from a VINCI program or for the SAS datasets through VINCI.

Okay, I’m going to turn this over right now to Ann to talk a little bit about this next slide.  What we’re doing here is we’re kind of transitioning to a real time result of having worked with some of the data that we’ve just discussed in general.

*Ann O’Hare:*Thanks, Jeff, that was great.  Just as an anecdote, I started working with DSS data back circa, I think, 2003.  Dan Bertenthal, who is one of our presenters, and I were working together in San Francisco, and the San Francisco *REAP*had designated some funding for Dan to help me with my research.  I was just starting out a VA career development award.  It was really perfect, because we did not have expertise in terms of working with national data sets for research at our center.  Some operations programmers had been very generous with their time, but that wasn’t going to be possible for sustaining a research program.

This is a way for me to do some of the work I wanted to do and for Dan to build his skills and expertise, which have actually gone on to serve many other researchers in San Francisco.  Dan will talk a little bit later about some of the limitations of DSS data.  The way this started was that Dan and I went down to the Palo Alto VA to attend a seminar on DSS by the HERC group.  I don’t think we had any particular goals in mind other than to learn more about VA data.  As we were sitting there, they started to talk about particular lab tests coming online nationally via the DSS laboratory results file.  I kind of pricked up and I was, asked a question—I put my hand up, and I said, “Which labs?  I mean, does this include, for example, serum creatinine?”  They were like, “Yes, I think serum creatinine’s in there.”

To me, that was really sort of a game-changing moment, because for a nephrologist, having information on serum creatinine is just an incredible, incredibly powerful tool, as compared with diagnostic codes for chronic kidney disease, which have the same limitations as diagnostic codes for other conditions, except perhaps more, because many people don’t necessarily recognize the presence of the condition.  Then it’s very hard to gauge severity of disease from diagnostic codes, especially at that time.  This really prompted us to start working nationally with DSS, and we used the very earliest form of DSS, which were SAS files organized by *VISN*and by year, so fairly cumbersome, but, once the data had been downloaded, fairly straightforward to identify the relevant tests.

Of course, we had to do some cleaning and get rid of unreasonable values, but we were able to assemble cohorts of patients from the early 2000s with information on renal function.  This translated into about 60 percent of veterans using the system.  We went ahead and started to do some work.  I’m showing you a slide here, the details of this slide are not important, but this is just to say that one of the first studies we did was looking at prevalence of kidney disease in veterans by level of GFR and by age group, which happened to be an interest of mine and something that seemed important to learn about from my clinical work.

The real strength of these data is, we were able to really look in very fine categories of GFR and age, and really try to understand some of the differences, potentially, in the relationships of—the relationship of GFR with different outcomes across age groups.  Differences and similarities.  This was our first study, and—using these data.

Did you hit the—sorry.  A simple advancing of the slide is proving—our next, because at the same time that we were working with DSS, Denise Hines’ group at VIReC was working very hard to acquire USRDF data.  USRDF is a national registry for end-stage renal disease.  They essentially, anyone who starts chronic dialysis or receives a kidney transplant enters this registry.  VIReC was able to, as part of the VA-Medicare data merge project, was able to acquire information on end-stage renal disease for veteran cohorts.

At some time after this initial study, where we—in our initial study, we looked at mortality outcomes.  We were then able to do another study in a cohort of veterans with kidney disease, a national cohort, and look at time to end-stage renal disease and the incidence of end-stage renal disease compared to death.  Again, we were able to look by fine age group categories and level of GFR, which allowed us, I think, to describe some relationships that hadn’t been quite described in other cohorts where there wasn’t that ability to fine, for fine stratification.  This was an exciting project for us early on.

One of my colleagues, the late Andy Choi, a very gifted nephrology researcher who unfortunately passed away in 2010, did some very elegant work.  He acquired the access to the immunology case registry for the VA, which—excuse me.  I’m jumping ahead.  This is another—apologize—this is another of his projects, where he actually looked at the relationship between race and end-stage renal disease and death among veterans with different levels of GFR.

Again, I think what you can see is that he was able to describe nuances, I guess, in the relationship between race and the incidence of end-stage renal disease and death in veterans by level of GFR.  For those of you not in the field, this is of interest because end-stage renal disease and kidney disease disproportionately affects African Americans.  Again, this is a, I think an example of the power of the VA data, having system wide information on some basic demographic characteristics, level of GFR and outcome information on end-stage renal disease.

Another colleague of mine, Judy Tsui, who at that time was a fellow in general internal medicine, had an interest in hepatitis C.  She did a very elegant study—and again, the details of this don’t matter.  It’s more just to sort of describe the broad scope here, was that she looked, again using our data on renal function, in combination with information on hepatitis C testing in the VA, which is actually available through DSS.  It was one of the lab numbers that we, that Jeff mentioned at the start.  It essentially told us whether patients were positive or negative for hepatitis C antibody.

Again, because hepatitis C screening is fairly widespread in the VA compared perhaps with other health systems, and also the prevalence is relatively high, we were again able to look at the relationship between hepatitis C and end-stage renal disease, again, by fine level of GFR and by age group.  This is important because this slide actually just shows that there were significant interactions by age and eGFR, which would have been really missed if we hadn’t been able to stratify like that, or at least wouldn’t have been as easy to understand.

Then this is a study I started to tell you about.  It got a little confused; I’ve been changing the order of slides a little bit.  Is Andy Choi’s study where he linked the immunology case registry to our data on kidney function in veterans.  The immunology case registry is a, it’s a sort of a parallel data source that collects very specific information on HI, veterans with HIV positivity.  Essentially, at least at that time it was difficult, there were a lot of steps to jump through to get it, but Andy was very persistent and was able to acquire the data and then I think do some very original work, looking at the relationship between HIV positivity and risk of end-stage renal disease in veterans.  It is, again, for those of you not in the field, HIV nephropathy almost—occurs almost exclusively in African Americans.

What he found in this study was that it was more or less only the combination of African American race with HIV that placed people at increased risk for end-stage renal disease.  Whites with HIV were actually at no greater risk for end-stage renal disease than whites with other co-morbid conditions, and here he used the example of diabetes.  Again, an example of how these data are very flexible, can be combined with a lot of different data sources within the VA.

I, at some point in 2007 I moved from San Francisco to VA Puget Sound, and we did some—this is a sort of a continuation of our work using VA data.  We wanted—there was a growing interest within the field of nephrology in proteinuria or albuminuria as a predictor of adverse outcomes.  At the time, the corporate data warehouse was not available for researchers or data from the corporate data warehouse was not available, lab data.  We looked to DSS to see what we could find, and in DSS there was information on albumin to creatinine ratio, which for the clinicians in the audience, this is a very common test if your patient has diabetes.  In fact, there are guidelines about how often you should check urine albumin to creatinine ratio.  It provides a pretty good quantitative estimate of how much albumin there is in the urine and is part of the classification system for kidney disease.

When I got up here, this is one of the first studies we did.  We did this in collaboration with folks at the CDC, Desmond Williams’ group at the CDC has supported this research.  We were able, again, to look by fine stratification of eGFR, level of urinary albumin and age, able to look at mortality outcomes as a function of those characteristics.  We looked only in patients with diabetes because, especially at that time, the urine albumin to creatinine ration would not routinely have been checked in patients who did not have diabetes.  This was somewhat limiting, because we would have like to have looked in a more broadly defined cohort.

Then finally, I think this is the lab study I’m going to describe before we move to a discussion of the limitations of DSS and then transitioning to a discussion at the end about CDW lab data.  This is another study that my colleagues up here helped to contribute to.  What we did here was we used our registry data from VIReC, from USRDF, and then we looked back among patients who started kidney dialysis or received a kidney transplant.

We looked back at what their trajectory of kidney function had been prior to initiating dialysis or reaching end-stage renal disease.  This is, again, for the—people with a nephrology background will be more familiar with this, but our registry includes really good, detailed data collected at the onset of end-stage renal disease and pretty good outcome data on whether people get transplanted or what their survival is.  But in terms of the pre-dialysis period, what happens to people before, there’s a limited number of measures and there’s no information on the trajectory of kidney function.  There’s some information on the level of kidney function when people start dialysis, but not looking back.

We copied shamelessly from a paper in the *New England Journal* by Tom Gill that looked at functional trajectories before death using a SAS procedure called TRAJ to look—trajectory modeling, group-based trajectory modeling.  We could have divided patients into different trajectories of renal function before initiation of dialysis and found very strong correlations with all kinds of care practices within the VA, depending on how quickly your illness had progressed, or your kidney function had been lost.  Again, just to give you an example of other opportunities that I think these data provide, that would otherwise not be possible in the national system, at least.  I’m just going to turn this over now to Dan to talk just in general terms about some of the strengths and limitations of DSS lab data.

*Daniel Bertenthal:*Hi everybody.  Okay, so the key asset of DSS, I think really as we’re kind of going into the CDW era, right, is simply that it goes much further back.  You can set up cohorts with a lot more follow up time than if we’re using CDW as the sole source.  That’s why it’s going to continue to stay relevant.  Along those lines, I think I’m going to focus more on some of the issues, especially kind of from the earlier years of the data.

When DSS came available in the early 2000s, the key strengths were, especially at the time, was that DSS data were automatically being captured simply because clinicians ordered the tests and results were delivered through Vista.  It’s a change from ICD-9 coding, in the sense that when clinicians are very busy, it’s not really discretionary that the data get recorded.  That’s the upside.  But you sort of turn that around, and the key issue is how the data were mapped.  I think as pretty much all of you know, is how well local Vista data are mapped to national data is just a huge, over-arching theme that kind of applies to everything involving the VA data.

Certainly looking back to some of the early years of DSS, the mapping was a pretty gradual process, and especially between the LAR and the LAB, it was relying on the local Vista staff to go in and to map all the correct tests.  Kind of an added problem with the DSS LAR is that compared to the other parts of DSS, it wasn’t a fiscal extract.  With DSS being mostly a financial system, it sort of didn’t have kind of that same priority, which—the account which the other parts of DSS for accounting had.

One pattern, for example, that you can observe a lot with the earlier parts of DSS, is that you can see entire facilities reporting no data for entire years, for parts of years, or sometimes you can even get other patterns.  For example, some of the VAs, the stations which have more than one medical center.  Example would be upstate New York, is that you can be finding that some medical centers are reporting and others are not.  To kind of cover that as mapping, yes, it applies to all sorts of VA data, but that’s especially something that if you’re really taking advantage of DSS, to go back to the earlier years, that you have to really account for.

An upside to that is that the missing data patterns tend to be more an issue of facilities and time period than patient-specific factors.  The last point, the lack of transparency that the mapping process being a black box, is to some extent, yes, there’s simply not a whole lot that you can really do. One approach a bunch of us have tried, and there’s some—and some people have written about this in the literature—is when people have tried merging the LAB and the LAR since for most of their history they were set up using a different method, and so to the extent that you’re getting data from both sources, it’s kind of a nice indication that hopefully you’re getting fairly complete data.  It’s still not a guarantee, of course.

Whereas I think with CDW in contrast, there’s just, I think there’s some more points along the way where you kind of, you take a look at some things aren’t necessarily being mapped right or being mapped the wrong thing, there’s a little bit more of a chance to kind of go in and try to set that straight.  It also has a plus side, which is simply that DSS data are a lot more simple, so—so, for better or worse, the data, they are what they are.  Where you have the data and where you’re seeing the consistent, plausible results that touched on.  It can be a more expedient process than going through CDW.  That can be an advantage there.  Okay.

*Ann O’Hare:*Great.  That’s great.  Thank you, Dan.  That’s great.  I’m going to actually—we wanted to talk a little bit, because I think this is relevant to what we have to say in a few moments about CDW, but I’m just going to turn this back to Jeff briefly to talk a little bit about the mapping process for DSS and how that has evolved over time.

*Jeff Todd-Stenberg:*I think Dan touched on it highly in his comments a minute ago, but I’ll focus specifically on something called the LOINC code.  It’s also called a Logical Observation Identifier Names and Code.  LOINC is a standardized set of codes, national codes.  It’s a proprietary set of codes by a company in Dallas, Texas that the VA uses, lots of organizations use it.  It provides a very standardized way of mapping tests or recognizing tests that are by those codes it references.

Now, we’ve used these LOINC codes in the Vista system for a long time, but from time to time they get out of synch with kind of a centralized DSS-LOINC file.  Then there are the historical references of not really using these particularly at all solely, but having the local staff at the Vista systems there in DSS map the codes in a standardized way themselves.  In 2009, the VA and the DSS system within the VA made it a standard to use the LOINC codes and to my understanding, they required all Vista sites to use the same reference file.

This doesn’t correct any problems that were from 2000 to 2008, but it does help towards standardization of 2009 forward.  Based on this change, we are able to see a little bit better match between the LAB and LAR records compared to previous methods based on the testing.  The final note there, I already talked a little bit about.  This code, and we’ll talk about this a little bit later when we talk about CDW data, this code is again something in the Vista system, and it’s basically, it’s also available through the CDW extract.  There’s a mapping in Vista of this file to the laboratory workload code file in Vista, which is also known as the national laboratory test file.

There are codes in CDW, lab domain, for the VA national lab code.  There’s a tie together there, both in DSS and also in the alternate CDW environment.  I’m not going to actually talk about these a lot anymore, but if you go to the VIReC website and you’re looking at information on DSS lab, you’ll find the list of the 91 tests that are currently available for which we have results.  You’ll see on the right-hand side all the LOINC codes, ranges of codes related to each test.  This is kind of a current standard.  Historically, there may be differences, but I think from 2009 forward we’re in better shape.  Did you have any comments on that, Dan, or did it—does that sound like your understanding as well?

*Daniel Bertenthal:*Yes.  I think certainly switching over to the LOINC has been great, because it means that all the efforts to standardize are kind of—it’s sort of a combination of efforts.  When you compare it with previously, when there was a separate—right, there was a separate system for LAR, there’s a separate system for LAB, yet at the same time, LOINC was already being used in some of the case registries.  It was a lot of duplication of effort and with the result that you could have different things being mapped to different systems.  Yes, by going through the same LOINC method it’s been a very big improvement.

I agree.

*Ann O’Hare:*You wanted us to finish at 10:45, is that correct?

*Moderator:*You can keep going, yes.  There’s a good 20 minutes, so you can keep going with you slides.

*Ann O’Hare:*We like to save some time for questions, but it would be nice, because I do want to spend some time on the CDW lab data, because I think moving forward a lot of people are going to be looking to potentially work with that.  We did start out with some audience poll questions here, the first one being have you worked with lab data in CDW, yes or no, and have you worked with serum creatinine or proteinuria data specifically in CDW.

Actually, it’s a really—that’s the current poll, right?  That’s very similar, actually, to the previous one.  It looks like the majority of people have not, three quarters of the people have not worked with CDW data and of those who have, perhaps 20 percent have worked with one of these renal measures.  That’s pretty helpful.  Okay, good.  How do I get the—okay, perfect.  Okay.  Moving on, I’m going to just say—Adam, I’ll just say a few, general things about CDW data, and then I’m going to give an example of how we use it.  Then I’m going to turn this over to Adam Batten, who has been on the front line of using this.  That’s a good choice of words!

*Adam Batten:*Well said, actually!  Well said.

*Ann O’Hare:*I will, I want to give Adam a chance to get into the sort of nitty-gritty, some of it, anyway, because I think it will give people a little bit of a feel if they haven’t worked with it before for what some of the issues might be.  Just to give you some background, CDW is, I would say, a parallel and increasingly relevant source of lab data in VA.  Essentially, as I had mentioned before, both DSS lab data and CDW data start out the same.  DSS data undergoes some processing by DSS that hasn’t always been as transparent as perhaps it is more recently, and so a little bit of a black box there for something happens.  There have been concerns about full capture of DSS data compared with CDW, which is the raw data coming out of the Vista lab package.

Just important for you to know that the source for both of these data systems, lab values in these data systems is the same coming from individual medical centers.  The advantage, CDW has a number of advantages over the DSS.  One is that it includes all lab data, so it’s not restricted to that list of 91 tests.  I’ll explain how this was relevant to us in a minute.  I would say the disadvantage, or as Dan sort of nice, put it nicely, it depends on your point of view and you priorities whether it’s a disadvantage or an advantage, but CDW data is basically what goes in in a Vista lab package.

There’s a lot of variability across medical centers; there’s a lot of different test names for the same test, the change over time, etcetera.  A lot more work on the front end for the end user in terms of actually being able to work with these data compared with DSS.  These data do go back to 2000.  They started coming online I think in the, around 2010?

*Adam Batten:*2002.

*Ann O’Hare:*But when we started, it was 2010-ish

*Adam Batten:*We selected for your study a little bit later, yes.

*Ann O’Hare:*We were waiting for these data, and I’ll explain why.  To a nephrologist, as I had mentioned, we wanted information on proteinuria in a wider cohort of patients, not necessarily with diabetes.  For that, you really have to look at, for a dipstick protein measurements, because these are going to be much more widely ascertained than what we call quantitative measurements of proteinuria, such as protein to creatinine ratio or albumin to creatinine ratio.  This was the, our trigger to go into the corporate data warehouse.

We did so, I want to say, around 2010.  We were literally waiting for these data to come on line, because CDW had not, when we wanted, started to want to do it, but the data were not available.  As it came on line nationally, we started to work with them.  We were eventually able to publish our work.  We also were waiting for some updated linkages with the registry that didn’t come for a little, didn’t come until sometime in, I think, 2012.  Eventually we were able to publish a study where we looked at a large cohort of veterans with information on their level of proteinuria.

That was, for us, that was for us the impetus we needed to grapple with the CDW data.  We just could not get these data in DSS; they weren’t there.  That’s I think, a good example of one of the advantages of CDW data.  I’m going to just, in favor—well, actually, I think Adam, these are your slides, so I’m going to turn this over to Adam to sort of conclude this.  I have a concluding slide at the end, so I’ll come back and sum up everything.

*Adam Batten:*Okay.  I’ll try to be, move through this pretty quickly.  It’s kind of technical, but—so this was my first assignment to CDW, I guess you could say.  First time playing with the data, first time seeing it.  Didn’t really know what was in there.  I kind of just took this process here, you can see, when I got the table, so I would query the lab table starting at the top.  Then look at the results, kind of map them to a table and see if they made sense.  Then take out any that were obviously strange or bizarre, that didn’t really map to proteinuria.  Then anything that was kind of outside that realm, I would take it back to Ann to see if it kind of maps up with her clinical background and her experience, to see if they were actually relevant, and then update the query to include those that were, and then cycle back through that entire process again.

Until we got, this is the next slide here, thousands of potential lab tests.  In CDW, because it’s free text, whatever whoever’s typing into Vista goes directly into CDW.  You kind of have to take that into account.  You have to have a pretty solid string search algorithms to pull out the meaningful information.  Like, you would hope that second row there, urine protein, you would hope that that would encompass a hundred percent of the labs, ideally.  But in this case, it was actually around 80 percent or so.

Then also within these tests, you can have text information or numeric information.  I mean, this doesn’t really apply to a lot of labs; it’s kind of unique to urine protein, but it’s something to watch out for if you’re, if you do receive a raw table from CDW.  That there is actually meaningful values in the text field.  Kind of a caveat, if you’re going to trust the numeric result, but verify through the text result.

Moving on to the next slide—yes, this just kind of gets into what’s actually in some of these proteinuria tests.  You should see there, it’s an awful lot of text.  A lot of different spellings.  You can have punctuation in there.  Most of the values are pretty good, so like negative, neg, that’s 35 percent of the data.  It’s variations on spelling of negative traits you kind of have to watch out for.  Yes, and then again, just taking out those oddities, the punctuation, whatnot and then taking other values back to Ann to review.

Here’s a quick poll.  I don’t know if we have time for this, but I was just curious to see what the familiarity is with regular expressions.  Okay.  Okay, great.  That’s pretty good.  Yes, regular expressions were invaluable for this problem.  I’m just going to go ahead and end it now, if I can.  Okay.  Then here’s just the logic that we put—I mean, obviously some of these are redundant.  I just wanted to show you pretty much every step that we went through to get the final meaningful lab results.  These are a lot of regular expressions here, kind of stepping through every possible bizarre value and also every possible spelling that you could come across for these labs.

The end result was pretty good.  I mean, would I use the same process next time?  Yes, but it needed to be updated because of the nature of the CDW is just constantly changing.  You kind of have to develop your method, but keep in mind that those will need to be modified down the road.  Yes, in the future now that the LOINC is good and is in CDW, I’d probably leverage both.  Do the LOINC poll to get your kind of base subset of labs and also combine that with the same text search that you saw before.  I think that’s pretty much it.  Ann, did you want to take over?

*Ann O’Hare:*Sorry, I was talking, but I was on mute!  Sorry about that.  Thank you very much, Adam.  We are—hopefully what comes across is, from Adam’s presentation and the other presentations earlier is that this is an evolving data environment.  I think what—our process may not be that relevant to people getting into this right now, but I think hopefully take home message is that you’re aware that there are now probably, there is not probably much more choice, much more options for how to work with VA lab data than there have ever been.

I think the availability of CDW lab data is obviously a huge benefit.  It does overcome many of the limitations of the DSS LAR file.  At the same time, I think it’s important to recognize that the process for building the DSS LAR file is becoming more transparent.  Depending on your goals, I think, it can certainly be a simpler process in terms of working with the data.  The final point is that hopefully we’ve communicated is that lab data from CDW require a lot more manipulation and decision, micro decisions, by the end user to transform it into a useable format compared with DSS.

That can be an advantage and a disadvantage.  It certainly might make it difficult to replicated the findings of one particular group, because there are just, I think, so many decisions that have to be made.  At the same time, I think if you want to know what those decisions are, you don’t want to just have blissful ignorance like we have working with DSS, then it makes sense to go to CDW.

We just wanted to, at the end, we’ve included a couple of good sources for learning more about both of these data sources.  The HSRData listserv, VIReC help desk, these are all, I think, really important resources for the research community in the VA.  These are also some SharePoints that you may want to familiarize yourself with.  Then we do need to also, I think, advertise an upcoming seminar for May 5th.  With that, I think we are—I thank Jeff, Adam and Dan for helping to present this, because it really has been such a team effort to get to work with these data.  We’re open for any questions.

*Moderator:* So far, we just have one question.  For the attendees, if you have questions, please type them in the box now.  The first question is, how do we access CDW?

*Jeff Todd-Stenberg:*Are you—if you’re talking about a research access, there’s a portal called a VHA data portal that, if you don’t have, if you don’t know the address to that, the web address to that, VIReC can provide that.  That provides the mechanism based on the type of data that you’re requesting to start something called the dart application, which is data access request tracking system.

Essentially, it boils down to you define the requirements of what your protocol is after.  You fill out a domain questionnaire, you provide supporting approval documents, and eventually when it’s approved by a national data systems to access the CDW data, a VINCI programmer will be assigned to work with you, take your specifications or list of cohort subjects by scrambled or real SSN, and they go up and mine the data and return it to you.  The data returned to you is based on your protocol and cohort design or specifications or subject list.

*Moderator:*Great.  Thank you for that.  Next question, are the data mapping-sash-data cleaning routines that DSS uses to clean test names available or documented anywhere?

*Jeff Todd-Stenberg:*There are DSS resource user guides that VIReC has available that probably gets as close as you can get without looking up the programming globals in Vista.  I think there’s probably still some technical limitations on what goes in the *Resource User Guide* that describe the data sources, but it will give you more background information.  The actual technical procedures are written in MUMPS in Vista.

*Moderator:*Next question: what specifically is reported from a work load cost perspective?  Essentially, what is tracked for lab work load?

*Jeff Todd-Stenberg:*Certainly the cost of the, there’s some cost of staffing.  There’s a cost of running the test.  All of these DSS costs by subgroup are essentially aggregated by the work done on that test on that day.  They include fixed direct and indirect cost, variable cost—there’s a whole slew of cost.  In the RUG, they do break that down to some extent in terms of describing it.  That’s the *Resource User Guide*.  Again, I would point you toward VIReC to get more specific detail.

*Moderator:*Lastly, we have a request from one of the attendees.  She’s asking if you could draft a document for data users.  She says the slides are too brief for use as a reference, but you have many helpful comments.

*Jeff Todd-Stenberg:*That’s very kind.  I’m not sure; we would have to talk, I think, with you, Arika and—I’m certainly happy to talk online also with the questioner.  Is that Grace?  Yes, it looks like Grace.  Feel free to email me or Jeff.  My email is just ann.ohare@va.gov.  No apostrophe.  We don’t have anything written up right now, but it’s certainly a good suggestion and perhaps something that we could talk with VIReC about.

*Moderator:*Great.  Thank you so much to our presenters for speaking today.  Our net session is scheduled for Monday, May 5th, from 1:00 to 2:00 PM Eastern, and is entitled, “Assessing Race and Ethnicity” and will be presented by Maria Mor.  If you have additional questions for today, please contact the VIReC help desk at virec@va.gov.  I hope that everyone has a great afternoon.

*[End of Audio]*

Page 1 of 15