**Transcript of Cyberseminar**

**VIReC Good Data Practices: Research Application**

**Presenter: Laurel Copeland, Ph.D., M.P.H.**

**September 12, 2013**

*This is an unedited transcript of this session. As such, it may contain omissions or errors due to sound quality or misinterpretation. For clarification or verification of any points in the transcript, please refer to the audio version posted at www.hsrd.research.va.gov/cyberseminars/catalog-archive.cfm or contact the VIReC Help desk at**virec@va.gov**.*

Moderator: Good morning and good afternoon everyone and welcome to day four of VIReC’s Good Data Practices Mini Series. The Good Data Practices Mini Series is a set of lectures that are focused on good data practices for research. The series includes five sessions that will take place this week, Monday through Friday, from one to two p.m. Eastern. The last two sessions of the series, beginning today, will be presented by experienced VA researchers who will describe their own research and how planning and documentation were applied at various stages. New researchers and research staff planning new projects may find this series beneficial. Thank you to CIDER for providing technical and promotional support for this series. 

Today’s lecture in our Good Data Practices series is a research application presented by Laurel Copeland. Dr. Copeland is associate director for the Center For Applied Health Research and a research investigator both in the VA and at Scott & White Health Care. She is also active in the HMO Research Network, holds an appointment as Associate Professor at Texas A&M in the College of Medicine and School Of Rural Public Health, and is currently interim associate chief of staff for research in the VA at Temple, Texas. 

Questions will be monitored during the talk and will be presented to Dr. Copeland at the end of the session. A brief evaluation questionnaire will pop up when we close the session. If possible, please stay until the very end and take a few moments to complete it. I am pleased to welcome today’s speaker, Laurel Copeland. 

Laurel Copeland:Hi. This is Laurel Copeland. And just to recap, earlier this week you've looked at early data planning for research, managing and documenting data workflow, and planning for data re-use. We’ll touch on these topics again today as we go through my project. And...I’m not advancing the slides. 

[Informal Background] 

Laurel Copeland:I’ll be using an actual research project of mine as an example. It had a pilot phase and then the main study. And, we’ll be highlighting good data practices in proposal planning and development. We’ll be looking at considerations with respect to funding, IRB, and study initiation. And of, focusing on documentation in terms of content and locations of documentation, and touching on study design and implementation. 

To get settled in and let me find out who I’m talking with I’d like to have poll question number one. The question is: What is your primary research role? Would you say you are a data analyst/programmer or statistician; research coordinator or research assistant; student, trainee, or fellow; an investigator with data skills; an investigator without data skills; or a manager, policy maker, or other non-research VA stakeholder? Looks like we are populating the fields here. We’ll have three polls today. 

Moderator: Yeah, it looks like the responses have slowed down a little bit here. 

Laurel Copeland:Should we end that poll? 

Moderator: Yeah, we could probably read through the results there. 

Laurel Copeland:Alright. Okay, so good representation of data analysts, programmers, statisticians, research coordinators, and research assistants and actually a nice chunk of investigators with data skills. That’s always good to see. Thank you all. 

The session outline consists of looking at proposal planning and development to begin with. When I was first recruited to Texas as a junior investigator one person interviewing me asked how long I expected to take to develop a proposal. I said three or four months. A lot of that time goes into developing and refining the aims. But during proposal development you could probably expect to spend one to three months developing the aims and tweaking them to the very end possibly. These aims will inform your methods and define why your methods will inform the aims. The aims have to be compelling but they also have to be feasible. The methods have to be affordable and therefore within your budget limits. You have to have the personnel available who have the skills required by the methods, and you need to be able to pay them according the budget. So, I actually usually start with the budget because it tells me who does what and how much time they need to do it. 

A part of that process is of course identifying your funder, which will tell you how much money you get to play with. This particular project that we’ll be talking about today is Surgical Treatment Outcomes for Patients with Psychiatric Disorders or STOPP. The STOPP project came about when I was paired with Val Lawrence as my primary mentor on my VA MRAP, which was a Career Development award. I was coming in from studies of VA patients with schizophrenia, where she’s a clinician researcher in the area of perioperative disability and recovery. Independent of the VA Career Development award, we wondered what’s at the intersection of these two areas? Do patients with serious mental illness have the same outcomes of surgery? Do they even get the same surgery as patients without these severe mental illnesses? 

What is known? To start to address this question we first conducted a systematic review of the literature. This was published in 2008 in *Annals of Surgery*. We were interested in articles that reported clinical outcomes on patients who had a preoperative established diagnosis of a severe mental illness. We relied on a VHA directive for...actually I think it’s 20-12-12, which came out later, to define severe mental illness. And these are four conditions: schizophrenia, bipolar disorder, post traumatic distress disorder, and major depressive disorder. We only found 12 papers in our systematic review. That meant we had identified a knowledge gap. That’s another way of saying we had found an opportunity. After that, we got to an IRB protocol and sought funding for our pilot study. The pilot study was Perioperative Outcomes and Safety in the Seriously Mentally Ill Elderly, which we nicknamed POSSE. 

Now we had a decision to make. We knew what we wanted to look at, but we didn't have the money to look at this topic. We had to decide whether we should get started on the work before the funding or get the funds first to do the work. The disadvantage of waiting to get the money is that you can lose a lot of time while the reviewers are mowing things over. The disadvantage of not having the funding while you're doing the work is there are a lot of time constraints in that case. 

I’d like to insert poll question number two right now. We’re going to get into this topic pretty heavily. Which best describes your research experience with VA administrative databases? Would you say you have worked directly with the VA administrative data yourself and are adept at DART processes, have worked directly with VA administrative data, have collaborated on their analysis but not pulled or manipulated them, or have not used the administrative data at all? Okay, so about 44% of the current audience, maybe a bit more, has not used the administrative data at all. Oh, you're starting out with an excellent serious here, I hope, to get you on your way. We have about a third, okay, using VA data yourselves, very good. Let’s give this a little bit more time. Okay, I’m going to broadcast these. 

[Informal Background] 

Laurel Copeland:Alright, so I call unfunded work a beautiful trap because it’s often work that I really want to do and it’s everywhere. There’s lots of it. It’s very interesting. But, of course where does the time come from? On the upside, if you have an IRB protocol you can generate preliminary data and baseline papers. It may even generate a very high impact study finding, it provides preliminary data for a larger proposal, and it can tell you a lot about feasibility of your proposed methods and processes. On the downside, it could be completely unproductive. My highest impact paper – I shall tell you – was from an unfunded study. It was published in medical care in 2006 and resulted in a directive in 2012. 

In terms of the pilot study POSSE, we had drafted the protocol for the IRB. Again, I was working with Val, whose initials are actually her name – Val Lawrence. And, we wanted some funding to work on this protocol. As you know, you do need IRB approval for pilot studies. Pilot study does not mean preparatory to research. It is research. You can run frequencies and means and things like that at VINCI or from Vista data, but you cannot record patient level data with any identifiers. That would be protected health information. You can’t record that without IRB approval. 

In order to get some money to fund this project we first needed to identify a mechanism. We found the VISN 17 New Investigator Award Program seemed to be a good mechanism. We didn't qualify as new investigators so we needed to identify a new investigator. Our study team started to expand. We identified John Zeber who was post doc at VERDICT in South Texas along with us at that time. Then, we started to look at what capacity we needed in our team. We added a data analyst, another non-clinician investigator, and two clinician investigators giving us a team of six people. 

The funding was awarded to John Zeber for two years by VISN 17 and it was our plan to identify major surgery in 2005. The term major surgery doesn't mean a whole lot when you're looking at administrative data. It’s not previously been defined. We wanted to define it using VA administrative data. We knew how to define inpatient surgery. We could define that by looking for patient who were admitted and then had an inpatient operation. So that would be an pre-admitted patient. Another version of inpatient surgery would be the day of surgery admission where the person was ambulatory, comes in, goes to the OR – the operating room, has the procedure, and then is admitted. In that second case, the day of surgery admission or DOSA, we don’t actually know if the admission was preplanned or was emergent form the operation itself. So that’s a sort of a limitation of administrative data. However, what the actual procedures that could be included in the idea of major surgery would be we didn't know, and that’s what we did for two years. We looked at lists of CPT and ICD9A procedure codes, we talked to surgeons, we went back to the list of codes, and went back and forth with the clinicians iteratively to define major surgery. 

About the codes: There are two kinds of codes, two schemas or two types of lists of codes for procedures. The CPT codes are separate from the ICD9A procedure codes. Don’t confuse the ICD9A procedure codes with ICD9 diagnosis codes. They are completely different things. A diagnosis is something that the clinician thinks you have. But a procedure is something the clinician will do to you. The ICD9A procedure codes do not look at the human body the same way the CPT codes do. So they don’t map well to each other. Therefore, we had to examine these two types of codes independently. Why did we have to look at them, because in VA data the outpatient procedures are defined by CPT codes, whereas the inpatient procedures – the ones that are listed on the inpatient records – are defined by ICD9A. So since we were including the doses, where the operation technically occurred on an outpatient basis, we had to include both types of codes. Ah, the things you learn from pilot studies. 

On to documentation content and locations: What we want to know is what’s in there and where is it? Okay, so part of the documentation was on the progress of the study in terms of meetings. The POSSE study met regularly. We documented the meetings immediately with meeting minutes that were emailed to all team members. We documented all the CPT and ICD9A procedure code lists very carefully, putting them on Word and Excel files with footers and page numbers to identify what was listed. If you're wondering about the specification of the page numbers it’s important to understand that these lists could be 40, 50, 80 pages long. If you printed one out and dropped it, you really wanted to know what page was which. 

We also tried to use good file names. Good file names are concise, but descriptive. We didn't want to use long sentences as file names because it’s not too difficult to exceed system limits on how long a path is and also it can get very hard to read in the window that your computer will present you. However, you can use some abbreviations and be creative and define what’s in there and when it was worked on and why and what the result was.  We saved all our documents to the project folder, which was of course on a VA research server as approved by the IRB. We kept focused on the goal to define major surgery, talking with clinical experts, which in our case in South Texas included nurses who were involved in VASQIP, which is the VA Surgical Quality Improvement Program. That was kind of an advantage of our location. 

In terms of project folders and binders, another factor comes into play, and that’s research oversight. So on our site, and as everywhere, we had our IRB, but over them is the Office of Research Oversight – or ORO. We usually call it ORO. They require that the PI maintains a hard copy study binder for every project. The reason it’s a hard copy at our site is that our standard operating procedures are taken from the handbook 1200.05 and we use section 9I, which specifies what goes into a study binder. If you read that handbook you'll see in that section it says a written copy. It just so happens that our branch of the ORO – western region – interprets the word written to mean printed on paper. So the take home from this part is you need to know your local policies and procedures. And, you also need to know a little bit about your regional oversight. Other sites might be allowed to have only an electronic copy of the study binder. I’m not going into the details of what’s in section 9I, but now you know where to find it. 

VA Data Security requires that VA Research Data be stored within VA in locations approved by the IRB. That is, you have to tell them specifically where the data are. Reason demands that you use an organized approach to your research project directories. I like to choose an organizational form and stick with it across projects. If I use the same organization across every project, then it makes it easier for my research team members to easily guess where something will be, even if they’re going from one project to another. 

Now we can pause and have poll question three. What level of experience do you have in organizing data, data structures, filenames, and variable names? The choices are expert, lots of experience, have worked with one or more schema; moderate, have used the schema for at least one study; low, aware of the general idea but haven’t implemented it; and novice, this is new to me. Good. That looks good – lots of people well aware of this important aspect of project organization. And...oh good we have some novices and new users, eager to learn, excellent. Thank you. 

In terms of documentation, there will be a few screen shots here which are going to have lots of small font. I’ll explain. This particular screen shot shows on the left-hand side that there are several folders starting with my name Copeland and then under bar and then one more world. This schema for a project or organization uses the PI’s last name under bar nickname for the study to name the research folder. So if you look further up and down you'd find my colleagues. We have opened up Copeland_STOPP, which is the project that we’re talking about today. It has three folders in it – data, docs, and report. Data has a specified structure within it. It always includes derive, export, extract, and import. And, I’ll talk about those later. In this case, it also has backup and POSSE. 

Now POSSE, as I explained, is our pilot study. Very few studies that I have actually have something like POSSE in their data, but in my case, I administratively combined my pilot study with my main study. This is a way to reduce your paperwork – your administrative overhead. If you ever want to administratively combine your pilot study with the subsequent main study, you need two steps. The first step is to amend the main study to add the pilot as a data source and the second step is to close the pilot study. 

Also in here you see the docs and report sub folders. The difference between them is that things that you put in report can be publicized. They can be emailed. They can be printed and sent to people. So in report we would put the analysis output and managed drafts that we were developing. Docs however, is not meant for publication or distribution. Docs would contain not only your IRB approvals, which you might be able to see there at the top, but also internal documents such as the list of CPT codes I mentioned, any preliminary data that included identifiers, QA and QI printouts, and things like that. There is a possibility that you would have protected health information or PHI in Docs. But you would never put any PHI in report. 

In the case of the structure that we’ve been looking at with the Copeland_nickname, that structure was developed at South Texas Veterans Health Care System with data analysts and the embedded OIT guy and me. I was the data manager. We were in the Verdict Research Center and we were really lucky to have Steve Bitant from OIT embedded with us in research. And the structure with these specific subfolders was developed by the data analysts and me based on Cleopatra Deliou’s experience in industry. So yes, I borrow my ideas from wherever I can get them. 

On this screen shot, this one is taken from my second site, which is where I am now. It is Central Texas Veteran Health Care System. This structure was developed when I was acting as a SIDS chair, which is the Subcommittee for Information and Data Security, working with the IRB chair and the IRB administrators. So, it has a slightly different tack. It starts out with the IRB protocol number – in the case of STOPP it’s 00412. We’ll see that later. Don’t squint - and then, the nickname STOPP. And those are included within Copeland. Now some of the slides are from a previous incarnation of our nomenclature. So bear with me. 

Here we see that there’s more than one way to skin a cat. You get to choose. You get to decide. But I would advise that you choose one model and stick with it to make finding your documents, our codes, your output, and your IRB documentation easy to find. One example here is to have the upper level folder for the project named 00412 – the IRB protocol number – space, Copeland – the PI’s last name, space STOPP – the nickname. Another idea we saw on the previous page which was similar only without the name Copeland, and the third example is Copeland under bar STOPPE and that one is from South Texas. 

Now, in the case of South Texas we actually have two folders for the studies. We have \Copeland_STOPP and we have \sas_Copeland_STOPP. The reason there’s two is they have a SAS server and there’s limited space attached to it. You want your data to be close to your SAS server if you have one because you want IO to be good. You want the input/output to be good. You don’t want to have hiccups while you’re reading in files and writing them out. In the case of the SAS server it has a BUS connection, which is a high quality connection to extra storage array. And I know you love the technical details, but if you need them later, email me. 

In terms of project documentation content, you need to be able to preserve all the decisions that you make during the conduct of the study so that you can use it during and after the study. These documented decisions will be turning up in your manuscripts in your method sections. They’ll be turning up in your responses to reviewers and of course in your reports to your funding agency. Good documentation of processes will help allow replication of your study. Every study should be able to be replicated. 

In documentation also sits data security, data management, and oversight review. You really can’t have too much organization. Also, you want your process written down, not in someone’s head. Also, in internal documentation within the files you want, for example, a new data analyst to be able to pick up after a previous data analyst if that person leaves. You want transition within the study to be easy. 

The project documentation for data includes of course the SAS code, the program files. I want these saved in a logical place. In my case, logical would be within data in a subfolder called derive, so all of my SAS programs are located, for every single project, inside of data/derive. Then, looking into the SAS program file itself I would expect a certain level of documentation. At a bare minimum I want the first three items. What is the name of the SAS program file itself, what was the data it was initiated or first begun to be written, and who was writing – who’s the programmer?  After that, it’s very nice to have a brief study description including the PI name. Within the program it’s especially helpful to put in little comments telling what you're doing and why you're doing it. There are often occasions where you need to include important decisions that were handed down from the study team or the PI. Sometimes copy and pasting in an email clip is very useful, especially if you date it so that you can always go back even years later and say yes, we threw out that subgroup for this reason on this date. That’s why they're not there. And always keep in mind succession. The next programmer may not have access to the original programmer. And also to replicability – studies must be replicable. 

Here is a little clip of a SAS program. I’ll read it for you because it’s small. The first three items of documentation are prediabetes.sas, which is the name of the file. It was created on 28, February, 2008. The coder was LA Copeland – that’s me. I took out some of the line breaks because I wanted to fit more on the page here. But normally those would all be separate lines. I used white space to make the file more readable, and I recommend that you do that. White space includes vertical lines that are blank and also tabbing in so that you'll have left-hand white space. In this one, it has a table of contents, Section1: Analysis and Section 2: Build Data Set. Another approach to that would be to have two separate files. That’s a very nice way to do it – I actually prefer it. And, that would separate the analysis into prediabetes_cr.sas where CR is for create. And a second file or program called prediabetes_an.sas where AN is for analysis. However, this one is combined. And there’s some comment about where the funding came from – VAHSRDIIRO5326. That’s pretty unimportant to the coder, the programmer, normally but what happens is somebody needs to know that number and this is certainly a place where you're not going to lose track of this file because you have to put it into every manuscript. 

Then there’s the cohorts are briefly defined. In this case I have diagnosed schizophrenia and how that’s defined in terms of ICD9 diagnosis codes and then there’s the diabetes cohort apparently. They have codes to define them. Then I have the data sources. I just wrote SU four source, but here are the files from Austin because the project was done quite a while ago – five years now. So, all of these data sets were coming off of Austin. There was no CDW at that time. The files that went into this particular project, which incidentally is not the STOPP project, but the file is much shorter and was easier to show. It included the PM, XM, and NM files. Those are inpatient files for the main VA...the VA hospitals – PM, the VA extended care facilities – XM, and the non-VA facilities or hospitals which is NM. SE is for outpatient events. An event is a visit to a clinic. So each row of that file represents a data clinic combination. Another way to look at outpatient visits if you were stacking your clinics is to look at the whole day. A veteran might go to two or three clinics in one day. Those data are organized in another set of files called SE. 

Each of these little abbreviations like PM, SE, and SS...I’m sorry. I misspoke. The by date one is SF. Each of these little abbreviations is created for each fiscal year with some variation because files come and go. Then I have a couple of the DSS national data extracts or NDE’s. We have LAR and PHA, which are short for lab results and pharmacy. Then we have BIRLS which is the old desk data, the MINIVITAL which I the new desk data, and ENONEPER which is enrolment data, and SHEP which is survey of health experiences of patients. Those are survey data. You don’t get them from Austin actually. You have to request them from the FSC. Then the LIBNAME you can see is for the data, the data level. That’s the upper level of data and that’s where I would put my patient level analytic database.

In terms of options, I want to draw your attention to two which aren’t used enough in my opinion. There’s mergenoby=warn. This option will warn you when you use a merge statement and forget to put in the by statement. It can create really messy data sets if you accidentally forget that. And, in a very long setup where you're doing a whole lot of merging it’s possible to forget. Another one I like is mesglevel-i. I is for information. This is the finest level of message. It produces messages in the log that tell you which variables you're overriding when you're merging data sets or setting them. No, just merging. Sorry. 

And there’s some comments and then the data step. You see after a few lines of the merge that there’s a comment. It’s study inclusion/exclusion criteria. For each if statement there’s a little number, like 39904. That is the number of records that were remaining after that if statement was executed. To get all of the numbers, I had to put the run after the first if statement, run it, write down the number, then move the run statement after the second if statement, run it, write down the number, etc. What is the point of this? It lets you draw for the manuscript that you are going to write the data flow. It tells you...it’s a very nice way to record for posterity, exactly where the patients were thrown out and how many when. 

I’m going to move on to a little bit on study design and implementation. We had successfully obtained the pilot, spent two years defining major surgery...actually, before that we had an idea. We first started with an idea. We systematically reviewed the literature on it. We found a knowledge gap, which we considered an opportunity. We wrote a protocol and got IRB blessing. Then, we got the pilot money and spent all that time doing the pilot study. Now, we wanted the big bucks. We had worked very hard and we wanted some money. So at this point we were ready to go after a merit award. 

We had used what we had learned in the pilot study to build a larger study with interesting aims, methods that made sense, and were informed by our missteps in the previous study. We used the preliminary data generated from POSSE in the STOPP proposal. And of course, we used any weaknesses to produce better methods in the larger proposal. So overall, what we chose to do – and this is a good use of time – is to reuse and adapt the POSSE SAS programs and experiences in the STOPP study. 

The aims of STOPP were to compare rates of surgery among VA patients by pre-existing severe mental illness status. They were...aim two was to assess 30-day, 90-day, and 1-year postop mortality. And aim three was to assess 30-day, 90-day, and 1-year postop complications. Why rates? Well, this was so I could have data on the denominator, which was all VA patients. I really wanted that. There were seven million of them. 

Okay, where did the data come from and how did we make our variables? Well, you can see inpatient and outpatient files, the PM/XM, SE files, to get the diagnosis variables, to define severe mental illness – which as I mentioned were schizophrenia, bipolar disorder, PTSD, and major depression. We used the same files again to define comorbidity. In particular, we were interested in several specific conditions like hypertension and substance abuse disorders, but we also wanted some comorbidity indices. So we made the Charlson comorbidity index and the Selim physical comorbidity score and the Selim mental comorbidity score. The difference between Charlson and Selim is that Charlson scores were created to quantify risk of one year post hospitalization death. So the patients were all discharge from the hospital...they were all inpatients discharged from the hospital and the point of the score was to predict mortality. 

The Selim on the other hand was developed much later to capture chronic illness. Between Charlson and Selim, we moved into the managed care arena. We deinstitutionalized and it got to be a lot more important to be able to work with outpatient data. Again we used the inpatient/outpatient files, PM/XM and the SE and some new ones, PS/XS and PP/XP to get procedure codes to define major surgery types, admission date, and this was of course going to be using our definition from POSSE. The PS and XS files are detailed files of the main records where the main records tells you that there was an hospital. The PS and XS files tell you about surgeries that happened during that hospital if any. So, there might be none. But of course, this being a surgery study, everybody had some. Then the PP/XP files are procedure files. Again, these are detailed files to the main records, the PM/XM records, and they contain procedures. 

What’s the difference between a surgery and a procedure? Well it’s very idiosyncratic. It varies by site and possibilities by coder as to where you're going to find your procedure code that you're after, between the surgery and the procedure files. So we used both. We knew what the codes were that we wanted because we spent two years figuring out that list. We weren’t going to accidentally miss some by not looking at one of the other files. So, I’d recommend that you use both if you're interested in some particular set of procedures or surgeries. 

We also used the inpatient files, the PM/XM, the familiar main records, and the bed section – PB/XB detailed files, to look at post-operative events. In particular, we wanted to look at ICU, which is a captured by a bed section. So you definitely need bed section files for that. And then we had these other things such as...the MI is myocardial infarcted acute coronary syndrome. These are complications of surgery that were primarily defined by date and diagnosis. 

We also used September PSSG files to define, to capture VA priority status. PA priority status is...it’s a veteran status which indicates why the patient is allowed to use VA health care. So a VA priority one is someone who has a 50-100% disability that was attributed to his military service. The other levels – two, three, four, five, six, seven, eight – have various meaning determined by your particular disability level, your special military experiences – such as being awarded a purple heart, non-service connected disability, and poverty. Therefore, the VA priority correlates with both socioeconomic status and also severity of illness. There’s a nice article by Louis Katsis [00:41:17] in 1998 that describes this. 

We also used MINI-VITALS to get best sex, best data of birth, and best date of death, and the use after death flag. There is some legitimate use after death because the VA will pay for family bereavement counseling. So, there can be legitimate use billed to the veteran’s account after he/she has died. But after 30 days it becomes so unlikely that it’s considered spurious and likely to be intended by a typo in somebody’s identifier. So, if use after death is positive or one we would actually drop that data. 

Then we wanted to have period of service, such as Vietnam era or World War II. And there are two sources for that variable – the inpatient and outpatient files. In this case, the outpatient file is the SF, which I described as the date level outpatient care files. For some reason, period of service is not in the SE files, but you can find it here in the SF files and also in the PM/XM files for inpatient data. However, period of service does not include operations Enduring Freedom and Iraqi Freedom nor OND, New Dawn. To get that, we had to go through a special DART application for access to the OEF roster files because it is owned by Department of Defense. Because it’s from DOD, it’s also not limited to VA patients. However, it’s available. 

We used the lab results files to get process measures such as had cholesterol assessed and outcome measures such as had high cholesterol. We used the pharmacy files to define pharmacy related covariates such as using corticosteroids within 21 days prior to admission for surgery. And all of these files that are described here as DSS NDE, which is DSS national data...okay, I’m blanking on that. But, they are no longer available. You have to go and get these data from the corporate data warehouse. At the time of study, they were just sitting there at Austin. 

Then we had this other files, the IE – inpatient encounter file. This is a set of files that we weren’t too sure what was in them. We could see that they had CPT codes in them but we wanted to explore them. So we put them into the protocol and we explored them. What we were hoping was that they would have really good overlap with the inpatient surgery data so that we could actually have CPT codes for the surgeries. But, that wasn’t the case. We spent quite a bit of time looking at the overlap this way and that way and regional variation in the overlap and decided we couldn’t use them. 

Now this is back to the organizational issue. In data we have the standard folders of Derive, Export, Extract, and Import. We also had Backup and POSSE. What’s important at this point is to think about all these types of files that I’ve been talking about and bringing them down from Austin to the project directory, which was local. To keep them straight, what we did is we made type specific subfolder extract for each kind of file. So, for the PM/XM files we had Extract\PM\XM. For the pharmacy data we had Extract\PHA. We put the files in there. For the lab results we had Extract\LAR and Extract\PSPP and so on. You can never have too much organization. 

Because of the great size of the flat ASCII files that were bringing down, keeping in mind that I had gone off that denominator which was seven million persons, and because of the very long time required to read these big files into SAS, we actually did something I would normally not do. We made binary SAS copies of the flat files once we got them to our research project directory. We stored the binary versions at the extract level, which is above the specific files I just mentioned but still inside of extract. We used compress=yes when we generated these files to keep them as small as possible. In some cases, that actually seemed to make them run faster also when we were using them. We did try indexing, but that seemed to increase run time. I find indexing is very useful when I frequently need to use the data sorted in more than one way. Mostly I would do this with drug data because sometimes I want it sorted by scrison [PH], date, drug and sometimes I need it sorted by scrison, drug, and then data depending on the recode. 

Another little methody thing that came up because of the size of the files is that we got a little tired of doing the same thing over and over and so we put a lot of things into macros. Macros are nice, not because they’re easy to read – they aren’t – but because you don’t have to keep pasting them into your main file – your creation file. So we made...we converted our algorithms for race, when we generated from all possible records of race and ethnicity and Selim comorbidity scores and Charlson comorbidity scores into macros. Then we could call them. We experimented with various approaches to data aggregation. We asked everybody for help. We asked HSR Listerv for input. We have colleagues, a large network of SAS programmers that we tapped. Basically, you can never ask too many questions. Don’t’ reinvent the wheel. Just ask for help. That’s my motto. 

Progress, after all this data we started to get to the point where we needed to write papers. This is interesting. Some of the tabs got lost, but that’s okay. Each paper that we would write had a subfolder within report. So we had Copeland_STOPP and then there were the main files - \DATA, \DOCS, and \REPORT. And then within REPORT we had \ABSTRACTS, \MORTCARD, \SUICIDE and so on. We put all of our abstracts into \ABSTRACTS altogether. And each paper had its own folder. So, \MORTCARD was actually a paper about mortality after cardiac ops and \SUICIDE was about race suicide and differences in suicide after surgery. 

Most papers needed a tailored analytic database as well, but that was saved in \DATA because all of my analytic databases go in the same place. And if you aren’t sure what an analytic database is, it is a database at the case level for your study. In this study STOPP, the case level was the person. But sometimes the case level could be admission and a person could have multiple admissions, or the case level could be facilities. So, the data that comes into these files obviously is at many different event levels which we have been talking about, such as a drug prescription, a clinic visit, a lab test, or an admission. But when you get to the analysis everything has to be recoded, aggregated, and merged to the case level. 

So, STOPP was progressing nicely and we facilitated this progress with team meetings where everybody was on the call. There was a move in between, but we had more than one site. So we had calls monthly with the whole team to develop definitions, talk about study design, and papers. We had data core meetings every week to monitor implementation of team decisions and to generate patient level data sets and analysis. We used the original aims to determine the primary papers that we wrote. But we were light on mentees and colleagues to determine the secondary papers. That is, we employed what we like to call the Kilbourne Model, named after up in Ann Harbor, of encouraging collaborators to write papers for your project. It’s easier than writing them all yourself and it greatly increases your dissemination abilities. 

Okay, about this move: So STOPP was funded in April of 2010 but John Zeber and I moved to a different VA in August of 2010 just after it got started. This is a big deal because before that it was a single site study. Prior to the move, I had to decide whether the data would stay inside Texas and I decided on yes, it would stay there. I had to file an application with a new VA before I moved to get that process started, because the minute I left my old station I wouldn’t be able to access my own data unless it was approved at the new one. And, I began discussions with the new VA’s administrative officer about moving my research funds that supported our salaries. This turns out to be a big deal. So, if you're every moving VA’s keep talking about it before you move and keep talking about it after you move until you're sure it has happened. 

Note that this move required that STOPP have study folders on two VA servers because we now had two IRB’s. And the documents for the second site had to be on their server. Now that we have CIRB, the central IRB, they might want oversight on a two site study like this or they might now. So you just have to ask. 

Now we come back to oversight. So there we are at Central Texas Veterans Health Care System and for some reason ORO – the Office of Research Oversight – visited our facility three times in 2012. Each time they came they wanted more changes. There was a lot going on with HIPPA at that time. It seems like a lifetime ago even though it was only last year. They wanted us to convert all our policies to SOP’s, which are standard operating procedures. The reason they wanted that is that policies have to go through really heavy duty concurrence and approval processes that involved a lot of service chiefs and the director, whereas standard operating procedures can be managed by committees, which are much more agile. The ORO was reviewing our SOP’s and our studies. They requested lots of changes and at some point they noted that some of our studies sounded like data repositories were being proposed. Oh boy, they didn't like that. But I thought hum...STOPP would be a good source of a data repository. Why do I get these ideas? Well, we didn’t have an SOP to cover data repositories, so I drafted one of those in 2013. And then I applied for the creation the STOPP data repository. 

STOPP itself concluded funding in March of this year, 2013. We had the SOP in place, which let me file for the creation of the STOPP data repository. This will permit us to keep using the massive data set to address aims that are not related to the original study. They don’t have to involve surgery. They don’t have to involve postoperative experiences of veterans with severe mental illness. We also added VINCI as a data storage to both STOPP and STOPP data repository – both of the protocols by IRB amendment. 

That is actually the end of my formal comments. That’s my contact information. If you answered yes to adept at DART, please sent me an email. I want you on my list of handy dandy contact people. I think Jeffrey Scehnet is getting tired of me. But we’re eager to take your questions at this point. 

Moderator: Okay, here’s one question: Can you identify the website or other useful resources used to identify CPT and ICD9 codes?

Laurel Copeland:Sure, CPT codes are put out by the AMA – the American Medical Association. That means they’re proprietary. So you either have to buy the book, which we do have a hard copy of, or you can work through CMS – the Centers for Medicare and Medicaid – or if you're associated with health care partner...what kind of health care partner? If you can get your clinical partners or your clinician researchers to help you, they should have access to CPT codes. So those are sort of the more difficult ones. There are also labels for CPT codes at Austin in the names file, and it would take me a while to explain that. So just write to me and I’ll send you how to hook up with those names files. 

And then, the other code is ICD9A procedure codes. And, like ICD9 codes they are created by WHO – The World Health Organization. That means they’re in the public domain. You can go to various website such as chrisendres.com and choose procedures and search for text strings or numbers. 

Moderator: Great. Thank you for that. Here’s another question: Could you provide a reference for defining Selim comorbidity score? 

Laurel Copeland:Yes, Selim...Alfredo Selim...I don't have that in my head right this second but what was the year of that? I can provide it by email. That would be easy. But yes, he published...his name is Alfredo A. Selim. 

Moderator: Excellent. Thank you for that. Here’s another question: Where outpatient files as well as inpatient files used to accumulate postoperative complication? 

Laurel Copeland:We only used inpatient because we wanted...why did we use that? We wanted things that were either happening immediately postop in the same stay or occasion to readmission. So in our case no, we didn't use both. We only used inpatient. 

Moderator: Okay. Here’s another question: Was the problem with the CPT codes that they could not be easily met to ICD9 procedure codes or was the problem that they were often missing for cases that you were pretty sure a surgery occurred?

Laurel Copeland:They were not missing. The problem is that the files are different files. So if the patient came in as an ambulatory patient he was registered as an outpatient and his data was recorded in the outpatient events. That files uses only CPT codes. There are no ICD9A procedure codes in that file. If the patient was pre-admitted, so he was already an inpatient when he had the operation, his experience was recorded in the inpatient file system in the PS or XS or NS files – the surgery detail files – and those files only use ICD9A procedure codes. They don’t use CPT codes. 

Moderator: Thank you. How about one last question: Can you describe how a collaborator might gain access to the STOPP data repository? 

Laurel Copeland:Yes. You have to be inside the VA, the way I set it up. But you can write to me and I have a form that you have to fill out. It’s pretty easy. But I don’t actually release the data. It’s got to be inside VA. I do have the reference for Selim now. It was 2004, J. Ambulatory Care Management. That’s the Journal of Ambulatory Care Management, volume 27, page 281, in 2004. 

Moderator: Thank you so much to Laurel Copeland for presenting today’s lecture. Tomorrow’s research application will cover planning for documentation of study design and measurement, data cleaning, construction of cohort, outcomes construction, covariate construction, linkage of primary survey data and VA secondary data, and summarize the value of documentation. Please join us tomorrow from one to two p.m. Eastern. Thank you, and have a great rest of the afternoon. 

00:59:18END OF TAPE

Page **1** of **14**