**Transcript of Cyberseminar**

**VIReC Good Data Practices Series**

**Planning for Data Re-use**

**Presenter: Linda Kok, MA**

**September 11, 2013**

*This is an unedited transcript of this session. As such, it may contain omissions or errors due to sound quality or misinterpretation. For clarification or verification of any points in the transcript, please refer to the audio version posted at www.hsrd.research.va.gov/cyberseminars/catalog-archive.cfm or contact the VIReC HelpDesk at virec@va.gov.*

Moderator:For those who are just joining, welcome to Session Three of VIReC’s Good Data Practices mini-series. The Good Data Practices mini-series is a set of lectures that are focused on good data practices for research. The series includes five sessions that will take place this week, Monday through Friday, from 1:00 to 2:00 p.m. eastern. New researches and research staff planning new projects may find this series beneficial. Thank you to Sider for providing technical and promotional support for this series.

Today’s lecture in our Good Data Practices series planning for data re-use is presented by Linda Kok. Mrs. Kok is Technical and Privacy Liaison for the VA Information Resource Center, VIReC. Questions will be monitored during this talk and will be presented to Mrs. Kok. A brief evaluation questionnaire will pop up when we close the session. If possible, please stay until the very end and take a few moments to complete it. I am pleased to welcome today’s speaker, Linda Kok.

Linda Kok:Hello, I am not on mute anymore. Can everyone hear me now?

Moderator:Yes, we can hear you, Linda.

Linda Kok:Great. And Heidi, have you given me control of the presentation?

Moderator:Yes, you have control. You should be able to use those arrows at the bottom left and the drawing tools up top.

Linda Kok:All right, wonderful. Thank you so much. I am Linda Kok. And I am very happy to be able to talk with you today about this new and growing topic, planning for data re-use. My screen is not moving. Great, got it. Since we just completed Session 2 a moment ago, I will not spend any time recapping what you have just heard. Let us turn to something that may be new and even maybe a little exciting.

We are going to focus today on data activities at project close. We will ask why we might want to make research data – research generated data available to re-use. We will look at VA and non-VA policies for sharing data for re-use. And we will go over planning for data sharing and review the documentation you need if you are going to decide to share your data for re-use. As a warning, I occasionally use the term secondary data use, data re-use, and data sharing. At least for the purposes of this presentation, please consider them interchangeable.

Before we begin, we would like to get an idea of how many of you are currently thinking about sharing data for re-use for a second project of your own or for re-use by others. Our question is are you planning or already working on a project that will produce a data set that might be shared for re-use? It is like watching a race to see which one is going to come in first. Heidi, do you think things have stabilized?

Moderator:Yeah, they are probably pretty good. You could probably read your responses out now.

Linda Kok:Okay. Well, it looks like we have a majority of the participants today who are interested in already planning or working on a project that produces a data set that might be shared. That is very exciting. And we hope we will be able to provide information for you that will – that you will find valuable. And to the others who are not already planning a data set for re-use, we hope that we can maybe inspire you. This is an important new area for the VA. So before we dive in to all that is involved in data sharing, let us begin with a review of data activities at project close and the decision to share data for re-use.

As Denise advised in Sessions One and Two, you may have already decided that you will create a research data repository when you wrote your proposal. But if you did not, you must decide before you close your project while it is still open and under continuing review. Once you close your project, it would be very difficult, if possible at all, to reopen it for sharing. Either way, you will need to notify the R&D Committee and IRB to get permission to close your project or amend it for sharing.

If you decide to close your project without data sharing, you must have your data secured by OINT with access permission for you and your project team removed. You will be allowed to retain your tables and charts and any data as long as they contain no protected health information at VHA. 

You should also be aware of VHA records retention requirements for research-generated data. That is kind of a trick question because right now, the records retention schedule for VA research generated data has not yet been agreed upon. So retaining it will be – is required to be indefinite. You have to retain your data indefinitely until the VHA adds it to the records control schedule. 

But what data should you be retaining? You would want to retain your analytic data set and any unique data sources so that if necessary, you or anyone else can replicate your findings to defend your research.

Here is an example of project closure process at time. We are required to turn in study data disks, if any still exist, to the research office. We need to discontinue using any identifiable data and we request that OINT staff remove access to all of our data sets and move – or to the data sets we want to retain and store them securely with access limited only to OINT staff.  

So now that we have covered what to do at project close, let us assume that you are interested in data sharing and that seems to be true for most of you. Why should we make research generated data available for re-use? Every day, we create 2.5 quintillion bytes of data so much that 90 percent of the data in the world today has been created in the last two years alone. That statement was made in August of 2012 by IBM as part of their Bringing Big Data to the Enterprise presentation. I imagine the numbers have gone up quite a bit since then. 

This requires – this imposes huge costs on the enterprise whether it is the VA or your university. To the extent that we can reduce those costs, we can – re-use the research data, we should try. Reduce, re-use, and recycle, a phrase familiar to us from the environmental movement. It can also be used to think about secondary use of research data. We can reduce redundant and expensive data preparation and purchase cost, re-use existing research generated data, and we can recycle our analytic data sets by selecting subsets of the variables or subsets of the original cohorts or perhaps merging it with other data to create an entirely new data set. If we can do this, we may be able to save our limited resources for other research activities.

In the previous sessions, you heard that planning for data sharing should begin during preparation of the research proposal. Depending on your funding, you may have been required to include a data-sharing plan with your submission already. The National Institute of Health’s goals of sharing data give us these – the reasons listed here for sharing our data. I will not go into them here because we have to move along. But they will give you a feel for why sharing is valuable.

Those are the NIH reasons, but why should you want to make your data available for re-use? You may want to re-use the data yourself or allow to access to your co-investigator for a new study. Or you may want others to re-use your data. And when they do it, they must cite it any publication. So allowing your data to be re-used can actually promote your own research. And finally, sharing it may enable new discoveries with your data by you and others. If your data are unique, if they cannot be easily replicated such as large surveys that are too expensive to replicate or studies of unique populations or studies conducted at unique times like immediately following a natural disaster. Or if your data are just too expensive to purchase again, if any of these are true, you might want to seriously consider sharing your data for re-use. 

Now things will get very exciting as we talk about policies that affect sharing data for re-use. As I mentioned, in NIH’s view, all data should be considered for data sharing. They feel it should be made as widely and freely available as possible while safeguarding the privacy of the participants. 

NIH actually requires research applications for over 500 thousand dollars or more in a single year to include a data-sharing plan or a statement explaining why data sharing is not possible. We have included a link there at the bottom of this screen so that you can read more about NIH’s data sharing guidelines.

The VA permits re-use of research data for additional VA research protocols if the data are in an IRB VA research data repository. Detailed requirements for creating a research data repository described in VHA Handbook 1200.12, Use of Data and Data Repositories in VHA research. I will not cover all the requirements described there. But there are three essential requirements that I want to mention.

First, an IRB approved protocol for the repository. It can be part of the original protocol, but the protocol must explain what you are doing with the research data repository. Second, and administrative structure including written policies and procedures for how the research data repository will be run. Third, there must be oversight. And this can be provided by scientific and/or ethic advisory committees.

One thing you may not know is that you can choose to establish your research data repository yourself or you can deposit your data in an existing research data repository within the VA. If you do that, of course, you are still going to have to include that information in your protocol. Connect with Brenda Cuccherini at ORD who will provide advice to help you establish your research data repository and help you in many ways in dealing with the research data repositories.

At this point, I wish the 1200.12 had called these data archives instead of research data repositories. It would be so much easier to say over and over with all of the R’s.

Before you can consider sharing any research data, VA or non-VA, you must have the authority. Let us take a look at the list of the – on the data sharing authorization checklist. If you are collecting data for consented subjects, be sure that the HIPAA authorization contains language that permits data re-use. If the patient authorization does not explicitly state that the data may be re-used, you may need to re-consent and get your subjects to sign a new authorization. This is one more reason why it is so important to think about and plan for data sharing when you are developing your proposal. 

If you do not have consented subjects, you must get a waiver of authorization approved by your IRB. And they must approve the creation or placement of the data in a research repository. The data owners/stewards for each VA data source used in the file must grant permission before any of their data can be shared for re-use. And finally, if your project has agreements with non-VA data sources that are used in creating the final data, they must grant permission before any of their data can be shared for re-use. 

ORD reports that there are several research data repositories already established. So far, most of them have been limited – limiting research to the primary investigator of the original study. Tomorrow, in Session Four, Laurel Copeland will tell us more about one of her projects in which a research data repository was created.

We are about halfway – at the halfway point. So let us pause here to check to see if there are any questions that come in about the material so far. Heidi?

Moderator:No, we do not have any kind of questions right now.

Linda Kok:Oh, good. Now that we have looked at reasons for sharing data and how it should be done in the VA, we will get into the details of planning for data sharing. There are general agreements in the materials we have reviewed for this series from MIT, NIH, and ICPSR that a data-sharing plan should include answers to these key questions. 

What data are to be shared? Usually, this would be the final analytic data set on which the summaries, statistics, and tables are done. 

What is the authority to share for re-use? Well, we have covered that. Where will the data be stored? In the VA, you can store data including data repositories on your local VA network server, a research center server if you have one, or on VINCI. 

How will data be use be approved? The PI can retain responsibility for approving each re-use or delegate that responsibility. Others may also review like the steward of the original data sources or ORG might review if the data contained real SSNs. If you need a – you will need to plan how the request for re-use will be reviewed. Your IRB will review that process.

In what form or format will the data be shared? Public research archives like ICPSR will accept data in several formats. In the VA, it seems like fast data and R are used most often for analysis. The format should work with multiple analytic formats or be easy to convert. If you are going to be archiving image data for re-use, planning the format is even more important.

How will data be provided to users? You can direct – you can allow direct access to the data or create custom data extracts for each request. And this will depend a little bit on where you host the data.

How will the data be protected? Within the VHA, network servers have a high level of security. Each research data repository will need a privacy plan, a gated security plan, and regular backup. Whether you allow direct access or provide data extracts, the process in data protection must be clearly described in your protocol.

A host is a place where your research data repository will be stored and maintained. What makes a good data repository host? Here are some of the attributes of a good data repository host. The interuniversity of consortium for political research has several data archives or data repositories, the way we call it in the VA that store data generated mostly by others for re-use. ICPSR has 700 universities, government agencies, and other institutions using or depositing data with them. They manage the activities required to maintain their research data long-term. And it also curates the data.

Data curation is like the work of an art or museum curator. In the curation process, data are organized, described, cleaned, enhanced, and preserved for use by others in the future. Without curation, data may be difficult to find, use, and interpret. If you create a research data repository, you may take on the role of data curator.

If we take another look at that earlier slide, what makes a good data repository host and focus on VA, all the elements are the same, data security, backup, file recovery, volume capacity, compatible data formats, retention capability, and data access positioning. Luckily, the VA network servers and data security policies provide most of these criteria. It is that last item, data access provisioning capabilities that needs support that is not always found on your local network server.

This brings us back to VINCI, the VA informatics computing infrastructure which already provides access to data for research projects. The PI for each research project identifies those who should be granted permission for access to the project workspace. And the VINCI data managers implement those permissions.

What you are seeing here is the opening page of a VINCI workspace showing the analytics and other software that Denise described earlier. The VINCI project workspaces might also be used to provide access to a research data repository for those who have been approved. The administrator of the research data repository can provide a list of all those who should be granted access. And the VINCI data managers would implement those permissions. 

In addition, VINCI provides extra security because the data – any data being downloaded from VINCI must pass through an auto to data transfer where all the downloaded data are monitored  so that you can feel more secure that your research data – the data in your research data repository are maintained and kept up on VINCI if that is your requirement. If a research data repository were to store its data on VINCI, the PI who created the repository would remain responsible for ensuring that the IRB of records reviewed and approved that placement. 

Creation and use of VA research data repositories in the VA is at its beginning. VIReC and VINCI would like to hear from you if you are interested in creating or have already established a research data repository. We would like to know what is out there and what your needs are.

I would like to stop here for a second question. If there were a central research data repository available in the VHA where you could deposit your research data, how likely would you be to share data from one of your research projects for re-use? Okay. Heidi, I think that they are pretty close. It looks almost equally with maybe winning out a little bit. But 60 percent of you are likely or very likely to think about sharing data from one of your research projects. I think – I am hoping that finding out more today has helped.

In our last topic, we will review the documentation required for re-use. In Sessions One and Two, Denise emphasized the importance of documenting as you go. From the development of a properly worded HIPAA compliant patient authorization form and if you have weaved the documentation into the daily workflow of your project, you will have built an accurate systematic record of your research process that captures your decisions and the reasons behind them. This will reduce mistakes, confusion, and time wasted during data management analysis. And if you decide to share your data for use, it will ensure that you have all the facts at hand for yourself or anyone else who reuses the data you created. The quality of your project and data documentation is in the details.

As the little figure on this slide hints, it takes work. But it is not difficult once you know what to include. On the next two slides, we will show you an excerpt of the ICPSR data deposit form and an example of a code style – code book style variable description from a research project. 

This slide shows ICPSR’s webpage for its Guide to Social Science Data Preparation and Archiving. Circles at left are major sections of the Data Preparation Guide. As you see, it covers the life cycle of a research project from proposal development through analysis and depositing data in an existing data research data repository. 

ICPSR collects information about the data that research wants to deposit using an online tool. And they were kind enough to share a paper version of that with us a few years ago. This slide shows details from the ICPSR data deposit form.

I want to point out three sections. The codebook that provides details of each variable like a cookbook that lists the ingredients and tells you how to put them together to make the dish. It also includes – requires data collection instruments to provide the exact wording of every question and every instruction that was used as you collected your data directly from your subject. The third, the data file inventory, asks for a list for all the data sets that you want to share with the detailed information about each. If you are interested in receiving a copy of the ICPSR Data Deposit Form that we have here at VIReC, please contact us and we will be happy to send a copy along to you. 

This slide shows an example of what a codebook entry for an analytic variable might look like. This is from a study impact of “Medicare Drug Benefits on VA Drug Use, Healthcare Use and Cost” by Kevin Stroupe here at Himes. Starting at the top, there is a brief description of the derived variable. I am not sure I can – I am not sure – oh there, a brief description of the variable and including the variable type and label, the name of the program which created the variable and detailed information about the data source for this variable. In this case, it is three variables in the A Edition enrollment files which code the veterans’ priority category and eligibility for benefits. 

At the bottom, you can see that this example also includes the original fast code algorithm used to derive the variable showing exactly how each of the three output code values, no copay, some copay, and all copay were created. This should give you a picture of what might be included in good data documentation. 

If your documentation is deficient, another researcher should be able to independently read and interpret the data collection and replicate your findings. We have included a data documentation checklist here from MIT Library’s guide. You might want to download this slide just to keep a copy of the checklist. There are five major elements of good data documentation, the project description, and data description, a description of methodology, the citation, and a study abstract.

Project description should include a brief narrative which provides summary information about the data collection. Study descriptions are valuable resources for data users and includes both general information such as the study title, creator, level of analysis, and identifier use, the timeframes for the data collected are used, your funder, and where you did your study.

The data description will include technical information describing each data set including the file format and structure. It will also include a variable list for each data set and a description and format for each variable. Variable labels and value labels should clearly describe the information or question recorded in that variable. Each variable in the data collection should have a set of exhaustives, mutually exclusive codes with definitions for each. Missing data codes should be defined and a description of each derived variable should be included. 

In order for your data to be used properly by you, your colleagues, and other researchers, they must know how you created the data so they can understand  your data in detail. The methodology description in your documentation should include your cohort definition with the reasoning behind the selection criteria. A flow chart can also often make the process of creating your cohort much more clear. The sources of data, for example, the data collection instruments that you used and any existing data sources that you used in your study, a description of the data processing and management techniques that you used to create the analytic data sets, you might want to include a description and perhaps a flow chart of all your data linkages. 

You will need to include – describe your cleaning processes when you were preparing the analytic data set. And any resulting data issues that were found and how you resolved them. The method you used to create the derived variables with detailed technical descriptions including the algorithm like we just saw in the example should be included. Also, in the methodology description section, you should include the reasoning behind each decision that formed your data and your analysis.

Your documentation should include the study citation that others must use to cite your work in developing the data. This way it will be a way for you to become famous, if not rich. You will not include the PI’s name and affiliation and any co-investigators and their affiliation, a descriptive title and study including the time periods and geographic locations. The place of production, places and production of data collection, the date of production, the organizational name of the data producer, your funding or sponsoring agency, your grant number, and you can read on. 

Finally, your documentation should include an abstract of the study that generated the data. The abstract should include theoretical framework that informs the study, the research questions addressed by the study, and any specific hypothesis testing.

Before you offer your data in a research data repository, review your data one last time and your documentation. And then build a good study description and enhance your documentation for understanding by others.

The MIT Library and ICPSR guides have been enormously helpful to us in developing these sessions. Please email us at VIReC@va.gov for these URLs. Remember, VIReC is always looking for ways to help you with data needs. We would like to hear from you about data sharing needs.

These past three sessions contained information that we at VIReC have not previously presented. We would really appreciate your feedback and your suggestions for additional topics of this type. Thank you very much for participating. Now Heidi, are there any questions?

Moderator:We do have one question out here right now. Can VINCI work for qualitative data?

Linda Kok:Is Dr. Heinz still on the phone?

Denise:The answer is yes. VINCI is – just think of it is as a computing platform. It can work – it can store data. There are some capacity issues. They are in the growing stages right now. But VINCI is a place where you can work, you can store, and manage data regardless of whether the data comes from VINCI. It can come from other places.

Linda Kok:Thank you, Denise. I think that we work so hard at making sure that we can fit all of our – the last of Session Two and all of Session Three in that we finished early. If there is anything else that anybody wants to ask, now would be the time to do it? Okay, hearing nothing from Heidi----

Moderator:Yeah, the questions should start rolling in right now so.

Linda Kok:Yeah, I think we have overwhelmed them with too much material.

Moderator:Here we go. Here we go. What do you recommend for staffing to appropriately accommodate the entire data management and data sharing process?

Linda Kok:Wow!

Denise:I can take that one.

Linda Kok:Thank you.

Denise:The answer is it depends. We have talked pretty generically about research with an emphasis on studies that fall into the domain of health services research and clinical research. But the fact is, it really depends on so many of – how complex your study is, how long it goes on, how many data sets, are they complicated to work with, how much analysis is proposed, and how much data sharing you plan. It really is a – it is not something that is easily answered. I would strongly recommend that in any project that is using data, you have somebody on your project who you can describe as a data manager, data coordinator. You need somebody who is going to help you with analytical work whether that is a stat efficient or stat analyst. If you are doing any primary data collection that has very different needs than if you are using all ex stance data. You need somebody who is going to manage the ongoing dynamic flow of data and the whole data collection process. So just some things to keep in mind.

Linda Kok:Thank you.

Moderator:Okay, and we have another question here. Do you know of any applications that help manage the information you just covered?

Linda Kok:I think we had some data management tools in Session One, did we not, as part of final slide on resources on that. And I think that was downloadable.

Denise:On Slide Set One and Two, I actually introduced something that we found on UCLA Library website called the Data Management – DMP Tool. I do not know if anybody has had a chance to try it. We have no experience using it. What I was able to show is – what it produces is a pretty high level data management plan that touches on basically a lot of the documentation checklist topics Linda and I reviewed that are similar to what MIT and ITTS recommend. What I am not aware of, and if anybody knows the answer to this it would be helpful to know whether the DMP Tool can really get at some of the granular kind of workflow and analysis workflow documentation.

Linda Kok:So the answer is there might be something out there that helps.

Denise:Yeah. It is also important to know – in the first lecture, we actually posed the question is anybody familiar with any data management workflow tools. And I think there were six people who responded that said they used them. But some of the examples that came up were tools like Access which I would – that would – I would say that you probably very creative and constructed your own data management planner within Access. But I am not aware that Access has specific customized tools to manage data and documentation. But if there are other software out there, I would love it if you would send VIReC and email so that we can become familiar with some of those tools and we can share it with others at VIReC@va.gov.

Moderator:Great. And that actually is all of the pending questions that we have at this time.

Linda Kok:Great. So I think I can turn it over to Erica now?

Erica:Sure. Thank you to Linda Kok for presenting today’s lecture “Planning for Data Re-use”. If there are any additional questions about today’s content, please contact the VIReC helpdesk at VIReC@va.gov. 

Tomorrow’s session will be a research application of good data practices highlighting proposal planning and development, funding, IRB, and study initiation considerations, documentation, content, location, and study design and implementation. Please join us tomorrow from 1:00 – 2:00 p.m. Eastern. Thank you and have a great rest of the afternoon.

Moderator:Thank you, Erica. And as Erica said at the beginning of the session, I am going to close the session here. And when that happens, we are going to pop up a feedback form. If you could take a few minutes to fill that out, we would appreciate that. Thank you for joining us for today’s VIReC cyber seminar.

Page 1 of 10