Good Data Practices 

VIReC Cyberseminar Series

May 2014

Good Data Practices 2014
GDP 2014 Acknowledgements

• Laurel Copeland, San Antonio VA

• Brian C. Sauer, Salt Lake City VA

• Kevin Stroupe, Hines VA

• Linda Williams, Indianapolis VA

• Brenda Cuccherini, ORD

• Denise Hynes, VIReC

• Arika Owens, VIReC

• Maria Souden, VIReC

2 
Research Life Cycle

3 

Proposal

Protocol & 

IRB

Data 

Collection

Data 

Management Analysis

Project Close

Data Sharing
Good Data Practices Series Overview

May 8

• The Best Laid Plans: Plan Well, Plan Early - 

Jennifer Garvin

May 15

• “The Living Protocol:” Managing Documentation 

While Managing Data - Matt Maciejewski

May 22

• Controlled Chaos: Tracking Decisions During an 

Evolving Analysis - Pete Groeneveld

May 29

• Reduce, Reuse, Recycle: Planning for Data 

Sharing - Linda Kok

4 
Poll #1: About you

• What is your role in research and level of experience? 

▫ Research investigator

New? Experienced?

▫ Data manager/analyst

New? Experienced?

▫ Project coordinator

New? Experienced?

▫ Other – please describe via the Q&A function

5 
Poll #2: GDP Experience 

• How many GDP sessions have you attended or viewed 

online this month, including today’s? 

 1 

 2 

 3 

 4

 

6 
Session 4: Reduce, Reuse, Recycle: 

Planning for Data Sharing

Linda Kok, MA

VIReC

7 
Poll Question #3

• Are you working on or planning a project that will 

produce data that might be shared for re-use?

▫ Yes, for re-use by me (other protocol, same PI)

▫ Yes, for re-use by others (other protocol, different 

PI)

▫ Not at this time

8 
Session 4: Reduce, Reuse, Recycle: 

Planning for Data Sharing

▫ Traditional project close activities

▫ Why re-use research data? 

▫ Project close activities if sharing data

▫ What is a research data repository?

▫ Requirements for a research data repository

▫ Creating a research data repository

9 
10

Traditional project close activities
Traditional project close – protocol 

▫ Notify R&D Committee 

▫ Notify IRB 

11
Traditional project close - data 

▫ Identify the datasets to be retained

▫ Arrange for secure storage until scheduled in 

VHA’s Records Control Schedule (RCS 10-1)

 Check with local facility for procedures

 Non-VA data use agreements may require data 

destruction – refer to the agreement

12
13

Why re-use research data?
Reduce, Reuse, Recycle!

▫ Reduce redundant (& expensive) data preparation

▫ Reuse existing research data

▫ Recycle subsets of research data 

14
Why make research-generated data 

available for re-use? 

“Every day, we create 2.5 quintillion bytes of data — 

so much that 90% of the data in the world today has 

been created in the last two years alone. “ 

August, 2012 

IBM, Bringing Big Data to the Enterprise

15

http://www-01.ibm.com/software/data/bigdata/
Open Data/Open Science

“To fuel entrepreneurship, innovation, and scientific 

discovery”

 - Exec. Order May 2013

16

http://www.whitehouse.gov/administration/eop/ostp
17

“If science is to be 

progressive and selfcorrecting it is critical 

for multiple groups to 

look at the data and 

draw their own 

conclusions and put 

the results in public 

view.”

Harlan Krumholz, MD 

Yale School of 

Medicine

http://www.forbes.com/sites/matthewherper/2014/01/30/instunning-win-for-open-science-johnson-johnson-decides-torelease-its-clinical-trial-data-to-researchers/
NIH data sharing goals

Data should be 

▫ Widely & freely available 

▫ With timely release and sharing

▫ While protecting patient/subject privacy

18

http://grants.nih.gov/grants/policy/data_sharing/data_sharing_guidance.htm
NIH: Reasons to share data

▫ Expedite translation of research into knowledge, 

products & procedures

▫ Facilitate the education of new researchers 

▫ Permit testing of new alternative hypotheses & 

methods

▫ Support methods and measurement studies 

▫ Reinforce open scientific inquiry

19

http://ori.dhhs.gov/education/products/RCRintro/c06/b2c6.html
Why make your data available for re-use? 

▫ To re-use the data yourself 

▫ Save data prep time & $$$ on your next project

▫ Promote your research

▫ Enable new discoveries with your data

20
Is your project data ready for sharing?

• Importance of documenting as you go 

▫ Accurate, systematic record of the research 

process

▫ Capture decisions and the reasoning behind them

▫ Must haves

 Project description

 Data description 

 Methodology description

21
Do you have authority to share?

▫ HIPAA compliant patient authorizations

▫ IRB approval of waiver of HIPAA authorization

▫ VHA data owners/steward permission

▫ Agreements with non-VHA data sources

22
23

Project close if sharing data
Traditional project close 

▫ Notify R&D Committee 

▫ Notify IRB 

▫ Identify and secure the data to be retained

24
Project close if sharing data

▫ Notify R&D Committee 

▫ Get IRB approval for a VA research data 

repository 

▫ Identify data to be shared

▫ And more...

25
26

What is a research data repository?
Research data repository (RDR)

▫ A dataset or collection of datasets produced and 

managed under an IRB approved research 

protocol to provide for re-use for subsequent 

research 

▫ Permits re-use of research-generated data within 

the VHA for IRB approved research protocols 

▫ Data collected for use exclusively for one specific 

research protocol do not constitute a research data 

repository 

27
Research data repository content

▫ Data generated during the course of one research 

protocol to be used by other subsequent research 

protocols 

▫ Data collected specifically for use by more than 

one research protocol 

28
Sources of data in RDRs

▫ Directly from research subjects or from review of 

the subject’s administrative, medical, or other 

records 

▫ Indirectly from existing databases

 Research data

 Non-research sources 

 VA or non-VA sources

29
Two management approaches

1. PI creates a new IRB approved research data 

repository to self-manage

2. PI delegates management of sharing the 

dataset(s) to an existing VA IRB approved 

research data repository 

Note: May need to amend 

the existing protocol either way

30
Summary: When is an RDR required?

31

Primary Data 

Collection?

Existing Data 

Sources?

Identifying 

Information?

To be used for more 

than one protocol?

Data Repository

Required?

Yes No De-identified No No

No Yes De-identified No No

Yes Yes De-identified No No

Yes No IIHI No No

No Yes IIHI No No

Yes Yes IIHI No No

Yes No De-identified Yes Yes

No Yes De-identified Yes Yes

Yes Yes De-identified Yes Yes

Yes No IIHI Yes Yes

No Yes IIHI Yes Yes

Yes Yes IIHI Yes Yes
32

Requirements for RDR
Scientific/ethical oversight committee(s)

▫ To provide scientific and ethical advice for 

repositories that contain a number of different 

databases or provide data broker services

▫ May be the IRB of record

▫ Consult with ORD about the requirement

33
Administration of RDR

▫ VA investigator responsible for all activities of the 

data repository – (not WOC or IPA)

 Name an administrator

 Does not have to be the PI

 A data repository specialist 

 May also be the administrator

 IT support available for granting data access 

34
Administrative Stability

▫ IRB and R&D Committee must approve changes 

to an RDR

 Changes to the repository administrator

 Combining administration of data with another 

research data repository 

35
RDR policies and procedures

▫ Criteria for releasing data 

▫ Request review process

 Procedures for verifying R&D Committee approval 

 Procedures for verifying IRB approval - if 

identifiable data 

 Record of requests & approvals

▫ Data privacy protection 

▫ Data security 

36
VHA data sharing requirements 

▫ VHA Handbook 1200.12 “Use of Data and Data 

Repositories in VHA Research” 

▫ Contact Dr. Brenda Cuccherini at ORD for 

guidance

37
38

Creating a VA RDR
RDR Example:

Policy & Procedures

39

IIR 09-335

Surgical Treatment Outcomes for 

Patients with Psychiatric 

Disorders (STOPP)

Laurel Anne Copeland PhD MPH BS

Central Texas Veterans Health Care 

System, Temple, TX
RDR Example:

Request review process

40

IIR 09-335

Surgical Treatment Outcomes for 

Patients with Psychiatric 

Disorders (STOPP)

Laurel Anne Copeland PhD MPH BS

Central Texas Veterans Health Care 

System, Temple, TX
Consideration for a data repository host 

• Data security

• Scheduled back-ups

• File recovery system 

• Adequate volume capacity

• Compatible data formats

• Long-term data retention capacity

• Data access provisioning capability 

41
42
My VINCI Workspace
Poll Question #4

• If there were a central research data repository available 

in the VHA, how likely would you be to share data from 

one of your research projects for re-use?

5 = Very likely

4 = Likely

3 = Maybe

2 = Unlikely

1 = Never 

44
Poll Question #5

• How likely would you be to use data from someone else’s 

research data repository?

5 = Very likely

4 = Likely

3 = Maybe

2 = Unlikely

1 = Never 

45
46

Resources
Massachusetts Institute of Technology 

(MIT) Libraries

47

http://libraries.mit.edu/guides/subjects/data-management/
Inter-university Consortium for 

Political and Social Research (ICPSR) 

http://www.icpsr.umich.edu/icpsrweb/content/DSDR/deposit.html
Contact Information

Linda Kok, MA

Hines VA Hospital

VIReC@ va.gov 

708-202-2413 

49
VIReC Resources

• VIReC Help Desk

▫ VIReC staff will answer your question and/or 

direct you to available resources on topics

▫ VIReC@va.gov

• HSRData Listserv

▫ Join at the VIReC Intranet web site

▫ Discussion among 800+ data stewards, managers, 

and users

▫ Past messages archived on intranet

• www.virec.research.va.gov

50
Recap 

51

• Why did we do this series?
Session1 The Best Laid Plans: Plan well, 

plan early

▫ Benefits of early data 

planning 

▫ Planning for data 

privacy & security

▫ Feasibility testing

▫ Planning for data reuse after the research

52
Session 2 The Living Protocol: Managing 

Documentation While Managing Data

▫ Living Protocol 

▫ Example: Managing 

secondary data

▫ Example: Managing 

linkage of primary & 

secondary data

▫ Conclusions: The 

value of 

documentation 

“Objective: Share examples 

of conducting these tasks in 

timely manner after....

 Trial and error

 Begging, borrowing and 

stealing best practices from 

other investigators”

53
Session 3 Controlled Chaos: Tracking 

Decisions During an Evolving Analysis

▫ Challenges of 

documentation 

▫ Why good 

documentation? 

▫ A schematic for 

document organization

▫ “Good practices:” 

Documents, 

communications & 

presentations 

▫ Why organize?

“Explain everything clearly 

to your “future self” who will 

have to decipher these 

documents while writing a 

scientific manuscript in the 

distant future.”

54
Whiteboard: 

55

▫ Things you learned?

▫ Things you will use in your research?

▫ Additional topics for future Good Data 

Practices sessions
56

Questions? 
57

Bonus slides
Data planning checklist – Handout to be 

developed

• Formalizing Your Data Management Plan

▫ Description of the project

▫ Description of the data to be collected

▫ Standards to be applied for formats, metadata, etc.

▫ Plans for short-term storage and data management: e.g., file formats, local storage and back up procedures, and security

▫ Description of legal and ethical issues: e.g., intellectual property, 

confidentiality of study participants

▫ Access policies and provisions: i.e., how will you make data 

available to others, any restrictions to data reuse, etc.

▫ Provisions for long-term archiving and preservation

▫ Assigned data management responsibilities: i.e., which persons 

will actually be responsible for ensuring data management; compliance monitoring over time

58
Importance of documenting as you go 

• Accurate, systematic record of the research process

• Captures decisions and the reasoning behind them

• Reduces mistakes, confusion and wasted time 

• Provides the basis for re-using data with confidence

It’s all in the details...

59
Documentation checklist (handout)

 Project Description

 Data description 

 Methodology description

 Study Citation 

 Study Abstract 

60
Project description

▫ A brief narrative & summary information 

about the data collection

 Creator (PI) 

 Level of analysis 

 Identifier used

 Time frame for the data collected or used 

 Funder 

 Where you did your study 

61
Data description

▫ Technical information describing each data set

▫ File format and structure 

▫ Variable list - Name, description, format, label, 

missing data...

• Session 3, slide 25

62
Methodology description

• The Cohort definition 

▫ with decisions & reasoning 

▫ a consort chart – (see Session 2, slide 40) 
Sources of data

▫ Data collection instrument 

▫ Existing data sources

 See table from Session 2, slide 34 

▫ Process description for creating the analytic 

dataset.

64
Documentation of derived analytic 

variable 

65

From: “Impact of Medicare Drug 

Benefit on VA Drug Use, 

Healthcare Use and Cost”; 

Funding Agency: Department of 

Veterans Affairs Health Services 

Research and Development; 

Study Number: IIR 07-165-2 - 

Kevin Stroupe, PI

Note: 

• Description 

• Source

• SAS code 
Study Citation & Abstract

▫ The study citation others must use to cite your 

work in developing the data. 

▫ Abstract of the study 

 Describe theoretical framework 

 Research questions addressed 

 Specific hypotheses tested