**Transcript of Cyberseminar**

**VIReC - Good Data Practices - The Best Laid Plans: Plan Well, Plan Early**

**Presented by Jennifer Garvin, PhD**

**May 8, 2014**

*This is an unedited transcript of this session. As such, it may contain omissions or errors due to sound quality or misinterpretation. For clarification or verification of any points in the transcript, please refer to the audio version posted at www.hsrd.research.va.gov/cyberseminars/catalog-archive.cfm or contact virec@va.gov.*

*Moderator:*Good afternoon and welcome to the first of four sessions of VIReC's Good Data Practices 2014, your guide to managing data through the research life cycle.  Thank you to CIDER for providing technical and promotional support for this series.  As a reminder, a couple housekeeping notes before we begin.  Questions will be monitored during the talk and will be presented to the speaker at the end of the session.  A brief evaluation questionnaire will also pop up when we close the session.  If possible, please stay on till the very end and take a few moments to complete it.  

At this time, I'd like to introduce Linda Kok.  Linda Kok is the Technical and Privacy Liaison for VIReC and one of the developers of this series.  Miss Kok will present a brief overview of the series and today's session and then introduce our speaker.  I am pleased to introduce to you now Linda Kok.

*Linda Kok:*Thank you, Melissa.  Can we start the slides?  There.  Good afternoon or good morning depending on where you are today, and welcome to VIReC's cyberseminar series, Good Data Practices.  The purpose of this series is to provide researchers with a discussion of good data practices throughout the research life cycle and provide real-life examples from VA researchers.  Before we begin, I want to take a moment to acknowledge those who have contributed to this series.  Several of the research examples that you will see were generously provided by Laurel Copeland of San Antonio VA, Brian Sauer at Salt Lake City, Kevin Stroupe here at Hines, and Linda Williams at the Indianapolis VA.  I'd also like to acknowledge the work of our team here at VIReC.  VIReC director Denise Hynes, our project coordinator Arika Owens, and Maria Souden, our VIReC communications director.  Of course, none of this could happen without the great support provided by the cyberseminar team at CIDER.  

The research life cycle, which you see here, begins when a researcher sees the need to learn more, formulates a question, and develops a plan to answer it.  It ends when the study is closed and the data are stored, perhaps for use by others or for destruction at the end of a required retention period.  The chart shown here walks through the research data steps.  The chart is from the Inter-University Consortium for Political and Social Research, or ICPSR, at the University of Michigan.  In addition to conducting research, ICPSR has been home to political and social research data archives since the early 1970s.  

Step one *[pause]* includes planning the project and writing the proposal.  Next is the project start-up and data management plan.  Step three, shown here, includes data collection and creating standards for files, variables, documenting the decisions made and the actions taken so far.  Next comes management of the analytic datasets and documentation of the methods and findings.  In this example from ICPSR, the last three steps are focusing on data sharing and depositing data in an archive, and managing the data once it's in an archive.

In the four sessions that make up this year's Good Data Practices cyberseminar series, we will follow the steps of the research life cycle.  Today, Jennifer Garvin will look at the importance of planning for data in the early phases of research before funding.  Next Thursday May 15th, Matt Maciejewski will describe a way to manage data management called “The Living Protocol: Managing Documentation While Managing Data.”  On May 22nd, Peter Groeneveld will present Controlled Chaos:  Tracking Decisions During an Evolving Analysis.  Finally on May 29th, I will present Reduce, Reuse, Recycle:  Planning for Data Sharing.

Before we jump into session one, we'd like to know more about today's participants.  For this question, we'd like to know about your role and about your experience.  Our question is what is your role in research and level of experience?  In the polling panel to the right, look for the combination that best describes you.  We're already getting some responses.

*Moderator:*Thank you, Linda.  If you are not seeing an adequate description of your position, go ahead and click number seven, other, and then you can write your specific role into the Q and A box in the upper right-hand corner of your screen and that will be displayed momentarily.  Looks like we've got a very responsive audience.  Lots of answers coming in, which we really appreciate.  Looks like our responses are starting to slow down.  We'll give it a few more seconds for anybody who wants to get in a last minute reply.  *[Pause]*  All right.  I'm going to go ahead and close the poll.  Linda, if you want to talk through those briefly, feel free.

*Linda Kok:*Yes.  Thank you, Molly.  It looks like we have a very well distributed group.  We have 15 percent new research investigators, 14 percent, yes, 14 percent new research investigators, 12 percent or 11.3 percent experienced research investigators, 10 percent new data managers or analysts, 23 percent of experienced data managers and analysts.  New project coordinators are 12 percent.  Experienced project coordinators are 13 percent.  I'm not sure if we—we didn't get any other titles in the Q and A box.

*Moderator:*The Q and A box is just now back up on the screen—

*Linda Kok:*Oh, okay.

*Moderator:*- so if anybody would like to specify their position in there, feel free to.

*Linda Kok:*It's nice that we have a good distribution of research experience and the project roles.

*Moderator:*Looks like some answers have come in.  We have a clinical research associate, a data access approval research investigator/analyst, research specialist, research coordinator, independent evaluator, and another evaluator.  Thank you to those respondents.

*Linda Kok:*Thank you very much.  We hope that you will all find helpful ideas for your own projects in today's session and in those to follow.  *[Pause]*  Now, Jennifer Garvin will present “The Best Laid Plans:  Plan Well, Plan Early.”  Dr. Garvin is a research health scientist at the Salt Lake City VA Healthcare System and leads implementation of the Congestive Heart Failure Information Extraction Framework, or CHIEF, a natural language processing system for heart failure quality measurement within the health informatics initiative.  She's also an associate professor in the Department of Biomedical Informatics at the University of Utah in Salt Lake.  Dr. Garvin?

*Dr. Garvin:*Thank you, Linda.  It's good to be with you today.  We have another polling question because we'd like to know more about how you undertake planning for data.  *[Pause]*  When do you start planning for data use for your research?  During the proposal stage, after I get funding, when I prepare the IRB submission.  If you'd please vote, it'd be very helpful.

*[5-second pause]*

*Moderator:*Thank you.  It looks like our audience is streaming in their responses.  We'll give people more time to go through that.  

*[10-second pause]*

All right.  It looks like most of the responses have come in.  We have about 82 percent saying that they start planning for data use during the proposal stage, about 5 percent say when they get the funding notice, and around 12 and a half percent say when they submit the IRB submission.  Thank you.

*Dr. Garvin:*Thank you so much.  It seems as though we have very experienced people today and who start planning early, so this is great.  Great to have you on the call.

*[5-second pause]*

Usually, we plan in advance what our use of data will be when we prepare our proposals, but there can be bumps in the road in actually undertaking the research.  Planning for data use and documenting what we are doing will help smooth out analysis and publication development as well as help us navigate staffing changes.  By documenting our data use, we are developing data provenance.  We should ideally detail how we obtain the data, describe what it is being used for, note how we measure the concepts we're interested in, describe where the data came from, and note if there will be any transformations or computation as well as describe linking datasets and any use of derived data.

My goal for today is to provide some thoughts about data planning and documentation with regard to how it can help your work.  I will also give a few examples from VA research, provide some lessons learned, and then suggest structure for the recording process you may want to consider, such as making the most of the documents you already have which reference data and use them as a start to the data documentation process.

*[5-second pause]*

In today's session, we suggest that you plan how the data will be acquired, where it will be stored, what privacy and security requirements are needed, and how these aspects of data acquisition and storage will affect what the research team plans to do.  We also suggest that you try to actualize the plans you have made to ensure that you can access and use the data you need.  Finally, we suggest thinking strategically about the value of the dataset that will result from your work and what may need to happen to the data after the study, including whether or not it may be reused. 

*[8-second pause]*

Let's talk about some specifics.  What is data planning?  It is thought and related documentation describing how data will be handled during the research project and after.  The benefits of having a plan include forcing you to think about the project work, and it helps to align the research team with the research goals and gives guidance for action.  It helps identify difficult issues before we begin and may help prevent delays.  It provides a written plan for the project team and can be used to provide details of needed action.  Planning for data helps you write your regulatory documents, the protocol, IRB, and provides supportive information for the requests for data.  

In terms of methods and results, data planning affects early methods section documentation, which helps with manuscript development, details the logic proposed, describes the logic of team choices during the course of the project.  Documenting as you go may reduce recall bias and error and will improve efficiency because it reduces searching through notes, emails, codes, or your memory to reconstruct the actions and decisions that led to your results.  It also helps reduce loss of research team memory when staff turnover occurs and it facilitates the process of documenting data provenance, which is becoming increasingly important.  *[Pause]*  Using existing documents such as those described here as a base for data planning will build awareness of potential difficulties and can result in better preparation for various phases of the research, including the analysis phase.  

We can have the best research plan and not anticipate some aspects of it.  Here's an example from a project done by the Stroke QUERI.  Data was abstracted through manual chart review for a cohort of ischemic stroke admissions from 2009 to 2012 for the Indianapolis VA INSPIRE project.  Chart reviews were completed, resulting in a dataset that includes chart-validated stroke admissions.  Individual data elements making up each stroke quality indicator were completed on all subjects so that the specific reason for the ineligibility or for failing any indicator can be determined.  In a new project, we used the abstracted data as a reference standard for comparison to an automated electronic quality indicator, data for which are obtained by using administrative and clinical data, as well as data from the use of natural language processing.  

While the original study focused on developing methods to automate VA stroke measures, one of the results of the project was that there was a manually abstracted dataset of information related to stroke and it's being used now as a reference standard to develop NLP techniques as I mentioned to accurately identify stroke patients who have symptom onset greater than 22 hours—greater than 2 hours before presentation to the medical center.  The onset of symptoms greater than 2 hours of presentation excludes the patient from thrombolysis.  This exclusion cannot be obtained via structured data, and so NLP is needed.  We will resume our research case study in a minute or two.  

Take a minute to review what's presented in the slide.  Will you need data directly from subjects?  If you need existing data, what will you need?  What is the time period for needed data and have you explored if the data is available for that time period?  How much data will be generated and is there server capacity and software capacity?  Do you need to link data from different sources and if yes, how will this be done?  What software will you need?  What methods will you use to protect data privacy and security?  Will you provide your data for reuse?  Even with the majority of these aspects thought through by the INSPIRE project team, Dr. Williams could not know in advance how useful the manual chart review data was to other researchers and reuse was not planned for.  This resulted in having to produce the original text report dataset again in another subsequent study.

Dr. Williams also suggests these lessons were learned.  It would've been good to standardize chart review and develop documentation, to develop standard chart review, a manual, and update it with local examples as they were noted.  To standardize search features and terms.  In other words, again, documenting what was done.  To organize the process for access requests and designate one person from her study to submit and stay in communication via the DART process.  After the fact, it seems that the process of chart review, the administrative process to request data, and the possibility of reuse could've had additional thought.

Usually we tailor our data plans to our methods.  *[Pause]*  These are the criteria that make up a good research question.  Is it feasible?  Interesting?  Novel?  Ethical?  Relevant?  This table from Hulley, Cummings, Browner, et al, in *Designing Clinical Research* from 2007 offers a helpful set of criteria for evaluating a research question.  How can we apply the FINER criteria to our research data?

*[7-second pause]*

We need data that is available and in a usable structure.  The data should be interesting because it answers our research questions and informs our understanding.  It is novel because we may use new methods, data, and develop new tools.  When we use data ethically, we protect subjects' privacy.  The use of it is relevant because we are undertaking research to benefit veterans, patients, and our healthcare systems.

Another important consideration is planning whether or not we will be using existing data or data gathered with a consent process.  This is important because while the use of existing data is associated with a waiver of consent, when we need to consent subjects and reuse is planned, the consent should specify that the data will be reused.  Linda Kok will discuss reuse of data in another cyberseminar, but we wanted to mention the implications of data reuse in planning data at this point as well.

Study design can be informed by prior use of datasets.  I provide a second research data story with lessons learned.  We obtained the discharge instruction document for each patient in our study to determine whether a pre-developed off-the-shelf informatics tool could accurately determine the completion status of discharge instructions for inpatients with CHF, congestive heart failure.  We compared the results of the automated method to both a manually developed reference standard and to the External Peer Review Process, EPRP, abstraction results.

The patient cohort obtained via a data request using ICD-9-CM codes for a principal diagnosis of CHF at the medical center resulted in 152 inpatients.  The number of patients with EPRP data from the same period with the same diagnosis of CHF was 98.  I learned a lesson.  While I understood that 100 percent of the patients with CHF from this medical center were abstracted by EPRP, I had not understood that some patients' abstraction could be delayed to a different quarter and reported later.  I also had not understood that EPRP does not abstract records usually in one month of a given year.  In sum, I learned that these were the reasons I had less patients than I expected.  In a later study, I used this knowledge to inform how I obtain data to have a better match between the patients in my cohort, the related document set, and the EPRP abstraction results.

*[7-second pause]*

Planning considerations are affected by study design.  Limitations of using a dataset that was not obtained in a controlled environment, for example using administrative data to measure comorbidities, plans for linking data, and adjusting for unexpected reductions in study participants based on dynamic operational settings as may happen in partner-based research.

*[7-second pause]*

As I mentioned when I discussed the second case study just now, I learned about EPRP data when I used it.  In a subsequent research project when I was developing methods to automate the inpatient heart failure quality measure, I used the EPRP data to establish my patient cohort and then I requested the text-integrated utility notes, or TIU notes, and structured data based on the patients abstracted by EPRP.  *[Pause]*  This meant that I could anticipate more accurately how many study subjects with documents I would have for the natural language processing task.  This is important because I needed to know in advance if our team would have enough patients to both train and test the NLP system, to undertake these sub-classifications you see here, and the overall classification of whether the patient's documentation met the qualities measure. 

When I was a VA medical informatics fellow, we used a secure VA research server that was associated with our research center.  I now use VINCI for my research.  While the variety of data and tools on VINCI is expanding, it is important to verify how often the data is updated that you need for your study and that the software for statistical analysis or text extraction is actually available.  There are processes to get new software uploaded, but it will be important to start as soon as possible so you can begin the research within your time frame.  

There are a series of steps needed to get access to VINCI.  First, we have to undertake the needed approvals.  Then we operationally access our VINCI folder, meaning sign on and be sure you can access what you expect to access.  Then request data by filling out the VINCI forms.  *[Pause]*  In my research when I request the text notes, I organize them by patient identifier and obtain a random sample.  I review the documents in the random sample manually to determine that I have the documents I need, that they are relevant to my study.  I potentially request specific notes if the ones I expect are not in the document set.  I create a mapping of each standardized name for each medical center that I need for my patient cohort and I use it to keep only documents that are relevant to my study and I  it or not.  In my research, I then assign a unique identifier to each patient and aggregate the notes in chronological order by patient, by discharge date.  This simulates a medical record with a limited document set for each patient and each discharge the patient has.  This provides a logical order so that we can undertake our research.  

We plan, but we may not be able to operationalize the plan fully perhaps at the time that we plan to do it.  For example, in the Houston PACT CREATE's beta-blocker titration project, we based our power analysis on plans to train and test our NLP system on documents from known CHF patients in the VISN, but the study sites we initially interact with are reduced to only two medical centers so we have fewer study subjects than in our original plan.  To address this, we undertook a revised power analysis to determine we have enough power under these circumstances.  Another example from this study is that in the original data pull of TIU notes from VINCI, the data pull returned more documents than we needed.  Once this was determined, we reduced the document set to only those reports with concepts we were interested in extracting.

Tailor your plans to your methods.  *[Pause]*  From this slide forward, I'm going to review some resources that exhibit best practices of data management.  I credit Dr. Salim Virani in the Houston VA for acquainting me with this dataset discussed on the slide.  The National Cardiovascular Data Registry website provides some good data management examples that can be used in planning for studies.  A data dictionary is a great tool.  This registry provides data resulting from a standardized data collection process.  As we learned from the INSPIRE study, in retrospect it would've been good to standardize more of the data collection process for that study.  This registry also espouses undertaking quality checks and advocates documenting changes to data.

Another resource is the Observational Medical Outcomes Partnership, or OMOP, common data model.  Its purpose is to standardize the format and content of observational data so standardized applications, tools, and methods can be applied to them.  The common data model combined with a method for standardizing its content usually via the vocabulary will ensure that research methods can be systemically applied to produce meaningfully comparable results.  I invite you to visit the OMOP web page and the links in this slide for more information, as well as to contact Dr. Ram Gouripeddi.  His email is on this slide.

OMOP related tools have been developed by VA researchers.  *[Pause]*  The generalized modules are called EpiTools.  EpiTools is a VA VINCI funded initiative.  EpiTools are analytic modules designed to be used in work flows that extract, transform, analyze data, and generate reports suitable for publication.  Please contact Dr. Brian Sauer for further information.  I will now turn the presentation back over to Linda Kok.  Before I do, I want to thank the VIReC staff and the content contributors I referenced in the slides for assisting me in developing this presentation.  Linda?

*Linda Kok:*Thanks, Jennifer.  Let's see.  As a reminder, Dr. Garvin and I will both be available for questions at the end of today's presentation, so be sure to put those into the Q and A while you're listening to the presentation so that we can answer all the questions that we can at the end.  I think we may have a good amount of time at the end for questions.  In the next few slides, we're going to take a quick look at additional planning needed prior to submitting your protocol to the IRB including data privacy, data security, and data sharing.  

In your IRB submission, you will be asked to discuss how you will protect the subjects' private health information and identity.  Your data privacy plan should address who will have access to the data, how the data access will be limited, what identifiers your project will need such as real or scrambled social security numbers, names and addresses, or text data that may contain these identifiers.  Be clear about why they're necessary and how they will be used, and describe what special handling you will provide to protect them.  If your study includes vulnerable subjects such as those with drug abuse issues or HIV, how will you handle the data for these special subjects?  

*[7-second pause]*

You'll also provide a data security plan.  This is a description of the technical and procedural protections that you will establish to secure the data.  It includes methods for how you will ensure patient confidentiality and how data access permissions will be granted.  All restrictions to access should be clearly described in your protocol.  Data security issues to consider include where the data will be stored.  It may be on a VHA network server at your local facility or you may want to use it on VINCI.  If your project requires data from an external source such as the Surveillance, Epidemiology, and End Results Program, or SEER, of the National Cancer Institute, you may need to provide real SSNs for your subjects to the agency or program that would provide the additional data.  These types of disclosures of protected health information must be described in your data security plan.  If your project includes multiple sites, you should include a description of how any data transfers between the sites will be done.  These examples are just a few of the security related issues that you need to plan for before you begin collecting data.

*[7-second pause]*

If there's a possibility that your project will produce a new dataset that might be shared at project close either to another protocol that you develop or to someone else, you may want to describe that possibility in your protocol.  Will you set up a new research data repository?  Perhaps you can place your data in an existing repository.  If you're doing primary data collection as Dr. Garvin mentioned and think that you may want to share the data later for additional research, you should include language in the HIPAA authorization signed by the patients that clearly permits the use of the data for subsequent research.  If you decide to place your data in research data repository, planning to get this authorization from the start could keep you from having to track down your subjects to get their permission once you've closed the project.  Early planning for data privacy, security, and data sharing will help you complete your IRB submissions and really get you started on the right foot.

Part of early data planning is finding information about available data and how to get access to it.  We'll look at four VA websites that can help.  *[Pause]*  One good source—if we must say so ourselves—for information about available VA data is the VIReC website.  This slide shows the VIReC home page.  VIReC is an HSR&D resource center that provides information about the content and use of data available in the VA.  Starting on the right, *[pause]* the top arrow points to resources for researchers.  These include data source descriptions, documentation such as frequencies and person counts, and also VIReC's research user guides which offer comprehensive information about a dataset and each variable in it.  Oh, here's the pointer.  I have a hard time with it.  That's what we're talking about right here, the data sources and research user guides.  There's also data reports, summary information, and a complete list of peer-reviewed articles published by VIReC.

The lower arrow down here points to VIReC's educational offerings, such as cyberseminars like this one and tutorials and other presentations.  At left, the upper arrow points to the HSR data listserv, a forum with over 800 subscribers hosted by VIReC where researchers can post questions about where to find data, issues to think about when using the data, or other questions about how to get access and locate and use the data that they need for their research.  The lower arrow indicates the link to VIReC's VA CMS Data for Research website that provides information about Medicare and Medicaid data, as well as the United States Renal Data System data for VA research use.

This slide is from HERC, the Health Economics Resource Center.  HERC is also an HSR&D resource center, but they focus on economic analyses.  They provide information and guidance on economic data in the VA.  HERC trains and advises researchers, makes VA data sources available to them, creates new economic datasets, and disseminates information about economic methods and findings.  The links circled here on the HERC home page, right here, lead to information such as HERC's average cost dataset, fee basis files, and decision support system data, as well as many others.  It's a remarkable source for information on economic data in the VA.

Health Information Governance, of HIG, is part of the VHA Office of Informatics and Analytics, or OIA.  The HIG Data Quality Program provides a strategic framework and operational support to improve the quality of data required to provide and manage healthcare.  It works closely with the VA's IT office to implement data governance and quality processes in the VA.  The Data Quality Program website shown here can provide very helpful information about using VA data, as well as information o specific data quality issues.  This image shows the drop-down menus *[pause]* here for finding their data quality and analysis documents.  First you click on “Business Product Management,” then “Documents.”  Then you'll see “Data Quality Analysis.”  Some titles found there include “Identifying Veterans in the CVW,” “CVW Race Data and Multiple Races,” and “Linking Patient Data and the CVW,” which many people have found extremely helpful when trying to figure out whether it's the ICN, the SSN, or the IEN that they should be looking at if they're working with CVW data.

This is the VINCI Central website.  VINCI's known as a secure high-performance computing environment that also provides many of researchers' favorite analytic tools, all for free.  VINCI is also a source for information about VHA data.  The drop-down menu indicated by the arrow in this slide will take you to information about data available in the CVW.  You would click on “Data,: and then “Available Data,” and it would give you a little information about CVW production and raw data and other data available from CVW.

This slide provides additional ideas on where to find information about available VA data.  It includes the Corporate Data Warehouse Metadata Report, the HDR data listserv that I mentioned.  Patient Care Services has a website that has a lot of information about the data available from them.  Pharmacy Benefits Management Service can provide information.  The VA Corporate Data Monograph lists reviews and publishes a list of all of the datasets available within the VA.  The VHA Data Portal that I mentioned before and the VIReC HelpDesk.

For data access information, the VHA Data Portal is a really good source.  It provides details of all of the request processes.  VIReC Database and Methods Series session on research access to data is updated every year and provided usually in the fall.  *[Pause]*  This slide shows the VHA Data Portal home page.  If you select “Research Access” from the drop-down menu circled here in red—first click on “Data Access” and then you'll have a list of other data pages.  Then click on “Research Access” and that will lead you to a list of available data sources with instructions for requesting access for each one.  

Shown here are the URLs for two sites that can provide comprehensive information on developing a data plan.  If you've downloaded the slides, you'll be about to keep these URLs handy.  The first, from MIT Libraries, includes several planning guides for researchers, including how to prepare a data management plan for research.  *[Pause]*  We used this website extensively as we began to develop this series in order to make sure that we were touching base with all of the important issues.  We found it to be a terrific resource.  From the UCLA Libraries, there's a guide to best practices for management of research data, along with a lot of other tools including the DMP Tools, which is a tool created for developing a complete and comprehensive data management plan.  

*[5-second pause]*

Okay.  That finishes my part of this presentation.  Next week, please join us for “The Living Protocol: Managing Documentation While Managing Data.”  It will be presented by Matt Maciejewski from Durham and we'll all look forward to hearing Matt talk about this concept of the living protocol that he developed for his own research.  Now we have time for questions.  

*Moderator:*Thank you, Linda.  For those of you that joined us after the top of the hour, to submit your questions and comments, please use the Q and A box that's located in the upper right-hand corner of the screen.  Simply type your question into the lower box and press the speech bubble and that will submit your question.  Melissa, I'll turn it over to you for moderation.

*Moderator:**[Pause]*  Thank you.  Right now, there's one question that's regarding the URLs for the websites and wondering if they're provided in the presentation or elsewhere.

*Linda Kok:*The presentation, I'm not sure which ones they're talking about.  The ones here that I talked about, the MIT and UCLA, those are given in the slides.  Molly, are the notes—the notes are not included in the slide sets, correct?

*Moderator:*All of the URLs that are in your slides are also in the handouts.  This person is specifically referring to the VA datasets.

*Interviewer:*Oh, okay.

*[7-second pause]*

I'm sorry, I don't under—I can't see any of the questions.

*Linda Kok:*The Q and A box is in the upper right-hand corner, Linda.  This person is asking for the URLs to the VA datasets.  I'm not sure if they're included in the slides.  

*Linda Kok:*Oh.

*Moderator:*I'm happy to add them.  It looks like somebody has written in that you can go to VIReC.researchva.gov/index.  Another one of vhadataportal.med at va.gov.

*Linda Kok:*Yeah.  Both of those websites are terrific sources for information for VA datasets, where to find them, what's in them, how to use them.  The portal also presents a lot more information on data access.  

*[7-second pause]*

*Moderator:*Thank you.  I'm going to go ahead and open up a chat section and I'll post those URLs there for people so that they can pull them out.

*Linda Kok:*Thanks, Molly.

*Moderator*I do have another question that just came in.  As a master's of science graduate student in biomedical informatics at Rutgers, I am curious about the opportunities for those of us who would like to pursue employment related to informatics.  What would you recommend rising professionals *[pause]* in—I'm sorry.  The question—

*Dr. Garvin:*I can take that question.  This is Jennifer Garvin.  I'm not sure if you're interested in going on to get your PhD, but there are postdoctoral fellowships within the VA for informatics, so that's a great way to get into the VA if you're interested in pursuing your doctorate.  In addition to that, there are VAs in New Jersey.  Normally you want to associate yourself with a research center if you're interested in doing research with informatics and there is a research center at East Orange, I know for sure, in New Jersey.  As well as you can apply for positions that are open on USAJOBS.  Let me know if you need any further information.

*Moderator:*There's an additional part of that question.  That is as current VA employees, how do we gain experience with VIReC or other avenues through VHA to more informatics related opportunities?

*Dr. Garvin:*Let's see here.  How can we gain—*[pause]* well, I know that the HI2, the Health Informatics Initiative, has opportunities for people to apply to be part of the VA AMIA 10x10 course.  That might be one way and that would allow you, then, to get connected to other people who are doing informatics in the VA specifically.  Those are the things that come to my mind.  Is there any other questions?

*Moderator:*The person works at the East Orange VA and wants to know what the research center is called that you were talking about.

*Dr. Garvin:*It's, I believe, there was an HSR&D research center there.  Is the person who's questioning that, are they familiar at all with the HSR&D research center?  *[Pause]*  That's something that I would look for.  It's a Health Services Research and Development research centers.  I believe there's one there.  If it's not at East Orange, there's another location, VA Medical Center with a research center in New Jersey.

*Moderator:**[Pause]*  Okay.  Thank you.  

*Dr. Garvin:*Mm-hmm.

*Moderator:*Another question.  Do we consider using clincialtrials.gov to help with data management?

*[7-second pause]*

*Linda Kok:*I haven't heard of that.  It's possible.  I'm not very familiar with that website.

*Dr. Garvin:*I'm not sure either, but I guess honestly, I think that we can learn lessons about data management from a variety of sources.  Sometimes, to me, the solution comes by looking at websites such as that to get ideas of what to do if my methodology is similar.  I'm not sure if this is referring to actually incorporating something into our dataset within VA and I'm not sure of that, if that's what the question is about.

*[7-second pause]*

*Moderator*Thank you.  We do have somebody requesting further information and an email offline.  I'm getting their request right now so I know who to have follow up with them.  Also, another one of our attendees is suggesting that people looking for the HSR&D resource centers visit the HSR&D home site web page and there are lots of resources there.  That is also listed down in the chat box that's going on at the bottom center of your screen.

*Interviewee:*That's great.  Yes, thank you for doing that.

*Moderator:**[Pause]*  Whoops.  All right.  Well, I am actually going to move the Q and A box over and I'm going to put up the feedback form for our attendees.  Please take a moment and provide us with some feedback.  As I mentioned at the beginning of the session, it is your feedback that helps guide where we go with these presentations and what topics specifically we have presented.  For those of you that are requesting any followup information, I strongly suggest that you contact the VIReC HelpDesk.  They will be able to either put you in touch with today's presenters, provide you additional resources, or perhaps just answer any lingering questions that we weren't able to get done today.  

I just want to mention once again that this was the first in a four-part series and we are going to have subsequent sessions each of the Thursdays of this month, also taking place at 1:00p.m. Eastern.  That will be the 15th.  That will be the 22nd, and the 29th.  You can register for all of those sessions by emailing HSR&D at cyberseminar at va.gov or you can just go to our online registration catalog and sign up there.  Melissa, I will turn it over to you for any concluding comment.

*Moderator:*I just want to remind anyone for any additional questions or information regarding the topic, they can email the VIReC HelpDesk at VIReC@va.gov.  Wanted to thank, again, Dr. Garvin and Miss Kok for taking the time to develop and present today's session.  We hope you'll join us next Thursday at 1:00p.m. for next week's session.  Again, it's entitled “The Living Protocol: Managing Documentation While Managing Data.”  Thank you all for attending today.

*Interviewer:*Thank you.

*Interviewee:*Thank you.

*Moderator:*Thank you.  For our attendees, I am going to leave up this feedback form, so take your time in submitting your responses.  Thank you for joining us today.  Have a good afternoon.

*[End of Audio]*

Page 1 of 13