**Improving Mortality Ascertainment Using the VA Vital Status Dataset**

>> Welcome to today's VIREC database and methods cyber seminar. The session today is: “Improving Mortality Ascertainment Using the Vital Status Data Set”. Today's presenter is social science analyst, Noreen Arnold and she will be presenting today's session. I would like to turn it now to Noreen. 

>> Thank you, Melissa. And welcome to everyone who is attending the cyber seminar today, we are going to be talking about the VHA vital status file. And hopefully everyone can see the slides. 

The objectives of the session are that at the end of program participants should be able to identify current approaches to ascertain mortality using the VHA vital status file, describe the recent research and methods regarding mortality ascertainment using the VHA vital status file, and describe limitations of the vital status file.  What is out of scope for today's session is cause of death.  We we will mention it briefly when we talk about some of the sources of the mortality data but it is generally out of scope. 

So to meet these objectives, our topics today will be the sources of mortality data, the background creation and contents of the VHA Vital Status File, we will review a couple of examples of published VA studies that have validated mortality data, and lastly we will talk about some possible future enhancements for the file and where to go to get more help in using the file. 

So to get started and I have a couple of questions just to understand who is on the call today.  If you could answer a couple of questions for me.  I would like to understand who has used the VHA Vital Status File and if you've used the master file or the mini file.  So it looks like most of the participants have not used the file yet,  but there are some who have used both the master file and the mini file.  

And the second question, for those of you who have used the vital status file or never used it, how would you rate your knowledge level? [pause] So good, it looks that most of the people on the call today have not used the file yet so there will be a lot of good information for you, and those of you who had used it would like to understand it a little bit more. 

So let's start by talking about the different sources of mortality data that are available. There are basically four sources of mortality data in the VA.  The first being the BIRLS death file and this is really the major VA death file.  It is an extract from the beneficiary identification and records locator system, the BIRLS database. It is produced by the veterans benefit administration and if you use this file you will probably obtain about 64 to 80% of veterans deaths depending on the age of your cohort. If your cohort is older you'll probably see more, 80% of the deaths.  If it’s younger, that’s where you’ll get the 64% capture. This file is updated monthly and it’s available at the Austin Information Technology Center.  

A second file is the VA Medicare Vital Status File and it is a file obtained from CMS. It is for veterans enrolled in Medicare.  And as I am talking today, when I mention veterans, this is not all veterans, but veterans who are known to the VBA and VHA.  I'm going to update this slide a little bit.  Actually what we have now is the annual file that was created on October 31, 2010.  So we have the more current file and information from this file is available in the VHA  vital status file now.  It indicates here in July but it was actually updated in April.  The Medicare vital status file will have deaths for people mainly over 65 because that is the Medicare population and you'll capture approximally 83% of veterans deaths using this file. It’s available from VIReC.

The VHA also has the SSA – I think I skipped ahead too far here.  It looks like we're missing a couple of slides. Oh here we go.  The VA also has the SSA death master file.  It is received from the Social Security Administration. Like the Medicare file, although it has deaths for the younger population, it’s more complete for the over 65 population.  It includes deaths for individuals enrolled in the SSA program since 1936 and there is approximately 87 million deaths on this file. It does contain deaths occurring outside the United States.  And using this file, you'll capture about 89% to 95% of veteran deaths again depending on the age of your cohort that you're looking at. It is updated monthly and also available at the AITC.  

The last source I want to talk available in the VA are deaths that occur in an inpatient setting. These you can find on the medical SAS inpatient datasets for deaths occurring in VA hospitals and if you use the fee basis inpatient data you can identify deaths occurring in non-VA hospitals when the cost of inpatient care is covered by the VA.  And because these are just inpatient deaths you will capture maybe  5 to 12% of veteran deaths using inpatient deaths.  Both of these files are available at the Austin Information Technology Center. 

There are several sources of mortality data that are not available within the VA but you can gain access to them outside of the VA. Those are death certificates and these can be obtained from either state vital statistics offices or the national death index, which is a combined database of all state vital statistics office death certificates.  The national death index is considered the gold standard for death ascertainment.  It is maintained by the national Center for health statistics but it is fairly costly to use and one other limitation of the national death index  is that there is a fair timeline between its update for recent deaths, it’s about 18 months.  So the 2009 deaths are expected to be released by the NDI in June of 2011.  It does contain the cause of death, so if you need the cause of death  for your study you will have to use deaths certificates  to obtain that information.  

And lastly I want to talk about the SSA epidemiologic search. This also provides date of death and it is basically using the same information that is on the SSA death master file.  But it provides a couple of other useful pieces of information.  It will validate an SSN so if you send the Social Security Administration an individual's name and their number will provide information to let him know if that is the same demographic and name and date of birth and their social security number, it will provide information to let you know if it’s the same demographic information and name that they have on their file for that SSN.  It also provides something called the ‘presumed living’ status which the SSA will tell you if they feel the individual is still presumed living based on their administrative data, including payroll deductions, railroad retirement or disability payments, and death claims filed by beneficiaries.  So if an individual is still making payroll deductions for social security, they would be considered ‘presumed living’. This can be used to reduce loss of follow-up in your studies. SSA does include deaths occurring outside of the United States which is somewhat different  than death certificates.  They mostly have deaths that occur in the US.  There are fees for using this file that are similar to the national death index fees. 

Now let’s take a look at the vital status file, its background, how it’s created, and the content. The impetus for the development of the VHA Vital Status File were several, there were several, the main is that mortality is a common and important outcome measure in research. And researchers have found conflicting results using a single data set. For example they have found that they may capture more deaths using the Medicare Vital Status File than the BIRLS death file, and that some of the deaths  on the BIRLS file may not be on the Medicare file and vice versa.  And as we were just discussing, each of these data sets may capture somewhere less than 90% of the deaths. So in 2003 VIReC initiated a study to look at the different sources of death data and what would happen if they were all combined?  How would that improve ascertainment?  And based on this study VIReC recommended the creation of a new VA file of combined mortality data.  If you go to VIReC’s website you can find more information about this study in the VIReC technical report listed here.  

So luckily national data system stepped up to the plate and took ownership of building this file and it became available in October of 2006. This website that I have listed here will provide information on how to gain access to the national -- to the vital status file.  The vital status file is updated quarterly and it includes veterans alive, on, or born after October of 1991.  The reason it is limited to these veterans is that the source files that are used to build the vital status file only go back to that date.  It is composed of actually three files, the master file, the mini file, and SSN conversion file.  

The master file is the largest of the three files and it includes all users of the VHA and VBA.  There is one record for SSN, date of birth, and sex combination.  For those of you who have used VA data quite a bit, I am sure have come across situations where the demographics, the date of birth, sex, and SSN, may vary across the different sources you try to combine.  And that is why you'll find more than one record for an SSN in those instances.  The file that was created in January had over 22 million records for 15.8 million unique SSNs.  Ten  million of those SSNs only had one record on the master file, 5.6 million had multiple records, and 1.7 million were records for non-veteran SSNs. The master file has 112 variables, so there is a lot of variables in the file, and its sort sequence is scrambled SSN, descending score, and we’ll talk about the score a little bit later, date of birth, and sex. 

The mini file is a smaller file , it only has one record per SSN, that is because the data for the multiple records for an SSN are combined into one record for the mini file.  It has 14 million records, but only 16 variables and its sort sequence is a scrambled SSN.  The SSN conversion file can be requested if you need real SSN access. The mini file and master file both the mini file both just have scrambled SSNs.  

If you need to match a cohort to the vital status file to obtain death information then there are a number of decisions that you'll have to make and some issues you may run into. You're going to have to decide whether you want to use the mini file or the master file, and how do you want to match your cohort.  Do you want to match on SSN, date of birth and sex and ensure that all match or just have partial matches, just year and sex, for example?  And when you're doing your merging and matching you may run into a couple of issues.  One, if the demographics that you have for your cohort for an SSN do not match those on the vital status file.  And also you might find instances where there is activity for your individuals and cohort after the date of death recorded on the vital status file.  We are going to cover construction of the file now to help you understand why these issues might occur and which files you may want to use when you're trying to use the vital status file for death ascertainment.

In the next few slides I’ll be covering the construction of the vital status file and this is the national data system’s methodology for building the file.

The first step of the process is to generate the master file.  And in that process VBA data and VHA data from those sources are combined into one record per SSN and date of birth and sex on the master file. So the information is merged on those three variables and NDS will pick up the last activity dates for each source, and also inpatient death dates in this step.

So what are the sources?  The sources include the medical SAS inpatient datasets and census, the inpatient and outpatient encounter, data, non-VA fee files, DSS pharmacy files, enrollment information, and VBA compensation and pension files.  In the last year a new file was added as a VBA source and that is the Veterans Service Network  Corporate Mini Master File.  This file eventually will replace the C&P Mini File but until that occurs both the C&P Mini File and the Veterans Service Network file – VETSNET file -- will be used.  I've included after each of these sources is a three character prefix that is used on the variables that contain information from the source file so when you look at the VHA vital status file master, any variable that has PTF, for example, as the prefix in its name, the data will be coming from the medical SAS inpatient dataset.  

So for each of these sources the data that is contributed to the master file is the data last utilization or activity that is found on that source, for example the last time an individual may fill a prescription through the VA would be indicated in the DSS source last activity date.  And also the inpatient death dates are picked up at this point. 

The next step in the process to build a master file is to merge the death information from the three other death sources: BIRLS , CMS, and SSA.  Now a key thing to note here, as this information is merged on SSN only.  And the information that is added to the master file at this point is the date of birth, sex, and date of death  from the death sources, and also there is a process to select the best date of death from these sources.  So we mean by best date of death?  

As we're going through this presentation today when we talk about best, that is using a routine to select a value for a demographic variable, either date of birth, date of death, or sex associated with an SSN when more than one value for that variable is found in the source data. So in instances where date of birth is recorded differently for an SSN across the multiple sources, the goal of this routine will be to select the true value for the variable in the majority of those types of cases. 

So when the death data is added from the CMS Medicare source, the date of the last CMS utilization is also included as well as race from CMS and a submission flag by year for CMS.  The death information from the BIRLS file is also added.  The SSA death file does not contain sex, just date of birth, so keep that in mind.  The death information from the BIRLS death file is also added.  This information is included based on SSN only, so the demographics, the date of birth and sex for an SSN may not necessarily match the key of the record, the SSN, date of birth, and sex combination.  

So the next step is to select the best date of death for each SSN/date of birth/sex combination found in the VA  or VHA or VBA data, and that routine selects an impatient death if it is present from PTF or FEE file.  Now if there are dates of death available from the other sources -- the inpatient, Medicare, SSA, or BIRLS – there is a routine for possibly selecting that date of death as the best.  That routine is contained in appendix A.  I'm not going to cover that here, but you can use that for reference material.  So once the best date of death is selected, that is included on the master file record. 

So now the master file record is created and I will briefly run through the contents.  As we indicated, there is one record for each SSN/date of birth/gender combination. The date of birth and sex for each of the sources is included on the record.   In the case of VA sources, because of the way the information from the VA is merged, they will have the same date of birth and sex -- all the sources.  But again, for BIRLS, Medicare and SSA since we only merge and match on SSN, they may have a different date of birth and sex on the record. 

The dates of death from each of the source will be on the master file.  Last activity dates from each of the sources will be on the file.  There is one exclusion in selecting the Medicare last date of activity in that durable medical equipment and unpaid carrier claims will not be included.  And flags indicating whether the individual is a veteran will also be on the file.  

As I mentioned before, we pick up additional information from Medicare: race and submission flags.  And the submission flags for each year will indicate if the SSN was submitted to CMS.  And if not submitted, it’d be set to zero, if it was submitted but the individual is not yet enrolled in Medicare it’s set to one, if the individual is enrolled and the SSN is also submitted it is set to two.  So this is also a good source to indicate what veterans are enrolled in Medicare.  And for each date on the master file the mini file there is an indicator that will tell you if the date from the source was complete (it had the month, day and year) or partial which means it may have only had the year or month present, or month and day, for example.  

OK once a master file is created now the mini file is created.  And as I indicated before, all information for one SSN is combined into one record.  Only veteran records are selected for the mini file.  And there is a process to select the best date of birth, and sex and date of death for the SSN, and there is a process to select the last activity date for that SSN.  

So only veteran records are selected.  In the case where there is one record for an SSN on the master file it is quite easy to create the mini file.  So the date of birth and sex on the master file record is just put on the mini file as well as the best date of death that was selected for that record.  If there are multiple records for an SSN on the master file it gets a little more complicated.  The best date of birth and sex from all the records will have to be selected for the mini file as well is the best date of death.

So the routine to select the best date of death for an SSN uses a scoring method, and that is the score we mentioned earlier.  The best date of birth for the utilization dates are selected with a score and the score uses the source as a priority and provides extra points if the data is complete, the sex is present or the date from the VA matches an external source such as SSA or Medicare. Scores can range from 0 to 40.  So each date of birth is scored, and the date of birth with the highest score will then be used in another routine.  The second routine selects the best date of birth from all sources.  Either the best utilization date of birth, Medicare, SSA, BIRLS, enrollment, and VBA comp and pen.  That method is in appendix B.  I will not cover that here but if you’re interested, take a look at it there. 

OK so once we have gone through that process now we have the mini file created .  

The contents of the mini file again are only 16 variables, one record per SSN, it has the best date of birth, sex, and date of death from all the records on the master file, it has the last healthcare utilization date, it has the last activity date from all the sources, and the healthcare utilization date would only be the last date from the sources that indicate healthcare utilization, the last activity date would include those sources as well as enrollment and VBA compensation and pension.  

There is a presumed living status indicator on the file.  I think I skipped ahead a little too quickly here.  This has three values, zero if there is no date of death, one of there is a death date, and two if the death date occurs 31 days after the most recent utilization date.  So using this indicator you can determine if an individual is deceased, if there's no death date and they have activity after the end of your study you can indicate they're still living.  If it is set to two, you may want to investigate these cases  a little bit more, and we'll talk about that shortly. 

So let's take a look at what the records look like on the current mini that was created in January.  For the date of birth, where is the most common best date of birth coming from?  Most of them are coming from and encounter records, 54%, followed by the inpatient file. 

We did a little review of the consistency of the date of birth when there’s multiple records.  We were looking at the master file created in December.  There are 19 million records, 14 million for veterans.  Of those 14 million, 1.4% had greater than one gender recorded.  36% had greater than one date of birth recorded.  And this includes dates of birth that had missing information.  So if you exclude those and look at just those cases where there is conflicting data in date of birth there were only 7.3% of the SSNs that actually had conflicting data.  

So for the date of death source on the mini file created in January , the best date of death was obtained from the Medicare vital status file and 56% of the cases, followed by SSA death master files 24%, the patient treatment files for inpatient deaths at 14%, 3% came from BIRLS, and another 1.5% came from the fee basis inpatient deaths.  So there are a total of 4 million deaths  on the vital status mini file created in January. 

Now that you understand a little bit more about how the vital status file is created and the sources of information, let’s go back to the possible issues that you may run into, where demographics may not match or you find activity after the date of death.  We will run quickly through three examples.  

In the first example, we do have a case where there are two records for an SSN.  They have different dates of birth, in one case it is 7/1/1944 and in the other case it’s 7/15.  These different dates of death are coming from the encounter source.  There is also a CMS date of death recorded.  And the CMS date of birth is 7/15/1944.  

So when the mini file is created, what date of birth goes to the mini file?  Well based on the routine that we talked about, it is going to select July 1st as the best date of birth.  The most recent utilization date is going to come from all -- the most recent of all utilization dates on the file for any of the records.  So in this case it is going to be September 5, 2006.  And we pick up the date of death from CMS, September 10th.  And the presumed living indicator is set to one, meaning deceased. 

In the second example, we again have 2 records for the same SSN but in this case they look like two different individuals.  They have completely different dates of birth and sex.  So what happens when we create the mini file from these two records? 

Well the routine is going to pick the July 15, 1943 date of birth.  But what happens with the date of last activity, because it is selecting the most recent activity from any of the activity dates on these two records, it is going to pick up June 12, 2009. And again CMS has the date of death as September 10, 2006.  So the presumed living indicated is going to be set to activity after death.  So, if you are using this record and you're using the mini file this may be a case where you want to go back to the master file records  to investigate why there is activity after death.  It looks like in one of these instances an incorrect Social Security number may have been recorded for a veteran. 

The last example is just one record for an SSN. In this case, the date of birth is not similar to dates of birth found on SSA and CMS.  So we have SSA and CMS death information from those sources but the date of birth and gender  do not match what we have for the veteran recorded in the VA data.  

So what is generated here on the mini file?  We are going to pick up the best date of birth from VA on the mini file which is August 30, 1980, and gender.  The date of last utilization will also becoming from this one record and the most recent activity data is from the PTF. We're also going to select a date of death from CMS  and SSA. These have different dates of death so the routine actually selects the SSA date of death as the best one.  And the living indicator is set to two. So again it looks like what has happened here for this individual, there is some issue with recording of SSN or their date of birth or gender.  But since we match the date of death information on SSN it is going to be pulled into the mini file.  So this individual truly may not be deceased, it is just that we have some issues with the information recorded for them. 

So points to consider again as you are using the vital status file, you may find some inconsistencies between the vital status file demographic and your cohort demographics.  And to resolve these inconsistencies the master file could be very useful.  There may be some people on this call today from operations, not from research.  Just want to remind you that if you're using the vital status file it should not be used for business operations regarding individual veterans.   It can be used if you're doing analysis at a high level but not for looking on whether an individual veteran is deceased.  

So what is a possible strategy in using the vital status file?  You may want to start with the mini file for your initial search and match just on SSN.  And then, if you look a little more closely and find that you have issues where your cohorts, date of birth, and sex do not match the information on the mini file,  you may want to investigate using the master file.  Because there may be records on the master file with matching demographics for your cohort.  But they were just not the one selected to be put on the mini file.  You may also find activity after death when you merge these.  To investigate those, you can look at the master file and check to see if the date of birth and sex for the date of death source on the master file actually matches the cohort or the individual’s date of birth and sex.  You may also find that the different date of death  sources on the master file may have different dates of death. 

There are some additional steps you may want to consider.  If you find cases on the mini file that have the date of death for your cohort, you still may want as a doublecheck to go back to the master file and obtain the date of death source, date of birth, and sex, and match those to your cohort’s date of birth and sex.  Just to make sure you are picking up a good date of death for the individual.  You may also want to consider using the national death index or state vital statistics offices to obtain death certificates for cases where you have lost this individual to follow up.  These may be cases where there is no death date on the file but there is also has been no VHA or VBA activity so the last activity date is not after your end of follow-up date or end of study date.  But you have to remember that the national death index has an 18 month lag, so that may affect your decision on whether to use the national death index.  And of course, if you need cause of death you'll need to go to the national death index or state vital statistics offices for cause of death. 

OK now I would like to briefly review a couple of studies that have validated mortality data and use the vital status file

The two studies I would like to cover today are one by Sohn on the accuracy and completeness of mortality data in the Department of Veterans Affairs.  And one by Savas titled, “Mortality Ascertainment of Women Veterans: A Comparison of sources of Vital Status Information”.  The Sohn study utilized 3000 veterans with information obtained from VA sources. There were 292 deaths forthese veterans.  The gold standard utilized in the study was the national death index and the study compared  deaths in BIRLS, PTF, Medicare and SSA to those found in the national death index.  The Sava study was similar in its approach but looked at women veterans. It identified those who have a death certificate in a state of Texas recorded and were also on the US national registry of women veterans. The gold standard used in this study was the Texas mortality database.  And the deaths captured in BIRLS, the inpatient data sets or PTF and SSA were compared to the gold standard.

The results of these studies were really quite similar and when you look at sensitivity by age and sensitivity here we mean percent of the gold standard deaths found in a source.  Both Sohn and Savas found that using individual sources you had lower sensitivity or found fewer deaths than when you combined both sources.  And that sensitivity increased with older cohorts, or death at older ages.  So you can get close to 98 to 99% sensitivity combining multiple sources for veterans 65 and older.  

Similarly they looked at sensitivity by data source and you can see that SSA had the greatest sensitivity or most deaths found, followed by Medicare, BIRLS, and lastly PTF because these are only inpatient deaths.  And if sources are combined you can get really quite high sensitivity, 97 to 98%.  The conclusions of both studies were that BIRLS should not be used alone, all sources provide unique cases, and sensitivity decreases with age.  

What are some feature enhancements that we are looking at and NDS is looking at regarding the Vital Status File?

Well VIReC right now is performing a data quality review where we are planning to quantify missing and conflicting demographics for an SSN, date of birth, gender, and date of death.  We are also evaluating the best date of birth selection methodology.  Doing some investigation of cases of activity after death to see if they have any, much bearing on activity after death.  And we plan to produce a technical report of these findings. There's also been some consideration of including the Medicaid data.  Deaths recorded in Medicaid data that VIReC has available for researchers, to see if that might improve the number of deaths  for individuals under the age of 65.  And although it is not on this slide, most recently NDS is looking at obtaining information from the VA’s National Cemetery Association on deaths to include those in the vital status file. 

OK, so where do you go to get more help and detail about the file?  National Data Systems has quite a bit of information on their website and the resident experts there are Dorothea Garrett and Larry Hughes and here is a link to their website.  VIReC also has webpages on the vital status file and on the VA data sources and VBA data sources that are used to build the file, and documentation of those data sets. 

Another good source for information is the HSR data listserv and you can join this listserv at VIReC’s website . And of course, you can always send VIReC Help Desk an email or give them a call for a specific question you have about the vital status file or other data sources.  

So that concludes the presentation today and I think we can open it up for questions now. 

>> Thank you so much Noreen.  There are a few questions from the audience. 

>> OK.

>> Let’s see, I’ll start at the beginning.  “For a pilot study in one VA medical center, is the local inpatient deaths sufficient?  Or the national data system has to be use?” 

>> If you're doing a pilot study you would still have to use the vital status file because -- oh I see, if you're using a local, I would still use the vital status file.  There may be instances where that death of a veteran is not recorded in the local file.  So you may still pick up that death in the Social Security Administration or the Medicare data.  So I would still use the national vital status file. 

>> OK, great thank you.  Next question.  “Does ADIS offer any additional date of death or post-date of death utilization information over and above what is offered by the use of VSF, or the Vital Status File?” 

>> Does the ADISH?  That is a good question.  We would probably have to get back to the individual on that.  There may be an ADISH file that would have information but, actually the vital status file really is updated monthly so it is going to pick up any recent activity throughout the VHA and VBA so I think it is a really good source of date of last activity. 

>> OK, next.  “I noticed on your slide that you say that presumed living status is zero when there is no death date but on the data dictionary for the mini status file it has zero equals ’presumed alive’”.  And they quote the VIREC website for vital status, which is correct.

>> OK

>> Sorry, it has “zero equals presumed dead and one equals presumed alive”.

>> OK

>> Do you want me to repeat that?  I’m sorry.

>> So it’s just reversed.  One is presumed living and zero is ... ?

>> On your slide, presumed living is zero when there is no death and the data dictionary has zero equals presumed dead and one equals presumed alive.   

>> OK.  You know I will have to check into that.  That could be an error.  So we will check into that and respond to that one to make sure which is the correct source. 

>> OK.  Another question.  “What is the best date of birth selection methodology?” 

>> The best date of birth selection methodology is used in those instances when you find more than one date of birth in the different sources.  So that methodology will score a date of birth based on its source and how complete it is, and then I think it is one of the appendices in the presentation, appendix B, that lists the best date of birth methodology for doing that.  So if the individual wants to look at appendix B that should explain it. 

>> OK, great.  Another question.   “Is there any separate procedure to request access to the mini master file that are different from the utilization files?” 

>> Yes. So the vital status mini master file, I think it is on one of my first slides, there is a link to the NDS website on how to obtain access to the vital status file that includes both the master and the mini file. That is a little bit different than methods for the other utilization files access methods. 

>> OK.  There is one more question.  “You had mentioned that it was costly to obtain NDI cause of death data.  For example, for a thousand person cohort, do you have any idea how costly?   Thousands, hundreds of dollars?”

>> For smaller cohorts it is probably not that expensive.  It gets really expensive when you have large cohorts.  For known deaths, I think it was five dollars and that is for known deaths, if you know they are deceased.  So it really starts adding up for the larger cohorts.  I would recommend going out to the national death index website and they have, they will tell you what the fees are and how to calculate them.  And also, if you want to give them a call they are very helpful and will work with you on that fee structure.   

>> OK great.  Thank you Noreen.  At this time there are no additional questions so if you do have questions I encourage you to enter them in.  In the meantime if you could fill out the survey and evaluation tool that is on your screen right now asking for feedback on this session, that greatly helps us in planning future and additional seminars.

>> It looks like we have finished up on the questions, looks like we don’t have anything else coming in right now so if you’d like, we can wrap up a couple minutes early here. 

>> Sure, that would be great. 

>> OK.  Sounds good.  Noreen, I want to thank you for taking the time to prepare and present for today's HSR&D cyber seminar.