Implementing and Evaluating INTERACT in VA CLCs
Phase: NA
Conditions: Quality Improvement
Sponsor: VA Office of Research and Development

 
 
 
2019 
COVER PAGE 
PROTOCOL: REDUCING VETERAN’S HOSPITALIZATIONS FROM 
CLCS 
NCT 04195880 
APPROVAL DATE: 06/28/2017 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 1 of 43 
Amendment 15 
 
 
 
 
Reducing Veteran’s Hospitalizations from Community Living Centers 
(CLCs)  
CRE-12-025 
PI: Vincent Mor, PhD 
Version 14:   
 
VA Health Services Research and Development 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 2 of 43 
Amendment 15 
Protocol Title: Reducing Veteran’s Hospitalizations from Community Living Centers (CLCs)  
 
A. ABSTRACT 
 
Although numerous interventions have aimed to reduce re-hospitalizations of community-dwelling older 
frail patients, fewer systematically developed interventions have aimed to reduce hospitalizations from 
nursing homes (NHs). One such program, the Interventions to Reduce Acute Care Transfers (INTERACT) 
program was designed by Dr. Joseph Ouslander and his colleagues. INTERACT is a comprehensive, multi-
level nursing staff training and quality improvement program designed to: help staff identify common 
clinical signs that are precursors of conditions that frequently cause hospitalization; communicate 
residents’ clinical conditions concisely and accurately to the physician; and examine potentially 
avoidable hospitalizations. A multi-state evaluation of 25 community NHs that implemented INTERACT 
found an overall 17% reduction in hospitalization rates and a 24% reduction among the 17 facilities 
rated as fully engaged in implementation. INTERACT has been adopted by CMS Quality Improvement 
Organizations and now has extensive on-line training and implementation documentation materials.   
Despite the promise of the INTERACT program for reducing avoidable hospitalizations, it has not been 
adapted to or tested in VA Community Living Centers (CLCs). Such a program is ripe for implementation 
in the VHA because it is an integrated delivery system without conflicting payment incentives across 
providers. The goal of this study is to implement and evaluate an intervention in VHA CLCs that is 
designed to improve the care of Veterans using CLCs who experience acute changes in their condition 
and at the same time reduce their rate of hospitalization. We are guided by the evidence based 
hypothesis that engagement in the INTERACT program will identify Veterans’ clinical problems earlier, 
help evaluate and safely initiate management of acute conditions in the CLC, communicate more 
effectively with physicians about Veterans’ conditions and thereby avoid the need for hospitalization. To 
test these hypotheses, the following specific aims will be undertaken: 
1. To characterize VHA CLCs in terms of alternate measures of hospitalization for long stay and post-acute 
care Veterans, including Medicare hospitalizations, and to examine regional variations in these rates 
over the period 2007 thru 2012; 
2. (a) To modify, for VHA implementation, and pilot the INTERACT  quality improvement program training 
materials; and  (b) To implement the INTERACT quality improvement program as the intervention in 8- 
15 randomly selected, pair matched CLCs for a 6 month intensive training period and an additional 24 
month implementation and data collection period.   
3. To conduct a quantitative and qualitative evaluation of the implementation of the INTERACT quality 
improvement program in order to characterize the fidelity with which CLCs in the intervention pairs 
participated in training, engaged in regular conference calls, undertook root cause analyses identifying 
why hospitalizations occurred and used the tools in which they were trained; 
4. To test the effect of the implementation of  INTERACT  as an intervention on the rate of reduction in 
hospitalizations from CLCs in the intervention as compared to the matched control facilities and to 
determine via a formalized “structured implicit review” whether the intervention was associated with a 
reduction in “avoidable” hospitalizations and did not lead to negative outcomes like mortality and ADL 
decline; 
5. To conduct a Budget Impact Analysis of the cost of the intervention relative to the “savings” associated 
with fewer hospitalizations 
 
 
 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 3 of 43 
Amendment 15 
B. INTRODUCATION/ BACKGROUND 
 
Elderly individuals with long-term care (LTC) needs are at particular risk for hospitalization; more than 
25% of long-stay nursing home residents are hospitalized in any given 6-month period.[12] Reducing 
hospitalizations from LTC settings is likely to have implications for health as well as health care costs. A 
clinical practice to transfer when indications are unclear is not optimal because the stress of transfer 
and the risk of nosocomial infection and other iatrogenic complications are not trivial for frail LTC 
recipients and can outweigh the health benefits from hospital treatment. A number of studies suggest 
that many hospitalizations for pneumonia could and should be avoided, as outcomes among those 
hospitalized can be worse than those treated without transfer.[13-15] While financial incentives across 
providers may vary, the cost to society of inpatient treatment is almost always more expensive than 
treatment elsewhere. Reducing potentially avoidable hospitalizations from LTC settings has the potential 
to improve health outcomes while simultaneously reducing overall costs for an important segment of 
the population.   
 
While reasons for hospitalization vary widely across diagnoses, there is compelling evidence that 
interventions based on readily available clinical practice guidelines and care paths could have a large 
impact on overall re-hospitalization rates. For example, a majority of hospitalizations in the population 
are associated with infections, cardiac and pulmonary conditions and acute renal failure or 
dehydration.[11, 16, 17] Not surprisingly, many interventions designed to reduce hospitalizations target 
these conditions, but some components of these interventions have potentially wider applicability 
across diagnoses.  
 
Hospitalizations can be loosely classified into those that are potentially avoidable versus unavoidable.[5, 
18, 19] Some hospitalizations may be seen as avoidable because the underlying event is avoidable with 
proper care of a chronic condition. For example, hospitalizations related to diabetes, heart failure (HF), 
and chronic obstructive pulmonary disease (COPD) could be avoided if proper diet, exercise, and 
medication management are maintained. A separate but related distinction can be made between 
hospitalizations that are discretionary versus nondiscretionary, once an adverse health event has 
occurred.[5, 18] While hospitalization is seen as always necessary for some acute conditions such as hip 
fracture, transfer to a hospital is discretionary for other conditions such as pneumonia and dehydration. 
In the discretionary cases, the decision to hospitalize may depend on resources available in the LTC 
setting and family preferences as well as patients’ clinical status.[20] Reducing hospitalizations is most 
feasible among avoidable and/or discretionary events.  
 
A recent report from the Research Triangle Institute examined the dually eligible (Medicare and 
Medicaid) population in community NHs and found that more than one-third of these dually eligible 
beneficiaries were hospitalized at least once, totaling almost 1 million hospitalizations per year.[21] Of 
these, 382,846, or 39%, may have been avoidable, either because the condition might have been 
prevented or because the condition might have been treated in a lower level of care setting than a 
hospital. These potentially avoidable hospitalizations include 241,000 hospitalizations (63%) originating 
from a Medicaid-covered NH stay, 73,000 (19%) from a Medicare-covered SNF stay. In both Medicare 
covered SNF stays and Medicaid covered nursing facility stays, pneumonia accounted for over 30% of 
potentially avoidable hospitalizations. 
 
To our knowledge no comparable hospitalization data exist on Veterans receiving post-acute care (PAC), 
approximately ½ of the CLC population. In community NHs, re-hospitalization of short stay, PAC patients 
is increasingly recognized as a major issue; however, these rates may differ between community NHs 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 4 of 43 
Amendment 15 
and VA CLCs. Case-mix adjusted regional and facility variation in the rate of re-hospitalization among 
PAC Medicare SNF patients has been shown to relate to states’ policies, facility resources and medical 
staff engagement.[7] VHA CLCs have medical and administrative staff that are under central control and 
therefore, at least theoretically, have the same organizational incentives as does the hospital director. 
These aligned incentives could mean that efforts to reduce re-hospitalizations in the VHA should be 
successful. 
     
One systematic review of interventions to reduce hospitalizations from NHs classified interventions into 
those that increased staffing, altered provider incentives, identified high risk patients and introduced 
changes into the system of care.[22] Although changing the financial incentives could decrease 
avoidable transfers, it can also result in inadequate care if the infrastructure to manage residents with 
acute changes doesn’t exist. Disseminating and implementing interventions that can assist NH staff 
manage acute changes in resident clinical status are critical to reducing avoidable hospitalizations of NH 
residents.  
 
Interventions to Reduce Acute Care Transfers (INTERACT) is a quality improvement program involving a 
set of evidence-based clinical practice tools and strategies initially developed under a Centers for 
Medicare & Medicaid Services (CMS) contract to Georgia’s Medicare Quality Improvement Organization. 
The original INTERACT tools were developed with input from a panel of expert NH clinicians and pilot 
tested in three Georgia NHs with high hospitalization rates.  More recently, INTERACT II was evaluated in 
25 NHs in three states in a 6-month quality improvement initiative that provided tools, on-site 
education, and teleconferences every 2 weeks by an experienced nurse practitioner. There was a 17% 
reduction in hospital admissions in these 25 NHs from the same 6-month period in the previous year. 
The group of 17 NHs rated as engaged in the initiative had a 24% reduction, compared with 6% in the 
group of eight NHs rated as not engaged and 3% in a comparison group of 11 NHs. The average cost of 
the 6-month implementation was $7,700 per NH while the projected savings to Medicare in a 100-bed 
NH were approximately $125,000 per year.[11] 
 
Significance to the VHA: In conjunction with our GEC partners, this project is being proposed as one 
component of a general effort to improve care and enhance the medical capacity of CLC nursing and 
medical staff.  If the INTERACT training program is effective, GEC anticipates expanding investment in 
the on-line training system and identifying local champions at CLCs across the country and implementing 
INTERACT.  The training of CLC nursing and medical staff in INTERACT is completely compatible with, but 
expands upon, VHA “culture transformation” efforts. Reducing hospitalizations from CLCs has 
immediate positive benefits on the experience of Veterans since they would be spared the trauma of 
unnecessary transfer to hospital. In addition, INTERACT training advances several principles central to 
culture transformation by promoting Veteran-directed advance care planning and training staff to 
understand Veteran patterns of daily activities to better identify early warning signs and symptoms of 
decline.   
 
Impact of Findings on the VHA: We expect that, compared to control-CLCs, intervention -CLCs which 
fully implement INTERACT will reduce hospitalizations by 20%, identifying avoidable hospitalizations.  
This figure is lower than the 25% reduction in hospitalization among facilities that were engaged in the 
INTERACT intervention.[11]  Consistent with findings from community NHs, these reductions in 
hospitalizations will be achieved without increasing mortality or accelerating functional decline. From 
the VAMC perspective, this means hospital beds will be available for more emergent cases without 
major increases in VHA costs.  

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 5 of 43 
Amendment 15 
C. 
STUDY PROCEDURES Overview:  
 
The INTERACT program, training curriculum and tools will be refined to be appropriate for 
implementation in a VA setting both in relation to standard CLC operating procedures and VHA language 
and policies.  
 
Based upon merged VIReC and Medicare inpatient and outpatient data and CLC nurse staffing and other 
organizational data, we will match CLCs in terms of hospitalization rate, proximity to their VAMC 
hospital, admissions per bed and the mix of short and long stay CLC Veterans they serve.   Up to 15 pairs 
of CLCs will be identified and from each pair one CLC will be randomly selected and approached about 
implementing the INTERACT program. We expect a minimum of 8 of the facilities approached to accept, 
adopt and implement the program over the 6 month training period and the additional 24 month 
implementation and data collection period (encompassing:  a 12 month monitoring and data collection 
period to review the results of the intervention, followed by 12 months of indirect data monitoring).  
Both members of the CLC matched pairs will be dropped from the study if the CLC that is randomly 
selected to be offered the INTERACT program declines to participate.  During the 6 month training 
period and the 12 month active monitoring and data collection period, extensive process measures, 
such as CLC staff participation in online training and support conference calls, will be gathered for use in 
both the implementation analyses as well as the Budget Impact Analysis (BIA).  Additionally, quality 
improvement tools applied by staff to examine the characteristics and “avoidability” of hospitalizations 
during the 6 month training and 12 month active monitoring period will be assembled and coded. There 
will be no contact with pair matched control CLC since outcome data on their hospitalization rates will 
come exclusively from existing data.  Changes in the case-mix composition and seasonally adjusted 
hospitalization rates for both the intervention and control CLC in each pair will be examined by 
comparing data from the two years prior to the introduction of INTERACT with data from the two years 
after the introduction of INTERACT.   
 
Data Sources: To estimate the impact of the INTERACT on the rate of hospitalization from CLCs a variety 
of administrative data will be used. Our strategy for merging data is described below. 
 
VHA Utilization and Costs:  VA utilization will be obtained from the national automated datasets 
maintained at the Austin Automation Center. The VHA Medical SAS Datasets are national administrative 
data for VHA-provided health care utilized primarily by Veterans. The datasets are provided in SAS 
format by fiscal year (Oct. 1 - Sept. 30). These data are extracted from the National Patient Care 
Database (NPCD).  Two particular datasets will be used: the “Bed Section” dataset for acute and 
extended (nursing home) utilization. The “Bed Section” files (as opposed to the Main files) will allow us 
to differentiate between acute hospitalizations and inpatient rehabilitation, which is necessary for the 
creation of the Residential History File (see below) as well. These datasets will be used to obtain specific 
dates of admission and discharge that will enable us to define study outcomes. They also contain 
variables describing patient demographics, primary/secondary diagnosis, length-of-stay and up to 10 
diagnostic code variables per admission.  In conjunction with Dr. Phibbs and staff at HERC (Health 
Economic & Resource Center) we will use HERC cost data based upon standardized cost accounting 
rules. 
 
 
Medicare Claims:  Medicare data for Veterans are available to VA researchers with approved 
projects. The VA Information Resource Center, VIReC, prepares the data and attaches it to VA's 
Scrambled Social Security Numbers, which allows researchers to link VA and Medicare healthcare 
utilization data. The data are distributed to VA researchers as SAS datasets. The VIReC cohort, first 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 6 of 43 
Amendment 15 
constructed for 1999, included a total of 6.4 million Veterans. Of these, there were approximately 6.1 
million Veterans who were alive as of January 1, 1999.  This cohort definition is updated annually to 
include the most recent source files to produce a cumulative finder file. Currently, files include claims 
from 1999 through 2011, and are updated continually.  
 
Medicare's routine administrative databases include detailed demographic, financial and clinical data.  
The enrollment files include demographic data (birthdate, age, gender, race, and place of residence), 
eligibility information (Medicare Part A and B eligibility and periods of HMO enrollment) and vital status 
(date of death).  Claims files are maintained for all beneficiaries not enrolled in HMOs. The hospital 
claims files include a record for each hospitalization (including DRG, multiple diagnoses, procedures, ICU 
days and reimbursements by revenue center). Other part A standard analytic files (SAF) include Skilled 
Nursing Facility, Home Health, and Hospice Files which contain claims with dates of service, charges, and 
the Outpatient file containing data on outpatient hospital procedures and reimbursements. Physician 
claims are part B claims and include dates, diagnoses, procedures and payments. All records in the 
claims and enrollment files include unique identifiers for Medicare enrollees that allow longitudinal 
linkage. Medicare claims are not available for beneficiaries, including Veterans, who enroll in a Managed 
Care plan, now constituting 25% of all Medicare beneficiaries.  We are very conscious of the incomplete 
data resulting from this, but the proportion of long term care users who are covered by Medicare 
Managed Care is considerably lower than for the general population; whether this is true for the VA is 
not known at present. 
 
Nursing Home Assessments: Minimum Data Set (MDS): The MDS resident assessment instrument 
(RAI) is a required clinical assessment done upon admission and periodically thereafter. In this study we 
will use the data elements included in the MDS as independent variables designed to adjust CLC 
hospitalization rates and as dependent variables to insure that reductions in hospitalization don’t 
adversely affect patient functioning. Repeated evaluations of the reliability of the MDS reveal that with 
proper training, 92% of the identification and background items are reliable with an average reliability 
estimate of 0.71, and all diagnoses in the MDS are reliable with an average reliability of 0.74. [24] Inter-
rater reliability data on over 6000 residents drawn from 209 facilities in 10 states were assembled on 
100 different MDS data elements, average Kappa values exceeded .7 in over 90% of the items. The 
validity and reliability of summary scales measuring dependence in physical and cognitive functioning, 
mood, etc. have been documented in comparison with research instruments and other criteria. [25-34]   
 
As of January 1st, 2012 VA CLCs began using the MDS 3.0 as the required resident assessment 
instrument.  Psychometric analyses done as part of the developmental work and field testing of the MDS 
3.0 reveal excellent inter-rater reliability for the new item definitions and many of those measuring 
function were not changed from the MDS 2.0.  Dr. Saliba, one of the investigators in this project, was the 
architect of the MDS 3.0 so we are confident that we’ll be in a position to work effectively with these 
data.[35, 36]  
 
Facility Level Data: Facility information on CLCs is constructed by aggregating MDS and VA as well 
as Medicare claims and utilization data based upon Veterans’ location.  Specifically, we will create 
facility level aggregate measures of hospitalizations from a CLC as well as the proportion of short and 
long stay residents in the facility and the proportion or residents with primary psychiatric diagnoses.  
Finally, geographic proximity to the VAMC inpatient unit will be obtained from GEC as part of the CLC 
matching process.  

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 7 of 43 
Amendment 15 
Data on staffing in the CLCs will be obtained from VA’s Corporate Data Warehouse (CDW) accessed 
through VINCI and stored locally on the secure research server in the designated restricted study drives 
through an existing DART request.  This will be used to characterize the intervention and control CLCs in 
terms of the level of skilled and unskilled staffing at each CLC.  This will help us to interpret the results of 
the comparison of hospitalization rates from the CLCs.   
 
 
Residential History File (RHF): Intrator and colleagues have constructed algorithms that concatenate 
the MDS and Medicare Claims files into individual patient histories able to document trajectories as 
patients move in and out of hospital and other settings. Continuous days of health care utilization at the 
same health care location are assigned in calendar time. In the case of overlapping dates, a hierarchy 
(inpatient claims, SNF, hospice, and home health) is used to assign location of care. Prior experience has 
found that very few claims have overlapping dates of service (less than 0.05%), reflecting the ongoing 
error checking CMS does.  More recently, under a current VA IIR, Intrator has applied the RHF to the 
combined VA-Medicare data, making it possible to track daily health care utilization of Veterans.  This 
algorithm makes it possible to count the number of patient days at risk of hospital admission amongst 
residents in a CLC during any period of time.  It is this feature of the RHF that makes it possible for us to 
precisely measure hospitalization rates in this study. 
 
Implicit Review of Potentially Avoidable Hospitalizations:  To estimate whether the rates of 
avoidable and inappropriate transfers changed with implementation of INTERACT, we will conduct a 
Structured Implicit Review (SIR) of the appropriateness of a sample of hospitalizations drawn from the 
CLCs participating in the intervention. To account for the heterogeneity and complexity of the CLC 
populations between sites, as well as the wide variety of factors that may influence the transfer 
decision, we will use SIR methodology to measure the appropriateness of transfer.  SIR specifies key 
data sources for review and guides reviewers to consider specific domains in evaluating care. SIR has 
been shown to have acceptable reliability where data elements have not been specified in a manner 
that fully supports explicit reviews and where consideration of care elements requires clinical 
synthesis.[5]  
 
Dr. Shay will assist the research team in recruiting up to 17 reviewers experienced with CLC care from 
VA GRECC clinical staff and from VA CLC Medical Directors not working at CLCs involved in the 
implementation of INTERACT.  We will use VA reviewers to facilitate credentialing and access to records. 
VA staff are more familiar with the VA population and the VA electronic health record. The expert 
reviewers will be carefully trained in the use of the SIR.  Dr. Saliba has experience in training clinical 
reviewers in performing SIR. Training will include reviewers independently completing a SIR on 
standardized practice charts. This will be followed by face-to-face meetings among reviewers to discuss 
their judgments and to clarify definitions and the information sources necessary to support a particular 
position viz. the potential “avoidability” of a hospitalization. Practice charts will be reviewed with the 
group discussion continuing until sources of disagreement have been resolved.  
 
Reviewers will return to their facility where they will be assigned records to review from a nested 
random sample of the records of 163 residents transferred from the intervention facilities prior to the 
intervention compared to a nested random sample 163 residents transferred during the supported 
routine care phase (i.e. after the 6 month training period has been completed). Each reviewer will be 
asked to review up to30 charts, each requiring between 45 and 75 minutes.  During the initial review 
period, 25 charts will have a second review to determine inter-rater agreement in the current sample. 
We will conduct monthly calls with the reviewers to encourage completion of reviews and to clarify 
questions or issues arising during reviews.  

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 8 of 43 
Amendment 15 
 
CLC Selection, Matching and Random Assignment:  We will match up to 15 pairs of CLCs to 
ensure at least 8 implement the INTERACT program. Prior to undertaking the facility matching process 
and in consultation with GEC officials, we may exclude facilities experiencing challenges or management 
problems that would make them poor intervention candidates.  We may also exclude from the matching 
process CLCs that are atypical or have very high proportions of long stay Veterans with histories of 
psychiatric conditions, since this population of Veterans has a different profile than our target 
population and are likely to be hospitalized in an inpatient setting for drastically different reasons. 
The following criteria will be considered in creating the matched CLCs for this project.  We will match 
CLCs based upon historical rates of hospitalization from CLC, the proportion of their Veterans who are 
short, post-acute care patients and the geographic proximity of the CLC to the VAMC inpatient unit.  
Additionally we will rely on two main types of characteristics: the monthly count of patient days alive in 
a CLC and the monthly re-hospitalization rate for each CLC. We restricted the analysis to 64 VA CLCs that 
had at least 50 beds, and were in operation during 2012. In addition, to calculate the matches we will 
only use data from 12/2007 to 12/2012.  
The matching was accomplished in two stages. In the first stage, we partitioned CLCs into 10 groups 
using k-means clustering, based on the pattern of the monthly number of patient days alive in each CLC. 
The number of clusters (10) was chosen based on a change in the sum of squared error (SSE) of the 
scree plot. This step ensured that all of the CLCs within a cluster will have similar number of patient days 
alive which also corresponds to the size of the CLC. In the second stage CLCs were matched within each 
of the clusters defined in step 1, based on the monthly re-hospitalization rates per days alive. The 
matching algorithm within each cluster is of the greedy type that searches for the pair of CLCs with the 
smallest Euclidean distance, which is calculated based on the re-hospitalization rates in the last five 
years. Once two of these CLCs are declared to be a “match”, they are removed from further 
consideration. The algorithm continues to find the closest pair remaining, and removes from 
consideration once identified. The process continues until all possible CLCs are matched. This two stage 
process ensures that matched CLCs have relatively similar sizes as well as other characteristics including 
relatively similar historical re-hospitalization rates.  
 
Once CLCs are matched, the randomly selected CLC will be approached to solicit their participation with 
the strong endorsement of GEC leadership. We will approach the randomly selected experimental CLCs 
until up to 15 have agreed.  If a CLC declines to participate, both members of the pair of matched CLCs 
will be dropped from consideration.  The matched “control” case of those CLCs that agree to participate 
in the INTERACT program will not be approached. The “control” status of a CLC will be known to no one 
outside of the project research team and will not be divulged in any project description or publication. 
 
Patient Selection:  We do not anticipate excluding any Veterans in CLCs from the population to 
be included in the outcome analyses. The CLC is the unit of random assignment, meaning the 
intervention changes the skills of CLC staff.  For each CLC agreeing to adopt the INTERACT program, it 
will be as if the CLC is introducing a focused quality improvement program to their staff. As such, no 
individual patient consent will be requested. We will separately examine hospitalization rates of 
Veterans classified as long stay or short stay based upon the pattern of recent admission and discharge 
data, but this will be regardless of their clinical condition or diagnoses. 
 
Outcome Measures:  The primary outcome (on which our statistical power estimates were 
based) will be the overall hospitalization rate calculated as the number of admissions to hospital from 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 9 of 43 
Amendment 15 
the CLC in a month per person year at risk of hospitalization in that month.  This measure has been 
constructed per CLC using VHA-Medicare merged data from 2004 and afterwards obtained under Dr. 
Intrator’s current HSR&D IIR.  The measure will be constructed covering the period from at least the 24 
months preceding the introduction of the INTERACT program until 24 months after the introduction of 
the INTERACT program.  Since hospitalization rates vary substantially during flu season, we have fit an 
ARIMA model to reduce the heterogeneity of variance in this monthly measure of hospitalization, 
thereby reducing the heterogeneity of residual variance.  An ARIMA model will be fit for each CLC in the 
study, then using the monthly difference between the observed to the expected value to compare 
between any two matched CLCs over time. The dependent measure is a facility aggregate which we will 
risk adjust using indirect standardization by stratifying the per CLC month measure separately for long 
stay and short stay person months at risk of hospitalization. [See the Statistical Considerations section 
below for a detailed presentation of the ARIMA model.] 
 
An alternate version of the outcome is a per-person, risk adjusted, probability of hospitalization 
variable which will be calculated and compared across intervention and control CLC pairs.  In 2007, 32% 
of Veteran’s were hospitalized at any time during the course of the year, ranging from under 10% per 
CLC to nearly 50%.  However, since each Veteran had a different number of person months of 
observation, something that varies across CLCs, these individual probabilities will be examined 
separately for short and long stay patients.  For short stay patients, we’ll calculate the proportion with a 
30 day re-hospitalization, adjusting for patient characteristics on the admission MDS and the originating 
hospital diagnosis and length of stay.  For long stay residents (those who are residents of a CLC for at 
least 100 days), we will calculate the likelihood of hospitalization over a 6 month period, adjusting for 
MDS measured clinical and functional characteristics relying upon the MDS assessment closest to the 
beginning of the 6 month period.  Both of these measures have been extensively used in the literature 
using Medicare fee for service claims and have been used by Dr. Mor and his colleagues at Brown in 
preparing these measures for use in the Commonwealth Scorecard system to rate states’ health care 
system.[2][[37]] 
 
The underlying intent of the intervention is to decrease transfers by avoiding decline, appropriately 
managing acute change or decline in the CLC, and improving the use of advance care plans. Transfer 
rates alone do not tell us whether these kinds of changes occurred in association with the focused 
intention of the intervention, i.e. to reduce the rate of potentially avoidable hospitalizations.  We will 
therefore also examine the overall rates of potentially avoidable and inappropriate transfers prior to 
the intervention compared to these rates during the consolidation phase of the intervention (among 
only the CLCs exposed to the INTERACT program), using a tested and reliable structured implicit review 
process.  The Structured Implicit Review (SIR) yields numerous judgments about the need for 
transferring the Veteran to the hospital and attributes this decision to whether identifiable clinical care 
problems had been missed and the expected benefits of hospitalization.  The final outcome variable is 
an ordinal variable ranking the hospitalization in terms of the degree of “avoidability”. 
 
To ensure that adoption of the INTERACT intervention does not result in an excessive reduction in 
hospitalizations to the possible detriment of Veterans, we will  adhere to the Data Safety Monitoring 
plan approved by the Data Safety Monitoring Board (DSMB).  See Data Safety Monitoring Section, on 
page 35 for full details.  
 
The INTERACT Intervention: 
 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 10 of 43 
Amendment 15 
Overview: INTERACT is a quality improvement program that includes extensive training, clinical 
practice, quality improvement 
tools, and strategies to 
implement and sustain the 
program. An overview of the 
program and tools can be 
viewed at 
http://interact.fau.edu.  The 
program’s effect is based on 
three core strategies enabling 
front-line CLC staff to (see adjacent Figure):  1.) identify acute conditions early in their course, thereby 
helping to prevent them from becoming severe enough to require acute hospitalization; 2.)  provide 
communication and decision support tools that assist with the safe and effective management of certain 
conditions in the CLC without transfer to the acute hospital; and 3) educate CLC staff in advance care 
planning and discussions about end-of-life and comfort care plans, thus increasing the use of advance 
directives, comfort care measures, and palliative and hospice care as an alternative to hospitalization 
when appropriate. 
 
INTERACT will be implemented as a quality improvement initiative in each participating CLC. Completion 
of Quality Improvement tools, Communication Tools and tracking hospitalizations (including those which 
may have been avoided) are critical components of the INTERACT program.  The Quality Improvement 
Tool guides the CLC staff in a review of the acute change in Veterans’ condition.  This is documented by 
the use of Communication Tools (Stop & Watch and SBAR Progress Note), what was done in the CLC to 
evaluate and manage the resident and decision making around transfer to the acute hospital when it 
occurs.  
 
Local adoption of new practices such as INTERACT requires a stable management team.  Pair matched 
facilities will be initially reviewed by GEC leadership who would know of management challenges that 
would make the CLC a poor candidate for adopting INTERACT as a quality improvement program.  The 
leadership at the pair matched CLC randomly selected to be offered INTERACT will be sent introductory 
material and will be called shortly thereafter by study team members  who will explain the program and 
what adopting it could mean to the CLC and their affiliated medical center.  We have made provisions 
for the possibility that up to 4 CLCs will decline the opportunity to participate which is why we have 
constructed 17 matched pairs; both members of the pair are dropped from consideration if the one 
randomly selected intervention site declines to participate.  Site visits may be conducted (depending on 
project timing, enrollment uptake and geographic proximity to study staff) to some or all CLCs agreeing 
to participate in order to meet and orient facility-based leadership and the project Champions. CLCs will 
be asked to sign a research agreement enumerating the expectations of the facility and facility 
Champions prior to participating in this project. This research agreement can be found in Appendix 3. 
During the recruitment period Nancy Henry, GNP, PhD, or Maria Carolina Rojido, MHA, (Senior Project 
Coordinators) will review the objectives of the program and mutual expectations, including 
demonstrating the online training materials and the various forms used to document the 
implementation of the intervention. Experience from previous community Nursing Homes quality 
improvement research projects suggests that such meetings are helpful to successfully initiate and 
sustain research and quality improvement projects. CLC project Champions will be responsible for 
involvement of CLC staff in the training and for leading the implementation of INTERACT at their CLC. 
 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 11 of 43 
Amendment 15 
The INTERACT program will be implemented in up to 15 CLCs that agree to participate involving various 
stipulations including: 1) identifying and appointing “Champions” who will be responsible for program 
implementation; 2) agreeing to incorporate the INTERACT program as a “quality improvement” project 
that will be evaluated on an ongoing basis; 3) agreeing to encourage relevant staff to engage in the web 
based training and regular conference calls to review the conditions surrounding each hospitalization; 
and 4) sustain engagement in the INTERACT program throughout the 6 month training and a 12-
18month monitoring and data collection period to review the results of the intervention.  
 
 
CLC-Based Project Champions: In addition to strong support by GEC and VAMC leadership, 
prior studies highlight the importance of selecting enthusiastic, experienced, respected leaders as 
project Champions. Implementing INTERACT with only one Champion is risky since loss of the project 
Champion was associated with a reduction in engagement, non-adherence or drop-out. Thus, at each 
CLC Champions will share responsibility for implementing the INTERACT program.  They will ensure 
participation of CLC staff in the INTERACT training, and engage the medical director, attending 
physicians, nurse practitioners, and physician assistants in the INTERACT program,  participate in regular 
conference calls and organize event briefings to review the conditions under which hospitalizations 
occurred and what the precipitating factors were.   
 
Training: Training and roll-out will occur over the course of a 6 month period followed by a 12-
18 month monitoring and data collection period to review the results of the intervention. The training 
program is divided into two broad periods:   
 
1.) 6 months to complete the training and to participate in the application of the INTERACT Program. 
The 6 month initial training and roll-out period is in line with our experience implementing the program 
at non-VA nursing homes.   This timeframe has proven to be effective for program initiation.  (See 
Appendix # 5, Webinars & Staff Modules); 
 
INTERACT Training Curriculum 
Webinar/Staff Module: VA-INTERACT  Introduction 
Webinar /Staff Module: VA-INTERACT Decision Support Tools 
Webinar /Staff Module: VA-INTERACT Stop & Watch Early Warning Tool 
Webinar /Staff Module: VA-INTERACT SBAR Change in Condition Progress Note 
Webinar /Staff Module: VA-INTERACT Advance Care Planning (ACP) Tools 
Webinar /Staff Module:  VA-INTERACT Quality Improvement Tools 
Webinar /Staff Module: VA-INTERACT Interacting with your Hospital 
Webinar /Staff Module: VA-INTERACT sustainability 
 
 2.) A 12-18 month monitoring and data collection period to review the results of the intervention 
following the 6 month training period. In this second phase of the intervention, research staff will 
continue to host conference calls and participate in more of an observational and advisory role. The 
research team may host occasional refresher trainings and other web-based modules for new or existing 
staff interested in participating.   
 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 12 of 43 
Amendment 15 
During the 6 month training period, the project Champions, as well as key CLC staff, including certified 
nursing assistants (CNAs), licensed nursing staff, ancillary staff, therapists, social workers, administrative 
nurses, the medical director, and medical providers (MDs, NPs, 
and PAs) will be expected to participate in the online training 
curriculum. Night and weekend staff will also be expected to 
complete the curriculum, as many acute care transfers occur at 
night and on weekends.  These types of staff members successfully 
completed training in community-based trials of INTERACT.  
 
The 6 month training curriculum includes a combination of online 
resources and teleconference calls facilitated by one or more of 
the VA-INTERACT research staff. Online resources include eight 
annotated Power Point presentations with voice over and video 
clips of case examples to illustrate the rationale and goals for the 
program and how the INTERACT tools can be incorporated 
effectively into every day practice, and a post-session feedback 
assessment. Individual CLC staff will be encouraged to take part in the designated staff modules hosted 
through the study SharePoint.  Staff will be able to anonymously complete the modules and feedback 
assessment identifying only their job category within the CLC.  Completion of individual webinars will be 
tracked and aggregate level reports generated for facility Champions in order to ensure that key staff 
members complete the curriculum during the training period.  Completion of the modules will be 
tracked through the project specific SharePoint. Training sessions will not be linked to individual CLC 
staff. 
 
Between sessions, suggested implementation activities will include practicing the use of the tools and 
tracking acute care transfer and hospitalization rates using existing information systems in the CLC, 
either paper or electronic formats.  Teleconferences will be held every other week and include 
opportunities for participating CLCs to ask questions, discuss implementation challenges, and share 
successes with other CLCs.  Individual CLC calls as well as calls with multiple intervention CLCs (Cohort 
calls) implementing INTERACT at the same time will be held.    The calls will include the Champions and 
key leadership staff at the CLC. 
 
The subsequent 12-18 month period will be characterized by every other week conference calls. The 
goal is to keep Champions and CLC staff engaged in the ongoing quality improvement process designed 
to reduce hospitalizations.  Calls will be with the Champions and will focus on CLC staff experiences 
using the INTERACT tools, particularly those related to documenting the reasons for hospitalization.   
 
 
The INTERACT Tools: The INTERACT tools were developed and refined with extensive 
input from national experts in nursing home care, as well as from all levels of direct care NH staff and 
medical providers. The tools are consistent with evidence-based and expert recommended clinical 
practice guidelines.[40-43]  
 
There are four types of tools within the INTERACT QI program, illustrated in the figure above:  
1. Communication tools: structured to document communication between nursing assistants and 
licensed nurses (RNs/LPNs/LVNs), between nurses and medical providers (MD, NP, PA), and between 
CLCs and hospitals. These tools include an Early Warning Tool (“Stop and Watch”), Situation Background 
Assessment Request (“SBAR”) Communication Form and Progress Note, Acute Care Transfer Checklist, 
CLC to Hospital Transfer Data List, Hospital to CLC Transfer Data List, and CLC Capabilities List. 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 13 of 43 
Amendment 15 
 2. Decisions Support Tools: including change in condition file cards and care paths for ten conditions 
that are common causes of potentially avoidable hospitalization of Veterans in CLCs including acute 
mental status change, change in behavior with new or worsening behavioral symptoms, dehydration, 
falls, fever, GI symptoms such a nausea, vomiting and diarrhea, shortness of breath, symptoms of CHF, 
lower respiratory illness and symptoms of UTI.  
3. Advance Care Planning tools (ACP): including an Advanced Care Planning Tracking tool, 
communication guide with specific strategies and examples of quotes to use when discussing palliative 
and end-of-life care with NH residents and their families, a template for “comfort care” orders, and 
educational handouts for staff (Identifying Veterans Appropriate for Palliative or Comfort Care) and 
Veterans and their families (Education on  CPR for residents and families, Education on Tube feeding for 
residents and families, Deciding about going to the hospital) .  
4. Quality Improvement tools: include a structured review of acute changes in condition and acute care 
transfers, a form to track transfers and hospitalizations, and a quality improvement summary tool 
 
The version of tools designed specifically for the VA CLCs can be found in Appendix 2, INTERACT VA CLC 
Version 1.0 Tools.   The INTERACT VA CLC tools are designed to be integrated into the everyday care 
process in CLCs, as illustrated in the figure above. The Advance Care Planning Tools are helpful in 
promoting the use of advance directives and palliative care plans with appropriate residents at or within 
a short time after admission, as well as throughout their stay. When direct care staff notices an acute 
change in a Veteran, they can document and communicate the change using the Stop & Watch Early 
Warning Tool. Licensed nursing staff can use the SBAR Form and Progress Note to document and 
communicate their findings to medical providers, and can use the Care Paths and Acute Change in 
Condition File Cards as decision support tools. If the decision to transfer to the acute hospital is made, 
inter-facility communication is structured and facilitated by the CLC Transfer Data List. The Quality 
Improvement tools include a structured review using a root cause analysis approach to assist CLC staff in 
reflecting on their experience in identifying, evaluating, and managing acute changes in condition, and a 
tracking form for acute care transfers and hospitalizations.   
 
A potential barrier to using the INTERACT tools within the CLC setting is that they are not integrated into 
CPRS as electronic templates where CLC clinical staff document encounters and communication.  To 
address the feedback from the pilot (Aim 2A) in Tampa for records to be maintained electronically, the 
SBAR Communication Form and Progress Note used by clinical staff to document a change in condition 
will be available for use electronically at CLCs that obtain approval through their local Medical Record 
Committee (MRC).   All forms will not be created electronically at this time due to considerable time and 
resources and required to make changes to existing VHA systems before we have the evidence that the 
intervention is effective in the CLC setting. 
 
Support by a Senior Research Project Coordinators: During the 6 month training period, the 
Research Project Coordinators will facilitate multi-site conference calls to foster cross pollination of 
ideas between CLCs about implementation, successes, and challenges to implementing INTERACT in the 
VA CLC.   These cohort calls will be held periodically during the 6 month period. The Champions from 
each facility will be participants on these calls. Dependent upon progress in the implementation of the 
INTERACT program the audience on these calls may be expanded to include others in the CLC clinical 
and/or administrative leadership. The calls will be used to:  review and reinforce the use of specific 
INTERACT tools and implementation strategies; share challenges and successes in implementing the 
intervention; review cases in which the INTERACT tools assisted in preventing a transfer and/or in 
managing residents without transfer; and review cases in which a resident was initially managed in the 
facility, but required transfer to the acute hospital.  

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 14 of 43 
Amendment 15 
 
Documenting Intervention Fidelity: The process of implementing and remaining true to the 
fidelity of INTERACT will be documented in a uniform manner across all intervention CLCs  using the web 
based training, documentation of attendance on conference calls, qualitative notes from site visits, and 
from fielding trouble shooting calls.  Specific items of interest that will be examined and reported will 
include: a change in the Champions; the proportion of staff who complete the web-based training; the 
number of Stop and Watch Early Warning Tools used; the number of SBAR Progress Notes completed 
relative to the number of hospitalizations per month; and any increase in the number of CLC residents 
with advance care directives.   Variation between CLCs participating in the INTERACT intervention will be 
one of the foci of an implementation report.  We will also document the association between a global 
assessment of “adherence” to the INTERACT intervention and the degree of reduction in the actual rate 
of hospitalization from CLC by CLC Veteran.  Counts of contacts, calls and numbers of forms filed will be 
differentiated between the first 6 month period and the second 12 month monitoring and data 
collection period. 
In order to evaluate intervention fidelity, we will use data on the number of CLC staff who complete 
INTERACT training, participate on teleconferences, and obtain information from project Champions 
during regularly scheduled calls on the use of specific INTERACT tools (Stop and Watch Early Warning 
Tool, SBAR, Quality Improvement Tools, and Advance Care Planning Tracking). The Project Coordinator 
will track receipt of data and will contact project Champions with reminders and follow-ups if data is not 
submitted. 
 
Aim #1: To characterize VHA CLCs in terms of alternate measures of hospitalization for long stay and 
post-acute care residents, including Medicare hospitalizations, and to examine regional variations in 
these rates over the period 2007 thru 2012. Using data from the merged VA SAS data system and 
Medicare claims, we will construct several different measures of hospitalization from CLCs that are 
applicable to both short and long stay residents.  A facility level measure with limited risk adjustment 
will be calculated as the number of CLC patient days in the year divided by the number of unique 
admissions from CLC.  This measure can be calculated per calendar year, per quarter or per month.  
Using 2007 and 2008 data, we’ve calculated this for all VHA CLCs, although some outliers suggest that 
there are likely inconsistencies in how unit transfers are coded which we will correct under the 
proposed study. Since hospitalizations vary by season, we’ll adjust for each CLC’s own seasonal variation 
(due to latitude) and characterize regional variation as well as variation attributable to the mix of short 
and long stay residents in the CLC and the proximity of the CLC to the VAMC. Of particular interest will 
be tracking changes over time in the rates of hospitalization since in the Medicare FFS system we have 
documented large increases over the last decade.[2]  We will undertake the same analyses using the risk 
adjusted, per patient level hospitalization measures, estimating the proportion of short and long stay 
residents, respectively, that were hospitalized between 2007 and 2012 and to test whether the patient 
level, risk adjusted rate of hospitalization increased, given changes in the acuity of individual entering 
and residing in CLCs. 
 
Aim #2A: To modify, for VHA implementation, and pilot the on-line INTERACT quality improvement 
program intervention training materials.  
The INTERACT quality improvement program was initially developed under a grant from the 
Commonwealth Fund and has since been modified extensively through interventions implemented 
under the auspices of CMS-sponsored quality improvement programs and select state-based efforts 
designed to reduce hospitalizations from nursing homes.[3, 44] INTERACT’s web-based training 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 15 of 43 
Amendment 15 
materials, forms for documenting hospitalizations, and avoidable hospitalization reviews include 
communication tools, quality improvement tools, and decision support tools, with care paths and 
advanced care planning education materials. These will all be modified to use appropriate VA Veteran 
centered language and practices.   
Activities related to modifying and updating the INTERACT training tools to be VA-centric will involve six 
on-site focus group sessions at the Tampa VA CLC. Focus group sessions will be conducted by Nancy 
Henry, PhD (Senior Project Coordinator), under the supervision of Joseph Ouslander, MD, who is based 
at Florida Atlantic University, in close proximity to the Tampa site.  Dr. Mor will serve as director for the 
pilot phase. The focus groups will not involve data collection, metrics, or any contact with CLC residents. 
The CLC personnel will be asked only to avail themselves of their professional expertise to make 
suggestions and/or feedback on the practical use of INTERACT. 
The Senior Project Coordinator (SPC) will coordinate with the Tampa Site Champion to identify those 
staff within the CLC who will volunteer to participate first in a free thirty minute discipline-specific 
introductory learning session about INTERACT. This will involve staff watching a 30 minute video 
provided via the Medline University website. Staff will be provided free access to the website and can 
schedule at their convenience prior to their involvement in the focus groups. Using a classroom setting 
in the CLC, the SPC (FAU) will facilitate (6) fifty minute focus groups which will build on the Medline 
University INTERACT introductory session. These focus groups will be facilitated by the SPC (FAU) along 
with the Site Champion to include scheduling at a convenient time for staff participation. During each 
focus group, CLC  volunteer staff  will be provided a Power Point presentation and paper versions of 
specific INTERACT education materials and tools.  The SPC (FAU) will lead a discussion about the usability 
of the tools in the CLC practice setting. Staff will have the option to offer verbal feedback and/or write 
their feedback on the paper materials during the session. The last of the six focus groups will include a 
discussion of recommendations and/or consideration for use of select INTERACT tools in electronic 
template form for use in CPRS, the VA electronic medical record. Input from these VA CLC volunteer 
staff will provide essential feedback on how best to modify the program and tools to ensure delivery of 
the most effective and efficient version of the INTERACT Quality Improvement Program for VA CLC 
intervention sites. A focus group guide containing detailed procedures around the introduction of the 
INTERACT program tools and training materials can be found in Appendix 4. The Tampa VA CLC staff will 
make a substantial and valuable contribution towards enhancing the use of INTERACT in preparation of 
introducing the modified INTERACT Quality Improvement Program to the CLC intervention sites.  
 
Interaction with the employees of the Tampa CLC is exclusively for engaging their voluntary participation 
in focus groups to gather feedback about improving the INTERACT tools and materials. Feedback from 
staff provided either verbally and/or in written format, will be summarized by the SPC (FAU) but no 
personal data will be collected from any staff or any resident in the Tampa CLC. Furthermore, no 
information gathered during the focus group process will involve topic areas beyond that of refining the 
INTERACT Quality Improvement tools nor any sensitive disclosures that might result from staff mix in 
attendance.  This process will involve minimal risk: the voluntary participation and input sought from VA 
CLC employees does not increase the probability of harm or discomfort. 
(Note:  As we are not gathering any data or information related to the participants in the focus group 
sessions, we are requesting both a waiver of informed consent and will not maintain a master list of 
subjects).  
Upon completion of the (6) focus groups, the SPC (FAU) will coordinate scheduling of thirty minute 
learning  sessions for the day, evening and night staff of one CLC unit with the Tampa Site Champion and 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 16 of 43 
Amendment 15 
CLC leadership, (3 learning sessions in total). During the learning sessions, staff will be provided an 
overview  of the INTERACT program and instructions on the use of 1) the Stop and Watch Early Warning 
Tool, and 2) SBAR tools for use in everyday care of residents.  The staff will be provided a supply of 
paper tools to complete as opportunities arise and submit them to their ward leader for a period of 30 
days. The Data Handling Plan that the Site Champion will utilize is described in the next paragraph. The 
Site Champion will track the number of tools staff use during this period of testing and gather feedback 
about usability. The SPC will provide site visits and/or telephonic support during the 30-day usability 
testing. The Site Champion will share with the SPC (FAU) the verbal or written feedback about the staff’s 
experience with the tools and the total number used. This thirty day period of feedback will add a 
substantial and valuable contribution towards understanding how best to revise the INTERACT tools in 
preparation for use in the CLC intervention sites. At the conclusion of the (6) focus groups and one 
month of tool usage in one CLC ward, the total sum of feedback and/or recommendations offered by 
staff will be used only to revise the INTERACT Quality Improvement Program prior to introducing it to 
the VA CLC interventions sites. This will conclude the activities at the Tampa VA CLC. 
Data Handling Plan (Tampa pilot): The staff will fill out paper tools and hand them off to the 
unit leader, who will ultimately submit them to the Site Champion. The Site Champion will keep the 
tools in a locked cabinet in a secure room at the CLC until the completion of the 30 days. Once a count is 
completed and feedback is summarized, the sample education materials and tools will be disposed of 
securely by the Site Champion, utilizing acceptable VA protocol. Specific details regarding data security 
and permissions for the INTERACT tools that may potentially be used throughout the whole study 
intervention period can be found in Appendix 1, pages 2-4. However, please note that the SBAR and 
Stop and Watch tools are the only two tools that will be tested at the Tampa pilot, and that resident 
information collected by these two tools will only be seen by CLC personnel, not research staff. 
However, research staff will receive verbal or written feedback about the staff’s experience with the 
tools and the total number of each tool used in the 30 day period. All feedback gathered in Tampa will 
be used to refine the INTERACT quality improvement program tools and the education materials in 
preparation for use as an intervention in 8-10 randomly selected VA CLCs. 
 
The INTERACT training tools are grouped into four categories, Quality Improvement Tools, 
Communication Tools, Decision Support Tools, and  Advanced Care Planning Tools  and can be found in 
Appendix 2. These tools will be modified for use in the VA CLCs based on the feedback obtained from 
the focus groups and through a one month period of use of select INTERACT tools in one ward of the 
Tampa CLC. Dr. Christa Hojlo will review all the updated INTERACT tools and materials following the 
completion of the work at the Tampa VA CLC prior to the roll out of the INTERACT intervention for study 
aim 2B (see below).  
Aim #2B: To implement the INTERACT intervention in up to 15 randomly selected, pair matched CLCs for 
a 6 month intervention training period and an additional 12-18 month observation period. 
CLC agreement to participate:  Up to 15 CLCs will be identified as potential participants in the 
INTERACT intervention program based on a pair matching system and random selection of one CLC from 
each pair.  If the selected CLC does not agree to participate the pair will be dropped and we will go back 
to the remaining list of potential pair matches and select another site until up to 15 have agreed to 
participate or we have contacted all eligible sites. 
 
The formal training at each CLC is scheduled to occur over a 6 month period followed by 12-18 
additional months of observation. A minimum level of participation in the training is required to 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 17 of 43 
Amendment 15 
ascertain that the intervention has been embraced by the CLC staff.   The Providence research staff will 
monitor the CLCs’ participation in the intervention. If a CLC is doing poorly and the Champions seem to 
be unable to engage and motivate the staff, we may drop the site or try to recruit an additional 
Champion(s).  This is one of the reasons for matching extra pairs of CLCs. One of the strengths of the pair 
matched design is that if one of the intervention CLCs is dropped we can readily drop the matched 
control site without having to worry about unbalancing the design. 
 
 Each CLC will follow the procedure below to become an active participating CLC in the INTERACT 
intervention. Selected CLCs will be contacted with a letter from the PI, Vincent Mor, PhD, Co-PI Debra 
Saliba, MD, MPH, INTERACT Developer, Joseph Ouslander, MD, and GEC leadership containing 
introductory material, describing the INTERACT training program being proposed.  After the letter, 
follow up contact will be made with each CLC to determine participation.  Sites who agree to participate 
will be asked to sign a Research Study Agreement (See Appendix # 3, Research Study Agreement).   
 
The intervention CLCs that agree to participate will agree to the following stipulations:  identify  
appropriate candidates as Champions who will be responsible for program implementation (and 
replacing a Champion if identified Champion can no longer complete assigned duties during study 
timeframe);  incorporate the INTERACT program as a “quality improvement” project that will be 
evaluated on an ongoing basis; encourage relevant staff to engage in the web based training, 
anonymous feedback and conference calls to review the conditions surrounding each hospitalization 
and provide information on the implementation of INTERACT; and sustain engagement in the INTERACT 
program throughout the training and 12-18 month monitoring of the INTERACT implementation at the 
CLC. 
Implementation of INTERACT Training Curriculum:  Site visits and/or calls will be conducted by 
a project coordinator to review the objectives of the program and mutual expectations.  This includes 
gathering preliminary information regarding QI activities already existing within the CLC, demonstrating 
the online training materials and the various tools, and working with sites to introduce the use of 
INTERACT to hospital personnel as necessary.  A resource handbook will be provided along with access 
to the secured VA-INTERACT CLC SharePoint providing all essential documentation and resources for the 
intervention. We will schedule training times and confirm with Champions to begin distributing 
information (electronic and paper materials will be used) about the upcoming INTERACT Intervention to 
personnel. Providence VAMC staff will assist sites with scheduling, material distribution and resources. 
ALL training will be scheduled based on site and study staff availability.  
 
Eight (30-50 minute) webinars will be held (online or in-person) (at convenient times for the site 
champions, other staff and moderator/facilitators) (See Appendix # 5, INTERACT Webinars & Staff 
modules).  Each live webinar will be recorded for the Champions to reference as well as for any 
Champion who is unable to attend the live webinar to listen after. The recorded (live Champion) 
webinars will be accessible through the individual secure CLC SharePoint or site-specific shared folder 
stored on the secure server at the Providence VAMC  in building 4, room 201  for the sites participating 
in the cohort for each webinar series and only Champions of each CLC in the cohort will have access.   
No recordings will be shared across the cohorts of CLCs participating.  At the beginning of each live 
webinar the Champions will be read a statement about the recording (See Appendix 7, Webinar 
Recording Statement) and given the opportunity to not participate in the live session if they do not want 
to be recorded. This statement will be posted on at the beginning of the each webinar before the 
recording begins for the Champions to read.  Additionally it will be posted on each CLC site SharePoint as 
a reminder to Champions throughout the project.  The recording will not begin until all questions are 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 18 of 43 
Amendment 15 
answered about this and those who wish to decline participation in the recording have been given the 
opportunity to exit the webinar.     
INTERACT VA CLC Version 1.0 tools will be introduced and suggested implementation will be provided. 
(See Appendix # 2, INTERACT VA CLC Version 1.0 Tools) Champions will be asked to engage key 
personnel in online training modules to assist personnel with implementation and get feedback on use 
of tools. The online modules will be available through a secured VA website portal for each site to have 
individual access.  Individual employees will remain anonymous, identifying only their position in the CLC 
when completing the modules and post session quizzes. (Only Providence VAMC Research staff will have 
access to individual responses.  Providence Research staff will quantify the results and share with 
clinicians so they can provide feedback to sites on their implementation status and evaluate process 
measures related to uptake of INTERACT implementation at a particular CLC (See Appendix # 5 , 
INTERACT Webinars & Staff modules).  In between webinars personnel will be asked to use the tools and 
track acute care transfers and hospitalization rates using existing information systems within the CLC 
(paper or electronic). Conference calls with each CLC will be held between webinar sessions to assess 
and support the implementation of INTERACT from the Champions and leadership perspective.  Cohort 
conference calls will be held periodically to get feedback across sites on the implementation process and 
INTERACT uptake, as well as an opportunity for sharing successes and challenges.  
SBAR usage will be monitored regularly throughout the study timeframe for each CLC. (Appendix # 6, 
Interview Guide)Beginning after the introduction and use of SBAR (approximately 2 months into the 6 
month training period), CLC Champions will review SBARs of all CLC Veterans for whom the form was 
completed during the timeframe of interest.  (These forms are completed by CLC nursing staff when 
they notice a clinical change in a patients’ condition that warrants a discussion with the physician 
regarding the need for changes in standing orders.)  
CLC Champions or other identified personnel within the CLC (i.e. informatics or clinical applications 
coordinators CACs personnel, who would have access, as part of their regular job duties at their local 
site) will produce a list of Veterans with a SBAR in their record.  This list will include Veterans names 
(including DOB or last 4 of SSN, only for reference when pulling the full SBAR data within the VistaWeb 
system and CPRS to confirm the correct record is being accessed) and the date the SBAR was used.  The 
list will be securely transferred to an access-controlled and site-specific shared folder stored on the 
secure server at the Providence VAMC in building 4, room 201, sent through encrypted PKI email to the 
project coordinator or to their secured SharePoint designated specifically for each site with a secured 
folder only accessible to those who upload to it from the CLC, the site Champions, the Providence VAMC 
research staff and the Project clinicians who will conduct the review.   
The Project Coordinator will be given access to VistaWeb for each local site participating in the study, to 
search by the ‘INTERACT SBAR Note’ title (named locally with specific naming conventions) created for 
the study locally and obtain the SBAR Note used for each Veteran listed.   This will lessen the burden on 
the CLC personnel as well as provide more accurate and timely data for the study.  The SBAR Note will 
be extracted from VistaWeb into PDF files including the Veterans name and DOB.  These files will be 
stored on the secure access-controlled and site-specific shared folder stored on the secure server at the 
Providence VAMC in building 4, room 201. (See Data Assurances on page 30 for details on data security).  
An excel file will be populated as a key and include identifiable data from each SBAR completed.  This 
key will include the code, linking collected SBARs to the limited data set, which will include dates but no 
other identifiable Veteran information.  This allows for future analysis of the collected data. The key will 
be kept secure by the project coordinator in Providence, stored on the secure drive designated for SBAR 
collection. 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 19 of 43 
Amendment 15 
The coded SBAR data (including a limited data set) and the SBAR review data collected for each case will 
be entered and stored in REDCap which will assist with future tracking, exporting and general analysis of 
the collected data.  
REDCap is a secure data storage method. We will use this data storage option because it provides: 1) an 
intuitive interface for validated data entry; 2) audit trails for tracking data manipulation and export 
procedures; 3) automated export procedures for seamless data downloads to common statistical 
packages; and 4) procedures for importing data from external sources.  
This has been reviewed by the local privacy officer and ISO.  All information  being stored here is 
acceptable and cannot be readily or easily re-identified.  There is no identifiable information stored here 
so it is acceptable to store the data here as well.  REDCAP is approved for use within the VA as a secure 
data storage mechanism.  
The SBARs collected will be shared with the Champions to gather further information on the outcome of 
each SBAR used.  The name, DOB or last 4 of SSN is shared only for the purpose of making it easier for 
the Champions, who already have access to this information in a search of the VISTA system locally, to 
gather the additional details about the SBAR usage for each case without having to do an additional 
search.  These PDF files will be share directly through the secured SharePoint designated specifically for 
site Champions, the Providence VAMC research staff and the Project clinicians who will conduct the 
review with the Champions.  
This process will continue throughout the 12-18 month active monitoring and data collection period.  
A standardized set of questions will be asked of the Champions or other personnel identified to have 
insight into the SBARs for review. The calls will focus on reviewing the SBARs completed within the 
designated timeframe and look to ascertain whether a Veteran was transferred to the hospital or should 
have been transferred sooner, and information about the Veteran’s advanced directives and end of life 
care wishes. (Appendix # 6, Interview Guide)  
 
INTERACT active monitoring and data collection period:  The CLCs will be followed for 12-18 
months after completion of the training period to determine the uptake of INTERACT at the CLCs. The 
12-18 months is an active monitoring and data collection period in which SBAR usage will continue to be 
monitored and the Champions and leadership will be asked to participate in monthly conference calls to 
ascertain that the intervention has been embraced by the CLC staff.  The goal is to keep Champions 
engaged and involve CLC staff in the ongoing quality improvement process designed to reduce 
hospitalizations. Calls will focus on CLC staff experiences using the INTERACT tools, particularly those 
related to documenting the reasons for hospitalization. Champions and leadership will serve as 
representatives of the CLC staff on these monthly calls. If desired the call group could be expanded to 
include others in the CLC clinical and/or administrative leadership, depending upon progress in the 
implementation of the INTERACT program.   The calls will be used to: review and reinforce the use of 
specific INTERACT tools and implementation strategies; share challenges and successes in implementing 
the intervention; review de-identified cases in which the INTERACT tools assisted in preventing a 
transfer and/or in managing residents without transfer; and review cases in which a Veteran was initially 
managed in the CLC, but required transfer to the acute hospital. (Appendix # 6, Interview Guide) 
 
Additionally, during the 12-18 month active monitoring and data collection period following the 6 month 
training, CLC staff will be encouraged by the Champions to continue using their CLC SharePoint  for 
resources on the implementation of INTERACT  and foster continued interest via refresher webinars or 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 20 of 43 
Amendment 15 
online trainings, and engaging newly hired clinical staff in the on line training.   They will also be 
encouraged to complete the Quality Improvement Tools (i.e. Quality Improvement Tool for Review of 
Acute Care Transfers and Quality Improvement Summary Worksheet) using the electronic versions 
(fillable PDF version of the paper forms found in Appendix 2) on their secure SharePoint.  The 
Champions can also complete these by paper and choose to scan and email them encrypted with PKI or 
fax paper copies to the Providence Research team for review as well.   
The Quality Improvement tools include a structured review using a root cause analysis approach to assist 
CLC staff in reflecting on their experience in identifying, evaluating, and managing acute changes in 
condition, and a tracking form for acute care transfers and hospitalizations.  The completed tools will be 
available for the research staff to review and identify areas to continue helping with implementation 
and improved outcomes.   
Additional training sessions to assist with proper usage and reinforce the use of the INTERACT tools will 
be offered as necessary to the Champions.   Also, the research team may host occasional refresher 
trainings and other web-based modules for new or existing staff interested in participating.     
 
 
Exit Survey:  To finalize the intervention and elicit feedback on the full intervention an exit 
survey (See Appendix 8- VA-INTERACT Exit Survey) will be conducted with CLC study Champions. The exit 
survey will gather specific information on how much of the VA-INTERACT QI program was utilized by 
each CLC, continued plans for use of VA-INTERACT within the CLCs, and in their opinion if the INTERACT 
QI program can be fully integrated into the daily care practices of CLC staff.   
This information will be gathered by research team members who did not participate in study aim 2b 
and did not have any interaction with CLC staff or Champions. The individual champion responses will 
remain anonymous to all research staff except those who are responsible for collecting the exit survey. 
The champions will be notified by the intervention staff about the exit survey. They will be contacted 
after completion of the intervention by other study staff members regarding this final survey.  It is 
entirely voluntary and their individual responses will remain anonymous to all other study staff except 
those sending and collecting the exit survey by email.  
The Exit Survey questions (See Appendix 8) will be emailed to all Champions within 30 days of the CLCs 
final contact with intervention research staff as part of the follow up and monitoring period.  The 
intervention research staff will notify the exit survey staff when this last contact occurs.  The exit survey 
research team will send an email message (See Appendix 8) with the attached interview questions to be 
completed at the Champions convenience and sent back to the exit survey research team by email. The 
exit survey team will send a reminder email to those champions who have not responded after one 
week and a second reminder after week two if no response received.  The reminder will be the same 
scripted email originally sent with an additional line in each indicating “This is a reminder we have not 
received your exit survey responses yet”. The final email will say “This is a reminder we have not received 
your exit survey response yet, this is our final attempt to collect this information.  Please respond by 
returning the attached survey.” Those are the only reminders that will be sent.   
A follow up phone call may be scheduled with any individual champion to clarify or expand on any 
specific content from the returned exit survey deemed important to further our understanding of the 
use of VA-INTERACT in the VA CLC setting.  This will be scheduled with those champions at their earliest 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 21 of 43 
Amendment 15 
convenience. Which champions receive a follow up phone call will be determined based on the 
following:   
First, they must be willing to be contacted by checking yes to the final survey question regarding future 
phone contact (if left blank, we will consider them as having checked no and they will not be eligible for 
follow up phone contact).  Additional criteria for determining who will receive a follow-up call includes: 
1.) Any responses that includes a novel or unique challenge or success  that further details could result 
in a better understanding of how VA-INTERACT could be integrated into the CLC setting or 2.) any 
incomplete or unclear narrative  response that with further conversation could contribute to a better 
understanding of how VA-INTERACT can be integrated into the CLCs. 
If determined a follow up call would provide valuable feedback on VA-INTERACT usage in the VA CLC 
setting based on the above criteria, the exit survey team will email the designated champion requesting 
a time to speak over the phone to expand or clarify some of their responses.  Through email, the exit 
survey team member and champion will determine a time to schedule this call. The exit survey member 
will call the champion at the designated time.  If they are unavailable at this time, a message will be left 
regarding a time to reschedule.   Also an email requesting at time to reschedule will be sent, this email 
will remind the champion that their participation is voluntary. If they no longer wish to participate, they 
can reply to the email letting the exit survey member know and no future contact will be made.  If no 
response is received from the champion a second email will be sent a week later to try to reschedule the 
call. This will be the final attempt and it will be noted in the email that this is the final contact.  Only if a 
response from the champion occurs will any further contact be made to schedule this call.  
The information gathered will be entered into their survey responses to those narrative questions being 
followed up on from that champion.   
The exit survey research team will compile the completed interviews, stripping them of the CLC and 
Champion name, replacing with an ID.  Only the exit survey research team will have access to connect 
the champion or CLC with their interview responses.  The exit survey staff will keep the coded list of 
champions’ names and IDs and their responses on a secure server within their local VA system.  The 
local VA exit survey team will be responsible for analyzing and providing the outcomes from these 
interviews to the rest of the research team 
 
Aim #3: To conduct a quantitative and qualitative evaluation of the implementation of INTERACT in 
order to characterize the fidelity with which CLCs in the intervention participated in training, engaged in 
regular conference calls, undertook root cause analyses identifying why hospitalizations occurred and 
used the other tools in which they were trained.  
 
 The proposed documentation of the implementation of the INTERACT intervention will have both a 
quantitative as well as a qualitative component.  We will rely upon the data recorded as part of the 
documentation of the intervention process to characterize the extent of adherence to INTERACT over 
the course of the training period and across the participating CLC sites. Each site will be characterized in 
terms of the following:  percentage of direct care staff who initiated and completed the on-line training; 
the percentage of training calls attended by the Champions; the proportion of hospitalizations for which 
a root cause analysis was completed by the nursing staff; and the number of SBAR and Stop & Watch 
tools reported on by the Champions per week. To document changes in the intervention from the 
beginning of the training period to the end of the monitoring period, we will calculate some metrics on a 
monthly basis to share with the CLC, allowing for the likelihood that the level of adherence of the 
intervention will change over time, presumably rising throughout the 6 month training period and 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 22 of 43 
Amendment 15 
beginning to decline during the monitoring period.  
 
The qualitative component of the intervention will be undertaken throughout the monitoring period 
during the numerous regularly scheduled calls with CLC representatives. An audit trail of anecdotal 
feedback derived during these calls will be kept in a restricted folder on the SharePoint which only 
Providence Research staff on this project will have access to.   
 
We are aware the data may be impressionistic and may not accurately reflect the variation in how the 
intervention was implemented over the duration of the project.  However, having information on the 
utility of the INTERACT tools, how they were used in practice, and how that use changed from the start 
of  the active 6 month training period  compared to the 12 month monitoring period will be very 
valuable in ascertaining fidelity  to INTERACT protocols and uptake of intervention. This may help explain 
possible differential results between facilities.  
 
Aim #4:  To test the effect of the INTERACT intervention on the rate of reduction in hospitalizations from 
intervention CLCs as compared to the matched control CLCs and to determine via a formalized structured 
implicit review (SIR) process whether the intervention was associated with a reduction in “avoidable” 
hospitalizations and that it did not lead to negative outcomes like mortality and ADL decline.  
 
We will test the effect of INTERACT on the rate of hospitalization from CLCs taking advantage of both the 
paired control CLCs as well as the differences in the hospitalization rates among intervention facilities 
before and after the implementation of the intervention. The primary outcome variable is the monthly 
rate of hospitalization, which, after adjusting for seasonality and trends in the rate over several years 
prior to the intervention, is compared to the rate of the paired comparison CLC.  To allow for regional 
differences in seasonality, separate ARIMA models will be run for each CLC and then we will use the 
monthly difference between the observed to the expected CLC hospitalization rate value to compare 
between the two matched CLCs, aggregating these differences overall pairs. Technically, the average 
monthly rate of hospitalization before the intervention will be compared to the average monthly rate 
after and the difference between these two rates will be compared to the differences in the rates of the 
comparison facilities, using a standard difference in difference design. 
 
A similar approach will be used with the person level, risk adjusted hospitalization rates for short and 
long stay residents calibrated as an annualized rate and there will be fewer facility years than were 
facility months under the approach described above.  The unit of analysis will be the CLC resident but 
the standard errors will be clustered at the facility level. 
 
Using the results of the Structured Implicit Review process, we will determine the percent of hospital 
transfers rated as avoidable through better quality of preventive care and early change recognition and 
the percent rated as inappropriate because the Veteran’s condition could have been safely managed in 
an average VA CLC.  We will compare the percentage hospitalized before the implementation to that 
after implementation. The sample size of 326 hospitalization records, 163 pre and 163 post-INTERACT, is 
based upon the assumption that the pre-intervention rate of potentially avoidable hospitalizations will 
be 40%. For the overall project, we are hypothesizing that there will be a 20% reduction in 
hospitalizations.  Logically, the reductions in hospitalizations will come primarily from the "avoidable" 
type (otherwise we have a causal inconsistency).  Thus, if 40 of 100 hospitalizations pre INTERACT are 
avoidable and all of the 20% reduction in hospitalizations (20 of 100) will come from among those that 
are avoidable, we expect a 50% reduction in the rate of avoidable hospitalizations.  So, if the pre rate is 
40% and the post rate is 20%, using a one tailed test at the alpha .05 level, we need about 100 records 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 23 of 43 
Amendment 15 
before and 100 after to achieve a power of .90.   In the event that some "unavoidable" hospitalizations 
are really avoided due to the intervention, we increase the sample size to 163 per time period. 
 
Finally, we will test for unintended consequences by examining changes in the rates of overall mortality 
and in the rate of decline in the late loss ADLs, as measured by the MDS 3.0 nursing home resident 
assessment, which is done upon admission, periodically thereafter and upon discharge.[45]  
 
Random selection of records with a transfer: A total of 326 medical records with a transfer to 
the hospital and across the eight intervention CLC sites will be reviewed, 163 pre-INTERACT intervention 
and 163 post-intervention.   
The 163 pre-INTERACT records will be selected from a list of transfers that occurred during the 18 
months prior to INTERACT being introduced to the CLCs.   This list will be compiled by the study 
programmer/analysts who will pull it from current CDW data and MDS data.   Once the list is generated, 
random selection will occur to obtain 163 records pre-intervention.   
A second list of transfers will be generated across the intervention CLCs for the post-intervention period 
and a random selection of 163 of these transfers will also be selected for review.   
Once each set of records is selected, the 326 records will be combined and randomized.  The 
randomized list will be randomly divided amongst the reviewers (and include a randomized portion of 
the records to be used for training on how to complete the SIR).   
 
Reviewer Recruitment and training:  Up to 17 physicians or other clinical experts from within 
the VA and with a geriatric background will be recruited by Dr. Debra Saliba and her team from 
LAVAMC.  These individuals will be added as study team members and will be required to complete a 
two-day training (in-person or web based), sign confidentiality and privacy agreement, be provided 
access through CAPRI (Compensation and Pension Record Interchange) /VistaWeb to the 8 intervention 
sites through an amendment to the DART DUA for this purpose, provided access to a secure restricted 
folder within the Providence VA Research Server and access to their secure SharePoint.   
To avoid the use of INTERACT influencing the reviewers’ structured implicit review we will NOT recruit 
anyone from our existing CLC intervention sites.   
After the reviewers complete their assigned chart reviews they will participate in a debriefing with the 
LA team, notifying them about the use of INTERACT as the QI intervention and the impact their reviews 
will have on determining INTERACT’s effectiveness at preventing ‘avoidable’ hospitalizations from 
occurring within the CLCs.    
The reviewers will be trained to use CAPRI/VistaWeb to complete the chart reviews.  CAPRI/VistaWeb 
provides a standardized, user-friendly method to access the charts for review within VA.   
 
Structured Implicit Review (SIR) Procedure:  Each reviewer will be randomly assigned an equal 
percent of the 326 charts to review based on the number of reviewers.  The charts will be randomly 
assigned, from across the intervention sites to the reviewers.  The reviewers will not be provided any 
specific information about the intervention conducted using INTERACT or any indication of whether it 
was a pre or post intervention chart.   
The reviewers’ will then complete their review of the charts using the SIR Instruction Manual and Guide 
(See Appendix 7, Structure Implicit Review Instructions and Guide).  They will utilize  access to a 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 24 of 43 
Amendment 15 
restricted folder within the Providence VA Research Server and PKI encryption by email for this purpose 
throughout the process to do the reviews and share information with the research team, from 
Providence and LAVMC related to assistance with the SIR process.  
 
The reviewers will complete each review using the SIR Instructions and Form (appendix 7).  They will 
save each review to the secure designated restricted folder.  If they choose to hand write their 
responses, they will scan the original review in and save on the secure restricted designated folder or 
send encrypted (PKI) to the project coordinator in Providence to be stored on the secure restricted 
folder within the Secure Providence VA Research Server.  The original hand written copies will be stored 
locally in a designated locked location by the reviewer for the specified time frame noted in RSC-10.  
 
While the reviewer will have access to complete medical records, only specific identifiable information 
from any medical record will be shared with the research staff (from Providence and LAVMC) to assist 
them in making sure they are accessing the correct record prior to their reviews.  That information 
would include the Veterans name, Date of birth, last four of SSN, treatment dates being reviewed and 
the information contained in the Review Form (appendix 7).   
 
REDCap may be used to input the reviews for analysis. The reviews would be input by Providence VA 
research team members, stripped of any identifying information and used to confirm the hypothesis 
that a reduction in avoidable hospitalizations was accomplished by the use of the INTERACT 
intervention, specifically the use of the SBAR change in condition progress note within CPRS.   
Review by the local privacy officer and local ISO confirms that the use of REDCap for the requested data 
is acceptable and within the guidelines of use.   
REDCap is a secure data storage method. We will use this data storage option because it provides: 1) an 
intuitive interface for validated data entry; 2) audit trails for tracking data manipulation and export 
procedures; 3) automated export procedures for seamless data downloads to common statistical 
packages; and 4) procedures for importing data from external sources.  
 
Aim #5:  To conduct a Budget Impact Analysis (BIA) of the INTERACT program from a VA perspective, 
using the 2-year study follow-up period as the time horizon.  Given the important public policy 
implications for Medicare, we will conduct a secondary BIA from the perspective of the entire Federal 
Government.  The purpose of the BIA is to provide VA managers with the budget impact information they 
need to manage the start-up and implementation of the program if this project demonstrates that 
INTERACT works in VA.  Preliminary non-VA studies have found INTERACT to save more than 15 times 
the cost of the intervention.  Thus, even if INTERACT is less effective in VA, it still should yield significant 
savings.   
 
The BIA will be driven by the other Aims’ estimates of the VA effectiveness of INTERACT.  As part of the 
intervention, we will carefully track both the start-up and on-going labor associated with INTERACT.  
Actual VA labor costs will be used to value this time.  The cost savings of the program are not directly 
observed as they are the hospitalizations that INTERACT prevents, and thus must be estimated.  We will 
obtain data on all VA and Medicare hospitalizations and other types of care.  VA costs will be obtained 
from the DSS fee-basis files, and the Medicare reimbursement will be used as the estimated costs of 
non-VA care.  The first analysis will be to compare the hospitalization costs between the intervention 
and control sites, including looking for offset effects for other types of care (reduced hospitalizations 
being partially offset by higher costs for other types of care, including higher costs in the CLC).   
 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 25 of 43 
Amendment 15 
We will conduct a full sensitivity analysis of the estimated savings, of how quickly they accrue, and of 
potential off-setting cost increases.  The extent of the savings will be driven by how the intervention 
affects both the rate of hospitalization, and of the types of hospitalizations (costs vary significantly by 
the types of conditions treated).  Another concern is that the DSS cost estimates include fully allocated 
overhead costs.  The details in the DSS cost data will be used to provide information about how the 
potential cost savings are distributed across variable direct care costs (e.g., supplies and nursing labor), 
costs that can only be changed over time (e.g., reallocation of space to other types of care), and 
allocated overhead costs that will not change with the program. 
 
 Statistical Considerations: 
 
We estimate our power to detect an expected decrease in hospitalization rate over a 24 month period 
using simulation analysis, over different scenarios with different sample sizes (# of CLCs), to predict the 
expected power of our pair matched experiment. Previous research reveals that a Auto-Regressive 
Moving Average (ARIMA) model characterizes hospitalization rate trends with high accuracy. In our 
simulations analysis we assume that each CLC’s rate follows an ARMA(2,2) model of the following form: 
, where ρi  the correlation of the current to the previous i  months, θi is 
the parameter that describe the effect of the error at time t-1 on the current hospitalization rate, εt is 
normal random with mean zero and standard deviation of σε and c is the overall expected 
hospitalization rate. The simulation analysis included 5 parameters: 1) the sample size in the treatment 
group; 2) an increase/decrease in the hospitalization rate variance σε (F); 3) the size of the effect (c); 4) 
the accuracy of the decrease σc;.; and, 5) the variability of ρi, θi of the nursing home that received 
treatment from the matched nursing home that received control σd. For each configuration of the five 
parameters, the following algorithm was repeated for 100 times: 1. a sample of n yearly hospitalization 
rates was chosen from currently observed empirical hospitalization rates and assigned to treatment and 
control CLCs; 2. sample  ρi’s, θi’s from the above ARMA(2,2) model randomly generated for each of n 
treatment CLCs; 3. for each i treated nursing home we perturbed the ρi’s, θi’s by a factor of σd, and 
assigned them to the pair matched control CLC; 4. generate 2n ARMA processes using the ρi’s, θi’s 
generated in step 2 and 3, and the variance σε/F; 5. decrease the hospitalization rate in the treated 
group gradually by 5c% to 20c% over 24 months + random deviation of σc; 6. calculate the monthly 
differences between the treated nursing home hospitalization rate to its matched control, and average 
across 24 months (τj) and calculate the standard error στ ; and finally, 7. use the average and variance 
calculated in 6 to obtain overall weighted average 
and its pooled variance 
. 
To find the factors with the largest effect on power, we calculated the MSE of all the main effects and 
their interactions. The F, n and c are factors that capture 93% of the variability in the data. Figure 1 
shows how the # of CLCs and F (variance of hospitalization rate) affect the power of our analysis. . Even 
if the monthly reduction in hospital rate will be as low as 10%, using 8 treatment CLCs will result in .77% 
power.  
 
Figure 4: Power Estimates, Number of CLC’s by Effect Size and Variance in Monthly Hospitalization Rates 
Effect 
20% Reduction in Hosp. Rate 
 
10% Reduction in Hosp. Rate 
2
2
1
1
t
t
i
t i
i t i
i
i
c
Y
Y


−
−
=
=
=
+
+
+


ò
ò
1
1
1
J
j
j
j
J
j
j





=
=


1
1
1
J
j
j


−
=










Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 26 of 43 
Amendment 15 
size 
# CLCs 
6 
7 
8 
9 
10 
 
6 
7 
8 
9 
10 
.5 
1 
1 
1 
1 
1 
 
.99 
1 
1 
1 
1 
1.0 
.98 
.99 
.99 
1 
1 
 
.7 
.74 
.77 
.81 
.84 
 
Preliminary Studies:   
 
The investigative team assembled to undertake this study is exceptionally strong and well suited to 
implementing the proposed study. Dr. Mor has been studying nursing home quality and the clinical, 
organizational and policy factors associated with the hospitalization and re-hospitalization of nursing 
homes for almost two decades in the Medicare and Medicaid environment.[2, 46-48] He has extensive 
experience managing large scale, multi-faceted and interdisciplinary research projects having had an NIA 
funded MERIT award and PI of an NIA Program Project grant making him well suited to direct this 
talented group of co-investigators on a complex intervention study. With funding from the VA HSR&D 
Center of Excellence for the Study of Healthcare Provider Behavior, Dr. Saliba developed the SIR 
instruction manual and form based on literature review, expert panel input into elements, a conceptual 
model for nursing home to hospital transfer and pilot testing. Reliability testing in 100 transfers from 8 
nursing homes to 10 hospitals showed that SIR methods demonstrated high inter-rater reliability for 
judging whether transfer could have been avoided.[5] The SIR form was revised based on input and 
questions from participating reviewers and was used in the Georgia QIO test of IMPACT.[3] Additionally, 
Dr. Saliba was the author of the MDS 3.0, leading an interdisciplinary group of clinicians and scientists in 
testing and reviewing the evidence for reliable and valid assessment items in both community and VA 
facilities.[36] In addition to his extensive research outside of VA settings, Dr. Ouslander did his Geriatric 
Medicine Fellowship at the Sepulveda VAMC, and subsequently served as the medical director of that 
facility’s CLC. He directed the Atlanta VAMC Rehabilitation Research and Development Center and 
served as an investigator in the Birmingham-Atlanta VA GRECC. During that time he successfully 
conducted a multi-CLC study of an exercise and incontinence intervention. Orna Intrator, Ph.D. is an 
applied mathematician, health services researcher who is a funded VA investigator studying use of long 
term care services by Veterans both within and outside of the VHA.  She has been a long-time 
collaborator with Dr. Mor examining nursing home hospitalizations and developed of the Residential 
History File in both the Medicare and VHA environment.[12, 19, 49]  Ciaran Phibbs is a health economist 
with a long time interest in estimating the cost of medical interventions, particularly for the chronically 
impaired.[50, 51]  He has integrated multiple sorts of health care utilization records in characterizing the 
costs of mental illness and experienced by women Veterans.[52, 53]  Dr. Dan Berlowitz is an experienced 
physician health services researcher who is one of the most experienced researchers in the long term 
care arena in the VA having done seminal research on the predictors of pressure ulcers, the 
measurement of nursing home quality indicators and the use of multiple pharmacologic agents by the 
frail elderly.[54, 55] 
 
Examination of CLC Hospitalization Rates has been undertaken using the linked SAS and Medicare files 
for the year 2008.  We accumulated all hospitalizations of Veterans directly from the CLC, both those 
occurring in VAMCs and those occurring in Medicare reimbursed hospitals.  Given the relatively high use 
of Medicare hospital services in other VA populations, we were surprised to see that only 7% of all 
hospitalizations of Veterans in CLCs occurred outside of a VAMC.  As expected, this varied from a high of 
nearly 50% in a small CLC to virtually no non-VAMC hospitalizations in many large CLCs.  We calculated 
the number of acute events per Veteran and constructed the rate of hospitalization defined as the 
number of hospitalizations per person year. We did this by applying the Residential History File 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 27 of 43 
Amendment 15 
algorithm initially developed using Medicare files and recently applied to VHA data by Dr. Intrator.[56]  
Across all CLCs, we found that there were .35 hospitalizations per Veteran but 23.5% of Veterans ever in 
the CLC in 2008 were hospitalized, suggesting that some individuals were hospitalized many times.  
Indeed, we find that the observed hospitalization rate is 1.3 per person year, ranging from a high of 1.1 
to a low of .1!  Both of these outliers had relatively few patient days suggesting that they house 
relatively few patients at any given time.  In spite of these outliers, we should be able to effectively 
match CLCs with hospitalization rates in the mid-range.   
 
Limitations: 
 
Like all research studies, ours has various limitations. First, our design requires comparisons over time 
and between our intervention and control VAMCs under which we assume no other competing forces 
such as policy changes that might confound our interpretation. Secondly, we assume that reductions in 
hospitalization are desirable from the Veterans’ perspective and that many of the current 
hospitalizations of Veterans from CLC are, indeed, avoidable.  Third, we assume that CLCs offered the 
opportunity to introduce a new training program will be interested in doing so, which, given the strong 
interest in the private sector, we feel will actually occur.  Fourth, we are explicitly treating the 
introduction of the INTERACT intervention as a quality improvement demonstration which CLCs and 
VAMC leadership can choose to adopt as an administrative practice rather than as research per se.  
Finally, as noted, it is likely that the intervention would be more smoothly implemented were its 
documentation features fully integrated into Veteran’s medical record, but the complications and cost 
associated with such a significant local departure from the medical record system may be too great to 
be borne by a single research project.    
 
Dissemination: 
 
As part of the LTC CREATE, the INTERACT trial is embedded within a broader dissemination structure 
that has direct access to senior officials within the VHA who are committed to transforming the 
character of the VHA long term care system to be more Veteran centered and evidence based. This is 
described in detail in the accompanying CREATE overview.  Our principle vehicle for dissemination is the 
Geriatric and Extended Care (GEC) service and the Office of Patient Centered Care and Cultural 
Transformation (OPCCCT).  Every 6 months the LTC CREATE investigators will meet with GEC and, in 
consultation with them, determine which insight or finding from a LTC CREATE project will be 
highlighted and incorporated into a report that Planetree (currently a major contractor to OPCCCT 
helping to transform the VHA) prepares and which identifies examples of positive instances of 
transformation around the country.  Thus, GEC leadership will be fully engaged in interpreting and then 
developing a practical implementation strategy for the findings of the INTERACT research program.   
 
Another aspect of our dissemination plan is to publish study results via the academic and clinical 
journals and to present study results at clinical meetings such as the American Geriatrics Society, the 
American Medical Directors Association and VAHSR&D annual meetings.  Since Drs. Saliba and Ouslander 
are leaders in the geriatrics and long term care communities and Dr. Berlowitz is a well-known figure in 
the VA HSR&D system, we are confident that presentations, including pre-conference workshops, will be 
well attended since they represent a great draw. As importantly, the whole issue of re-hospitalization 
and hospitalization from nursing home is increasingly recognized as a major issue in clinical and hospital 
practice meaning that the results of a trial revealing reductions in hospitalizations among either short or 
long stay residents would generate great interest among mainstream medical journals.  
 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 28 of 43 
Amendment 15 
Management Plan: 
 
Vincent Mor, Ph.D. will serve as Principal Investigator of the proposed project and will be responsible for 
all scientific and operational aspects of the study. Dr. Deb Saliba will serve as Co-PI and, in addition to 
being the senior clinical investigator, will be responsible for the implementation of the Structured 
Implicit Review (SIR) and all analyses related to estimating the impact of the INTERACT intervention on 
the level of avoidable hospitalizations.  Dr. Dan Berlowitz will provide senior oversight to the evaluation 
design and Dr. Joseph Ouslander, and Nancy Henry, GNP, PhD under an IPA, will be responsible for 
revising and implementing the INTERACT intervention. Dr. Orna Intrator will direct all the data file and 
outcome variable construction and will collaborate with Dr. Ciaran Phibbs in creating the outcome and 
cost data analysis files necessary for the final hospitalization impact analyses and the budget impact 
analyses.  Dr. Roee Gutman will serve as the statistician directing the monthly hospitalization reduction 
analyses and fitting the ARIMA seasonality models necessary to reduce the heterogeneity of variance in 
the CLC monthly hospitalization measures. Dr. Mor will lead monthly project status conference calls and 
speak bi-weekly with Drs. Ouslander, Henry, Berlowitz and Saliba while the INTERACT intervention is 
underway and bi-weekly with Drs. Intrator, Phibbs and Gutman about the data and analyses.  Progress 
with respect to meeting project milestones will be monitored carefully and difficulties in recruiting CLCs 
will be discussed with the GEC leadership who are committed to this project. 
 
Figure 5 below presents the GANNT chart for the 4 year project which begins in the second year of the 
LTC CREATE.  The availability of up to date baseline data comes from the other CREATE projects. 
FIGURE 5 
TASK 
9/13-
3/14 
4/14-
8/14 
9/14-3/15 
4/15-8/15 
9/15-3/16 
4/16-8/16 
9-16-3/17 
4/17-
8/17 
Select CLCs 
 
XXXX 
XXXX 
 
 
 
 
 
RCT IRB 
 
   XXX XXX 
 
 
 
 
 
Adapt 
INTERACT 
 
    
XXX 
 
 
 
 
 
 
Train CLCs 
 
 
 
XXXX 
 
 
 
 
Support CLCs 
 
 
 
 
XXXX 
XXXX 
 
 
Create Hosp. 
Data  
 
 
XXXX 
XXXX 
XXXX 
XXXX 
XXXX 
XXXX 
Implicit Review 
 
 
 
 
 
 
XXXX 
 
Implementation 
Analysis 
 
 
 
 
 
 
 
XXXX 
Avoidable 
Hospitalization 
 
 
 
 
 
 
 
XXXX 
Impact Analysis 
 
 
 
 
 
 
 
 
Budget  Impact 
Analysis 
 
 
 
 
 
 
 
 
 
 
Personnel: 
 
Vincent Mor, Ph.D. (37.5% Yrs. 1-3, 32.5% Yrs. 4-5): Vincent Mor, Ph.D. will serve as Principal 
Investigator of the proposed project, responsible for all scientific and managerial aspects of the project. 
Dr. Mor has managed numerous, large scale multi-site research studies, both observational and 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 29 of 43 
Amendment 15 
intervention studies. He will coordinate the activities of the intervention teams (Drs. Saliba and 
Ouslander) as well as the research and data teams (Drs. Intrator, Guttman and Phibbs) via bi-weekly 
conference calls with monthly combined calls and quarterly calls that include our Geriatrics and 
Extended Care partners. 
 
Orna Intrator, Ph.D. (15% Yrs. 1-5): Dr. Intrator is an applied mathematician and 
health services researcher who has specialized in the creation of complex, integrated, longitudinal and 
hierarchical research analysis files that merge Medicare and VA data together. Given the complexities of 
creating historical, contemporaneous overall, and risk adjusted rates of hospitalization from CLC, Dr. 
Intrator will directly supervise the programmer and coordinate carefully with Dr. Phibbs to insure that 
the files necessary to undertake the intervention impact analyses also meet the needs of the budget 
impact analyses. 
 
Danielle Cote, Project Manager (50% - Yrs 1-5): Ms. Cote will facilitate all communication between the 
project sites; serve as a liaison between the intervention and the research groups, particularly the work 
being done under Dr. Saliba’s direction of the Structured Implicit Review of “potentially avoidable” 
hospitalizations. Ms. Cote will file IRB requests; make modifications to these, run interference in the 
event that participating VAMC CLCs have difficulties with their individual IRB approval. She will be 
shared half time with the other LTC CREATE proposal emanating from the Providence VAMC for 3 years 
and in the last year will be 100% on the INTERACT study working primarily with Drs. Mor and Intrator 
preparing analysis tables, conducting literature reviews and facilitating communication among the 
investigators. 
 
Amy Mochel, Project Coordinator (100% - Yrs 2-5):  Ms. Mochel will serve as a liaison between 
intervention project sites, other project staff and clinicians.  Ms. Mochel will focus on the work being 
done to implement the INTERACT intervention, assisting sites in adhering to the fidelity of the QI 
program.  She will file IRB requests, make modifications, work with individual IRBs across sites, and 
produce and file other study related reports as necessary.   
 
Health Research Scientist, TBN (50%; Yrs. 1-4): This individual will work directly with Dr. Intrator applying 
algorithms from the residential history file to create the Veteran level measures of hospitalization and 
then aggregating these up to the level of CLC specific aggregated measures of hospitalizations per 
Veteran day at risk in the CLC. The programmer will also create all the measures of adverse effects 
based upon the MDS assessments looking at functional decline and other complication rates. The 
programmer will create analysis files at both the individual and aggregated levels for Dr. Gutman, the 
statistician who’ll be modeling the hospitalization rates and other outcome measures. 
 
Analyst, TBN (12.5% Yr. 1, 25%- Yrs. 2-5): This individual will work directly with Dr. Intrator and the study 
Programmer to create the Veteran level measures of hospitalization and then aggregating these up to 
the level of CLC specific aggregated measures of hospitalizations per Veteran day at risk in the CLC. The 
analyst will assist Programmer in creating all the measures of adverse effects based upon the MDS 
assessments looking at functional decline and other complication rates. 
 
Ciaran Phibbs, Ph.D. (12.5% Yrs.1-5): Dr. Phibbs is a health economist at HERC who has pioneered the 
application of Budget Impact Analysis to the world of health care operations and policy changes, moving 
beyond the world of drugs and devices where the techniques were developed. He will be responsible for 
conducting the Budget Impact Analysis and will engage the intervention team, relying upon the 
extensive process data collected and will work with Dr. Intrator to insure that the appropriate utilization 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 30 of 43 
Amendment 15 
data are available to apply various cost adjustments to them. Most importantly, he will direct 
discussions with our GEC partners to best determine the appropriate budget assumptions that will be 
most helpful to them in making the case for broad scale implementation of the INTERACT program if it is 
found to be effective at an acceptable cost. 
 
Brian Mittman, Ph.D. (5% Yrs.2-5): Dr. Mittman is an organizational and implementation researcher who 
is nationally recognized for his leadership in the field of implementation science. He serves as co-editor 
in chief of the journal Implementation Science and as Director of CIPRS (VA Center for Implementation 
Practice and Research Support) through which he provides technical assistance, consultation and 
support to the VA implementation research community to further develop and strengthen 
implementation science theory, methods and research practice. Dr. Mittman and Saliba have 
collaborated on projects, including an ongoing VA project to evaluate provider practices and Veteran 
and caregiver preferences surrounding goal setting for diabetes. Dr. Mittman will serve as co-
investigator throughout all of the proposed project phases and assist Dr. Saliba in the design and 
interpretation of the qualitative interviews to improve the identification of barriers and facilitators to 
implementing INTERACT. 
 
Debra Saliba, MD, MPH (project co-Principal Investigator; GLA site principal investigator  (Yrs.1- 5, 5% In 
kind): Dr. Saliba is a geriatrician and a research physician in the VA Greater Los Angeles HSR&D Center of 
Excellence and the Geriatric Research, Education and Clinical Center (GRECC). She also holds the Anna 
and Harry Borun Endowed Chair in Geriatrics at UCLA and is a senior natural scientist at RAND. Dr. Saliba 
brings content expertise in assessing and improving care quality for persons with long term care needs, 
including developing and testing the structured implicit review (SIR) system that will be used as part of 
the outcome evaluation in the proposed project. Dr. Saliba will serve as co-PI on the overall research 
project and work closely with Dr. Mor to provide content and methodological expertise across all project 
phases and to interpret results and conduct dissemination activities. As site PI, she will be responsible 
for overall conduct of the SIR and the qualitative interviews with providers in participating facilities. 
 
Deborah Riopelle, MS (project manager – 75% FTE years 3 and 4): Ms. Riopelle is a Supervisory Health 
Science Specialist in the GLA VA with over 17 years of experience managing research projects. She has 
managed several large intervention and evaluation projects during that time, participating in survey 
design and administration, IRB application and data safety management, subject recruitment, staff 
trainings and supervision. In the proposed project, Ms. Riopelle will organize training activities and 
reviewer recruitment, identify transfer records, maintain reviewer assignment algorithm, assign records 
for review, track reviews, abstract data from records, maintain communication with reviewers, and 
organize monthly review calls. She will manage data safety protocols and privacy protections. She will 
also work closely with Ms. Yosef to 
organize staff recruitment for qualitative interviews and will alternate note-taking responsibilities with 
Ms. Yosef during provider discussions. 
 
Barbara Simon, MA (survey/IRB SPA leader- 5% FTE year 3) Ms. Simon is the COE’s expert in qualitative 
methods (both focus groups and semi-structured interviews) and survey development; her expertise 
builds on over 30 years’ experience in VA, RAND’s Survey Group and the National Opinion Research 
Center (NORC). She worked with Dr. Saliba in the development and testing of the preference and pain 
interviews for the MDS. For the proposed project, she will work directly with Dr. Saliba to design 
appropriate interviewer guides for qualitative assessment of INTERACT materials and program 
implementation and assist with appropriate set up of data security, review storage, and training 
activities. 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 31 of 43 
Amendment 15 
 
Julia Yosef, RN, MS Psychology (survey expert - 30% FTE year 3) Ms. Yosef is a Survey Interviewer and 
Project Coordinator with the Greater Los Angeles VA HSR&D Center of Excellence who has led both 
qualitative and structured interviews on a variety of projects, including working in both the pilot and 
national study for the MDS 3.0 project in VA CLCs. She will work with the team in developing discussion 
probes, schedule and conduct interviews with providers in the VA CLCs that implemented INTERACT, 
alternating interviews and note taking with Ms. Riopelle, and will participate in analysis of interview 
results. 
 
Alissa Simon, MA (survey expert - 4% FTE year 3) Ms. Simon has been working in survey design and 
development for over 13 years, both for the VA GLA HSR&D Center of Excellence and as a consultant for 
other entities (UCLA, RAND). She is an expert in survey content, field methods, logic, flow, format, 
consistency and web programming. Ms. Simon will be responsible for converting the Structured Implicit 
Review form into a web-based format for reviewer use that will be password access protected and 
housed on the COE’s protected server, compliant with established VA data security protections. 
 
Clinician Reviewers (TBD – 4% FTE) Up to 17 clinicians will be recruited to serve as  reviewers. 
 
These reviewers, as credentialed VA staff members, will donate their time for review activities. We will 
recruit and train up to 17. Although we plan to distribute charts among all recruited for review, we have 
intentionally recruited a number that could accommodate “drop out” because of changes in 
appointment status or availability of review... Our goal is to identify reviewers familiar with VA CLCs at 
non implementation sites and who have clinical experience in identifying change in condition and in 
making transfer decisions.  
 
Ken Shay, DDS, MS who is Director of Geriatric Programs in the VA Office of Geriatrics and Extended 
Care will also assist Dr. Saliba in identifying GRECC individuals to serve as reviewers. 
 
Off-Site Consultants 
Joseph Ouslander, MD. (20% Yrs. 1-3; 5% Yrs. 4-5): Dr. Ouslander was the developer of the INTERACT 
quality improvement program. He will supervise Nancy Henry, GNP, PhD and direct his team to initially 
modify the existing program to be more appropriate for implementation in the VA CLCs by initially 
engaging  the Tampa VA Medical Center CLC staff to offer feedback on best methods for modifying and 
adapting INTERACT for practical use in VA CLCs and then participate in recruiting and implementing the 
intervention and training in 8 sites across the country. He will be a participating author in all analyses 
emerging from this study and will be available to provide instruction in the optimal manner in which the 
intervention is to be generally rolled out across all VA CLCs.  
IPAs 
 
Nancy Henry, GNP, PhD, Senior Project Coordinator will lead the modification and adaptation of the 
existing INTERACT program for use in the VA CLCs in collaboration with Joseph Ouslander, MD. Before 
testing the effect of the INTERACT intervention it is essential to work with the Tampa VA Medical Center 
VA about the best methods for modifying and adapting the INTERACT program for use in VA CLCs. This 
will involve educating CLC leadership and clinical staff about INTERACT, its website, tools and related 
education materials to enhance practical use of INTERACT and how best to refine the program to make 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 32 of 43 
Amendment 15 
it a vital part of improving care for Veteran’s in the CLCs; and participate in implementing the 
intervention and training in 8 sites across the country. 
Maria Carolina Rojido, MD, Research Coordinator will assist in the modification and adaptation of the 
existing INTERACT program for use in the VA CLCs in collaboration with Joseph Ouslander, MD.  She will 
work collaboratively with Nancy Henry, GNP, PhD, Senior Project Coordinator to develop and implement 
the INTERACT QI program in the CLCs.   
Roee Gutman, Ph.D. (10% Yrs.1-4 5% Yr 5): Dr. Gutman is a biostatistician who specializes in 
observational data analysis and propensity score modeling. He will be part of the data analysis team and 
will be responsible for undertaking the outcome analyses and specifically fitting an ARIMA model to the 
hospitalization rates from CLC over the entire study period in such a manner as it optimizes the 
statistical 
power of our test of the INTERACT intervention. He will work closely with Dr. Intrator and Dr. Mor. 
 
Project Team Members 
First   
Last 
Project Role  
Site 
Access to 
secondary 
data  
(aim #1) 
Participating 
in pilot as 
described in 
qualitative 
aim #2a 
Participating in 
INTERACT 
Intervention as 
described in aim 
#2b 
Vincent 
Mor 
Principal Investigator 
Providence 
Yes 
No 
Yes 
Danielle 
Cote 
Project Manager 
Providence 
Yes 
No 
Yes 
Rajesh 
Makineni 
Programmer 
Providence 
Yes 
No 
Yes 
Roee 
Gutman 
Biostatistician 
Providence 
Yes 
No 
Yes 
Rouba 
Youssef 
Analyst 
Providence 
Yes 
No 
Yes 
Maxwell 
Cutty 
Research Assistant 
Providence 
Yes 
No 
Yes 
Nancy  
Henry 
Senior Project 
Coordinator 
Providence 
No 
Yes 
Yes 
Debra 
Saliba 
Co-Principal 
Investigator 
Greater LA 
Yes 
No 
No 
Brian 
Mittman 
Co-Investigator 
Greater LA 
No 
No 
No 
Deborah 
Riopelle 
Qualitative Analyst 
Greater LA 
No 
No 
No 
Alissa 
Simon 
Qualitative Analyst 
Greater LA 
No 
No 
No 
Barbara 
Simon 
Qualitative Analyst 
Greater LA 
No 
No 
No 
Julia 
Yosef 
Sr. Qualitative Analyst 
Greater LA 
No 
No 
No 
Kisa 
Fulbright 
Research Assistant 
Greater LA 
No 
No 
No 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 33 of 43 
Amendment 15 
Ciaran 
Phibbs 
Co-Investigator 
Palo Alto 
Yes 
No 
No 
Sonji 
Blanks 
Local Site Investigator 
Tampa 
No 
Yes 
No 
Carol 
Rueter 
Local Site Investigator 
Tampa 
No 
Yes 
No 
Orna 
Intrator 
Co-Investigator 
Canandaigua 
Yes 
No 
No 
Dan  
Berlowitz 
Co-Investigator 
Bedford 
No 
No 
No 
Maria 
Carolina 
Rojido 
Research Coordinator 
Providence 
No 
No 
Yes 
Amy  
Mochel 
Project Coordinator 
Providence 
No 
No 
Yes 
Kristen 
Morgan 
Research Assistant 
Providence 
No 
No 
Yes 
 
 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 34 of 43 
Amendment 15 
Human Subjects: 
 
Risks to Subjects: This proposed project has two different aspects of human subjects research: 
1. Use of de-identified, sensitive health data which will be accessed for all Veterans receiving long term 
care over the period 2008 through 2017; and 2. CLC staff of the up to 15 intervention CLCs which 
participated in the implementation of the INTERACT staff training and quality improvement program.  
Our approach to addressing the risks and benefits of the proposed research for each of these two 
subject groups is described separately in the sections below.  
 
Data Sources and Data Handling:  There are two major types of data used in this project:  
Veteran specific data and organization level data. These data will be assembled from existing 
administrative data sources available either from the VA or from sources assembled by our study team 
in other studies.  All individually identifiable datasets made available by the VHA have a unique 
identifier, the scrambled SSN, which is a formula-based encryption of the individual's Social Security 
Number. The identifier is consistent for a given patient across datasets and this project’s fiscal years and 
will enable us to access and link data from all of the data sources required for this project.  
 
All organization level data are identified by either a federal identification number (Medicare/ Medicaid 
certified community nursing homes), or by their VA site identifier (station). Organization level data 
include public available data from the Online Survey Certification and Reporting (OSCAR) of Medicare/ 
Medicaid certified nursing homes, the Area Resource File (ARF) obtained from the U.S. Department of 
Health and Human Services (http://arf.hrsa.gov/overview.htm).  
 
VA data will be extracted from Austin Automated Center via VA intranet. Following a signed DUA with 
VIReC/CMS, VIReC will set up a secure share for us to transfer the finder file containing scrambled SSNs 
of Veterans residing in VA CLCs, as required for the project.  
 
Data Assurances: 
 
Adequacy of Protection against Risk: The procedures described in this section address our 
efforts to minimize the risk of breach of confidentiality.  All research personnel involved in the project 
will have successfully completed the Collaborative IRB Training Initiative (CITI) at the VA and all other 
elements of human research protections education programs required by the Providence VAMC.  
 
The primary risk for this study is the potential of loss of confidentiality resulting from a breach of data 
security. This risk is low. We will reduce this risk by storing the data in a protected location in the 
Providence VA Medical Center and by not removing the data from this VA location. All data analytic 
work will occur in the Providence VAMC on the secure cluster present there after VA and Medicare files 
are downloaded from the VA system in Austin. All personal identifiers are removed from the dataset 
prior to generating analytic files thus further minimizing the risk of loss of identifiable data.  
 
The Providence COIN computing infrastructure consists of a Windows server (COIN server), accessed via 
remote desktop sessions and client Windows PCs. Network security is provided by a combination of VA 
firewalls, local network access controls, and continuous auditing and monitoring for security 
breaches. Access from systems external to the VA’s intranet and COIN server is limited to encrypted 
channels, e.g. VPN. Access to the COIN server is independent from the general VA network to provide an 
extra layer of protection.  
 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 35 of 43 
Amendment 15 
All VA data are stored on the COIN server, which is maintained and backed-up on a daily basis by IRM 
personnel.  Personally-identified as well as partially-de-identified data housed on the COIN server are 
restricted to the system analyst at IRM and to the project’s programmer and investigator. No data may 
be transferred to other computers (desktops or laptops) unless they are stripped of all identifiable 
information. COIN employees sign a VA privacy agreement to ensure confidentiality of all sensitive 
information, and its violation may result in criminal charges and a fine of from $5,000 to $20,000.  
 
The COIN’s computer system is highly secure, and accessible only to authorized users.  Within the group 
of authorized users, access to project data is restricted to critical research support staff and INTERACT 
Study Facilitators, as well as CLC specific Champions (Champions only have access to PHI from their own 
CLC site).Data uploaded to the project SharePoint is stored in the VA environment on a secure VA server 
behind the VA firewall.  
 
Data are de-identified before analytic files are created, providing an additional level of protection. 
Furthermore, COIN employees have signed an oath of confidentiality, and its violation is sufficient 
grounds for immediate termination. Finally, analysis results are always presented in aggregate form with 
minimum cell sizes restricted to greater than 10 in accordance with VHA and CMS Data Use Agreements 
and policies.   
 
The project will use real time CDW data available from VINCI to provide CLCs with hospitalization rates 
and mortality data for the selected INTERACT CLCs on a regular basis. The Providence research staff will 
also track this data for the pair matched facilities and provide aggregate, de-identified data for the Data 
Safety Monitoring Board. 
 
Data Safety Monitoring Plan: 
 
To ensure the safety of Veterans in the CLC and confirm the intervention is not reducing hospitalizations 
to the detriment of Veterans’ we have created a safety monitoring plan as described below which has 
been reviewed and signed off on by the Data Safety Monitoring  Board (DSMB).  
1. We will generate mortality data for all CLC residents in both INTERACT (experimental) and 
control facilities, charting changes in mortality rates relative to the historical and contemporary 
mortality rates of paired facilities; 
2. The INTERACT (experimental) CLCs will complete SBAR Progress Notes regarding acute changes 
in conditions for Veterans. These forms are completed by CLC nursing staff when they notice a 
clinical change in a patients’ condition that warrants a discussion with the physician regarding 
the need for changes in standing orders.  There is considerable detail on patients’ clinical 
condition at the time along with information on the disposition of the case; 
3. Staff will transfer the names of cases for which an SBAR is completed to Providence VAMC 
research staff via a joint SharePoint and Providence staff will keep records of SBARs used.   
4. The INTERACT (experimental) CLCs, during every other week calls with the site Champions will 
review the record of all CLC Veteran’s for whom an SBAR form was completed during the 
previous timeframe.  
5. During these qualitative interviews/calls with CLC site Champions intervention staff will review 
cases where a Veteran had an SBAR completed, but was not transferred to the hospital that 
subsequently dies and/or is hospitalized within seven days of the documented change in 
condition record.  These cases will be further discussed based upon the content of the SBAR 
progress note.  A standardized set of open ended questions will be asked focusing on whether 
the Veteran should have been transferred to the hospital sooner, based upon the SBAR progress 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 36 of 43 
Amendment 15 
note, as well as information about the patients’ advanced directives and end of life care wishes. 
(See Appendix # 6, Interview Guide) 
 
These summarized data will be reported to the DSMB on a quarterly basis. The number of 
hospitalizations and deaths occurring in the Control group of CLCs will also be reported quarterly and 
cumulatively (See tables below). Additional aggregate data on outcomes will be provided as requested 
to confirm the intervention is safe for residents and staff.   
 
 
 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 37 of 43 
Amendment 15 
VA Informatics and Computing Infrastructure (VINCI): 
 
As VA and VHA research progresses, large amounts of data are being collected into databases 
maintained by a variety of investigators, studies, and locations. Individual investigators and multiple 
databases may lack sufficient resources to ensure consistency and quality control, or a long-term 
commitment to data storage and access. Therefore, there are less consistent standards for the 
protection of Veterans data, data quality, and data access compared to a centralized repository. A 
centralized research data repository, such as the VA Informatics and Computing Infrastructure (VINCI), 
offers a number of important advantages: Consistent, defined, and transparent security and standards 
for access to data; a common point of entry for all investigators who use the data; tools for analysis and 
reporting; tighter and more consistent control over the standards and quality of the data included; and 
the ability to standardize and update terminology and format as technology and methodology improve. 
VINCI is a partnership between the VA Office of Information Technology (OI&T) and the Veterans’ Health 
Administration Office of Research and Development (VHA ORD). VINCI provides the storage and server 
technologies to securely host suites of databases integrated from select national data. These servers 
reside at the Austin Information Technology Center (AITC), located in Austin, Texas. To ensure the 
protection of Veterans data, VINCI maintains compliance with the guidelines set forth by Veterans 
Health Administration (VHA) Handbook 1200.12, Use of Data and Data Repositories in VHA Research and 
all other applicable VA and VHA policies and regulations. In addition, VINCI has undergone all security 
certification activities in support of obtaining an Authorization to Operate (ATO). Access to VINCI 
resources will be approved in accordance with the requirements of National Data Systems (NDS), VHA 
Handbook 1200.12, Use of Data and Data Repositories in VHA Research, and all other applicable VA and 
VHA policies and regulations. Researchers and Operations staff will access the data along with the tools 
for analysis and reporting in the secure, virtual working environment through a certified VHA network 
computer using the VA INTRANET (NOTE:  VINCI is not accessible through the INTERNET).  If not working 
within a VA or VHA hosted office environment containing VA network access, researchers may access 
VINCI through an approved Virtual Private Network (VPN) and Remote Desktop application. The remote 
computing environment will enable data analysis to be done directly on VINCI-CDW servers located at 
the Austin Information Technology Center, thus keeping all data from being transmitted to local PC hard 
drives. 
 
Data Collection 
VA provides care to veterans at over 1,400 points of care. At the core of virtually all care processes is a 
broadly scoped and extensively used electronic health record system known as the Veterans 
Information System Technology Architecture (VistA). VistA provides a longitudinal view for patients 
receiving care nationwide including diagnosis, procedures, pharmacy, orders, labs, microbiology, 
physiologic measurements, and text documents. VA uses 128 VistA implementations to provide 
longitudinal electronic health record services nationwide for more than 25 million veterans historically. 
The aggregate content of these 128 VistA systems includes just over 1.03 Billion documents (e.g., 
Progress Notes, Discharge Summaries, Reports) accumulating at a rate of 638,000 each workday; 1.65 
Billion orders (+955,000 each workday); 590 Million images (+884,000 each workday); 1.06 Billion vital 
sign measurements (+729,000 each workday) and 850 Million medication administrations (+607,000 
each workday). 
 
VA Informatics and Computing Infrastructure (VINCI) aggregates data sources from individual VistA 
systems, data from the Regional Data Warehouses for all 4 VA regions, the VA Corporate Data 
Warehouse, and the VA Health Data Repository and prepares them for research use. Other data 
published by the VHA Decision Support System (DSS) and Inpatient and Outpatient Medical SAS 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 38 of 43 
Amendment 15 
(MedSAS) can be requested through VINCI. VA National Data Services and other data stewards regulate 
the right to use the data, but VINCI facilitates the process. VINCI servers for data, applications and 
virtual sessions are physically located in the VA Automation Center in Austin, Texas. This secure enclave 
with 20 racks of high-performance servers and 72 terabytes of high-speed data storage has multiple 
layers of security to prevent data loss. When study data requested through VINCI is approved for use, it 
is extracted from source databases and placed in SQL tables accessible only to the research team and VA 
Automation Center OI&T operations personnel. 
 
The Communication tools and Quality Improvement Tools will be used to track Veterans changes in 
condition and hospitalizations.  During calls with CLC Champions and study clinicians the tools will be 
referenced to determine if in retrospect, a transfer could have been prevented.  A root cause analysis 
assessment will be performed to determine better practices for future instances of similar conditions. 
The research team will track data for the pair matched facilities and provide aggregate, de-identified 
data for the Data Safety Monitoring Board (See page 31). 
 
Natural Language Processing (NLP): 
 
The VINCI application library has a suite of Natural Language Processing (NLP) tools for extracting 
information from unstructured text. The ability to create textual reports offers flexibility to clinicians for 
describing symptoms, vital signs, behaviors, attitude, instructions, patient and family history, and much 
more; but free text is not a data format well suited to the analytical tools familiar to researchers.  
VINCI has an NLP “Pipeline”, a collection of configurable NLP modules available as a Service Oriented 
Architecture (SOA) within the VINCI processing environment. This SOA pipeline, named V3NLP, is more 
easily configured than other NLP pipelines and it is easier to adapt existing GATE or UIMA NLP modules 
to the SOA environment. VINCI and their customers have adapted V3NLP to data patterns specific to the 
VA and V3NLP has been used for several clinical use cases. 
 
Potential Benefits of the Proposed Research to the Subjects and Others: 
 
Although there are no direct benefits to human subjects, this study would provide information that 
would assist policy makers and facility administrators. The minimal risks to subjects are reasonable for 
the importance of the information that will help understand the variation and outcomes of health care 
transitions. Reducing hospitalizations from CLCs has immediate positive benefits on the experience of 
Veterans since they would be spared the trauma of unnecessary transfer to hospital. In addition, 
INTERACT training advances several principles central to culture transformation by promoting Veteran-
directed advance care planning and training staff to understand Veteran patterns of daily activities to 
better identify early warning signs and symptoms of decline. 
 
Importance of the knowledge to be gained: 
 
By assembling and analyzing the linked data it will be possible to perform analyses relating to how the 
health care system performs to meet the needs of its frail Veterans.  
 
Women and Minority Representation: 
 
The cohort selected for this study will be population based, thus ensuring adequate representation of 
minorities and women, as represented among the overall veteran population served in nursing homes. 
According to a recent report, among newly admitted veterans to VHA NHCU between June 1999 and 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 39 of 43 
Amendment 15 
October 2004, 97% of the population were men, 82% were white, 13% were African American, and 5% 
were of other racial or ethnic background25.  
 
Since 2003, the VA has collected self-reported race in compliance with a new federal guideline from the 
Office of Management and Budget (OMB). In FY 2003 the variables RACE1 - RACE6 were added to the 
SAS Inpatient Datasets and in FY 2004 RACE1 - RACE7 were added to the SAS Outpatient Datasets to 
allow for multiple race reporting. These new variables are coded to incorporate both race and method 
of collection. Hispanic ethnicity is recorded as a separate variable under the new coding.  The variable 
ETHNIC was added to the SAS Inpatient Datasets in FY 2003 and to the Outpatient Datasets in FY 2004. 
The RACE variable is only partially populated in the FY 2003 Medical SAS Inpatient Main Dataset and is 
entirely empty in the FY 2004 Inpatient Main Dataset. VIReC conducted a study examining issues related 
to the transition to the new race/ethnicity data collection standards in the VA 60. Using Medical SAS 
Inpatient and Outpatient Datasets for FY 1997 through FY 2002 and FY 2004, the researchers found that 
the agreement between observer-recorded race before the transition and self-reported race/ethnicity in 
FY 2004 was high overall and among White and African American (AA) VHA users.  This result suggests 
that observer-recorded and self-reported data for these groups can be used across years without 
creating serious bias. Observer-recorded race was less reliable for non-AA minorities.  The investigators 
demonstrated that combining those groups to create a higher-level more inclusive group improved 
accuracy of observer-reported race/ethnicity and that the AA and non-AA distinction provided the 
greatest agreement between observer- and self-reported race/ethnicity. Consistent with VIREC’s 
recommendation (http://www.virec.research.va.gov/DataSourcesCategory/DataQuality/HSRD 
_Data_Quality_Alert.pdf), we plan to use race/ethnicity as a 3 level categorical variable identifying 
Whites, African Americans, and other minorities.  For veterans enrolled in Medicare, the Medicare 
denominator file is considered of good quality with few missing values, and will be preferred to the VHA 
recorded race/ ethnicity. Other veterans’ not enrolled in Medicare who have missing race/ ethnicity 
information will be examined for evidence of race/ ethnicity in prior year’s data. 
 
 
REFERENCES: 
 
1. 
French DD, Campbell RR, Rubenstein LZ: Long-stay nursing home residents' hospitalizations in 
the VHA: the potential impact of aligning financial incentives on hospitalizations. J Am Med Dir 
Assoc 2008, 9(7):499-503. 
2. 
Mor V, Intrator O, Feng Z, Grabowski DC: The revolving door of rehospitalization from skilled 
nursing facilities. Health Aff (Millwood) 2010, 29(1):57-64. 
3. 
Ouslander JG, Lamb G, Perloe M, Givens JH, Kluge L, Rutland T, Atherly A, Saliba D: Potentially 
avoidable hospitalizations of nursing home residents: frequency, causes, and costs: [see 
editorial comments by Drs. Jean F. Wyman and William R. Hazzard, pp 760-761]. J Am Geriatr 
Soc 2010, 58(4):627-635. 
4. 
Grabowski DC, O'Malley AJ, Barhydt NR: The costs and potential savings associated with 
nursing home hospitalizations. Health Aff (Millwood) 2007, 26(6):1753-1761. 
5. 
Saliba D, Kington R, Buchanan J, Bell R, Wang M, Lee M, Herbst M, Lee D, Sur D, Rubenstein L: 
Appropriateness of the decision to transfer nursing facility residents to the hospital. J Am 
Geriatr Soc 2000, 48(2):154-163. 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 40 of 43 
Amendment 15 
6. 
Grabowski DC, Feng Z, Intrator O, Mor V: Medicaid bed-hold policy and Medicare skilled 
nursing facility rehospitalizations. Health Serv Res 2010, 45(6 Pt 2):1963-1980. 
7. 
Intrator O, Grabowski DC, Zinn J, Schleinitz M, Feng Z, Miller S, Mor V: Hospitalization of nursing 
home residents: the effects of states' Medicaid payment and bed-hold policies. Health Serv Res 
2007, 42(4):1651-1671. 
8. 
Coleman EA, Parry C, Chalmers S, Min SJ: The care transitions intervention: results of a 
randomized controlled trial. Arch Intern Med 2006, 166(17):1822-1828. 
9. 
Naylor MD, Brooten D, Campbell R, Jacobsen BS, Mezey MD, Pauly MV, Schwartz JS: 
Comprehensive discharge planning and home follow-up of hospitalized elders: a randomized 
clinical trial. JAMA 1999, 281(7):613-620. 
10. 
Jack BW, Chetty VK, Anthony D, Greenwald JL, Sanchez GM, Johnson AE, Forsythe SR, O'Donnell 
JK, Paasche-Orlow MK, Manasseh C et al: A reengineered hospital discharge program to 
decrease rehospitalization: a randomized trial. Ann Intern Med 2009, 150(3):178-187. 
11. 
Ouslander JG, Lamb G, Tappen R, Herndon L, Diaz S, Roos BA, Grabowski DC, Bonner A: 
Interventions to reduce hospitalizations from nursing homes: evaluation of the INTERACT II 
collaborative quality improvement project. J Am Geriatr Soc 2011, 59(4):745-753. 
12. 
Intrator O, Castle NG, Mor V: Facility characteristics associated with hospitalization of nursing 
home residents: results of a national study. Med Care 1999, 37(3):228-237. 
13. 
Thompson RS, Hall NK, Szpiech M: Hospitalization and mortality rates for nursing home-
acquired pneumonia. The Journal of family practice 1999, 48(4):291-293. 
14. 
Fried TR, Gillick MR, Lipsitz LA: Whether to transfer? Factors associated with hospitalization 
and outcome of elderly long-term care patients with pneumonia. Journal of general internal 
medicine 1995, 10(5):246-250. 
15. 
Fried TR, Gillick MR, Lipsitz LA: Short-term functional outcomes of long-term care residents 
with pneumonia treated with and without hospital transfer. Journal of the American Geriatrics 
Society 1997, 45(3):302-306. 
16. 
Murtaugh CM, Freiman MP: Nursing home residents at risk of hospitalization and the 
characteristics of their hospital stays. The Gerontologist 1995, 35(1):35-43. 
17. 
Freiman MP, Murtaugh CM: The determinants of the hospitalization of nursing home 
residents. Journal of health economics 1993, 12(3):349-359. 
18. 
Carter MW: Variations in hospitalization rates among nursing home residents: the role of 
discretionary hospitalizations. Health services research 2003, 38(4):1177-1206. 
19. 
Intrator O, Zinn J, Mor V: Nursing home characteristics and potentially preventable 
hospitalizations of long-stay residents. J Am Geriatr Soc 2004, 52(10):1730-1736. 
20. 
Konetzka RT, Spector W, Shaffer T: Effects of nursing home ownership type and resident payer 
source on hospitalization for suspected pneumonia. Medical care 2004, 42(10):1001-1008. 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 41 of 43 
Amendment 15 
21. 
Walsh EG, Freiman, M, Haber, S, Bragg, A, Ouslander, J, Weiner, JM.: Cost Drivers for Dually 
Eligible Beneficiaries: Potentially Avoidable Hospitalizations from Nursing Facility, Skilled 
Nursing Facility and Home and Community Based Services Waiver Programs. In. Waltham, MA: 
Research Triangle Institute 2010. 
22. 
Konetzka RT, Spector W, Limcangco MR: Reducing hospitalizations from long-term care 
settings. Medical care research and review : MCRR 2008, 65(1):40-66. 
23. 
Baldwin LM, Adamache W, Klabunde CN, Kenward K, Dahlman C, J LW: Linking physician 
characteristics and medicare claims data: issues in data availability, quality, and 
measurement. Med Care 2002, 40(8 Suppl):IV-82-95. 
24. 
Hawes C, Morris JN, Phillips CD, Mor V, Fries BE, Nonemaker S: Reliability estimates for the 
Minimum Data Set for nursing home resident assessment and care screening (MDS). 
Gerontologist 1995, 35(2):172-178. 
25. 
Phillips CD, Morris JN, Hawes C, Fries BE, Mor V, Nennstiel M, Iannacchione V: Association of 
the Resident Assessment Instrument (RAI) with changes in function, cognition, and 
psychosocial status. J Am Geriatr Soc 1997, 45(8):986-993. 
26. 
Phillips CD, Morris JN: The potential for using administrative and clinical data to analyze 
outcomes for the cognitively impaired: an assessment of the minimum data set for nursing 
homes. Alzheimer Dis Assoc Disord 1997, 11 Suppl 6:162-167. 
27. 
Morris JN, Nonemaker S, Murphy K, Hawes C, Fries BE, Mor V, Phillips C: A commitment to 
change: revision of HCFA's RAI. J Am Geriatr Soc 1997, 45(8):1011-1016. 
28. 
Banaszak Holl J, Zinn JS, Mor V: The impact of market and organizational characteristics on 
nursing care facility service innovation: a resource dependency perspective. Health Serv Res 
1996, 31(1):97-117. 
29. 
Hartmaier SL, Sloane PD, Guess HA, Koch GG: The MDS Cognition Scale: a valid instrument for 
identifying and staging nursing home residents with dementia using the minimum data set. J 
Am Geriatr Soc 1994, 42(11):1173-1179. 
30. 
Hartmaier SL, Sloane PD, Guess HA, Koch GG, Mitchell CM, Phillips CD: Validation of the 
Minimum Data Set Cognitive Performance Scale: agreement with the Mini-Mental State 
Examination. J Gerontol A Biol Sci Med Sci 1995, 50(2):M128-133. 
31. 
Blaum CS, Fries BE, Fiatarone MA: Factors associated with low body mass index and weight loss 
in nursing home residents. J Gerontol A Biol Sci Med Sci 1995, 50(3):M162-168. 
32. 
Voelkl JE, Fries BE, Galecki AT: Predictors of nursing home residents' participation in activity 
programs. Gerontologist 1995, 35(1):44-51. 
33. 
Frederiksen K, Tariot P, De Jonghe E: Minimum Data Set Plus (MDS+) scores compared with 
scores from five rating scales. J Am Geriatr Soc 1996, 44(3):305-309. 
34. 
Resnick NM, Brandeis GH, Baumann MM, Morris JN: Evaluating a national assessment strategy 
for urinary incontinence in nursing home residents: reliability of the minimum data set and 
validity of the resident assessment protocol. Neurourol Urodyn 1996, 15(6):583-598. 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 42 of 43 
Amendment 15 
35. 
Housen P, Shannon GR, Simon B, Edelen MO, Cadogan MP, Jones M, Buchanan J, Saliba D: Why 
not just ask the resident? Journal of gerontological nursing 2009, 35(11):40-49. 
36. 
Saliba DaB, J.: Development and Validation of a Revised Nursing Home Assessment Tool: MDS 
3.0. In. Santa Monic, CA: RAND Corporation; 2008. 
37. 
System TCFCoaHPH: Why Not the Best? Results from the National Scorecard on U.S. Health 
System Performance, 2011. In. NY NY: Commonwealth Fund; 2011. 
38. 
Mor V, Gruneir A, Feng Z, Grabowski DC, Intrator O, Zinn J: The effect of state policies on 
nursing home resident outcomes. J Am Geriatr Soc 2011, 59(1):3-9. 
39. 
Mor V, Gruneir A, Feng Z, Grabowski DC, Intrator O, Zinn J: The Effect of State Policies on 
Nursing Home Resident Outcomes. J Am Geriatr Soc 2011. 
40. 
Ouslander JG, Diaz S, Hain D, Tappen R: Frequency and diagnoses associated with 7- and 30-day 
readmission of skilled nursing facility patients to a nonteaching community hospital. Journal of 
the American Medical Directors Association 2011, 12(3):195-203. 
41. 
Lohfeld L, Loeb, M, Brazil, K. : Evidence-based pathways to manage urinary tract infections in 
long-term care facilities: a qualitative case study describing administrators and nursing staff 
views. J Am Med Dir Assoc 2007, 8:477-484. 
42. 
Loeb M CS, Goeree R, et al. : Effect of a Clinical Pathway to Reduce Hospitalizations in Nursing 
Home Residents With Pneumonia --A Randomized Controlled Trial. JAMA 2006, 295:2503-
2510. 
43. 
Molloy DW GG, Russo R, et al.: Systematic Implementation of an Advance Directive Program in 
Nursing Homes: A Randomized Controlled Trial. JAMA 2000, 283:1437-1444. 
44. 
Ouslander JG, Perloe M, Givens JH, Kluge L, Rutland T, Lamb G: Reducing potentially avoidable 
hospitalizations of nursing home residents: results of a pilot quality improvement project. J 
Am Med Dir Assoc 2009, 10(9):644-652. 
45. 
Morris JN, Fries BE, Morris SA: Scaling ADLs within the MDS. J Gerontol A Biol Sci Med Sci 1999, 
54(11):M546-553. 
46. 
Castle NG, Mor V: Hospitalization of nursing home residents: a review of the literature, 1980-
1995. Med Care Res Rev 1996, 53(2):123-148. 
47. 
Mor V, Intrator O, Fries BE, Phillips C, Teno J, Hiris J, Hawes C, Morris J: Changes in 
hospitalization associated with introducing the Resident Assessment Instrument. J Am Geriatr 
Soc 1997, 45(8):1002-1010. 
48. 
Gruneir A, Miller SC, Intrator O, Mor V: Hospitalization of nursing home residents with 
cognitive impairments: the influence of organizational features and state policies. 
Gerontologist 2007, 47(4):447-456. 
49. 
Intrator O, Hiris J, Berg K, Miller SC, Mor V: The residential history file: studying nursing home 
residents' long-term care histories(*). Health Serv Res 2011, 46(1 Pt 1):120-137. 

Version 14:  06/28/17                                VA Central IRB Protocol                                       Page 43 of 43 
Amendment 15 
50. 
Phibbs CS, Bhandari A, Yu W, Barnett PG: Estimating the costs of VA ambulatory care. Medical 
care research and review : MCRR 2003, 60(3 Suppl):54S-73S. 
51. 
Yoon J, Scott JY, Phibbs CS, Wagner TH: Recent Trends in Veterans Affairs Chronic Condition 
Spending. Popul Health Manag 2011. 
52. 
Friedman SA, Phibbs CS, Schmitt SK, Hayes PM, Herrera L, Frayne SM: New women veterans in 
the VHA: a longitudinal profile. Womens Health Issues 2011, 21(4 Suppl):S103-111. 
53. 
Frayne SM, Yu W, Yano EM, Ananth L, Iqbal S, Thrailkill A, Phibbs CS: Gender and use of care: 
planning for tomorrow's Veterans Health Administration. J Womens Health (Larchmt) 2007, 
16(8):1188-1199. 
54. 
Berlowitz DR, Wilking SV: Risk factors for pressure sores. A comparison of cross-sectional and 
cohort-derived data. Journal of the American Geriatrics Society 1989, 37(11):1043-1050. 
55. 
Berlowitz DR, Halpern J: Evaluating and improving pressure ulcer care: the VA experience with 
administrative data. Jt Comm J Qual Improv 1997, 23(8):424-433. 
56. 
Intrator O, Hiris J, Berg K, Miller SC, Mor V: The Residential History File: Studying Nursing Home 
Residents' Long-Term Care Histories(*). Health Serv Res 2010.