Changing Talk Online (CHATO): A Pragmatic Trial to Reduce Behavioral Symptoms in Dementia Care
Phase: NA
Conditions: Staff Attitude, Dementia
Sponsor: University of Kansas Medical Center

NIA Protocol Template, Version 2.0  
 
 
 
 
 
 
 
 
 
 
Changing Talk Online (CHATO)  
Pilot Study Protocol  
 
 

 
 
 
 
 
Changing Talk Online (CHATO): A Pragmatic Trial to 
Reduce Behavioral Symptoms in Dementia Care (Pilot) 
 
 
Principal Investigator:  
 
Kristine Williams, RN, PhD, FNP-BC, FGSA, FAAN 
E. Jean Hill Professor 
University of Kansas School of Nursing 
Supported by: 
The National Institute on Aging 
 Grant number: 1R61AG061881 
 
Protocol & Statistical Plan 
Last IRB Approval Date: November 1, 2020 
 
 
 
 
 

     
 
ii 
TABLE OF CONTENTS 
Page 
Changing Talk Online (CHATO): A Pragmatic Trial to Reduce Behavioral Symptoms in 
Dementia Care (Pilot) ................................................................................................................... i 
TABLE OF CONTENTS ............................................................................................................. ii 
PRÉCIS .......................................................................................................................................... v 
Study Title ................................................................................................................................... v 
Objectives .................................................................................................................................... v 
Design and Outcomes .................................................................................................................. v 
Interventions and Duration .......................................................................................................... v 
Sample Size and Population ........................................................................................................ v 
STUDY TEAM ROSTER ............................................................................................................. 1 
Principal Investigator: Kristine Williams, RN, PhD, FNP-BC, FGSA, FAAN ....................... 1 
Co-Investigators:  Carissa Coleman, PhD ................................................................................ 1 
Co-Investigators:  Yelena Perkhounkova, PhD ........................................................................ 1 
Co-Investigators:  Nor Hashidah Abd-Hamid, PhD ................................................................. 1 
Consultant: 
   Robyn Stone, PhD ............................................................................................ 2 
Consultant: 
   Natasha Bryant, MA ......................................................................................... 2 
PARTICIPATING STUDY SITES .............................................................................................. 2 
1 
Study objectives ..................................................................................................................... 3 
1.1 
Primary Objective ............................................................................................................ 3 
1.2 
Secondary Objectives ...................................................................................................... 3 
2 
BACKGROUND AND RATIONALE ................................................................................. 3 
2.1 
Background on Condition, Disease, or Other Primary Study Focus ............................... 3 
2.2 
Study Rationale ............................................................................................................... 3 
3 
STUDY DESIGN ................................................................................................................... 6 
4 
SELECTION AND ENROLLMENT OF PARTICIPANTS ............................................. 7 
4.1 
Inclusion Criteria ............................................................................................................. 7 
4.2 
Exclusion Criteria ............................................................................................................ 8 

     
 
iii 
4.3 
Study Enrollment Procedures .......................................................................................... 8 
5 
STUDY INTERVENTIONS ................................................................................................. 8 
5.1 
Interventions, Administration, and Duration ................................................................... 8 
5.2 
Handling of Study Interventions ..................................................................................... 9 
5.3 
Concomitant Interventions ............................................................................................ 10 
5.4 
Adherence Assessment .................................................................................................. 10 
6 
STUDY PROCEDURES ..................................................................................................... 11 
6.1 
Schedule of Pilot Evaluations in NHs ........................................................................... 11 
6.2 
Description of Evaluations ............................................................................................ 12 
6.2.1 
Screening and Randomization ............................................................................... 12 
6.2.2 
Consenting and Enrollment ................................................................................... 12 
6.2.3 
Treatment: CHATO Training ................................................................................ 12 
6.2.4 
Completion and Final Evaluation .......................................................................... 13 
7 
SAFETY ASSESSMENTS .................................................................................................. 13 
7.1 
Specification of Safety Parameters ................................................................................ 13 
7.2 
Methods and Timing for Assessing, Recording, and Analyzing Safety Parameters ..... 14 
7.3 
Adverse Events and Serious Adverse Events ................................................................ 14 
7.4 
Reporting Procedures .................................................................................................... 15 
7.5 
Follow-up for Adverse Events ....................................................................................... 15 
7.6 
Safety Monitoring .......................................................................................................... 15 
8 
INTERVENTION DISCONTINUATION ........................................................................ 15 
9 
STATISTICAL CONSIDERATIONS ............................................................................... 16 
9.1 
General Design and Outcomes ...................................................................................... 16 
9.2 
Sample Size and Randomization ................................................................................... 16 
9.3 
Interim analyses and Stopping Rules ............................................................................ 16 
9.4 
Data Analyses ................................................................................................................ 16 
10 
DATA COLLECTION AND QUALITY ASSURANCE ............................................. 17 
10.1 
Data Collection Forms ................................................................................................... 17 
10.2 
Data Management .......................................................................................................... 18 
10.3 
Quality Assurance ......................................................................................................... 19 
10.3.1 Training ...................................................................................................................... 19 

     
 
iv 
10.3.2 
Quality Control Committee ................................................................................... 19 
10.3.3 
Metrics ................................................................................................................... 19 
10.3.4 
Protocol Deviations ............................................................................................... 19 
10.3.5 
Monitoring ............................................................................................................. 20 
11 
PARTICIPANT RIGHTS AND CONFIDENTIALITY .............................................. 20 
11.1 
Institutional Review Board (IRB) Review .................................................................... 20 
11.2 
Informed Consent Forms ............................................................................................... 20 
11.3 
Participant Confidentiality ............................................................................................. 20 
11.4 
Study Discontinuation ................................................................................................... 21 
12 
ETHICAL CONSIDERATIONS ................................................................................... 21 
13 
PUBLICATION OF RESEARCH FINDINGS ............................................................. 21 
14 
REFERENCES ................................................................................................................ 21 
15 
APPENDIX ...................................................................................................................... 26 
15.1 
Changing Talk Online (CHATO) Implementation Toolkit ........................................... 26 
15.2 
Changing Talk Online (CHATO) Training Overview .................................................. 26 
15.3 
CHATO Implemenation Strategies Survey ................................................................... 26 
15.3.1 
Artifacts of Culture Change Survey ...................................................................... 26 
15.4 
CHATO Leadership Inverviews .................................................................................... 26 
 
 
 

     
 
v 
PRÉCIS 
Study Title  
     Changing Talk Online (CHATO): A Pragmatic Trial to Reduce Behavioral  
             Symptoms in Dementia Care (Pilot) 
Objectives 
AIM 1. Establish acceptability and preliminary efficacy of online CHATO modules through pilot 
testing with NH staff.  
AIM 2. Develop and pilot test the data collection tool with consultant and advisory panel 
input. Interviews of NH administrators and staff who participate in the pilot testing of CHATO 
and a process evaluation will be used to identify and develop supports for implementation and 
sustainability in preparation for future CHATO testing. 
Design and Outcomes   
The R61 will prepare for the R01 pragmatic trial by establishing feasibility of online modules and 
preliminary efficacy of CHATO with NH staff. The research design is a randomized clinical trial. 
One NH will provide initial feasibility testing. Any modifications to the modules will be made. 
Then six nursing homes (estimated N=150 staff) will be randomly assigned to intervention or 
wait-list control groups. The primary outcome will be knowledge gain for staff completing 
CHATO training. Additional outcomes include facility level data related to resident quality 
measures of behavioral and psychological symptoms of dementia (BPSD) and inappropriate use 
of psychotropic medications to control BPSD. Implementation strategies will be assessed by 
survey and leadership interviews completed by an external evaluator. 
 
Group 
Time 1 
1 Month 
Time 2 
1 Month 
Time 3 
Immediate 
3 NHs 
Baseline 
assessment1 
CHATO training2 
Post-training 
assessment3 
 
 
Wait list 
3 NHs 
Baseline 
assessment1 
 
Repeat baseline 
assessment1 
CHATO training2 
Post-training 
assessment3 
1Pre-test knowledge, Communication rating; 2 Three modules over a one-month period;  
3 Post-test knowledge, Communication rating, Program Evaluation, Diffusion of Innovation 
Interventions and Duration  
Changing Talk Online (CHATO) training is a course designed to increase awareness of the 
importance of effective communication with older adults and to use evidence-based person-
centered communication during interactions with older adults in nursing homes and other health 
care settings. The total program is approximately 3 hours, split into 3 modules. Each module is 
approximately an hour, depending on the individual user. Each NH will work with the research 
team for three months to plan, implement, and collect data.   
Sample Size and Population  
This course is designed for staff in nursing homes, independent and assisted living, and health 
care settings in the community that include registered nurses, nursing assistants, nursing home 
dieticians, direct care professionals, other administrations and support employees. All the 
employees at all seven nursing homes will be asked to participate. Assignment of NHs to 
intervention and wait-list control groups will be at random. We estimate a total sample of 150 
training participants.  

 
1 
STUDY TEAM ROSTER  
 
Principal Investigator: 
Kristine Williams, RN, PhD, FNP-BC, FGSA, FAAN 
E. Jean Hill Professor 
University of Kansas School of Nursing 
3043 School of Nursing 
Kansas City, KS 66160 
kwilliams1@kumc.edu 
913-588-1673 
Main responsibilities: Research Lead  
 
Co-Investigators: 
 
Carissa Coleman, PhD 
 
University of Kansas School of Nursing 
3043 School of Nursing 
Kansas City, KS 66160 
ccoleman3@kumc.edu  
970-270-6839 
Main responsibilities: Project Director 
 
Co-Investigators: 
 
Yelena Perkhounkova, PhD 
University of Iowa School of Nursing 
414 College of Nursing Building 
Iowa City, Iowa 52242 
elena-perkhounkova@uiowa.edu 
319-335-7078 
Main responsibilities: Statistician  
 
Co-Investigators: 
 
Nor Hashidah Abd-Hamid, PhD  
University of Iowa College of Public Health 
5183 Westlawn  
Iowa City, Iowa 52242 
norhashidah-abdhamid@uiowa.edu  
319-335-8812 
Main responsibilities: Instructional Designer 
 
Study Personnel:              Maria Hein, MSW 
                                           Office of Nursing Research and Scholarship 
416 College of Nursing Building 
Iowa City, IA 52242 
maria-hein@uiowa.edu 
319-335-7076 
Main responsibilities: Data Manager 
 
Study Personnel:              Tim Beachy, BA 
University of Iowa College of Public Health 
4203 Westlawn  
Iowa City, Iowa 52242 
tim-beachy@uiowa.edu 

     
 
2 
319-353-5955 
Main responsibilities: Learning System Coordinator 
 
Study Personnel:              Clarissa Shaw, RN, MSN 
University of Iowa School of Nursing 
414 College of Nursing Building 
Iowa City, Iowa 52242 
clarissa-shaw@uiowa.edu 
319-335-7078 
Main responsibilities: Graduate Research Assistant  
 
Consultant:  
 
 
Robyn Stone, PhD 
Senior Vice President of Research, LeadingAge and Co-Director, 
LeadingAge LTSS Center @UMass Boston 
2519 Connecticut Avenue NW Washington, DC 20008 
202-508-1206 
rstone@LeadingAge.org 
Main responsibilities: National Consultant and NH Industry Leader 
 
Consultant:  
 
 
Natasha Bryant, MA 
LeadingAge Managing Director/Senior Research Associate  
2519 Connecticut Avenue NW Washington, DC 20008 
202-508-1214 
nbryant@leadingage.org  
                                    Main responsibilities: External Evaluator 
 
 
 
PARTICIPATING STUDY SITES  
 
University of Kansas School of Nursing 
3901 Rainbow Blvd.,  
Kansas City, Kansas 66160 
913-588-1619 
 
University of Iowa School of Nursing 
101 College of Nursing Building 
50 Newton Road,  
Iowa City, Iowa 52242  
319-335-7018 
 
 

     
 
3 
1 
STUDY OBJECTIVES 
1.1 Primary Objective 
AIM 1. Establish acceptability and preliminary efficacy of online CHATO modules through pilot 
testing with NH staff. 
1.2 Secondary Objectives 
AIM 2. Develop and pilot test the data collection tool with consultant and advisory panel 
input. Interviews of NH administrators and staff who participate in the pilot testing of CHATO 
and a process evaluation will be used to identify and develop supports for implementation and 
sustainability in preparation for future CHATO testing. 
2 
BACKGROUND AND RATIONALE  
2.1 Background on Condition, Disease, or Other Primary Study Focus 
The population afflicted with Alzheimer’s disease and other dementias will expand from 5 to 16 
million by 2050, increasing dementia care costs from $259 billion to $1.1 trillion.1 Of today’s 1.4 
million nursing home (NH) residents, 61% have moderate to severe dementia, and up to 90% of 
them exhibit behavioral and psychological symptoms of dementia (BPSD) such as physical and 
verbal aggression, agitation, and wandering.2-4 These behaviors are associated with depression as 
well as reduced quality of life and lower survival rates in persons with dementia (PWD).3  BPSD 
also stress family caregivers and precipitate NH placement.5,6  In the NH, BPSD increase time to 
provide care, and NH staff, primarily Certified Nursing Assistants (CNAs) who provide most 
direct care, report that BPSD represent the most stressful aspect of their job.7-9 Considering 
additional costs for CNA burnout and turnover, It is estimated that  BPSD increase costs of 
dementia care by 25 to 35%.10  With national NH rates for a semi-private room of $82,200 per 
year, reducing BPSD may save up to $20,000 per resident annually.11  
 
As cognitive and communication abilities decline due to dementia, NH residents become unable 
to convey care preferences and needs and staff communication becomes infantilizing, impersonal, 
and task-oriented resulting in BPSD. As verified in our research using behavioral coding and 
sequential analyses of video-recorded care, staff elderspeak (communication that sounds like 
baby talk) is linked to resident resistiveness to care (RTC), a subset of BPSD that disrupt nursing 
care. In our study, NH residents were more than twice as likely to be resistive to care when staff 
used elderspeak compared to normal communication.12 Thus, improving communication has great 
potential as a nonpharmacological intervention to reduce BPSD in NH care.13 
2.2 Study Rationale 
The Communication Predicament of Aging theory establishes the link between elderspeak and 
BPSD. Elderspeak derives from stereotypical views of older adults as less competent than 
younger persons.14 When younger people talk with older adults, they modify their speech by 
simplifying, clarifying, and altering the underlying affective quality of messages.15,16 The 
resulting implicit message of incompetence begins a negative feedback loop for older persons, 
who react with depression, withdrawal, and dependency.14 Elderspeak is especially threatening to 
self-concept and personhood, critical to the wellbeing of PWD who are likely to respond with 
BPSD.17,18 The Need-driven Dementia-compromised Behavior model recognizes BPSD as the 
expression of unmet needs of PWD.19,20 Communication, that staff can modify to prevent BPSD, 
is an essential constant part of the environment connecting PWD to others and affirming their 
self-concept.   
 

     
 
4 
Psychotropic medications are often used inappropriately to control BPSD in NH residents with 
dementia.21,22 Alarmingly high rates persist, despite negative outcomes,23 an FDA black box 
warning of increased mortality for older adults with dementia,21 and a recent Centers for 
Medicare and Medicaid Services (CMS) mandate to reduce off-label prescribing of 
antipsychotics.24 CMS and the National Partnership to Improve Dementia Care target reductions 
in psychotropic drug use as top priority.25 Despite reductions in antipsychotic rates (one type of 
psychotropic medication) ranging from 3 to 12% from 2011-2016, up to 20% of NH residents 
received inappropriate antipsychotic medication in 2017.26 Research demonstrates that educating 
direct care providers in behavioral interventions to control BPSD also reduces psychotropic drug 
use (antipsychotics, hypnotics, antidepressants, antianxiety, sedative, anticonvulsant and mood 
stabilizers),27-29 although evidence is limited by lack of rigorous clinical trials that also evaluate 
approaches that influence intervention effects.27  
 
Our recently completed R01 clinical trial that tested Changing Talk (CHAT) communication 
training (NR011455) provides preliminary data for the proposed research. CHAT decreased staff 
elderspeak that reduced resident RTC.30 On average, elderspeak declined from 34.6% (SD = 18.7) 
at baseline by 13.6 percentage points (SD = 20.00) post-intervention and 12.2 percentage points 
(SD=22.0) after 3 months (see Figure 1). RTC declined from 35.7% (SD = 23.2) by 15.3 
percentage points (SD = 32.4) post-intervention and 13.4 percentage points (SD=33.7) at follow-
up. Linear mixed modeling determined that change in elderspeak was predicted by CHAT (b = -
12.20, p = .028) and baseline elderspeak (b = -0.65, p < .001), while RTC change was predicted 
by elderspeak change (b = 0.43, p < .001), baseline RTC (b = -0.58, p < .001), resident 
communication disability, and comorbid illnesses.  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Figure 1. CHAT effects on elderspeak & RTC. 
 
CHAT increases staff awareness of elderspeak’s negative effects and guides practice of more 
effective communication. CHAT includes three, hour-long sessions, with replicated effects on 
communication in three studies among staff (N=89) and residents (N=194) in over 20 NHs.30-32 
Effect sizes ranged from η2 = .35 to .62 for reducing elderspeak diminutives (inappropriately 
intimate terms of endearment) and collective “we” pronoun substitutions; effects were maintained 

     
 
5 
over 2 months. Person-centered (vs task-focused) communication also increased.30,33 CHAT was 
highly rated by past participants (N=217); mean program ratings ranged from 4.5 to 4.8 on a 
1(low) to 5 (high) scale.34  
 
To evaluate the potential impact of CHAT on psychotropic medication use, we extracted CMS 
NHQM data for use of antipsychotics for the 10 CHAT NHs for which data were available. We 
compared each NH’s rates for two quarters before and two quarters after each participated in 
CHAT. The percentage of residents prescribed antipsychotics decreased by an average of 4.9 
percentage points, from 20.7% (SD=8.5) to 15.8% (SD=9.0) after CHAT. We used state average 
rates of antipsychotic medication use for the same time periods to control for changes that may 
have occurred due to the CMS mandate and other factors.47 The state benchmark for prescribed 
antipsychotics averaged 25.4% across quarters before and declined to 24.9% for quarters after 
CHAT, a decrease of 0.5%. The average decrease in CHAT NHs (4.9%) compared to the other 
state NHs (0.5%) was significant (p=0.05 for two-tailed t-test).48 
 
Despite the success of CHAT in reducing elderspeak and RTC, we found challenges to educating 
NH staff that limited participation in CHAT including turnover, absenteeism, heavy workloads, and 
personal conflicts.35,36 37 Each CHAT session was held multiple days and times. Still, as few as 44% 
of staff in one NH completed at least two of the three sessions,30 although this rate is higher than 
that noted for other NH staff training programs.38 Although successful in reducing RTC, the 
classroom format limits staff access and participation and feasibility for widespread dissemination. 
Creative, efficient approaches are needed to overcome NH staff education barriers.39,40 We first 
evaluated online web conference training with multiple NHs as an alternative format for 
increasing access and dissemination.41 However, engagement of individual staff was limited with 
this approach. To facilitate dissemination, online CHAT modules (CHATO) were developed to 
provide the same CHAT content with asynchronous and independent access for busy NH staff.37  
 
The PI worked with an instructional designer, item writer, and media team to transition CHAT 
content, including 20 video clips of NH staff-resident interactions, to the online CHATO 
modules.37 Scripts from the original CHAT were narrated to maintain content, integrating adult 
learning theories and principles for online learning, and eliminating a need for advanced literacy 
skills. Interactive scenario and game-based activities engage staff. For example, participants 
watch a video clip, select problem communication in the transcript, type their improved 
communication, and compare it to suggested corrections. Moderated online discussions are 
included in the modules that are supported on Training-Source.org, a free and publicly available 
learning portal. 
 
IT functionality and content equivalency of the newly developed CHATO online modules was 
demonstrated by a convenient university-affiliated sample of nurses, CNAs, and students (N=9) 
with NH experience. They rated the online modules at 4.7 to 5.0 on a 1 to 5 scale (1 = strongly 
disagree to 5 = strongly agree) for increasing their knowledge, usefulness, clarity, and 
satisfaction. CHATO was rated as easy by 57%, while 43% found it somewhat challenging but 
not too difficult.  All reported moderate to great improvement in recognizing and understanding 
ineffective communication and developing effective communication skills.37  Pre to post test 
scores improved from M = 82.4% (SD 10.62) to M = 91.21% (SD 8.22). Effect sizes for pre-post 
changes for identifying ineffective, appropriate, and person-centered communication in a video 
clip were greater for CHATO participants compared to past CHAT participants (N = 217).34,37 
Although the group testing CHATO may not represent all NH staff, findings confirm that 
CHATO is feasible and comparable in content and effects. 
 
While CHAT effectively reduced RTC, the in-person classroom format required an onsite 
interventionist, which limits accessibility and feasibility for dissemination. A pilot test of 

     
 
6 
acceptability and preliminary efficacy of online CHATO modules is the next logical step. The 
pilot will prepare for a pragmatic clinical trial that will test the effects of improved staff 
communication (from CHATO) on resident BPSD, using available clinical Minimum Data Set 
(MDS) data. Effects on psychotropic medication use, measured by MDS and Nursing Home 
Quality Measures (NHQM) data, will also be evaluated. This research will address the gap in 
rigorous trials testing nonpharmacological interventions to decrease BPSD that also identify 
strategies to improve intervention dissemination.27The goal is to increase access to CHATO 
training, as a tool to reduce BPSD and inappropriate psychotropic medication use to improve 
dementia care.  
3 
STUDY DESIGN 
The R61 will prepare for the R01 pragmatic trial by establishing feasibility of online modules and 
preliminary efficacy of CHATO with NH staff. The research design is a randomized clinical trial. 
One NH will provide initial feasibility testing.  Any modifications to the modules will be made. 
Then 6 nursing homes (estimated N=150 staff) will be randomly assigned to intervention or wait-
list control groups. The primary outcome will be knowledge gain for staff completing CHATO 
training. Additional outcomes include resident quality measures related to behavioral and 
psychological symptoms of dementia (BPSD) on both resident and facility levels and facility 
level data related to inappropriate use of psychotropic medications to control BPSD. 
 
Staff Knowledge Gain. At time 1, nursing home staff in both groups will complete baseline pre-
tests of knowledge and rating of communication. The immediate intervention group will then 
complete the online CHATO modules within a one-month period.  This includes assessments of 
post-test knowledge, communication rating, program evaluation, and diffusion of innovation 
surveys at Time 2. Also, at Time 2, the wait-list control group will repeat the pre-tests of 
knowledge and communication rating and will then complete the CHATO online module training 
over a one-month period, followed by Time 3 collection of post-test knowledge, communication 
rating, program evaluation, and diffusion of innovation surveys. Knowledge gain and 
communication rating data will be compared between the intervention and wait-list control 
groups and within nursing homes before and after the CHATO training. Changes in knowledge 
from Time 1 (baseline) to Time 2 will be compared between immediate and wait-list groups using 
model estimates obtained with a linear mixed model (LMM) approach to account for repeated 
measures and clustering within nursing homes.  Next, pre- to post-training changes in knowledge 
will be combined for immediate and wait-list groups and also tested using LMM approach. The 
models will be adjusted for covariates, such as staff role (RN, LPN, CNA, other), experience, and 
whether the nursing home has a SCU or not.  
 
TABLE 1. R61 Study Design 
Group 
Time 1 
1 Month 
Time 2 
1 Month 
Time 3 
Immediate 
3 NHs 
Baseline 
assessment1 
CHATO 
training2 
Post-training 
assessment3 
 
 
Wait list 
3 NHs 
Baseline 
assessment1 
 
Repeat 
baseline 
assessment1 
CHATO 
training2 
Post-training 
assessment3 
1Pre-test knowledge, Communication rating; 2 Three modules over a one-month period;  
3 Post-test knowledge, Communication rating, Program Evaluation, Diffusion of Innovation 
 
Resident Quality Measure Outcomes. In addition, nursing homes in both groups will provide 
monthly summary reports for behavioral symptom occurrence for the facility (in aggregate as 
well as for individual residents [deidentified]). A nursing home aggregate antipsychotic 
medication use report will also be collected and analyzed. The BPSD and medication reports will 

     
 
7 
be provided for the one-month period before baseline data collection and for each month 
thereafter until the post-training assessment of the wait-listed nursing homes is completed (a total 
of seven months for both groups). Resident outcomes data will be compared between the 
intervention and wait-list control groups and within nursing homes before and after the CHATO 
training. Seven months of data will be plotted to examine changes in nursing homes and resident 
outcomes.  
 
Implementation Strategies and Process Evaluation. An Implementation Toolkit for NHs and a 
CHATO Training Manual (See Appendices) have been created to provide support and 
implementation suggestions to pilot nursing homes. Several consultants provided feedback and 
additional resources to be included in these materials. Each NH will be given a three-month 
period to complete the CHATO training. One month for orientation, team development, and 
planning, one month to complete the three-week training, and one month for staff recognition and 
follow up. The CHATO Research Team will meet with the NH leadership at the beginning and 
end of this three-month period, manage CHATO training virtual discussion board, and provide 
technical assistance as necessary. The process evaluation includes an online implementation 
survey (See Appendices) which identifies the strategies NHs used to implement the training and 
includes the Artifacts of Culture survey. The survey uses the Diffusion of Innovation framework 
and mirrors the Implementation Toolkit. Additional process evaluation activities include: 
Leadership phone interviews completed by the consultants and external evaluators at LeadingAge 
and open-ended questions asked of direct care staff in the CHATO virtual discussion. 
 
TABLE 2. R61 Research Timeline 
 
Quarter 1 
Quarter 2 
Quarter 3 
Quarter 4 
Quarter 5 
Quarter 6 
Planning (IRB, DSMB, 
Manuals, Measurement) 
Pilot CHATO Modules 
Analyses:  
   Knowledge Gained 
   Process Evaluation 
   Resident Outcomes 
Dissemination 
 
 
 
 
 
 
4 
SELECTION AND ENROLLMENT OF PARTICIPANTS  
4.1 Inclusion Criteria  
NH Participating in CHATO. A NH home is eligible to participate in the study if they a) serve 
people with dementia, and b) have internet available for staff to complete the CHATO training, 
and c) are willing to complete leadership interviews and surveys. Seven NHs from Presbyterian 
Manor will participate in the pilot due to past relationships with organization. The nursing homes 
that were unable to participate in the original CHAT study will participate in the CHATO pilot. 
 
Staff Participants. NH staff including CNAs and nurses who are permanent employees and who 
provide direct care at least 8 hours weekly will be invited to complete the CHATO training, 
available by URL link. Other personnel, such as housekeeping, dietary, and administrators may 
also participate. Our prior research found that a variety of staff benefit from CHAT, but we will 
focus on direct care nursing staff who communicate most with residents. Participation by as many 
staff as possible is desired to achieve facility-wide communication change. NHs will provide the 
staff that will take the training. Their roles and other demographics will be collected in the 
training module.  
 
Resident Data. Aggregate, deidentified data for residents will be collected from participating 
NHs. Those with Alzheimer’s disease or non-Alzheimer’s dementia documented on the MDS 

     
 
8 
Active Diagnoses list will be included in the analyses.   
4.2 Exclusion Criteria  
NH Participating in CHATO. Assisted Living facilities or other types of facilities are excluded 
as well as NHs that previously participated in the CHAT study. 
 
Staff Participants. All staff are welcome to participate in the CHATO training, no staff 
exclusions. 
4.3 Study Enrollment Procedures  
For the R61, we partnered with seven Presbyterian Manor sites to pilot the CHATO intervention. 
These facilities have already been screened for eligibility and/or ineligibility. Letters of 
agreement will be collected from each NH as well as executive leaders from Presbyterian 
Manors. All staff at these sites will be invited to participate and will read and accept an exempt 
statement prior to taking the CHATO training.  
5 
STUDY INTERVENTIONS  
5.1 Interventions, Administration, and Duration  
CHATO is a direct Internet module adaptation of the CHAT program, developed by the PI and an 
instructional design team to include all CHAT content and interactivity. Standards for online 
education were used to transition to the module format. CHAT(and CHATO) have 
interchangeable content and are conceptually based on the Communication Predicament of Aging 
16 that describes how stereotypes of older adults as incompetent prompt younger persons (NH 
staff) to alter their communication with older adults (NH residents). The exaggerated 
simplification and clarification strategies of elderspeak result in speech that sounds like baby talk 
and is perceived as demeaning, contributing to isolation, depression, assumption of dependency, 
and BPSD.16,62 CHATO alerts NH staff to elderspeak and its negative effects, involves them in 
taking the older adult’s perspective, and guides self-evaluation and practice of effective 
communication.63 Improved communication, based on individual assessment of resident abilities, 
supports positive self-concept and meets socialization needs, reducing need-driven dementia-
compromised behavior or BPSD. 
CHAT and CHATO include review of actual NH video examples and practice of improved 
communication strategies that staff readily apply and monitor in practice, significantly reducing 
elderspeak in just three modules. Both CHAT and CHATO target adult learners using applied 
strategies tailored to improve specific aspects of elderspeak that are described in the literature and 
are modifiable. Limiting content and complexity minimizes burden to staff and their NH 
employers and increases the likelihood of skill enactment.64  
CHAT was developed step-wise and iteratively through the NIH Stage Model for Behavioral 
Intervention Development.65 Early Stage 1 development led to Stage III testing of CHAT in 
controlled NH (community) settings to establish efficacy.  After additional Stage I modification 
of the original CHAT content to online module format, CHATO is now positioned for 
effectiveness testing (Stage IV) with NH (community) providers to maximize external validity in 
preparation for future Implementation and Dissemination research (Stage V).66 
The CHATO program is presented in three modules.37,63 Session 1 introduces effective versus 
ineffective communication. Participants identify communication issues in video vignettes. 
Session 2 focuses on identification and negative effects of elderspeak, using video-recorded 

     
 
9 
examples. Session 3 includes strategies for improved communication based on assessment of 
each resident’s abilities. Participants critique communication in videos and correct transcripts to 
eliminate elderspeak. Training is limited to three, 1-hour sessions to assure feasibility in busy 
NHs. The modules include interactive exercises that must be successfully completed to advance 
through the program, assuring delivery of content fidelity and dose of the intervention. CHATO is 
provided to as many staff as possible to achieve facility-wide effects of reduced elderspeak.67   
NH sites will receive the link to the trainings in three-month intervals according to the 
randomization plan. The NH leadership will determine the best method of implementation for 
their site. The link to the training will be shared with staff and the staff will have three months to 
complete all three modules. The modules contain pre and posttests (scenario-based questions and 
rating of staff communication in video clips) to evaluate knowledge gain from CHATO. The 
Modified Duke Diffusion of Innovation58 survey provides information about CHATO participant 
intentions to use learned skills in practice and the likelihood for translation.  
During the R61, the instructional designer will assure that data for identifying each participant’s 
NH and metrics including time spent in each module are collected for analyses.34,37 
5.2 Handling of Study Interventions  
The study intervention is the Changing Talk Online (CHATO) training. The goal of course is to 
increase awareness of the importance of effective communication with older adults and to use 
evidence-based person-centered communication during interactions with older adults in nursing 
homes and other health care settings. The course is designed for staff in nursing homes, 
independent and assistive living, and health care settings in the community that include registered 
nurses, nursing assistants, nursing home dieticians, direct care professionals, other 
administrations and support employees. The total program is approximately 3 hours, split into 3 
modules. Each module is approximately an hour, depending on the individual user. Upon 
completion of all three modules and 70% on a posttest, a certificate of completion (3 nursing 
contact hours) will be awarded. 
 
Treatment 1:  CHATO Training Module 1 
This module contains information on the importance, benefits, and components of effective 
communication. This module covers the overarching goals and objectives, the benefits and 
components of effective communication, communication challenges of aging, and identification 
of effective communication (Videos). 
 
Learning Outcomes 
• 
Recognize the importance of communication to older adults. 
• 
Identify barriers to communication in care of older adults. 
• 
Distinguish between effective and ineffective communication strategies 
 
Treatment 2:  CHATO Training Module 2 
This module contains information on common communication barriers and challenges, 
elderspeak communication, and effective and ineffective communication strategies.  It reviews 
benefits and components of effective communication and introduces Elizabeth Layton paintings 
and thought exercises. The module also covers communication barriers and nursing home 
communication, ignoring talk vs. task talk vs. interpersonal talk, the Communication Predicament 
Model and elderspeak, and the identification of elderspeak and ineffective strategies (Videos). 
 
Learning Outcomes 
• 
Identify elderspeak and its potential negative messages. 

     
 10 
• 
Contrast effective and ineffective communication strategies. 
• 
Revise transcripts to reduce elderspeak and ineffective communication. 
 
Treatment 3:  CHATO Training Module 3 
This module contains information on common problems during communication, guidelines for 
improving communication, and characteristics of person-centered communication. The modules 
review nursing home communication, ineffective strategies, and elderspeak and discusses the 
Communication Enhancement Model. Affirming and effective communication and evaluating 
communication are practiced (Videos and Rewriting of Scripts). 
 
Learning Outcomes 
• 
Identify affirming communication. 
• 
Critique communication in video recordings of nursing home staff-resident communication. 
• 
Rewrite transcripts to reduce elderspeak and ineffective communication. 
The course content is presented by Dr. Kristine Williams, RN, PhD. There are learning activities, 
discussion forums, list of resources, pre and posttests, and evaluations at the end of the course. 
The practice activities were designed to provide opportunities to apply the knowledge and skills 
presented in the course and a virtual discussion forum is available to share experiences and reflect 
with others. 
5.3 Concomitant Interventions 
NHs will agree not to participate in other communication education programs during study 
involvement. Monitoring and technical assistance will be provided to each study site to ensure 
adherence. 
5.4 Adherence Assessment  
Monitoring of user statistics including: staff demographics, time spent in the modules, completion 
rate and program evaluation will be done by the Instructional Designer and her staff. Technical 
assistance including CHATO discussion boards will be provided by the Project Director and the 
Learning Systems Coordinator as necessary. 
 

     
 
11 
6 
STUDY PROCEDURES 
6.1 
Schedule of Pilot Evaluations in NHs 
 
 
Assessment 
Initial 
Leadership 
Meeting 
Treatment 
CHATO  
Planning 
(1st Month) 
Treatment  
CHATO 
Training  
(2nd Month)  
Treatment 
CHATO 
Follow-up  
(3rd Month) 
Closing 
Leadership 
Meeting 
Final Data 
Collection  
NH Agreement 
X 
 
 
 
 
 
Staff CHATO Participation 
 
X 
 
 
 
 
Staff Demographics 
 
 
X 
 
 
 
Staff CHATO Knowledge Pre-Test 
 
 
X 
 
 
 
Communication Rating Sheet Pre-Test 
 
 
X 
 
 
 
Staff CHATO Knowledge Post-Test 
 
 
X 
 
 
 
Communication Rating Sheet Post-Test 
 
 
X 
 
 
 
Modified Duke Diffusion of Innovation Intention Survey 
 
 
X 
 
 
 
Staff CHATO Evaluation 
 
 
X 
 
 
 
Implementation Survey 
 
 
 
X 
 
 
The Artifacts of Culture Change Tool 
 
 
 
X 
 
 
Staff Wage Data 
 
 
 
 
X 
 
NH Level Resident Data 
 
 
 
 
 
X 
Leadership Phone Interviews 
 
 
 
 
 
X 

     
 12 
6.2 Description of Evaluations  
6.2.1 Screening and Randomization 
Screening 
For the R61, we partnered with seven Presbyterian Manor sites to pilot the CHATO 
intervention. These facilities have already been screened for eligibility and/or ineligibility. 
Letters of agreement will be collected from each NH as well as executive leaders from 
Presbyterian Manors. All staff at these sites will be invited to participate and will read and 
accept an exempt statement prior to taking the CHATO training.  
 
Randomization 
Randomization of pilot NHs will occur to prior to any data collection. Those assigned to the 
pilot intervention group will receive the training first while the pilot control group is 
waitlisted. Randomization will occur by using a random number generator.   
6.2.2 Consenting and Enrollment 
Consenting Procedure 
The project director will meet with NH leadership and obtain the Letter of Agreement at the 
initial leadership meeting at the start of the NH’s assigned three-month intervention period. 
Signed Letters of Agreement will be stored electronically on a secure KUMC server. This 
procedure has been approved by the KUMC IRB.  
 
Enrollment 
Enrollment in the study begins with the initial leadership meeting and a signed letter of 
agreement from the participating NH. During the initial meeting, we will provide leadership 
and overview and implementation strategies as well as links to additional resources. We will 
also collect a list of staff emails. The training system will send the staff a link to the CHATO 
training and reminder emails. 
 
§ 
Initial Leadership Meeting 
o Letter of Agreement 
o List of staff and their emails that will participate in CHATO training 
6.2.3 Treatment: CHATO Training 
The treatment schedule may vary according to the needs of each individual nursing home; 
however, all modules will be completed by as many staff as possible during the three-month 
intervention period. Technical assistance will be provided as needed.     
  
§ 
Treatment:  CHATO Training Module 1 
o Staff Demographics 
o Communication Rating Sheet Pre-Test 
o CHATO Knowledge Pre-Test 
 
§ 
Treatment:  CHATO Training Module 2 
 
§ 
Treatment:  CHATO Training Module 3 
o Communication Rating Sheet Post-Test 
o CHATO Post-Test 
o Modified Duke Diffusion of Innovation Intention Survey 

     
 13 
o CHATO Program Evaluation 
6.2.4 Completion and Final Evaluation 
Competition 
The NH’s three-month intervention period will conclude with a closing leadership meeting 
that will include implementation strategies, challenges and successes, overall lessons learned, 
collection of surveys, and staff wage data.  
 
§ 
By Closing Leadership Meeting 
o Implementation Survey 
o The Artifacts of Culture Change Tool 
o Staff Wage Data 
 
If NHs need to discontinue study interventions early, circumstances and reasons will be 
documented. Discontinuation from the CHATO intervention is not anticipated, however, 
organizational confounds or special circumstances could interfere with participation rate.   
 
Final Evaluation 
Deidentified resident data will be provided by Presbyterian Manor for one-month prior to the 
training and one-month post. External evaluators will conduct leadership phone interviews 
following the closing leadership meeting. Interviews with leadership staff will provide insight 
into most effective implementation strategies and overall evaluation of the CHATO training 
process. 
 
§ 
Final Data Collection  
o NH Level, Deidentified Resident Data from executive management 
o Leadership Interviews by external evaluator 
7 
SAFETY ASSESSMENTS  
7.1 Specification of Safety Parameters  
This is a minimal risk pilot test of an online version of an educational program for nursing home 
staff that has been established as acceptable and effective. There are minimal risks for 
participants and participating in online continuing education is a regular job-related activity for 
nursing home staff. Participants may experience concern about their performance and ability to 
successfully complete the post-tests and about any sharing of performance-related data with their 
supervisor (only aggregate results that do not identify specific staff will be shared). Participants 
may experience frustration if they are challenged to navigate the online modules (only basic 
computer skills are required). Participants may experience some burden including the time and 
effort required to complete the online training modules, however this is limited to a 3-4 hour 
period.   
 
CNA staff risks of participation include revealing CHATO training performance and survey 
responses to supervisory staff and administrators who may use this information for performance 
evaluations. Thus, NH administrators will not have access to data that identifies individual staff. 
Staff will be assured that descriptive data, CHATO performance, and survey responses will not be 
shared with administrators, supervisory staff, or other staff. In addition, recordings that may be 
used for future training materials or presentations will not identify the CNA or their NH 
employer. No appreciable risk of physical, psychological, social, legal or other harm to staff 
participants is expected. No information about past, present, or future physical or mental health or 

     
 14 
payment of health care benefits (HIPAA protected information) will be collected. 
 
Another potential risk is accidental disclosure of data outside the research team. Accordingly, all 
data will be handled as confidential by assigning codes, not including names, from the point of 
archiving through data analyses. The original data files and file linking data identification codes 
to participants will be stored in locked secure files or on secure KUMC servers. Data access will 
be controlled by the research protocol.  
7.2 Methods and Timing for Assessing, Recording, and Analyzing Safety Parameters 
Data Safety Monitoring Board (DSMB) will provide oversight to assure the safety of participants 
and integrity of the data. The members of the DSMB include a team of interdisciplinary 
professionals who are not a part of the research team. The DSMB members includes clinical and 
research experts on NH dementia care and NH staff education as well as a statistician.  
 
The DSMB will meet twice yearly and additionally if needed at the request of the PI (due to 
issues arising in study conduct or major changes in study protocols). The DSMB will review 
reports of the research team concerning data quality and timeliness, adherence to study protocols, 
participant recruitment and accrual, and consenting procedures.  The research team (under the 
direction of the PI) will report accrual, retention figures, and all adverse and other events and 
outcomes. 
 
 
The Principal Investigator (PI) will be responsible for ensuring participants’ safety daily and will 
monitor participant safety, evaluate progress of the study, and assure confidentiality, data 
management, and analysis. This includes continuous active review of research activities that will 
be completed with assistance of the Project Director (PD).     
 
The PI will be responsible for submitting necessary reports (including minutes of annual and any 
additional DSMB meetings) to the funder. The PI will provide timely reporting of a) any 
unanticipated problems or unexpected adverse or serious adverse events that are determined to be 
related to the study protocol, b) IRB-approved revisions to the study protocol that indicate a 
change in risk for participants, c) a summary of recommendations made by the DSMB and if 
applicable the action plan for response, and d) notice of any actions taken by the IRB or 
regulatory bodies regarding the research and any responses to those actions.  
 
To assure data control and safety for participants, each research team member will be trained and 
supervised by the site PI and/or PD, including instruction in all research activities, data handling, 
and in reporting subject complaints or other behaviors indicating a negative response to 
participation in the study immediately to the PI. Research team staff will be supervised during 
direct interactions with research study participants and provided with feedback.   
 
All data identifiers will be removed for secured computer storage; a code will be assigned to each 
subject for data storage and the key linking the code number to subject identification will be kept 
secure by each site PI. All data will be deidentified prior to dissemination outside the research 
team.  
7.3 Adverse Events and Serious Adverse Events  
The OHRP Unanticipated Problem and AE Guidance definitions will be used for this study. 
Adverse Events (AE) will be documented on the Adverse Event Form and unanticipated events 
that are not adverse events will be recorded on the Unanticipated Problem form. Adverse events 
and unanticipated problems (Ups) will be reported to the IRB, DSMB and NIH Program Officer 

     
 15 
as required. The PI will evaluate each AE for severity, whether it was expected or unexpected, 
and will determine relationship to participation in the study. 
 
A Serious Adverse Event (SAE) occurs when an unanticipated event which may result in death, 
life threatening or places participant at risk of death, requires of prolongs hospitalization, causes 
persistent or significant disability or incapacity, results in congenital anomalies or birth defects, 
or otherwise is judged to represent a significant hazard. SAE are not expected in this minimal risk 
exempt pilot study, however, the Principal Investigator (PI) will be responsible for ensuring 
participants’ safety on a daily basis and will monitor participant safety. 
7.4 Reporting Procedures 
When SAEs occur that are unanticipated and are related to the intervention, they will be reported 
to the NIA Program Officer and to the DSMB Chair or designee, within 48 hours of the research 
team’s knowledge.  
 
Ups involving risks to study participants or others will be reported to the NIA program officer 
within 48 hours unless they are also SAEs  
7.5 Follow-up for Adverse Events 
A summary of all SAEs will be reported to the NIA Program Officer and to the DSMB quarterly.    
7.6 Safety Monitoring 
The DSMB will initially review and approve the written study protocol; data collection and 
storage protocols, intervention materials, informed consent procedures, and the plan for data and 
safety monitoring. Any outside agreements with vendors and subcontractors will be reviewed.  
The DSMB will also review the general reporting guidelines based on the study procedures 
approved by IRB. Reporting rules will be established taking into consideration the population 
under study and anticipated adverse outcomes (reflected in the human subjects’ consent 
documents).  
 
The DSMB will meet twice annually, either in-person or by teleconference call to review study 
progress, data quality, and participants safety. Safety reports are sent to the SO if necessary twice 
a year and will include a detailed analysis of study progress, data and safety issues. The content 
of the data and safety monitoring report will include: (e.g., study status, participant descriptive 
information, safety information, study quality). Blinded reports will be produced for open 
sessions and unblinded reports for closed sessions. 
8 
INTERVENTION DISCONTINUATION  
This is a low risk study consisting of pilot testing an educational intervention. Thus, it is difficult 
to anticipate specific, a-priori stopping rules. If the research team, IRB, NIA program officer, 
and/or DSMB identify a pattern to these events, an interim analysis of safety data will be 
conducted to inform a decision for premature stoppage of study. Since this is a low-risk, short 
(six-month) pilot test of an educational intervention, no interim analyses on the primary study 
outcomes are planned. 

     
 16 
9 
STATISTICAL CONSIDERATIONS  
9.1 General Design and Outcomes  
Primary Objective 
 
AIM 1. Establish acceptability and preliminary efficacy of online CHATO modules through pilot 
testing with NH staff. 
 
Staff Outcomes: Knowledge gain and communication rating data will be compared between the 
intervention and wait-list control groups and within nursing homes before and after the CHATO 
training. 
 
Resident Outcomes: The BPSD and medication reports will be provided for the one-month 
period before baseline data collection and for each month thereafter until the post-training 
assessment of the wait-listed nursing homes is completed (a total of seven months for both 
groups). Resident outcomes data will be compared between the intervention and wait-list control 
groups and within nursing homes before and after the CHATO training.  
 
Secondary Objectives 
 
AIM 2. Develop and pilot test the data collection tool with consultant input. Interviews of NH 
administrators and staff who participate in the pilot testing of CHATO and a process evaluation 
will be used to identify and develop supports for implementation and sustainability in preparation 
for future CHATO testing. 
 
 
Implementation Outcomes: We will analyze NH strategies used to engage staff in CHATO 
(approach, motivation, incentives). Strategies will be described, categorized, and correlated with 
participation rates. A survey for CHATO group NHs, will describe how CHATO was 
implemented and concurrent activities that may have influenced BPSD and psychotropic use 
rates. The Artifacts of Culture Change Tool will describe the NH care environment and practices, 
leadership and workplace practices, staffing outcomes, and occupancy.100 In addition, a simple 
cost-effectiveness analysis (CEA) for the cost associated with hypothesized BPSD reductions will 
be calculated. 
9.2 Sample Size and Randomization 
Seven NHs will be used to pilot the CHATO intervention during the R61. One nursing home will 
be used to test overall feasibility. The additional six will be randomized into three intervention 
and three waitlist controls using a random number generator.  We estimate a CHATO training 
sample size of 150 participants.  
9.3 Interim analyses and Stopping Rules 
This is a low risk study consisting of pilot testing an educational intervention. Thus, it is difficult 
to anticipate specific, a-priori stopping rules. If the research team, IRB, NIA program officer, 
and/or DSMB identify a pattern to these events, an interim analysis of safety data will be 
conducted to inform a decision for premature stoppage of study. Since this is a low-risk pilot test 
of an educational intervention, no interim analyses on the primary study outcomes are planned. 
9.4 Data Analyses 
SAS software71 will be used for all analyses. Initial analysis will calculate descriptive statistics at 

     
 17 
all data collection points. Outliers will be investigated for accuracy and possible entry errors. 
Graphic representation of data and correlations among variables will be examined. Patterns and 
site effects of missing data will be examined in relation to variables of interest.72  
 
Knowledge gain and communication rating data will be compared between the intervention and 
wait-list control groups and within nursing homes before and after the CHATO training. Changes 
in knowledge from Time 1 (baseline) to Time 2 will be compared between immediate and wait-
listed groups using model estimates obtained with a linear mixed model (LMM) approach to 
account for repeated measures and clustering within nursing homes. Next, pre- to post-training 
changes in knowledge will be combined for immediate and wait-list groups and also tested using 
LMM approach. The models will be adjusted for covariates, such as staff role (RN, LPN, CNA, 
other), experience, and whether the nursing home has a SCU or not. Distributions of residuals 
will be evaluated for normality and variance homogeneity and variables will be transformed as 
needed or statistical analyses for non-normal data will be utilized. 
 
The BPSD and medication reports will be provided for the one-month period before baseline data 
collection and for each month thereafter until the post-training assessment of the wait-listed 
nursing homes is completed (a total of seven months for both groups). Resident outcomes data 
will be compared between the intervention and wait-list control groups and within nursing homes 
before and after the CHATO training. Seven months of data will be plotted to examine changes in 
nursing homes and resident outcomes.  
 
We will analyze NH strategies used to engage staff in CHATO (approach, motivation, 
incentives). Strategies will be described, categorized, and correlated with participation rates. A 
survey for CHATO group NHs, will describe how CHATO was implemented and concurrent 
activities that may have influenced BPSD and psychotropic use rates. The Artifacts of Culture 
Change Tool will describe the NH care environment and practices, leadership and workplace 
practices, staffing outcomes, and occupancy.100  In addition, a simple cost-effectiveness analysis 
(CEA) for the cost associated with hypothesized BPSD reductions will be calculated. 
 
Scores for each engagement strategy category will be calculated, summarized using means and 
standard deviations, and correlated with participation rates. Strategy categories (approach, 
motivation, incentives) will be summarized using frequencies and percentages and examined in 
relation to participation rates using ANOVA or Kruskal-Wallis test, depending on the distribution 
of participation rates. Participation rates will be calculated using data from the NHs (eligible 
staff) and from the modules (CHATO participants). Data describing CHATO participant time 
spent on each module and completion rates, all collected by the learning system, will also be 
analyzed. We will also evaluate NH attrition and within-NH dropout/completion rates for 
CHATO participants. We will also evaluate CEA differences for the three typical NH scenarios.56  
10 
DATA COLLECTION AND QUALITY ASSURANCE 
10.1 Data Collection Forms 
Data will be collected according to Table 4. The data itself will be collected via the Moodle 
learning system managed by the Instructional Designer or RedCAP managed by the Project 
Director. The person responsible for collection will send data to the data manager for data 
cleaning and analysis. The data manager is the individual designated for managing data files and 
for assuring confidentiality. She will provide data directly to the DSMB and Program Officer 
upon request. She will serve as the data coordinator for the study.  
 

     
 18 
 
 
TABLE 4. Data Collection Forms, Method of Collection, and Person Responsible. 
Data Collection Form 
Method of Collection 
Person Responsible 
NH Agreement/Letter of Support 
NH Leadership/Word 
Project Director 
Staff CHATO Participation 
NH Leadership/Excel 
Project Director 
Staff Demographics 
CHATO Training/Moodle 
Instructional Designer 
Staff CHATO Knowledge Pre-Test 
CHATO Training/Moodle 
Instructional Designer 
Communication Rating Sheet Pre-Test 
CHATO Training/Moodle 
Instructional Designer 
Staff CHATO Knowledge Post-Test 
CHATO Training/Moodle 
Instructional Designer 
Communication Rating Sheet Post-Test 
CHATO Training/Moodle 
Instructional Designer 
Modified Duke Diffusion of Innovation Intention Survey 
CHATO Training/Moodle 
Instructional Designer 
Staff CHATO Evaluation 
CHATO Training/Moodle 
Instructional Designer 
Implementation Survey 
NH Leadership/RedCAP 
Project Director 
The Artifacts of Culture Change Tool 
NH Leadership/RedCAP 
Project Director 
Staff Wage Data 
NH Leadership/RedCAP 
Project Director 
NH Level Resident Data 
NH Executives/Internal Data System 
Project Director 
Leadership Phone Interviews 
NH Leadership/Phone Interview 
LeadingAge Evaluators 
10.2 Data Management  
The participating nursing homes will not be responsible for collecting any data onsite. They will 
provide a NH Agreement, identify staff taking the training, take the Implementation Survey and 
The Artifacts of Culture Change Tool, and provide staff wage data. NH Chain Executives will 
provide NH level deidentified resident data. 
 
Data will be collected in RedCAP and the Moodle learning system and provided to the data 
manager for data monitoring, cleaning, and analysis. Data will be stored on the collection 
platform and on secure servers at the University of Kansas Medical Center and/or the University 
of Iowa.  
 
TABLE 5. Data Collection Forms Description 
Data Collection Form 
Description 
NH Agreement 
A signed letter of agreement from the participating NH indicating participation in the 
research. 
Staff CHATO Participation 
List of participating staff and their email address where they can receive the link to 
the CHATO training. 
Staff Demographics 
Participants’ age, sex, race, ethnicity, NH employer, role, education, and length of 
time in current NH and health care role. 
Staff CHATO Knowledge Pre-Test/Post-Test 
Two forms (Forms A and B) measuring knowledge gained from training. 
Communication Rating Sheet Pre-Test/Post-Test 
Participant watches a video and answers questions which tests their ability to 
visually and audibly identify effective vs ineffective communication strategies and 
recognize elderspeak vs. person-centered care.  
Modified Duke Diffusion of Innovation Intention Survey 
Participants rates the training intervention on four scales: complexity and 
compatibility, observability/image, organizational support, and intention to use.  
Staff CHATO Evaluation 
Program evaluation of the CHATO training.  
Implementation Survey 
Measures approach, motivation, and incentives used to implement CHATO. 
The Artifacts of Culture Change Tool 
Measures NH care environment and practices, leadership and workplace practices, 
staffing outcomes, and occupancy. 
Staff Wage Data 
Wages per hour by NH role. 
NH Level Resident Data 
Resident behavioral and medication data. 
Leadership Phone Interviews 
Qualitative data regarding implementation strategies, lessons learned, and overall 
evaluation of the CHATO pilot from the NH leadership perspective.  

     
 19 
10.3 Quality Assurance  
10.3.1 Training 
The PI will conduct an annual review of research team member compliance with IRB required 
training and certification. The PI or PD will train and supervise research team members in 
recruitment and consenting procedures, data collection, provision of intervention, and data 
analysis and coding on a quarterly basis.  
10.3.2 Quality Control Committee  
The DSMB responsibilities are to: review the research protocol, informed consent documents 
and plans for data safety and monitoring; advise the NIA on the readiness of the study staff to 
initiate recruitment; evaluate the progress of the trial, including periodic assessments of data 
quality and timeliness, recruitment, accrual and retention, participant risk versus benefit, 
performance of the trial sites, and other factors that can affect study outcome; consider factors 
external to the study when relevant information becomes available, such as scientific or 
therapeutic developments that may have an impact on the safety of the participants or the ethics 
of the trial; review study performance, make recommendations and assist in the resolution of 
problems reported by the Principal Investigator; protect the safety of the study participants; 
report to NIA on the safety and progress of the trial; make recommendations to the NIA, the 
Principal Investigator, and, if required, to the Food and Drug Administration (FDA) concerning 
continuation, termination or other modifications of the trial based on the observed beneficial or 
adverse effects of the treatment under study; if appropriate, review interim analyses in 
accordance with stopping rules, which are clearly defined in advance of data analysis and have 
the approval of the DSMB; ensure the confidentiality of the study data and the results of 
monitoring; and, assist the NIA by commenting on any problems with study conduct, 
enrollment, sample size and/or data collection. 
The DSMB Members include: 
Dr. James Powers, Vanderbilt University, is a Geriatrician with relevant background in Quality 
Improvement, Medication Safety, and Long-Term Care.   
Dr. Kimberly Van Haitsma, Penn State University, is a Clinical Geropsychologist with 
extensive clinical experience, expertise in nursing home staff training, and improving dementia 
care.  She also brings extensive pragmatic clinical trial expertise.  
Dr. Daniel Bontempo, Texas Tech University, is a statistician with substantive social science 
expertise as a consultant as well as in multivariate and multilevel modeling methodology.  
10.3.3 Metrics 
Accurate data entry and export will be evaluated prior to the start of the pilot study. The 
research team will create accounts and log in as participants, provide dummy descriptive data, 
and complete the modules (including assessments). Results will be exported from the Moodle 
platform and reviewed to assure that all data are captured in the proper format and that the 
exported data reflect the entered data.  Accuracy will be reevaluated after the first nursing home 
completes the pilot study. 
10.3.4 Protocol Deviations 
A protocol deviation for this study will be defined by the DSMB and is outlined below. All 
protocol deviations will be recorded in the Protocol Deviations Log and reported to the DSMB 
quarterly. Only protocol deviations that impact participant safety will be reported within 24 
hours of occurrence if possible, or as soon as they are discovered. A log for recording protocol 
deviations will be used.  

     
 20 
Protocol deviations/violations include, but are not limited to, the following: 
• 
Randomization of an ineligible participant 
• 
Failure to obtain Informed Consent 
• 
Enrollment of a participant into another study  
• 
Failure to keep IRB approval up to date 
• 
Wrong treatment administered to participant 
 
Variations in how individual nursing home implement the CHATO training will also be 
monitored. Any implementation protocol deviations for the 7 pilot homes will be recorded by 
the instructional designer and/or project director. 
10.3.5 Monitoring 
The instructional designer and her staff will monitor the Moodle system. The project director 
will monitor RedCAP. All monitoring will be done in conjunction with the data manager and 
statistician. All coordination of research activities will be monitored by the PI. Review of these 
systems will be done daily during the pilot to ensure the training and any data collection are 
being completed as determined by the protocol. 
11 
PARTICIPANT RIGHTS AND CONFIDENTIALITY  
11.1 Institutional Review Board (IRB) Review 
Approval of the Kansas University Medical Center (KUMC) Institutional Review Board (IRB) 
for the protection of human subjects was obtained on February 5, 2019. The KUMC IRB will 
act as the one IRB for this study with agreement from the IRB at the University of Iowa sub 
award site.  
 
Research personnel will abide by all tenets of the University confidentiality policies as well as 
the Privacy Protection for Research. All research staff will remain current in the NIH required 
Human Subjects protections and HIPAA certifications. KUMC partners with the Collaborative 
Institutional Training Initiative at the University of Miami (CITI) to provide Human Research 
Protection training and other types of training to our research community. This tutorial training 
is available on line for all research team members and will be completed by all research team 
personnel. Consultants will be required to submit proof of Human Subjects Protections and 
HIPAA training from their respective institutions or may complete the CITI program. 
11.2 Informed Consent Forms 
A letter of agreement will be obtained from nursing homes (NHs) who participate in the study.  
The KUMC IRB will act as their IRB and the letter of agreement to participate from an 
authorized administrator will be submitted to the KUMC IRB for approval. Staff read an 
exempt statement prior to taking the first module of the CHATO training.  
11.3 Participant Confidentiality 
All data will be handled as confidential by assigning codes, not including names, from the point 
of archiving through data analyses. The original data files and file linking data identification 
codes to participants will be stored in locked secure files. Data access will be controlled by the 
research protocol. Any data suggesting physical or mental harm or illegal behaviors will be 
reported to the NH administration and state authorities following mandated reporting guidelines. 
No information about past, present, or future physical or mental health or payment of health 
care benefits (HIPAA protected information) will be collected. 

     
 21 
11.4 Study Discontinuation  
The study may be discontinued at any time by the IRB, the NIA, the OHRP, or other 
government agencies as part of their duties to ensure that research participants are protected.  
12 
ETHICAL CONSIDERATIONS 
This study will include NH staff. In addition, NH resident data from the MDS will be used to 
evaluate study outcomes. Of particular importance to these populations is the protection of 
privacy and confidentiality of materials that might be used for performance evaluations or 
released to unauthorized parties. This is a minimal risk study because protections and protocols 
limit the probability and magnitude of harm or discomfort from participation. Harm and 
discomfort are not greater than that ordinarily encountered in daily life or during the 
performance of routine physical or psychological examinations or tests. 
13 
PUBLICATION OF RESEARCH FINDINGS 
Any presentation, abstract, or manuscript will be made available for review by the sponsor prior 
to submission. 
14 
REFERENCES 
1. 
Alzheimer's Association. Alzheimer's disease facts and figures. 2016. 
http://www.alz.org/documents_custom/2016-facts-and-figures.pdf. Accessed July 26, 2016. 
2. 
Family Caregiver Alliance. Selected Long-Term Care Statistics. National Center on 
Caregiving;2015. 
3. 
Cerejeira J, Lagarto L, Mukaetova-Ladinska E. Behavioral and Psychological Symptoms of 
Dementia. Frontiers in Neurology. 2012;3(73)  
4. 
Kales HC, Gitlin LN, Lyketsos CG. Assessment and management of behavioral and 
psychological symptoms of dementia. BMJ : British Medical Journal. 2015;350  
5. 
Buhr GT, Kuchibhatla M, Clipp EC. Caregivers' reasons for nursing home placement: clues 
for improving discussions with families prior to the transition. Gerontologist. 
2006;46(1):52-61  
6. 
Toot S, Swinson T, Devine M, Challis D, Orrell M. Causes of nursing home placement for 
older people with dementia: a systematic review and meta-analysis. International 
Psychogeriatrics. 2016;29(2):195-208  
7. 
Brodaty H, Draper B, Low L. Nursing home staff attitudes towards residents with 
dementia: Strain and satisfaction with work. Journal of advanced nursing. 2003;44(6):583-
590  
8. 
Grant L, Pothoff S, Ryden M, Kane R. Assignment, Staff Ratios, and Training in 
Alzheimer's Special Care Units. Journal of Gerontological Nursing. 1998;24:9-16  
9. 
Maas M, Buckwalter KC. Providing quality care in Assisted Living Facilities: 
Recommendations for enhanced staffing and staff training. Journal of Gerontological 
Nursing. 2006;32:14-20  
10. 
Beeri MS, Werner P, Davidson M, Noy S. The cost of behavioral and pshychological 
symptoms of dementia (BPSD) in community dwelling AD patients. International Journal 
of Geriatric Psychiatry. 2002;17:403-408  
11. 
Genworth Financial. Cost of Care Survey. 2016. 
http://www.genworth.com/content/etc/medialib/genworth/us/en/Long_Term_Care.Par.1429
1.File.dat/37522%20CoC%20Brochure.pdf. Accessed November 19, 2008. 
12. 
Williams K, Herman R, Gajewski B, Wilson K. Elderspeak communication: Impact on 
dementia care. American Journal of Alzheimer’s Disease and Other Dementias. 
2009;24:11-20.PMC2823803. 

     
 22 
13. 
Gitlin LN, Winter L, Dennis MP, Hodgson N, Hauck WW. Targeting and Managing 
Behavioral Symptoms in Individuals with Dementia: A Randomized Trial of a 
Nonpharmacological Intervention. Journal of the American Geriatrics Society. 
2010;58(8):1465-1474  
14. 
Ryan EB, Giles H, Bartolucci RY, Henwood K. Psycholinguistic and social psychological 
components of communication by and with the elderly. Language and Communication. 
1986;6(1/2):1-24  
15. 
Kemper S, Ferrell P, Harden T, Finter-Urczyk A, Billington C. Use of elderspeak by young 
and older adults to impaired and unimpaired listeners. Aging, Neuropsychology, and 
Cognition. 1998;5(1):43-55  
16. 
Ryan EB, Hummert ML, Boich LH. Communication predicaments of aging; Patronizing 
behavior toward older adults. Journal of Language and Social Psychology. 1995;14(1-
2):144-166  
17. 
Kitwood T, Bredin K. Towards a theory of dementia care: Personhood and wellbeing. 
Ageing and Society. 1992;12:269-287  
18. 
Kitwood T. The experience of dementia. Aging and Mental Health. 1997;7:15-22  
19. 
Kolanowshi A, Hurwitz S, Taylor L, Evans L, Strumpf N. Contextual factors associated 
with disturbing behaviors in institutionalized elders. Nursing Research. 1994;43:73-79  
20. 
Talicero K, Evans L, Strumpf N. Mental health correlates of aggression in nursing home 
reisdents with dementia. The Gerontologist. 2002;42:169-177  
21. 
Maher A, Maglione M, Bagley S, Suttorp M, Hu JH, Ewing B, Wang Z, Timmer M, 
Sultzer D, Shekelle PG. Efficacy and comparative effectiveness of atypical antipsychotic 
medications for off-label uses in adults: A systematic review and meta-analysis. JAMA. 
2011;306(12):1359-1369  
22. 
Stevenson DG, Decker SL, Dwyer LL, Huskamp HA, Grabowski DC, Metzger ED, 
Mitchell SL. Antipsychotic and benzodiazepine use among nursing home residents: 
findings from the 2004 National Nursing Home Survey. The American journal of geriatric 
psychiatry : official journal of the American Association for Geriatric Psychiatry. 
2010;18(12):1078-1092.PMC3009456. 
23. 
Schneider  LS, Tariot  PN, Dagerman  KS, Davis  SM, Hsiao  JK, Ismail  MS, Lebowitz  
BD, Lyketsos  CG, Ryan  JM, Stroup  TS, Sultzer  DL, Weintraub  D, Lieberman  JA. 
Effectiveness of Atypical Antipsychotic Drugs in Patients with Alzheimer's Disease. New 
England Journal of Medicine. 2006;355(15):1525-1538  
24. 
Center for Medicare and Medicaid Services. National Partnership to Improve Dementia 
Care in Nursing Homes, June 2016 update,. 
25. 
Borson S, Boustani MA, Buckwalter KC, Burgio LD, Chodosh J, Fortinsky RH, Gifford 
DR, Gwyther LP, Koren MJ, Lynn J, Phillips C, Roherty M, Ronch J, Stahl C, Rodgers L, 
Kim H, Baumgart M, Geiger A. Report on milestones for care and support under the U.S. 
National Plan to Address Alzheimer's Disease. Alzheimer's & Dementia: The Journal of the 
Alzheimer's Association. 2016;12(3):334-369  
26. 
Human Rights Watch. "They Want Docile:" How Nursing Homes in the United States 
Overmedicate People with Dementia. United States, 2018. 
27. 
Thompson Coon J, Abbott R, Rogers M, Whear R, Pearson S, Lang I, Cartmell N, Stein K. 
Interventions to Reduce Inappropriate Prescribing of Antipsychotic Medications in People 
With Dementia Resident in Care Homes: A Systematic Review. Journal of the American 
Medical Directors Association. 2014;15(10):706-718  
28. 
Fossey J, Ballard C, Juszczak E, James I, Alder N, Jacoby R, Howard R. Effect of 
enhanced psychosocial care on antipsychotic use in nursing home residents with severe 
dementia: cluster randomised trial. BMJ. 2006;332(7544):756-761  
29. 
Tawaih P, Black M, Scott-Walkder M, Johnson E, Vaughan, C.,. Reducing antipsychotic 
use through culture change: An interdisciplinary effort. Annals of Long Term Care. 2016  

     
 23 
30. 
Williams K, Perkhounkova Y, Herman R, Bossen A. A Communication Intervention to 
Reduce Resistiveness in Dementia Care: A Cluster Randomized Controlled Trial. The 
Gerontologist. 2017;57(4):707-718.PMID: 27048705, PMC-In Process. 
31. 
Williams K. Improving outcomes of nursing home interactions. Research in Nursing and 
Health. 2006;29:121-133  
32. 
Williams K, Kemper S, Hummert ML. Improving nursing home communication: An 
intervention to reduce elderspeak. The Gerontologist. 2003;43(2):242-247.PMID 
12677081. No PMCID. 
33. 
Williams K, Ilten T, Bower H. Meeting communication needs: Topics of talk in the nursing 
home. Journal of psychosocial nursing and mental health services. 2005;43:38-45.PMID 
14116925. No PMCID. 
34. 
Williams K, Bossen A, Perkhounkova Y, Hein M. Knowledge to Practice: Nursing Home 
Staff Perceptions and Intentions for Learned Communication Skills. Journal of 
Gerontological Nursing. 2016;42(3):26-34.  PMC4841447. 
35. 
Beeber AS, Zimmerman S, Fletcher S, Mitchell CM, Gould E. Challenges and strategies 
for implementing and evaluating dementia care staff training in long-term care settings. 
Alzheimer's Care Today. 2010;11(1):17-39  
36. 
Low L-F, Fletcher J, Goodenough B, Jeon Y-H, Etherton-Beer C, MacAndrew M, Beattie 
E. A Systematic Review of Interventions to Change Staff Care Practices in Order to 
Improve Resident Outcomes in Nursing Homes. PLOS ONE. 2015;10(11):e0140711  
37. 
Williams K, Abd-Hamid NH, Perkhounkova Y. Transitioning Communication Education to 
an Interactive Online Module Format. Journal of  Continuing Education in Nursing. 
2017;48(7):320-328.PMC5492957. 
38. 
Mitchell CM, Zimmerman S, Beeber AS. Dementia Care Training for Long‐term Care 
Staff: If You Provide It, Will They Come? Alzheimer's Care Today. 2010;11(1):40-50  
39. 
Institute on Medicine. Redesigning Continuing Education in the Health Professions. 2009. 
40. 
Silvestre JH, Bowers BJ, Gaard S. Improving the Quality of Long-Term Care. Journal of 
Nursing Regulation. 2015;6(2):52-56  
41. 
Coleman CK, Fanning K, Williams KN. Comparing person centered communication 
education in long term care using onsite and online formats. Journal of Gerontological 
Nursing 2015;41(11):1-6.PMC 5025030. 
42. 
Hebert LE, Weuve J, Scherr PA, Evans DA. Alzheimer disease in the United States (2010–
2050) estimated using the 2010 census. Neurology. 2013;80(19):1778-1783  
43. 
Register F. Proposed  Rules: Reform of Requirements for Long-Term Care Facilities   In: 
Services DoHaH, ed. Vol CMS-3260-P2016. 
44. 
Williams K, Ayyagari P, Perkhounkova Y, Bott M, Herman R, Bossen A. Costs of a Staff 
Communication Intervention to Reduce Dementia Behaviors in Nursing Home Care. 
Journal of Nursing Home Research Sciences. 2017;3:22-27.PMC5425099. 
45. 
Harris-Kojetin L, Sengupta M., Park-Lee E., Valverde R. H, MD:. Long-term care services 
in the United States: 2013 overview. Hyattsville, MD2013. 
46. 
Berridge C, Tyler DA, Miller SC. Staff Empowerment Practices and CNA Retention. 
Journal of Applied Gerontology.0(0):0733464816665204  
47. 
Mollot RJ, Butler D. Federal Requirements & Regulatory Provisions Relevant to Dementia 
Care & the Use of Antipsychotic Drugs. New York, NY: Long Term Care Community 
Coalition;2013. 
48. 
Wiliams K, Shaw C, Perkhounkova Y. Improved Communication: Effects on Inappropriate 
Psychotropic Medication Use. Paper presented at Midwest Nursing Research Society 42nd 
Annual Research Conference; April 13, 2018; Cleveland, Ohio. 
49. 
Francis D, Roberts I, Elbourne DR, Shakur H, Knight RC, Garcia J, Snowdon C, Entwistle 
VA, McDonald AM, Grant AM, Campbell MK. Marketing and clinical trials: a case study. 
Trials. 2007;8:37-37  

     
 24 
50. 
Wu YW. Hierarchical linear models: A multilevel data analysis technique. Nursing 
Research. 1995;44:123-126  
51. 
Bennett DA, Wilson RS, Schneider JA, Evans DA, Bechkett LA, Aggarwal NT, Barnes 
LL, Fox JH, Bach J. Natural history of mild cognitive impairment in older persons. 
Neurology. 2002;59:198-205  
52. 
Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P. Methods and processes of the 
CONSORT group: Example of an extension for trials assessing nonpharmacologic 
treaments. Annals of Internal Medicine. 2008;148 W-60-W-66  
53. 
Keppel G, Wikkens TD. Design & Analysis: A Researcher's Handbook. 4th edition ed. 
Upper Saddle River, NJ: Prentice Hall; 2004. 
54. 
Fielding E, Beattie E, O'Reilly M, McMaster M. Achieving a National Sample of Nursing 
Homes: Balancing Probability Techniques and Practicalities. Res Gerontol Nurs. 
2016;9(2):58-65  
55. 
Harrington C, Swan J. Nursing home staffing, turnover, and case mix. Medical Care 
Research and Review. 2003;60(3):366-392  
56. 
Zimmerman S, Williams CS, Reed PS, Boustani M, Preisser JS, Heck E, Sloane PD. 
Attitudes, stress, and satisfaction of staff who care for residents with dementia. The 
Gerontologist. 2005;45(Special Issue 1):96-105  
57. 
Bowman CS, Schoeneman K. Development of the Artifacts of Culture Change Tool, Report 
of Contract HHSM-500-2005-00076P. Centers for Medicare & Medicaid Services; April 
21, 2006 2006. 
58. 
McConnell ES, Corazzini K.N., Lekan D., Bailey D.E., Sloane R., Landerman LR, 
Champagne MT. Diffusion of innovations in long-term care measurement battery. 
Research in Gerontological Nursing. 2012;5(1):54-76  
59. 
Murray D, Varnell S, Blitstein J. Design and analysis of group-randomized trials: A review 
of recent methodological developments. American Journal of Public Health. 2004;94:423-
432  
60. 
Sloane PD, Hoeffer BC, Mitchell M, McKenzie DA, Barrick AL, Rader J, Stewart. B.J., 
Talerico KA, Rasin JH, Zink RC, Koch GC. Effect of Person-Centered Showering and the 
Towel Bath on Bathing-Associated Aggression, Agitation, and Discomfort in Nursing 
Home Residents with Dementia: A Randomized, Controlled Trial. Jounal of the American 
Geriatrics Society. 2004;52(11):1795-1804  
61. 
Meyer G, Warnke A, Bender R, Muhlhauser I. Effect on hip fractures of increased use of 
hip protectors in nursing homes: cluster randomised controlled trial. Journal of the 
American Geriatrics Society. 2003;326(7380):76-  
62. 
Kemper S. Elderspeak: Speech accommodations to older adults. Aging and Cognition. 
1994;1(1):17-28  
63. 
Williams K, Kemper S, Hummert ML. Enhancing communication with older adults: 
Overcoming elderspeak. The Journal of Gerontological Nursing. 2004;30:1-10  
64. 
Banazak DA, Mickus M, Averill M, Colenda CC. Herding cats: Barriers to implementing a 
nurse aide educational program. Annals of Long-Term Care. 2000;8(10):68-71  
65. 
National Institue of Aging. Stage Model for Behavioral Intervention Development. 2016; 
https://www.nia.nih.gov/research/dbsr/stage-model-behavioral-intervention-development. 
Accessed September 9, 2016. 
66. 
Onken LS, Carroll KM, Shoham V, Cuthbert BN, Riddle M. Reenvisioning Clinical 
Science: Unifying the Discipline to Improve the Public Health. Clinical Psychological 
Science. 2014;2(1):22-34  
67. 
Stolee P, Esbaugh J, Aylward S, Cathers T, Harvey D, Hillier L, Keat N, Feightner J. 
Factors associated with the effectiveness of continuing education in long-term care. The 
Gerontologist. 2005;45(3):399-405  

     
 25 
68. 
Hirdes JP, Frijters DH, Teare GF. The MDS-CHESS Scale: A New Measure to Predict 
Mortality in Institutionalized Older People. Journal of the American Geriatrics Society. 
2003;51(1):96-100  
69. 
Morris JN, Fries BE, Mehr D. MS cognitive performance scale. Journal of Gerontology. 
1994;49:M174-182  
70. 
New England Quality Innovation Network-Quality Improvement Organization. 
Understanding the New MDS 3.0 Quality Measures. Centers for Medicare and Medicaid 
Services; Updated March 2016 2016. 
71. 
SAS [computer program]. Cary, North Carolina, USA: SAS Institute, Inc. 
72. 
Dawson JD. Stratification of summary statistic tests according to missing data patterns. 
Statistics in Medicine. 1994;13(18):1853-1863  
73. 
Brown H, Prescott R. Applied mixed models in medicine. 3 rd. ed. Hoboken, NJ: John 
Wiley & Sons, Ltd.; 2015. 
74. 
Molenberghs G, Kenward MG. Missing data in clinical studies. West Sussex, England: 
John Wiley & Sons, Ltd.; 2007. 
75. 
Allison PD. Handling Missing Data by Maximum Likelihood. Orlando Florida: SAS Global 
Forum; 2012. 
76. 
Demirtas H, Schafer JL. On the performance of random-coefficient patern-mixture models 
for non-ignorable drop-out. Statistics in Medicine. 2003;22:1553-2575  
77. 
Weiss RE. Modeling longitudinal data. New York; London: Springer; 2005. 
78. 
Westfall PH, Tobias RD, Wolfinger RD. Multiple comparisons and multiple tests using 
SAS (2nd ed.). Cary, NC: SAS Institute, Inc.; 2011. 2nd. ed. Cary, NC: SAS Intitute, Inc.; 
2011. 
79. 
Chen Y, Briesacher BA, Field TS, Tjia J, Lau DT, Gurwitz JH. Unexplained variation 
across us nursing homes in antipsychotic prescribing rates. Archives of Internal Medicine. 
2010;170(1):89-95  
80. 
Thompson Coon J, Abbott R, Rogers M, Whear R, Pearson S, Lang I, Cartmell N, Stein K. 
Interventions to reduce inappropriate prescribing of antipsychotic medications in people 
with dementia resident in care homes: a systematic review. J Am Med Dir Assoc. 
2014;15(10):706-718  
81. 
PASS15 Sample Size [computer program]. Kaysville, Utah, USA: NCSS Statistical 
Software; 2017. 
82. 
Williams K, Harris B, Ward K, Wassmer R, Weber A, Lueger A. Visual Cues for Person-
Centered Communication. Clinical Nursing Research. 2011;20(4):448-461.PMID: 
21828242. No PMCID. 
83. 
Williams K, Warren CAB. Communication in assisted living Journal of Aging Studies. 
2009;23(1):24-36.PMC2811280. 
84. 
Williams KN, Warren CAB. Assisted living and the aging trajectory. Journal of Women 
and Aging. 2008;20(3/4):309-327.PMCID 18983114. No PMCID. 
85. 
Efron B, Tibshirani RJ. An Introduction to the Bootstrap. New York: Chapman & Hall; 
1993. 
86. 
Polit DF, Beck CT. Nursing Research Principles and Methods. 7th edition ed: Lippincott, 
Williams, and Wilkins; 2007. 
87. 
Munro CL, Savel RH. Rigor and Reproducibility in Critical Care Research. American 
Journal of Critical Care. 2017;26(4):265-267  
88. 
Green LW, Glasgow RE. Evaluating the Relevance, Generalization, and Applicability of 
Research: Issues in External Validation and Translation Methodology. Evaluation & the 
Health Professions. 2006;29(1):126-153 

     
 26 
15 
APPENDIX 
15.1 Changing Talk Online (CHATO) Implementation Toolkit 
15.2 Changing Talk Online (CHATO) Training Overview 
15.3 CHATO Implemenation Strategies Survey 
15.3.1 Artifacts of Culture Change Survey 
15.4 CHATO Leadership Inverviews