Implementing and Evaluating Computer-Based Interventions for Mental Health (CDA 14-420)
Phase: NA
Conditions: Insomnia
Sponsor: VA Office of Research and Development

Version 4.0 30 Oct 2018 
Page 
1 
 
Project Description 
Last Update: 28 May 2019 
 
Implementing and Evaluating Computer-Based Interventions for Mental Health: Testing an 
Implementation Strategy for VA Outpatient Care 
 
NCT03151083 
 
 

Version 4.0 30 Oct 2018 
Page 
2 
 
1. Purpose: The purpose of this study is to evaluate the preliminary (1) effectiveness of an 
implementation strategy for Internet-based Self-help Interventions and (2) clinical outcomes of a 
specific Internet-based self-help program for insomnia, SHUTiTM, in VACT outpatient primary 
care. Internet-based self-help programs are personalized, self-guided interventions delivered 
over a computer, mobile device, or other Internet platform and focused on improving knowledge, 
awareness, or behavior change for a mental or physical health problem.1 Through our previous 
and current work at VACT (protocol EH0002), we are developing a general strategy for 
implementing Internet-based Self-help programs in VA primary care, specifically among Patient 
Aligned Care Teams (PACT) and Primary Care Mental Health Integration (PCMHI) teams. An 
implementation strategy is defined as a systematic intervention to integrate evidence-based 
health innovations into usual care.2 The strategy we propose to test consists of four core 
components: (1) a clinical intermediary for patient support, (2) provider/staff facilitation and 
education, (3) patient education, and (4) stepped-care for those requiring additional treatment. 
Our currently approved protocol (EH0002) involves interviewing VA providers, administrators, 
and staff in order to expand and modify the components of this strategy. In the proposed study, 
we will compare this strategy to an enhanced usual care (control) strategy with respect to the 
implementation related outcomes of patent engagement, provider adoption through referral to 
the program, and patient completion, over a eight-month active implementation period. There are 
evidence-based Internet-based self-help programs for many mental and behavioral health 
problems common in primary care such as depression, anxiety, problematic alcohol use, etc.  
We will target insomnia and use the SHUTiTM program, a 6-week self-guided program utilizing 
standard cognitive-behavioral therapy (CBT) techniques for the treatment of chronic insomnia. 
The program has been shown to improve insomnia severity and other sleep related outcomes in 
multiple randomized controlled trials.3-5 The clinical effectiveness of SHUTiTM will be evaluated 
using clinical insomnia outcomes obtained on all patients enrolled over the two 8-month active 
implementation periods.  
 
2. Background 
 
Insomnia Treatment: A Prototypical Gap in Access to Quality Mental Healthcare 
Insomnia is one of the most common mental health disorders among Veterans and is likely 
under-diagnosed in routine care.6 Most Veterans receive treatment for insomnia (and other common 
mental health conditions) in primary care, where psychotropic medications such as sedative-
hypnotics are the primary treatment.7,8 unfortunately, sedative-hypnotic medications have risks such 
as accidents, falls, and abuse potential. Medications are also of limited efficacy for the extended 
treatment of insomnia.9 Several non-pharmacologic evidence-based practices (EBPs), including CBT 
for insomnia, have equal or better effectiveness and far fewer risks.10,11 EBPs are generally 
underutilized, and engagement in and completion of EBP-driven treatments are limited within VA.  
 
VA Patient Aligned Care Teams and Primary Care Mental Health Integration 
One method to improve the deployment of EBPs is to leverage the use of Internet-based 
self-help interventions within VA PACT in cooperation with PCMHI providers. PACT/PCMHI teams 
distribute mental healthcare activities across a range of providers, including physicians, 
psychologists, nurses, care managers, and health technicians, who use a collaborative care model 
to treat a range of mental health conditions, most commonly mild to moderate depression, anxiety, 
and insomnia.  
 
Internet-Based Self-Help Interventions 
Internet-based interventions are personalized, self-guided programs delivered over a 
computer, mobile or other Internet access device and are focused on improving knowledge, 
awareness, or behavior change.1 Most Internet-based interventions are based on CBT principles 
and focused on fostering behavior change for a range of problems. Such programs present 

Version 4.0 30 Oct 2018 
Page 
3 
standardized yet individualized material in rich and varying formats, including audio, video, text, 
standardized symptom assessments, and graphical presentation of treatment response. Because 
patient interaction with these programs is relatively standardized through their format, Internet-based 
interventions provide a high degree of fidelity to psychotherapeutic EBP models. Many programs 
also include measurement tools, providing the opportunity to collect and monitor relevant outcome 
data. Most are designed around a “self-help” format and are accessed at a pace and in a setting 
chosen by the user, but are also available with varying levels of professional support.  
The potential health services benefits of Internet-based self-help interventions are extensive. 
They can (1) reduce travel barriers, providing treatment for individuals who cannot visit a clinic 
during business hours, (2) reduce barriers associated with the stigma of seeking mental health care 
and increase self-care, (3) increase clinic productivity, and (4) reduce costs associated with provider 
time.12-14 As a result, Congress mandated that VA implement Internet-based interventions through 
the Veterans’ Mental Health and Other Care Improvements Act of 2008.15 Evidence from systematic 
reviews indicates that Internet-based self-help programs on the whole are efficacious for a range of 
mental health conditions, and when compared with face-to-face therapy some programs may provide 
similar, yet more cost-effective care.16-19  
During the development of this project, VACT PACT/PCMHI leaders and providers indicated 
that they would like to target insomnia, one of the most common conditions encountered in primary 
care, for intervention via Internet-based self-help. Multiple Internet-based self-help programs for the 
treatment of insomnia have shown efficacy in randomized controlled trials. We will use the SHUTiTM 
program, based on its effectiveness in trials involving over 1500 individuals, in which it showed 
improvements in insomnia severity ratings, depression severity ratings, and sleep indices up to six 
months following program engagement.3-5   
 
Implementing Internet-based Self-help in VA Primary Care 
The development and testing of effective strategies to support the implementation of 
empirically supported Internet-based self-help programs in current outpatient treatment models is a 
novel area of research. The objective of this proposal is to evaluate the preliminary effectiveness of 
an implementation strategy for Internet-based self-help Interventions in VA Primary care using a 
specific Internet-based self-help program for insomnia. An implementation strategy is defined as a 
systematic intervention to integrate evidence-based health innovations into usual care.2 This strategy 
will provide guidance for the implementation of this novel intervention platform in a specific context, 
outpatient care in VA PACT/PCMHI.  
 
Past Pilot Study of Internet-based Self-Help At VACT 
 
In 2014, we completed a pilot feasibility study implementing Internet-based self-help in VA 
outpatient care.20 In this study an evidence-based program for insomnia (RESTORETM) was 
implemented in the VACT substance abuse clinic using provider and patient education sessions, on-
site Internet access, and clinician telephone support. The feasibility of this strategy was evaluated by 
assessing engagement/completion rates, participant reported acceptability, and clinical outcomes. 
Please see the abstract from the published article in the appendix of this proposal for the results of 
this study. These findings demonstrate the ability of our team to perform research on the 
implementation of Internet-based self-help in VA clinical care, as well as the feasibility of the strategy 
we propose in this study.  
 
3. Significance  
 
The proposed project has major innovations:  
1. The development of a general strategy for the implementation of evidence-based Internet-
based self-help programs in VA primary care, independent of the target condition, specific 
program, or treatment site is a novel undertaking.  
2. The testing of an implementation strategy for such interventions compared to an enhanced 
usual care strategy has not been attempted. Specific aspects of our proposed strategy have 

Version 4.0 30 Oct 2018 
Page 
4 
been tested, but none have codified core components into a defined strategy as they pertain 
to Internet-based self-help implementation.  
 
Positive benefits for Veterans and VA care include: 
1. Better access to effective mental health care, especially among Veterans from more recent 
conflicts who may find Internet-based self-help a more attractive treatment modality. 
2. Provision of mental health treatment earlier in the course of illness. 
3. Intervention-related improvements in health outcomes (insomnia severity). 
4. Greater self-care associated with the use of self-help programs. 
5. The ability to access care 24/7. 
6. Improved clinic productivity. 
7. Potentially reduced the risks associated with long-term use of sedative-hypnotic medications, 
such as benzodiazepines.  
 
In addition to providing a safer alternative for treating insomnia, the insights gained from developing 
and testing this implementation strategy will be transferable to other common mental health 
disorders for which other evidence-based self-help programs are currently available, such as 
depression, generalized anxiety, problematic alcohol use, smoking cessation, chronic pain, diabetes 
management, etc. 
 
4. Research Plan 
 
A. Study Design and Procedures 
 
Table 1. Timeline of Research Activities  
 
Year 1 
Year 2 
Year 3 
1 2 3 4 1 2 3 4 1 2 3 4 
Implement Program Using Enhanced Usual Care 
X X  
 
 
 
 
 
 
 
 
 
Finalization of Experimental Implementation Strategy 
 
 
X  
 
 
 
 
 
 
 
 
Identification and Training of Support Clinician 
 
 
X  
 
 
 
 
 
 
 
 
Provider/Staff Training for Experimental Strategy 
 
 
X  
 
 
 
 
 
 
 
 
Piloting in two individuals  
 
 
X X  
 
 
 
 
 
 
 
Implement SHUTi Using phase 1 Exp. Strategy 
 
 
 
X X  
 
 
 
 
 
 
Implement SHUTi using Phase 2 Exp. Strategy 
 
 
 
 
 
X X  
 
 
 
 
Analysis of Data from active Implementations 
 
 
 
 
 
X X X X  
 
 
 
As described by Curran et al. (2012), we will employ a hybrid type 3 implementation-
effectiveness study design through which we will primarily test the effectiveness of the 
experimental implementation strategy we are developing and secondarily evaluate the 
clinical effectiveness of SHUTiTM.21 We will use a quasi-experimental pre-/post-cohort design 
whereby the program will initially be implemented using an enhanced usual care (control) 
strategy, followed by implementation using experimental implementation strategies. The 
program’s association with clinical response will be evaluated in an uncontrolled pre-/post-
format. The timeline of events for this study is described in Table 1.  
 
SHUTiTM will initially be implemented using an enhanced usual care (control) strategy 
(described in detail below), over a eight-month period. For this strategy, VACT PACT/PCMHI 
providers will be given a 30-min education session on how to refer to and use SHUTiTM in 
clinical practice. This education will be followed by a eight-month period in which VACT 
PACT/PCMHI patients will be recruited to use SHUTiTM using this strategy. PACT/PCMHI 
providers will identify patients with insomnia who are appropriate for SHUTiTM, and recruit 

Version 4.0 30 Oct 2018 
Page 
5 
them through the normal course of clinical interactions. During recruitment, patients will be 
given a pamphlet with information about the SHUTiTM program including: educational 
information about insomnia and CBT treatment for insomnia, information about the content of 
the program, as well as website, login, privacy and provider contact information. Individuals 
will then meet with the research assistant for the application of final inclusion/exclusion 
criteria and research consent. Individuals who do not meet the inclusion/exclusion criteria will 
be referred back to the PACT/PCMHI provider. The research assistant will also provide a de-
identified user name for login, password resets, and will collect paper and pencil baseline 
measures. Subjects will then engage in the program at their own convenience. The research 
assistant will not provide program education or support directly to patients. Patients will be 
explicitly instructed on how to use regular PACT/PCMHI follow-up channels or the Veteran’s 
crisis line in the case of emergency, as they would do in the course of typical PACT/PCMHI 
treatment. Patients will be contacted by the research assistant after 10 weeks for the 
purpose of follow-up measure completion, which will take place either face-to-face or over 
the phone. Subjects will be compensated $10.00 for participation in each of two visits with 
the research team. See Table 2 for a full list and description of visits for individuals involved 
in the study.  
 
Table 2. Schedule of Subject Visits 
 
Phase 
Visit 
Visit description 
Context 
Location 
Duration (Min) Payment 
Enhance Usual Care (Control) 
Provider 
Initial 
Contact 
1. Identification of Insomnia 
2. Discussion of program with patient 
3. Provider gives pamphlet 
4. Provider/Patient Contacts Research Staff 
As part of regular treatment 
encounter 
VACT PACT Offices 
5-15 
None 
Baseline 
Visit with 
Research 
Personnel 
1. Application of Inclusion/Exclusion criteria 
2. Informed Consent 
3. Baseline measures 
4. Issue userID and temporary password 
Individual contacted by 
research assistant to 
coordinate in-person visit 
VACT PACT Offices 
10-25 
$10 
 
Provider 
Follow-up 
As need support contacts to address issues 
that arise, coordinated by provider/patient 
Provider: In-person or 
phone 
Phone or PACT 
Offices 
As needed 
None 
Research 
Follow-up 
Complete final follow-up measures 10-
weeks after baseline visit 
Research Assistant: In-
person or phone 
Phone or PACT 
Offices 
5-15 
$10 
Experimental Strategy (Phases 1&2) 
Provider 
Initial 
Contact 
1. Identification of Insomnia 
2. Discussion of program with patient 
3. Provider/patient contacts support 
clinician 
As part of regular treatment 
encounter 
VACT PACT Offices 
10-20 
None 
Support 
clinician 
Initial 
Contact 
1. Education session 
2. Initial tour of program 
3. Issue userID and temporary password 
Initial in-person education 
session 
VACT PACT Offices 
30-45 
None 
Baseline 
Visit, 
Research 
Personnel 
1. Application of Inclusion/Exclusion criteria 
2. Informed Consent 
3. Baseline measures 
 
Research assistant 
contacted by support 
clinician. Visit takes place 
prior to support clinician 
visit 
VACT PACT Offices 
10-20 
$10 
Support 
Clinician 
Follow-up 
Scheduled and as needed support contacts 
to address issues that arise. Coordinated 
by support clinician and patient 
Support clinician: In-person 
or phone 
Phone or PACT 
Offices 
As needed 
none 
Research 
Follow-up 
Complete final Follow-up measures 10-
weeks after baseline visit. 
Research assistant: In-
person or phone 
Phone or PACT 
Offices 
10-15 
$10 
 
After the initial 8-month testing period, referrals to SHUTiTM in VACT PACT/PCMHI will stop 
for three to four months. During this critical period, several activities will take place:  

Version 4.0 30 Oct 2018 
Page 
6 
(1) Focus groups consisting if VACT PACT/PCMHI staff will take place, as described in our 
other protocol (EH0002). 
(2) Final specific modifications to the experimental implementation strategy using information 
from interviews with VA-wide providers and administrators as well as the focus groups. (A 
final description of the implementation strategy will be supplied to the VACT HSC via a 
protocol amendment during this period.) 
(3) Identification and training of the support clinician (described in the experimental 
implementation strategy below). 
(4) Provider education for the experimental strategy (described in the experimental 
implementation strategy below). 
(5) Pre-piloting of the SHUTiTM program with one-two individuals using this strategy. 
 
After this period, patients will be recruited to participate in SHUTiTM using the experimental 
implementation strategy over a period of eight months. The experimental implementation 
strategy is described at length in the intervention section below. First, a provider education 
session will take place. Then over a 8-month period, providers will identify and recruit 
PACT/PCMHI patients to participate in the program. Providers will refer patients they recruit 
to the support clinician for an initial education session. The support clinician will also provide 
patients with user names, password resets, and privacy information. As part of this initial 
session, research staff will meet with the patient and apply final inclusion/exclusion criteria, 
consent individuals, and complete baseline measures. Subjects will then engage in the 
program at their own convenience with support from the support clinician. The research 
assistant will not provide program education or support directly to the patient. Patients will be 
instructed on how to use the support clinician, regular PACT/PCMHI follow-up channels, or 
the Veteran’s crisis line in the case of emergency or with any questions about the program. 
Patients will be contacted by the support clinician over the telephone at regular intervals for 
check in. Patients will be contacted by the research assistant after 10 weeks for the purpose 
of follow up measure completion which will happen either face-to-face or over the phone. 
Subjects will be compensated $10.00 for participation in each of two visits with the research 
team. See Table 2 for a full list and description of visits for individuals involved in the study. 
 
After the first experimental implementation period, patients will be recruited to participate in 
SHUTiTM using a modified experimental implementation strategy over another period of 8 
months, the phase 2 experimental strategy. No changes to the core components of the 
experimental implementation strategy will be made (table 2), only to the individuals who will 
provide the clinical support, who will be psychologists, nurses, and peers in the West Haven 
VA PCMHI clinic. Providers will continue to identify and recruit PACT/PCMHI patients to 
participate in the program and refer patients they recruit to the support clinician for an initial 
education session. Referrals will be made through a modification to the PCMHI consult 
request form in CPRS. Research staff will still apply the same final inclusion/exclusion 
criteria, review project information documents, and complete baseline measures. Subjects 
will engage in the program at their own convenience with support from the support clinician. 
As in the other phases, subjects will be instructed on how to use the support clinician, regular 
PACT/PCMHI follow-up channels, or the Veteran’s crisis line in the case of emergency or 
with any questions about the program. Patients will still be contacted by the support clinician 
over the telephone at regular intervals for check in. Patients will be contacted by the research 
assistant after 10 weeks for the purpose of follow up measure completion which will happen 
either face-to-face or over the phone. Subjects will be compensated $10.00 for participation 
in each of two visits with the research team.  
 
B. Subjects. Participants will include individuals from three different groups: West Haven VACT 
PACT/PCMHI providers and staff, a clinical intermediary for patient support who will be a 
VACT PACT/PCMHI staff member, and West Haven VACT PACT/PCMHI patients. Please 

Version 4.0 30 Oct 2018 
Page 
7 
see the appendix and Initial review application for a description of the Subject sample, roles, 
recruitment, inclusion/exclusion criteria, and consent procedures. Roles, recruitment, 
inclusion/exclusion criteria, and consent procedures will remain the same for the additional 
phase 2 experimental implementation strategy.  
 
C. Interventions  
 
The Experimental Implementation Strategy. Components of the experimental implementation 
strategy consists of (1) a clinical intermediary for patient support, (2) provider/staff facilitation 
and education to empower adoption, (3) patient education to facilitate engagement and 
completion, and (4) a process of stepped-care to identify and refer individuals requiring a 
higher level of care. The components of this strategy, especially the educational content, will 
be expanded and modified using results from our ongoing work in protocol EH0002. 
 
1. Clinical Intermediary for Participant Support (Support Clinician) 
The support clinician will accept provider referrals to the program and provide support to 
patients as they engage in the Internet-based self-help program. This position is consistent 
with research showing that engagement and completion with Internet-based self-help 
programs is higher when such programs are supported,16,17,19,22 the well-known impact of 
collaborative care models on the treatment of mental health disorders in primary care,23 as 
well as implementation research recognizing the need for “Technical Assistance” and 
“Program Champions” who can provide direct support.24  
The support clinician will have six duties: (1) interacting with providers to facilitate 
referrals, (2) performing face-to-face introductory patient education sessions, (3) encouraging 
program engagement and completion, (4) resolving technical problems such as log-on issues 
and password loss, (5) resolving clinical questions associated with insomnia treatment, and 
(6) identifying individuals who require elevated levels of care. In our prior work, 96% of 
support contacts were for encouragement, 3% for technical issues, and 1% for insomnia 
treatment-related questions, with no emergent clinical issues identified.25 Support will be 
provided during initial contact, by phone at regular intervals, as well as through as needed 
phone and face-to-face contact. All patient interaction will be logged, using de-identified 
research numbers, in the treatment log kept by the support clinician and documented in the 
medical record. The training and facilitation provided to the support clinician is described 
below, and fidelity to support plans will be monitored as described below. 
For this study, a current PACT/PCMHI staff member, such as a nurse, case manager, 
tele-case-management clinician, health-technician, or peer support individual will be 
identified in the 3-4 month period prior to implementation using the experimental strategy 
through focus groups described in the current VACT HSC protocol (EH0002) and discussion 
with VACT PACT/PCMHI and case management leadership. Such leaders (Drs. Chris Ruser, 
Kirsten Wilkins, and Donna Vogel) have already agreed to carve out four hours per week in 
this person’s schedule to support this work (see support letter). When selected, this person 
will be identified to the VACT HSC through an amendment to this protocol.  
 
For the second phase of the experimental implementation strategy, Mr. Battagliotti will 
no longer be providing clinical support duties. We have identified individuals in the PCMHI 
clinical group as being willing to perform SHUTi clinical support tasks. We had previously 
identified a peer support specialist within PCMHI, Mr. Greg Dante, to share “internal 
champion” duties with Mr. Battagliotti. Both Anne Klee and Nicole Campbell, leasers of the 
Peer Support program at West Haven, and Lisa Osborne, Mr. Dante’s supervisor, have 
agreed to allow him to take part. Lisa Osborne will continue to supervise his clinical duties 

Version 4.0 30 Oct 2018 
Page 
8 
while he acts as internal champion for approximately one hour per week. Additional members 
of the PCMHI team, psychologists, psychiatrists, and the nurse care manager within PCMHI 
have agreed to take on these duties as well.  
 
These specific activities are as follows:  
A. Provide academic detailing: PCMHI personnel will liaise with providers in individual 
meetings, team meetings, and larger meetings to educate them on program content 
and the referral process.  
B. Patient education: PCMHI personnel will discuss the availability of the program with 
patients during the normal course of his clinical duties. Referrals to the program will 
come from providers as per the original project description. Once patients are referred, 
PRMHI personnel will discuss the content of the program with them and answer 
questions. Per the original project description, the research team will provide informed 
consent and access codes to the program.  
C. Patient motivational enhancement and educational support for engagement and 
completion: After subjects have been consented, PCMHI personnel will provide follow-
up phone (and as needed face-to-face) contact. He will encourage patients to engage 
in the program and answer program related questions that are within his skill level. 
Clinical questions or safety concerns during these contacts which are beyond his skill 
level will be referred to the appropriate clinician, in accordance with current clinic 
protocol.   
D. Using VA clinical quality improvement databases to identify patients who may be 
appropriate for the program: The VA Central Office, Offices of Academic Detailing and 
the Psychotropic Drug Safety Initiative have developed informatics dashboards which 
identify patients who are treated with recurrent prescriptions of sedative-hypnotics 
(https://vaww.portal2.va.gov/sites/ad/SitePages/Chronic%20Insomnia.aspx). The 
dashboards can be searched by facility, clinic, team and provider to identify patients 
who may be appropriate for evidence-based non-pharmacologic treatment of insomnia 
(such as the SHUTiTM program).  VACT academic detailing and PCMHI personnel will 
use this dashboard to identify patients who may benefit from non-pharmacologic 
treatment of insomnia. Patients identified through these dashboards will be mailed a 
letter describing the reason they are being contacted, the potential benefits of 
behavioral treatment for insomnia, and the treatment options (either face-to-face or 
online versions of this treatment). The letter will include the name and contact 
information of the peer support specialist, Greg Dante, who will be able to describe 
behavioral treatment for insomnia, answer questions, and mediate referrals to 
treatment through providers, if the patient is interested. The SHUTi information 
brochure, will be included with the letter. This brochure has information about 
insomnia, behavioral treatment, and the SHUTi program, as well as Greg Dante’s 
contact information. 
E. Audit and feedback to support provider champions within the clinic. We will 
identify the WHVA primary care providers who have made the most referrals to 
the program since it began and identify the outcomes of the patients these 
providers referred. Three providers will be identified in Firms A and B, 
respectively, and one in the Woman’s clinic. The following outcomes will be 
identified:  (a) the number of referrals they have made to the program, (b) the 
number of those referred who went on to engage, (c) the number of those 
referred who completed the program,  and (d) the change in insomnia level and 
sedative hypnotic use after using the program in those who engaged and 

Version 4.0 30 Oct 2018 
Page 
9 
completed the program. This information will be given to the respective 
providers in a deidentified and pooled format in the form of an encrypted email 
(using the WHVA outlook encryption system) and face to face discussions. No 
patient personally identifying information will be shared with the provider. The 
aim will be to recruit providers to become program champions, communicating 
the success they have seen with the program to other providers in order to get 
more patients referred.   
 
As an additional point of clarification, the above activities are clinical activities related 
to the care of patients within the primary care clinic in West Haven. Research activities will 
remain the same as described in the original project description.   
A. The research team will educate the PCMHI team as to aspects of the SHUTiTM 
program allowing them to perform the provider academic detailing, patient 
education, and patient support contacts. 
B. The research team will continue to perform informed consent on those referred to 
the program as described in the original project description. 
C. The research team will educate the PCMHI team as to aspects of the VACO 
insomnia dashboard in order to allow him to identify WHVA primary care patients 
with insomnia.  
D. The research team will have weekly meetings with the PCMHI team to provide 
coaching, answer questions, and track activities. 
 
2. Provider/Staff Facilitation and Education 
PACT/PCMHI providers will be given a 30-minute educational session by study 
personnel, which will include a discussion of (1) insomnia prevalence, diagnosis, and 
treatment options; (2) efficacy and content of CBT for insomnia and specifically the SHUTiTM 
program; (3) program referral; and (4) the role of the support clinician and the shared 
responsibility for identifying those who require a step-up in care. The use of this program as 
part of a menu of treatment options for insomnia and as a potential alternative to long-term 
sedative-hypnotic use will also be emphasized. These sessions will take place during 
regularly scheduled PACT/PCMHI meetings. The support clinician will receive two separate 
hour-long educational sessions focusing on training for the key duties described above. The 
content of these sessions will be developed in collaboration with Dr. DeViva and influenced 
by findings from our ongoing qualitative work (HSC EH0002). To facilitate implementation 
during the experimental period, research staff will be available during regularly scheduled 
PACT/PCMHI meetings and day-to-day (in person or by telephone) to answer questions and 
troubleshoot problems. The support clinician will meet with research staff bi-weekly to review 
progress and troubleshoot. 
 
3. Patient Education and Facilitation 
 
Patients enrolled in the program during the experimental implementation phase will 
be provided an initial short education session with the support clinician in the experimental 
phases. This session will allow for relationship development and is hypothesized to be 
essential to support patient engagement. The session will include information on: (1) CBT 
concepts; (2) SHUTiTM program content and components; (3) program access (website, 
username, passwords, privacy, etc.); and (4) access to clinical support for the program 

Version 4.0 30 Oct 2018 
Page 
10 
during participation. The content of these sessions will be developed in collaboration with Dr. 
DeViva and influenced by findings from ongoing work (EH0002).  
 
4. Stepped-Care 
Stepped-care refers to adjusting the intensity of treatment in stages according to 
factors such as changes in symptomatology or lack of engagement or efficacy. In short, 
patients who fail a less intense level of treatment should be offer progressively more intense 
treatment services. This approach is derived from the extensive literature available on the 
efficacy of collaborative care approaches for mental health treatment in primary care, which 
forms the basis of the PACT/PCMHI teams and has been used for the implementation of 
Internet-based self-help interventions within the U.K.’s National Health Service.66-68  
From our initial feasibility work25 and our continued work as part of HSC protocol 
EH0002, we have noted three general categories of clinical issues during the course of 
Internet-based self-help program participation that would trigger a change in care: (1) 
consideration of dropping out related to a lack of efficacy, dissatisfaction, or competing 
demands; (2) new psychiatric and/or medical problem interfering with participation; and (3) a 
clinical question related to participation in Internet-based self-help such as how to manage 
specific stimulus control or sleep timing issues. The support clinician will monitor participants 
for these issues as part of phone contact with participants and will refer participants 
accordingly. Referrals may include time-limited consultation with PCMHI providers 
(psychologists) to address a low-intensity problem or referral back to PACT/PCMHI providers 
after treatment failure, drop out, or identification of additional clinical concerns. These 
processes will be further developed in our current study (EH0002) and consultation with Dr. 
DeViva, an expert in CBT for insomnia. 
 
Enhanced Usual Care (Control) Implementation Strategy. Currently, Internet-based self-help 
interventions are not widely utilized in U.S. healthcare organizations, including VACT. This 
fact prohibits a “treatment as usual” competitor developed from the literature. Therefore, we 
developed a “low-intensity” enhanced usual care strategy, which minimizes aspects of the 
four core components of the experimental strategy, to serve as a control strategy.  This 
strategy can also be thought of as “enhanced usual care,” because it represents the process 
by which Internet-based self-help programs can be implemented using a method with the 
lowest logistical impact on the current PACT/PCMHI treatment process.  
The enhanced usual care strategy will consist of (1) a 30-minute introductory session 
with providers to explain the SHUTiTM program and how it will be used; (2) provider use of a 
referral pamphlet that includes website, login, and clinic contact information for patients; and 
(3) clinical support provided by PACT/PCMHI providers. Research staff will consent patients, 
provide log-in information/password resets, and collect measures only. They will not provide 
program education or support. Patients will be explicitly instructed on how to use regular 
PACT/PCMHI follow-up channels or the Veteran’s crisis line in the case of emergency, as 
they would do in the course of typical PACT/PCMHI treatment.  
 
Internet-based Self-help Insomnia Intervention. We will use the program, SHUTiTM for the 
treatment of insomnia. SHUTiTM was developed by Drs. Lee Ritterband, Francis Thorndike, 
and Charles Morin. It is currently openly available for a fee as a self-help program at 
http://www.myshuti.com/ and marketed for healthcare delivery by BeHealth Solutions, LLC. 
The intervention is a CBT-based 6-week, self-administered course accessed via the Internet 
on mobile, desk-top, and other devices. The program is split into six modules, completed 
weekly, which include instruction on psycho-education, stimulus control, relaxation training, 

Version 4.0 30 Oct 2018 
Page 
11 
sleep restriction, medication tapering, and addressing cognitive distortions. The content is 
delivered via text, video vignettes, case histories, interactive learning tools, interactive skills 
assessments, symptom assessments, and a sleep log. Homework is assigned after each 
module. Providers and support staff can follow treatment progress via access to a dashboard 
of patient information collected by the program. Such information includes progress through 
the modules, date of last engagement, symptom assessment outcomes, and sleep log data 
abstracted into commonly used sleep indices such as sleep onset latency and sleep 
efficiency.  
 
SHUTiTM has been empirically validated across four randomized and controlled trials. In 
addition to the specific trials listed below, there are four other ongoing International trials 
involving the program.  
(1) NIH-R34 Pilot. This project studied 45 adults with chronic insomnia. After 8 weeks and 
again at eight months following the intervention, SHUTiTM users demonstrated statistically 
significant sleep improvements (insomnia severity, sleep efficiency, and wake after sleep 
onset) compared to wait-list controls.26  
(2) Study of Cancer Survivors. This project studied 28 adult cancer survivors with chronic 
insomnia. After 8 weeks, SHUTiTM users demonstrated statistically significant 
improvements in insomnia severity, sleep onset latency, and sleep efficiency compared 
to wait-list controls.4 
(3) Australian Study Evaluating Depression Prevention. This project studied 1149 Australians 
with depression and insomnia recruited over the Internet. Compared to an Internet-based 
information control group, SHUTiTM users demonstrated lowered depression symptoms 
on the PHQ-9 and improved insomnia severity at 6 weeks and 6 months compared with 
the control group.5 
(4) NIH ROI Study (Currently Under Review). This project is studying 303 adults with chronic 
insomnia randomly assigned to either SHUTiTM or an Internet-based education control. 
Recent discussion with investigators from this study indicated that SHUTiTM users 
demonstrated improved insomnia severity, sleep onset latency, and wake time after sleep 
onset at 6 weeks and one year after participation. Within group effect sizes show that the 
magnitude of the improvements from baseline to post assessment were medium to large 
for the SHUTiTM group. This study is currently under review for publication.  
 
Although SHUTiTM is a commercial product, the program is similar to programs for 
insomnia currently under development by the VA. We have extensive experience using a 
different but similar Internet-based self-help program for insomnia, RESTORETM, in our pilot 
study.25 We have chosen SHUTiTM for this study because its superior evidence base as well 
as functionality and content. In the execution of our pilot project and the development of this 
proposal, we have been in contact with Dr. Carolyn Greene, VA National Program Manager 
for Web Services, regarding the status of VA sponsored Internet-based self-help programs 
for insomnia. Currently, a program is being efficacy-tested while funds for the development of 
another are still waiting to be allocated. As VA-developed Internet-based self-help programs 
become available and show efficacy, the implementation strategy described here will easily 
apply to these VA-developed products.  
We currently have a verbal agreement with the CEO of BeHealth Solutions, LLC to 
provide the SHUTi program for the purposes of this study. BeHealth Solutions, LLC has also 
agreed to supply de-identified data which participants will input during their interaction with 
the program.  We will sign a letter of agreement with the CEO of BeHealth Solutions, LLC 
which documents this agreement once the VA Connecticut IRB has reviewed this protocol.  
 
D. Outcomes. This is a hybrid implementation-effectiveness study. Therefore, summative 
implementation, clinical, and implementation process outcomes will be evaluated. Please see 
the appendix to this application for a full description of the outcomes that will be assessed.  

Version 4.0 30 Oct 2018 
Page 
12 
a. Summative Implementation Outcomes. For the two 8-month control and experimental 
strategy implementation periods, data will be collected by a research assistant who will 
not play a role in intervention delivery.  
i. Program Engagement (Primary Outcome). Engagement will be defined in two ways in 
order to encompass the breadth of the construct: (1) completion of the first module of 
the SHUTiTM program (binary) and (2) the number of modules completed (counts). 
ii. Program Completion (Secondary Outcome). Completion of all six modules of the 
program will serve as a secondary outcome (binary). 
iii. Provider Adoption (Secondary Outcome). This measure will be operationalized as a 
PACT/PCMHI provider referring a patient with insomnia to the program. In the 
enhanced usual care (control) implementation condition, the referral will be to the 
research assistant for the purposes of informed consent and assessments. In the 
experimental implementation strategy condition, referral will be to the support clinician. 
The support clinician will then perform the education session and involve the research 
assistant to obtain written informed consent and assessments. Referral will be logged 
by the research assistant in the enhanced usual care condition. The support clinician 
in the experimental condition will capture this outcome. 
iv. Exploratory Implementation Outcomes. SHUTiTM also allows the tracking of homework 
completion and para-data (screens viewed, log-in frequency, etc.). This data will be 
used to explore the level of interaction with the program and whether there are 
associations between program interaction and clinical outcomes. 
b. Insomnia Treatment Outcomes. Clinical outcomes will be collected by the research 
assistant via structured assessments at baseline (following informed consent) and at 
follow-up (10-weeks after baseline) during the two eight-month implementation periods. 
Follow-up measures will be completed during face-to-face meetings by all participants, 
including those who do not engage or drop out prior to completion. 
i. The Insomnia Severity Index (ISI). The ISI will be the primary clinical outcome and is a 
self-report seven-item measure that targets the subjective symptoms and functional 
consequences of insomnia.27,28 
ii. Beck Depression Inventory (BDI). The BDI II is a self-report measure of 
depression.29,30  
iii. Sedative-Hypnotic Medication Use. The use of sedative-hypnotics will be measured by 
self-report of the type and dose of specific medications used in the last week.  
iv. Sleep Indices. Change in sleep indices (total sleep time, sleep onset latency, sleep 
efficiency, wake time after sleep onset, and nocturnal awakenings) will be measured in 
sleep logs, which are integral to the SHUTiTM program. Such measures can only be 
completed by those who engage in the program and, therefore, cannot be used as the 
primary clinical outcome.  
c. Implementation Process Outcomes. Implementation process outcomes will have two 
functions: (1) to identify mediators of summative outcomes and (2) to evaluate fidelity to 
the experimental implementation strategy. The measures used in this project are 
developed from validated measures of technology acceptance and use, derived from the 
Unified Theory or the Use and Acceptance of Technology (UTAUT).31  
i. Patients. Patients will complete implementation process outcomes during the control 
and experimental implementation conditions at the pre- and post-intervention 
assessment time points. 
ii. Providers and staff. Providers will complete process measures (1) prior to the initial 
enhanced usual care implementation phase, (2) following the enhanced usual care 
implementation phase, and (3) following the experimental implementation. 
iii. Support Clinician. The support clinician’s fidelity to the experimental implementation 
strategy will be measured during the 8-month experimental implementation period by 
data obtained from the support clinician’s clinical log. This log will track interactions the 
support clinician has with PACT/PCMHI patients referred to the program. This log will 

Version 4.0 30 Oct 2018 
Page 
13 
track patients via user name and will not contain PHI. Information tracked will include: 
date of referral, date of education session, date of support contacts, date of completion, 
date patient indicated he or she were dropping out, last module and date completed, 
and whether the individual required a step-up in care. Specific measures developed 
from this log will include: (a) The proportion of individuals referred who attend the initial 
education session with the support clinician, (b) The proportion of individuals referred 
who receive weekly phone calls during their participation, (c) The number and content 
of clinical and technical help sessions with the support clinician during participation. 
 
E. Data Analysis. Please see the appendix for a full discussion of the statistical analysis plan.  
a. Summative Implementation Outcomes. For the three active 8-month implementation 
periods, four separate but related dependent variables representing both the patient and 
provider levels of implementation will be analyzed. Summative implementation outcomes 
will be analyzed by comparing the two experimental implementation periods with the 
enhanced usual control period. Co-primary outcomes will include: (1) the proportion of 
participants engaging in the program (completing at least one module) among the unique 
Veterans treated in VACT primary care over the respective time periods; (2) the mean 
number of modules completed among all participants enrolled (Poisson mean as the 
distribution is expected to be non-normal, heavy skewing with large numbers of zeroes 
and ones).Secondary outcomes will include: (1) The proportion of individuals completing 
all six modules among all individuals enrolled. (2) The proportion referred to the program 
(whether they engaged or not) among all unique Veterans treated in VACT primary care 
over the respective time periods. 
b. Clinical Outcomes. Pre- and post-intervention total ISI scores obtained for all participants 
in each implementation period will be calculated and treated as a continuous variable for 
the primary dependent variable of the clinical outcome. A 7-point change pre-post will be 
considered clinically meaningful. A secondary dependent variable representing sedative-
hypnotic use will be calculated as the mean number of total weekly sedative-hypnotic 
doses. Exploratory continuous variables representing sleep indices developed from sleep 
logs internal to the SHUTiTM program will also be analyzed, but only in participants who 
engage in the program. In exploratory analyses, clinical outcome data will be evaluated 
by clinical and demographic data extracted from the just, such as medical and mental 
health diagnoses as well as prescribed medications.  
c. Implementation Process-Level Outcomes 
i. Patients. Process-level outcomes will be analyzed as ordinal variables in frequencies 
and proportions based on Likert responses. Responses will be compared between the 
enhanced usual care and the two experimental conditions using chi-square tests. The 
association between process-level and primary summative implementation outcomes 
will be analyzed using chi-square tests and multivariable  
ii. Providers. Process-level outcomes at the provider level will be analyzed as ordinal 
variables in frequencies and proportions based on Likert responses. For providers, the 
change in responses over three periods (prior to the enhanced usual care 
implementation, following the control phase, and following the 8-month active 
facilitation of the experimental Implementation) will be analyzed using Cochran-
Mantel-Haenszel and GEE marginal models. The association between changes in 
process-level provider outcomes and provider referral will be analyzed using chi-
square. 
iii. Support Clinician. Fidelity of the support clinician to the experimental implementation 
strategy will be analyzed as the proportion of patients (1) who attend the initial 
education session among those referred, and (2) who receive weekly phone calls 
during the experimental implementation phase among enrollees. Bivariate 
associations between these proportions and key patient level outcomes as well as 
baseline characteristics will be analyzed.  

Version 4.0 30 Oct 2018 
Page 
14 
 
5. Other Information 
 
A. Study Location. This study will be conducted in the VACT (West Haven) PACT/PCMHI 
clinics. The intervention will be accessed via the Internet using a computer or other device of 
the participant’s choosing at a time and in a location of their choosing. Research records will 
be located in the PI’s office. The PI’s office is located in the New England MIRECC offices 
located in Building 35, lower level. 
 
B. Payment. Subjects who are VACT PACT/PCMHI patients will be paid $10.00 for their 
participation in each baseline and follow up meeting with research personnel, but not for their 
interactions with PACT/PCMHI providers or staff such as the support clinician (See Table 2). 
PACT/PCMHI providers and the support clinician will not be compensated for their 
participation.  
 
C. Funding Source. VA Office of Research and Development, HSR&D, via a carrier 
development award to the PI, Eric Hermes, MD, as well as New England Mental Illness 
Research, Education, and Clinical Center (MIRECC) and/or funds from the VACREF. 
 
D. Duration. The expected duration is 5 years.  
 
6. References 
1. 
Marks IM, Cavanagh K, Gega L. Hands-on help: Computer-aided psychotherapy. 
Psychology Press; 2007. 
2. 
Powell BJ, McMillen JC, Proctor EK, et al. A compilation of strategies for implementing 
clinical innovations in health and mental health. Medical Care Research and Review. 
2012;69(2):123-157. 
3. 
Ritterband LM, Thorndike FP, Gonder-Frederick LA, et al. Efficacy of an Internet-based 
behavioral intervention for adults with insomnia. Archives of general psychiatry. 
2009;66(7):692-698. 
4. 
Ritterband LM, Bailey ET, Thorndike FP, Lord HR, Farrell-Carnahan L, Baum LD. Initial 
evaluation of an Internet intervention to improve the sleep of cancer survivors with insomnia. 
Psycho-Oncology. 2012;21(7):695-705. 
5. 
Christensen H, Batterham PJ, Gosling JA, et al. Effectiveness of an online insomnia program 
(SHUTi) for prevention of depressive episodes (the GoodNight Study): a randomised 
controlled trial. The Lancet Psychiatry. 2016;3(4):333-341. 
6. 
Hermes ED, Rosenheck RA. Prevalence, Pharmacotherapy, and Clinical Correlates of 
Diagnosed Insomnia Among Veterans Health Administration Service Users Nationally. Sleep 
medicine. 2014. 
7. 
Hermes E, Sernyak M, Rosenheck R. Use of second-generation antipsychotic agents for 
sleep and sedation: a provider survey. Sleep. 2012;36(4):597-600. 
8. 
Hermes E, Sernyak M, Rosenheck R. The use of second generation antipsychotics for post-
traumatic stress disorder in a US Veterans Health Administration Medical Center. 
Epidemiology and psychiatric sciences. 2013:1-8. 
9. 
Glass J, Lanctôt KL, Herrmann N, Sproule BA, Busto UE. Sedative hypnotics in older people 
with insomnia: meta-analysis of risks and benefits. Bmj. 2005;331(7526):1169. 
10. 
Morgenthaler T, Kramer M, Alessi C, et al. Practice parameters for the psychological and 
behavioral treatment of insomnia: an update. An American Academy of Sleep Medicine 
report. SLEEP-NEW YORK THEN WESTCHESTER-. 2006;29(11):1415. 
11. 
Morin CM, Hauri PJ, Espie CA, Spielman AJ, Buysse DJ, Bootzin RR. Nonpharmacologic 
treatment of chronic insomnia. An American Academy of Sleep Medicine review. Sleep. 
1999;22(8):1134-1156. 

Version 4.0 30 Oct 2018 
Page 
15 
12. 
Cucciare MA, Weingardt KR, Humphreys K. How Internet technology can improve the quality 
of care for substance use disorders. Current Drug Abuse Reviews. 2009;2(3):256-262. 
13. 
Berger M, Wagner TH, Baker LC. Internet use and stigmatized illness. Social science & 
medicine. 2005;61(8):1821-1827. 
14. 
McCrone P, Knapp M, Proudfoot J, et al. Cost-effectiveness of computerised cognitive–
behavioural therapy for anxiety and depression in primary care: randomised controlled trial. 
The British Journal of Psychiatry. 2004;185(1):55-62. 
15. 
Carroll KM, Ball SA, Martino S, et al. Computer-assisted delivery of cognitive-behavioral 
therapy for addiction: a randomized trial of CBT4CBT. The American journal of psychiatry. 
2008;165(7):881. 
16. 
Griffiths KM, Christensen H. Review of randomised controlled trials of Internet interventions 
for mental disorders and related conditions. Clinical Psychologist. 2006;10(1):16-29. 
17. 
Kaltenthaler E, Sutcliffe P, Parry G, Beverley C, Rees A, Ferriter M. The acceptability to 
patients of computerized cognitive behaviour therapy for depression: a systematic review. 
Psychological medicine. 2008;38(11):1521. 
18. 
Wantland DJ, Portillo CJ, Holzemer WL, Slaughter R, McGhee EM. The effectiveness of 
Web-based vs. non-Web-based interventions: a meta-analysis of behavioral change 
outcomes. Journal of medical Internet research. 2004;6(4). 
19. 
Cavanagh K, Shapiro D, Berg S, Swain S, Barkham M, Proudfoot J. The effectiveness of 
computerized cognitive behavioural therapy in routine care. British Journal of Clinical 
Psychology. 2006;45(4):499-514. 
20. 
Hermes E, Rosenheck R. Implementing Computer-Based Psychotherapy Among Veterans in 
Outpatient Treatment Using a Supported Strategy. Psychiatric Services. 2015;In press. 
21. 
Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid 
designs: combining elements of clinical effectiveness and implementation research to 
enhance public health impact. Medical care. 2012;50(3):217. 
22. 
Foroushani PS, Schneider J, Assareh N. Meta-review of the effectiveness of computerised 
CBT in treating depression. BMC psychiatry. 2011;11(1):131. 
23. 
Hedrick SC, Chaney EF, Felker B, et al. Effectiveness of collaborative care depression 
treatment in Veterans' Affairs primary care. Journal of General Internal Medicine. 
2003;18(1):9-16. 
24. 
Kilbourne AM, Neumann MS, Pincus HA, Bauer MS, Stall R. Implementing evidence-based 
interventions in health care: application of the replicating effective programs framework. 
Implementation Science. 2007;2(1):42. 
25. 
Hermes ED, Rosenheck RA. Implementing Computer-Based Psychotherapy Among 
Veterans in Outpatient Treatment for Substance Use Disorders. Psychiatric Services. 2015. 
26. 
Ritterband LM, Thorndike FP, Gonder-Frederick LA, et al. Efficacy of an Internet-based 
behavioral intervention for adults with insomnia. Archives of general psychiatry. 
2009;66(7):692-698. 
27. 
Morin CM, Belleville G, Bélanger L, Ivers H. The insomnia severity index: psychometric 
indicators to detect insomnia cases and evaluate treatment response. Sleep. 2011;34(5):601. 
28. 
Bastien CH, Vallières A, Morin CM. Validation of the Insomnia Severity Index as an outcome 
measure for insomnia research. Sleep medicine. 2001;2(4):297-307. 
29. 
Beck AT, Steer RA, Brown GK. Beck depression inventory-II. San Antonio. 1996;78(2):490-
498. 
30. 
Dozois DJ, Dobson KS, Ahnberg JL. A psychometric evaluation of the Beck Depression 
Inventory–II. Psychological assessment. 1998;10(2):83. 
31. 
Duyck P, Pynoo B, Devolder P, Voet T, Adang L, Vercruysse J. User acceptance of a picture 
archiving and communication system. Applying the unified theory of acceptance and use of 
technology in a radiological setting. Methods of information in medicine. 2007;47(2):149-156.