Remotely-Delivered Benefits Counseling for Service Connection Applicants
Phase: NA
Conditions: Job Satisfaction, Unemployment
Sponsor: VA Office of Research and Development

Remotely-Delivered Benefits Counseling for Service 
Connection Applicants 
NCT02677428 
4/9/19 
 
 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
2 
 
 
Project Description 
 
1. Principal Investigator 
Marc Rosen, MD  
 
2. Project Title 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
 
3. Purpose 
The purpose of this study is to evaluate the utility of Remotely-Delivered Benefits Counseling in 
a 24-week clinical trial. This study will evaluate the effectiveness of Benefits Counseling on 
work and work-related outcomes with Veterans who are applying for service-connection. This 
study has the following specific aims and hypotheses: 
 
Primary Aim 1: To estimate the effect of Remotely-Delivered Benefits Counseling on the slopes 
of paid work over time.  
Hypothesis: The slopes of hours of paid work over time will differ between the control 
and Benefits Counseling groups, as evidenced by a significant, positive treatment-by-time 
interaction effect. 
 
Exploratory Aim One: To estimate the effect of Remotely-Delivered Benefits Counseling on 
other outcomes. 
Hypothesis: The slopes of hours of volunteer work; days attending school, days in 
vocational rehabilitation settings, days engaged in any one of the work-related activities; 
and income earned over time will differ between the control and Benefits Counseling 
groups. 
 
Exploratory Aim Two: To conduct bivariate analyses of two of the components of mediation of 
Benefits Counseling’s effects---correlation with the group assignment and with the outcome: 
Hypothesis 2A: Assignment to Benefits Counseling (vs. control) will be associated with 
one or more of the following: (i) more positive changes in ratings of work’s importance, 
(ii) greater decreases in beliefs that working and service connection are linked, and (iii) 
more positive changes in number of weeks attending any mental health or vocational 
counseling. 
 
Hypothesis 2B: Individual slopes in hours worked over time will be associated with one 
or more of the potential mediators (i, ii, iii) as above in Hypothesis 2A. 
 
4. Background 
In FY 2013, a total of 1,110,050 Veterans were being compensated for mental disorders. Of the 
1.4 million new claims awarded for various conditions, 100,515 were for a mental disorder [8]. 
More than half of all OEF/OIF Veterans are expected to apply for some form of service 
connection [9]. The disability decision has far-reaching implications for Veterans’ financial and 
work status [10]. Despite the predominantly salutary effects of disability payments [11, 12], 
Veterans [13, 14] and Social Security recipients [15] who receive disability payments are less 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
3 
 
likely to work for pay, even after statistically adjusting for the fact that people awarded benefits 
are generally more disabled than those denied benefits. Working is associated with better quality 
of life and other benefits [9, 16], so underemployment is a problem. A strong argument for 
engagement of Veterans applying for service-connection in occupational rehabilitation was made 
by Sally Satel, a Yale-affiliated psychiatrist, in 2011 Congressional Hearings. In her testimony, 
titled “Bridging the Gap Between Care and Compensation for Veterans” [17] she cited our 
editorial on engaging Veterans in treatment when they apply for service-connection [18] in 
describing the need for rehabilitation at the time that Veterans apply for service-connection. 
 
Benefits Counseling maximizes beneficiaries’ time engaging in work-related activities in the 
following ways: 1) Providing information about how work might impact receipt of disability 
payments: One reason people who receive disability payments are disproportionately 
underemployed is the perception that beneficiaries who work will definitely lose their disability 
benefits [16, 19]. However, service-connection ratings are based primarily on the average 
impairment in earning capacity, and therefore an individual Veteran is not supposed to be 
penalized for overcoming a disability by working (38 CFR Section 4.15). There are some 
exceptions [20] but nevertheless, the best available data are that only about 6% of Veterans who 
are service connected for PTSD lose these benefits during a ten year period [21]. 2) Amplifying 
applicants’ existing motivation to engage in work and related-activities, using a Motivational 
Interviewing approach: Motivation has been associated with success in rehabilitation [22, 23]. 
Benefits Counseling brings out motivation to work in a non-judgmental, individually-tailored 
way. 3) Guiding Veterans to available vocational rehabilitation, education, and job-finding 
services: Veterans applying for disability compensation can benefit from the extensive treatment, 
rehabilitation, and employment-related resources available to them. There is little in the current 
VA disability application process that informs patients of vocational rehabilitation opportunities. 
Results of the 2010 National Survey of Veterans suggested that rehabilitation benefits were 
underused [24]; only 14.8% of Veteran respondents who applied for disability compensation 
indicated that they had ever used vocational rehabilitation services. Many who applied 
for disability benefits but did not use any vocational rehabilitation services indicated reasons that 
were remediable such as not knowing how to apply and not knowing they were eligible for these 
services. 4) Encouraging Veterans to seek treatment for psychiatric disorders that are barriers to 
working. A substantial proportion of Veterans presenting for Compensation examinations have 
not received VA services before---75.3% in one published study [25]--- making this a valuable 
opportunity to engage Veterans in treatment and thus reduce health-related barriers to working. 
 
Evidence of Benefits Counseling's Efficacy 
Benefits Counseling was evaluated in an observational cohort study among Social Security 
recipients with psychiatric conditions who had expressed some interest in working [26]. 
Participants received an average of 8 hours of counseling. Remarkably, those beneficiaries who 
received this Benefits Counseling earned an average of $1,256 more per year than matched 
controls. Subsequently, a large, randomized controlled trial of Benefits Counseling as part of a 
multi-component bundled intervention found a large and significant effect; 61 percent of 
Benefits Counseling participants were working at follow-up compared to only 41 percent of 
control participants [27]. Our group’s work extended these findings to Veterans applying for 
service-connection for psychiatric conditions. We developed a Veteran-specific manual based on 
published literature---primarily that of Robert Drake and colleagues [26] and Nina Sayer [28-30]. 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
4 
 
Vocational rehabilitation materials were developed with input from our VISN 1 MIRECC 
collaborators, primarily Charles Drebing and Lisa Mueller, and input from rehabilitation experts 
at VA Connecticut (Morris Bell, Joanna Fiszdon). The material was discussed with Veterans, 
Veterans’ Service Organizations, and Veterans Benefits Administration stakeholders. It was then 
pilot tested and further refined. We demonstrated Benefits Counseling's efficacy among Veterans 
in a randomized clinical trial published in Psychiatric Services [31]. The study utilized many 
methodologies that are in the proposed study -recruitment of Veterans applying for service-
connection for a psychiatric condition, use of a timeline followback calendar to assess daily 
amount of work done, and assessment 4, 12, and 24 weeks after randomization. The main 
outcome analysis found that Benefits Counseling was associated with a significantly greater 
slope in days worked for pay than the control condition (p=.01). The effect size for this 
difference was 0.69 (standardized mean difference in slope estimates between the Benefits 
Counseling and control groups), a large effect for a behavioral intervention. 
 
The Development of Benefits Counseling in a Remotely-delivered Format in Addition to Face-to-
Face. Motivational Interviewing interventions targeting complex behaviors have been effectively 
delivered by computerized programs. Computer-delivered treatments employing Motivational 
Interviewing have shown some efficacy for diverse conditions including self-management of 
epilepsy [32], reduction of drug and alcohol use [33], reduction of risky sexual behaviors [34], 
and quitting smoking [35]. Because of their economies of scale [36, 37], online therapies are 
often developed in addition to face-to-face, in order to expand access to treatment, a particularly 
relevant advantage in counseling Veterans who come from remote areas for one-time C&P 
evaluations. Our group developed the internet-based version of Benefits Counseling using 
methods described by Dilorio and colleagues [32]. They identified characteristics of a website 
that effectively delivers an intervention: simple layout; navigability to different modules, 
resources and links; aesthetic appeal (e.g video clips); graphics; and ease of use. Other 
investigators have made similar recommendations [38, 39]. 
 
5. Significance 
Early intervention is particularly important for most psychiatric conditions affecting Veterans 
[40] that may become chronic [41]. Benefits Counseling can engage more Veterans in work and 
productive activity. Work has myriad health-related benefits, and can provide pride and a sense 
of purpose. A remotely-delivered Benefits Counseling intervention can be widely offered with a 
modest investment. There is considerable public pressure to integrate Veterans into the 
workforce, and to look at the Compensation and Pension process as a place to do it. In widely 
publicized testimony to the House Committee on Veterans Affairs, Linda Bilmes called for 
something like the proposed intervention: “VBA should shift its focus away from claims 
processing and onto rehabilitating and reintegration of Veterans” (House Committee on Veterans 
Affairs, 2007). Finally, the proposed study’s findings among Veterans applying for service 
connection for mental health conditions may be generalizable to Veterans applying for service-
connection for non-psychiatric, medical disabilities. The study results may hasten the day that a 
Compensation examination is routinely seen as an opportunity for both evaluation and 
rehabilitation for Veterans. 
 
6. Research plan 
A. Study Design Overview  

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
5 
 
Data collection will be conducted at the Northampton VA Medical Center and the West Haven 
VA Medical Center. Veterans will complete baseline evaluations, and then be given a user ID 
and instructions to log-on to a study website at home to receive their randomization assignment. 
Only those who do so will continue in the study and be asked to complete the telephone-
delivered assessments on weeks 4, 12, and 24. This randomization after logging in restricts 
participation to people who demonstrate some motivation and ability to access a website. 
 
Sequence of procedures for the clinical trial: 
___  
Potentially eligible Veterans scheduled for a Compensation examination are identified by 
reviewing Compensation clinic schedules and charts. 
___  
Potentially eligible Veteran receives letter explaining that a study recruitment telephone 
call will be coming, thus allowing the Veteran to opt out of receiving it (see Appendix). 
Veterans can also opt out by calling a toll-free phone provided in the letter. 
___  
Telephone call inviting participation. The Research Assistant calls and asks the Veteran 
questions to determine eligibility. Following a script (see Appendix), the Research 
Assistant describes the study to the Veteran and schedules an appointment for informed 
consent and data collection. Appointments will be scheduled before a Veteran’s planned 
Compensation examination and up to 3 months after the exam. Veterans vary in how able 
they are to schedule appointments immediately before or after Compensation 
examinations. Flexible scheduling balances feasibility and experimental rigor. 
___  
Informed consent and baseline data collection by Research Assistant. 
___  
The Research Assistant gives the Veteran a user ID and password to log on to the study 
Website. 
___  
Veteran goes home, logs onto the website, and receives directions to log-on to either the 
Benefits Counseling or Control websites (i.e. the Veteran is randomized). 
____  Veteran is called by the study therapist within two days of logging on and is counseled 
with the assigned treatment. If the Veteran has not logged on for randomization within a 
week, then the therapist will call and encourage the Veteran to log-on for randomization 
and continuation in the study. If after one more week the Veteran has not been 
randomized, that participant will be discontinued from the study. 
___  
Assessments by phone at weeks 4, 12, 24 (only of randomized Veterans). 
 
B. Remotely-Delivered Benefits Counseling 
1. Integration of Website Use and Therapist Phone Calls: 
a. Purposes of Phone Calls: The purpose of the Benefits Counseling phone calls is to 
complement the web-based modules and to support the Veteran in applying the website 
content to his/her particular situation. 
b. Introduction: The therapist explains that his/her role is to help the Veteran get as much 
as possible out of the Benefits Counseling website. The therapist plans the dates and 
times for phone calls and provides the Veteran with a phone number to call him/her for 
rescheduling or brief questions. 
c. Timing and Content of Session Calls: The therapist tries to schedule calls soon after the 
Veteran has completed each of the first two modules. At each of the calls, the therapist 
makes general inquiries about the Veteran’s experience using the site, the information 
covered, and any follow-up the Veteran plans. Based on these responses, the therapist 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
6 
 
will follow-up with questions about the content of the module just completed (domains 
described below). 
d. Timing and Content of Session-Three Call: Therapists will instruct Veterans to 
complete Module Three after receiving their service-connection decision and to call after 
completing it. If the Veteran has not called by week 12, the therapist will call the Veteran 
to determine the status of the C&P claim, and module completion. If the C&P decision 
has not been received, the therapist will call again at 16 and 20 weeks to review the 
decision and module, if the Veteran hasn’t called first. 
e. Overall Call Limit: Therapists will limit calls to no more than 30 minutes each, and no 
more than three therapy calls per Veteran; calls lasting fewer than five minutes (e.g. for 
scheduling or brief questions) will not count towards the three call limit. 
 
2. Website 
 
a. The VACHS IRB reviewed the website’s content as part of protocol MR0031, IRB 
#01389, for preliminary use by Veterans.  Screenshots of the website are attached as an 
Appendix. 
The website can also be accessed by 
Going to vabenefitseducation.org 
Entering username marc 
Password bluefish 
Selecting go to cloned interview. 
   
b. The Goals and Content of the Three Modules: In keeping with published literature and 
our pilot experience, we kept the intervention modules brief (about 20-30 minutes each). 
Module 
Module’s Goals 
Session Content in Brief 
Session One: Your 
Benefits and Your 
Goals 
To engage Veteran in the 
program, explain that Veterans 
can work and earn service-
connection 
-Instruction on how to use the website and its 
purpose 
-Review Veteran’s experience applying for benefits 
-Inquire about Veteran’s perception of relationship 
between disability benefits and work, explanation of 
relationship between disability benefits and work 
Session Two: Your 
Options-Work and 
Otherwise 
To enhance the Veteran’s 
motivation to work. Based on 
goals, make an Action Plan. 
-Elicit situation, feelings and attitudes towards 
employment 
-Motivational Interviewing exercises (e.g., 
Decisional Balance, Values Card Sort) 
-Review barriers to work including the disabling 
condition; consideration of treatment for disabling 
condition 
-Develop an action plan 
Session Three: 
After Your Service 
Connection 
Decision 
To review the goals and plans 
identified in the previous 
sessions in light of the award 
(or denial) of service-
connection benefits 
-Inquiries about Veteran’s reaction to service-
connection decision; provide information about how 
to appeal decision 
-Reconsider work and financial goals (from Session 
Two) in light of service-connection award (or 
denial) 
-Review action plan 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
7 
 
-Troubleshoot barriers, summary, and follow-up 
resources 
c. The relationship between work and benefits is explained using graphics, and video 
testimonials from Veterans who are service-connected and work.  
 
C. Control Condition: 
1. Description of the Control Condition and Rationale for Choosing this Specific Control 
Condition: This control condition, involving referral to VA websites, represents enhanced 
treatment-as-usual. A Veteran who completes a Compensation examination ordinarily has no 
further treatment or referral as part of the Compensation examination. The information about 
benefits-related websites controls for having information available about benefits and receiving 
encouragement to pursue that information. 
 
2. Control Condition Content: The control condition involves being urged to explore links to 
three VA websites with Compensation & Pension-related information: 
a. The VBA website with links about VA disability compensation. The questions 
answered are: What is VA Disability Compensation?, Who Is Eligible?, How Much Does 
VA Pay?, How Can You Apply? And Upcoming Events. 
b. The VBA site with fact sheets about the different types of benefits available to 
Veterans. The topics covered in these fact sheets are: General Benefit Information, 
Veterans with Service-Connected Disabilities, Veterans with Limited Income, Education 
and Training for Veterans, Home Loans, Dependents and Survivors' Benefits, Burial and 
Memorial Benefits, and Insurance. 
c. The VBA websites for the local C&P office where issues pertaining to an individual 
Veteran's specific application are addressed. The site includes phone numbers to call with 
questions, e-mail contacts and directions to the VBA office. 
 
3. Role of Therapists in Control Condition: The therapists in the control condition will differ 
from the Benefits Counseling therapist as shown in the table below 
 
Control Therapist 
Benefits Counseling Therapist 
Initiate calls to Veterans at weeks 1 and 2 
Initiate calls to Veteran at weeks 1 and 2 and 
around time of Module Three (or thereafter) 
Up to three therapy calls of up to 30 minutes each 
Up to three therapy calls of up to 30 minutes each 
Convey information about Veterans Benefits 
Convey information about Veterans Benefits AND 
vocational issues 
Offer to elaborate on information on control 
websites 
Inquiry about modules on Benefits Counseling 
website. Follow-up inquiries about domains 
covered on site.  
 
Justification of Sample Size 
We estimate that 190 Veterans will be enrolled and complete baseline assessments; 130 
will log-in to the randomization site and be assigned to a treatment condition; 104 (52 per 
condition) will complete at least one follow-up assessment. 
 
Precision Analysis  
Consistent with the purpose of a pilot study, the sample size was selected to 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
8 
 
estimate the standardized effect of treatment with reasonable precision, rather than to power for a 
statistically significant effect [50]; A sample size of 52 per treatment condition permits 
estimation of an expected standardized difference in slopes of hours worked between Benefits 
Counseling and control groups (e.g., (slopeBC-slopeCON)/SDslope) of δ=.40 with a 95% 
confidence interval of (.01, .79). This effect would reflect an average 6% increase in baseline 
number of hours worked per month, or a total increase of 43% by month 6. Assuming an average 
baseline event rate of 40 hours/28 days and no change in the control group, this effect is 
equivalent to approximately 17 additional hours (approximately 2 days) of work per month by 
month 6. 
We planned our precision analysis around an effect size of 0.4 because (a) the effect size 
in our RCT of face-to-face Benefits Counseling was δ=.69 and (b) an effect size of 0.4 is 
clinically meaningful. An additional 2 days worked reflects earning a substantial amount of 
additional money for people with limited incomes, in addition to other benefits associated with 
working. 
 
D. Experimental Participants 
1. Inclusion and Exclusion Criteria 
a. Inclusion Criteria: All enrolled participants will have difficulty working as evidenced 
by a “Yes” response to any sub-part of the question below (from the SF-36, [51]): “During the 
past 4 weeks, have you had any of the following problems with your work or other regular daily 
activities as a result of any emotional problems (such as feeling depressed or anxious): (a) Cut 
down on the amount of time you spent on work or other activities? (b) Accomplished less than 
you would like? (c) Didn’t do work or other activities as carefully as usual?” 
Participants will also meet the following inclusion criteria: (i) A Compensation and 
Pension evaluation by a psychiatrist or psychologist is scheduled; (ii) Does not have a service-
connected physical disability rated over 30%; (iii) Is not service-connected for a psychiatric 
condition (0% service-connection is allowed, as Veterans with 0% service-connection have not 
received any benefit payments); (iv) Age 18-65; (v) Reports access to an internet-connected 
computer/tablet and a working telephone; (vi) Reports working  no more than 24 hours per week 
for pay over the preceding 4 weeks; (vii) Self-reported ability to participate psychologically and 
physically, able to give informed consent and complete assessments; (viii) 
Reports being able to read (this is necessary for understanding the website); (ix) Expressed 
interest in sampling whatever website is provided. 
b. Exclusion Criteria: (i) Already receiving service-connection, SSI or SSDI; (ii) Has a 
conservator. 
 
2. Feasibility of Randomizing 130 Veterans: A review of Compensation examinations conducted 
at Northampton over a 12 month period showed that 547 were conducted for psychiatric 
conditions. Of these, 348 were initial examinations, 190 were reviews, and 9 were for other 
reasons. We are estimating that approximately 60% of the 348 will be working fewer than two 
days per week and receiving less than 30% service-connection. In a chart review of 
Compensation examinations conducted at our MIRECC site in Bedford, approximately half (17 
out of 38) of Veterans coming for new psychiatric Compensation evaluations were both not 
working at all and less than 30% service-connected. Some Veterans will be excluded for other 
reasons. Therefore, we estimate that 209 eligible Veterans will be approached in a year, which is 
enough to randomize 130 over two and a half years of recruitment. 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
9 
 
 
3. Balancing the study groups: Participants will be randomized by the Project Director, with 
assignment delivered by the website, using an urn randomization procedure [52] that offers the 
benefits of balancing allocation of important prognostic variables in treatment groups, while still 
retaining other benefits of random assignment. The factors in the urn will be those potentially 
related to vocational outcomes in Veterans: 
• Self-reported use of an illicit drug and/or risky alcohol use within the past 3 months (as 
measured by the ASSIST, using standard definitions [53,60,61,62]) within the last 28 
days. Opioid use will qualify if the opioids were not prescribed to the Veteran. 
• Current enrollment in VA psychiatric or substance abuse treatment (yes/no). 
• Race (African American –yes/no) has been associated with responses to remotely-
delivered behavioral messages [54]. 
 
Justification for Why Follow-up Data Collection after Baseline will be Done via Telephone:  
Timeline followbacks have been administered by phone for substance abuse evaluation 
and are highly consistent with face-to- face assessments conducted in the same participants---in 
fact reports of substance use were numerically higher with the telephone assessment, suggesting 
a greater likelihood of truthful reporting of embarrassing information by phone [55]. The follow-
up assessments are also relatively brief, taking 30 minutes or less per session. Veterans will have 
already completed these assessments at the face-to-face baseline visit so they will be familiar 
with them when they are re-administered by phone. 
1. Separation of Therapies from Assessments: Veterans will be instructed not to tell the 
Research Assistant if they are receiving Benefits Counseling or not. To determine if the blind 
was maintained, we will ask the Research Assistant to rate the condition each Veteran was 
assigned to and the Research Assistant’s certainty about the assignment on a 0-100 visual 
analogue scale. 
2. Participant Tracking Procedures: After randomization, participants will schedule 
phone assessments with the Research Assistant. The Research Assistant will offer to place 
reminder calls or a text message/email reminder prior to the assessment call to help Veterans 
remember them. Within two days of an upcoming appointment, participants will receive a 
reminder call/email/text. (Sample text/email would read: “You have a follow-up phone call for 
the VA research study scheduled for [date] at [time]. Please call [Research Assistant’s name] at 
[phone number] if you need to reschedule. Do not reply to this message as the account is not 
monitored. Thank you.”) If a participant misses a scheduled assessment, the Research Assistant 
will place follow-up calls. 
To the extent that participants are not able to be reached for either the phone assessments 
(or therapist telephone calls), participants’ VA medical records will be accessed to obtain 
updated telephone numbers, in addition to updated resident addresses. This will allow us to call 
participants and send letters to participants using the most up to date information on file. We also 
will seek to obtain information from participants' VA records regarding upcoming appointments 
and additional locations where they can be located so as to re-establish contact. Participants will 
have the option to not allow this via an opt-out statement included in the consent form.  
Through these proposed means we believe we will be able to increase participant 
retention and better track the outcome of participation in the current study.   

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
10 
 
 
3. Schedule of Assessments 
Visit 
Domain 
Assessment 
BL 
Characterize 
Participants 
Baseline Questionnaire, SF 36; Alcohol, Smoking and Substance Involvement 
Screening Test (ASSIST); Disability Application Appraisal Inventory; PTSD 
Checklist (DSM5 version); Brief Symptom Inventory 
BL, 4, 12, 24 
Baseline and 
Outcomes 
Timeline Follow-Back Calendar: paid work, unpaid work, voc rehab, 
education; 
Income from different sources; Brief Symptom Inventory; PTSD Checklist-5 
BL, 4, 12, 24 
Potential 
Mediators 
Work-Related Attitudes Scale: Work Importance and Confidence Ruler 
Therapy 
Session 
Process 
Audio-recorded Phone Calls 
Week 8, 24 
Process 
Satisfaction with Counseling and with Websites (mailed assessment) 
End-of-Study 
Process 
Extent of Study Participation from Therapist Cell-Phone (# minutes on phone 
with Therapist) 
Extent of Benefits Counseling Website Use (# days logged on to website, # 
modules completed) 
End-of-Study 
Extract Data 
from C&P Exam 
and VA Database 
Mental Health Diagnoses and service connection outcome and award 
determination from C&P Examination Reports and VBA databases and VA 
Mental Health Service Use from VA Databases 
 
Justification for Schedule of Assessments: The initial 4-week assessment follow-up date was 
chosen because results of Motivational Interviewing interventions have been detectable this 
quickly in other studies [56]. In our randomized clinical trial of in-person Benefits Counseling, 
week 24 assessments allowed for detection of later emerging effects of Benefits Counseling. 
 
Baseline Assessment Measures to Characterize Participants: The following measures will be 
administered by the Research Assistant in a face-to-face meeting prior to beginning the 
intervention:  
1. Baseline Characteristics Questionnaire: A questionnaire to characterize study participants has 
been used in our recently-completed Benefits Counseling study. It covers the following domains: 
demographics, military history, psychiatric history, employment, legal status, and last 28 days of 
income by source. It also includes 10 items from the Attitudes toward Seeking Professional 
Psychological Help Scale-Short Form [57]. This widely-used scale consists of two factors, one 
reflecting openness to seeking treatment for emotional problems and the other reflecting value of 
seeking treatment and need for it. The Questionnaire also includes a four-question screen for 
TBI, a screen which is intended to be sensitive but not specific, when a single positive answer is 
interpreted as a positive screen [58]. Internet usage is measured using items derived from a 
clinical trial of an interactive voice response system [59]. Veterans are asked whether they own 
different technology types (e.g. a working cell phone), whether they make different uses of it (e-
mail, text messaging, searching) and number of days engaged in activities such as sending e-
mail. 
2. The SF-36 will be used to measure self-rated quality of life. The SF-36 was derived from the 
Medical Outcomes Study to assess health-related quality of life [51]. Patients answer 36 
questions about health related impairments in daily activities, impairments from illness, and 
psychological states. 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
11 
 
3. Alcohol, Smoking and Substance Involvement Screening Test (ASSIST): The ASSIST 
consists of eight questions concerning use of alcohol and or drugs, mostly within a three-month 
timeframe. It includes inquiries about amount of use, and incorporates questions that are 
somewhat indirect such as whether anyone else asked the Veteran to cut-down on use [60]. The 
ASSIST has excellent reliability [61] and validity [62]. 
4. The Disability Application Appraisal Inventory [28] was developed to better understand VA 
disability claimants’ views of disability application and the role the beliefs play in clinical 
presentation. It is composed of 3 scales assessing (i) understanding of the claims process 
(Knowledge scale), (ii) expectations specific to the process (Negative Expectations scale), and 
(iii) the reasons becoming service connected is important to Veterans (Importance scale). The 
Disability Application Appraisal Inventory’s test-retest reliability, internal consistency, factor 
structure and validation against selected MMPI subscales were tested and the measure has good 
reliability and construct validity [28]. 
5. The Post Traumatic Stress Disorder Checklist for DSM5 is a 20-item update [3] of the 
extensively validated earlier version [63], which had excellent agreement with comprehensive 
observer ratings [64, 65]. Veterans are asked to rate their agreement on a 5-point Likert scale 
with statements about how bothered the Veteran has been by each symptom in the past month, 
and the total score is the sum of the 20 item scores. The wording is general enough to rate PTSD 
symptoms related to Military Sexual Trauma as well as Combat Trauma and it has been sensitive 
to treatment effects in behavior therapy trials [66]. 
 
Outcomes Measured Repeatedly (Baseline, Weeks 4, 12, and 24) 
1. Work and rehabilitative activities. A time-line-follow-back method will be used to determine 
whether and on what days the Veteran engaged in different work-related activities.  
--- work for pay (in-kind compensation such as reduced rent is only counted when it is a formal 
arrangement) 
---working without pay (includes formal volunteer work, and informal work such as babysitting, 
cleaning grounds for a neighbor, cleaning basements and attics, informal in-kind arrangements) 
---participated in any kind of vocational rehabilitation (includes Compensated Work Therapy, 
Supported Employment) 
---enrolled in classes, attended school 
Additional coding is made for illicit work, and an “other” category for other work-related 
activities. 
A calendar avoids information being collected twice for the same service, such as Compensated 
Work Therapy being counted as a paid job and as rehabilitation. Separate summary measures 
will be the number of hours and days the participant worked for pay; and days worked without 
pay, participated in any kind of vocational rehabilitation, and attended school. A secondary 
outcome will be the number of days out of 28 the Veteran engaged in any one of these activities 
(i.e. range of 0-28 with each day scored 0/1 for whether any of the above activities were engaged 
in). Income earned will be tabulated. 
 
2. The Brief Symptom Inventory [4] is a 53-item self-report inventory of psychiatric symptoms 
that asks patients to rate items on a 5-point scale of distress. It yields nine primary symptom 
dimensions: anxiety, depression, hostility, interpersonal sensitivity, obsessive-compulsive, 
paranoia, phobic anxiety, psychoticism and somatization. Depression, most notably, has been 
strongly associated with function among the disabled [68]. 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
12 
 
 
Attitude Measures that Might Mediate Treatment Response 
1. Work Importance and Confidence Ruler: One potential mediator of change that will be 
measured is “importance of work” and “confidence in ability to work” rated on a similar scale 
and asked in a similar way as used in Motivational Interviewing [69] and for many other 
behavioral measures [70]. Our VISN 1 MIRECC colleague Lisa Mueller used this scale to show 
that Veteran-rated importance of work was associated with engagement in Compensated Work 
Therapy among Veterans with vocational and psychiatric difficulties. 
 
2. Work-Related Attitudes Scale. This instrument asks the Veteran to rate agreement with 
statements asserting an absolute connection between several work situations and receiving 
service-connection. The items were modified from those used in a completed study of Supported 
Employment at VA [71] and from those that showed range restriction in our study of in person 
Benefits Counseling. 
 
Time Veterans Need to Complete Assessments: It is estimated that the assessments collected at 
the first face-to-face assessment will take approximately 90 minutes to complete and the 
assessments collected serially by phone at 4, 12, and 24 weeks will take an average of 25 
minutes per call to complete. 
 
During Therapy Session Audio Recordings 
Counselors will attempt to audio record all telephone sessions. From these, a random 
sample of approximately 20 will be rated. Procedures for rating audio recordings and items will 
follow the format of the Yale Adherence and Competence Scale (YACS) [72] in that sessions 
will be rated for expected elements (review of work and compensation claim), proscribed 
elements (pressuring Veterans to work against their will) and general elements (therapist 
supportiveness).  
Ratings will be conducted by the Project Director and an experienced clinician-rater. 
Only the independent rater’s ratings will be analyzed; the Director’s ratings will be for training 
and supervision only. The rater will be trained until adequate inter-rater reliability with the 
Project Director’s ratings is achieved. 
 
Process Measures (weeks 8 and 24): 
a. Satisfaction with Counseling and Websites will be rated on Likert-scaled items. This form will 
be mailed to the Veteran at weeks 8 and 24 with instructions to complete it, place it in a sealed 
envelope, and mail it to the  Project Director. The reason to administer this assessment by mail is 
to maintain the study blind, which would be compromised by having the Research Assistant 
obtain them. The initial portion of this scale asks for ratings of the counseling overall. The 
second portion, used in our “Think Aloud” study, asks participants to rate characteristics of the 
website they were randomized to (e.g. clarity of layout, usefulness of information, desirability 
compared to face-to-face presentation of same material). 
b. Benefits Counseling Treatment Engagement. The hosting server will record the date and time 
of each log-in by study ID. We will track the number of separate days the Veteran logged on to 
each website module and responses for each participant. We will also track the number of 
completed modules (out of three possible). The main treatment participation measure will be 
number of days logged on to the website. Note that the program does not collect any PHI. It also 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
13 
 
does not record how long was spent on the website because it cannot distinguish between when 
someone is using the program, has left the browser open, or has left the program without logging 
out. Study therapists will complete a contact log for each encounter. The contact log will 
document whether there was a scheduled contact that was not attended, the date and time of 
contact, and time spent in minutes. 
 
End of Study Measures 
1. The current diagnoses, service-connection outcome, and award determination will be obtained 
from the completed Compensation and Pension examination and VBA database. Diagnoses will 
be coded within DSM-V with specifiers for definite diagnoses and rule-outs. 
2. VA Service Use Outcomes. Analyses will be conducted by Robert Rosenheck, a national 
authority on service use analyses. Episodes reflecting use of VA substance abuse, specialized 
PTSD and other mental health treatment will be collected from VA administrative records as 
described previously [73, 74]. Information will be accessed by an agreement between the 
MIRECC and the West Haven-sited Northeast Program Evaluation Center (NEPEC), which 
routinely accesses this information for VA reports. VA bed section codes indicate hospitalization 
on specialized units and VA clinic codes indicate outpatient treatment in specialized clinics. 
Mental Health service use data will be collected from 24 weeks before the Compensation 
examination (to allow analyses to control for baseline service use) until 24 weeks after claims 
adjudication. 
3. Cost of Therapist Time Providing the Intervention: The direct costs of providing Remotely-
Delivered Benefits Counseling will be calculated but a full cost-effectiveness analysis is beyond 
the scope of this proposal. The amount of time the therapist spends will be calculated by having 
therapists maintain logs of time spent on study-related activities including conducting telephone 
calls, scheduling phone calls, and reminding Veterans to access the study website. The product 
of the therapist’s time spent and the average therapist cost will be used to calculate the labor cost. 
Therapist salaries will be calculated including fringe benefits but not the overhead rate of the VA 
facilities. We will not include research costs, facility costs, or the costs of developing the 
intervention. 
 
Data Analysis  
1. Determination of Study Outcomes: We list below the outcome measures whose scoring 
requires explication. Many of the scales’ scoring is embedded in our earlier description of the 
measure. 
a. Primary Outcome: The a priori definition of a primary outcome reduces the risk of Type I 
error. The number of hours paid work will be the primary treatment outcome. As noted 
previously, this includes work that is formal or under-the-table, and includes formal in-kind 
arrangements. It does not include days engaged in informal in-kind arrangements (housekeeping) 
or illegal activities such as gambling. 
b. Secondary Outcomes: Secondary outcomes include each of the other work-related activities 
considered separately (unpaid work, attending school, engaging in vocational rehabilitation). 
Another secondary outcome is days engaged in any of the above work-related activities. This 
measure of work-related activities has a range of 0 to 28 in that each day is scored for whether 
any work-related activity was engaged in on that day as a dichotomous (0,1) value. 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
14 
 
Dollars earned will be log-transformed (adding “1” to totals before transformation to allow 
calculation for veterans with no income) for outcome analyses if it is heavily skewed by outliers 
with high incomes. 
c. Scoring of Clinical Measures.  
i. Psychiatric Symptomatology: The Global Severity Index from the Brief Symptom 
Inventory will be the main outcome from the BSI. The depression subscale will be scored 
too because of the association between depression and poor work function. 
 
ii. The SF-36 will be scored to yield standardized Mental Health and Physical Health 
Component Scores, with a mean of 50 and population standard deviation of 10. As noted 
earlier, the PTSD Checklist items are summed to yield a single score. 
 
iii. For the Work-Related Attitudes Scale, EFA will be conducted if it can be 
accomplished with the sample, i.e., if there is approximately a 10:1 participant:item ratio 
and item communalities are high [.6 or greater] after dropping any items with poor 
variability. If EFA is not feasible, we will analyze inter-item correlations and estimate 
internal consistency reliabilities to identify items that cluster together within a domain 
that reflect the perceived work-benefits relationship. A mean score of all items in this 
domain (or domains, if more than one is identified), will be generated. 
 
iv. The ASSIST yields an overall substance use severity score. 
 
d. Process Measures 
i. Treatment engagement: This will be measured as number of days logging onto the 
website, number of modules completed, number of minutes spent in telephone calls with 
the Counselor. 
 
ii. Treatment satisfaction will be assessed using mean score on subscales of the Likert-
scaled Satisfaction questionnaire (with reverse scoring). Satisfaction with discrete 
elements of the websites will be considered as separate items. 
 
iii. Number of weeks attending any VA substance abuse and/or mental health treatment 
will be derived from the service use data. 
 
2. Approach to Missing Data: For analyses involving HLM, maximum likelihood estimation 
techniques permit missing data on the dependent variable. Valid parameters are estimated from 
all available data from all cases under the assumption that data are missing at random. To 
improve the plausibility of the missing-at-random assumption, we may include missing data 
correlates in the model. For normal models (ANOVA, Regression) requiring complete data, if 
data are not missing completely at random (MCAR), we will use NORM software to multiply 
impute missing values. We will first determine whether data are MCAR using Little’s test [75]. 
If data are not MCAR, we will characterize study drop-outs in terms of differences in mean 
scores on relevant variables. Variables determined to be related to missingness or to the 
incomplete variables (missing data correlates) will be included in models to improve estimation 
[76]. In specifying the imputation model, we will follow guidelines of Rubin [77], Schafer [78, 
79]) and Collins et al. [76], including all variables that will be included in the analytical models, 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
15 
 
as well as all variables that are associated with the variable(s) with missing values, and with 
missingness itself. We will combine parameter estimates and estimate standard errors according 
to Rubin’s rules [80]. To determine what observed variables are associated with missingness on a 
given variable, we will create dichotomous “missingness” indicators (missing or not missing; 
1/0) and estimate logistic regression models. Variables that significantly predict missingness will 
be included in the imputation model. 
 
3. Evaluation of Treatment Effects: For baseline values potentially related to outcome, evaluation 
of comparability of the two treatment groups will be conducted using chi-square tests for 
categorical variables and t-tests for continuous variables. If an unexpected difference is found 
and is related to the outcome, the potentially significant confounding variable will be added to 
the primary outcome analysis as a covariate.  
The principal strategy for assessing the efficacy of the study treatment will be random 
effects regression, which will be used to determine the differences in slopes of outcomes 
between Veterans assigned to Benefits Counseling compared to Control. The model will include 
covariates described in the Introduction Section, time measured in weeks, and the contrasts of 
Benefits Counseling versus Control. A linear effect of time on outcome variables is being 
hypothesized but ancillary analyses will be conducted to determine if the effect of Benefits 
Counseling treatment is better explained by a quadratic function (as described in the Introduction 
section of this grant). Models will be specified to accommodate the observed distributions of 
outcomes (e.g., Poisson, normal, Bernoulli) and theory, model estimates, and fit indices will 
inform the specification of effects as fixed or random, and the structure of the variance-
covariance matrix. 
Our principal data analyses will be conducted on all randomized Veterans. We will 
attempt to follow all participants who are randomized for assessments regardless of their 
retention in treatment. Intention-to-treat analyses will be supplemented by analyses to describe 
the consistency of findings across different definitions of the sample: e.g., module-completers 
and treatment completers. Measures of engagement in Benefits Counseling will also be tested for 
their association with outcomes (number of days using the website, modules completed, minutes 
on phone with Benefits Counselor). 
 
4. Evaluation of Processes that Might Mediate Treatment Response: This study is not powered 
for full mediation analyses but will allow for bivariate analyses of the components of mediation. 
Bivariate analyses will be conducted of the relationship between the slope of each potential 
mediator and (1) group assignment and (2) slope of hours worked per week. The partial 
correlation of group assignment with the slope of hours worked per week will be calculated, 
controlling in turn for each potential mediator’s slope. 
The general potential mediators of treatment response will include ratings of 
importance/confidence in ability to work, work-benefits attitudes (attitudes about one’s personal 
situation, understanding of work-benefits relationship), service-connection award, and number of 
weeks engaged in any mental health and/or substance abuse treatment. 
 
5. Evaluation of Process Measures: The Likert-scaled therapy process ratings and other measures 
reflecting single values (e.g. number of times logging into the study website) will be compared 
between groups using parametric or non-parametric group comparisons. Only the Benefits 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
16 
 
Counseling group will have ratings of audiotaped therapy sessions and these will be tabulated for 
descriptive statistics. 
 
6. Evaluation of Factors Associated with Effects of Study Treatments on Working (Moderators) 
To determine what factors are associated with response to counseling, potential predictor 
variables will be serially entered into the primary outcome analyses as separate covariates, and as 
terms interacting with group-assignment. To avoid finding significant associations by chance, 
potential predictors will be screened for their intercorrelations with each other and redundant 
measures (e.g., r>.7; R2 ≥.6; VIF≥2.5) will be pared or 
combined. Potential predictor variables include: 
--Demographic variables (age, race, gender), 
--Conditions that Make Working More Difficult: Substance use severity, baseline PTSD severity 
as measured by the PCL-5, overall psychological symptomatology as measured by the BSI, 
depression as measured by the BSI. 
--Attitudes Toward Seeking Professional Psychological Help Scale-Short Form (two domains--- 
openness to seeking treatment for emotional problems and value of seeking treatment). 
---Veterans' rated reasons for seeking compensation as described by the Disability Application 
Appraisal and Inventory [28]: we hypothesize that Veterans who rate themselves higher in 
having applied for financial gain will be more likely to work in response to Benefits Counseling. 
 
7. Evaluation of factors associated with progress through different steps of study participation 
(i.e. going to the website for randomization vs. not doing so; using the Benefits Counseling 
website if assigned to it vs. not following the assignment). Potential predictors will be screened 
for their intercorrelations with each other and redundant measures will be pared or combined. 
We will then use bivariate analyses (t-test, chi-square) to identify factors that differentiate the 
groups of interest at a significance level of alpha = .10 or less. Identified factors will be entered 
into a logistic regression with group as the dependent variable (e.g. randomized vs. not). Factors 
to be considered include internet-associated factors (ease of internet access, extent of internet 
use), disability related factors (knowledge of disability process, desire to learn more about it), 
work-related factors (e.g. distress about not working), and demographic variables (age, gender, 
race, education status), and conditions that make internet use more difficult (e.g. BSI global 
severity). 
 
Human Subjects  
A. Consent Procedures  
1. Recruitment: A waiver of informed consent and HIPPAA authorization will be obtained to 
review the Compensation and Pension materials and charts of Veterans scheduled for 
Compensation and Pension examinations to identify those who are not more than 30% service-
connected for physical conditions and scheduled for examinations for a psychiatric condition. 
The waiver will allow for the collection of basic demographic information concerning Veterans 
screened for study participation to determine if Veterans who consent to study participation 
differ systematically from those who do not.  
The recruitment strategy involved identifying Veterans who have submitted requests for 
mental health Compensation examinations. Potentially eligible Veterans will be Express Mailed 
a letter from the P.I. explaining the basics of the study and that unless the Veteran opts out, a 
member of the research team will call to further explain the study. The letter will include a phone 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
17 
 
number to call to opt not to receive the research call. No phone calls will be made to Veterans 
calling to opt out.  
At that call, the Research Assistant will explain the study and invite the Veteran for a 
face-to-face meeting with further screening, informed consent, and baseline assessments. A 
Waiver of written informed consent will be obtained to screen potential participants over the 
phone. 
 Justification for recruitment via mailed letter: The proposed letter recruitment strategy, 
which was used in several of Dr. Rosen’s other approved studies including two SBIRT studies 
(projects which deliver SBIRT to Veterans applying for disability for painful musculoskeletal 
conditions or PTSD) and the QUERI study (a multi-site study of the Compensation and Pension 
process directed by Theodore Speroff), was effective. The main reason for recruitment by letter 
is that it would not be feasible to recruit Veterans at the time they come for their Compensation 
examination because  Veterans may not have time to consider study participation when they are 
already at their Compensation examinations. Veterans at their Compensation and Pension 
evaluations may be nervous and rushed. They may be accompanied by other people who cannot 
or will not wait while the Veteran participates in an unplanned study. Additionally, assessments 
are designed to be collected before the Compensation examination. 
A second mode of recruitment will involve having C&P clinic staff (but not the C&P 
examiner) give potentially eligible Veterans a study brochure and ask them if they are willing to 
discuss study participation with a member of the study staff. If they are, a member of the staff 
will contact them at the C&P office, if feasible, or by phone.  
 
2. Consent: Voluntary informed consent will be obtained from all Veterans prior to participation. 
Consent will be obtained by one of the investigators or his/her designee after the research 
procedures and risks associated with participation have been explained to the candidate. A signed 
copy of the consent form will be provided to all potential Veterans.  
Participants will be asked several questions to be certain they understand the risks 
associated with study participation. Only participants who correctly answer all the questions 
(with a second chance to answer questions if incorrectly answered the first time) will be enrolled. 
The questions will be the following:  
a. What topics will you be asked about?  
Answer should mention working.  
b. Is it possible that someone will learn that you were at the study website?  
The correct answer is “Yes.”  
c. Can you drop out of this study whenever you want? Yes.  
 
B. Risks and Discomforts 
Risks associated with participation in the study include risks associated with:  
1. Completing questionnaires: Administration of study instruments may be boring, frustrating, or 
upsetting.  
2. Becoming upset while discussing traumatic events: There is a risk that individual Veterans 
will become upset by discussing their traumatic events during assessments or counseling. In 
practice, study-related adverse events in clinical trials involving PTSD assessments among 
Veterans have been very rare. Research assistants will emphasize that answering questions is 
voluntary and will allow Veterans to answer questions at their own pace and without duress.  

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
18 
 
3. Information from the counseling being told to participants’ clinicians at the VA or to someone 
who reviews their disability application. The counseling is unlikely to impact the Veteran’s 
disability application. The therapist will not volunteer any non-emergent information in the 
Veterans’ chart or to other clinicians, as this is contrary to the voluntary, Motivational 
Interviewing stance that is integral to Benefits Counseling. The therapist will write a note in the 
Veteran’s VA chart after any phone counseling indicating that the participant was in this research 
study and spoke to the therapist.  
Nevertheless, the therapist is going to convey information to other VA clinicians if the 
Veteran indicates information during the therapy that requires emergency intervention. If the 
Veteran indicates information suggesting imminent danger to the Veteran or to others, the 
Benefits Counseling therapist will share this information with other VA clinicians so they can 
take appropriate action. Therefore, Veterans will be told the risk that information that they tell 
the study therapist may be told to their VA clinicians and would then become available to 
someone who reviews their disability application. It is very unlikely that an emergency 
disclosure would prevent receipt of service-connected disability.  
4. Potential effect of work on the benefits application process: Veterans compensated for 
physical or mental problems that developed during or were made worse by military service are 
not supposed to be penalized for coping well with these problems by working. However, the 
Veterans Benefits Administration reviews claims on a case-by-case basis, and it is unknown to 
what extent working will influence any individual Veteran’s disability determination process. 
Veterans will be told this as part of the informed consent process.  
5. Breach of Confidentiality: The website will not contain any PHI.  
However, the Veterans’ computer that accesses the website will leave a trace (in the 
history of browsed sites on the local computer).Therefore, if a subsequent user of a computer 
knows the specific Veteran who used an account, the user may infer that the Veteran has a 
Compensation claim and/or a psychiatric disorder. 
Research Assistants and Therapists will clarify communication procedures with Veterans. 
They will ask the Veteran’s preference in terms of whether or not the Veteran wants messages 
left on voicemail, and if so, if the Veteran wants the study therapist’s phone number left on the 
voicemail. Therapists will establish that the Veteran is the person answering the phone by using 
personal identifiers drawn from a standard list of security questions (mother’s maiden name, etc.) 
that the Veteran has supplied. 
  
C. Protection of Participants  
1. Confidentiality of Research Materials: All research information is considered confidential. All 
computers used by research staff are password protected. Limits to confidentiality include only 
disclosure of acute suicidality, homicidality, or abuse of a minor, as is standard in clinical 
practice. If a Veteran selects any value over 1 (“a little bit”) on the Brief Symptom Inventory 
question about “thoughts about ending your own life,” the Research Assistant will contact Dr. 
Kessler, his covering psychiatrist or, if neither is available, the clinician handling intakes to 
arrange for an evaluation of the Veteran.  
All screening forms will have a study code number but no identifying information. 
Electronic files containing identifiable information will be kept on a secure VA server available 
only to research staff.  
Research data will be identified by code number and will not include names, although 
enrollment records must also be kept and these records will include names. Our study forms have 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
19 
 
been designed to avoid collecting identifiable information; no Protected Health Information 
(PHI) will be collected on study forms. We generally collect only protocol session dates. These 
dates are changed to ‘number of sessions completed’ when data sets are anonymized and 
released to other investigators. Research data, containing no PHI, will be uploaded to a secure 
VA server using SharePoint. SharePoint is a platform that allows users to create and share 
information, documents, and reports through a common secure VA intranet portal. Files 
containing audio-recorded sessions will be uploaded using SharePoint for rating at West Haven, 
after which they will be deleted. Data storage will use the HSR&D PRIME Center’s data storage 
servers (support letter attached).  
Personal identifiers will be retained to obtain VA service use data. After the service use 
data has been extracted, the identifiers will be removed to create files with de-identified data for 
subsequent analyses. Source documents (e.g. Compensation and Pension examinations) are 
stored on VA clinical servers behind a secure firewall, and will not be stored with research data. 
Other forms that contain PHI such as consent forms will be collected by the two site PIs and 
stored in locked file cabinets by the site PI’s until seven years from the last study-regulated 
publication, unless VA regulations stipulate a different date and mode for their destruction.  
All possible precautions will be taken to prevent a breach of confidentiality. Study IDs 
will be assigned at the time of recruitment and replace names and record numbers. Clinical 
enrollment records are kept separate from research data, and are not available without a 
Veteran’s written consent. Participants’ names will appear only on the consent form, HIPAA 
authorization form, and “key” form kept by the site P.I. All paper forms will be stored and 
secured in a locked file cabinet under the jurisdiction of the site P.I.  
At the conclusion of the study, all locator data will be destroyed. The crosswalk file will 
be stored separately from all other study data on VA secure network computers. Data Security 
will be insured via the servers, which all reside completely behind the VA firewall. All systems 
utilize Secure Socket Layer (SSL) technology for internet protocol data transfer, which encrypts 
all network transmissions and transmits only within the VA network backbone. There is no 
public internet access. Another layer of protection for sensitive information is ascribed to the 
system by use of granular access privileges. Only site-specific authorized staff will have data 
access. The Study Coordinator will have the capability of granting access and read-write 
privileges to users. All systems undergo daily backup and 24-hour security.  
The research records will be used to prepare reports that do not include patient 
identification. All research staff and therapists receive annual Good Clinical Practice, Human 
Subjects Protection, and HIPAA training through the Yale Department of Psychiatry. Our data 
collection and management procedures are fully compliant with HIPAA.  
 
2. Confidentiality of Benefits Counseling Website:  
Information to be collected via the internet-based Benefits Counseling program will include 
participants’ responses to the questions, and dates and times that participants’ logged on to the 
site. Participants will not be asked to provide any PHI in the course of their engagement in the 
internet-based Benefits Counseling intervention.  
At the initial meeting for consenting and baseline assessment, participants will be 
provided with a personal user login ID and a temporary password. The participant will be 
required to change the password at initial logon to the website to a personal one that s/he will 
recall. Only the participant and the study staff will have the authority to access the participant’s 
responses to the components of the intervention. If the participant forgets or loses his/her 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
20 
 
password, s/he will be issued a new temporary password and will again be prompted to change it 
upon the next logon to the program. The information collected via the internet-based program 
will be captured in a database (again, containing no PHI, only the participant’s user ID, which 
will not be linked to his/her identifying information or study number in that database).  
Google Analytics and other server software allows for tracking the IP address and/or the 
address of the router via which the Benefits Counseling website was accessed. Thus, there is a 
risk that someone accessing the server hosting the web site could learn that a computer at a 
particular network address accessed the Benefits Counseling site. However, host sites (i.e. 
Heroku) have security policies and procedures to prevent the unauthorized accessing and use of 
this information.  
Patients will be warned that if they use their computers or tablets to access the study 
website, it is possible that someone else who uses the computer or tablet later will be able to tell 
that their computer or phone accessed the Benefits Counseling web site.  
 
3. Confidentiality of Audio recordings (Recording phone calls between Veterans and 
counselors): To assure the confidentiality and protection of participants with respect to audio 
recording, the following steps will be taken:  
• Participants will provide informed consent for audio recording. Participants will be 
allowed to participate in the study and refuse the audio recording. Participants who do 
consent to audio recording will be informed that they have the right to stop recording at 
any time.  
• Audio recordings will be treated as confidential research records. The digital file, labeled 
with a unique study ID code will be uploaded to a secure network VA server, separate 
from other study data. The file name will use study ID codes.  
• Access to audio files will be limited to members of the investigative team including 
trained research raters who will rate the tapes according to the therapist rating scales. A 
separate file will link the file code to the participant’s unique identification code, the 
therapist name, and the date of the session.  
• When the study is completed and the ratings have been properly coded, all audiotapes 
will be deleted according to VA regulations. Participants will be told about this process 
when they provide informed consent.  
• Additional consent will be requested of a patient if that patient’s audiotape may be added 
to materials that will be disseminated, such as a training tape.  
 
4. Training of Study Therapists: The risk of angering or alienating Veterans is very small 
because of the non-confrontational nature of Benefits Counseling. The risk will be further 
minimized by hiring staff who have experience with the study population. Study therapists will 
be trained to adopt a collaborative approach. Study therapists will also be taught the phone 
numbers to call in case of an emergency during a telephone call with Veterans.  
 
5. Training of Research Assistants: Research Assistants will also be taught where the emergency 
facilities are including the location of emergency rooms, VA and public police and crisis 
services. Research Assistants will also receive training in de-escalating agitated Veterans.  
 
D. Data Safety Monitoring Plan:  
1. Data Management Plan 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
21 
 
Careful training of evaluators will precede study initiation. The main instrument on 
which training is needed is the baseline Work-Related Activities timeline-follow-back calendar. 
Training to conduct the Work-Related Activities Calendars will involve at least three supervised 
ratings.  
Our research group will be using REDcap, a free, secure VA Web application that allows 
for the collection and entry of research data. In addition to enabling users to develop surveys and 
databases without additional software, REDcap helps researchers enter, store and manage project 
data. All data entries will have a study code number but no identifying information. Registration 
records including the signed consent form will be stored separately from these study records. 
  
2. Safety Monitoring Plan: The risk to Veterans who do go on to be randomized to Benefits 
Counseling or Control is low (greater than minimal). Veterans presenting with PTSD symptoms 
and other psychiatric conditions for Compensation and Pension examinations have a high 
frequency of medical and psychiatric difficulties. Therefore, changes in medication regimen, the 
presence of medication side effects and symptom exacerbations are not untoward and do not 
constitute adverse events. Anticipated adverse events include complaints about the study 
procedures and the counseling, and any occurrence in which the patient attributes discomfort, 
harm or disability to the study procedures.  
In the clinical trial, assessment for adverse events will be a routine part of every study 
and therapy visit. Any report that a patient required overnight treatment in any facility 
(emergency room, detoxification, and hospitalization) will constitute an SAE, as will any report 
of death or serious disability. Hospitalization or death of another individual due to direct action 
of patient will also be considered an SAE.  
Any SAEs will be immediately reported to Dr. Rosen and, if he is unavailable, to Dr. 
Black. Reports to Dr. Rosen will include a description of the event and a summary of recent 
contacts with the patient. This summary will include any warning signs of the adverse event, the 
patient’s general state and any information suggesting a causal link between study participation 
and the event. Dr. Rosen will report these to the IRB within 3 working days.  
The New England MIRECC’s D.S.M.B. will be the D.S.M.B. for this study. The 
D.S.M.B. will be contacted within 3 days about all significant study-related adverse events.  
The main risks of the study are frustration with the study procedures, frustration with the 
Benefits Counseling website, and someone learning about the Veteran accessing the Benefits 
Counseling website without the Veteran’s permission. Because of the very low risk that either 
therapy would worsen outcomes, no a priori stopping criteria are proposed. Every year, Dr. 
Rosen and the D.S.M.B. will review study information on a standardized form. Board members 
will review recruitment, retention, and follow-up rates for the study-to-date, SAEs and AEs and 
other data as the D.S.M.B. deems necessary.  
Based on this report, each DSMB member will complete a form making one of two 
recommendations: 1) continue recruitment as planned; or 2) schedule formal DSMB meeting 
immediately. If any DSMB member recommends a meeting, this will be scheduled within one 
week, minutes will be kept, the report will be reviewed with the PI, and the committee will vote 
on whether the study should: 1) continue recruitment unchanged; 2) continue with a protocol 
amendment; 3) stop recruiting pending further investigation. If, after this meeting, any DSMB 
member votes to stop recruitment or requests a protocol modification, the Yale IRB will be 
informed.  

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
22 
 
A written summary including this information will be provided to the IRB with the 
protocol renewal.  
 
D. Potential Benefits  
1. Benefits to the Participants: The potential benefits to participants is that they may improve 
their vocational functioning as a result of engaging in the Benefits Counseling or Control 
interventions.  
2. Benefits to Society: The study will provide important information concerning the attitudinal 
and other factors that determine whether Veterans applying for Compensation engage in work 
and related activities. The study findings may suggest modifications to existing VA 
Compensation policies to facilitate engagement in vocational rehabilitation. The studies will 
contribute to the development of a novel format for implementing Benefits Counseling (i.e., an 
internet-based version) that may be appealing and accessible to Veterans applying for VA 
benefits.  
3. Financial Compensation to Participants for Completing Assessments: It is our usual practice to 
compensate participants for studies like this one that require considerable time completing forms 
and ratings that are not a part of routine clinical care. Compensation offsets the inconvenience of 
these procedures. Participant payments will be $60 for the baseline evaluation and $30 for each 
of weeks 4, 12 and 24. This totals $150 per participant. These amounts were chosen to be 
sufficient reimbursement for participants who have competing, paid demands on their time, and 
who come for their initial Compensation Examinations from all over Connecticut, often despite 
no treatment relationship with the VA Healthcare System. In addition, participants will be 
reimbursed 57.5¢ per mile traveled to their residence and from the Compensation examination.  
 
E. Risk/Benefit Ratio  
The study has the potential to further the development of effective treatment for Veterans who 
have psychiatric and vocational problems. Consideration of the risks and benefits mentioned 
above suggests that potential benefits for both the participants and science outweigh the risk of 
minor harm associated with the study procedures.  
 
F. Enrollment of Women and Minorities  
Enrollment of women and minorities will reflect their proportions in the pool of Veterans 
presenting for Compensation and Pension evaluations for psychiatric conditions at the 
Northampton VA. As noted in the Feasibility Section, approximately 8% of recent Veterans 
evaluated were women, 4% were Hispanic, and 12% were African American. We anticipate 
approximately these proportions among the Veterans enrolled in this study.  
Our group has a good record of enrolling representative proportions of women and minorities in 
C&P-based RCTs. In our completed study of face-to-face Benefits Counseling, 17% of Veterans 
were women, 13% were African American, and 17% were Hispanic. 
 
 
 
 
 
 
 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
23 
 
 
 
 
 
 
 
 
 
References 
1. 
Lord, S. Development of an Efficacious Online STI/HIV Prevention Program for Young 
Adults. in College on the Problems of Drug Dependence. 2012. Palm Springs, California. 
2. 
Rounsaville, B.J., K.M. Carroll, and L.S. Onken, A stage model of behavioral therapies 
research: Getting started and moving on from stage I. Clinical Psychology: Science and 
Practice, 2001. 8(2): p. 133-142. 
3. 
Weathers, F.W., et al., PTSD Checklist for DSM-5 (PCL-5), N.C.f. PTSD, Editor. 2013. 
4. 
Derogatis, L.R., Brief Symptom Inventory (BSI): Administration, scoring, and procedures 
manual. 2nd ed. 1992, Baltimore, MD: Clinical Psychometric Research. 
5. 
Murdoch, M., et al., Regional variation and other correlates of department of veterans 
affairs disability awards for patients with posttraumatic stress disorder. Medical Care, 
2005. 43(2): p. 112-121. 
6. 
Jackson, J.C., et al., Variation in practices and attitudes of clinicians assessing PTSD-
related disability among veterans. Journal of Traumatic Stress, 2011. 24(5): p. 609-613. 
7. 
Department of Veterans Affairs. Disability Benefits Questionnaires (DBQs).  [Online]. 
(2014) Cited October 16, 2014. 
8. 
U.S. Department of Veterans Affairs, V.B.A., ANNUAL BENEFITS REPORT, Fiscal 
Year 2011. 2011. 
9. 
Marchione, M. U.S. vets' disability filings reach historic rate. USA Today, 2012. 
10. 
Murdoch, M., et al., Long-term Outcomes of Disability Benefits in US Veterans With 
Posttraumatic Stress Disorder. Arch Gen Psychiatry, 2011. 68(10): p. 1072-1080. 
11. 
Rosenheck, R.A., et al., Outcomes after initial receipt of social security benefits among 
homeless veterans with mental illness. Psychiatric Services, 2000. 51(12): p. 1549-1554. 
12. 
Murdoch, M., et al., Mitigating effect of department of veterans affairs disability benefits 
for post-traumatic stress disorder on low income. Military Medicine, 2005. 170(2): p. 
137-140. 
13. 
Rosenheck, R., L. Frisman, and J. Sindelar, Disability compensation and work among 
veterans with psychiatric and nonpsychiatric impairments. Psychiatric Services, 1995. 
46(4): p. 359-365. 
14. 
Drew, D., et al., Effects of disability compensation on participation in and outcomes of 
vocational rehabilitation. Psychiatric Services, 2001. 52(11): p. 1479-1484. 
15. 
Rosen, M.I., et al., Effect of Social Security Payments on Substance Abuse in a Homeless 
Mentally Ill Cohort. Health Services Research, 2006. 41(1): p. 173-191. 
16. 
Meshberg-Cohen, S., et al., Veterans' attitudes toward work and disability compensation: 
Associations with substance abuse. Addictive Behaviors, 2014. 39(2): p. 445-448. 
17. 
Prepared Statement of Sally Satel, M.D., Resident Scholar, American Enterprise Institute, 
in House Committee on Veterans Affairs. 2011: Washington DC. 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
24 
 
18. 
Rosen, M.I., Compensation examinations for PTSD--an opportunity for treatment? 
Journal of Rehabilitation Research & Development, 2010. 47(5): p. xv-xxii. 
19. 
Tremblay, T., et al., The impact of specialized benefits counseling services on social 
security administration disability beneficiaries in Vermont. Journal of Rehabilitation, 
2004. 70(2): p. 5-11. 
20. 
Council, C.o.V.C.f.P.S.D.I.o.M.a.N.R., PTSD Compensation and Military Service. 2007, 
Washington D.C.: The National Academies Press. 
21. 
Sayer, N.A., et al., Gender Differences in VA Disability Status for PTSD Over Time. 
Psychiatr Serv, 2014. 
22. 
Grahn, B., C. Ekdahl, and L. Borgquist, Motivation as a predictor of changes in quality 
of life and working ability in multidisciplinary rehabilitation: A two-year follow-up of a 
prospective controlled study in patients with prolonged musculoskeletal disorders, in 
Disabil Rehabil. 2000. p. 639-654. 
23. 
Drebing, C.E., et al., The impact of enhanced incentives on vocational rehabilitation 
outcomes for dually diagnosed veterans. Journal of Applied Behavior Analysis, 2005. 
38(3): p. 359-72. 
24. 
 Planning, and Preparedness Office National Survey of Veterans 2010 Final Report. 
Office of Policy of the Department of Veterans Affairs Assistant Secretary for Policy. 
25. 
Sayer, N.A., M. Spoont, and D.B. Nelson, Disability Compensation for PTSD and Use of 
VA Mental Health Care. Psychiatr Serv, 2004. 55(5): p. 589. 
26. 
Tremblay, T., et al., Effect of benefits counseling services on employment outcomes for 
people with psychiatric disabilities. Psychiatric Services, 2006. 57(6): p. 816-821. 
27. 
Frey , W.D., et al., Mental Health Treatment Study Final Report July 2011. 2011, Westat. 
28. 
Sayer, N.A., et al., Development and psychometric properties of the disability application 
appraisal inventory. Psychological Assessment, 2004. 16(2): p. 192-6. 
29. 
Sayer, N.A., M. Spoont, and D.B. Nelson, Post-traumatic stress disorder claims from the 
viewpoint of veterans service officers. Military Medicine, 2005. 170(10): p. 867-70. 
30. 
Sayer, N., et al., A qualitative study of determinants of PTSD treatment initiation in 
veterans. Psychiatry, 2009. 72(3): p. 238-255. 
31. 
Rosen, M.I., et al., Work Outcomes After Benefits Counseling Among Veterans Applying 
for Service Connection for a Psychiatric Condition. Psychiatr Serv, 2014. 
32. 
DiIorio, C., et al., WebEase: development of a Web-based epilepsy self-management 
intervention. Prev Chronic Dis, 2009. 6(1): p. A28. 
33. 
Hester, R.K., D.D. Squires, and H.D. Delaney, The Drinker's Check-up: 12-month 
outcomes of a controlled clinical trial of a stand-alone software program for problem 
drinkers. Journal of Substance Abuse Treatment, 2005. 28(2): p. 159-169. 
34. 
Kiene, S.M. and W.D. Barta, A Brief Individualized Computer-Delivered Sexual Risk 
Reduction Intervention Increases HIV/AIDS Preventive Behavior. Journal of Adolescent 
Health, 2006. 39(3): p. 404-410. 
35. 
Escoffery, C., L. McCormick, and K. Bateman, Development and process evaluation of a 
web-based smoking cessation program for college smokers: innovative tool for 
education. Patient Educ Couns, 2004. 53(2): p. 217-25. 
36. 
Kaltenthaler, E., et al., The acceptability to patients of computerized cognitive behaviour 
therapy for depression: a systematic review. Psychological Medicine, 2008. 38(11): p. 
1521-30. 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
25 
 
37. 
Waller, R. and S. Gilbody, Barriers to the uptake of computerized cognitive behavioural 
therapy: a systematic review of the quantitative and qualitative evidence. Psychol Med, 
2009. 39(5): p. 705-12. 
38. 
Ondersma, S.J., et al., Computer-based brief motivational intervention for perinatal drug 
use. J Subst Abuse Treat, 2005. 28(4): p. 305-12. 
39. 
Steele, R., K.W. Mummery, and T. Dwyer, Development and process evaluation of an 
internet-based physical activity behaviour change program. Patient Educ Couns, 2007. 
67(1-2): p. 127-36. 
40. 
Seal, K.H., et al., Bringing the war back home: Mental health disorders among 103 788 
US veterans returning from Iraq and Afghanistan seen at Department of Veterans Affairs 
Facilities. Archives of Internal Medicine, 2007. 167(5): p. 476-482. 
41. 
Bremner, J., et al., Chronic PTSD in Vietnam combat veterans: course of illness and 
substance abuse. Am J Psychiatry, 1996. 153(3): p. 369-375. 
42. 
House Committee on Veterans' Affairs. [Online]. (2007). Cited May 23, 2007. 
43. 
Speroff, T., et al., Impact of evidence-based standardized assessment on the disability 
clinical interview for diagnosis of service-connected PTSD: a cluster-randomized trial. J 
Trauma Stress, 2012. 25(6): p. 607-15. 
44. 
Rosen, M.I., et al., Racial differences in veterans' satisfaction with examination of 
disability from posttraumatic stress disorder. Psychiatr Serv, 2013. 64(4): p. 354-9. 
45. 
Rosen, M., Compensation examinations for PTSD--an opportunity for treatment? Journal 
of Rehabilitation Research & Development, 2010. 47(5): p. xv-xxii. 
46. 
Serowik, K.L., et al., Developing a benefits counseling website for Veterans using 
Motivational Interviewing techniques. Computers in Human Behavior, 2014. 37: p. 26-
30. 
47. 
Christensen, H., K.M. Griffiths, and L. Farrer, Adherence in internet interventions for 
anxiety and depression. Journal of Medical Internet Research, 2009. 11(2): p. e13. 
48. 
Mander, H., The impact of additional initial face-to-face sessions on engagement within 
an Improving Access to Psychological Therapies service. The Cognitive Behaviour 
Therapist, 2014. 7: p. e1-e8. 
49. 
Johansson, R. and G. Andersson, Internet-based psychological treatments for depression. 
Expert Rev Neurother, 2012. 12(7): p. 861-9; quiz 870. 
50. 
Julious, S.A. and S.D. Patterson, Sample sizes for estimation in clinical research. 
Pharmaceutical Statistics, 2004. 3: p. 213-215. 
51. 
Ware, J.E., Jr. and C.D. Sherbourne, The MOS 36-item short-form health survey (SF-36). 
I. Conceptual framework and item selection. Medical Care, 1992. 30(6): p. 473-83. 
52. 
Stout, R.L., et al., Ensuring balanced distribution of prognostic factors in treatment 
outcome research. Journal of Studies on Alcohol Supplement, 1994. 12: p. 70-5. 
53. 
Saitz, R., Unhealthy Alcohol Use. New England Journal of Medicine, 2005. 352(6): p. 
596-607. 
54. 
Aronson, I.D. and T.C. Bania, Race and emotion in computer-based HIV prevention 
videos for emergency department patients. AIDS Education & Prevention, 2011. 23(2): p. 
91-104. 
55. 
Pedersen, E.R., et al., Concurrent validity of an online version of the Timeline 
Followback assessment. Psychology of Addictive Behaviors, 2012. 26(3): p. 672-677. 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
26 
 
56. 
Burke, B.L., H. Arkowitz, and M. Menchola, The efficacy of motivational interviewing: a 
meta-analysis of controlled clinical trials. Journal of Consulting & Clinical Psychology, 
2003. 71(5): p. 843-61. 
57. 
Elhai, J.D., W. Schweinle, and S.M. Anderson, Reliability and validity of the Attitudes 
Toward Seeking Professional Psychological Help Scale-Short Form. Psychiatry 
Research, 2008. 159(3): p. 320-329. 
58. 
GAO, Milld Traumatic Brain Injury Screening and Evaluation Implemented for 
OEF/OIF Veterans, but Challenges Remain. 2008: Washington DC. 
59. 
Moore, B.A., et al., The Recovery Line: A pilot trial of automated, telephone-based 
treatment for continued drug use in methadone maintenance. J Subst Abuse Treat, 2013. 
45(1): p. 63-9. 
60. 
Ewing, J.A., Detecting alcoholism. The CAGE questionnaire. JAMA, 1984. 252(14): p. 
1905-7. 
61. 
Group, W.A.W., The Alcohol, Smoking and Substance Involvement Screening Test 
(ASSIST): development, reliability and feasibility. Addiction, 2002. 97(9): p. 1183-94. 
62. 
Humeniuk, R., et al., Validation of the Alcohol, Smoking And Substance Involvement 
Screening Test (ASSIST). Addiction, 2008. 103(6): p. 1039-47. 
63. 
Blanchard, E.B., et al., Psychometric properties of the PTSD Checklist (PCL). Behaviour 
Research & Therapy, 1996. 34(8): p. 669-73. 
64. 
Forbes, D., M. Creamer, and D. Biddle, The validity of the PTSD checklist as a measure 
of symptomatic change in combat-related PTSD. Behaviour Research & Therapy, 2001. 
39(8): p. 977-86. 
65. 
Monson, C.M., et al., Change in posttraumatic stress disorder symptoms: do clinicians 
and patients agree? Psychological Assessment, 2008. 20(2): p. 131-8. 
66. 
Monson, C.M., et al., Cognitive processing therapy for veterans with military-related 
posttraumatic stress disorder. Journal of Consulting & Clinical Psychology, 2006. 74(5): 
p. 898-907. 
67. 
Drake, R.E., et al., Assisting Social Security Disability Insurance beneficiaries with 
schizophrenia, bipolar disorder, or major depression in returning to work. Am J 
Psychiatry, 2013. 170(12): p. 1433-41. 
68. 
Cook, J.A., Depression, disability, and rehabilitation services for women. Psychology of 
Women Quarterly, 2003. 27(2): p. 121-129. 
69. 
Miller, W.R., et al., Motivational Enhancement Therapy Manual: A Clinical Research 
Guide for Therapists Treating Individuals With Alcohol Abuse and Dependence. Vol. 2. 
1992: U.S. Department of Health and Human Services. 
70. 
Williams, E.C., et al., Do brief measures of readiness to change predict alcohol 
consumption and consequences in primary care patients with unhealthy alcohol use? 
Alcoholism: Clinical & Experimental Research, 2007. 31(3): p. 428-35. 
71. 
Rosenheck, R.A. and A.S. Mares, Implementation of supported employment for homeless 
veterans with psychiatric or addiction disorders: Two-year outcomes. Psychiatric 
Services, 2007. 58(3): p. 325-333. 
72. 
Carroll, K.M., et al., A general system for evaluating therapist adherence and 
competence in psychotherapy research in the addictions. Drug & Alcohol Dependence, 
2000. 57(3): p. 225-38. 
73. 
Desai, R.A., et al., Suicide among discharged psychiatric inpatients in the Department of 
Veterans Affairs. Military Medicine, 2008. 173(8): p. 721-8. 

Marc Rosen, MD 
Remotely-Delivered Benefits Counseling for Service Connection Applicants 
Version Date: 4/9/19 
27 
 
74. 
Harpaz-Rotem, I., et al., Tracing the flow of knowledge: geographic variability in the 
diffusion of prazosin use for the treatment of posttraumatic stress disorder nationally in 
the Department of Veterans Affairs. Archives of General Psychiatry, 2009. 66(4): p. 417-
21. 
75. 
Little, R.J.A., A test for missing completely at random for multivariate data with missing 
values. Journal of the American Statistical Association, 1988. 83: p. 1198-1202. 
76. 
LM, C., S. JL, and K. C, A comparison of inclusive and restrictive strategies in modern 
missing data procedures. Psychological Methods, 2001. 6: p. 330-351. 
77. 
Rubin, D.B., Multiple imputation after 18+ years. Journal of the American Statistical 
Association, 1996. 91(434): p. 473-489. 
78. 
Schafer, J.L. and M.K. Olsen, Multiple imputation for multivariate missing-data 
problems: A data analyst's perspective. Multivariate behavioral research, 1998. 33(4): p. 
545-571. 
79. 
Schafer, J.L., Analysis of incomplete multivariate data. 2010: CRC press. 
80. 
Schafer, J.L., Analysis of incomplete multivariate data. 1997, New York: Chapman & 
Hall.