Relational Agent to Improve Alcohol Screening and Treatment in Primary Care: RCT
Phase: NA
Conditions: Unhealthy Alcohol Use, Alcohol Use Disorder, Alcohol Disorders
Sponsor: VA Office of Research and Development

IIR 11-334 
Relational Agent for Alcohol Screening and Treatment 
NCT02030288 
 
April 10, 2017 

Relational Agent to Improve Alcohol Screening and Treatment in Primary Care: RCT 
PI: Steven Simon, MD 
Overview: We would like to amend this study by adding a protocol for Aims 2 and 3 of our grant. Aim 2 
involves conducting a RCT to evaluate the effectiveness of the Relational Agent for Screening, Brief 
Intervention and Referral to Treatment (SBIRT) to improve the management of risky alcohol use in primary 
care. To the extent possible, we are integrating the program into the practices of the Primary Care Clinic. Aim 3 
is a formative evaluation of Veterans’ experiences with the Relational Agent. We present the full RCT protocol 
first, then the formative evaluation.  
To facilitate review we are again attaching the Specific Aims and Research Plan for the grant proposal, 
although it was attached to Aim 1 when the initial review was conducted.  
Specific Aim 1: Tailor the Relational Agent Intervention to the Veteran population.
Already approved.  
Specific Aim 2: Conduct a RCT Comparing Two Different Treatments; Treatment as Usual versus
Treatment as Usual plus the Relational Agent for unhealthy alcohol use. 
(1) Rationale
(a) Statement of the Problem.  As stated in Aim 1, there is ample evidence that screening and brief
intervention for alcohol problems in primary care is effective in reducing high risk alcohol use. There is also 
evidence that in the VA (as elsewhere) the implementation of these procedures in primary care is less than 
adequate. The purpose of this project is to develop and evaluate the effectiveness of a computerized program 
to deliver the screening and, brief intervention, and referral for specialty care. In Aim 2, we are using the 
relational agent program developed in Aim 1 in a randomized clinical effectiveness trial. We will randomize 
Veterans who screen positive for unhealthy alcohol use in primary care to Treatment as Usual (TAU) or TAU 
plus additional treatment with the Relational Agent (TAU+RA). We will measure outcomes including quantity 
and frequency of alcohol use, rates of brief intervention, and referral to specialty treatment at three months 
following randomization. We will also measure Veterans’ satisfaction with treatment for alcohol problems in 
primary care.  
(b) Hypotheses or Key Question.
The primary hypothesis is that 1) the Relational Agent will be more effective than TAU in reducing the 
quantity (drinks per day) and frequency (number of drinking days per week) of alcohol consumption.  
The two secondary hypotheses are that 2) the Relational Agent will be more effective than TAU in 
improving rates of brief intervention; and 3) the Relational Agent will be more effective than TAU in improving 
rates of referral. Additional exploratory hypotheses are listed in the Analysis section. 
(2) Background, Significance, and Relevance to Veteran’s Health
Please see Aim 1 protocol and grant proposal.
(3) Work Accomplished
Please see grant proposal for pilot work. So far in this study we have completed part of Aim 1. We
interviewed 20 veterans and collected information on their preferences for the appearance of the relational 
agent, as well as some information on how the setting should look. We are in the process of collecting 
feedback on the sections of the relational agent program that we have  programmed so far and making final 
adjustments to the program. 
(4) Work Proposed
1

Methods 
Setting and Population 
We will recruit a total of 180 Veterans from primary care clinics, both men and women. We expect to recruit 
approximately 10-40 participants per month over 11 months, finishing the 3 month follow-up visits in 14 months 
from the start of recruitment. The study will be conducted first in the Jamaica Plain primary care clinic (JP-PCC) 
of the VA Boston Healthcare System. If we are unable to meet our recruitment goals, we will also attempt to 
recruit at the West Roxbury primary care clinic (WRX-PCC) or Brockton primary care (BR-PCC). In any case, 
we will exclude the veterans who participated in Aim 1.  
Recruitment 
We are working collaboratively with the team leaders and clinic staff to identify a process by which 
Veterans who screen positive on the AUD-C can be referred to the research assistants without disruption to the 
workflow of primary care encounters. The usual practice in this setting is that the health technician or LPN 
administers the AUD-C and informs the physician if the Veteran has a positive screen. The physician (or other 
responsible primary care clinician) is then responsible for delivering the brief intervention. Per current practice, 
referral to Mental Health for alcohol problems is routinely made by computerized order to one of the specialty 
substance abuse clinics.  
For this study, clinic staff will fill out the “Permission to Contact for Research” form with the veteran and 
pass it to the research assistant in clinic, or hold it in a predetermined place for the research assistant to pick 
up. If possible the research assistant will interview the patient during the visit. We will aim to have participants 
complete the baseline interview (and, for Veterans ultimately allocated to the Relational Agent arm, the 
Relational Agent interaction) prior to seeing their clinician for the scheduled clinical encounter; however, if 
workflow requires the Veteran to complete his encounter first, the research assistant will meet the Veteran 
immediately following the visit to complete the baseline interview. If necessary, the veteran will be scheduled for 
a return visit to finish the baseline interview and/or session 1. The research assistant will have a private space to 
conduct interviews and have the veteran use the Relational Agent program.  
In addition, each week, clinic staff will provide us with a list of patients who are due for an AUD-C 
screening.  We depend on staff to recruit and refer patients to us, and it has been slow going. 1. We would like 
to use CPRS to access the medical files of anyone who was supposed to be screened with the AUDIT-C that 
week. If the person scored positive on the AUDIT-C we will send them a letter containing a brief description of 
our study and an opt out card. If we do not receive an opt out card in the two weeks following the letter, our 
study team will try and contact the individual by phone to see if they would be interested in participating in the 
study. If they are interested, we will conduct the screening on the phone and, if eligible, we will schedule them 
for a baseline interview in person. 2.  
 
Study flyers have been placed around the hospital with the study’s telephone number around the 
hospital.  To include veterans who call our study line directly, we would check their eligibility by directly administering
the AUD-C over the phone. This AUD-C will not go into CPRS; it will stay in study records.  Then we will follow our 
standard eligibility requirements and procedures.  
Eligibility Criteria 
In general, we approach this study as a pragmatic clinical trial, such that we intend to maximize inclusion to 
reflect actual clinical practice and to demonstrate effectiveness, rather than efficacy, of the intervention.  
Inclusion criteria include: (1) having a primary care provider at VA Boston PCC; (2) having a positive screen on 
the AUD-C; (3) drinking above CDC guidelines in the past 30 days; (4) willing to provide us with at least 1 “locator 
person” ( and preferably 2 locator persons) who always knows their whereabouts (explanation on p. 5). 
Exclusion criteria are: (1) Veterans who do not have a valid telephone number and mailing address, and who 
expect to move in the next 6 months. This information is needed to ensure 3-month follow-up contact and 
completion of Aim #3; (2) Veterans who report less than 14 drinks per week or less than 5 drinks on any one 
occasion in the past 30 days when screened on the Quick Drink Screen (described in Measures). The purpose 
of adding this screening is to account for the fact that the AUD-C measures Veterans’ alcohol consumption over 
the past year, and it is possible for Veterans to screen positive on the AUD-C but to have stopped drinking 
2

 
during the past 30 days. We will use the Quick Drinking Screen (see Measures) to exclude these Veterans who 
screen positive on the AUD-C but are not consuming unhealthy quantities of alcohol at the time of the study, 
because the primary study outcomes include quantitative measures of alcohol consumption.  These Veterans 
may not be currently in need of brief intervention because they have recently cut down their drinking. (3) 
Veterans who have participated in substance abuse treatment in the past 30 days because they are already 
receiving care. 
The table below summarizes the eligibility criteria. 
We note that the study will not exclude patients with mental illness, traumatic brain injury, or other mental or 
neurological conditions.  Second, we will measure health literacy to ensure the program can be used by any 
Veteran, no matter their familiarity with health issues. 
Measures 
 
The study measures will be derived from data collected at the interviews, as well as from the VA’s 
computerized patient record system, CPRS. The measures are summarized in the table below and attached in 
Appendix A. Data will be entered into REDCap, a free, secure Web application that facilitates the collection 
and entry of research data. User-friendly electronic data capture (EDC) tools enable users to quickly develop 
surveys and databases from conception to production on the Web without additional software requirements. 
This tool helps researchers enter, store, and manage their project data in a systematic manner. REDCap is a 
VA Intranet Web application; surveys can only be administered to respondents while logged into the VA 
Intranet. 
Research Measures and Schedule of Use 
Measures 
Screening 
Baseline 
Session 2 
(intervention 
group only) 
3 Month 
Follow-up 
AUD-C (from health technician) 
X 
Quick Drink Screen 
X 
X 
Demographics and other 
X 
REALM-SF (health literacy) 
X 
MINI-Alcohol 
X 
SIP 
X 
X 
System Usability Scale 
X 
ECA Questionnaire 
X 
Satisfaction with alcohol treatment in primary 
care (1 item) 
X 
CPRS data extraction 
X 
Demographics and other. Age, gender, ethnicity, branch of military service, and other descriptors or questions of 
relevance to this study (e.g., previous substance abuse treatment) are included. See Appendix A for document. 
 Study Eligibility Criteria 
Inclusion Criteria 
Exclusion Criteria 
AUD-C ≥ 5 
Participation in Aim 1 of the study 
Receive primary care  where recruited 
Drank <14 drinks/wk and <5 drinks on one occasion in past 30 days 
Drinking more than guidelines in past 30 days 
Unsure of phone and address over next 6 months 
Stable phone and address over next 6 months 
Alcohol treatment in last 30 days 
Supply at least 1 (up to 2) locator person(s) 
Unwilling to supply locators. 
3

Quick Drink Screen (QDS). The QDS will be used to assess quantity and frequency of drinking over the past 30 
days. When compared with the Timeline Followback over a one-year period, intra-class correlations on all five 
variables ranged between 0.66 and 0.82, which indicates that the QDS may be an expedient way of gathering 
summary drinking data.  
Rapid Estimate of Adult Literacy in Medicine—Short Form (REALM-SF). The REALM-SF is a 7-item word 
recognition test that provides a valid and quick assessment of health literacy.   
Alcohol Use Disorders Identification Test (AUDIT): The AUDIT is a 10-item self-report screening tool that 
produces a total score from 0-40. The AUDIT was developed and used by World Health Organization in 
multinational trials of brief interventions, demonstrates good sensitivity and specificity in many populations and 
is generalizable across populations. The first three items are used in the VA and called the AUD-C. The AUD-C 
has been demonstrated to have a high rate of agreement with scores on the full AUDIT. The AUD-C only asks 
about consumption; the full AUDIT has questions that reflect hazardous drinking and alcohol abuse (items 4-6) 
and questions that reflect likely alcohol dependence (items 7-10). 
Short Index of Problems-Recent Consequences Form (SIP). The SIP is a short version of the Drinker Inventory 
of Consequences that measures negative consequences of alcohol use in the past three months. This 
measure yields an overall score and scores for five sub-scales: Physical, Interpersonal, Intrapersonal, Impulse 
Control, and Social Responsibility. This will allow us to determine if there have been additional consequences 
since baseline or whether problems have diminished. The SIP has published norms.
System Usability Scale (SUS) is a widely used simple, ten-item questionnaire (Likert scale) giving a global view 
of subjective assessments of usability. 
Satisfaction with Alcohol Treatment in Primary Care. Consumer Assessment of Healthcare Providers and 
Systems (CAHPS) has emerged as a standard for measuring satisfaction with health care delivery, including 
primary care. We will use a single item, modeled on the CAHPS, to assess satisfaction: “On a 0 to 10 point 
scale, with 0 being the worst care you can imagine and 10 being the best care you can imagine, how would you 
rate the care you have received in the last 3 months from the VA for alcohol problems?”.  
ECA Questionnaire. 
This measure assesses participants’ satisfaction with and trust of the Relational Agent. It has been used in Dr. 
Bickmore’s lab for all of his relational agent studies. 
Multiphasic  International Neuropsychiatric Inventory. 
The MINI is a widely used structured diagnostic interview designed to assess psychiatric disorders in 
clinical and research settings where a long diagnostic interview is not possible. The latest version of the MINI, 
7.0, yields diagnoses that are compatible both with the International Classification of Diseases 10th Revision 
(ICD-10) and the Diagnostic and Statistical Manual of Mental Disorders, Fourth and Fifth Edition (DSM-5).   A 
comparison of the MINI to the Composite International Diagnostic Interview showed good to very good 
agreement for most diagnoses and, more specifically, had very good agreement for alcohol dependence 
diagnoses with a kappa of 0.82. In addition, inter-rater and test/retest agreement for the MINI was high for 
alcohol dependence (kappa = 1.0 and 0.86, respectively) and alcohol abuse (kappa = 0.9 and 0.85, 
respectively). The MINI Alcohol Scale, which is the only part of the MINI we will use, takes approximately 7 
minutes to administer. 
Study Arms 
Treatment as Usual. Treatment as Usual will be the standard delivery of health care in PCC. Usual care 
includes annual screening for alcohol problems using the 3-item AUD-C (see description of AUDIT in Measures, 
below), prompted by clinical reminders in the electronic health record. In the context of this randomized trial, 
4

 
Veterans allocated to Treatment as Usual will proceed with their scheduled clinical encounter with the primary 
care clinician (PCP). Because each of these participants will have screened positive on the AUD-C, the clinic 
staff who completed the AUD-C screening will notify the PCP of the positive screening. This notification occurs 
either by verbal communication or by passing of a written (paper) notification. In addition, the positive screening 
score on the AUD-C generates a new clinical reminder that prompts the clinician to complete a brief 
intervention regarding alcohol use. In the course of Treatment as Usual, the clinician may elect to refer the 
Veteran for specialty care (e.g., Mental Health) for assessment and management of alcohol-related problems. 
Relational Agent Intervention. The Relational Agent is an on-screen human-like representation that uses 
gestures, words and phrases based on psychology and communication science to engage the Veteran in a 
simulated face-to-face dialogue. The Relational Agent talks using synthetic speech and animation, and the 
Veteran responds by selecting from a multiple-choice list of phrases on a touch-screen tablet. 
The Relational Agent will be delivering the same kind of care as the PCP would ideally deliver. The 
Relational Agent will be providing a form of brief Motivational Interviewing and recommending a follow-up visit 
in one month. The Relational Agent (whose name is Laura) will commence with a brief assessment, which will 
be administered in order to provide personalized feedback to the Veteran. The assessment includes the Quick 
Drinking Screen (QDS) and the full Alcohol Use Disorders Identification Test (AUDIT) (see Measures, below). 
Although the Veteran will have completed the AUD-C in the initial interaction with the primary care health 
technician, we will use the full AUDIT to enable us to provide the Veteran feedback about other symptoms of 
alcohol problems. We are also using the QDS to give feedback on drinking levels and how they compare to 
national guidelines. The feedback derived from these measures is a standard method of delivering feedback 
and using drinking problems they bring to light to try and motivate the person to change.   
Next Laura will take the Veteran through the decisional balance exercise (pros and cons of change). 
Importance, confidence, and readiness rulers and responses to those will be used. At the end, Laura will ask 
for a commitment to change. If a Veteran has an alcohol use disorder, as determined below, the Veteran will be 
offered treatment through referral to specialty substance abuse clinics. A personal feedback report will be 
printed out for the Veteran that includes a summary of findings and patient input. A clinic report will also be 
generated that will be used to write a progress note and as data for a referral (see Appendix B for all reports).  
In the second visit, one month later, Laura will again ask Veterans about their drinking, and give them 
feedback about how their current drinking compares to what they reported earlier. If the Veteran has reduced 
consumption, they will be congratulated. If they have not changed drinking patterns, Laura will give the Veteran 
the option to revisit sections of the motivational intervention from session 1. If they haven’t changed their 
drinking, or if they did not follow-up on a previous referral, the Veteran will again be offered a referral to 
specialty care. Veterans will again receive printed feedback and a clinic report will be prepared.  
Procedures 
Screening Interview 
Clinic staff will refer patients who score 5 or above on the AUD-C, which is the VA-established cutoff for 
alcohol-related problems. Research assistants will see the patient while they are onsite for the primary care visit 
if at all possible, or set up a time for a return visit. The elements of this interview are as follows: 
1.
Brief overview description of the study, noting the following key points (see script in Appendix B):
(a)
The Veteran may be eligible, depending on the responses to the screening items.
(b)
If eligible, the Veteran would be randomly allocated either to two visits with
a computer-based system of brief intervention for individuals who report unhealthy alcohol
consumption, or not. All Veterans still receive Treatment as Usual.
The Veteran would be asked to complete a baseline interview at the time of the visit, and by
telephone in 3 months, regardless of the study arm to which he or she is allocated.
(c)
Upon completion of each interview (baseline and 3-month), the Veteran will receive $50 in
appreciation of his or her participation.
2.
Completion of the Quick Drinking Screen (QDS). The results of the QDS will be compared with the
Center for Disease Control and Prevention (CDC) guidelines to ascertain if the Veteran is engaging in
risky alcohol use (more than 5 drinks on any one occasion or more than 14 drinks a week for men, and
5

more than 4 drinks on any one occasion or more than 7 drinks a week for women in the last 30 days). 
3.
Ascertain whether the veteran meets criteria for having a stable phone and address for the next 6
months.
4.
Ascertain whether the veteran is willing to provide us with contact information for at least one locator.
Veterans who express interest in participating and meet the 3 criteria above will be asked to participate.
The research assistant will read the ICF with Veterans and obtain signed informed consent. (See Human 
Subjects for details of this process.) Upon completion of the signed informed consent process, we will ask 
whether they have been in substance abuse treatment in the last month. If they have not been in treatment in 
the last 30 days they are eligible and the rest of the baseline interview continues. In addition to the veteran’s 
telephone and mailing address, we will ask for contact information for 2 “locators”, family or friends who always 
know how to find the veteran. Dr. Rubin has done this many times before, as substance-abusing participants in 
general tend to have chaotic lives and move around a lot or change phone numbers. We will ask the veteran to 
address an envelope to the locator, sign the locator letter (in Appendix B), and we will insert the letter to locators 
letting them know they may be contacted. We will only mail the letter if we cannot locate the participant for a 
follow-up interview. We have found this helps facilitate cooperation when we call.   If they have not been in 
treatment, and are willing to provide at least 1 locator, they are eligible for the study. We will collect all contact 
information at this point.   
The baseline interview will commence immediately, if possible. If the veteran cannot stay, we will schedule the 
baseline to occur as soon as possible. 
See the Figure below for flow of patients through the study. 
Primary Care 
Clinic 
Appointment
Intake by health 
tech/LPN, incl AUD-C 
if due
AUD-C negative
AUD-C positive
Yes
Permission
 to contact?
No
Eligibility Screening 
by Res. Asst (in 
person, phone)
Eligible so 
far?
Yes – consent 
and baseline
No
Randomize
Treatment as 
Usual
Treatment as Usual 
+Relational Agent
Subst. abuse 
treatment in last 30 
days?
Yes - ineligible
Baseline Interview 
The research assistant will administer the baseline interview and schedule the 3-month telephone follow-up 
with the Veteran. We will collect demographic information and other items – see Demographics Form, attached 
6

 
This interview will assess the Veteran’s severity of alcohol use in greater detail and with greater precision 
than the AUD-C by employing the MINI. Although we could use the recommended cutoffs on the AUDIT, the 
MINI is a diagnostic measure that will provide greater confidence that Veterans are classified correctly. In 
addition, the research assistant will administer the SIP, which measures negative consequences of alcohol use. 
Finally, the research assistant will administer the Realm-SF. The baseline interview will take about 45 minutes, 
including the consent process. Veterans receive $50  in appreciation for their participation in completing the 
baseline interview. 
Stratification 
Based on the results of the MINI Alcohol Scale, the research assistant will stratify each participant as 
having either risky drinking or an alcohol use disorder.  Veterans stratified to the risky drinking group reflect a 
population of individuals among whom brief interventions have been most frequently demonstrated to be 
efficacious. In contrast, the efficacy of brief intervention among individuals with Alcohol Use Disorder has rarely 
been studied. Because we undertake this study as a pragmatic clinical trial, we do not want to exclude these 
individuals from participation; therefore we are stratifying our sample for two purposes. First, stratification will 
enable us to examine a priori the effectiveness of the Relational Agent on primary and secondary outcomes 
separately in these two strata. As described below, we hypothesize that the effect of the intervention will vary 
between the two strata.  Second, stratification allows tailoring of the intervention to each stratum; we will 
configure the Relational Agent always to recommend referral for further evaluation and treatment to Veterans in 
the Alcohol Use Disorder group, whereas it will only recommend referral to Veterans meeting certain response 
criteria within the risky drinking group (no desire to change, wish to discuss drinking with a person instead). 
After the research assistant determines the stratum, the research assistant and Veteran proceed to 
randomization.  
Randomization 
Following the baseline interview and stratification, the research assistant will randomize Veterans to 
Treatment as Usual or Treatment as Usual plus Relational Agent Intervention. As described above, Veterans 
will be stratified by the severity of alcohol use disorder. Within strata, block randomization with variable block 
sizes will be used to ensure that treatment groups balance over time. The block sizes will be randomly varied 
and of sufficient size to minimize the ability of any study staff or Veterans to guess the next random allocation 
assignment. Randomization assignments will be generated using computer-generated random numbers and 
will be available immediately to the research assistant upon entering the Veteran-participant’s information and 
responses from the baseline interview. We will use the randomization procedure available in RedCAP.  After the 
research assistant determines the random allocation assignment of the Veteran-participant, the Veteran will be 
directed either to proceed to his scheduled clinical encounter (Treatment as Usual) or will be introduced to the 
Relational Agent Intervention.  
Relational Agent Intervention-procedure 
When a Veteran is allocated to Relational Agent + Treatment as Usual, the research assistant will first 
enroll the participant into the Relational Agent system (entering the participant’s ID, gender, given name, and 
presence/absence of AUD), introduce the Veteran to the Relational Agent, explain briefly how the Relational 
Agent works, then allow the Veteran to complete the Relational Agent program in privacy using headphones. 
We are using AUDIT scores and previous substance abuse treatment in addition to the MINI as 
conservative indicators of the need for specialty treatment. Negative results on these measures will result in a 
brief intervention only. A positive result for alcohol abuse or dependence from the MINI during the baseline 
assessment, previous substance abuse treatment, or a score higher than 20 on the AUDIT will trigger a referral 
to specialty treatment at the end of the intervention.  
 When session 1 is over, the research assistant will print out the patient feedback report for the 
participant, and make sure they are scheduled for session 2 and the 3 month research interview. The patient 
feedback report will contain information about the participant’s drinking and their list of pros and cons (see 
Appendix B). The clinic report will contain information about the participant’s drinking, and whether or not a 
referral was agreed upon. The information from the clinic report will go into a progress note in CPRS, and will be 
7

 
used for referral if needed. The research assistant will check the clinic report to see if a referral is indicated, and 
contact Dr. Simon or Dr. Rubin if a consult (referral) is indicated. Dr. Simon or Dr. Rubin will speak with the 
participant on the phone to gather information for the consult and place the consult for the participant. If, in the 
unlikely event that either Dr. Rubin or Dr. Simon are not available, the research assistant will contact the 
postdoctoral fellow working on this project, or walk the participant down to the offices of the Primary Care 
Mental Health staff for assistance. If there are any concerns about suicidality or homicidality from the telephone 
consult process, the research assistant will walk the participant over to Primary Care Mental Health staff for 
assistance. If they are not available the research assistant will alert the primary care team and hand off the 
patient to one of the PCC staff who can bring the patient to urgent care. See “Human Subjects – Protection 
Against Risk” for more detail. 
When the participant returns for session 2, the research assistant will again set up the computer for the 
participant and allow them to use the program in privacy, using headphones. At the end of the session, the 
research assistant will print out the patient feedback report, confirm the 3 month follow-up appointment, and 
check the clinic report for instructions. If a referral is needed the research assistant will follow the same 
procedure as above. Veterans receive $10 in for completing questionnaires after session 2. 
The research assistant will also ask the Veteran to fill out the System Usability Scale and the ECA 
Questionnaire. 
Three-Month Follow-up Interview 
The research assistants will conduct the 3-month follow-up interview by telephone with each enrolled study 
participant.  Each of these telephone interviews will take approximately 15 minutes and will include completion 
of the QDS and SIP. The research assistant will also ask whether the Veteran received brief intervention or 
referral for alcohol problems at the index primary care visit or in the intervening 3-month period; this 
ascertainment will complement the documentation of brief intervention and referral that may be present in the 
computerized patient record system (CPRS).  In addition, the research assistant will ask about Veterans’ 
satisfaction with care for alcohol problems in PCC. Veterans receive $20 in appreciation for their participation in 
completing the 3 month follow-up interview. 
Follow-up Procedures 
To minimize loss-to-follow-up and maximize participation, we will send each participant a reminder letter 
approximately one week prior to each interview. We also leave a reminder call a few days before. Dr. Rubin will 
train the research assistants to conduct the in-person and 3 month telephone interviews and will monitor their 
quality and accuracy. The research assistants will follow a written protocol for conducting these calls, including 
a script (see Appendix B) for how to interact with automated telephone answering systems (machines or voice 
mail) and how to confirm the identity of the Veteran to ensure privacy and confidentiality. Based on our prior 
studies, we developed a protocol for multiple call attempts if the Veteran is unreachable, followed by up to 3 
letters attempting to reschedule the missed appointments. If we are still unable to contact the participants we 
will send out the locator letters and follow them up with a phone call to the locator person(s). We will also check 
CPRS for changes in phone numbers or addresses, and to check if a participant is scheduled for a visit in the 
near future where we can speak with them.  Data from each of these interviews will be entered into RedCAP on 
a study laptop computer (equipped with all required encryption and passwords) and eventually converted to 
Statistical Package for the Social Sciences Software (SPSS) data file format for statistical analysis. 
Review of the Computerized Patient Record System (CPRS) 
Upon completion of the 3-month follow-up interview, we will review the electronic health record of each 
enrolled Veteran-participant to conduct a structured abstraction of key data elements for the statistical analysis. 
Dr. Simon, Dr. Rubin and study team members will develop a data collection form to guide the data abstraction 
process. Table 3 shows a partial list of the data elements to be abstracted from CPRS. We will closely follow 
the methodology of Hawkins et al, who developed and implemented a structured process to review and  
abstract data on brief intervention and referral for alcohol treatment from CPRS.7 In that study, trained medical 
record reviewers used computerized algorithms to ensure reliable and accurate data abstraction. Their 
8

 
reviewers recorded documented brief intervention with the following standardized instructions: “At any time 
since the most recent alcohol screening, does the record document any of the following components of brief 
alcohol counseling for past-year drinkers? Indicate all that apply and the date counseling was noted in the 
record: 
Patient drinks within recommended limits per self-report, (2) Advice to abstain, (3) Personalized 
counseling regarding relationship of alcohol to the patient’s specific health issues, (4) General alcohol-related 
counseling (not linked to patient’s issues), (5) Patient advised to drink within recommended limits, (6) No 
alcohol counseling documented.” 
We will use these instructions and, as done by Hawkins et al, we will provide reviewers with examples 
that satisfy each of the above components of brief alcohol counseling. Similarly, we will follow the approach of 
Hawkins et al in ascertaining referral for alcohol problems. Referral to alcohol treatment will be defined as a 
documented discussion in CPRS of referral for VA or non-VA services to address alcohol misuse or a 
scheduled appointment for VA services. We will train reviewers to identify and abstract information of any 
referral to VA or non-VA alcohol treatment services and document type of referred services (e.g., addiction 
specialty care, non-VA primary care services?) and date of referral appointment. Alcohol treatment services 
will be defined as any of the following with documentation indicating the purpose of the service was to evaluate 
or address alcohol misuse: VA inpatient or outpatient specialty-care addiction services, VA alcohol-related 
telephone counseling, or referral to non-VA facility for alcohol treatment. Reviewers will also be instructed to 
indicate if a discussion of a treatment referral was documented in CPRS without actual referral being made. 
For Veterans who received appointments to address alcohol misuse, reviewers will ascertain the date 
Veterans attended their referral appointment. 
In addition, we will abstract and record other Mental Health diagnoses, especially Substance Use 
Disorders, PTSD, and TBI, to describe the sample and examine any subgroup differences. 
Dr. Simon, an experienced primary care physician and active user of CPRS, will train the research 
assistants to conduct the CPRS review and abstraction. The research assistants will be blinded to the 
treatment status of the Veterans. Dr. Simon, who will also be blinded to the treatment status, will oversee the 
data abstraction and ensure its quality and accuracy. Abstracted data will be entered into the REDCAP 
database for eventual conversion to SPSS format for analysis. 
Study Data Management 
Telephone communications. We will have one telephone line where participants or clinic staff can call 
and leave messages. Research staff will answer this phone or return calls promptly. 
Data collection and storage. Research interviews and other relevant data will be collected via the 
REDCap program behind the VA firewall and stored on a VA server. REDCap allows the assignment of 
different levels of access privileges and audit trails, thus ensuring proper data management and minimizing 
mistakes. For instance the Project Manager can be the only one who is allowed to edit data, while RAs can be 
allowed to enter and download data only. At the time of this IRB amendment (2/9/16), we were given 
permission to use REDCap while it is under review for VA Research by VACO. If REDCap is disallowed, we 
will identify and propose alternative systems.   
The Relational Agent software is installed on VA tablet/laptops and the database for the program will be 
on a VA server. The software installed on the tablets includes custom software created by Northeastern 
University computer scientists, Unity software used for animation, and XXAMP, a web application. Also, the 
system uses MySQL database on both the tablets and the VA server. It is installed on the tablets for local 
storage, and as insurance against being unable to run the program if wifi connection is unavailable. It gets 
backed up to the VA servers. Both the database on tablets and VA server and the transmission to the server 
can be encrypted.  
Study Design for Aim 2 
This study is a mixed design, with Alcohol Severity Groups and Treatment Arms as between-groups 
variables and Time of Testing (Baseline vs. 3-Month Follow-up) as a repeated measure variable. The main 
focus of this initial RCT is to compare Treatment as Usual (TAU) to TAU plus the Relational Agent Intervention 
9

 
(TAU+RA) on (a) alcohol consumption, (b) rates of brief intervention, and (c) rates of referral to treatment. We 
will use a stratified randomization scheme, stratifying on whether a Veteran is a risky drinker or has the more 
severe designation of an alcohol use disorder, to ensure that the two study groups balance on severity.  
Veterans allocated to Treatment as Usual will continue to receive all usual care as delivered in the PCC. 
Veterans allocated to Treatment as Usual plus Relational Agent Intervention will interact with the Relational 
Agent as soon as possible following the baseline assessment, and still receive usual care. 
Alcohol use and related behaviors will be assessed by in-person survey at baseline and by telephone 
survey at 3 months after randomization. Principal intervention effect analyses will be performed at 3 months. 
The primary dependent (outcome) variables are number of drinking days per week (frequency) and average 
drinks per drinking day (quantity). Secondary dependent variables are percent receiving brief intervention and 
percent referred for further evaluation or treatment of alcohol problems.  
Independent Variables 
The two independent variables are the experimental condition, Treatment as Usual vs. Treatment as Usual 
plus Relational Agent Intervention, and the severity of alcohol use, dichotomized as Risky Drinking vs. Alcohol 
Use Disorder. The experimental condition will be assigned by random allocation (see Section 4.e.7). 
The severity of alcohol use variable, which will be used to allocate Veteran-participants into two strata prior 
to randomization, will be ascertained by the baseline completion of the MINI. 
Dependent Variables 
The primary dependent variables (i.e., the main study outcome measures) are number of drinking days per 
week and average drinks per drinking day. Because the Relational Agent will target reduction of both the 
frequency (number of drinking days per week) and the quantity (average drinks per drinking day) of alcohol 
consumption, each of these measures will be considered separately as primary outcome measures. These 
variables are measured by the QDS. 
Secondary dependent variables include completion of brief intervention (yes/no) and referral for 
evaluation/treatment (yes/no); these secondary dependent variables will be ascertained from review of CPRS. 
We will also ascertain completion of brief intervention and referral for evaluation/treatment from the interviews 
with participants; we will assess concordance between CPRS and Veterans’ self-report and, for main analyses, 
we will count brief intervention or referral as completed if they are ascertained from either source. 
We will measure alcohol-related consequences present at baseline and at 3-month follow-up, as measured 
by the SIP. The intervention is designed and hypothesized to reduce the quantity and frequency of risky 
drinking, which can be logically expected to lead to reduced frequency of consequences associated with 
drinking. However, the Relational Agent is not specifically designed to reduce these consequences, and the 
relatively short (3-month) follow-up period is not likely to be sufficient to demonstrate a differential rate of 
consequences between the treatment groups; as a result, data on consequences will be collected for 
exploration and hypothesis generation for future studies, rather than a priori formal outcome measures. The 
consequences will be ascertained by means of the SIP. The SIP will be administered at the baseline and 3-
month interviews. 
In addition, we will measure Veterans’ satisfaction with treatment for alcohol problems. To our knowledge, 
there exists no scale or question designed specifically to measure patients’ satisfaction with the care they 
received for alcohol problems in primary care. Prior studies have used various questions and scales to assess 
clients’ satisfaction with mental health or substance abuse treatment and the Consumer Assessment of 
Healthcare Providers and Systems (CAHPS) has emerged as a standard for measuring satisfaction with health 
care delivery, including primary care. We will use a single item, modeled on CAHPS, to assess satisfaction: 
“On a 0 to 10 point scale, with 0 being the worst care you can imagine and 10 being the best care you can 
imagine, how would you rate the care you have received in the last 3 months from the VA for alcohol 
problems?”  
Descriptor Variables 
We will collect data at baseline for the purpose of describing the sample and also analyzing potential 
moderating variables (i.e., covariates). This information will include age, educational level, race/ethnicity, 
branch of service, location of deployment, if any, and months of deployment. We will ask about substance 
10

 
abuse treatment in the last 12 months. We will also collect the following information from CPRS: mental health 
diagnoses, especially post-traumatic stress disorder (PTSD); and diagnosis of traumatic brain injury (TBI). It is 
unknown if these variables affect outcomes for brief intervention, and thus a fruitful area for exploratory 
analyses.  
Other variables that we will use to describe the sample and explore in cross-tabulations with the outcome 
variables include summary scores from the REALM health literacy questions and .the ECA Questionnaire. 
Statistical Analysis 
The figure below shows a schematic representation of the research design. For visual clarity, it does not 
show the descriptive variables mentioned above. 
Preliminary Analyses: In preliminary analyses, we will examine the distributions of all dependent 
variables, and consider appropriate transformations. In particular, we will consider an arcsine transformation 
(appropriate for reported proportions) for the “number of drinking days per week” and a log transformation 
(appropriate for highly skewed distributions) for “average drinks per drinking day”.  The 4 groups will be tested 
for any differences in socio-demographic variables. All quantitative variables (e.g. Age) will be tested by 2-way 
between groups Analysis of Variance (ANOVA); all qualitative variables (e.g. Gender) will be tested by 3-
dimensional Chi Square (Mantel-Haenszel test). Any variable found to be significant will be entered into all 
subsequent analyses as a covariate. 
Checks for “non-Response” Bias: The potential for differential drop-out rates among the four groups is 
itself an interesting and important empirical question, with clinical implications. Drop-out rates will be tested by 
Chi Square. Factors that may contribute to drop-out rate will be tested by logistic regression. In effect, the four 
groups can be stratified at Baseline into Completers and non-Completers. Then, these groups will be 
compared on Baseline values to determine which, if any, are significant systematic “predictors” of Completion. 
Statistical Analyses: If there is no significant differential drop-out rate attributable to Alcohol Group, to 
Treatment Group, or, especially, to Alcohol x Treatment interaction, then the non-completion rate can be 
considered as a random loss, and Completers as a representative sample. In that case, the dependent 
variable(s) will be tested by 3-way mixed design AN(CO)VA. If it is determined that there is a differential drop- 
out rate such that the data are not missing completely at random (NMCAR), then we will utilize a generalized 
estimation equation (GEE) approach, which should provide an unbiased estimate of the intervention effect. 
GEE utilizes inverse probabilities to correct for any bias associated with NMCAR, with participants who are 
Completers weighted with the drop-out probabilities of participants with the “same” characteristics at Baseline. 
These weighting values will be estimated from the logistic analyses described above. 
Independent Variable
RA + Treatment as Usual 
vs. Treatment as Usual 
Baseline Moderator Variables 
1)
Presence/Absence of 
Alcohol Use Disorder 
(MINI) 
2) Sociodemographics
Primary Dependent 
Variable
Quantity and Frequency 
of alcohol use (QDS) 
Secondary Dependent 
Variables
1.  % Receiving BI  
2.  % Referred for 
Treatment 
3.  Alcohol-Related 
Consequences (SIP)
Abbreviations: RCT= randomized controlled trial; RA = relational agent intervention; BI = brief intervention; QDS = Quick Drink Screen; SIP = 
Short Index of Problems 
11

 
The overall design is a three-way mixed-design  with Treatment Arm (Treatment as Usual vs. Treatment 
as Usual  plus Relational Agent Intervention) and Drinking Severity (Risky vs. Alcohol Use Disorder) and 2 
occasions (0 and 3 months) as the repeated measures design. We will take an intent-to-treat approach to the 
analysis; Veterans will be included in our primary analyses regardless of their level of adherence with their 
assigned intervention. Participants who do not complete the 3-month follow-up assessment will be considered 
to have missing data for that assessment and will not be included in the analysis of intervention effectiveness. 
The secondary outcomes, which will be “incidence rates”, will be tested by logistic regression. 
Primary Hypothesis: The Relational Agent will be more effective than Treatment as Usual in 
reducing the quantity and frequency of alcohol use. 
Our primary analyses of the effectiveness of Treatment as Usual vs. Treatment as Usual plus Relational 
Agent Intervention will focus on measurement outcome variables at the 3-month follow-up. Separate analyses 
will be performed for the two primary outcome measures: number of drinking days per week (frequency) and 
average drinks per drinking day (quantity), both reported over a month. Here we describe the analytic approach 
for the outcome of quantity of alcohol use (average drinks per drinking day); the approach for frequency will be 
analogous.  Initially our analysis will be through analysis of covariance models. We will include baseline values 
of the outcome variable as a covariate in these models, so our analyses may be interpreted as focusing on the 
change in outcome from baseline to three month follow-up. If any key characteristics of our sample were found 
to differ between groups, we will enter those as covariates as well. 
Secondary Hypothesis: The Relational Agent will be more effective than Treatment as Usual in 
improving rates of brief intervention. 
For our secondary outcome of increased rate of delivering brief intervention, we will compare the two study 
groups on the proportion of patients receiving brief intervention through the chi-square test.  Logistic regression 
will be used to calculate odds ratios and 95% confidence intervals in order to describe differences in rates of 
brief intervention between groups. To further examine baseline patient characteristics that predict brief 
intervention, and control for any covariates found to differ between treatment groups at baseline, we will 
develop a multivariable logistic regression model predicting rate of brief intervention from baseline 
characteristics as well as indicator variables representing the intervention group and presence/absence of 
alcohol use disorders. 
Secondary Hypothesis: The Relational Agent will be more effective than Treatment as Usual in 
improving rates of referral to specialty treatment. 
Secondary analysis also includes a comparison of referral rates, and analyses similar to those described 
for the secondary Hypothesis above will be employed. We will compare study groups on the proportion of 
patients referred through the chi-square test. Since we expect that Veterans with alcohol use disorders will be 
more likely to be referred regardless of group, we will test this hypothesis separately, i.e., we will use chi- 
square to compare rates of referral based solely on presence or absence of alcohol use disorders. Logistic 
regression will be used to calculate odds ratios and 95% confidence intervals in order to describe differences in 
referral rates between groups. To further examine baseline patient characteristics that predict brief intervention, 
and control for any covariates found to differ between treatment groups at baseline, we will develop a 
multivariable logistic regression model predicting referral rates from baseline characteristics as well as indicator 
variables representing the intervention group and presence/absence of alcohol use disorders. 
Additional exploratory hypotheses: 
Negative consequences for alcohol use, as measured by the SIP, may also decrease as a result of 
intervention, and this hypothesis will be explored as well through a random effects linear regression model. 
Sample Size and Power Considerations 
Power considerations focus on our primary hypotheses, which examine intervention effects on two primary 
outcomes, drinking days per week and average drinks per drinking day, at 3 months follow-up.  For our principal 
analyses we will separately examine effects on these outcomes at 3 months through analysis of covariance, 
controlling for baseline levels of the outcome variable. Reported effects of brief intervention vary in the literature. 
12

 
For instance, a meta-analysis by Moyer et al. found an effect size for alcohol consumption in non-treatment-
seeking samples of 0.67 for drinks per day at three months, but it quickly drops to 0.14 at six months, and 0.24 
at 12 months.95 On the other hand, Bien et al. reported an average effect size of 0.64 at 12 months for the 
reduction in drinks per month, but it varied in studies they reviewed from 0.27 to over 1.0.17 More to the point for 
our study, Riper et al96 conducted a meta-analysis of web-based brief interventions and reported a medium 
effect size (d=0.44). There are little data to suggest that those with alcohol use disorders derive more or less 
benefit from brief intervention.33 However, VA clinical guidelines suggest that people with alcohol use disorders 
need to be referred to specialty treatment, implying that they might not be expected to benefit solely from brief 
intervention. With these prior studies in mind, and with the explicit design of this study as an initial, single-site 
RCT to assess the potential effectiveness of the Relational Agent Intervention on frequency and quantity of 
alcohol use among Veterans, we have powered the trial for the detection of a large effect size, as defined by 
Cohen.97 
We begin our sample size estimation with the observation that 5,571 underwent AUD-C screening in our 
study setting during fiscal year 2010. We estimate that 1.25 * 5,571 = 6,963 will be screened during the 15- 
month study enrollment period. National samples indicate that about 20% of Veterans screened receive a 
positive score on the AUD-C,26  which would be consistent with the estimated prevalence in the larger 
population. Therefore, 6,963 * 0.2 = 1,393 Veterans will be potentially eligible. Because of clinic workflow and 
research assistant availability, we anticipate approaching approximately 500 of these Veterans. Based on Dr. 
Bickmore’s pilot study of a Relational Agent to screen for alcohol problems in primary care (described above, 
Section 4.a.1) and the two recently completed pilot studies of the Relational Agent in the Primary Care Clinic of 
VA Boston, we estimate that 36% of the 500 Veterans approached, or 180 Veterans, will agree to participate. 
We estimate that these 180 Veterans will be distributed in a 2:1 ratio as those with alcohol use disorders 
(N=120) and those with risky drinking (N=60), based on studies in VA.  
Rates of attrition, or loss-to-follow-up, in prior studies of brief intervention for alcohol problems have 
typically ranged from 10% to 30%. Based on our familiarity with following up study participants in alcohol and 
substance abuse interventions, and particularly noting our use of financial compensation for participation, we 
anticipate that our loss-to-follow-up will be on the lower end of this range. However, in the table below we show 
power estimates for three possible rates of loss to follow-up: 20%, 25% and 30%. Table 5 shows estimated 
power to detect a medium (Cohen’s) effect size, at an alpha level of 0.05, given the three possible rates of 
loss-to-follow-up. 
Power estimates for a range of loss-to-follow-up rates 
% Loss-to- 
Follow-Up 
Experimental 
Group 
Alcohol Use 
Disorders (N) 
Risky Drinking 
(N) 
Total (N) 
Power 
20% 
TAU 
48 
24 
72 
0.84 
TAU+RAI 
48 
24 
72 
25% 
TAU 
45 
22.5 
68 
0.82 
TAU+RAI 
45 
22.5 
68 
30% 
TAU 
42 
21 
63 
0.78 
TAU+RAI 
42 
21 
63 
Note: TAU=Treatment as Usual; RAI=Relational Agent Intervention. 
Thus, for our primary outcome, we estimate that we will have more than 80% power to detect a medium 
effect of the intervention with loss-to-follow-up rates approaching 30%. We will monitor loss-to-follow-up closely 
and, if we discover that the attrition rate is exceeding 30%, we will increase our sample size to account for the 
loss-to-follow-up. 
We will recruit a total of 180 Veterans from PCC, with approximately 60 participants in the Risky Drinking 
Group and 120 participants in the Alcohol Use Disorders Group. Accounting for 20-30% loss to follow up, we 
expect that at least 126 to 144 Veterans will complete the 3-month follow-up.  
13

Specific Aim 3: Examine in-depth Veterans’ experience with the Relational Agent Intervention.
Because this Aim is exploratory in nature, there are no associated hypotheses. Following the RCT, we will 
conduct a formative evaluation (Aim 3), with in-depth interviews, to characterize the elements of the Relational 
Agent Intervention that Veterans prefer, and the features that emerge as most effective and those that seem 
extraneous or even counter-productive. 
Rationale for formative evaluation 
Formative evaluation will inform future evaluation and implementation efforts, specifically in the area of 
Relational Agents for alcohol problems and more generically for interventions using health information 
technology to engage Veterans in their care in the primary care setting. Formative evaluation is valuable in the 
context of an intervention that results in quantitative improvement of outcomes because it can yield information 
regarding those characteristics of the intervention that seemed most useful and effective from the perspective 
of the study participants. Perhaps more importantly, formative evaluation is crucial in the context of an 
intervention that does not result in quantitative improvement in the target outcome (i.e., in the setting of a 
“negative study”), because it can identify the factors that may have prevented an otherwise-potent intervention 
from achieving the intended outcome. 
In-depth Interviews 
We will carry out in-depth interviews with Veterans soon after completion of the study to minimize 
perturbation of the experimental setting. In-depth interviews are characterized by extensive probing and open- 
ended questions. We will prepare a semi-structured interview guide that includes a list of questions or issues 
that are to be explored and suggested probes for following up on key topics. The content of the in-depth 
interviews will focus on identifying and characterizing factors related to successful implementation of the 
Relational Agent intervention, from study recruitment and enrollment, through actual engagement with the 
Relational Agent. The interview will encourage Veterans to provide feedback on specific details of the 
intervention that they found helpful and useful, as well as those features that were of little value or even counter-
productive. Broad topic areas will include the ease or difficulty associated with using the tablet computer, 
attitudes toward and reactions to the Relational Agent, and perceptions of the Relational Agent’s effect on 
Veterans’ behavior. Veterans will receive $20 in compensation for their time. 
We will use purposive sampling to identify potential participants allocated to the Treatment as Usual plus 
Relational Agent Intervention group, recruiting approximately equal numbers of Veterans who had a reduction in 
alcohol use and those who had no reduction. We will recruit a total of 20 Veterans.  All interviews will be 
conducted by telephone and, with the permission of the participants, will be audio-recorded for transcription and 
subsequent analysis. (See Human Subjects for details on recruitment and protection of human subjects.) 
These post-intervention interviews will be conducted by the research assistants, who will be trained by Drs. 
Goldman and Simon. 
Analysis 
We will perform a content analysis of the transcribed in-depth interviews, incorporating the principles of the 
immersion-crystallization method. This qualitative approach consists of repeated cycles of immersion into the 
collected data with subsequent emergence, after reflection, of an intuitive crystallization of the dominant 
themes.63 Dr. Rubin and research staff will independently listen to selected interview tapes, read all transcripts 
and write analytic notes for each observation or interview.  They will meet regularly to discuss each transcript 
and will compile detailed notes of each meeting to document emerging themes and maintain a permanent 
record of the analysis.  This process will allow thorough immersion in the data within the context of each 
interview in its entirety. We will then compare the data from the transcript under discussion with the data from 
other analyzed transcripts. Through this process, we will identify salient themes that crystallize from the 
interviews. Dr. Rubin and the research staff will each then code the transcripts and observation guides based 
on these themes, as well as broad content areas suggested by overall goals of the sessions. Dr. Rubin and the 
research staff will meet regularly with Dr. Simon, Dr. Goldman and other members of the team to review the 
emerging themes and coding strategies. We will continue analysis until no new major themes emerge. 
Examples from the transcribed interviews will be used to illustrate the findings that emerge. We will address 
recognized criteria for qualitative research: credibility, fittingness, auditability, and confirmability. We will 
establish credibility by including the research associate in discussions with the study team to determine the 
14

 
concordance between coded themes and the real-time research associates’ experiences. In addition, having 
multiple individuals read and code each of the transcripts will enhance credibility. We will assess fittingness 
through line-by-line analysis of the transcripts and by linking extensive verbatim examples to each theme. 
Through iterative discussions among the study team, these qualitative findings will be used to characterize 
the degree to which the Relational Agent Intervention considerations for future implementation of the 
intervention. 
(5) Human Subjects
Aims 2 and 3: Risk to Subjects
The overall goal of this project is to develop a Relational Agent Intervention to improve brief intervention 
for alcohol problems in primary care, and compare its effectiveness with treatment as usual. Aim 1 of this study 
has already been approved.  Aim 2 is quantitative: an initial RCT to compare the effect of an intervention on 
Veterans' consumption of alcohol to that of standard treatment in primary care. Human subjects’ risks and 
protections will be addressed for Aims 2 and 3 here. Benefits, importance of knowledge to be gained, the data 
safety monitoring plan and inclusion of women, children and minorities are the same for all parts of the project, 
and will be copied from the Aim 1 approval for the convenience of the reviewers.  
Human Subjects Involvement and Characteristics 
For Aim 2, 180 primary care patients who are attending a primary care appointment at VA Boston will be 
recruited into this study. Both men and women will be recruited.  Veterans identified by clinic staff as screening 
positive on the AUD-C for alcohol use (score of >=5) will be eligible for screening.  As detailed above, Veterans 
will be eligible if drinking above CDC guidelines in the past 30 days. They will be ineligible if  they do not have a 
stable  telephone number and mailing address for the next 6 months, or if they have participated in substance 
abuse treatment in the past 30 days.  
Participants will participate in research interviews at baseline (right after screening) and 3 months. 
Participants randomized to the experimental arm will also come in at one month for a second visit with the 
Relational Agent. 
For Aim 3, 20 of the RCT participants in the experimental arm will be invited to participate after their 3 
month interview is complete. We will use purposive sampling to identify potential participants allocated to the 
Treatment as Usual plus Relational Agent Intervention group, recruiting approximately equal numbers of 
Veterans who had a reduction in alcohol use and those who had no reduction. We will recruit a total of 20 
Veterans.   
Sources of Materials 
Data for Aim2, the RCT, will be obtained from two sources. Data will be obtained specifically for research 
from questionnaires, from the computer program providing screening and brief intervention, and from 
participant interviews. Data will also be obtained from CPRS to record whether the expected screening and 
brief intervention occurred in the “Treatment as Usual” arm of the study, and to gather information on other 
psychiatric problems that may affect the outcomes of the study. 
Veterans will be asked questions to assess their levels of health literacy, computer literacy, and attitudes 
toward the Intervention, as well as drinking problems and general descriptors (e.g., age, gender, previous 
substance abuse treatment). We will also have information about process variables from the participants who 
use the Relational Agent program (e.g., pros and cons of drinking, confidence level to change, etc.)we will 
obtain that information from participants. 
Data for Aim 3, the formative interviews, will be obtained via in-depth interviews with participants about 
their reactions to and assessment of the Relational Agent intervention.  
Potential Risks 
15

 
The research risk is minimal and involves additional questions and the administration of a computerized 
screening, brief intervention, and referral to treatment for alcohol problems (commonly referred to as SBIRT). 
Since each group receives treatment as usual, and primary care clinicians are supposed to provide a brief 
intervention and referral if appropriate for patients who screen positive on the AUDIT, the research-related 
SBIRT is in addition to usual care.   
The potential risks to participants in this study are minor discomforts from using a touch screen computer, 
wearing headphones, or answering questions about themselves and their alcohol use. There are no apparent 
physical risks for any of the participants.  There is also a risk of information being accidentally disclosed 
outside of the research team or, for certain information, outside of the VA primary care clinic. This risk will be 
minimized by keeping data secure (on secure servers, in locked cabinets and locked offices) and identifying 
information is kept separate from study data.  Documentation of the research visit and actions recommended to 
the participant will be found in a research progress note. This information may also be found in a consult 
generated to refer willing participants to specialty treatment.  There is also a potential risk that the questions 
and conversations in the interviews could be psychologically upsetting to the Veterans. This risk is low, and 
research assistants are trained to manage these by calling for professional help from Drs. Simon or Rubin, or 
from primary care staff.  In the section below, “Protection Against Risk,” we outline approaches to ensure that 
all risks are minimized. 
Adequacy of Protection from Risk 
Recruitment and Informed Consent 
We will recruit Veterans from the PCC and Women’s Health Clinics in Jamaica Plain. If this proves to yield 
an insufficient flow of participants, we will extend recruitment of Veterans to the West Roxbury and Brockton 
campuses. Clinic staff will do the actual recruitment by informing Veterans who screen positive on their annual 
AUD-C of the study. If the Veteran is interested, they will sign the “Permission to Contact” form and clinic staff 
will pass on the form to research staff. If it fits in the workflow of the clinic, clinic staff will provide a “warm 
handoff” of the veteran to the research staff.  Potential participants will be informed that the collection of data 
will be strictly confidential and that the participation of all human subjects will be completely voluntary. Our 
study will include any and all Veterans in Primary Care who are positive on the AUD-C and pass other eligibility 
requirements, as long as they are able to communicate and understand the Informed Consent (in English) 
document and physically interact with the Relational Agent computer system. 
Clinic staff administer the AUD-C once a year to all Veterans. If a Veteran scores above the cutoff, clinic 
staff will inform the Veteran about the study and ask them to sign the “permission to contact” form if they are 
interested. This form will be passed on to the Research Assistant, who will meet with the Veteran to conduct 
the screening. The RA will briefly explain what the study is about and ask if the Veteran will have a stable 
phone number and address for the next 6 months. If the answer is yes, the research assistant will administer 
the Quick Drinking Screen (quantity and frequency of drinking in the past 30 days). If the Veteran is drinking 
above CDC guidelines, additional screening questions will be administered. If the Veteran is eligible, the 
consent form will be read to the participant. The RA will stop after each page or each section of the ICF and 
ask if the veteran understands or has any questions. Once the ICF has been signed, the RA will ask about 
substance Abuse treatment in the last month. If the veteran has not had treatment in the last month, the RA 
administer baseline questionnaires, randomize the participant, and inform the participant of their assignment, 
and an appointment for the 3 month follow up will be scheduled. If the Veteran has been assigned to the 
Relational Agent, the RA will set up the program, explain what the veteran needs to do, then let the veteran 
use the program. When the veteran finishes the program, the one month appointment for the second session 
will be scheduled if appropriate.  
Protection against Risk 
As described above, the potential risks to subjects in this study are of low likelihood and generally not 
serious. Nevertheless, we have given these risks careful consideration and have developed a plan to ensure 
that research subjects are protected against them. With respect to ensuring confidentiality and data security, 
16

we will take the following precautions. We will only collect identifiable data when absolutely necessary and 
scientifically justified. All data will be stored on VA-encrypted laptops and on VA servers behind the VA firewall. 
Data transmissions between study staff will be minimized, but when necessary, will meet or exceed all 
standards for encryption and security that are in place for the electronic transmission of Veterans’ clinical 
information, with which the Principal Investigator is familiar. The master list will be stored separately from the 
coded data.  
The data collected on laptops includes a study ID, age, gender, and alcohol-related questions. This data is 
stored in the MySQL database on the laptop and is backed up to the database on a VA server when the 
session with the relational agent is completed.  
Recordings of Veterans participating in formative interviews in Aim 3 will be treated with similar care. The 
audio recordings will be made with a portable digital recording device that will always be maintained under 
lock-and-key of study staff (either in locked file cabinets within a locked office. or, when in transport between 
facilities, within a locked box). Transcription will be conducted by members of the study staff. The digital 
recordings will be downloaded to password-protected computer files on the VA network server, and 
transcriptions from these recordings will be similarly stored and handled. Prior to processing the transcriptions 
for qualitative analysis, study staff will remove all names and other personal identifiers from the transcripts. 
Transcripts will be printed as hardcopy only after all identifiers are removed; moreover, study staff will be 
instructed to treat the transcripts as confidential and as potentially harboring PHI, and as such, they will always 
be maintained in locked file cabinets in locked offices, and disposed of according to the existing VA policies 
regarding the disposal and destruction of such documents. We expect these processes, which our team 
members have employed in a variety of studies both within and outside the VA, will be highly effective in 
protecting subjects from the risks of data security and confidentiality breaches. With respect to protecting 
Veterans’ from the risk of psychological distress and physical taxation as a result of participating in the 
interviews, we will take the following precautions. First, we will ensure that the facility available for conducting 
the interviews is maximally accessible to all Veterans, with special attention to needs of Veterans with physical 
disabilities. Second, we will train the research associate to be sensitive to any discomfort that the Veteran may 
be experiencing; Drs. Simon and Rubin have extensive clinical and research experience related to interviewing 
and will ensure that the research associates have sufficient training in this regard.  
There are two possible scenarios in which a Veteran may express a risk of harm or self or others, although 
we consider it an unlikely occurrence. First, the Veteran may answer in the affirmative to the risk management 
questions during the telephone consult process conducted by either Dr. Simon or Dr. Rubin. In that case, Drs. 
Simon or Rubin will ascertain the level of risk (i.e., passive thoughts of suicide, or a detailed plan) and 
recommend the appropriate steps to the RA and the Veteran. In the second scenario, the Veteran may 
spontaneously tell the RA of their thoughts. In either case, the RAs will have the following procedures in place 
to follow: 
1.
The RA will contact Dr. Simon or Dr. Rubin immediately to inform us and get support for next steps
(if we are not already in contact).
2.
The RA will not leave the Veteran alone.
3.
Whether or not senior research staff are available, The RA will walk the Veteran over to the
Primary Care Mental Health (PCMHI) staff and hand off the Veteran.
4.
If PCMHI are not available the research assistant will alert the primary care team and hand off the
patient to one of the PCC staff who can bring the patient to urgent care.
5.
At no time will the RA be solely responsible for the care of a Veteran at risk for suicide or homicide.
Potential benefits of research to subjects and others 
This application proposes to develop a Relational Agent Intervention to improve brief intervention for 
alcohol problems in primary care, and compare its effectiveness with treatment as usual. The potential benefits 
of this research to Veterans include more frequent and higher quality intervention for this high priority area, 
resulting in improved health of Veterans. We also anticipate that this intervention, if proven effective and, 
ultimately, integrated successfully into the primary care clinic, will allow clinicians more time with their Veteran 
patients, with a lower burden of training specific to unhealthy alcohol use, while still being involved in their 
17

patients' needs in this area. Finally, this line of research can lead to the demonstration of the utility of an 
eHealth tool that Veterans with less comfort with technology can use, leading the way toward additional 
behavioral interventions that do not add to the clinic's workload. We believe that these benefits do outweigh the 
minimal risk of harm to research subjects, thus justifying the research. 
Importance of knowledge to be gained 
This project is unique in that it brings together three high priority areas for VA; improving mental and 
behavioral health interventions, specifically, brief alcohol interventions in primary care; transforming care 
through the use of healthcare informatics, specifically, the Relational Agent; and reducing health care 
disparities, through the use of an eHealth tool that has few barriers to use by Veterans who may have poor 
health and computer literacy. If ultimately proven effective, the use of this eHealth tool for multiple behavioral 
issues can be integrated into the interface between primary care and behavioral health, and can fit well into the 
model of Patient Aligned Care Teams. 
The knowledge to be gained from this study will be significant in addressing three high priority areas at 
once, and point to the pathway for using patient-facing technology to meet the 21st century requirements for all 
of our Veterans to be more involved in their own care. It will provide information on integrating eHealth tools 
into primary care as well as the effectiveness of this tool, at least for unhealthy alcohol use. The potential 
knowledge to be gained from this project outweighs the minimal risks to participants. 
18