The Utility of the Computerized Assessment and Referral System (CARS) Screener for Mental Health Evaluations in the Emergency Setting
Phase: NA
Conditions: Mental Health
Sponsor: University of Arkansas

Version #: 10 
Date: 08/28/19 
Page 1 
 
 
 
 
Study Title: 
The utility of the Computerized Assessment and Referral System 
(CARS) screener for mental health evaluations in the emergency setting 
 
Principal Investigator:  
Michael Wilson 
 
University of Arkansas for Medical Sciences 
  
4301 W. Markham Street, Slot # 584 
Little Rock, AR 72205 
Telephone: 501-686-5515 
Email: MPWilson@uams.edu 
 
 
Sub-Investigator (s): 
Sara Landes 
 
Department of Psychiatry 
 
University of Arkansas for Medical Sciences 
 
4301 W. Markham Street, Slot # 584 
Little Rock, AR 72205 
Telephone: 501-686-5515 
Email: SJLandes@uams.edu 
 
Manuel Alvarez Romero 
Department of Emergency Medicine 
University of Arkansas for Medical Sciences 
4301 W. Markham Street, Slot # 584 
Little Rock, AR 72205 
Telephone: 501-686-5515 
Email: Malvarezromero@uams.edu 
 
Austin Porter 
University of Arkansas for Medical Sciences 
4301 W. Markham Street 
Little Rock, AR 72205 
Telephone: 501-686-5515 
Email: APorter@uams.edu 
 
Eva Woodward 
University of Arkansas for Medical Sciences 
4301 W. Markham Street 
Little Rock, AR 72205 
Telephone: 501-686-5515 
Email: ENWoodward@uams.edu 
 
 
 
 
Study location: 
UAMS 
 
 
 
 

Title: The utility of the Computerized Assessment and Referral System (CARS) screener for mental health 
evaluations in the emergency setting 
PI: Wilson, University of Arkansas for Medical Sciences 
 
Version #: 10 
Date: 08/28/19 
Page 2 
 
Background and Rationale 
Scientific rationale: Mental health visits to the emergency department are increasing 
• 
Given time pressures, computerized methods have been proposed as a more accurate and faster 
way of assessing mental health diagnoses that are not obvious from the chart alone and that may 
be incidental to the emergency department visit (ED). However, this has not been rigorously 
tested in the ED setting. 
• 
The CARS has been tested for patients with alcohol abuse, but has not been tested in a more 
general ED population. 
• 
Previous literature has shown increase in mental health diagnoses of approximately 9% using a 
computerized screener, although this research used a different system than CARS (1).   
• 
Hybrid effectiveness-implementation designs focus on both clinical effectiveness and 
implementation outcomes. This is far more efficient than pursuing separate lines of effectiveness 
and implementation research, allowing the field to move forward at a quicker pace. It also offers 
more useful information for decision makers (e.g., what is needed to implement an intervention 
should it be effective). 
 
Hypothesis & Specific Aims 
• 
Assess whether access to the CARS results modifies the mental health assessment and disposition 
of ED patients. Hypothesis: Use of the CARS will increase the number of mental health diagnoses 
made by emergency physicians or psychiatrists, an increase in psychiatry evaluations (ie 
consults) in the emergency department, or an increase in mental health referrals from the ED. 
• 
Qualitatively assess the barriers and facilitators to implementing the CARS screener in the 
emergency department, including variables related to the intervention itself, the context and 
culture of the ED, provider attitudes, and patient needs. Hypothesis: Provider attitudes towards 
the intervention will vary by provider type. Both barriers & facilitators will be identified. 
 
Please note that the study will not otherwise alter usual & customary care in the emergency department, 
and patients may still be referred to any specialist that the emergency physician deems necessary. Thus, 
withholding a CARS report does not impede physician diagnosis or harm the patient in any way. All 
patients will be notified of their CARS diagnosis (please see study design below). 
 
Study Design and Procedures 
Background of the CARS: 
• 
The CARS is based on the Composite International Diagnostic Interview (CIDI), a validated 
instrument for assessing mental health diagnoses (2). According to the CARS manual, “The 
Computerized Assessment and Referral System (CARS) is the result of a collaboration between 
the Division on Addiction and the Foundation for Advancing Alcohol Responsibility, a nonprofit 
organization with a focus on preventing DUI…CARS adapts questions from the internationally-
recognized Composite International Diagnostic Interview (CIDI), a tool designed for lay 
interviewers who may not have expertise or training in mental health assessment.” CARS was 
piloted at 6 different sites in 2016, all in DUI populations. As CARS has not been specifically 
validated on non-DUI populations, investigators will collect alcohol history (see below). The 
CARS self-assessment version (CARS-SA) will be utilized in this study. 
 
Please see Figure 1 for a schematic of study flow.  

Title: The utility of the Computerized Assessment and Referral System (CARS) screener for mental health 
evaluations in the emergency setting 
PI: Wilson, University of Arkansas for Medical Sciences 
 
Version #: 10 
Date: 08/28/19 
Page 3 
 
 
Figure 1. Study flow. 
This is a hybrid effectiveness-implementation randomized controlled trial which will be conducted at the 
emergency department (ED) at the University of Arkansas for Medical Sciences (UAMS) in Little Rock, 
Arkansas. The UAMS ED is an urban emergency department which sees approximately 60,000 patients 
per year. 
 
This pragmatic mixed methods clinical trial will compare the intervention of providing a CARS screener 
report to emergency providers to not providing the report. It will also collect qualitative data regarding 
implementation of the CARS screener and report. Consecutive patients presenting when a research 
associate is available will be approached to participate. Patients will be approached after evaluation by an 
emergency physician. 
 
Inclusion criteria: 
• 
>18 years of age 
• 
Not incarcerated 
• 
English-speaking and English-writing (as translators will not be available for this study) 
 
Exclusion criteria: 
• 
Unable or unwilling to complete CARS screener 
• 
Unable or unwilling to provide any history of alcohol abuse 
• 
Currently presenting with a chief complaint of a psychiatric symptom 
• 
Currently presenting with a substance use complaint 
• 
History of a known psychiatric condition 
• 
Patients taking psychiatric medication 
• 
Unwilling to answer all CARS screener questions 

Title: The utility of the Computerized Assessment and Referral System (CARS) screener for mental health 
evaluations in the emergency setting 
PI: Wilson, University of Arkansas for Medical Sciences 
 
Version #: 10 
Date: 08/28/19 
Page 4 
 
 
No distinction will be made between a current presentation for psychiatric symptoms and current 
treatment for a mental illness. Additionally, patients with previous diagnosis of mental illness will be 
ineligible for enrollment in the study even if not currently seeking treatment. 
 
If informed consent is obtained (see “Ethical Considerations” below), patients with a CARS diagnosis 
will be randomized in a 1:1 fashion after completing the CARS screener to either the CARS report, in 
which the patient’s emergency providers will be given a copy of the CARS summary findings, or a no-
CARS report, in which the patient’s emergency providers will not be given a copy of the CARS summary 
findings. Patients will not be informed of their group assignment, and no alteration to usual care will be 
provided. All patients will be offered a copy of their own screener in a sealed envelope, along with a 
statement that results are suggestive, not definitive. If wished by patients who have a positive CARS 
screener in the control condition (ie, screener is not given to the ED staff), referral materials will be given 
to appropriate community mental health clinics. Please note that as standard of care, all patients are 
screened at triage for their reason for visit, for suicidal ideation, and for domestic abuse. Thus, even if 
patients are in the control condition, they are not at risk for having a serious mental health issue that is left 
untreated. 
 
Patients will be allowed to answer questions on the CARS screener via computer in the privacy of their 
ED treatment room. Patients will not be approached in the waiting room. Research assistants (RAs) will 
be immediately available, but will not read questions to patients unless asked. Patients do not need to 
answer any question if they feel uncomfortable, although this will cause their withdrawal from the study. 
Participants (both patients and clinicians) may withdraw from the study at any time. 
 
Data collection 
After the conclusion of the visit, patient charts in the electronic medical record will be searched for the 
following variables by blinded research associates: mental health diagnoses documented at time of visit; 
any psychiatry consults obtained in the ED; any referrals to PMD or mental health clinic; admissions for a 
mental health diagnosis; transfers to outside mental health facilities; number, dates, and chief complaints 
of any visits to the ED six months before and after the date of study enrollment; age, past medical history, 
past psychiatric history, substance abuse history, and alcohol use history. Research associates will collect 
these variables for all patients enrolled and included in the study. The CARS screener will specifically 
collect information about symptoms that may be indicative of mental illness. 
 
Primary outcome/hypothesis 
The primary outcome measure is a composite measure in CARS+ patients whose providers have been 
given a copy of the CARS report versus CARS+ patients whose providers have not been given a copy of 
the report. The composite measure includes: the number of CARS mental health diagnoses made in the 
ED; the number of psychiatry evaluations in the ED (ie, consults); and the number of mental health 
referrals made from the ED; the number of admissions for a mental health diagnosis; the number of 
transfers to outside mental health facilities; and the number, dates, and chief complaints of visits to the 
ED six months before and after the date of study enrollment. As the CARS screener was initially 
validated in alcohol-abuse patients, the performance of the screener will be compared separately in 
patients with and without alcohol abuse histories (operationalized as DUI or ED presentation for alcohol-
related reasons within the past 6 months). The primary outcome measure for patients (ie, the composite 
measure above) will be analyzed using logistic regression, with proportion of individuals seeking 
treatment as the dependent variable. Independent variables will include age, group assignment, and 
alcohol history. Other variables specified below may be included as deemed necessary by the 
investigators. 

Title: The utility of the Computerized Assessment and Referral System (CARS) screener for mental health 
evaluations in the emergency setting 
PI: Wilson, University of Arkansas for Medical Sciences 
 
Version #: 10 
Date: 08/28/19 
Page 5 
 
 
Justification for a composite measure 
Although use of CARS is expected to have a positive influence on evaluation and assessment by mental 
health workers, it is unclear exactly how providers will react to this additional information. Therefore, a 
composite outcome measure capturing different possible outcomes is appropriate. The measure is 
identical to that used in previous research by Schriger et al (2001) while investigating a similar type of 
screener with a similar endpoint. In that study, utilizing the PRIME-MD system, there was approximately 
9% more patients identified and/or treated in the above manner after being shown the screener. It is 
reasonable to expect that the same effects in this study utilizing the CARS screener. It is currently 
unknown whether admissions for mental health concerns will increase after use of the CARS screener. It 
is currently unknown whether the CARS screener will be more accurate in patients with a history of 
alcohol abuse or dependence than in patients without such a history. 
 
Sample size 
In a test of two proportions with 80% power to find at least a 9% difference between groups, 
approximately 238 patients per arm will be required (R Studio version 1.0.136) for a total of 476 patients 
who have a CARS diagnosis. As the investigators cannot control which patients will or will not have an 
occult mental illness, it is likely that many more patients will need to be enrolled to account for patients 
who do not have a CARS diagnosis (please see below). As the UAMS sees approximately 60,000 patients 
per year (approximately 5093 in January 2017 alone), this number of patients should be easily achievable 
in a 1-year timeframe, even planning for approximately 30-50% refusal rate. This project will include an 
analysis, by a PhD level statistician, of effect size after approximately 30% and 60% of planned patients 
are enrolled. Based on this interim analysis, the final number of patients may be slightly adjusted. 
 
Study Population 
Please see Figure 1 (study flow). Approximately 476 patients (238 in each arm) will need to be enrolled at 
UAMS. Given that only 40% are likely to have a CARS diagnosis based on previous research (1) and 
some participants will be unable to complete the mental health screener before their disposition (ie, 
discharge or admission) from the ED, we expect to enroll a minimum of 1400 patients. This number may 
be adjusted on the basis of an interim calculation of effect size (please see sample size above). 
 
If non-agitated, patients will be approached by research staff in the UAMS emergency department in a 
private room so as to protect patient privacy. Patients in the waiting room will not be approached. In the 
UAMS emergency department, research assistants may scan the trackboard and triage screen to identify 
patients 17 years of age and younger or who have a chief complaint of a psychiatric nature since these 
patients will not be invited to participate in the study. Patients who are critically ill or who are unable to 
be consented for any other reason will not be approached. A partial waiver of HIPAA is therefore 
requested for recruiting purposes only. The justification for this waiver is included in the IRB submission 
pages. The investigators assure the IRB that no PHI will be reused or disclosed for other purposes. 
 
Implementation Evaluation 
Qualitative interviews will be conducted with ED staff (ie, nurses, physicians) or patients and families. 
Interviews will be conducted with at least five of each person type, or until theme saturation is achieved 
(3). Although these interviews are not of a sensitive nature, the PI will not be informed of interviewee 
names so as to avoid the possibility of coercion. Please note that participants of any type who grant an 
interview, including patients, will be given a $10 gift card for their time and expertise. 

Title: The utility of the Computerized Assessment and Referral System (CARS) screener for mental health 
evaluations in the emergency setting 
PI: Wilson, University of Arkansas for Medical Sciences 
 
Version #: 10 
Date: 08/28/19 
Page 6 
 
 
If informed consent is provided for qualitative interviews, the research assistant will schedule individual 
semi-structured qualitative interviews in a masked fashion. These will be performed either in person or 
via telephone, based on the participant’s preference and research assistant’s availability. The interview 
guide will be developed based on the Consolidated Framework for Implementation Research (CFIR) and 
will examine the following variables known to impact implementation of an intervention (4):  
 
• 
Intervention characteristics (e.g., complexity, trialability) 
• 
Outer setting characteristics (e.g., patient needs and resources, local or state policies) 
• 
Inner setting characteristics (e.g., organizational or department culture, infrastructure changes 
needed) 
• 
Characteristics of individuals (e.g., attitudes, self-efficacy/confidence in using intervention) 
• 
Process (e.g., identifying local champions) 
 
Rapid Qualitative Analysis (RQA) strategies will be used to ensure timely communication of findings to 
our funding partner (5). Interviews will be audio-recorded and transcribed; each transcription will be read 
by two members of the qualitative team, who will then distill information pertaining to previously 
identified domains of interest based on CFIR, as well as emergent themes, into an “episode profile.” 
Episode profiles will be compared and any discrepancies resolved by referring back to the original 
transcription and/or presented to the larger team for discussion. Completed episode profiles will be 
entered into tables to create a composite matrix of data for each site, in order to be able to compare across 
participant responses.  
 
Risks and Benefits 
The main potential risk to study participants is in loss of confidentiality. Measures to protect the 
confidentiality of study participants will be implemented as described in the Data Handling and 
Recordkeeping section below. 
 
If patients require treatment for a mental health condition, there may be direct benefit to the study 
participants as they may receive additional evaluation by healthcare providers. In addition, knowledge 
gained from the study could potentially benefit patients in the future. 
 
Data Handling and Recordkeeping 
The Principal Investigator will carefully monitor study procedures to protect the safety of research 
subjects, the quality of the data, and the integrity of the study. All study subject material will be assigned 
a unique identifying code or number in REDCap. The key to the code will be kept in a locked file in the 
principal investigator’s office. Only Dr Wilson, the study statistician, and select study staff will have 
access to the code and information that identifies the subject in this study. However, audits of deidentified 
data may be performed by the study sponsor at their discretion. Interview material of clinicians will also 
be kept in a locked file in the office of the PI or co-investigator. Access will be strictly limited to study 
staff. 
Records will be maintained for 7 years per IRB requirements. At the discretion of the PI, records may be 
scanned and maintained in electronic format instead of paper format once the study is complete. If so, 
electronic records will be audited to ensure high fidelity with the originals. These electronic copies will 

Title: The utility of the Computerized Assessment and Referral System (CARS) screener for mental health 
evaluations in the emergency setting 
PI: Wilson, University of Arkansas for Medical Sciences 
 
Version #: 10 
Date: 08/28/19 
Page 7 
 
also be maintained on secure password-protected UAMS servers. When eventually destroyed, copies will 
be shredded per UAMS disposal guidelines. 
 
Data collection will be performed through both the CARS screener and the Research Electronic Data 
Capture (REDCap) system, and will be set up in cooperation with the UAMS Translational Research 
Institute (TRI) in order to assure 21 CFR Part 11 and HIPAA compliance. 
 
Data Analysis 
The primary outcome for this study will be a composite measure (please see above). 
 
Ethical Considerations 
After consent of clinicians, families, or patients (please see relevant consent form), the investigators will 
collect information regarding the usefulness of the CARS screener. Clinicians will not be approached by 
the PI, an emergency department physician, in order to avoid coercion. No PHI will be collected from any 
interviewee. All patients will be consented before randomization and survey administration. This study 
will be conducted in accordance with all applicable government regulations and University of Arkansas 
for Medical Sciences research policies and procedures. This protocol and any amendments will be 
submitted and approved by the UAMS Institutional Review Board (IRB) to conduct the study.  
 
Potential participants will be identified from the track board by a research assistant, who will approach 
the patient in a ED treatment room if this can be done privately. The consent process will also be done 
privately in the same room. No patients will be approached in the waiting room or in a hall bed. The 
formal consent of each subject, using the IRB-approved consent form, will be obtained before that subject 
is submitted to any study procedure. All subjects for this study will be provided a consent form describing 
this study and providing sufficient information in language suitable for subjects to make an informed 
decision about their participation in this study. The person obtaining consent will thoroughly explain each 
element of the document and outline the risks and benefits, alternate treatment(s), and requirements of the 
study. The consent process will take place as described in the “Study Population” section above, and 
subjects may take as much time as needed to make a decision about their participation.  
Participation privacy will be maintained and questions regarding participation will be answered. No 
coercion or undue influence will be used in the consent process, including the fact that the PI will not be 
involved in the recruitment process. This consent form must be signed by the subject or legally authorized 
representative and the individual obtaining the consent. A copy of the signed consent will be given to the 
participant, and the informed consent process will be documented in each subject’s research record. 
 
Dissemination of Data 
Results of this study may be used for presentations, posters, or publications. The publications will not 
contain any identifiable information that could be linked to a participant.  
 
 
 

Title: The utility of the Computerized Assessment and Referral System (CARS) screener for mental health 
evaluations in the emergency setting 
PI: Wilson, University of Arkansas for Medical Sciences 
 
Version #: 10 
Date: 08/28/19 
Page 8 
 
 
References 
1. 
Schriger DL, Gibbons PS, Langone CA, Lee S, Altshuler LL. Enabling the diagnosis of occult 
psychiatric illness in the emergency department: a randomized, controlled trial of the computerized, self-
administered PRIME-MD diagnostic system. Ann Emerg Med. 2001;37(2):132-40. 
2. 
Holmes E, Dalbec M. Computerized Assessment and Referral System: Implementation process 
evaluation. 2017. Personal communication. 
3. 
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering 
implementation of health services research findings into practice: A consolidated framework for 
advancing implementation science. Implement Sci. 2009 Aug 7;4:1-15. 
4. 
Qualitative methods in rapid turn-around health services research.  VA HSR&D Cyberseminar.  
11 Dec 2013. 
5. 
Beebe J. Rapid Assessment Inquiry: An Introduction. Walnut Creek, CA: AltaMira Press, 2001.