Implementing Computerized Substance Use and Depression Screening and Evidence-based Treatments in an HIV Primary Care Population
Phase: NA
Conditions: Substance Use Disorders, HIV, Depression, Anxiety
Sponsor: University of California, San Francisco

KPNC IRB Version: 06/22/2018 
TITLE: 
Computerized Substance Use and Depression Screening and Behavioral Treatment in HIV Primary Care 
NCT NUMBER: 
NCT03217058 
VERSION DATE: 
October 5, 2022 

 
 
 
 
KPNC IRB Version: 06/22/2018 
Research Question: What are the key barriers to implementation, factors associated with effectiveness, and cost 
of tablet-based screening for the continuum of SUD severity and depression, followed by BHS-delivered 
interventions for those who screen positive. 
Aim 1. To evaluate the implementation of computerized SUD and depression screening and BHS-delivered 
intervention in HIV primary care. 
Hypothesis 1: SUD and depression screening rates and BHS-delivered intervention will be higher in the 
intervention phase compared with the observation phase and will be maintained long-term. 
Hypothesis 2: Greater HIV severity and younger age will increase screening rates for SUD and depression. 
Hypothesis 3: More BHS-delivered intervention will occur with higher HIV, SUD and/or depression severity.  
Aim 2. To examine the effectiveness of computerized screening and BHS-delivered intervention in HIV primary 
care among patients with moderate or high SUD risk or depression severity.  
Hypothesis 1: SUD and/or psychiatry specialty care treatment initiation and antidepressant prescription use 
will be greater in the intervention phase compared with the observation phase. 
Hypothesis 2: HIV outcomes, including ART adherence, HIV RNA control, VACS index score, and retention in 
HIV care will be improved in the intervention phase compared with observation phase. 
Hypothesis 3: SUD and depression symptoms will decrease after implementation of the intervention.  
Aim 3. To determine implementation costs and cost-effectiveness of screening and BHS-delivered 
intervention.  
We will measure the cost per patient screened, and among those screening positive, the cost per patient 
initiating BHS-delivered intervention. We will also evaluate the cost-effectiveness of the intervention for 
achieving improvements in each HIV, SUD and depression outcome. 
Aim 4: Perform key informant interviews to evaluate provider- and clinic-level implementation barriers and 
facilitators to computerized screening and BHS-delivered intervention.  
Semi-structured interviews for selected providers and clinic staff, including qualitative and quantitative 
components, will be performed in each clinic. 
 
 
 
Study Design 
 
Description of Study Phases 
Phase 1: Data only 
Observational phase: The 12 months prior to implementation at each study clinic will be considered the 
observational phase consisting of usual care with access to specialty clinic-based SUD and depression care. 
Hazardous drinking screening has been implemented systematically using NIAAA guidelines,169 along with 
standardized tobacco screening; both are part of the clinical workflow in the health system and are captured in 
the “MA Rooming Tool” used by medical assistants to capture essential preventive health measures. Screening 
for other substances is not standardized and instead relies on physicians’ clinical practices. A combination of 
appointment codes and appropriate International Classification of Diseases, (ICD) 9th and 10th edition will be 
used to determine screening for alcohol and drug use. Drug use and depression data are captured in the Social 
History section of KP Health Connect, and will be extracted from the EHR. Treatments provided to depression 
patients are based on current best practices for medication management and psychotherapy. 229,230 Mild to 
moderate depression is often addressed in primary care with antidepressant medication, whereas patients with 

 
 
 
 
KPNC IRB Version: 06/22/2018 
more severe symptoms or complex presentations are usually referred to psychiatry clinics for specialty care. We 
will track these aspects of patient care in the EHR. 
 
Phase 2: TAPS/AOQ Administration 
Overview: This hybrid design intervention study will evaluate the implementation, effectiveness and cost of 
screening (via KP.org, tablet, or clinic-based computer during visit [“suspended hyperspace”]) for the continuum 
of SUD severity and depression, followed by BHS-delivered interventions for those who screen positive. The 
study will be performed within the 3 largest clinics in KPNC at Oakland, Sacramento and San Francisco, which 
collectively serve ~5,000 patients. Through this project, clinical and Division of Research staff will partner to 
develop and deliver patient messaging regarding SUD and depression screening. Division of Research staff will 
provide initial support to clinics to attach screening questionnaires to patient appointments, with the goal of 
transitioning this work over to clinical staff during the course of the project to inform the feasibility of long term 
implementation.  
The study team will assist the clinics in programming computerized screening measures, contacting patients via 
secure message on behalf of clinicians, and training a BHS to deliver evidence-based interventions. Patients will 
be informed that completing the questionnaire is part of Kaiser Permanente research and completion is 
voluntary. This information is included in both the secure message language (for those who have KP.org) as well 
as in the questionnaire instructions that are displayed on the tablet or computer device during administration 
(see language in “Secure Message” document and “TAPS/AOQ Questionnaire”). 
 
Prior to and during routine annual HIV clinic visits, HIV patients will be asked via secure message to complete the 
questionnaire in KP.org. Patients who do not complete the questionnaire prior to registration in the clinic will be 
instructed by a receptionist to complete it on a tablet in the waiting room. Clinicians will also have an 
opportunity to administer the questionnaire on an as needed basis (e.g. if the tablet is unavailable) via 
suspended hyperspace during the clinical visit. Clinic staff will be trained by Drs. Satre and Leibowitz on 
questionnaire administration and interpretation. The results will be directly incorporated into the EHR and 
viewable by clinical staff including physicians and BHSs. Next, BHSs will review screening results and initiate MI- 
and CBT-based interventions as needed during the clinic visit and/or by phone. The study will employ a stepped 
wedge design, with implementation and effectiveness outcomes in the intervention phase compared with 
outcomes in the 12-month observational phase (i.e., usual care in 12 months prior to implementation). Rollout 
of the intervention will occur sequentially at the 3 clinics over two years (including 6-months overlap between 
clinics), allowing for refinement of the intervention accommodating the unique implementation challenges at 
each site.  
 
The study team will assist the clinics in programming computerized screening measures, contacting patients via 
secure message on behalf of clinicians, and training a BHS to deliver evidence-based interventions. Prior to and 
during routine annual HIV clinic visits, HIV patients will be asked via secure message to complete the 
questionnaire in KP.org. Patients who do not complete the questionnaire prior to registration in the clinic will be 
instructed by a receptionist to complete it on a tablet in the waiting room. Clinicians will also have an 
opportunity to administer the questionnaire on an as needed basis (e.g. if the tablet is unavailable) via 
suspended hyperspace during the clinical visit. Clinic staff will be trained by Drs. Satre and Leibowitz on 
questionnaire administration and interpretation. The results will be directly incorporated into the EHR and 
viewable by clinical staff including physicians and BHSs. Next, BHSs will review screening results and initiate MI- 
and CBT-based interventions as needed during the clinic visit and/or by phone. 
 

 
 
 
 
KPNC IRB Version: 06/22/2018 
For more detail regarding Phase 2, see “Procedures” Section. 
 
Study Population 
 
Inclusion and Exclusion Criteria 
 
Phase 1: All patients who are HIV-positive patients from the 3 largest KPNC HIV clinics at Oakland, Sacramento 
and San Francisco will be included in the observational phase. 
 
Phase 2: All patients who are HIV-positive patients from the 3 largest KPNC HIV clinics at Oakland, Sacramento 
and San Francisco will be included in the study. There will be no additional screening for eligibility as patients 
will already be identified as members of the KPNC HIV registry and have a scheduled appointment with an HIV 
care provider at one of the participating clinics to be included. 
 
Recruitment Methods 
Phase 1: No recruitment 
 
Phase 2: KPNC patients will not be recruited as part of this study as we propose to implement screening and 
treatment for SUDs and depression as part of routine HIV primary care and the implementation of the BHS 
intervention will be part of standard of care practices.  
 
 
a. Informed Consent Process 
Phase 1 (Data Only): We are requesting a Waiver of Informed consent. 
Phase 2 (TAPS/AOQ Administration): We are a requesting a Waiver of Signed Informed Consent 
Waiver of Informed Consent  
 
Phase 1 (Data Only):  
This study is minimal risk, The data only phase involves no active participation for patients and will be looking at 
retrospective data regarding access to usual care and utilization of SUD and depression care prior to the 
implementation of the systematized screening tool. It would be impractical to obtain informed consent from 
~5,000 HIV patients who receive their routine care from one of the three participating KPNC HIV clinics. 
 
Waiver of Signed Informed Consent  
 
Phase 2 (TAPS/AOQ Administration): 
–Condition 1 applies to our request for waiver of signed informed consent. Prior to and during routine annual 
HIV clinic visits, HIV patients with a KP.org account will be asked via secure message to complete the 
questionnaire in KP.org. The secure message will contain the elements of consent as well as a link for 
participants to click to complete the questionnaire. Clinicians will also have an opportunity to administer the 
questionnaire on an as needed basis (e.g. if the tablet is unavailable) via suspended hyperspace during the 
clinical visit. The elements of consent are also reiterated in the instructions displayed at the beginning of the 
questionnaire to ensure that patients who complete their screening via tablet or suspended hyperspace also 
review the necessary information and consent is assumed when patients continue to fill out the questionnaire 

 
 
 
 
KPNC IRB Version: 06/22/2018 
after reading the instructions. We are requesting a waiver of written informed consent since it would be 
impractical to obtain written consent from ~5,000 HIV patients who may receive the survey more than once.  
 
In addition, we originally envisioned that clinic staff would assist with sending surveys and adding in written 
informed consent to the workflow would be overly burdensome to both providers and their patients. Finally, the 
research involves no more than minimal risk for participants as these routine, clinically important questions 
about substance use and mental health are already being asked by their providers in HIV clinics (with no written 
consent required) during their appointments.. 
 
Associates) and all PHI will be removed from transcriptions prior to data analysis. Quantitative measures will 
also be collected from study participants via online survey hosted by KPNC IT-approved Datstat technology. No 
PHI will be collected during the online survey and subjects will only be identified by their medical center (OAK, 
SAC, SF) and their job category (clinician, support staff, BHS, etc.). 
 
HIPAA Privacy Rule Authorization – if study will use or disclose Protected Health Information (PHI) 
Phase 1 (Data Only): We are requesting a Waiver of HIPAA Privacy Rule Authorization. 
Phase 2 (TAPS/AOQ Administration): We are a requesting an Alteration of HIPAA Privacy Rule Authorization 
without signature. 
Waiver of HIPAA Privacy Rule Authorization  
 
Phase 1 (Data Only): The study will only recruit all adult HIV primary care patients from 1 of 3 primary HIV 
clinics. It is necessary to access PHI on patients including data drawn from the electronic medical record. We 
need to be able to link these sources of data using identifying patient information and dates of service, which 
includes PHI. We are requesting a waiver since it would be impractical to obtain written authorization from 
~5,000 HIV patients who are eligible to participate. 
 
Phase 2 (TAPS/AOQ Administration): We propose to implement screening and treatment for SUDs and 
depression as part of routine HIV primary care. Without an alteration we would be required to request signed 
authorization from all the clinic’s patients in order to review their records. This would put an unnecessary 
burden on patients and would be disruptive to the clinic, due to the large number of patients who use the clinic 
to receive primary care. We are proposing to implement screening and treatment into routine clinical care.. In 
addition to the elements of consent, we will also include information regarding access to and use of PHI for this 
project in the secure message provided to patients who have KP.org as well as in an information sheet provided 
to all patients who complete the questionnaire in clinic via the tablet or suspended hyperspace function. Thus, 
all patients will be informed that their PHI will be accessed if they choose to participate in this project and have 
the opportunity to choose not to participate. 
 
As described, the primary source of risk is potential loss of confidentiality. There is also a small risk that some 
patients may feel uncomfortable or experience emotional distress when answering some of the questions 
related to depression and thoughts of suicidality on the AOQ measure. We do not anticipate this risk is any 
greater than the routine care already received. Risk is further minimized by participation in the study because 
we will be actively monitoring TAPS/AOQ responses and providing timely notification to clinic staff for patients 
who endorse thoughts of self harm which may require more immediate clinical action. Under standard of care, 
AOQ responses that are submitted prior to visits are not flagged in such a way and a clinician may not be aware 
that patients endorsed thoughts of self-harm until they come in for their visit. We will also include information 

 
 
 
 
KPNC IRB Version: 06/22/2018 
regarding mental health resources to patients in both the secure message, for patients who receive the 
questionnaire via KP.org, as well as in the information sheet that is provided to patients who complete the 
questionnaire in clinic via the tablet or the suspended hyperspace feature.  
 
 
Study Procedures 
Phase 1: Data Only 
 
Observational phase: The 12 months prior to implementation at each study clinic will be considered the 
observational phase consisting of usual care with access to specialty clinic-based SUD and depression care. 
Hazardous drinking screening has been implemented systematically using NIAAA guidelines,169 along with 
standardized tobacco screening; both are part of the clinical workflow in the health system and are captured in 
the “MA Rooming Tool” used by medical assistants to capture essential preventive health measures. Screening 
for other substances is not standardized and instead relies on physicians’ clinical practices. A combination of 
appointment codes and appropriate International Classification of Diseases, (ICD) 9th and 10th edition will be 
used to determine screening for alcohol and drug use. Drug use and depression data are captured in the Social 
History section of KP Health Connect, and will be extracted from the EHR. Treatments provided to depression 
patients are based on current best practices for medication management and psychotherapy. 229,230 Mild to 
moderate depression is often addressed in primary care with antidepressant medication, whereas patients with 
more severe symptoms or complex presentations are usually referred to psychiatry clinics for specialty care. We 
will track these aspects of patient care in the EHR. 
Sleep subanalysis: We will be doing additional analysis on sleep disorders related to mental health and 
substance use that will require the following data: 1) patient reported outcomes on sleep, and 2) ICD9/10 codes 
for sleep disorders  
 
Phase 2: TAPS/AOQ Administration 
 
Overview: This hybrid design intervention study will evaluate the implementation, effectiveness and cost of 
screening (via KP.org, tablet, or clinic-based computer during visit [“suspended hyperspace”]) for the continuum 
of SUD severity and depression, followed by BHS-delivered interventions for those who screen positive. The 
study will be performed within the 3 largest clinics in KPNC at Oakland, Sacramento and San Francisco, which 
collectively serve ~5,000 patients. Through this project, clinical and Division of Research staff will partner to 
develop and deliver patient messaging regarding SUD and depression screening. Division of Research staff will 
provide initial support to clinics to attach screening questionnaires to patient appointments, with the goal of 
transitioning this work over to clinical staff during the course of the project to inform the feasibility of long term 
implementation.  
The study team will assist the clinics in programming computerized screening measures, contacting patients via 
secure message on behalf of clinicians, and training a BHS to deliver evidence-based interventions. Patients will 
be informed that completing the questionnaire is part of Kaiser Permanente research and completion is 
voluntary in both the secure message language (for those who have KP.org) as well as in the questionnaire 
instructions that are displayed on the tablet or computer device during administration (see language in “Secure 
Message” document and “TAPS/AOQ Questionnaire”. 
 
Computerized patient self-administered screening measures (TAPS/AOQ). As part of annual primary care visits 
delivered by clinic staff, patients will complete the validated Tobacco, Alcohol, Prescription Medication, and 
Other Substance Use (TAPS) tool. The taps includes 4 initial screening items asking about tobacco, alcohol, 

 
 
 
 
KPNC IRB Version: 06/22/2018 
prescription drug misuse, or other drug use in the prior 12 months; with 3 follow-up questions in each of these 
categories regarding substance-related problems. [J. McNeely et al., Performance of the Tobacco, Alcohol, 
Prescription Medication, and Other Substance Use (TAPS) Tool for Substance Use Screening in Primary Care 
Patients. Ann Intern Med. 2016;165:690-699. doi:10.7326/M16-0317] . It is important to note that ALL HIV 
patients (regardless of cognitive status) are already asked questions regarding their substance use as a part of 
their routine clinical care, however patient responses are not documented in the health record in a consistent 
way across providers, which requires clinicians to look in several different areas of the health record for 
responses when reviewing a patient. The addition of the TAPS is an opportunity to collect substance use 
responses from patients in a systematic and consistent way for clinicians to refer to when reviewing patient 
records.  
 
Depression measures include the PHQ-9207, administered as part of the widely-used Kaiser Permanente Adult 
Outcomes Questionnaire (AOQ) that is already utilized in both electronic and paper formats for many patients 
throughout KPNC and procedures are already in place for clinician response to suicidal ideations indicated by 
patients completing the AOQ. Existing procedures will still remain in effect. Patients are routinely sent this 
questionnaire electronically and are told that their provider may not see the responses until the time of their 
appointment in the clinic. Responses to the measure are currently used as a starting point for clinicians to assess 
depression severity, including suicidal ideation, during clinic visits. In addition to this existing procedure, project 
staff will (1) include mental health resources in both the secure message as well as on the information sheet 
provided to patients who complete the questionnaire in clinic for patients who may experience emotional 
distress when answering the questions and (2) send a Staff Message to physicians when patients endorse this 
item on questionnaires completed prior to arrival at the clinic – so that clinic staff can respond appropriately. 
For example, in some cases providers may choose to reach out to a patient by phone prior to a scheduled 
appointment, based on their knowledge of the patient’s mental health history. Thus, the proposed procedures 
reduce patient risk even further than standard of care administration procedures.  
 
Automated reporting of the results of the measures into the EHR is enabled via a survey feature of KPNC’s 
shared medical record. The generated reports, displayed in a clinical progress note, will include yes/no 
responses for substance use and depression, severity scores (low, medium and high risk for dependence) for 
each substance reported, and depression severity scores (mild, moderate, moderately severe, and severe). The 
intent of the screening report is to serve as a resource in the EHR for usual-care providers, a prompt for BHS 
providers to initiate patient outreach, and a starting point for BHSs as they conduct patient-centered clinical 
assessment and MI- and CBT-based intervention. Results will also be included in a new report within “iHIV”, a 
clinical reporting tool in use at all KPNC HIV clinics and currently maintained by DOR HIV Registry staff. DOR has 
standard training procedures for any new iHIV users, and for assistance as needed for existing users in reviewing 
new reports and functionality.  
 
Modalities of Administration: There are three modalities that will be utilized to administer the TAPS/AOQ 
questionnaires: 1) online via the KP.org questionnaire feature sent via secure message; 2) in clinic on a secure 
tablet provided by reception prior to the appointment; 3) during a clinic visit via the “suspended hyperspace” 
feature.  
 
All HIV patients who have an appointment with an HIV care provider at one of the three participating KPNC 
clinics within two weeks will be flagged in the tracking system to have a TAPS/AOQ questionnaire attached to 
their upcoming visit in Health Connect. After the questionnaire is attached to the appointment, patients who 
have KP.org will also receive a secure message through the KP.org system with the “Secure Message” text that 
includes a link that allows them to complete the questionnaire prior to their appointment. Patients will receive 

 
 
 
 
KPNC IRB Version: 06/22/2018 
up to two secure messages regarding completing the questionnaire prior to their visit. Appointments scheduled 
within 1 week of the visit will only receive one secure message. Appointments scheduled >1 week before the 
visit will receive two secure messages, an initial message sent up to two weeks prior to the appointment date 
and an additional reminder message sent 7 days after the first message if they did not yet complete the 
questionnaire online. Patients who indicate to clinicians that they would no longer like to receive secure 
messages regarding completing the questionnaire will be flagged will no longer receive secure messages 
regarding completing the questionnaire prior to the appointment.  
 
As noted above, the AOQ is already administered as standard care throughout KPNC and existing procedures are 
in place in the clinics for addressing patients who endorse thoughts of self harm on the measure when 
completing the questionnaire outside of an office visit. For patients who complete the questionnaire as part of 
the TAPS/AOQ prior to their appointment, these existing procedures will remain in effect and we will also 
minimize risk even further by adding a resource list in the secure message text for patients who may experience 
emotional distress when answering the questions as well as an additional level of oversight to monitor PHQ9 
item responses for endorsement of “thoughts of self harm”. When these results are received by study staff, a 
flag will occur in the tracking system and a study staff member will send a Staff Message in Health Connect 
within 1 business day to the clinic providers to alert them of the patient’s response. Additionally, there the 
information sheet will be given to all patients who complete the TAPS/AOQ questionnaire in clinic to ensure that 
they have access to the same mental health resources as individuals who receive the questionnaire via secure 
message. 
 
Patients who do not have KP.org or patients who received the secure message but did not complete the 
questionnaire online prior to their appointment will have a flag associated with their appointment in Health 
Connect which will alert reception to provide them with a tablet to complete the questionnaire when they check 
in prior to their visit. Patients who have a TAPS/AOQ questionnaire attached to their appointment that has not 
yet been completed will have a pop-up message displayed at the appointment desk. A Qx icon column AND a 
check-in pop-up message will display in HealthConnect. Patients who are given a tablet to complete the 
questionnaire in clinic will also be given an information sheet with information regarding the use of their PHI as 
part of this study as well as with a resources list for patients who may experience emotional distress when 
answering the questions. We are requesting an alteration of HIPAA Authorization for this phase of the study. 
 
Clinicians will also have an opportunity to administer the questionnaire on an as needed basis (e.g. if the tablet 
is unavailable or on an ad hoc basis) via suspended hyperspace during a clinical visit. Suspended hyperspace is a 
feature that allows the MA or clinician to pull up the questionnaire on the computer in the exam room that 
allows the patient to complete the questions but restricts their access to their medical record, PHI or any other 
sensitive information. The questionnaire results will be directly incorporated into the EHR and viewable by 
clinical staff including physicians and behavioral health specialists (BHS). Next, BHSs will review screening results 
and initiate MI- and CBT-based interventions as needed during the clinic visit and/or by phone. 
 
The study will employ a stepped wedge design, with implementation and effectiveness outcomes in the 
intervention phase compared with outcomes in the 12-month observational phase (i.e., usual care in 12 months 
prior to implementation). Rollout of the intervention will occur sequentially at the 3 clinics over two years 
(including 6-months overlap between clinics), allowing for refinement of the intervention accommodating the 
unique implementation challenges at each site. Patient recruitment will conclude at all three sites the end of 
project year 4 (7/7/2020).  
 

 
 
 
 
KPNC IRB Version: 06/22/2018 
Patients will only be flagged to be asked to complete the questionnaire every 6 months. Clinicians may choose 
to administer the questionnaire via the suspended hyperspace feature in the interim period if they deem it 
appropriate follow up for clinical care, however no appointment will be flagged to have a questionnaire directly 
attached for patient completion prior to their visit if a TAPS/AOQ has been completed within 6 months.  
 
BHS-delivered intervention. Drs. Satre and Leibowitz will meet with clinic BHSs to introduce (or review) core 
cognitive behavioral therapy (CBT) - and motivational interviewing (MI)-based interventions. BHSs will be trained 
to review results of all computerized screenings and to provide outreach by phone210 and/or web-based patient 
portal (KP.org) using secure messages. They will be coached on patient engagement and patient-centered goal 
setting (e.g., noting screening results but also focusing on patients’ priorities and concerns), and instructed to 
offer individual MI/CBT sessions by phone or in person, according to clinical assessment and patient preference. 
MI and CBT can be effectively combined to address co-occurring depression and substance problems.217-219 BHSs 
will use MI to enhance motivation to initiate specialty SUD or psychiatry treatment within KPNC for higher 
severity patients, which is available for all members in facilities nearby to each primary care clinic.  
 
Intervention training. The study team will meeting with staff and providers early in the implementation phase 
to assist with planning for tablet or suspended hyperspace-based administration based on our prior work, e.g., 
work flow considerations.91 Orientation for physicians and other providers (e.g., nurse practitioners) will include 
review of the new screening measures, how to locate scores in the EHR, and the BHS role in patient care and the 
intervention. 
BHSs are Masters or Doctoral-level psychologists or licensed clinical social workers with a training 
foundation and experience in behavioral interventions. Currently they provide care across medical centers; and 
our study will support their time dedicated to HIV clinics. They often co-manage patients’ medical conditions 
through non-pharmacologic and behavioral interventions, collaborate with primary care providers, and provide 
consultation to other team members in the areas of mental health, behavioral medicine, and health psychology. 
Dr. Satre will provide BHSs training on MI and CBT for depression and substance-use problems. Training will 
include 15 hours of instruction and supervised practice of brief, solution-focused, MI/CBT interventions involving 
3 training cases (volunteer actors posing as patients), which is consistent with the average number of hours 
required for training in past clinical trials.95  
The initial training with BHSs will also include new technology and workflow (e.g., accessing screening data 
and documenting encounters in the EHR), and guidelines for when to help patients initiate specialty SUD and 
psychiatry care (i.e., for high severity patients), suicide risk assessment, and follow-up (monitoring patient 
engagement and following up as needed with patient and care team to ensure linkage to treatment). Monthly 
consultation with Drs. Satre and Leibowitz will reinforce key clinical skills, enable clinicians to share case material 
and obtain feedback, and troubleshoot system and workflow challenges.  
 
Study Timeline. This study has a 5-year timeline (Table 2). This will be sufficient for observation and follow-up of 
4600 patients, data extraction and analysis, and interpretation and dissemination of results. 
 
Table 2. Project Timeline  
Study Activities 
Year 1 
Year 2 
Year 3 
Year 4 
Year 5 
Study start up activities 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Clinic 1 implementation 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Clinic 2 implementation 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Clinic 3 implementation  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Ongoing BHS support and coaching 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Semi-structured clinician interviews 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
One year clinic data follow up period  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Data analysis 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Manuscript Preparation 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Dissemination activities 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
KPNC IRB Version: 06/22/2018 
. 
Data Sources. Study measures will be obtained from the EHR, HIV Registry, and informant interviews. 
 
KPNC EHR. The EHR databases links information using unique member identifiers from multiple data sources 
including demographics, membership, ambulatory, inpatient, laboratory and prescription data. Unit costs of 
services obtained for each patient encounter and service are also available and can be calculated by member 
over time, or aggregated to physician or facility level. Vital status is captured comprehensively from member 
proxy reporting, deaths during a KP hospitalization, regional cancer registry vital status, quarterly Social Security 
Administration vital status files and death certificates. Additional detail on all EHR data sources is provided in 
KPNC Resources. After implementation of the intervention, results of KP.org-, tablet-, and suspended 
hyperspace-based SUD and depression screening during routine primary care visits will be added to the EHR. 
HIV registry. The HIV registry includes all HIV patients since the early 1980s, with HIV-infection confirmed by 
chart review. It maintains up-to-date lists of all HIV patients, HIV transmission risk factors, dates of known HIV 
infection, AIDS diagnoses, and complete HIV-related lab and pharmacy data. A total of 8,186 HIV patients are 
current KPNC members, including 4,600 (56% of all KPNC HIV patients) from the three study clinics.   
 
Study Measures. We will include relevant patient demographic factors in all analyses, including age, sex, 
race/ethnicity, and HIV risk group. Evaluating these demographics in the context of an insured population allow 
for the evaluation of potential disparities in screening and treatment rates, independent of differences in access 
to care. Other study measures for each study Aim are described below. 
 
Aim 1 measures (implementation) 
• SUD and depression screening during the observational phase will include EHR-based provider-documented 
screening for SUDs and depression. Intervention phase screening will include provider-documented screening 
and KP.org-, tablet-, and suspended hyperspace-based screens, also in the EHR.  
• BHS-delivered intervention during the observational and intervention phases will be defined as one or more 
in-person or phone visits with a BHS. We will also measure the number of phone and in-person visits.  
Aim 2 measures (effectiveness)  
• SUD and depression treatment initiation. Patients scoring in the high SUD risk level on the TAPS or moderate 
to severe depression on the PHQ-9 will be encouraged to initiate specialty SUD or psychiatry treatment, and 
the BHS will use MI to help enhance motivation to initiate care. We will analyze SUD and depression 
treatment initiation rates using electronic administrative data; that is, whether the patient attended at least 
one in-person or phone visit with a BHS and/or in SUD or psychiatry treatment.  
• Antidepressant prescription rates. Any prescription for antidepressants, including standard medications 
based on best practices guidelines in the treatment of depression.211,230  
• Antiretroviral (ART) medication adherence. ART adherence will be defined using the Medication Possession 
Ratio (MPR)234,235 for each ART prescribed.236 MPR is calculated using a numerator of days’ supply dispensed 
from first fill to end of the interval (i.e., 12 months), and a denominator of total days between first fill to end 
of the interval, with values ranging from 0% to 100%. Mean refill adherence will be calculated across all 
individual ARTs dispensed in the interval237 distinguishing between true non-adherence or a change in ART 
(i.e., a new ART medication prescribed within 30 days) as in our previous work.238 179 . 
• HIV RNA response. HIV RNA control, defined as undetectable HIV RNA levels below 75 copies/ml.  
• Retention in care. We will use the Institute of Medicine’s encounter-based definition of retention, defined as 
≥2 HIV primary care visits within a 12-month period, ≥90 days apart.155  
• VACS index score. The VACS score has been shown to correlate with mortality.186 It incorporates 7 routinely 
collected clinical variables, including age, CD4, HIV RNA, hemoglobin, fibrosis index 4 (FIB-4), Hepatitis C, and 

 
 
 
 
KPNC IRB Version: 06/22/2018 
estimated glomerular filtration rate. The FIB-4 index incorporates age, platelets, and two liver function test 
results, including aspartate aminotransferase and alanine aminotransferase.  
• TAPS (SUD risk level) and PHQ-9 (Depression). See section C.6.b. 
 
 
Data Analysis  
  
Overview. The proposed study’s primary aims are to examine the implementation and effectiveness and cost of 
computerized screening and BHS-delivered intervention. In context of the stepped wedge study design, HIV, 
SUD and depression outcomes will be compared between the intervention and observational phases. We will 
evaluate both short-term (i.e., within intervention phase) and long-term (i.e., within 1-2 years after initial 
intervention phase). As described in detail below, we will also employ two complementary study designs: 1) a 
repeated cross-section design with a different mix of individuals in each cluster (clinic) for examining 
implementation outcomes (e.g., Aim 1: screening rate) and 2) a cohort design for examining effectiveness in 
outcomes over time within the same individuals (e.g., Aim 2: retention in HIV care). Finally, we will evaluate 
implementation cost and cost-effectiveness, and perform qualitative interviews to assess provider- and clinic-
level implementation barriers.  
 
General Random Effects Model. Data analysis for both the cross-sectional and cohort approaches will use the 
random effects modeling framework which accounts for correlation between individuals within clusters and 
over time and is generalizable to non-normal distributions.11 The general linear model is written as: Yijk=mij + ai + 
bj + QXij + 
eijk where Yijk denotes the outcome for individual k in cluster i at time j. mij is the mean for cluster i at time j, ai is 
the random effect for cluster i, βj represent fixed effect corresponding to time j, and Xij is the treatment mode in 
cluster i at time j. αi and εijk are independently distributed with appropriate distributions (e.g., logistic for binary 
outcomes, normal for continuous outcomes) and individual level covariates (e.g., gender, age, severity) to 
describe mean responses may be added to this model. The underlying distribution from the model can be linked 
to a linear predictor by appropriate transformations (e.g. binary outcome linearized using a logit link). These 
models may be estimated using SAS PROC MIXED or PROC NLMIXED for normal or non-normal distributions 
allowing for correlation between observations and variable cluster sizes. 
 
Aim 1. To evaluate the implementation of computerized SUD and depression screening and BHS-delivered 
intervention in HIV primary care. Every cluster (site) will have 4 repeated measures (at least 1 in the 
observation phase) and up to 3 in the intervention phase. We use screening rate to illustrate estimation and 
hypothesis testing (Aim 1, Hypothesis 1). Using patient-level data on screening (=1 if screened, 0 otherwise) we 
have a dichotomous dependent variable with patients clustered within clinics, implying correlated observations. 
The random effects (mixed) model framework above will be used to estimate model parameters. With the 
observation phase as the reference condition, we expect a significant positive coefficient associated with the 
indicator variable for intervention phase. We expect negative coefficients for age (implying younger individuals 
more likely to be screened) and CD4 count, implying those with greater HIV disease severity will be more likely 
to be screened (Aim 1, Hypothesis 2). We examine association between clinic factors using indicator variables 
for clinics since we have one clinic per intervention phase. For example, using San Francisco as the reference 
group, we will examine the coefficient of the indicator variables for the other facilities (e.g., OAK=1 if Oakland, 0 
otherwise). The sign and magnitude of these coefficients will determine if there are clinic differences in 
implementation outcomes. We will replicate these analyses for examining BHS-delivered intervention among 
those who screen positive for SUD or depression (Aim 1, Hypothesis 3).We expect significant positive 
coefficients for the indicator variable for high severity indicating that more severe individuals are more likely to 

 
 
 
 
KPNC IRB Version: 06/22/2018 
receive BHS-delivered intervention than those with low or moderate severity. We will also examine the 
intervention’s sustainability by comparing outcomes in the intervention’s first and second year; a one-sided test 
of the coefficient of time will indicate whether outcomes were sustained in year 2. 
 
Aim 2. To examine the effectiveness of computerized screening and BHS-delivered intervention in HIV primary 
care among patients with moderate or high SUD risk or depression severity. We will first compare treatment 
initiation in specialty clinic for those with severe SUD or depression problems and anti-depression medication 
prescription rates, among all patients who screen positive (Aim 2, Hypothesis 1). The repeated cross-section 
analyses will be conducted using the random effects (mixed) model framework as above with the dependent 
variable (whether a patient received any prescription orders initiation with SUD specialty care or Psychiatry 
treatment for their SUD (depression) being dichotomous. Drawing from preliminary analyses, we will include all 
patient-level predictors (e.g. demographics such as age and gender, severity) that are likely to be related to 
these outcomes. These models will use logistic regression since the outcome is dichotomous. We will use the 
SAS® NLMIXED or similar procedure for analyses. Sample size permitting, we will also examine the use of 
Buprenorphine/Naloxone for the subset of HIV patients with opioid addiction.243,244 
We will also examine the longitudinal (cohort) effect of the intervention on clinical outcomes by 
comparing changes in HIV clinical parameters (Aim 2, Hypothesis 2) and SUD and depression severity (Aim 2, 
Hypothesis 3) for those who have return visits within the study observation period, adjusting for time between 
visits. We will examine changes in these values between the patient’s initial screening and their subsequent 
visits using the random effects model described above. For example, we expect the coefficient of interest of the 
time x intervention variable to be positive, which would imply that BHS-delivered intervention results in greater 
increase in ART adherence. 
 
Aim 3. To determine implementation costs and cost-effectiveness of screening and BHS-delivered 
intervention. Costs will be obtained as described in C.8.c. We will assess the incremental cost-effectiveness ratio 
as the additional cost per unit of outcome (e.g., 1% increase in mean ART adherence) between the observation 
and intervention phases. For example, if the BHS-delivered intervention costs $100 more per patient than during 
the observational phase and increases mean ART adherence from 85% to 90%, then the intervention costs 
$2000 per additional unit of adherence. A cost-effectiveness acceptability curve will be obtained from the 
distribution of the ICER which will be constructed using the bootstrap method based on the variances of 
estimates of costs and effectiveness as we have done in prior work.166 This can be used by policy-makers to 
determine potential gain in effectiveness for acceptable cost thresholds.