Prospective Validation Study of a Antiretroviral Therapy (ARV) Decision Support Tool
Phase: NA
Conditions: Hiv
Sponsor: Johns Hopkins University

36  
  
 
 
Prospective Validation Study of 
a Antiretroviral Therapy (ARV) 
Decision Support Tool 
 
 
 
NCT04080765 
 
 
 
July 1, 2019 
 

37  
  
****************************************************************************** 
********  
  
1.  
Abstract  
  
The Centers for Disease Control and Prevention (CDC) estimate that 1.2 million people currently 
live with HIV in the United States; however, less than half have been prescribed antiretroviral 
(ARV) therapy and are virally suppressed [1,2,3]. ARV regimens generally include multiple 
drugs from various mechanistic classes. There are currently over 25 ARV drugs approved by the 
U.S. Food and Drug Administration (FDA), leading to a wide array of potential ARV 
combinations [4,5].   
Treatment of HIV with antiretroviral drugs (ARV) requires complex medical decision-making 
that incorporates multiple individual patient variables. The existing guidelines provided by the 
Department of Health and Human Services (DHHS) provide excellent recommendations on 
choosing initial ARV regimens, but they are dense, cumbersome to navigate, especially for 
inexperienced providers and for patients with significant co-morbidities or drug resistance 
mutations [4]. This is particularly problematic given that recent studies have shown a large 
proportion of HIV care is being provided by primary care practitioners, who may not have 
received specific training in HIV care.[6] There is thus a need for innovation to improve provider 
knowledge of HIV evidence based practice recommendations and emerging scientific data, and 
to assist clinicians in decision making when selecting an appropriate ARV regimen.  
Faculty at Johns Hopkins have recently created HIV-ASSIST (https://www.hivassist.com/) an 
online interactive patient-based tool that allows for delivery of tailored educational content that 
may assist providers to learn HIV treatment principles and support decision making on ARV 
usage.   
Our study design involves a randomized trial of trainees from the Johns Hopkins School of 
Medicine (SOM) and Residents from Johns Hopkins Medical Institutions, comparing ARV 
selection for ten hypothetical cases among trainees that have access to the current DHHS 
guidelines versus trainees that have access to HIV-ASSIST.  Our hypothesis is that there will be 
a statistically significant higher percentage of appropriate ART selection (comparing to a 
reference standard of HIV experts) in the group using HIV-ASSIST compared to the group using 
DHHS guidelines alone.   
  
We have previously constructed ten standardized case scenarios in an electronic survey format, 
and ascertained ARV prescribing preferences from experienced HIV clinicians to serve as the 
reference standard, to define ‘appropriate ART selection’.   We will enroll student and resident 
trainees from Johns Hopkins, who will be randomized to a control group (with access to current 
DHHS guidelines) or an intervention group (with access to HIV-ASSIST and current guidelines).  
We will deliver the same ten case scenarios via electronic survey to study participants, who will 
be asked to recommend ARV regimens for each case. We will report the percent concordance to 
the reference standard (i.e percent appropriate ART selection), comparing the intervention group 
to the control group.   
  
  

38  
  
2.  
Objectives   
a) 
Primary: To determine the difference in percent concordance (with reference standard;  
i.e. percent appropriate ART selection) between the group using the DHHS guidelines for ARV 
selection and the group using HIVASSIST for ARV selection.   
b) 
Secondary: To determine the difference in time forARV selection for standardized case  
scenarios between trainees using HIVASSIST and those using the DHHS guidelines.  
  
3.  
Background   
  
The Centers for Disease Control and Prevention (CDC) estimate that 1.2 million people currently 
live with HIV in the United States; however, only less than half have been prescribed 
antiretroviral (ARV) therapy and are virally suppressed. ARV regimens generally include 
multiple drugs from various mechanistic classes. There are currently over 30 ARV drugs in 7 
different classes that have been approved by the U.S. Food and Drug Administration (FDA), 
leading to a wide array of potential ARV combinations.   
Choosing these drugs is a complex medical decision, and current guidelines advocate 
individualized therapy that accounts for multiple patient characteristics. Factors that must be 
considered include: previous ARV treatment, adherence, HIV resistance mutations, viral load, 
comorbidities (e.g., renal function, cardiovascular disease), and drug interactions.  Synthesizing 
each of these modifying factors is necessary, but complex and time consuming for providers.  
Challenges in prescribing ARV regimens in a primary care setting become increasingly relevant 
as HIV continues to become a manageable chronic illness. Recent studies have shown that a 
large proportion of HIV care is provided by primary care practitioners, and often infectious 
disease specialists feel uncomfortable managing patient’s comorbidities like hypertension and 
diabetes. With the growing need for increasing involvement of primary care physicians in HIV 
care management, there is also a growing need to develop tools that allow more effective and 
efficient learning of HIV treatment principles and that support ARV decision making.   
To solve this problem, faculty at Johns Hopkins have created HIV-ASSIST, an online interactive 
patient-based tool that allows for individualized treatment recommendations through the 
synthesis of patient-specific factors with existing guidelines and evidence. HIV-ASSIST offers 
efficient real-time learning of treatment principles and augmented decision support for care 
providers by delivering informational material about the evidence behind suggested regimens 
during the use of the tool.   
Our goal is to evaluate the impact of using HIV-ASSIST compared to DHHS guidelines in 
making ARV selections (for both ARV naïve and experienced patients) among trainees that are 
not experts in HIV care. We have previously ascertained ARV preferences for 10 case scenarios 
from HIV experts to serve as the reference standard. We will the proportion of ARV selections 
that are concordant with the reference standard (% appropriate ART selection) between a group 
of trainees using DHHS guidelines, and a group of trainees using HIVASSIST as well as the 
guidelines   
  
3.  
Study Procedures  
  

39  
  
a. 
Study design: Randomized trial of students and trainees to assess the impact of 
HIVASSIST utilization on ARV selection for standardized case scenarios compared to usage of 
DHHS guidelines alone.   
  
b. 
Study participants and settings:  We will enroll up to 200 participants. Students in the 
Johns Hopkins University SOM or SON (Baltimore, Maryland), and/or internal medicine 
residents at Johns Hopkins Hospital will be eligible to participate.  Participants will be invited to 
participate via an email to class or program listservs or via flyers posted in the school of 
medicine or nursing buildings.  Individuals that are interested will have a link to a Qualtrics 
survey where they will receive additional study details.   
  
c. 
Enrollment/Randomization: An email will describe the study and will contain a link to 
the survey; the email will also inform them that their participation is completely voluntary, and 
that their decision to participate (or not) will in no way affect their academic standing. They will 
also be informed that no information about individual results/surveys will be shared with their 
school administration.  All participants enrolling in the study will be offered an incentive of $50 
gift card or equivalent; they will receive a link to this gift card via email after their participation. 
Study enrollment will take place over 12 months. Participants will be recruited on a rolling basis 
throughout the study period.   
• 
Qualtrics will be used to deliver the survey, including an initial page explaining the study 
and asking for consent.   
• 
Qualtrics will automatically randomize participants to different groups as a part of the 
survey process, using Qualtrics internal random sequence generator. We do not include any 
stratifications for randomization as limited demographic detail is collected. The participants were 
not blinded to their allocation, as they were told which decision-support aids they would have 
available.  
• 
The survey will be conducted in one session, and the time to complete the survey will be 
captured. Individuals not completing the survey in one session will be excluded. d. Data 
collection/Survey:   
• 
Development of standardized case scenarios: We have previously designed 10 clinical 
case scenarios for a range of patient situations, including a) ARV naïve b) ARV experienced, 
suppressed or c) ARV experienced, failing patients (with and without mutations, comorbidities, 
and common co-medications).  In a prior prospective survey, we have ascertained ARV 
preferences for each case scenario among a cohort of experienced HIV clinicians. These ARV 
choices for each case scenario will be considered a reference standard to judge acceptability of 
ARV selections in the study cohort in this study.  
• 
We will collect basic demographic details of study participant including degree program 
(NP,MD), level of training (M1, M2), and level of experience with HIV care.  
• 
Each survey will have its own video tutorial briefly explaining how to use the DHHS 
guidelines and/or HIV-ASSIST. A link to the video will be provided in the survey. The control 
group will have a link to the DHHS guidelines in the survey. The experimental group will have a 
link to both the DHHS guidelines, and the HIVASSIST tool. Each participant must acknowledge 
that they have successfully accessed their respective resources to help inform their ARV regimen 
selections. A question at the beginning of both surveys will restrict the participants from 

40  
  
advancing until they have filled information from either DHHS guidelines or HIV-ASSIST 
correctly. This will be a test of the participant’s understanding of how to use the survey and the 
respective regimen selection aid.  
• 
Study participants will be guided through a survey in which the ten standardized case 
scenarios (vignettes with relevant HIV and patient variables are provided) are presented, and 
participants will be asked to propose their ARV treatment regimens.  The participant’s ARV 
choices will be recorded. Through Qualtrics software, we will also record the time needed to 
complete the study.  
  
  
e.  
Statistical considerations:   
  
Definitions of appropriate ART selection:  
Current DHHS guidelines recommend an individualized approach to ARV selection based on 
multiple considerations.  Several regimens are listed as acceptable for ARV-naïve patients as 
‘regimens recommended for most people living with HIV’, with specific instructions on 
comorbidities or drug interactions that may lead to selection from a list of ‘regimens 
recommended in some clinical scenarios’.  For ARV-experienced patients, the guidelines outline 
a list of general treatment principles to consider, such as including “at least two, and preferably 
three, fully active agents.”  
  
For each case-scenario in the survey, we have previously collected ARV selections from a cohort 
of experienced HIV clinicians,  recognizing that there may be multiple reasonable ARV regimen 
options for each case scenario. The ARV choices from the prior survey will be used as the 
reference standard.    
  
Each question will be graded as ‘concordant/appropriate’ if the ARV regimens selected by the 
respondent matches an ARV regimen of any HIV expert, or is consistent with current DHHS 
guidelines (e.g., selection from the list of “regimens recommended for most people living with 
HIV’’ for treatment-naïve case-scenarios; regimens which “include at least two, and preferably 
three, fully active agents”    
  
Our primary outcome is the proportion of appropriate ARV selection for standardized case 
scenarios, comparing between trainees and expert HIV clinicians and/or DHHS guidelines. 
Specifically, we will assess:   
  
a) 
The difference in percent appropriate/concordant between the control group, and the 
intervention group that has access to HIV-ASSIST  
Hypothesis:  There will be a statistically significant higher percent concordance in the group 
using HIV-ASSIST compared to the group using DHHS guidelines alone.   
  
Our secondary outcome will look at the time required for each group to reach a correct answer. 
Specifically, we will assess:   
  

41  
  
b) 
The difference in time to ARV selection for standardized case scenarios between trainees 
using HIV-ASSIST and those using the DHHS guidelines.   
Hypothesis:  There will be a statistically significant lower time to correct ARV selection in the 
group using HIV-ASSIST compared to the group using DHHS guidelines.   
  
Sample Size:  
There is limited a priori data to estimate the likely percent concordance between trainees/students 
and HIV experts. We assumed that those participants using DHHS guidelines alone (control) will 
have an average of up to 50% appropriate responses, with a wide standard deviation of up to 0.5. 
We anticipate that there will be approximately 80% appropriate ART selections with a standard 
deviation of 0.1 among those in the intervention arm. With this assumption, we would need 18 
individuals in each arm to be adequately powered to detect a difference (two-sided alpha, 0.05). 
Table 1 shows needed sample size to have 80% power to detect a difference between the control 
and intervention arms under varying assumptions. In secondary analysis, we will compare the 
control and intervention arms, stratified by level of training.  
  
DHHS MEAN %  
APPROPRIATE  
RESPONSES  
0.3  0.3  
0.3  
0.5  
0.5  
0.5  
0.3  
0.3  
0.3  
0.5  
0.5  
0.5  
HIVASSIST MEAN  
% APPROPRIATE  
RESPONSES  
0.7  0.7  
0.7  
0.7  
0.7  
0.7  
0.8  
0.8  
0.8  
0.8  
0.8  
0.8  
DHHS SD  
0.1  0.3  
0.5  
0.1  
0.3  
0.5  
0.1  
0.3  
0.5  
0.1  
0.3  
0.5  
HIVASSIST SD  
0.1  0.1  
0.1  
0.1  
0.1  
0.1  
0.1  
0.1  
0.1  
0.1  
0.1  
0.1  
Sample size needed 
per group for 80% 
power  
1  
4  
11  
4  
16  
41  
1  
1  
7  
2  
7  
18  
  
Table 1. Sample Size Variations by Expected Group Means and Standard Deviations  
  
  
4.  
Drugs/ Substances/ Devices  
a.  
N/A  
  
  
  
5.  
Risks  
a. 
This study only involves a survey with standardized cases to ascertain clinician 
preferences for a variety of HIV case scenarios. Participants may feel insecure regarding 
their self-perceived lack of knowledge regarding HIV management. However, all 
responses to the survey will be visible only to the study members. There are no other 
biological risks identified.  

42  
  
  
b. 
Steps taken to minimize the risks.: Participants will be permitted to stop participating in 
the survey at any time.  
  
c. 
Plan for reporting unanticipated problems or study deviations: Participants will be 
instructed to contact the PI if any problems occur during survey completions. Such 
problems will be reported to the IRB immediately.   
  
d. 
Financial risks to the participants: There are no financial risks to participants.  
  
6. 
Benefits: Once the data are compiled, analyzed, and published, the participants will learn 
about their collective understanding of HIV antiretroviral management. In addition, the 
results of this study will allow us to improve HIV-ASSIST, a tool that will greatly expand 
providers’ ability to provide guideline-recommended HIV care. This is particularly 
important in places where there may not be dedicated HIV training (such as other 
countries) and in places where there are no HIV providers/a provider with no formal 
training in HIV care is managing a handful of patients with HIV (which occurs in many 
rural parts of the United States).  Managing HIV in an optimal way not only provides a 
benefit to patients, it benefits society by reducing patients’ viral loads and thus reducing 
transmission to other persons.   
  
7. 
Payment and Remuneration  
•  
Participants who complete the study will receive a link through email for an electronic 
$50 Amazon gift card. There will be no penalties for not completing the study.  
  
  
8.  
Costs and Funding  
•  
The only cost of the study is the Amazon incentives that we will offer our participants to  
complete the study. These costs will be paid using internal Johns Hopkins Institute of 
Educational Excellence Berkheimer Grant, awarded to Maunank Shah (PI).  
  
9. 
Trial Registration: clinicaltrials.gov  
10. 
Roles and Responsibilities: The study was designed by Dr. Shah (PI) and Jesus Aaron 
Ramirez (Co-investigator), who also led protocol development, obtained IRB approval, 
conducted the study, and led analysis and manuscript writing.  The funders had no role in 
the design, collection, management, analysis or interpretation of data or writing of the 
report.   
11. 
The study received approval by the Ethics/IRB committee of Johns Hopkins University  
  
  
Protocol References  
1. 
CDC. HIV in the United States: At A Glance.   
https://www.cdc.gov/hiv/statistics/overview/ataglance.html [Accessed March 2018].  
2. 
Foundation KF. The HIV/AIDS Epidemic in the United States: The Basics.   

43  
  
https://www.kff.org/hivaids/fact-sheet/the-hivaids-epidemic-in-the-united-states-the-basics/.  
[Accessed May 2017].  
3. 
Weiser J, Beer L, West BT, Duke CC, Gremel GW, Skarbinski J. Qualifications, 
Demographics, Satisfaction, and Future Capacity of the HIV Care Provider Workforce in 
the United States, 2013-2014. Clin Infect Dis 2016,63:966-975.  
4. 
DHHS Panel on Antiretroviral Guidelines for Adults and Adolescents. Guidelines for the 
Use of Antiretroviral Agents in HIV-1-Infected Adults and Adolescents. 2016.  
http://aidsinfo.nih.gov/guidelines [Accessed 11/2016].  
  
5. 
Gunthard HF, Saag MS, Benson CA, del Rio C, Eron JJ, Gallant JE, et al. Antiretroviral 
Drugs for Treatment and Prevention of HIV Infection in Adults: 2016 Recommendations 
of the International Antiviral Society-USA Panel. JAMA 2016,316:191-210.  
6. 
Medicine AAoH. HIV Clinician Workforce in the US.  
https://aahivm.org/wpcontent/uploads/2017/03/FINAL-August-2016.pdf [Accessed 
March 2018].