Information Visualizations to Facilitate Clinician-patient Communication in HIV Care (Info Viz: HIV)
Phase: NA
Conditions: HIV, Communication Research
Sponsor: University of Colorado, Denver

Protocol Template 
Page 1 
CF-146, Effective 7/10/11 
 
 
Protocol #: 20-0163 
Project Title: Information Visualizations to Enhance HIV-Related Communication in Diverse   
                      Clinical Settings  
 
 
 
Principal Investigator:  Dr. Samantha Stonbraker 
Version Date: September 20, 2023 
 
 
 
 
  
 
I. Hypotheses and Specific Aims:   
 
Brief background 
 
In preliminary studies, the PI developed and evaluated a set of HIV-related infographics to improve 
clinician-patient communication and patient outcomes among Latino persons living with HIV 
(PWH). This work was conducted in the Dominican Republic (DR) and in New York City. 
Simultaneously, the PI developed a Substitutable Medical Applications and Reusable Technology 
(SMART)® application (app) compatible with the Fast Healthcare Interoperability Resources 
(FHIR)®, which is the standard for connecting mobile apps and electronic health records. She has 
incorporated her evidence-based infographics into this app so her intervention can be scaled up to 
reach clinicians and their Latino patients living with HIV globally. The next steps in this work are to 
increase generalizability of the intervention by adapting it to an additional Latino group (persons of 
predominantly Mexican descent) and to test the efficacy of the intervention to improve clinician-
patient communication and health outcomes. 
 
The aims of this study are therefore to: 
 
Aim 1 Adapt previously designed and evaluated infographics for relevance to persons with HIV 
(PWH) of predominantly Mexican descent through individual interviews and incorporate final 
infographics into the mobile app prior to the proposed RCT. 
 
Aim 2 Conduct a multi-site randomized controlled trial to assess the efficacy of the intervention 
(infographics administered through a mobile app) as compared to standard care in improving 
treatment outcomes (CD4 count, viral load, self-efficacy to manage HIV, among others) of 
Latino PWH and determine how resultant effects vary over time (baseline to 9-months) and 
between sites (DR and CO). 
 
Aim 3 Assess implementation factors of integrating the SMART on FHIR app into EHRs through in-
depth interviews and leverage findings to ensure the app effectively interfaces with EHRs in the 
CO and the DR. 
 
Complementary to this work, a better understanding of how distinct Latino groups perceive HIV 
infection and prefer to communicate regarding HIV is needed so interventions can be modified 
accordingly. Thus, the last aim of this study is exploratory.  
 
Aim 4 (Exploratory) Conduct in-depth interviews with PWH of predominantly Mexican descent to 
explore their perceptions of HIV infection and preferences for health communication. Qualitative 
findings will be integrated with those from the NY and DR cohorts to detail variations and 
similarities, and identify areas for intervention refinement. 
 
II. Background and Significance:  
 
w 
COMIRB Protocol  
COLORADO MULTIPLE INSTITUTIONAL REVIEW BOARD 
CAMPUS BOX F-490    TELEPHONE:  303-724-1055  Fax:  303-724-
0990 
 
 

Protocol Template 
Page 2 
CF-146, Effective 7/10/11 
 
Latinos are the largest and fastest growing minority group in the United States (US)1 and 
they are disproportionately affected by HIV.2-5 Latinos experience higher HIV prevalence, faster 
disease progression, more HIV-related deaths, and a marked delay in the diagnosis of new 
infections.2,4,6 Many factors contribute to HIV-related health disparities, of which low health literacy 
(LHL) is a major contributor.7,8 LHL is an established concern globally9,10 and Spanish-speaking, 
less educated, and/or foreign-born Latinos are more likely to have LHL than those born in the 
US.9,11 LHL can lead to lesser ability to self-manage which leads to worse health outcomes.10,12 
The proposed intervention is designed to meet the self-management information needs of Latino 
PWH with LHL, which can lead to better health outcomes.13-15 
 
Infographics are technologies that can help clinicians effectively provide health information 
to individuals with varying language capabilities and LHL.16,17 Communication regarding 
complex chronic conditions, such as HIV, is challenging, as providers must provide large amounts 
of data to patients in a meaningful way.18 This is especially difficult when a patient’s preferred 
language and culture differs from that of their provider.2,19 When clinician-patient communication is 
enhanced, it can lead to better self-management by improving behaviors such as adherence to 
antiretroviral therapy (ART).20-22 Infographics can improve clinical communication because, when 
rigorously designed, they contain a depth and breadth of content displayed in a visually logical 
way.23,24 This is useful amidst culture and language differences as infographics provide images 
familiar to patients, which can help explain complicated processes, augment attention span, and 
improve information recall.25-28 Infographics such as the ones in this intervention can, therefore, 
improve self-management by helping clinicians provide information in an understandable way.24,28  
 
Therapeutic communication methods, such as motivational interviewing (MI), targeted at 
HIV self-management behaviors can help clinicians influence behavior change.29,30 During 
MI, specific techniques such as reflective listening and asking open-ended questions are used to 
emphasize collaboration and acceptance.29 Incorporating targeted training in this method, even if it 
is brief,29,31,32 into interventions enables providers to deliver health education messages that 
promote healthier behaviors.29,33 The proposed intervention includes a brief training in MI, so 
clinicians can more effectively provide information to PWH. 
 
The integration of evidence-based infographics into EHRs will enable infographics to be 
easily accessible and efficiently scaled up for use at clinical sites locally and globally. The 
applicant’s previously designed and tested infographics are now automated in a Substitutable 
Medical Applications and Reusable Technology (SMART)® Application (app) compatible with the 
Fast Healthcare Interoperability Resources (FHIR)®.34 This is the standard for communicating 
between apps and electronic health records (EHR) via an application programming interface 
(API).34 The resultant app will enable providers to access evidence-based infographics to facilitate 
HIV-related communication at a large scale. Unfortunately, electronic health tools, and their 
advantages, are more widely available in developed settings.35-37 But, a recent push to develop 
open-source electronic health records in low and middle income countries will enable FHIR 
compatible devices to be accessible to those living in limited-resource settings.38-40 The proposed 
study will leverage this advancement and provide a valuable platform and guide through which 
other researchers and developers can expand technologies to meet the needs of organizations and 
populations in limited-resource settings globally. 
 
Understanding if infographics are acceptable to non-Latino populations is an 
important step towards generalizability and future scalability of this 
intervention.  As the infographics in this study were designed and tested with Latino 
persons with HIV, we do not yet know if the infographics are relevant for other 
populations with HIV. Other US-based races/ethnicities that experience high HIV 
prevalences and negative HIV-related health outcomes are non-Latino Black and 
White persons (65% of HIV infections), especially those in high-risk groups such as 
Men who have Sex with Men and Transgender Women.3,41,42 Therefore, it is critical to 
explore if the intervention infographics are meaningful in these populations. As such, 

Protocol Template 
Page 3 
CF-146, Effective 7/10/11 
 
we propose an extension of the study to ascertain the perspectives of PWH who 
identify as Black or White.III. Preliminary Studies/Progress Report 
 
The PI has rigorously and iteratively developed an intervention to assist clinicians 
provide HIV-related health information to PWH in several preliminary NINR-funded 
studies. Her dissertation (T32 NR013454), conducted in the Dominican Republic 
(DR), identified a need for this type of intervention, among other findings.43-45 In 
response, her postdoctoral work (T32 NR007969) was to develop evidence-based 
infographics to assist clinician-patient communication in the DR,46,47 as well-designed 
infographics can improve patients’ understanding of concepts and health 
behaviors.23,25,26,28 During her K99 (K99 NR017829), the PI first assessed the 
feasibility of using infographics to improve clinical communication with, and outcomes 
of, PWH (CD4 count among others) in the DR (first K99 feasibility study). She then 
adapted infographics to Latino PWH in New York (NY), who are mainly of Dominican 
origin/descent, and completed a second feasibility trial in New York. 
 
Summary of K99 feasibility study results 
Results from the first K99 feasibility study, conducted with a validated methodology in 
the DR, demonstrate that using infographics to improve clinician-patient 
communication and health outcomes is feasible and acceptable. Qualitative 
responses from clinicians who administered the intervention were overwhelmingly 
supportive of not only continuing to use the intervention, but indicated infographics 
should be further developed to encompass more topics and be made available 
outside of the study, to other providers at the clinic. Statistically significant changes 
were observed in the primary outcome measure (CD4 count) of patient participants 
between baseline and 9-month visits. As several other outcome measures (viral load, 
engagement with clinician, HIV-related knowledge) showed statistically significant 
improvements over this time period as well, it is likely the intervention had a positive 
effect on these individuals. As with all studies, there were limitations. First, all 
participants were recruited from one health clinic in the Dominican Republic so results 
may not be generalizable. The second feasibility trial, recently completed in NY, will 
provide further evidence of feasibility and usability among a similar population in a 
different clinical context. Another main limitation of both feasibility studies is that there 
was not a control group, so from current data we are unable to establish how much of 
the improvement seen in the Dominican cohort was due to infographics and what was 
attributable to confounders. When larger cohorts are randomized to the treatment and 
control groups in the planned multi-site trial during the R00 phase, it will be possible 
to compare results of participants who received the intervention to those who did not, 
which will provide further evidence of the actual treatment effect.  
 
IV. Research Methods  

Protocol Template 
Page 4 
CF-146, Effective 7/10/11 
 
 
A. Outcome Measure(s):  Outcomes were selected using a conceptual model depicting 
distinct pathways through which clinician-patient communication can influence health 
outcomes.57 In this model, outcomes that are affected by clinician-patient communication 
are separated into three distinct categories, proximal, intermediate, and long-term. Table 1 
summarizes the corresponding outcome measures and the scale or type of data that will 
be collected for each.  
 
Please note, the primary outcome of interest is CD4 count, which will be collected 
with a review of participants’ medical records following each study visit. Participants’ 
viral load and current antiretroviral therapy regimen will also be extracted from participants’ 
medical records following each study visit. Other measures will be in the REDCap 
questionnaires that will be administered using iPads. 
 
B. Description of Populations to be Enrolled:   
Individual interviews to adapt infographics to the Latino population of Denver, CO 
Patient participants for interviews to identify necessary changes to infographics will be 
adults (≥ 18 years), Hispanic/Latino, English or Spanish-speaking, and living with HIV. If 
patient participants are pregnant, they may still be included. 
 
RCT - Interventionists 
Inclusion criteria for clinicians who will administer the intervention during the RCT will be 
those who have offered HIV-related care for greater than one year and are not planning to 
leave the clinic for the next year.  
 
RCT - Patient participants  
Inclusion criteria for patient participants for the RCT will be that patients are adult (≥ 18 
years), Hispanic/Latino, English or Spanish-speaking, living with HIV with a detectable viral 
load (>30 or >40 copies/mL depending on the laboratory test) at any point in the past year 
or be newly diagnosed with HIV, or have an indication of adherence risk as identified by 
their provider and be planning to receive care at the study site for the next year. If patient 
participants are pregnant, they may still be 
included in the study. Participants from the US 
site are expected to reflect the demographics 
of Latino patients at the Center for Positive 
Health (CPH), who are predominantly male 
(85%) and Mexican (90%). Participants from 
the DR will reflect the population at CFLR, 
which is majority female (57%) and Dominican 
(81%). 
 
Key-stakeholders who participate in in-
depth interviews 
Key-stakeholders for in-depth interviews to 
assess implementation factors of incorporating 
the SMART on FHIR app into electronic health 
records will be clinicians or other employees at 
both sites who provide HIV-related education 
to patients, have been working in HIV care for 
greater than one year, and interact with the 
electronic health record at their site on no less 
than a weekly basis. 
 
Usability testing of Creole version of the 
mobile app 
Table 1. Outcome measures  
Outcome 
Measure 
Proximal Outcomes 
Information 
recall 
Questions pertaining to final 
infographic content  
Engagement 
with clinician 
Engagement with Health Care 
providers Scale*48  
Satisfaction 
with care 
1. Two questions assessing 
satisfaction*49 
2. CHAPS Survey 2.0*50 
Intermediate Outcomes 
Self-efficacy to 
manage HIV 
Self-efficacy to manage 
chronic conditions scale*51 
Adherence 
1. SMAQ 6 item scale*52 
2. Medication possession 
ratio (MPR)53,54 
Long-term Outcomes 
Health related 
quality of life 
PROMIS 29*55 
Health status 
1. CD4 count and viral load 
2. Health Status 
Assessment*56 
Note: * scale validated in Spanish  

Protocol Template 
Page 5 
CF-146, Effective 7/10/11 
 
Participants will be employees of the research site in the Dominican Republic who are 
Spanish/Creole bilingual and provide HIV-related health information to patients in Creole. 
 
Validation of infographics with non-Latino PWH 
Inclusion criteria for this phase of the study will be 1) Does not identify as Hispanic/Latino 
2) Identifies as White or Black race 3) Has HIV infection, confirmed in electronic health 
record 4) (≥ 18 years) of age. 
 
C. Study Design and Research Methods   
 
Study Design 
 
This will be a mixed-methods study with several components:  
 
1) Individual interviews (Aim 1) 
 
2) Multi-site randomized controlled trial (RCT) (Aim 2) 
 
3) In-depth individual interviews with Key stakeholders (Aim 3) 
 
4) In-depth interviews with patient participants from the RCT (Aim 4) 
 
5)  Interviews with non-Latino PWH (additional component of study added 09/20/23) 
 
 
 
Study Settings 
The trial will take place at two outpatient clinics specializing in HIV treatment. The US site 
will be CPH, in Denver, CO and the Dominican site is Clínica de Familia La Romana 
(CFLR). These are the ideal clinical locations for this trial because they offer services to 
two large, distinct Latino populations who are the target of this intervention (Mexican and 
Dominican PWH). Including a site in the US and one in the DR will also further augment 
intervention generalizability. Additionally, the EHR infrastructures at these sites are ideal 
for further development of the SMART on FHIR app, which will enable scalability to sites 
across the US and abroad. CPH in CO uses EPIC as their EHR and CFLR uses the open-
source EHR, Odoo. Both systems can interface with SMART apps,58 and their 
administrative structures permit app integration with EHRs. 
 
Individual interviews to adapt infographics to Colorado context 
To satisfy Aim 1, to adapt previously designed and tested infographics for relevance to 
persons with HIV (PWH) of predominantly Mexican descent through individual interviews 
and incorporate final infographics into the mobile app prior to the proposed RCT, up to 15 
individual interviews will be conducted (in English and in Spanish) with PWH after they 
attend a regularly scheduled clinic visit. During interviews, infographics (n=16) will be 
shown and participants’ feedback on their interpretability and acceptability and suggestions 
for refinement will be solicited. Interviews will be audio recorded and transcribed, then 
transcripts will be reviewed to extract meaning and identify necessary refinements to 
infographics. Corresponding refinements will be incorporated into infographics prior to the 
proposed trial. 
 
SMART on FHIR App Development 
A prototype of the app will be ready for the proposed trial, with a planned start date in 
January 2021. But, because full integration with the EHR at both sites by then is not 
feasible, the EHR functionality of the app will be continuously developed, separate from the 
trial, so the final product of this study will be a functional app that can be incorporated into 

Protocol Template 
Page 6 
CF-146, Effective 7/10/11 
 
the EHRs at both sites. To maintain intervention fidelity, the prototype will be used 
throughout the trial. 
 
Infographic Intervention  
Intervention infographics are available in English and in Spanish and were created using a 
rigorous iterative design process.46 They are intended to assist clinicians deliver patient 
education through culturally appropriate and visually appealing representations of key HIV-
related concepts, systematically identified in a preliminary study.47 For the trial, up to five 
clinicians at each site will be selected to use infographics. These “interventionists” will first 
attend a training, held by the study team, on how to use infographics to guide education 
using motivational interviewing (MI),30 the mobile health application that contains the 
infographics, and on how to maintain intervention fidelity.59,60 During training, key concepts 
of infographics and MI will be reviewed then attendees will conduct “mock visits” to review 
learned components of MI, the mobile application and infographic content, confirm 
understanding of study procedures, and ensure interventionists uniformly administer the 
intervention. Additional training sessions will be held during the study to reinforce 
interventionists’ ability to implement learned skills. Also, during the second training session, 
interventionists will complete a questionaire (the system usability scale (SUS)) to assess 
usability of the mobile health app containing infographics. During the study, in normal clinic 
visits, interventionists will use MI and infographics to guide education with participants 
using the infographics that will appear in the app, 
easily accessible on iPads. If a provider is 
conducting a telehealth visit with a study 
participant in the treatment group, during times of 
COVID-19 surges or for any other reason, they 
will have paper copies of the infographics to 
show participants during video visits. 
 
 
 
RCT study procedures 
During recruitment, clinic staff at each site will 
work closely with the Research Assistants (RA) 
to identify eligible participants on or before each 
recruitment day by verifying eligibility criteria in 
the electronic health records (EHR).  
 
At the DR site, the RA will notify clinic staff of 
eligible participants at the beginning of each 
recruitment day. Then, when an eligible 
participant presents to the clinic for their regularly 
scheduled appointment, initial contact will be 
made by clinic staff (face to face) who will 
introduce the study by providing a brief description of the study using the approved 
recruitment script and will then introduce interested participants to the RA, who will further 
explain the study, confirm eligibility, and complete informed consent and randomization.  
 
At the US site, the RA will identify eligible participants for recruitment days the day before. 
Then, the RA will call eligible participants to confirm eligibility and explain the study by 
providing a brief description using the approved recruitment script. If a potential participant 
is interested, they will be invited to come to the clinic an hour before their scheduled 
appointment to complete the study visit. When a participant presents to the clinic, the RA 
will further explain the study, confirm eligibility, and complete informed consent and 
randomization. If the baseline/enrollment visit will be completed virtually, the procedures for 
recruitment/enrollment will be exactly the same, they will just be conducted over the phone 
Table 2. Study activities at each visit in 
the RCT 
Time 
point 
Activity and group (Treatment 
(Tx), Control (Cl), or Both) 
Baseline 
 
• Randomization 
• Baseline assessment (Both)  
• Standard Care (Cl)  
• Exposure to intervention (Tx) 
3 months 
• Standard Care (Cl) 
• Exposure to intervention (Tx)  
• Follow-up assessment (Both) 
6 months 
• Standard Care (Cl) 
• Exposure to intervention (Tx) 
• Follow-up assessment (Both) 
• In-depth interviews with PWH 
begin (Both) 
9 months • Follow up assessment (Both) 
Study end 
• Follow-up interviews for 
interventionists (5 per site) 
• In-depth interviews with key 
stakeholders (15 per site) 

Protocol Template 
Page 7 
CF-146, Effective 7/10/11 
 
or via a HIPAA compliant platform such as Skype or Zoom. During virtual enrollment visits, 
the RA will complete informed consent electronically through REDCap. 
 
Permuted block randomization stratified by clinical site will be computer generated a priori 
and placed in ordered opaque envelopes by a study team member who is not recruiting; 
thus, group assignments will be blind to the study team.61 Procedures will then take place 
as described in Table 2.  
 
Time intervals of study visits (baseline, 3-, 6-,and 9-months) were selected as CD4 count 
and viral load may change at three months after ART initiation.62 Study participants will 
have visits with interventionists (treatment group) or other providers (control group). Prior 
to the study, standard education at each site will be evaluated to ensure the only difference 
between treatment and control groups is the infographic intervention. The intervention will 
not be administered at 9-month visits to enable assessment of long-term effects. No extra 
time will be allowed for intervention study visits and participants will be reminded of visits 
via phone. Half of intervention visits (randomly selected) will be audio recorded and a 
checklist of intervention components will be independently completed by two study team 
members, then compared to verify intervention fidelity. 
 
In-depth interviews with key stakeholders  
Semi-structured individual interviews will be conducted with up to 15 clinicians at each site 
who provide patient education to establish preferences and recommendations for 
integrating the app into each EHR (Aim 3). Interview guides will be rigorously developed63 
and questions will be guided with a theoretical model.64 
 
Usability testing of Creole version of the mobile application 
 
As the study team had the opportunity to translate the mobile application for this project 
into Haitian Creole, we will recruit five additional clinic employees who provide HIV-related 
health education to PWH to use and evaluate the Creole version of the app. Following 
recruitment, they will complete informed consent and then will participate in a 1 hour 
training session, conducted by the study team, where they will learn about the infographics 
in the app and also learn about the app itself. These five participants will then use the app 
with patients who speak Creole for two weeks. Then, they will participate in a focus group 
where they will provide feedback on the app and suggestions for improvement. During the 
group, participants will be provided prinotouts of the infographics and of the screens in the 
mobile app so they can make changes to the Creole versions directly. They will also 
complete the SUS, the same usability scale that the study interventionists completed. 
 
Assessment of infographics with non-Latino PWH 
 
We will recruit an additional 50 PWH (N=25 Black and N=25 White) from the Center for 
Positive Health at Denver Health to help us understand if infographics are meaningful to 
non-Hispanic/Latino Persons with HIV. During Recruitment, Medical Assistants at the 
Center for Positive Health will introduce eligible participants (as confirmed in their medical 
records) and if they are interested in participating, will refer them to speak with an on site 
study team member. The study team member will then further explain the study, answer 
any questions. When interested and eligible patients agree to participate, the on-site study 
team member will complete verbal informed consent (see attached document) and 
complete the study visit. 
 
During the visit, the study team member will show participants all of the infographics in the 
set and ask them questions about each one. After seeing the full set of infographics, 
participants will answer a set of questions that is about infographics generally and will then 

Protocol Template 
Page 8 
CF-146, Effective 7/10/11 
 
complete a very short (3 question) recorded interview to ascertain their perspectives of 
infographics and their potential use in the clinic.  
 
After completion of the study visit, the study team member will provide the participant with 
a $40 gift card for their time. 
 
At study completion, we will provide a $50 gift cards to the Medical Assistants and clinic 
staff who assist in recruiting. 
 
D. Description, Risks and Justification of Procedures and Data Collection Tools 
 
Description of data collection tools 
 
D.1. Individual interviews to adapt infographics to the Colorado context. 
Participants will initially complete a brief demographic questionnaire. Then, they will 
complete individual interviews with a study team member. Interviews will be audio recorded 
and later professionally transcribed. 
 
D.2. RCT 
 
D.2.1. Medical record review 
Data will be extracted from medical records after each study visit and will include the 
primary outcome CD4 count as well as most recent viral load and currently prescribed 
ART regimen. 
 
D.2.2. Baseline assessments  
Baseline assessment will be administered either virtually or in-person at baseline visits and 
will collect demographics (age, length of time living with HIV, sex, gender, income, level of 
education among others), assess health literacy levels with two validated assessments, the 
Short Assessment of Health Literacy - Spanish & English (SAHL S&E)65 and the Newest 
Vital Sign (NVS),66 include a measure of acculturation, the Short Acculturation Scale for 
Hispanics (SASH),67 and all other outcome measures not extracted from medical records 
(Table 1). 
 
D.2.3. Follow up assessments  
Follow up assessments will be administered either virtually or in-person at 3-, 6-, and 9-
month visits and will include outcome measures (Table 1) and open-ended questions 
exploring infographic usability, satisfaction with infographic design, and recommendations 
for improvement. Questions regarding interventionists’ communication strategies and 
intervention delivery will be included to further assess intervention fidelity and enable 
assessment of providers’ communication variability. 
 
D.2.4. Semi-structured in-depth interviews  
In depth interviews with patient participants at the CO site will be conducted using a 
rigorously developed iterative interview guide.63 Interviews at the CO site will include 
questions and probes to generate an explanatory model of participants’ experience living 
with HIV and communicating about it in a clinical setting,68 this data was already collected 
from participants in the DR and NY in preliminary studies. Questions to obtain feedback on 
how to adapt infographics and clinical communication will be included for all participants. 
Interviews will be audio recorded and later transcribed. 
 
D.2.5. Follow up interviews with interventionists  
Final interviews with interventionists will explore clinicians’ perspectives on the use of 
visual communication aids (infographics) during clinic visits (barriers and facilitators), 

Protocol Template 
Page 9 
CF-146, Effective 7/10/11 
 
explore their experiences working with participants in the study, and identify areas where 
infographics can be further refined. Interviews will be audio recorded and later transcribed. 
 
D.3. In-depth interviews with key stakeholders  
In depth interviews will be conducted with up to 15 clinicians at each site who provide 
patient education to establish preferences and recommendations for integrating the app 
into each EHR (Aim 3). Interview guides will be rigorously developed63 and questions will 
be guided with a theoretical model.64 Interviews will be audio recorded and later 
transcribed. 
 
E. Interviews with non-Latino PWH 
Interviews with non-Latino PWH will be conducted using an interview guide that includes 
the questions to ask for each infographic, the set of questions to ask after the participant 
has seen the entire set of infographics as well as the three interview questions to ask 
during the recorded portion of the interview. 
 
Potential Risks 
In all research there is a risk of a loss of confidentiality. Possible additional risks are 
detailed below. 
 
In initial individual interviews, participants may feel uncomfortable discussing HIV or 
answering questions about infographics. 
 
In the RCT, interventionists will be trained in a new communication technique (motivational 
interviewing) and will be using the intervention during clinic setting so no more than 
minimal risks beyond their normal daily work are anticipated. Patient participants in the 
RCT will receive health information using infographics (intervention group), otherwise they 
will receive standard care. Both treatment and control group participants will complete 
baseline and follow up questionnaires at all study visits with a study team member. 
Therefore, participants may feel uncomfortable with questions asked. Also, two study visits 
will be randomly selected to be recorded, which may make participants feel uncomfortable.  
 
Though it is unlikely because interviews will not include sensitive questions, key-
stakeholders may also feel uncomfortable answering questions in an interview setting. 
 
Participants may have difficulty accessing study visits in virtual formats. If this case 
presents itself, the RA will provide additional assistance as needed to help them gain 
familiarity with the electronic platforms to be used. 
 
The study team will strictly adhere to the procedures detailed in this protocol, 
including methods to mitigate risks to participants. 
 
Justification of procedures 
Because this study involves no more than minimal risks to participants, the above 
described procedures and data collection techniques are justified as they are all necessary 
to satisfy the research aims of this study. Additionally, having the option to complete visits 
virtually or in-person will ensure maximum safety for patients and providers in times of 
COVID-19 surges. 
    
E.   Potential Scientific Problems   
As attrition may be a concern in hard to reach populations,69 we accounted for 22% attrition 
in our sample size calculations. To encourage retention, as was successful in the K99 trial 
in the DR, participants will be reminded of visits via phone calls prior to study visits and 
incentives appropriate for each site will be provided. Recruitment will be monitored to 
ensure a sufficient pace of enrollment and timelines will be extended if needed. 

Protocol Template 
Page 10 
CF-146, Effective 7/10/11 
 
Intervention fidelity and contamination are concerns as participants may be scheduled to 
see the incorrect provider, providers may use infographics with control group participants, 
or simply may not use infographics. Intervention fidelity will be assessed by providers, 
patients, and a study team member, and contamination addressed with intent-to-treat 
analysis.  
 
F. Data Analysis Plan  
 
Sample size calculations 
 
Individual interviews 
 
Up to 15 participants will participate in individual interviews. This sample size was 
established as 15 participants are likely to be able to identify 90% of usability concerns in 
informatics-based studies70 and as few as 10 are likely to achieve saturation in qualitative 
studies on specific themes (e.g. the experience of living with HIV) with relatively 
homogenous samples (e.g. Latino persons of predominantly Mexican origin/descent living 
in Denver, CO).71 
 
RCT – Interventionists 
 
Up to 5 interventionists at each site will be purposefully selected to participate. This sample 
size is based on preliminary feasibility studies. 
 
RCT – patient participants (n=164, 82 at each site, 41 per group) 
 
To establish sample size for the RCT, we used a longitudinal power analysis approach for 
testing the “time averaged difference” in two group means for a continuous outcome over 
time.72 This effect size is analogous to Cohen’s D: (𝜇𝜇1-𝜇𝜇2)/σ, where 𝜇𝜇1, 𝜇𝜇2 are the outcome 
means (averaged over time) in the treatment and control groups, and σ is the standard 
deviation of the outcome (assumed constant over time and equal for both groups).73  
Assuming a slightly less than “medium” effect size of D=0.45,73 80% power, and two-sided 
significance level α=0.05, the required total sample size ranges from 68-126 depending on 
the assumed constant pairwise correlation between the 4 repeated measures (which we 
varied from 0.25 to 0.75) (Table 3). We conservatively assume a correlation of 0.75 which 
requires a total sample size of 126. To account for 22% attrition, higher than our attrition 
rate that we experienced in the pilot tests in the Dominican Republic (16%) to account for 
complications with this hard to reach population of Denver, we increased the sample size 
to 164 (161.5 rounded up to ensure equal numbers in each group). 

Protocol Template 
Page 11 
CF-146, Effective 7/10/11 
 
 
Interviews with key stakeholders 
 
Up to 15 interviews with key stakeholders will be conducted at each site (CO and DR). This 
is based on estimates that 12 interviews per site can lead to saturation of identified 
themes.74,75 
 
Summary of total sample size per site: 
 
DR total # of participants to be consented to achieve study aims: 100 
(maximum recruitment at this site will be 115 to account for failures and withdrawals). 
 
Denver Health total # of participants to be consented to achieve study aims: 121 
(maximum recruitment at this site will be 140 to account for failures and withdrawals). 
 
Analytic plan for each study aim 
 
Aim 1: See individual interviews in section C above. 
 
Aim 2: Conduct a multi-site randomized controlled trial to assess the efficacy of the 
intervention (infographics administered through a mobile app) as compared to standard 
care in improving treatment outcomes (CD4 count, viral load, self-efficacy to manage HIV, 
among others) of PWH and determine how resultant effects vary over time (baseline to 9-
months) and between sites (US and CO). 
 
Primary 
Analysis: A linear mixed effect model (LMM) will be used to compare the effects of the two 
groups (treatment vs. standard of care) on the primary longitudinal outcome of CD4 count 
between baseline and the three follow up visit time points (3-, 6-, and 9-months). Subject-
specific random intercepts and slopes will be used to model heterogeneity in the subjects’ 
trajectories. Three primary tests will be considered: 1) testing the time-averaged difference 
in the outcome between the two treatment groups (main effect for treatment), and 2) test 
whether change-over-time differs between the two treatment groups (treatment*time 
interaction), and 3) test whether the treatment effect differs between the two sites 
(treatment*site interaction and treatment*site*time interaction).  
 
Secondary Analyses: The same analysis plan will be repeated with the remaining 
longitudinal outcomes Table 2 of the Research Strategy section of this application. Linear 
mixed effect models (LMM) will be used for continuous outcomes (CD4 count, viral load, 
knowledge questions, engagement with health care provider scale, satisfaction with care, 
Table 3. Necessary sample size for testing the time-averaged difference 
between two groups of equal size  
 
𝜌𝜌 (constant correlation of 4 repeated 
measures over time) 
Effect Size (Average diff. in 2 
group means over time, analogous 
to Cohen’s D: (𝜇𝜇1-𝜇𝜇2)/σ)   
0.25 
0.50 
0.75 
0.20 (Small) 
344 
491 
638 
0.35 
113 
161 
209 
0.40 
86 
123 
160 
0.45 
68 
97 
126 
0.50 (Medium) 
55 
79 
103 
0.80 (Large) 
22 
31 
40 
Note: Calculations with 80% power and α=0.05 with 4 repeated measures 
assuming exchangeable/ constant pairwise correlation “ρ” varied from 0.25 to 0.75. 

Protocol Template 
Page 12 
CF-146, Effective 7/10/11 
 
self-efficacy, health related quality of life PROMIS-29), logistic generalized linear mixed 
effect model (GLMM) will be used for the dichotomous adherence score (SMAQ-6), and 
ordinal GLMM will be used for the health status assessment score while LMM will be used 
for the additional health status continuous score.  Lastly, as a simpler alternative to 
longitudinal mixed effect models (that use all time points), standard general linear models 
will be used to model outcomes at 9 months adjusting for baseline, thus testing whether an 
outcome differs between the two interventions at 9 months.  
The System Usability Scale (SUS) that providers complete during the second training 
session will be scored according to published guidelines.76 
 
D.9.4. Aim 3: Assess implementation factors of integrating the SMART on FHIR app into 
EHRs through in-depth interviews and leverage findings to ensure the app effectively 
interfaces with EHRs in the CO and the DR. 
 
Transcripts from key stakeholders at each site will be analyzed using directed content 
analysis to identify implementation factors regarding the SMART app’s integration with 
EHRs.77-80 Key considerations to enhance perceived usefulness and ease of use will be 
identified and incorporated into the final app. 
 
D.9.5. Exploratory Aim 4: Conduct in-depth interviews with PWH in CO to explore their 
perceptions of HIV infection and preferences for health communication. Qualitative findings 
will be integrated with those from the NY and DR cohorts to detail variations and 
similarities, and identify areas for intervention refinement. To deeply and rigorously explore 
participants’ perceptions of HIV infection and how they influence clinical communication, 
explanatory models of HIV infection among PWH at each site will be generated.68,81-83 
Directed content analysis guided by McGuire’s Classic Input-Output Framework for 
Constructing Persuasive Messages will be used to analyze transcripts from in-depth 
interviews.84 Gioia’s methodology will be followed to enhance qualitative rigor.85 Results 
from each cohort (DR, CO, and NY) will be synthesized and integrated to establish 
variations and similarities in the HIV experience among distinct Latino cultures as well as 
establish how their unique perspectives shape intervention design and delivery. 
 
Usability testing of Creole version of the app 
 
Changes to the app will be made directly on the paper copies provided during the group. 
The scores of the SUS will be calculated according to the instrument guidelines. 
 
Validation of infographics with non-Latino PWH 
 
Quantitative data from the questionnaire that will be administered during interviews will be 
scored and descriptive statistics calculated. Audio recordings will be transcribed and 
analyzed for patterns. Representative quotes will be extracted. 
 
E. Summarize Knowledge to be Gained 
  
HIV continues to be a global concern, but the focus on HIV treatment and research have 
shifted from considering HIV to be a devastating acute condition to a chronic condition that 
is manageable with optimal self-management and treatment adherence. This shift in 
attention and disease progression requires persons living with HIV to effectively manage 
their health for long periods of time. However, individuals with limited health literacy may 
have difficulty accessing and using the health information that enables this extended 
health management, especially when they are presented with information that is not 
culturally relevant or is language inappropriate. This study will contribute evidence to 
support the provision of health information to diverse Latino populations in numerous 
ways. 

Protocol Template 
Page 13 
CF-146, Effective 7/10/11 
 
 
This study aims to develop and evaluate an evidence-based and culturally appropriate 
technology (infographics embedded in a mobile application) to facilitate HIV-related 
communication between Latino persons living with HIV and their clinicians. By 
incorporating these evidence-based infographics into an app that interfaces with EHRs 
both locally and globally, this work will contribute to the informatics literature regarding the 
effective development and implementation of SMART on FHIR apps at a global scale. 
Additionally, it will provide valuable information regarding the effectiveness of infographics 
at improving clinical communication between patients and providers, which has 
widespread relevance to the growing information visualization literature. Additionally, there 
is little in the literature regarding effective ways to tailor interventions to heterogeneous 
groups, as an intervention designed for one Latino group may not be relevant to others. 
This work will provide evidence-based guidance regarding methods to effectively tailor 
health related interventions to diverse Latino groups. Furthermore, this intervention is 
scalable and will be accessible to clinicians working with Latino patients with low health 
literacy in the United States and around the world. 
 
Including an additional portion in this study to ascertain if the infographics are meaningful 
and acceptable to non-Latino persons with HIV will provide preliminary data about the 
generalizability of these infographics and set the stage for further scalability of the 
intervention so that more individuals in need are likely to benefit from it. 
 
F. 
References 
 
1. 
Passel J, Cohn D, Lopez M. Hispanics account for more than half of nation’s growth in past 
decade. 2011. chrome-
extension://efaidnbmnnnibpcajpcglclefindmkaj/https://assets.pewresearch.org/wp-
content/uploads/sites/7/reports/140.pdf 
2. 
Henao-Martínez AF, Castillo-Mancilla JR. The Hispanic HIV epidemic. Curr Infect Dis Rep. 
2013;15(1):46-51.  
3. 
Sullivan PS, Purcell DW, Grey JA, et al. Patterns of racial/ethnic disparities and prevalence 
in HIV and syphilis diagnoses among men who have sex with men, 2016: a novel data 
visualization. Am J Public Health. 2018;108(S4):S266-S273.  
4. 
Gonzalez JS, Hendriksen ES, Collins EM, Durán RE, Safren SA. Latinos and HIV/AIDS: 
examining factors related to disparity and identifying opportunities for psychosocial intervention 
research. AIDS Behav. 2009;13(3):582-602.  
5. 
Xia Q, Braunstein SL, Wiewel EW, Hadler JL, Torian LV. Persistent racial disparities in HIV 
infection in the USA: HIV prevalence matters. J Racial Ethn Health Disparities. 2016;4(1):87-93.  
6. 
Vega WA, Rodriguez MA, Gruskin E. Health disparities in the Latino population. Epidemiol 
Rev. 2009;31(1):99-112.  
7. 
Wawrzyniak AJ, Ownby RL, McCoy K, Waldrop-Valverde D. Health literacy: impact on the 
health of HIV-infected individuals. Curr HIV/AIDS Rep. 2013;10(4):295-304.  
8. 
Osborn CY, Paasche-Orlow MK, Davis TC, Wolf MS. Health literacy: an overlooked factor 
in understanding HIV health disparities. Am J Prev Med. 2007;33(5):374-378.  
9. 
Koskan A, Friedman D, Messias D. Health literacy among Hispanics: a systematic 
research review (1992-2008). Hispanic Health Care International. 2010;8:65-76. doi:10.1891/1540-
4153.8.2.65 
10. 
Berkman N, Sheridan S, Donahue K, Halpern D, Crotty K. Low health literacy and health 
outcomes: an updated systematic review. Ann Intern Med. 2011;155(2):97-107.  
11. 
Becerra BJ, Arias D, Becerra MB. Low health literacy among immigrant Hispanics. J Racial 
Ethn Health Disparities. 2017;4(3):480-483.  
12. 
Moore J, Boyer E, Safren S, et al. Designing interventions to overcome poor numeracy and 
improve medication adherence in chronic illness, including HIV/AIDS. J Med Toxicol. 
2011;7(2):133-138.  

Protocol Template 
Page 14 
CF-146, Effective 7/10/11 
 
13. 
Iribarren S, Siegel K, Hirshfield S, et al. Self-Management Strategies for Coping with 
Adverse Symptoms in Persons Living with HIV with HIV Associated Non-AIDS Conditions. AIDS 
Behav. 2018;22(1):297-307.  
14. 
Mackey LM, Doody C, Werner EL, Fullen B. Self-Management Skills in Chronic Disease 
Management What Role Does Health Literacy Have? Med Decis Making. 2016;36(6):741-759.  
15. 
Millard T, Elliott J, Girdler S. Self-management education programs for people living with 
HIV/AIDS: a systematic review. AIDS Patient Care and STDs. 2013;27(2):103-113.  
16. 
Schiavo R. Health communication in health disparities settings. Journal of Communication 
in Healthcare. 2014;7(2):71-73. doi:10.1179/1753806814Z.00000000073 
17. 
Kasper J, Roemer A, Pöttgen J, et al. A new graphical format to communicate treatment 
effects to patients—A web‐based randomized controlled trial. Health Expect. 2016;20(4):797-804.  
18. 
Storey D, Seifert-Ahanda K, Andaluz A, Tsoi B, Matsuki J, Cutler B. What is health 
communication and how does it affect the HIV/AIDS continuum of care? a brief primer and case 
study from New York City. JAIDS, Journal of Acquired Immune Deficiency Syndromes. 
2014;66:S241-S249.  
19. 
Sorlie V, Lopez R. When language, health literacy, and miscommunication collide: tremors 
versus seizures. Case Reports. Fam Med. 2011;43:48-50.  
20. 
Zolnierek KB, DiMatteo RM. Physician communication and patient adherence to treatment: 
a meta-analysis. Med Care. 08 2009;47(8):826-34.  
21. 
Beach MC, Keruly J, Moore RD. Is the Quality of the Patient‐Provider Relationship 
Associated with Better Adherence and Health Outcomes for Patients with HIV? J Gen Intern Med. 
2006;21(6):661-665.  
22. 
Mogobe KD, Shaibu S, Matshediso E, et al. Language and Culture in Health Literacy for 
People Living with HIV: Perspectives of Health Care Providers and Professional Care Team 
Members. AIDS Research and Treatment. 2016;2016(Article 5015707):1-11.  
23. 
Garcia-Retamero R, Okan Y, Cokely ET. Using visual aids to improve communication of 
risks about health: A review. Sci World J. 2012;81(4):257-269.  
24. 
McCaffery KJ, Dixon A, Hayen A, Jansen J, Smith S, Simpson JM. The influence of 
graphic display format on the interpretations of quantitative risk information among adults with 
lower education and literacy: a randomized experimental study. Med Decis Making. 
2012;32(4):532-544.  
25. 
Arcia A, Suero-Tejeda N, Bales ME, et al. Sometimes more is more: iterative participatory 
design of infographics for engagement of community members with varying levels of health literacy. 
J Am Med Inform Assoc. 2016;23(1):174-183.  
26. 
Houts P, Doak C, Doak L, Loscalzo M. The role of pictures in improving health 
communication: a review of research on attention, comprehension, recall, and adherence. Patient 
Educ Couns. 2006;61(2):173-190.  
27. 
Zikmund-Fisher BJ, Witteman HO, Dickson M, et al. Blocks, ovals, or people? Icon type 
affects risk perceptions and recall of pictographs. Med Decis Making. 2014;34(4):443-453.  
28. 
Garcia‐Retamero R, Dhami MK. Pictures speak louder than numbers: On communicating 
medical risks to immigrants with limited non‐native language proficiency. Health Expect. 
2011;14(s1):46-57.  
29. 
Dillard PK, Zuniga JA, Holstad MM. An integrative review of the efficacy of motivational 
interviewing in HIV management. Patient Educ Couns. 2017;100(4):636-646.  
30. 
Grodensky CA, Golin CE, Pack AP, et al. Adaptation and delivery of a motivational 
interviewing-based counseling program for persons acutely infected with HIV in Malawi: 
Implementation and lessons learned. Patient Educ Couns. 2018;101(6):1103-1109.  
31. 
Andersen LS, Magidson JF, O’Cleirigh C, et al. A pilot study of a nurse-delivered cognitive 
behavioral therapy intervention (Ziphamandla) for adherence and depression in HIV in South 
Africa. Journal of health psychology. 2018;23(6):776-787.  
32. 
Beach MC, Laws MB, Rose G, et al. Effects of minimal versus intensive intervention to 
enhance motivational interviewing in HIV care. AIDS Behav. 2018;22(1):276-286.  
33. 
Brandt CP, Paulus DJ, Garza M, Lemaire C, Norton PJ, Zvolensky MJ. A novel integrated 
cognitive-behavioral therapy for anxiety and medication adherence among persons living with 

Protocol Template 
Page 15 
CF-146, Effective 7/10/11 
 
HIV/AIDS. Cognitive and Behavioral Practice. 2018;25(1):105-118. 
doi:10.1016/j.cbpra.2017.01.007 
34. 
Mandel JC, Kreda DA, Mandl KD, Kohane IS, Ramoni RB. SMART on FHIR: a standards-
based, interoperable apps platform for electronic health records. J Am Med Inform Assoc. 
2016;23(5):899-908.  
35. 
Chang B, Bakken S, Brown S, et al. Bridging the digital divide: reaching vulnerable 
populations. J Am Med Inform Assoc. 2004;11(6):448-457.  
36. 
Levy H, Janke AT, Langa KM. Health literacy and the digital divide among older 
Americans. J Gen Intern Med. 2015;30(3):284-289.  
37. 
López L, Green AR, Tan-McGrory A, King RS, Betancourt JR. Bridging the digital divide in 
health care: the role of health information technology in addressing racial and ethnic disparities. 
The Joint Commission Journal on Quality and Patient Safety. 2011;37(10):437-445.  
38. 
Chen W, Akay M. Developing EMRs in developing countries. IEEE Transactions on 
Information Technology in Biomedicine. 2010;15(1):62-65.  
39. 
Syzdykova A, Malta A, Zolfo M, Diro E, Oliveira JL. Open-source electronic health record 
systems for low-resource settings: systematic review. JMIR medical informatics. 2017;5(4):e44.  
40. 
Akhlaq A, McKinstry B, Muhammad KB, Sheikh A. Barriers and facilitators to health 
information exchange in low-and middle-income country settings: a systematic review. Health 
policy and planning. 2016;31(9):1310-1325.  
41. 
Centers for Disease Control and Prevention. Basic HIV statistics. Division of HIV 
Prevention, National Center for HIV, Viral Hepatitis, STD, and TB Prevention, Centers for Disease 
Control and Prevention. Accessed September, 2023. https://www.cdc.gov/hiv/basics/statistics.html 
42. 
Sullivan PS, Johnson AS, Pembleton ES, et al. Epidemiology of HIV in the USA: epidemic 
burden, inequities, contexts, and responses. Lancet. 2021;397(10279):1095-1106.  
43. 
Stonbraker S, Smaldone A, Luft H, et al. Associations between health literacy, HIV-related 
knowledge, and information behavior among persons living with HIV in the Dominican Republic. 
Public Health Nursing. 2017;35(3):166-175. doi:10.1111/phn.12382 
44. 
Stonbraker S, Befus M, Lerebours Nadal L, Halpern M, Larson E. Factors associated with 
health information seeking, processing, and use among HIV positive adults in the Dominican 
Republic. AIDS Behav. 2016;21(6):1588–1600. doi:10.1007/s10461-016-1569-5 
45. 
Stonbraker S, Larson E. Health-information needs of HIV-positive adults in Latin America 
and the Caribbean: an integrative review of the literature. AIDS Care. 2016;28(10):1223-1229. 
doi:10.1080/09540121.2016.1173645 
46. 
Stonbraker S, Halpern M, Bakken S, Schnall R. Special Section on Visual Analytics in 
Healthcare: Developing infographics to facilitate HIV-related patient-provider communication in a 
limited-resource setting. Appl Clin Inform. 2019;10(4):597-609. doi:10.1055/s-0039-1694001 
47. 
Stonbraker S, Richards SD, Halpern M, Bakken S, Schnall R. Priority topics for health 
education to support HIV self-management in limited-resource settings. J Nurs Scholarsh. 
2019;51(2):168-177. doi:10.1111/jnu.12448 
48. 
Chen W-T, Wantland D, Reid P, et al. Engagement with health care providers affects self-
efficacy, self-esteem, medication adherence and quality of life in people living with HIV. Journal of 
AIDS & Clinical Research. 2013;4(11):256.  
49. 
Dang BN, Westbrook RA, Hartman CM, Giordano TP. Retaining HIV patients in care: the 
role of initial patient care experiences. AIDS Behav. 2016;20(10):2477-2487.  
50. 
O’Brien M, Shea J. Disparities in patient satisfaction among Hispanics: the role of language 
preference. Journal of Immigrant and Minority Health. 2011;13(2):408-412.  
51. 
Ritter PL, Lorig K. The English and Spanish self-efficacy to manage chronic disease scale 
measures were validated using multiple studies. J Clin Epidemiol. 2014;67(11):1265-1273.  
52. 
Knobel H, Alonso J, Casado JL, et al. Validation of a simplified medication adherence 
questionnaire in a large cohort of HIV-infected patients: the GEEMA Study. AIDS. 2002;16(4):605-
613.  
53. 
Sperber CM, Samarasinghe SR, Lomax GP. An upper and lower bound of the medication 
possession ratio. Patient preference and adherence. 2017;11:1469.  

Protocol Template 
Page 16 
CF-146, Effective 7/10/11 
 
54. 
Monroe A, Pena J, Moore RD, et al. Randomized controlled trial of a pictorial aid 
intervention for medication adherence among HIV-positive patients with comorbid diabetes or 
hypertension. AIDS Care. 2018;30(2):199-206.  
55. 
Paz SH, Spritzer KL, Morales LS, Hays RD. Evaluation of the patient-reported outcomes 
information system (PROMIS®) Spanish-language physical functioning items. Qual Life Res. 
2013;22(7):1819-1830.  
56. 
ACTG Outcomes Committee. Overall Health Status Assessment. Outcomes Committee of 
the AIDS Clinical Trials Group. Accessed July, 2018. 
https://www.frontierscience.org/apps/cfmx/apps/common/QOLAdherenceForms/index.cfm?project=
ACTG 
57. 
Street RL, Makoul G, Arora NK, Epstein RM. How does communication heal? Pathways 
linking clinician–patient communication to health outcomes. Patient Educ Couns. 2009;74(3):295-
301.  
58. 
Stirbu V. A RESTful architecture for adaptive and multi-device application sharing. ACM; 
2010:62-65. 
59. 
Dickens A, Utley-Smith Q, Stott G, Relf M. Evaluation of communication techniques used 
by HIV specialty providers caring for patients with low health literacy in an outpatient HIV clinical 
setting. J Nurs Educ Pract. 2013;3(7):114. doi:10.5430/jnep.v3n7p114 
60. 
Gearing RE, El-Bassel N, Ghesquiere A, Baldwin S, Gillies J, Ngeow E. Major ingredients 
of fidelity: A review and scientific guide to improving quality of intervention research 
implementation. Clin Psychol Rev. 2011;31(1):79-88.  
61. 
Broglio K. Randomization in clinical trials: permuted blocks and stratification. Jama. 
2018;319(21):2223-2224.  
62. 
Toren KG, Buskin SE, Dombrowski JC, Cassels SL, Golden MR. Time from HIV diagnosis 
to viral load suppression: 2007–2013. Sex Trasm Dis. 2016;43(1):34.  
63. 
Kallio H, Pietilä AM, Johnson M, Kangasniemi M. Systematic methodological review: 
developing a framework for a qualitative semi‐structured interview guide. Journal of Advanced 
Nursing. 2016;72(12):2954-2965. doi:10.1111/jan.13031 
64. 
Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of 
information technology. MIS Q. 1989:319-340.  
65. 
Lee S, Stucky B, Lee J, Rozier R, Bender D. Short assessment of health literacy-spanish 
and english: a comparable test of health literacy for Spanish and English speakers. Health Serv 
Res. 2010;45(4):1105-1120. doi:http://dx.doi.org/10.1111/j.1475-6773.2010.01119.x 
66. 
Weiss BD, Mays MZ, Martz W, et al. Quick assessment of literacy in primary care: The 
Newest Vital Sign. Ann Fam Med. 2005;3(6):514-522. doi:10.1370/afm.405 
67. 
Ellison J, Jandorf L, Duhamel K. Assessment of the Short Acculturation Scale for Hispanics 
(SASH) among low-income, immigrant Hispanics. Journal of Cancer Education. 2011;26(3):478-
483.  
68. 
McSweeney JC, Allan JD, Mayo K. Exploring the use of explanatory models in nursing 
research and practice. J Nurs Scholarsh. 1997;29(3):243-248. doi:10.1111/j.1547-
5069.1997.tb00992.x 
69. 
Carriquiry G, Fink V, Koethe JR, et al. Mortality and loss to follow-up among HIV-infected 
persons on long-term antiretroviral therapy in Latin America and the Caribbean. J Int AIDS Soc. 
2015;18(1) 
70. 
Faulkner L. Beyond the five-user assumption: Benefits of increased sample sizes in 
usability testing. Behavior Research Methods. 2003;35(3):379-383.  
71. 
Sandelowski M. Sample size in qualitative research. Res Nurs Health. 1995;18(2):179-183.  
72. 
Diggle P, Diggle PJ, Heagerty P, Liang K-Y, Heagerty PJ, Zeger S. Analysis of longitudinal 
data. Oxford University Press; 2002. 
73. 
Cohen J. Statistical power analysis for the behavioral sciences. Routledge; 2013. 
74. 
Hagaman AK, Wutich A. How many interviews are enough to identify metathemes in 
multisited and cross-cultural research? Another perspective on Guest, Bunce, and Johnson’s 
(2006) landmark study. Field Methods. 2017;29(1):23-41.  

Protocol Template 
Page 17 
CF-146, Effective 7/10/11 
 
75. 
Guest G, Bunce A, Johnson L. How many interviews are enough? An experiment with data 
saturation and variability. Field Methods. 2006;18(1):59-82. 
doi:https://doi.org/10.1177/1525822X05279903 
76. 
Sevilla-Gonzalez MDR, Loaeza LM, Lazaro-Carrera LS, et al. Spanish Version of the 
System Usability Scale for the Assessment of Electronic Tools: Development and Validation. JMIR 
Human Factors. 2020;7(4):e21161.  
77. 
Willis DG, Sullivan-Bolyai S, Knafl K, Cohen MZ. Distinguishing features and similarities 
between descriptive phenomenological and qualitative description research. Western journal of 
nursing research. 2016;38(9):1185-1204.  
78. 
Vaismoradi M, Turunen H, Bondas T. Content analysis and thematic analysis: Implications 
for conducting a qualitative descriptive study. Nursing & health sciences. 2013;15(3):398-405.  
79. 
White MD, Marsh EE. Content analysis: A flexible methodology. Libr Trends. 
2006;55(1):22-45.  
80. 
Elo S, Kyngäs H. The qualitative content analysis process. Journal of advanced nursing. 
2008;62(1):107-115.  
81. 
Laws MB. Explanatory models and illness experience of people living with HIV. AIDS 
Behav. 2016;20(9):2119-2129. doi:10.1007/s10461-016-1358-1 
82. 
Oetzel J, Wilcox B, Avila M, Hill R, Archiopoli A, Ginossar T. Patient–provider interaction, 
patient satisfaction, and health outcomes: testing explanatory models for people living with 
HIV/AIDS. AIDS Care. 2015;27(8):972-978. doi:10.1080/09540121.2015.1015478 
83. 
Kleinman A, Eisenberg L, Good B. Culture, illness, and care: clinical lessons from 
anthropologic and cross-cultural research. Ann Intern Med. 1978;88(2):251-258. doi:10.7326/0003-
4819-88-2-251 
84. 
McGuire WJ. McGuire’s classic input-output framework for constructing persuasive 
messages. Public communication campaigns. 2013;4:133-145.  
85. 
Gioia DA, Corley KG, Hamilton AL. Seeking qualitative rigor in inductive research: Notes 
on the Gioia methodology. Organizational research methods. 2013;16(1):15-31.