A Mobile Health Application for Engagement in Care Among Youth Living With HIV
Phase: NA
Conditions: HIV/AIDS, Mobile Health, Young Adults
Sponsor: University of California, San Francisco

WYZ Study Protocol 
 
Study Title: A Mobile Health Application for Engagement in Care Among Youth Living With HIV  
National Clinical Trial Number: NCT03587857 
IRB Study Number: 16-20393 
Date: 3/26/2019 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

WYZ Study Protocol 
 
Contents 
Protocol Title ................................................................................................................................................. 2 
Background ................................................................................................................................................... 2 
Objectives ..................................................................................................................................................... 3 
Sample Size and Eligibility ............................................................................................................................. 4 
Recruitment .................................................................................................................................................. 4 
Procedures .................................................................................................................................................... 4 
Outcome Measures ....................................................................................................................................... 5 
Data Analysis Plan ......................................................................................................................................... 6 
References .................................................................................................................................................... 7 
 
 
Protocol Title 
WYZ: A Mobile Health Application for Engagement in Care Among Youth Living With HIV 
Background 
Globally, youth and young adults are recognized as being particularly vulnerable to HIV and 
bear a disproportionate share of the HIV burden, with HIV being the second leading cause of 
death.1 In the USA, youth and young adults living with HIV (YLWH) have the lowest likelihood of 
linking to care, being retained in care, adhering to antiretroviral therapy (ART) and achieving 
viral suppression, compared with other age groups.2–4 One review estimated that among all 
YLWH in the USA, fewer than 6% are virally suppressed.5 The personal and public health 
consequences of suboptimal adherence among YLWH include poor health outcomes, increased 
drug resistance, reduced efficacy of existing treatment options, widening health disparities and 
potential transmission of drug-resistant virus. Efforts to adapt healthcare services for YLWH are 
critical, lest we risk a future generation of adults who are more susceptible to developing AIDS. 
While barriers to engagement in HIV care and ART adherence are generally similar across 
settings and populations, YLWH are more likely to report barriers such as forgetting, being away 
from home, change in daily routine, depression and alcohol and substance use compared with 
other age groups.6 Additionally, studies have shown that YLWH face many challenges across the 
HIV care continuum, and clinical outcomes are worse in this group compared with adults.7 The 
combination of these health disparities and developmental challenges in this transitional period 
may further hinder YLWH’s ability to maintain optimal ART adherence and sustain virological 
suppression.1 7 The link between younger age and lower ART adherence and higher risk of 

virological failure has been reported in numerous studies.8–11 However, there are few studies to 
inform the design of youth-friendly healthcare services and interventions.12 
 
The widespread use of personal mobile phones presents an opportunity to develop novel 
interventions capable of bridging structural and systemic gaps in service and improving the 
equity of healthcare delivery. Mobile health (mHealth), the use of mobile technology for health, 
has the potential to reduce barriers to healthcare access, increase use of healthcare services 
and improve health outcomes among underserved populations.13 Reducing barriers like travel 
to healthcare settings, mHealth interventions hold the potential to increase accessibility for 
underserved populations and allow for more efficient use of provider resources.14 Additionally, 
technology-based interventions have the capacity to consistently deliver interventions with 
high fidelity and to complement and integrate with other healthcare services and technologies. 
As the largest consumers of mobile technology and internet use, youth and young adults hold 
the greatest potential to benefit from such technology-based interventions.15 In the USA, 
among 18–29 year-olds, greater than 77% download mobile applications and over 90% are 
users of social media.16 17 
 
There is a rapidly growing body of evidence demonstrating efficacy of mHealth interventions 
among various populations and for a variety of conditions, yet the field is nascent and 
confronting challenges related to limited generalisability,18 lack of rigorous long-term 
evaluations and demonstration projects,13 and complexities in evaluating efficacious theory-
driven components.19 The purpose of our research is to build on our theory-guided model and 
formative research to develop an mHealth application called WYZ for improved engagement in 
HIV care and ART adherence, and pilot test it among 18–29 year-olds living with HIV.  
Objectives 
WYZ is an mHealth intervention designed and developed in collaboration with YLWH to 
improve ART adherence and engagement in HIV care among this age group. WYZ contains three 
main features, My Health, My Team, and My Community. My Health allows users to keep track 
of their ART medication information, visualize their adherence and laboratory data, and 
understand their health; My Team provides community resources and facilitates 
communication with health care team members; and My Community allows for social support 
from peers through anonymous and moderated discussion forums and allows users to stay up-
to-date on health-related news. These features were developed with guidance from youth and 
young adults living with HIV and further refined through focus groups with youth and young 
adults living with HIV and iterative field testing with our Youth Advisory Panel (YAP), and were 
chosen to address specific barriers to ART adherence and engagement in HIV care (e.g., 
social isolation and lack of community support). 
 
We will conduct a 6-month single-arm pilot study to examine WYZ feasibility and acceptability 
among YLWH (18–29 years old) who live or receive care in the San Francisco Bay Area. The 
objective of this forthcoming phase is to refine the design and development and pilot test WYZ. 

All study activities, including recruitment, screening, enrolment, study assessments, provision 
of incentives and exit interviews, will be conducted remotely. 
Sample Size and Eligibility 
We will enroll up to 10 health care providers and up to 85 YLWH study participants. Of these 85, 
we will select a subset of 25 participants to take part in a 30 minute qualitative interview about 
their experience with the app. 
 
Inclusion Criteria: 
1. All YLWH will be confirmed HIV-positive, age 18-29, be able to speak and read in English, 
have a smartphone (iOS V10.0 and above or Android V5.0 and above), and be able to 
understand the consent procedure and voluntarily sign consent forms. 
2. Health care providers are primary care providers, nurses, social workers and pharmacists 
who provide care to participating individuals. 
 
Exclusion Criteria: 
Those with evidence of severe cognitive impairment or active psychosis that impedes their 
ability to provide informed consent will be excluded. 
Recruitment 
A combination of recruitment strategies will be used to help achieve the study’s enrolment 
goals. These include strategies such as posting flyers at clinics and community-based 
organizations, participant submission of contact information through the study website 
(wyz.ucsf.edu) and contacting healthcare team members to refer application users. We will also 
work with a city-wide consortium of youth service providers and the CAPS Youth Advisory Panel 
to disseminate study information to the target population. Enrolled participants will be 
encouraged to refer eligible peers and receive $10 incentive for the referee and the referred 
individual who participates. Lastly, participants from the principal investigator’s prior studies 
who had expressed interest and consented to be contacted about future research will be given 
the opportunity to participate, if still eligible. 
Procedures 
Eligibility screening 
Interested individuals will be provided a brief overview of the study goals and given the chance 
to ask questions. Individuals wishing to enroll in the study will be screened for eligibility. To 
confirm an individual’s age, we will require a photo identification showing their date of birth. To 
confirm an individual’s HIV serostatus, we will require a clinician’s letter of HIV diagnosis, copy 
of laboratory test results (for HIV antibody or HIV viral load), or the individual’s ART prescription 
showing their name. These documents can be sent as a text message to the encrypted and 
secure study phone for verification by study staff. 
 

Consent and enrollment 
To enroll in the study, individuals will review and sign an electronic consent form sent to them 
via Qualtrics (Provo, UT, USA; version March 2017). All participants will be emailed a copy of the 
Experimental Subject’s Bill of Rights as mandated by the State of California and a signed copy of 
their consent. If a participant does not have an email address, the study staff will assist them in 
signing up for one, as it will be necessary for gaining access to WYZ. 
 
Participant retention 
A number of steps will be taken to engage and retain participants throughout the study period. 
First, participants will be asked for multiple forms of contact information (including emergency 
contacts, clinical contacts and social media contacts) at the initial visit. They will receive 
monthly follow-up text messages to confirm that their contact information has not changed. 
Finally, participants will receive reminder text messages 24 hours before scheduled study 
activities. 
 
Participant incentives 
Participants can receive up to a total of $215 incentive for completing all study activities. They 
will receive $40 for the baseline session, which includes consent, application installation and 
baseline survey. They will then receive $10, $10, $15, $15, $20, $20 and $25 for participating in 
check-ins at weeks 1, 2, 4, 8, 12, 16 and 20, respectively. For the final exit survey at week 24, 
participants will receive $40, and those who are invited to participate in the exit qualitative 
interview at this time will receive an additional $20. Part of this incentive is aimed at offsetting 
any data and text messaging fees associated with the application use. We estimate that 
participants will spend a total of 4–4.5 hours for study activities. Participants will be given a 
ClinCard, a reloadable debit card, which will be used for the remote provision of study 
incentives. Finally, we will offer $50 to each healthcare provider for participating in a one-time 
qualitative interview. 
Outcome Measures 
The main outcomes of the pilot trial include feasibility and acceptability as indicated by meeting or 
exceeding the benchmarks presented in table 1. Feasibility will be assessed via user metrics by 
examining the participant’s interactions with WYZ via a mobile analytics service called Flurry and 
backend (Salesforce) reporting tools (table 1). Acceptability will be examined during monthly check-ins 
to examine satisfaction with application use, the system usability scale (SUS),20 a study-developed 
acceptability survey and an exit qualitative interview. HIV viral load and CD4+ cell count will be collected 
to determine feasibility of collecting these measures at both baseline and study completion. 
Table 1 
Feasibility metrics and threshold for determining feasibility 
Measure 
Threshold (per user) 
General 
 
   Recruitment 
Recruit at least 55 participants (i.e., 70% of target n) 
   Number of log-ins 
Mean of 1 log-in per week 

   Mean number of minutes in 
application 
Mean of 15 min in application per week 
My Health 
 
   Use of refill feature 
Use of feature once monthly (if receiving 30-day ART 
supply) 
   Use of ART adherence tracking 
Track meds at least three times per week 
   Access of laboratory data 
Review of laboratory data at least one time per month 
My Team 
 
   Communications with clinical team 
member 
Mean of one communication per month with clinical team 
member 
My Community 
 
   Postchat or response to chats 
Mean of one post or response to a post per week 
   Calendar and news 
Minimum access once per week 
Administrative 
 
   Time for participant onboarding 
Mean of 30 min to onboard (including downloading app 
and reviewing app features) 
   Time to maintain and support 
application 
<1 hour per participant per week (excludes first visit) 
   Mean number of app crashes reported 
Fewer than one self-reported app crash per week 
 
Given that application use engagement is a complex and multifactorial concept, we aim to move beyond 
simple metrics. Therefore, in addition to feasibility measures detailed in table 1, we will examine an 
engagement index (EI) that has been detailed in other mHealth interventions.21 The EI includes the 
following subindices: (1) click depth (number of pages a user views per session), (2) loyalty (measures 
how frequently users access the application during the study period), (3) recency (the time difference 
between each session the user accessed the application), (4) interaction (number of push notifications 
opened from those sent through the application), and (5) feedback (subjective measure of participants’ 
satisfaction with the application). 
Data Analysis Plan 
We have established minimum thresholds for determining the feasibility and acceptability of the 
intervention. Table 1 details these thresholds for the feasibility metrics. For acceptability, SUS scores 
range from 0 to 100 and scores >68 are considered above average. We will use a threshold of 70% or 
greater satisfaction to determine acceptability. Means and proportions will be computed at the study 
endpoint to enable descriptive evaluation of feasibility and acceptability. For EI, we will add all five 
subindices and examine cut-off points based on the distribution of the total samples’ EI scores and 
categorize participants as either poorly, moderately or highly engaged. 
On completion of the study, up to 20 participants will be invited to complete a one-time, audio-recorded 
semistructured qualitative exit interview with study staff by phone to further assess the acceptability of 
WYZ. We will select participants based on their EI category (i.e., poor, moderate or high engagement), 
and interviews will occur until data saturation or until 20 participants have been interviewed. The 
interviews will focus on the participant’s (1) attitudes towards exiting features; (2) attitudes and 
perceptions of privacy and information security; (3) features that they would modify, add or eliminate; 
and (4) suggested modifications to increase engagement and sustained use of WYZ. 

Finally, we will conduct in-depth qualitative interviews with clinical team members whose patients 
participated in the pilot study. The interview will focus on perceived (1) burden or ease of study 
procedures; (2) attitudes and beliefs regarding impact of the intervention on users’ engagement in HIV 
care and ART adherence; (3) sustainability and scalability of the application to other youth in the clinic; 
(4) challenges of integrating an application into clinical work; and (5) suggested modifications to 
improve usability. All interviews will be done by phone, digitally recorded, transcribed verbatim and 
deidentified. Transcriptions will be analyzed using thematic and content analysis frameworks. This 
combination is useful in identifying implicit and explicit ideas within the data while also allowing for 
quantification of qualitative data.22 
References 
1. 
Centers for Disease Control and Prevention. HIV and Youth: Viral Suppression. 2021; 
https://www.cdc.gov/hiv/group/age/youth/viral-suppression.html. Accessed March 9, 2022. 
2. 
Chen M, Rhodes PH, Hall IH, et al. Prevalence of undiagnosed HIV infection among persons aged >/=13 
years--National HIV Surveillance System, United States, 2005-2008. MMWR Morbidity and mortality weekly report. 
2012;61 Suppl:57-64. 
3. 
Torian LV, Wiewel EW, Liu KL, Sackoff JE, Frieden TR. Risk factors for delayed initiation of medical care 
after diagnosis of human immunodeficiency virus. Arch Intern Med. 2008;168(11):1181-1187. 
4. 
Rutstein RM, Gebo KA, Flynn PM, et al. Immunologic function and virologic suppression among children 
with perinatally acquired HIV infection on highly active antiretroviral therapy. Med Care. 2005;43(9):15-22. 
5. 
Zanoni BC, Mayer KH. The Adolescent and Young Adult HIV Cascade of Care in the United States: 
Exaggerated Health Disparities. Aids Patient Care St. 2014;28(3):128-135. 
6. 
Kim SH, Gerver SM, Fidler S, Ward H. Adherence to antiretroviral therapy in adolescents living with HIV: 
systematic review and meta-analysis. Aids. 2014;28(13):1945-1956. 
7. 
Lee M, Lee H, Kim Y, et al. Mobile App-Based Health Promotion Programs: A Systematic Review of the 
Literature. International Journal of Environmental Research and Public Health. 2018;15(12). 
8. 
Gold J, Lim MSC, Hocking JS, Keogh LA, Spelman T, Hellard ME. Determining the Impact of Text Messaging 
for Sexual Health Promotion to Young People. Sex Transm Dis. 2011;38(4):247-252. 
9. 
Mulawa MI, LeGrand S, Hightow-Weidman LB. eHealth to Enhance Treatment Adherence Among Youth 
Living with HIV. Current HIV/AIDS reports. 2018;15(4):336-349. 
10. 
Liu AY, Laborde ND, Coleman K, et al. DOT Diary: Developing a Novel Mobile App Using Artificial 
Intelligence and an Electronic Sexual Diary to Measure and Support PrEP Adherence Among Young Men Who Have 
Sex with Men. Aids Behav. 2021;25(4):1001-1012. 
11. 
Macaraig M, Lobato MN, Pilote KM, Wegener D. A National Survey on the Use of Electronic Directly 
Observed Therapy for Treatment of Tuberculosis. Journal of Public Health Management and Practice. 
2018;24(6):567-570. 
12. 
Garfein RS, Doshi RP. Synchronous and asynchronous video observed therapy (VOT) for tuberculosis 
treatment adherence monitoring and support. J Clin Tuber Oth Myc. 2019;17. 
13. 
Ngwatu BK, Nsengiyumva NP, Oxlade O, et al. The impact of digital health technologies on tuberculosis 
treatment: a systematic review. Eur Respir J. 2018;51(1). 

14. 
Parmar P, Mackie D, Varghese S, Cooper C. Use of Telemedicine Technologies in the Management of 
Infectious Diseases: A Review. Clin Infect Dis. 2015;60(7):1084-1094. 
15. 
Shiovitz TM, Bain EE, McCann DJ, et al. Mitigating the Effects of Nonadherence in Clinical Trials. J Clin 
Pharmacol. 2016;56(9):1151-1164. 
16. 
Ernsting C, Dombrowski SU, Oedekoven M, et al. Using Smartphones and Health Apps to Change and 
Manage Health Behaviors: A Population-Based Survey. Journal of medical Internet research. 2017;19(4). 
17. 
Labovitz DL, Shafner L, Gil MR, Virmani D, Hanina A. Using Artificial Intelligence to Reduce the Risk of 
Nonadherence in Patients on Anticoagulation Therapy. Stroke. 2017;48(5):1416-+. 
18. 
Galarraga O, Genberg BL, Martin RA, Laws MB, Wilson IB. Conditional Economic Incentives to Improve HIV 
Treatment Adherence: Literature Review and Theoretical Considerations. Aids Behav. 2013;17(7):2283-2292. 
19. 
de Walque D. The use of financial incentives to prevent unhealthy behaviors: A review. Soc Sci Med. 
2020;261. 
20. 
Brooke J. System Usability Scale (SUS). 1986. http://www. usability. gov/ how- to- and- tools/ methods/ 
system- usability- scale. Html (Accessed 9 Mar 2017). 
21. Taki S, Lymer S, Russell CG, et al. Assessing User Engagement of an mHealth Intervention: Development and 
Implementation of the Growing Healthy App Engagement Index. JMIR Mhealth Uhealth 2017;5:e89. 
22. Vaismoradi M, Turunen H, Bondas T. Content analysis and thematic analysis: Implications for conducting a 
qualitative descriptive study. Nurs Health Sci 2013;15:398–405.