Piloting a Sequential Multiple-Assignment Randomization Trial to Evaluate AllyQuest: an mHealth Intervention for HIV-positive Young MSM and TWSM to Optimize HIV Medication Adherence and Care Outcomes
Phase: NA
Conditions: Hiv, HIV/AIDS, Medication Adherence, Sexual Behavior, Sexually Transmitted Infections (Not HIV or Hepatitis)
Sponsor: University of North Carolina, Chapel Hill

STUDY PROTOCOL 
 
Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest: An mHealth Intervention for HIV-Positive Young 
MSM to Optimize HIV Medication Adherence and Care Outcomes 
 
 
NCT number NCT03916484 
Document Date 02/01/2023 
 

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 1 of 39
HRP-503 Revised: December 2, 2022
PROTOCOL TITLE:
Piloting a Sequential Multiple-Assignment Randomization Trial to Evaluate AllyQuest:
An mHealth Intervention for HIV-Positive Young MSM to Optimize HIV Medication 
Adherence and Care Outcomes
PRINCIPAL INVESTIGATOR:
Kathryn Muessig, PhD
College of Nursing
Phone: (850) 644-5260
E-mail: kmuessig@fsu.edu 
VERSION NUMBER/DATE:
Modified from UNC Version 01/14/2022
FSU Version 1.0 01/11/2023
REVISION HISTORY
Revision 
#
Version Date
Summary of Changes
Consent 
Change?
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 2 of 39
HRP-503 Revised: December 2, 2022
Table of Contents
1.0
Study Summary .......................................................................................................3
2.0
Objectives ................................................................................................................6
3.0
Background..............................................................................................................6
4.0
Study Endpoints.......................................................................................................9
5.0
Investigational Test Articles or Products.................................................................9
6.0
Procedures Involved ..............................................................................................14
7.0
Data and Specimen Banking..................................................................................19
8.0
Sharing of Results with Subjects ...........................................................................20
9.0
Study Timelines.....................................................................................................20
10.0
Inclusion and Exclusion Criteria ...........................................................................20
11.0
Vulnerable Populations..........................................................................................22
12.0
Local Number of Subjects .....................................................................................23
13.0
Recruitment Methods.............................................................................................23
14.0
Withdrawal of Subjects..........................................................................................24
15.0
Risks to Subjects....................................................................................................25
16.0
Potential Benefits to Subjects ................................................................................28
17.0
Data Management and Confidentiality..................................................................29
18.0
Provisions to Monitor the Data to Ensure the Safety of Subjects..........................36
19.0
Provisions to Protect the Privacy Interests of Subjects .........................................37
20.0
Compensation for Research-Related Injury...........................................................38
21.0
Economic Burden to Subjects................................................................................38
22.0
Consent Process.....................................................................................................38
23.0
Process to Document Consent in Writing..............................................................38
24.0
Setting....................................................................................................................39
25.0
Resources Available ..............................................................................................39
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 3 of 39
HRP-503 Revised: December 2, 2022
1.0
Study Summary
Study Title
Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest:
An mHealth Intervention for HIV-Positive Young MSM to 
Optimize HIV Medication Adherence and Care Outcomes
Study Design
This study is a two arm Sequential Multiple-Assignment 
Randomization Trial (SMART) that will test the feasibility 
and acceptability of the intervention and trial design in a 6-
month pilot intervention study among young men who have 
sex with men (YMSM) and young transgender women who 
have sex with men (YTWSM) living with HIV, ages 15-29. 
Participants will be randomized to one of three treatment 
strategies: (TS1) AQ alone followed by Continued AQ for 
Responders and Escalation to AQ+NSC for Non-
Responders; (TS2) AQ+NSC followed by Continued 
AQ+NSC for Responders and Continued AQ+NSC for Non-
Responders; and (TS3) AQ+NSC followed by De-escalation 
to AQ for Responders and Continued AQ+NSC for Non-
Responders. Responder status will be assessed at 3 months, 
based on app reported adherence and available viral load 
(VL) data. 
Primary Objective
To test the feasibility, acceptability, and preliminary efficacy 
of the AQ/AQ+ intervention app among 75 YMSM and 
YTWSM, ages 15-29, by conducting a SMART with 
assessments at baseline (month 0), month 3 (intervention arm 
reassignment) and month 6 (end of intervention). The 
primary outcome measures are: (a) Intervention feasibility: 
Average proportion of days of any app use, average 
proportion of days of HIV medication tracked and (b) 
Intervention acceptability: Mean intervention acceptability, 
composite score AHAA.
To analyze app paradata to determine which treatment 
strategies embedded in the SMART (escalation, de-
escalation, and maintenance combinations) are most 
promising to test for sustained app engagement, self-reported 
adherence, and VS. 
Secondary 
Objective(s)
The secondary outcome measures (preliminary efficacy) are: 
(c) Change in ART medication adherence from Baseline to 
Month 6 (self-report); (d) Difference in ART medication 
adherence between treatment arms at month 6; and (e) Rate 
of viral suppression (VS) at 6-month follow-up
Research 
Intervention(s)/ 
N/A 
This project is coming to FSU as data analysis only.
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 4 of 39
HRP-503 Revised: December 2, 2022
Investigational 
Agent(s) 
IND/IDE # 
(see section 5)
N/A
Study Population
Eligible participants: 1) are aged 15-29 at enrollment visit; 2) 
were assigned male sex at birth; 3) report sex with men or 
transgender women (lifetime); 4) are able to speak and read 
English; 5) have reliable daily access to an Android or iOS 
smartphone with a data plan during the intervention period 
(approx. 6 months); 6) are HIV+; 7) Prescribed ART; 8) At 
least one of the following: Having failed to show up for or 
missed 1 or more scheduled HIV care appointment in the 
past 12 months OR Last HIV care visit was more than 6 
months ago OR Self-reporting less than 90% ART adherence 
in the past 4 weeks OR have a detectable viral load measure 
in the past 12 months OR recently diagnosed with HIV (past 
3 months) Not eligible: 1) unable to be consented due to 
active substance use or psychological condition; 2) 
participating in another HIV-related study.
Sample Size
SMART Trial: 71 YMSM and YTWSM living with HIV 
recruited across 6 SRV sites (Birmingham, AL; Chapel Hill, 
NC; Charleston, SC; Charlotte, NC; Detroit, MI; and 
Newark, NJ) and nationally through social media recruitment 
and online enrollment.
Study Duration for 
individual 
participants
The total duration of study participation and active 
intervention for each participant was 6 months. 
Study Specific 
Abbreviations/ 
Definitions 
AIDS
Acquired Immunodeficiency 
Syndrome
AHAA                        Acceptability of Health Apps among       
Adolescents
ART
Antiretroviral medicine 
AQ
AllyQuest
AQ+
AllyQuest with Integrated Next Step 
Counseling
CASI
Computer assisted self-interview
CFR
Code of Federal Regulations
CRF
Case report form
DBS
Dried blood spot
DHHS
U.S. Department of Health and 
Human Services
GCP
Good Clinical Practices
HIV
Human Immunodeficiency Virus
iNSC
Integrated Next Step Counseling
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 5 of 39
HRP-503 Revised: December 2, 2022
IRB
Institutional Review Board
MSM
Man (or men) who has sex with men
NIH
National Institutes of Health
NIMH
National Institutes of Mental Health
PrEP
Pre-exposure prophylaxis
PI
Principal Investigator
OHRP
Office of Human Research Protection
SID
Study ID number
SMART
Sequential Multiple-Assignment 
Randomization Trial 
SRV
Subject Recruitment Venue
SSL
Secure Socket Layer
TW
Transgender woman (or women) 
TWSM
Transgender woman (or women) who 
has sex with men 
VL
Viral load
YAB
Youth Advisory Board
YAC
Youth Advisory Council
YMSM
Young man (or men) who have sex 
with men
YTWSM
Young transgender woman (or 
women) who have sex with men
Study Schema:
Key terms: AQ: AllyQuest; NSC: Integrated Next Steps Counseling; R: Randomization
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 6 of 39
HRP-503 Revised: December 2, 2022
2.0
Objectives
2.1
The primary objective is to test the feasibility of the SMART and to 
assess the acceptability of the AllyQuest app.  
The secondary objective is to assess the preliminary efficacy of the 
AQ/AQ+ intervention at improving self-reported ART adherence 
and viral suppression.  
Exploratory research question: Exploratory analyses will assess 
whether app engagement and adherence vary by possible tailoring 
variables (e.g. education level, depression, substance use). 
Qualitative research questions: We will thematically code and 
analyze in-app text data (iNSC session transcripts, daily discussion 
forum posts) and qualitative exit-interview data to assess app 
engagement and the acceptability and feasibility of the app features.
2.2
H1: The intervention design will be feasible as defined by average 
app use for the majority of participants on 30% of trial days or more. 
H2: The intervention will be acceptable as defined by average 
composite score on the AHAA.
H2a: Greater app use will be positively associated with 
higher AHAA scores. 
H3: Average self-reported ART medication adherence will show 
trends toward improvement from Baseline to Month 6 across 
intervention arms.
H3a: Greater app use will be positively associated with 
improvement in ART medication adherence.
H4: The proportion of virally suppressed participants will increase 
from baseline to 6-month follow-up 
H4a: Greater app use will be positively associated with 
achieving viral suppression.  
3.0
Background
3.1
In 2015, men who have sex with men (MSM) accounted for 92% 
of all new infections among people ages 13 to 24. While recent 
epidemic trends suggest some slowing of new HIV infections 
among MSM overall, significant disparities still exist for youth 
and Black and Latino MSM. The United States (US) HIV 
epidemic future depends in large part on what happens among 
young MSM (YMSM). 
Compared to adults, youth (ages 18 – 24) have worse outcomes 
across the HIV care continuum. Nationally, 33 to 38% of diagnosed 
HIV-positive youth aged 13 to 24 achieve VS, with rates lower for 
Black YMSM (16%). Among 13 sites in the US Adolescent Trials 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 7 of 39
HRP-503 Revised: December 2, 2022
Network (ATN) only 7% of a predominantly Black YMSM sample 
of HIV-positive youth achieved VS. In another ATN study, only 
59% of VS youth attending clinics with comprehensive supportive 
services maintained 12-month VS. As most YMSM with detectable 
viral load (VL) report condomless anal sex, onward transmission is 
likely to continue. Effective interventions for YMSM are needed to 
support sustained VS for individual health and HIV prevention. 
Sustained antiretroviral (ART) adherence is critical for VS but 
suboptimal among YMSM. A 2014 meta-analysis of adherence for 
youth living with HIV found overall low self-reported adherence of 
62.3%. Barriers to adherence and VS among youth and YMSM 
include forgetting or not feeling like taking ART, low adherence 
self-efficacy, psychological distress (depression, anxiety), substance 
use, structural barriers (transportation, homelessness, insurance), low 
social support, and HIV-related stigma. Many YMSM experience 
multiple barriers, impacting adherence and VS in a dose-response 
nature. 
Despite clear need, few adherence interventions have focused on 
YMSM. A 2017 systematic review of interventions along the HIV 
care continuum identified 117 medication adherence interventions; 
only 9 (8%) focused on adolescents/youth and only 2 (2%) focused 
on MSM. A 2014 review of adherence-focused Evidence Based 
Interventions found none exclusively designed for MSM or youth. 
While several adherence intervention studies for MSM are underway 
or under development, none appear to utilize smartphone technology 
and engagement features in the innovative ways we propose. 
The flexibility of mobile health (mHealth) platforms and the power 
of paradata allow real-time assessment of user engagement and 
adaptive delivery of tailored content to fit individual HIV 
management challenges. Daily mobile phone-based contact is 
acceptable to youth living with HIV and is associated with improved 
adherence. High smartphone ownership among diverse YMSM 
provides capacity for national scale up and diffusion. With extensive 
input from HIV-positive YMSM, we developed the AllyQuest (AQ) 
smartphone app as a tailored adherence and social support 
intervention. In a 4-week pilot study, AQ was rated highly and 
greater app use was associated with significant increases in HIV 
knowledge and self-management. 
3.2
Despite these promising results, some participants minimally 
engaged with the app and daily app use declined over the course of 
the pilot (from 4.3 to 2.8 days/week). Based on the pilot results, we 
propose to enhance AQ engagement and effectiveness by adding 
tailored content, small financial incentives, and personalized text-
based adherence counseling. AQ can be programmed to deliver 
individualized, tailored content through complex algorithms, but a 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 8 of 39
HRP-503 Revised: December 2, 2022
combination approach (auto-tailoring plus two-way texting) may 
benefit some YMSM via higher intervention engagement and 
adherence support. Our ART counseling model is based on Next 
Step Counseling (iNSC), a brief, client-centered motivational 
approach to supporting medication adherence. iNSC was developed 
for the iPrEx pre-exposure prophylaxis (PrEP) trial and was found to 
be highly acceptable among MSM. iNSC, which is grounded in the 
Information-Motivation-Behavioral skills (IMB) model of ART 
adherence, provides flexible, individualized adherence support. Core 
components of iNSC include: reviewing participant experiences with 
adherence, exploring adherence facilitators and barriers, identifying 
adherence needs and strategies, and developing an adherence action 
plan. Our study team is already adapting iNSC for two-way text chat 
in a PrEP adherence app for YMSM (U19-HD089881). 
Sequential multiple assignment randomized trials (SMART) have 
numerous advantages over traditional trial designs. In these adaptive 
interventions, the type or dose of the intervention (app alone vs. app 
+ iNSC) is adjusted based on participant characteristics or response 
(adherence/VS). Thus, an adaptive intervention is a sequence of 
decision rules that specify how the intensity or type of treatment 
should change depending on the patient's needs rather than applying 
a “one size fits all” approach. SMARTs are an efficient and rigorous 
way to maximize clinical utility and real-world applicability. A pilot 
SMART will allow us to estimate an optimal adaptive adherence 
intervention using a design that more closely mirrors clinical 
practice.
This study has been previously reviewed and approved by the 
University of North Carolina at Chapel Hill IRB. All study 
activities outlined in this protocol, including recruitment, 
enrollment, data collection, and follow-up with participants were 
completed while the research team was at UNC-CH. By 
submitting this protocol to Florida State University, we are 
seeking IRB approval for purposes of data analysis only. The 
original protocol, study documents, and approval letters are 
appended in the IRB submission.
3.3
Improving ART adherence among YMSM is an urgent public health 
issue. Given estimates that only 7 to 38% of youth/young adults 
living with HIV achieve VS, advancing innovative ART adherence 
interventions for YMSM with HIV is a pressing priority. This 
proposal expands on our existing theory-based app, AQ, which 
showed promise for engagement and impact on HIV knowledge and 
HIV self-management, while adding innovative features via 
enhanced app engagement strategies and SMART adaptive design. 
Using a SMART design will allow us to evaluate the impact of 
multiple evidence-based intervention treatment strategies. mHealth 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 9 of 39
HRP-503 Revised: December 2, 2022
technologies’ have an underutilized ability to monitor behaviors in 
real-time via app paradata (usage metrics) and adapt strategies when 
a participant is not responding to an initial intervention approach. 
Adaptive strategies that adjust for intervention non-response and 
increase in intensity to meet the needs of non-responders represent a 
more “real-world” approach, which would have a high public health 
impact and great potential for rapid scale-up and dissemination. 
Simultaneously evaluating the efficacy of a treatment strategy that 
de-escalates intervention intensity among responsive participants 
more closely mirrors clinical practice, follows the scientific rationale 
that some patients will benefit from a more intensive intervention 
up-front, and optimally applies resources where and when they are 
most needed. 
4.0
Study Endpoints
4.1
The primary study endpoint of 1) feasibility is app use, measured by 
number of trial days using AQ/AQ+; 2) acceptability is average 
composite score on the AHAA.
The secondary study endpoint of preliminary efficacy of the 
AQ/AQ+ intervention is average 6-month self-reported ART 
medication adherence and end-of-study proportion of viral 
suppression.
4.2
N/A
5.0
Investigational Test Articles or Products
5.1
AllyQuest is not an investigational device as it does not meet the 
requirements specified in Section 201(h) of the FD&C Act (21 USC 
321(h)). However, to provide context, we have included an overview of the 
intervention and its intended use in the study below: 
AQ is a theory-informed, multi-feature medication adherence and social 
support app. AQ+ adds a component of one-on-one guided support provided 
through the app by an adherence coach. The intervention app is currently 
designed to provide six months of educational content and adherence 
coaching. In the current trial, the app will be tested using an adaptive trial 
design which will allow participants to move between the two intervention 
conditions (AQ vs. AQ+) to best meet their current medication adherence 
and health needs. 
AQ development was guided by health behavior change theories including 
Social Cognitive Theory (SCT), narrative communication (e.g. storytelling), 
and the principles of persuasive technology. AQ addresses key principles of 
SCT including: (1) observational learning by doing daily activities; (2) 
modeling and vicarious experiences (observing and participating in Daily 
Discussions, exploration of narrative “choose-your-own-adventure” stories); 
(3) self-efficacy and verbal persuasion from expert sources (multi-media 
knowledge center, tailored messages) and (4) reinforcements (virtual 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 10 of 39
HRP-503 Revised: December 2, 2022
rewards, FI, achievements). The Fogg Behavioral Model (FBM) of 
persuasive technology informed the development of our technology partner 
Ayogo’s Empower™ platform employed for AQ. According to the FBM, 
the principal factors to promote behavior change using technology include 
triggers, ability, and motivation. In AQ app notifications are triggers for 
healthy behaviors while app content also helps participants identify their 
own daily triggers. Regular self-reported adherence prompts act as 
additional triggers. Ability is increased through knowledge and by defining 
steps toward behavioral goals (e.g. coping with side effects, knowing how to 
fill a prescription). Participants get tips from others dealing with similar 
issues and through narrative stories that reinforce the consequences of health 
behavior choices. App motivators include social support, rewards, goal 
setting, and achievement. Table 1 presents all features of the app along with 
their purpose and evidence base. 
Table 1: Ally Quest intervention components and scientific rationale 
Feature: Description
Scientific Rationale
Profile Page
Privacy features: avatars, pseudonyms, confidential pin 
number to open app, app time-out after 5 minutes of 
inactivity, medication tracker allows participant to 
choose any name (real or made-up) they want for 
medication reminder.
Work by our team and others identify 
anonymity and privacy as critical 
features for YMSM mHealth. 
Virtual bank account: Shows $ gained and lost based 
on daily app use. Account seeded at BL and 3-mo FU 
with $40. Gain $0.25/day for completing ≥1 in-app 
activity; lose $0.50/day for <1 activity. At 3- and 6-mo 
FU, user given account balance amount (possible range 
at each FU: $0 - $60).
Financial incentives support behavior 
change. and are being employed to 
maximize engagement and 
intervention retention.
App progression meter: Visual display of current app 
“level” and in-game currency, visible to other 
participants. YMSM level up and earn in-game 
currency based on app use. Redeem currency to unlock 
narratives and other app features.
Game-based elements (e.g. levels, 
competition) influence intervention 
engagement and impact. 
Daily Discussion
Social prompts: (e.g. How do you remember your 
meds?) Daily Discussion prompts foster community, 
peer sharing, and model and reinforce successful health 
behaviors. AQ/AQ+ will be optimized by increasing 
prompts (to 2 per day) to facilitate increased user 
interaction. Notifications are sent when users comment 
or “like” posts.
Most engaging AQ feature, users 
requested >1 daily prompt.  Social 
support and connection are important 
app features for HIV+ YMSM. 
Medication Tracker
Medication reminder system: Personalized reminders 
and habit building solutions to promote ART 
adherence. 
Medication reminders improve 
adherence, but may not be sufficient. 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 11 of 39
HRP-503 Revised: December 2, 2022
Tailored adherence strategies:  Upon set-up, 
participants enter medication details (doses/day, 
preferred time of day taken, food restrictions). The app 
uses this information to provide suggested adherence 
strategies (e.g. Take when I brush my teeth). 
Participants who are having adherence difficulties 
receive tailored feedback on new strategies and 
adherence tips.
Dynamic tailoring and unique 
feedback based on frequent 
assessments effectively promotes 
behavior change for many conditions, 
including HIV prevention and ART 
adherence. 
Appointment & refill reminders: Participants can create 
personalized reminders for ART refills
Requested by AQ usability and pilot 
YMSM. Feature of “ideal” apps. 
Adherence dashboard: Provides study team with an 
easily interpretable, real-time overview of each user’s 
ART tracking and app use. Automated and tailored 
messages can be delivered to provide support and 
encouragement. AQ+ arm can communicate 1:1 with 
iNSC counselor between sessions.
AQ pilot participants wanted more 
personalized reminders, messages and 
additional accountability regarding 
medication taking. 
Daily Challenge
Daily Quest: Actionable routine tasks help users set 
goals, build knowledge/skills. 
Brain games: Articles, quizzes and interactive exercises 
help users check knowledge and skills.
Rated highly by usability and pilot 
participants.  Gamification increases 
intervention engagement & impact. 
Knowledge Center
Multimedia: Presentation of information that includes 
HIV-related, safer-sex, relationships and general health 
and wellness. Users prompted with a reflection 
question after articles to apply the material to their 
lives. Visual meter shows progress toward completing 
each section.
Formative work of our team and 
others has identified that HIV+ 
YMSM desire information on both 
HIV-related issues and general health 
and wellness. 
Tailored Content Sets: Participants receive tailored 
daily content (quests, quizzes, articles) based on 
creation of an initial profile at app set-up (e.g. high vs. 
low HIV literacy and ART adherence, and health areas 
of interest). Level and preferences are reassessed 
monthly to adjust tailored content.   
AQ pilot trial users wanted articles 
most relevant to them. Content sets 
allow prioritization of information 
based on each user’s life situation.
Character-based Narratives
Text and video-based narratives feature HIV+ YMSM 
navigating situations that impact care engagement and 
adherence (e.g. unstable housing, substance use, 
disclosure). Stories facilitate self-reflection on contexts 
and choices that impact health; problem solving; and 
visualizing relevance in one’s own life.
Narrative communication through role 
modeling facilitates health behavior 
change. 
AQ+ iNSC
Two-way chat based iNSC (AQ+): Ability to deliver 
iNSC-based adherence counseling sessions via in-app 
two-way text-based chat.
Connection to provider/counselor and 
iNSC support ART adherence.
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 12 of 39
HRP-503 Revised: December 2, 2022
5.2
AQ+ Integrated Next Step Counseling Description 
Integrated Next Step Counseling (iNSC) is an interactive, client-
centered motivational intervention that was developed and 
implemented during the iPrEx PrEP trial and was found to be highly 
acceptable among MSM. The intervention is based on the IMB 
model and provides flexible, individualized counseling with the goal 
of supporting adherence and the accurate reporting of adherence. 
The key components of iNSC counseling include: review of 
participant experiences with adherence, exploration of adherence 
facilitators and barriers, identification of adherence needs, 
identification of strategies to meet needs, and development of an 
adherence action plan. Our team has adapted iNSC for use as a two-
way text-based intervention to support ART within the context of the 
AQ app. Those assigned to AQ+ for 3 months will receive 7 iNSC 
sessions; those who do not de-escalate at month 3 and are thus 
assigned to AQ+ for all 6 months will receive 15 iNSC sessions. 
Depending on when escalation occurs (due to VL testing window), 
time between chats may vary but will not exceed 1 chat per week. 
Table 2 outlines the goals, components, and timing of iNSC sessions 
based on participant study trajectories. Coaches will conduct brief 
text messaging “check ins” with YMSM in between sessions using 
AQ’s provider dashboard. Text check ins are an evidence-based 
method to keep participants engaged and can provide reminders of 
personal goals set during sessions. During the pilot SMART, staff 
will document all iNSC sessions and the study team will regularly 
review session transcripts for protocol fidelity. 
Table 2: Schedule of iNSC sessions 
Week
Type of 
Contact
Session Description
Key Takeaway
1
Session 
1
Coach (C) will introduce the concept of 
adherence counseling while answering 
participant’s initial questions. AC will work 
with participant to articulate what they hope 
to get out of participation.
Rapport building. C will learn 
participant’s motivation for 
participating in Ally Quest.
2
Session 
2
C will introduce SMART (Specific, 
Measurable, Achievable, Realistic, Timely) 
goals. C will guide participant in 
brainstorming goals. 
 Participant will learn what 
SMART goals are and how to 
set them. 
3
Session 
3
C will guide participant in goal 
solidification. Participant will verbalize a 
clear path forward.
Participant will set a SMART 
goal.
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 13 of 39
HRP-503 Revised: December 2, 2022
4
Check In
C will contact participant for the purpose of 
rapport building. C will check in on 
concerns.
5
Session 
4
C will work with participant to evaluate 
progress toward goal. C will guide in goal 
maintenance or reshaping, as needed.
Evaluate SMART goal 
progress.
6
Check In
C will contact participant for the purpose of 
rapport building. C will check in on 
concerns.
7
Session 
5
C will guide in goal maintenance or 
reshaping, as needed. Participant will assess 
their progress.
Participant will be able to 
successfully evaluate SMART 
goal progress independently.
8
Check In
C will contact participant for the purpose of 
rapport building. C will check in on 
concerns.
9
Session 
6
C will assist in goal maintenance or 
reshaping, as needed. Participant will begin 
to discuss post-intervention plans and will 
verbalize potential obstacles and ways to 
overcome them.
Participant will begin making 
plans for continuing progress 
post-intervention.
10
Check In
C will contact participant for the purpose of 
rapport building. C will check in on 
concerns.
At this point, we will be assessing intervention non-response through a combination of daily app-
reported adherence and VL measurement. Non-responders will remain on AQ+ and receive 
refocused iNSC support, while responders will either remain on AQ+ or de-escalate to AQ 
(depending on the treatment strategy they were randomized to at Baseline). De-escalated 
participants will still have access to the counselor but will not be asked to schedule sessions
Among those initially assigned to AQ, non-responders will be escalated to AQ+ and will begin 
their intervention at ’Week 1’ as listed in this chart.
(Non-responders) C will prep participant to 
begin contemplating whether or not they 
want to continue with the same goal.
Assess progress on SMART 
goal and determine factors that 
prevented participant from 
meeting the goal.
11
Session 
7
(De-escalators) C will guide participant in 
creating a maintenance plan. The two will 
address any final concerns.
Participant will have a plan for 
maintaining their progress.
12
13
Session 
8
C will answer questions and address 
concerns from participants, if any. Examine 
the previous 12 weeks and goal. The two 
will examine victories, challenges, and 
lessons learned.
C and participant will identify 
past challenges and strengths of 
the participant. Participant will 
be able to identify past goals 
they successfully met.
14
Session 
9
C will guide participant in setting new goal 
or revising their previous goal.
Participant will be able to 
differentiate their new SMART 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 14 of 39
HRP-503 Revised: December 2, 2022
goal from their previous 
SMART goal.
15
Session 
10
C will explore any concerns that the 
participant expresses, if any. C will work 
with participant to create a clear path 
forward for their SMART goal.
Participant will set plan for 
SMART goal.
16
Session 
11
C will lead participant in exploring and 
expressing the similarities and differences 
that they’re experiencing from the previous 
goal.
C will evaluate participant’s 
mindset and reframe as 
necessary.
17
Check In
C will contact participant for the purpose of 
rapport building. C will check in on 
concerns.
18
Session 
12
C will work with participant to evaluate 
progress toward goal. C will guide in goal 
maintenance or reshaping, as needed.
Evaluate SMART goal 
progress.
19
Check In
C will contact participant for the purpose of 
rapport building. C will check in on 
concerns.
20
Session 
13
C will guide in goal maintenance or 
reshaping, as needed. Participant will assess 
their progress.
Participant will be able to 
successfully evaluate SMART 
goal progress independently.
21
Check In
C will contact participant for the purpose of 
rapport building. C will check in on 
concerns.
22
Session 
14
C will guide participant in creating a 
maintenance plan.
Participant will have a plan for 
maintaining their progress.
23
Session 
15
C will address any final concerns.
Participant will leave with a 
sense of their strengths and 
abilities. 
24
Check In
C will send well wishes and reiterate the 
participant’s strengths.
5.3
N/A
5.4
N/A
6.0
Procedures Involved
6.1
Sequential Multiple Assignment Randomized Trial (SMART) 
Design 
We conducted a 6-month pilot SMART of the AQ intervention to 
test trial design feasibility and app acceptability. This was a two 
stage SMART including AQ and AQ+, with 71 YMSM and 
YTWSM living with HIV with assessments at baseline (month 0), 
month 3 (escalation/de-escalation) and months 6 (end of 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 15 of 39
HRP-503 Revised: December 2, 2022
intervention). Progress through the trial was dependent on the 
participant’s responsiveness as determined by self-reported daily 
adherence in the app and/or viral load status. As presented in Table 3 
below, at 3 months, among those initially assigned to AQ, non-
responders were escalated to AQ+ and responders remained on 
AQ; among those initially assigned to AQ+, non-responders 
remained on AQ+ and received refocused iNSC support, while 
responders either remained on AQ+ or de-escalated to AQ 
depending on the treatment strategy they were randomized to at 
baseline. The primary outcome measure is the feasibility of the 
SMART and the acceptability of the AllyQuest app. Secondary 
outcomes include self-reported ART adherence and viral 
suppression. As a secondary analysis, we will compare 
escalation/de-escalation decisions based on in-app reported ART 
adherence to month 3 viral loads. We also piloted home DBS 
collection for all participants. Qualitative exit interviews were 
conducted with a subset of participants (up to 25 total).  
Table 3: Escalation/De-Escalation Treatment Decision Guidelines 
Baseline
Treatment strategy decision rules at 3-month 
follow up
Status
Action 
AQ
<80% past month app-reported adherence OR 
≥5 consecutive days non-adherence; OR 
detectable VL
Non-responder
Escalate to AQ+
AQ
≥80% past month app-reported adherence 
AND do not have ≥5 consecutive days non-
adherence; AND are VS
Responder
Continue AQ
AQ+
<80% past month app-reported adherence OR 
≥5 consecutive days non-adherence; OR 
detectable VL
Non-responder
Continue AQ+ 
Revise adherence 
strategies as needed 
AQ+
≥80% past month app-reported adherence 
AND do not have ≥5 consecutive days non-
adherence; AND are VS
Responder
Assign to AQ+ or 
AQ based on 
treatment strategy 
randomized to at 
baseline
Table 4: Schedule of Visits and Procedures (completed while staff was at UNC)
Baseline/enrollment
(in-person or remote)
Month 3
(in-person or remote)
Month 6
(in-person or remote)
Visit-specific 
procedures
Informed Consent
Randomization
App install & tour 
SMART decision (re-
randomize, escalate, 
de-escalate, maintain)
Exit interview (optional)
Visit window
30 days from 
screening
14 days before and 30 
days after 
14 days before or 30 days 
after 
Viral load 
Chart review or blood 
draw or self-collected 
DBS or participant 
Chart review or blood 
draw or self-collected 
DBS or participant 
Chart review or blood 
draw
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 16 of 39
HRP-503 Revised: December 2, 2022
shared (either by 
documentation or 
showing result to 
SRV staff)
shared (either by 
documentation or 
showing result to SRV 
staff)
or self-collected DBS or 
participant shared (either 
by documentation or 
showing result to SRV 
staff)
ACASI survey
Qualtrics
Qualtrics
Qualtrics
Remuneration
$75 (survey, app 
download)
$25 (self-collected 
DBS)
$50 (survey)
$25 (self-collected 
DBS)
Up to $60 (app use)
$50 (survey)
$25 (self-collected DBS)
Up to $60  (app use)
$50 (interview)
$50 (bonus for 
participants who 
completed all 3 study 
visits)
Long chain referral 
(up to 5 referral 
coupons per 
enrolled 
participant)
$10 provided to the participant for each referred contact who screens 
eligible and presents for a baseline study visit. Total possible $50 additional 
remuneration. 
6.2
We conducted a 6-month pilot SMART intervention trial with 71 
HIV-positive YMSM/YTW on ART with detectable viral load (VL) 
or poor adherence. Participants were randomized to one of three 
treatment strategies (TS): (TS1) AQ alone followed by Continued 
AQ for Responders and Escalation to AQ+NSC for Non-
Responders; (TS2) AQ+NSC followed by Continued AQ+NSC for 
Responders and Continued AQ+NSC for Non-Responders; and 
(TS3) AQ+NSC followed by De-escalation to AQ for Responders 
and Continued AQ+NSC for Non-Responders. Daily app-reported 
adherence and 3-month VL were used as decision rules to escalate, 
de-escalate, or maintain intervention intensity. Participants 
completed surveys and VL tests at baseline, 3- and 6- months. We 
will measure the trial procedures’ feasibility and acceptability via 
surveys, in-depth exit interviews, and study records (e.g. attrition, 
staffing needs, and completeness of VL data). We will utilize 3 types 
of paradata to assess intervention feasibility, acceptability, and 
preliminary efficacy: 1) frequency and total time spent on each AQ 
feature; 2) in-app daily adherence reports; and 3) user-contributed 
content (adherence counseling chat logs, user-entered daily 
discussion content and adherence strategies). We will use paradata 
sources 1 and 2 to determine which treatment strategies embedded in 
the SMART (escalation, de-escalation, and maintenance 
combinations) result in sustained app engagement, self-reported 
adherence, and VS. Exploratory analyses will assess whether app 
engagement and adherence vary by possible tailoring variables (e.g. 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 17 of 39
HRP-503 Revised: December 2, 2022
education level, depression, current substance use). We will 
thematically code and analyze paradata source 3 to assess app 
engagement and the feasibility of two-way text-based adherence 
counseling. 
Feasibility and Acceptability analyses
To determine the SMART procedures’ feasibility we will adapt 
Thabane et al.’s framework for assessing pilot studies across 4 
domains: study Processes (e.g. recruitment rates, escalation 
procedures); Resources (e.g. retention, appropriateness of eligibility 
criteria); Management (e.g. staff capacity, time to deliver NSC); and 
Science (e.g. completeness of VL data, frequency of app use; 
estimated intervention effect, variance of intervention effect). To 
measure the SMART procedures’ acceptability we will use a 
combination of survey measures Acceptability of Health Apps 
among Adolescents (e.g. AHAA), open-ended interview questions, 
and paradata metrics (Table 5). CASI surveys will also measure 
moderators (Table 5) associated with adherence to assess feasibility 
of using these as tailoring variables in a future fully-powered 
SMART. 
Paradata overview: AQ’s Empower™ platform captures 
timestamped usage metrics (paradata) that can be compiled and 
analyzed by app feature and user across time. We will utilize three 
types of paradata to assess intervention feasibility, acceptability, and 
preliminary efficacy: 
Source 1: frequency and total time spent on each AQ 
feature; 
Source 2: in-app self-reported daily adherence; and 
Source 3: user-contributed content (adherence counseling 
chat logs, user-entered in-app content, including social wall 
posts and adherence strategies). 
We will assess AQ intervention feasibility by thematically coding 
and analyzing paradata source 3 to assess the nature of user-
contributed content and delivery of two-way text-based adherence 
counseling via AQ+. We will code whether adherence strategies are 
identified within the chat sessions and then implemented within the 
user’s in-app reported adherence strategies, and whether a user’s in-
app daily adherence reports (paradata source 2) increase after an 
NSC session. We will use paradata metrics as a proxy measure of 
AQ intervention acceptability by creating usage variables for each 
AQ feature (paradata source 1). We will supplement paradata with 
participant-reported intervention acceptability using an app 
acceptability questionnaire (BL, 3M, 6M) and the Acceptability of 
Health Apps among Adolescents (AHAA) via CASI at 3- and 6-
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 18 of 39
HRP-503 Revised: December 2, 2022
months and open-ended exit interview questions (6-months) 
regarding each app component.  
Table 5:  SMART Pilot Study Measures
Primary Outcomes
Measures
Feasibility
Adaptation of Thabane et al. framework for assessing successful pilot 
studies: Processes, Resources, Management, Science. 
Acceptability
Acceptability of Health Apps among Adolescents (AHAA): 22-item 
measure of global intervention satisfaction; In-depth exit interviews 
(intervention features, study protocols); and App use paradata: frequency 
and time and participant-contributed content (social wall posts; iNSC 
adherence counseling chat logs).
Secondary 
Outcomes
Self-reported ART 
adherence
BL, 3 & 6-months CASI: Missed doses (Last 30 days, Visual Analog Scale 
and 7-day recall) 
Daily: in-app reported adherence. 
Viral suppression
BL, 3 & 6 months: clinic/diagnostic VL, VS defined as VL below limit of 
detection per assay used.   
Moderators
Measures (BL, 3 & 6 months)
Sociodemographics
Education, income, race/ethnicity, age, housing stability.   
Depression
Patient Health Questionnaire (PHQ-8): 8-item measure to identify probable 
depression. 
Anxiety
General Anxiety Disorder scale (GAD-7): 7-item measure to screen for and 
measure anxiety. 
Substance use
Alcohol, Smoking and Substance Involvement Screening Test (ASSIST): 
8-item measure that screens for all levels of problem or risky substance 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 19 of 39
HRP-503 Revised: December 2, 2022
More details on the analysis plan are provided later in the protocol 
(see Section 17.0)
All data used for analysis will be stored securely in HIPAA-
compliant, cloud-based storage systems such as Microsoft Teams or 
OneDrive hosted by FSU. 
6.3
To minimize risks to confidentiality, we will secure study data with 
all appropriate physical, electronic and operational protections. Data 
will be stored in a physically secure environment. All data files will 
have encryption and strong password protection. Access to data will 
be on a role-based standard; only those study staff who require 
access to each type of data to complete their study-related roles will 
be allowed access. All study staff will be trained in security and 
confidentiality procedures and will sign a confidentiality agreement 
before receiving access to any participant data. 
All study-related information will be kept in double-locked, limited 
access areas at each study site. SIDs will not be entered into the 
mobile app and instead a unique app ID will be assigned to each 
participant and used when logging into the app. These unique App 
IDs will be provided by the developer and recorded into CRFs 
during enrollment. Original source documents (e.g., Contact 
Information Worksheet) for individual participants will be 
maintained at the respective SRV and will be accessible only to the 
study staff. Data from original source documents will be transcribed 
on CRFs as applicable. 
6.4
N/A
6.5
N/A 
6.6
N/A
7.0
Data and Specimen Banking
7.1
N/A
7.2
N/A
7.3
N/A
7.4
N/A
use. 
Social support
Medical Outcomes Survey- Social Support Scale: 19-item scale to measure 
perceived support including emotional/informational, tangible, 
affectionate, and belonging support. 
HIV stigma
HIV Stigma Scale: 40-item measure (4-point Likert scale) of perceived 
stigma among HIV+ persons. 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 20 of 39
HRP-503 Revised: December 2, 2022
8.0
Sharing of Results with Subjects
8.1
After completion of analyses, overall study results will be shared 
with participants in the form of a lay summary and/or infographic. 
This information will be sent to participants via the email they 
consented to using for study purposes. 
Any manuscripts for academic publication or other publicly 
available materials generated as a result of this work will not 
individually identify participants in this study. 
9.0
Study Timelines
9.1
Primary analyses of previously collected study data are expected to 
be completed within 6 months. 
10.0
Inclusion and Exclusion Criteria
10.1 The SMART included 71 youth (YMSM and YTWSM) who met 
one of the following five conditions 1) one or more detectable VL 
test result (above the lower limit of detection for the clinical assay if 
medical-chart verified) in the past 12-month while on ART for at 
least 3 months; 2) having failed to show up for or missed 1 or more 
scheduled HIV care appointment in the past 12 months; 3) last HIV 
care visit was more than 6 months ago; 4) self-reporting less than 
90% ART adherence in the past 4 weeks; 5) newly diagnosed with 
HIV in the past 3 months).
10.2 Inclusion Criteria 
Ages 15-29, inclusive at enrollment visit 
Male sex at birth
Report sex with men or transgender women (lifetime)
Able to speak and read English
HIV-positive status (self-report)
Reliable daily access to Android or iOS smartphone with a data plan 
during the intervention period (approximately 6 months)
Prescribed ART
Meet one of the follow five conditions 
o One or more detectable VL test result (above the lower limit of 
detection for the clinical assay if medical-chart verified) in the 
past 12-month while on ART for at least 3 months
o Having failed to show up for or missed 1 or more scheduled HIV 
care appointment(s) in the past 12 months
o Last HIV care visit was more than 6 months ago
o Self-reporting less than 90% ART adherence in the past 4 weeks
o Newly diagnosed with HIV (in the past 3 months)
Exclusion Criteria 

Aged younger than 15 years or older than 29 years at enrollment visit 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 21 of 39
HRP-503 Revised: December 2, 2022

Not available to meet (in person or remotely) with study staff for 
baseline study visit

Non-English speaking 

Unable to be consented due to cognitive impairment from active 
substance use or psychological condition

HIV-negative (self-report)

Not currently prescribed ART

Anticipate not having reliable access to a smartphone with a data plan 
for 2 or more days during field testing or 1 or more weeks during the 
SMART intervention period 

Unwilling or unable to comply with protocol requirements.

Participating in any other experimental HIV interventions

Does not meet any of the following criteria by medical chart or self-
report:
o
One or more detectable VL test result (above the lower limit of 
detection for the clinical assay if medical-chart verified) in the 
past 12 months while on ART for at least 3 months
o
Having failed to show up for or missed 1 or more scheduled HIV 
care appointment(s) in the past 12 months
o
Last HIV care visit was more than 6 months ago
o
Self-reporting less than 90% ART adherence in the past 4 weeks
o
Newly diagnosed with HIV (in the past 3 months)
10.3 Special populations: 

This study does not include adults who are unable to consent

This study DOES include teenagers who are 15 years or older

This study does not include pregnant women

NIH has concluded that this protocol does NOT meet Federal 
requirements governing prisoner participation in human subjects 
research and should NOT be considered by local IRBs for the 
recruitment of prisoners. Subjects enrolled who subsequently 
become incarcerated or are placed in detention may not continue 
study participation. Study visits cannot be conducted during the 
period of incarceration or detention
For all participants, study staff reviews the informed consent/assent 
to make an assessment of the youth’s decisional capacity and ability 
to provide consent/assent prior to signing, using a 2-step process. 
First, staff determines if the person understands the study goals by 
asking “Can you tell me what this study is about?” In step 2, 
potential participants will be asked questions designed to assess their 
capacity to understand, appreciate, reason with, and express a choice 
about participation in our specific protocol. Participants will be 
asked to: name things they will be expected to do during the study; 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 22 of 39
HRP-503 Revised: December 2, 2022
explain what they would do if they no longer wished to participate in 
the study; explain what they would do if they experienced distress 
during the study; and identify potential risks for participating in the 
study. For youth who cannot answer these questions, the study staff 
will go back and review the relevant elements of consent with the 
participant again and repeat the process. Youth who appear not to 
understand after repeated review will not be enrolled in the study. If 
the enrollment process occurs online, research staff will talk with 
potential participants via phone or online teleconference software 
such as Zoom or other comparable platform to assess for decisional 
capacity and to address any questions or concerns the person may 
have.
As this is a minimal risk study, we sought a waiver of parental 
consent for individuals under age 18. A request for a waiver of the 
requirement for parental permission is requested for 2 reasons:  1) 
many youth would be reluctant to participate in this study – which 
focuses on HIV and risks for poorer medication adherence – if they 
are required to get parental permission; and 2) many of the youth in 
our study are likely to be gay, bisexual, or have an attraction to 
persons of the same gender, but may not be out to their parents; 
requiring parental permission may place participants at risk for 
outing themselves as part of the LGBT community or having HIV. 
For these reasons, we believe it is important to be granted a waiver 
for parental permission for this study population.
11.0
Vulnerable Populations
11.1 Waiver of parental consent:  Participants under 18 years of age were 
enrolled with a waiver of parental consent granted by UNC-CH IRB 
as the central IRB (IRB of Record at the time of enrollment) to 
participate in this research study for youth participants who were 15 
to 17 years of age. The research team has been granted waivers of 
parental permission for prior studies with sexual minority youth. 
Under 45 CFR 46.408 (c), an IRB has the authority to waive parental 
permission if it determines that “a research protocol is designed for 
conditions or a subject population for which parental or guardian 
permission is not a reasonable requirement to protect the subjects” 
and “an appropriate mechanism for protecting the children who will 
participate as research subjects is substituted” and “that the waiver is 
not inconsistent with Federal, State, or local law.” A waiver of 
parental permission for studies with lesbian, gay, bisexual, 
transgender, and questioning (LGBTQ) youth that do not involve 
greater than minimal risk is a common practice among researchers 
working in the area of gay and lesbian health/mental health. This is 
done to avoid the selection biases operating in only recruiting youth 
whose parents are both aware of and comfortable with their sexual 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 23 of 39
HRP-503 Revised: December 2, 2022
orientation. Commonly these youth have explored their sexual 
orientation without their parents’ knowledge as the youth struggle 
with issues of disclosure and its consequences within the social, 
religious, and economic context of their families. A requirement for 
parental permission in this type of study could not only affect a 
person’s willingness to participate, but could also potentially impact 
the ability of researchers to engage in this type of research with 
sexual minority youth. Additionally, minors can often seek sexually 
transmitted infection (STI) and HIV prevention services without 
parental/legal guardian permission, depending on each site’s state 
laws.
If the purpose of requiring parental permission as stated in CFR is to 
protect the minor subject, then requiring parental permission for 
youth in these circumstances is not a reasonable requirement.
12.0
Local Number of Subjects
12.1 N/A
12.2 N/A
13.0
Recruitment Methods
13.1 We worked closely with the participating SRVs to engage youth. 
This included utilizing in person, venue-based (including recruitment 
from locally specific HIV clinics/providers) and online recruitment 
mechanisms. All participating SRVs either provided ART or had 
close relationships with sites in their community that prescribe. 
Recruitment procedures varied slightly depending on the SRV. This 
study requested a limited waiver of HIPAA to allow clinical sites to 
review patient medical records for recruitment purposes. We also 
followed respondent-driven sampling (RDS) methods and used a 
long-chain referral method to supplement recruitment, especially 
with the adolescents (15-17) who may be harder to reach than young 
adults (18-29).  Online recruitment was also conducted, including 
but not limited to recruiting via online social media outlets (e.g., 
Facebook, Instagram, Snapchat, etc.). In addition to recruitment and 
enrollment through the SRVs, recruitment also happened nationally 
through national online recruitment, including but not limited to 
social media recruitment.
13.2 Participants were sourced from venue-based and online recruitment 
methods implemented at Medical University of South Carolina, 
Wayne State University, University of Alabama Birmingham, 
University of North Carolina at Chapel Hill, Rutgers University, and 
RAIN Inc.
13.3 Potential participants recruited in any of the aforementioned ways 
were directed to the study screening survey (hosted on a HIPAA-
compliant online software program, e.g. Qualtrics). The screening 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 24 of 39
HRP-503 Revised: December 2, 2022
survey included information about the study. Those who expressed 
interest in the study were asked to provide informed consent/assent 
for eligibility screening. Those who provided consent/assent viewed 
the aim-specific eligibility screener questions. The screener survey 
could be viewed on participants’ own device or on a computer or 
tablet located in a confidential room at the SRV to determine if they 
meet eligibility criteria. Screening happened onsite and remotely. 
Study staff also administered the screener to participants by phone if 
desired. The participant’s clinical provider could also provide 
clinical information to complete the screener on behalf of the 
participant or with study staff. 
For those who expressed interest in participating, we asked for and 
recorded the first name, e-mail, and phone number of the individual 
via the online screener. We also collected preferred means of contact 
(e.g. call, text, email) and permission to leave a message. We used 
SSL encryption for transfers of information online and data were 
stored in secure, HIPAA-compliant servers. The study staff accessed 
the screener database to determine eligibility. Eligible individuals 
were contacted by study staff and scheduled for an enrollment visit. 
Screening and enrollment happened during one visit if possible. 
Enrollment and follow up visits happened in person or remotely.
13.4 N/A
13.5 N/A
13.6 N/A
14.0
Withdrawal of Subjects
14.1 The principal investigator has the authority to withdraw any 
participant at any time if it their opinion it would be in the best 
interest of the participant. The participant will be informed of this 
withdrawal and explained the rationale. Withdrawal will be 
documented in the study tracking system.
Subjects will be prematurely discontinued from the study if any of 
the following occurs: 

The subject withdraws consent/assent; 

The participant is unwilling or unable to comply with study 
procedures;

The investigator believes that ongoing participation may cause harm 
to the participant or study staff;

The investigator believes that ongoing participation may impact the 
integrity of the study data;

The study is cancelled by the NIH;

The study is cancelled for other administrative reasons;   
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 25 of 39
HRP-503 Revised: December 2, 2022

The participant repeatedly posts hostile or inflammatory information 
on the app (see Section 8.3)

The subject becomes incarcerated or placed in detention during the 
study; or 

Death of the subject.
14.2 Participants may end their participation in the study at any time. A 
study CRF will be completed for any participant who is prematurely 
discontinued from the study. No further data collection will occur 
from the date the decision is made to permanently discontinue the 
subject from the study. Participants who experience distress during 
the study while in the SRV clinic will be offered counseling on site. 
Participants who experience distress during the study and do not 
come to the SRV clinic for a visit will be provided a list of 
community referrals via phone or e-mail. 
14.3 N/A
15.0
Risks to Subjects
15.1 We identified the following 4 items as possible risks to subjects and 
describe how we plan on addressing those risks:
Breach of Confidentiality: A potential risk to participants is violation 
of confidentiality. We will take the utmost caution to protect the 
confidentiality of all responses. We will minimize this risk by 
maintaining confidentiality and discretion throughout the trial. Files 
– audio, paper, and electronic – will not have any identifying 
information about the study participants and will be tracked through 
a unique SID and participant code. All audio recordings will be 
downloaded and stored on a password-protected, encrypted 
computer in locked offices. Transcription of audio files will be 
conducted using a HIPAA compliant transcription service. Any 
names mentioned in the audio files will be redacted during 
transcription. Hard copies will be kept in locked files. This research 
specifically includes a vulnerable population, children (YMSM ages 
15-17). We will take every available step to minimize the risk of 
identifying/linking data being subpoenaed, stolen, or inadvertently 
released. First, we have a Certificate of Confidentiality from the 
NIH. Second, all research staff members are required to complete 
ethical clearance certification regarding protection of human’s 
subjects. Third, the study will safeguard against the risk of the 
linking information being stolen by keeping such information in a 
locked Excel file stored on a secure server to which only essential 
study personnel who have completed CITI certification for human 
subjects’ research ethics training (http://citiprogram.org) will have 
access. We have also included numerous features to ensure app 
security and privacy. All relevant app communications (e.g. those 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 26 of 39
HRP-503 Revised: December 2, 2022
between participants or those between participants and staff) will be 
secured via industry standard encrypted SSL communications links. 
These connections will ensure that all communications are 
inaccessible to unauthorized third parties. Furthermore, the app can 
be updated regularly to address any unforeseen security updates to 
the software libraries underlying the secured communication links. 
Beyond encrypting communication, we will allow users to enable an 
extra PIN to secure the app within their phone beyond the PIN 
required to unlock the phone itself. This will allow the user to share 
their phone generally with others without granting access to the AQ 
study app. These software security solutions will provide the layers 
of both communications security and physical access security to 
ensure that only authorized users have access to the information 
stored on the phone as well as the information being shared over 
communications links. We will take special care to ensure that AQ 
addresses participant privacy. We have chosen the app name, AQ, 
because it does not relate to health care and is therefore designed to 
be non-stigmatizing and un-interpretable by anyone observing a 
participant using the app on their mobile phones. During app 
onboarding, study staff will assist youth in choosing a discreet and 
anonymous username. Moreover, mobile phone screens themselves 
are also constructed to prevent surreptitious observation.
Emotional discomfort: It is possible that the study may precipitate 
discomfort and/or an emotional response when YMSM/YTW who 
have sex with men answer questions about potentially sensitive 
topics such as living with HIV and HIV medication adherence. 
Further, participants may feel embarrassed about discussing sensitive 
issues. All participants will be told during the informed consent 
process that their participation is voluntary and that they can chose to 
stop participating at any time without any consequences. Based on 
our experiences using similar data collections methods with 
YMSM/YTW who have sex with men in past studies, the likelihood 
and seriousness of this risk is minimal and we will strive to create a 
safe and comfortable environment for all study participants.
Discomfort related to blood draws for viral load testing: Participants 
will require three viral load measures during the study: at baseline, 3 
and 6 months. Staff will make every effort to align participants’ data 
collection time points with standard clinic viral load measures. When 
these do not align, an additional blood draw or participant self-
reported VL(via participant shared documentation or participant 
showing SRV staff a VL test result from their online health portal)  
or self-collected DBS will be needed. The risk of discomfort due to 
blood sample collection is considered minimal – and no greater than 
regular standard of care diagnostics. For both standard VL testing 
and DBS, physical harms are minimal. The type of safety lancet 
selected for use in the study was designed to minimize the potential 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 27 of 39
HRP-503 Revised: December 2, 2022
for infection or significant injury. Cleaning the area with an alcohol 
pad first lowers the already low infection risk further; pads are 
included in every kit. The spring-loaded lancet retracts fully after 
one firing, and thus does not pose a risk for “needle stick” injury to 
anyone after the device has been used. Subjects could experience 
dizziness, diaphoresis and nausea associated with the procedure but 
in prior clinical studies using dried blood spots, adverse events have 
been rare. 
Protecting against hostile interactions and inaccurate information 
from one participant to another. participants will be informed during 
the consent process of the “group rules” regarding interactions with 
one another (e.g., “Honesty is important; however, hostile or abusive 
language will not be tolerated and may be grounds for immediate 
removal from the study”). These “terms of use” will also be 
available with a link on the app for review by participants at any 
time. The project coordinator and/or research assistant will manually 
review participants’ posts on a daily basis to identify any hostile 
interactions or inaccurate information. Hostile interactions between 
participants will be handled by, first, reminding the participants in 
the interaction of the “group rules” regarding appropriate 
interactions. If the hostility continues, the offending participants will 
be given a warning that the continued hostility will result in 
withdrawal from the study if it continues. On the third offense, the 
offending participant will be withdrawn from the study. Text 
containing hostile exchanges will be removed from the study app 
and unavailable to view. In cases in which inaccurate information is 
found, project staff will be guided by experts on the team to post a 
comment that provides accurate information on the topic. In extreme 
cases, the PI may decide to withdraw a participant before the third 
offense. We will ensure all SRVs have a clear clinical protocol to 
address major issues that may come up at study visits or in online 
interactions. The major issues addressed in the procotol will be 
suicidal ideation, homicidality or violent ideation, emotional and 
cognitive disregulation, violent/aggressive or disruptive behavior, 
and intoxication. If research staff see concerning comments or 
messages online from participants regarding self-harm or harm of 
others, they will contact clinicians at the site and take immediate 
precautions.
All sites have specific policies governing the treatment of human 
subjects. These policies specify that medical and psychological 
assistance will be available in the immediate environment in the 
event a participant should experience any adverse reactions resulting 
from study procedures.
While participants will be informed that they may refuse to answer 
any question at any time, responses or reactions to certain questions 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 28 of 39
HRP-503 Revised: December 2, 2022
may indicate distress on the part of the participants. If at any time 
during the study, a participant divulges that they are at risk for harm, 
including but not limited to being abused or experiencing violence, if 
harm is suspected or likely, or if the participant states they are 
suicidal/homicidal, measures will be taken to ensure their safety. 
Reporting will be done as appropriate to the situation and the legal 
statutes, including reporting to child protection agencies or other 
appropriate agencies, and referrals will be provided to appropriate 
support, counseling, or treatment resources.
15.2 N/A
15.3 N/A
15.4 N/A
15.5 N/A
15.6 N/A 
16.0
Potential Benefits to Subjects
16.1 The risk to individual participants is small and the potential benefit 
to both the individual and society is substantial. The main benefit of 
the proposed study to society is the development of a potentially 
feasible and acceptable mobile app that improves HIV medication 
adherence, retention in HIV clinical care, and increased social, 
emotional and informational support among YMSM/YTWSM that 
can be scaled up for use in a variety of clinical and community 
settings. Participants may experience improvements in their own 
ART adherence, retention in ART clinical care, and social and 
emotional well-being thereby potentially reducing their risk for poor 
health outcomes related to HIV (e.g. opportunistic infection, 
detectable viral load) and risks and costs to society (i.e. forward HIV 
transmission, expense of additional medical care, loss of productive 
workforce labor). Therefore, the risk/benefit ratio is favorable. Study 
participants will be compensated for their time.
YMSM and YTWSM account for nearly two thirds of all new HIV 
infections in the US and YMSM are the only risk group experiencing 
a significant increase in HIV incidence. Youth are also less likely to 
link to care, and become and maintain consistent viral suppression. 
Given this high potential impact and low potential hazards to 
participants, we find that a clear examination of these research 
questions outweighs the previously mentioned risks. The 
effectiveness of a novel, scalable, technology driven, intervention to 
address ART adherence and retention in clinical care is understudied 
with this population. Given the significant health sequelae associated 
with HIV infections, and the paucity of intervention programs for 
this population of young adults, the knowledge to be gained from 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 29 of 39
HRP-503 Revised: December 2, 2022
this research is significant. The risks to participants are reasonable in 
relation to the importance of the knowledge to be gained.
16.2 N/A
17.0
Data Management and Confidentiality
17.1 Statistical analyses for preliminary intervention efficacy
Using the variables created from paradata sources 1 and 2, we will 
determine which Treatment Strategies (TS) embedded in the 
SMART (escalation, de-escalation, and maintenance combinations, 
Figure 1, TS1 – TS3) result in sustained app engagement, daily app-
reported ART adherence, and VS. First, we will assess VS at 3 
months. While pilot SMARTs are not intended to determine efficacy, 
we expect to see increases in VS at 3- and 6-months that vary by app 
use. We will calculate the change in the proportion of VS in the AQ 
and AQ+ arms from baseline to month 3. At a significance level of 
0.05, using a Bonferroni correction for multiple comparisons, and 
assuming 20% attrition (conservatively assumed to occur at 
baseline), the proposed sample size has 85% power to detect a 
medium effect size (Cohen’s h=0.5) and over 99% power to detect a 
large effect size (Cohen’s h=0.8). Second, using AQ paradata and 
participant profiles, we will compare the 3 TS embedded in the 
SMART to determine optimal intervention engagement approaches 
for maximizing app use, daily in-app reported ART adherence, and 
VS. We will compare the following TS which are embedded in the 
trial design: (TS1) assign AQ initially, escalate non-responders to 
AQ+, and have responders continue AQ; (TS2) assign AQ+ initially 
for 3 months, and continue AQ+ regardless of responder status; and 
(TS3) assign AQ+ initially, de-escalate responders to AQ, and have 
non-responders continue AQ+. TS comparisons will be made in 
terms of total app use (minutes and days), in-app daily reported 
adherence, and the rate of VS at 6 months. For VS, at a significance 
level of 0.05 and adjusting for all pairwise comparisons using a 
Bonferroni correction there is 80% power to detect a medium effect 
size (Cohen’s h=0.50).
Exploratory analyses 
The pilot SMART design creates a unique data resource that 
facilitates hypothesis-generating analyses including the estimation of 
an optimal adaptive treatment strategy (TS) that maps individual 
patient characteristics to treatment recommendations so as to 
maximize ART adherence/VS. While the pilot study is not powered 
for these analyses, we will conduct them in order to assess feasibility 
and generate hypotheses and estimates for full SMART sample size 
calculations. To estimate an optimal adaptive TS, we will use an 
estimation method known as non-parametric Q-learning with policy-
search. This method uses a sequence of regressions to estimate the 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 30 of 39
HRP-503 Revised: December 2, 2022
marginal mean outcome under any possible adaptive TS and 
subsequently searches for an optimal TS within a parsimonious and 
clinically interpretable class of candidate TS, e.g., those that can be 
expressed as flow-charts or as a sequence of if-then statements. For 
example: if a patient meets clinical criteria for depression at baseline 
screening, then recommend AQ+ initially, otherwise recommend 
AQ; if the patient starts on AQ and responds, then recommend 
continuing on AQ; if the patient starts on AQ but does not respond, 
then recommend escalating treatment to AQ+; if the patient starts on 
AQ+ and responds, then recommend de-escalating to AQ if the 
patient does not report depression in the past 3 months, otherwise 
recommend continuing on AQ+;  if the patient starts on AQ+ and 
does not respond, then recommend continuing on AQ+.   
In addition to generating new clinical hypotheses about how patient 
characteristics should be used to tailor treatment, estimation of an 
optimal adaptive TS can be used to conduct moderation analyses. An 
intermediate step in the Q-learning with policy-search estimation 
algorithm is that one must postulate and fit a model for the 
regression of the outcome on covariates and treatment. This model 
can be used to conduct standard moderation analyses, e.g., test for a 
differential effect of AQ relative to AQ+ among subjects with 
depression.  Furthermore, these models can be used to test whether 
moderators are tailoring variables, i.e., those that would be important 
for choosing a TS. Continuing the example of depression, to test if 
depression is a tailoring variable, we would test if the optimal TS 
that does not make use of depression status results in significantly 
lower adherence/VS rates compared with the overall (unconstrained) 
optimal TS. We will test for moderation and tailoring among known 
risk factors for non-adherence (depression, substance use, unstable 
housing, low social support, and HIV-related stigma) as well as 
additional variables identified during model building for the Q-
learning with policy search algorithm. The results of these analyses 
will inform the design of our full SMART. 
To augment the primary quantitative analyses, we will evaluate 
participants’ app usage and engagement by measuring app usage 
(e.g. numbers of times per week participants access app, completion 
of daily quests, and average daily use of app), number and content of 
social wall posts, amount of virtual currency collected and rewards 
unlocked. The research team will have access to these data as they 
are transferred from mobile phones to the secure server, which will 
occur any time the participant is connected to the internet via 
broadband or Wi-Fi. For AQ+ we will collect data on number of 
sessions, and content of sessions. 
Missing Data
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 31 of 39
HRP-503 Revised: December 2, 2022
Several procedures will be used to conduct data analysis when data 
for either outcomes or covariates are missing. The first step will be 
to assess the extent and pattern of missing data. If data are missing 
for only a few cases, then data analysis will be conducted only on 
study participants with complete data. However, when such a 
strategy would result in loss of data from a substantial proportion of 
participants, or if this approach would lead to biased or inaccurate 
results, the multiple imputation scheme for sequential multiple 
assignment randomized trials proposed by Shortreed et al. (2014) 
will be used to account for the missing data.
17.2 All laboratory specimens, questionnaires, evaluation forms, reports, 
transcripts, and other records will be identified by SID and 
participant code only, to maintain participant confidentiality. All 
paper records with personally-identifying information will be kept in 
a locked file cabinet in a limited secure access area at each SRV site. 
All computer entry and networking programs will be done with 
coded numbers only. Clinical information will not be released 
without written permission of the participant, except as necessary for 
monitoring by NIH.
Every effort will be made to ensure that study participants are 
protected from risks. 
Breach of Confidentiality:  A potential risk to participants is 
violation of confidentiality. We will take the utmost caution to 
protect the confidentiality of all responses. We will minimize this 
risk by maintaining confidentiality and discretion throughout all 
research procedures and data management and analysis. 
Participants may be concerned about the security of their data, 
particularly since it is collected and stored electronically. The 
research team has significant experience developing security 
protocols for Internet-based studies, and we will take a variety of 
steps to ensure participant security, including using a dedicated 
server behind a firewall, encryption of data, separation of identifiers 
from responses, and password-protected access to data. Therefore, 
we believe that this risk will be minimal. As described above, the 
AQ app includes a number of features to ensure app security and 
privacy. 
Participants will create a unique username that does not contain any 
identifying information but will be their username on the AllyQuest 
app. Intervention participants will choose an anonymous avatar 
(cartoon superhero head) to represent themselves on the app and are 
told at the enrollment visit and in multiple places on the app that it is 
a violation of community guidelines to reveal identifying 
information including name, address, phone number, email address, 
social media handles, places of employment, or locations to meet for 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 32 of 39
HRP-503 Revised: December 2, 2022
personal or business use. Participants will not be able to privately 
communicate with each other on the app - all conversations are 
viewable on the main feed. Trained research staff will monitor the 
app daily to ensure violations to privacy, even through self-
disclosure, are addressed. The AllyQuest app is also password-
protected.  
17.3 Certificate of Confidentiality: 
This research specifically targets a vulnerable population, children – 
YMSM/YTWSM ages 15-17. We will take every available step to minimize 
the risk of identifying/linking data being subpoenaed, stolen, or 
inadvertently released. First, per Section 2012 of the 21st Century Cures Act 
as implemented in the 2017 NIH Certificates of Confidentiality Policy, all 
ongoing or new research funded by NIH as of December 13, 2016 that is 
collecting or using identifiable, sensitive information is automatically issued 
a CoC.  As noted on the NIH website 
(http://grants.nih.gov/grants/policy/COC/faqs.htm#187), a Certificate of 
Confidentiality will help the research team “...avoid compelled ‘involuntary 
disclosure’ (e.g., subpoenas) of names and other identifying information 
about any individual who participates as a research subject (i.e., about 
whom the investigator maintains identifying information) during any time 
the Certificate is in effect.” 
17.4 To minimize risks to confidentiality, we will secure study data with 
all appropriate physical, electronic and operational protections. Data 
will be stored in a physically secure environment. All data files will 
have encryption and strong password protection. Any identifiable 
data will either be stored on secure servers or will be on fully 
encrypted laptops. CASI assessments and online eligibility screening 
will take place on an encrypted commercial survey website, 
Qualtrics. This site has been used by the investigators for thousands 
of online surveys with MSM with no data security breaches. Access 
to data will be on a role-based standard; only those study staff who 
require access to each type of data to complete their study-related 
roles will be allowed access. All study staff will be trained in 
security and confidentiality procedures, and will sign a 
confidentiality agreement before receiving access to any participant 
data. 
We will also develop procedures to minimize indirect disclosure that 
a participant is participating in an HIV- related research study, or a 
study that enrolls MSM/TWSM. For each mode of contact 
information, we will ask specifically whether anyone else potentially 
has access to that mode of communication, and if it is acceptable to 
leave a non-specific message about participation in a health study. 
No study-related messages will ever mention HIV or the nature of 
the research study. Additionally, all scripts for email, text message, 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 33 of 39
HRP-503 Revised: December 2, 2022
and telephone contact with participants will be reviewed and 
approved by the IRB before being used for contact with participants. 
The research team will use Dedoose software to perform all 
qualitative analyses. Dedoose is a web-based application for 
organizing and analyzing textual, audio, and video data (qualitative) 
along with outstanding functionality for their integration with 
survey, test score, ratings, and demographic data (quantitative). 
Dedoose employs the highest levels of data encryption available for 
a web application in all data storage, back up, and transmission. 
Dedoose allows for project specific encryption feature. When using 
this feature, only Dr. Muessig or her designee will hold the 
additional encryption key needed to be entered in order to view the 
project. This gives Dr. Muessig exclusive control over who can view 
the project under any circumstances.
REDCap - a HIPAA-compliant web-based platform – for participant 
management. REDCap was supported by UNC’s TraCS Clinical 
Research Data Management Service. As specified in their user 
manual: “Standard features of electronic clinical research data 
management systems are available in the web-based systems 
provided with REDCap. These include interactive data entry with 
real-time field validation, lab data imports, audit logs to record 
database modifications, database integrity checks, security (in logins, 
permissions based on need, and encryption), reporting, forms 
inventory, and exports to common statistical packages for analysis. 
Logging tracks all data entered in REDCap so that it can be traced 
back to the person who entered it. No data can be changed without 
showing who has made the changes. This allows the study team to 
ensure the security and integrity of the data collected and submitted; 
therefore, there are controls surrounding this aspect. REDCap also 
provides for principle investigator sign-off on data, as required in 
FDA studies. Although users can modify data based on their 
permissions, they cannot delete the subject or history of that subject.  
Requests to delete a subject must be made to the REDCap system 
administrator. The data is encrypted during transmission.  The 
servers are located in a secure campus area with all appropriate 
physical security measures in place. Access is by individual user id, 
and is restricted to the forms and/or functions that the user needs to 
have.  The applications themselves are written using open source 
tools, and have also been scanned by the campus security office to 
ensure that the applications are also protected from known exploits.  
The data is backed up to electronic media on a daily basis.  The 
electronic media is secured by ITS stored in a secure area separate 
from the servers.
The team will use Zoom or a comparable, HIPAA-compliant 
platform to remotely conduct study visits and the qualitative exit 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 34 of 39
HRP-503 Revised: December 2, 2022
interviews at 6-months. When using Zoom, participants will have the 
option to use Zoom in several formats: face-to-face video chat, video 
chat in which they can see the interviewer but the interviewer cannot 
see them, audio chat only, or a text based conversation. Zoom is 
compatible on PCs, tablets, and smartphones; as well as maintains 
the option to conduct an audio conference without the video 
component. All SRVs will be asked to use a HIPAA-compliant 
version of Zoom or a comparable video conferencing platform for 
study visits and qualitative exit interviews. 
End-to-end encryption. Zoom encrypts all presentation content at the 
application layer using the Advanced Encryption Standard (AES) 
256-bit algorithm. Zoom end-to-end (E2E) chat encryption allows 
for a secured communication where only the intended recipient can 
read the secured message. Zoom uses public and private keys to 
encrypt the chat session with Advance Encryption Standard 
(AES256), and session keys are generated with device unique 
hardware ID to avoid data being read from other devices. This 
ensures that the session cannot be eavesdropped or tampered with.
Cloud Control Infrastructure. A distributed network of low-latency 
multimedia routers (software) resides on Zoom’s communications 
infrastructure. With these low-latency multimedia routers, all session 
data originating from the host’s device and arriving at the 
participants’ devices is dynamically switched — never stored 
persistently through the Zoom communications infrastructure. 
Zoom’s communications infrastructure for real-time video, audio, 
and data communications resides on Zoom dedicated servers, which 
are housed in SSAE 16 SOC2 compliant datacenters on opposite 
sides of the US. Zoom sessions are completely temporary and 
operate analogously to the popular mobile conversation over the 
public mobile network. In addition to unique security beneﬁts, 
Zoom’s communications infrastructure also enables an extremely 
scalable and highly available meeting infrastructure unrestricted by 
the limitations of physical data centers.
The Zoom client communicates with the multimedia router to 
establish a reliable and secure connection. At the time of 
instantiation, the Zoom client will determine the best method for 
communication, attempting to connect automatically using udp and 
tcp port 8801, 8802 and 8804 or HTTPS (port 443/TLS).
The Zoom sessions will contain identifying information, but this 
information will be stripped form the recorded Zoom sessions before 
they are sent to the analysis team for content analysis. 
Qualtrics may be used for study surveys, online informed 
consent/assent forms, and online HIPAA waivers.
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 35 of 39
HRP-503 Revised: December 2, 2022
Qualtrics uses Transport Layer Security (TLS) encryption (also 
known as Hypertext Transfer Protocol Secure (HTTPS)) for all 
transmitted data.  Survey data are protected with passwords and 
HTTPS referrer checking.  The data is hosted by third party data 
centers that are Statement on Standards for Attestation Engagements 
(SSAE)-16 Service Organization Control (SOC) II certified.  All 
data at rest are encrypted, and data on deprecated hard drives are 
destroyed by U.S. Department of Defense methods and delivered to 
a third-party data destruction service.
Qualtrics deploys the general requirements set forth by many Federal 
Acts including the Federal Information Security Management Act 
(FISMA) of 2002.  They meet or exceed the minimum requirements 
as outlined in Federal Information Processing Standards (FIPS) 
Publication 200.
Health Insurance Portability and Accountability Act (HIPAA) 
Statement: With some restrictions, Qualtrics may be designated as a 
Business Associate when the Qualtrics BA Agreement is signed with 
a Covered Entity—those organizations that are required to comply 
with HIPAA privacy rules.  All client data are considered 
confidential, and treated as such.
Related to HIPAA, Health Information Technology for Economic 
and Clinical Health Act (HITECH) are updated assessment rules to 
ensure that data are properly protected and best security practices are 
followed.  By using secure and certified data centers, Qualtrics 
ensures the highest protection and testing as per HITECH 
requirements.
Back Up Recording
All qualitative interviews may also be recorded using a back-up 
digital audio recorder. Audio files will be erased after being 
transcribed and quality control checked and transcripts will be de-
identified. All audio files will be kept confidential and stored in a 
locked/limited access folder on secured servers, which are only 
accessible to designated study staff. All members of the research 
team will be trained in confidentiality and have signed 
confidentiality agreements. A professional transcription service, 
experienced in the handling of confidential data, will be used to fully 
transcribe verbatim all audio files. Prior to receipt of the first audio 
file, the transcription service will be instructed to redact from the 
typed transcript identifying information (e.g., a name) that may have 
been verbalized during the course of the interviews. 
App paradata
Ayogo (technology partner and creator of AQ) servers can be hosted 
on Amazon Web Services or ClearDATA (HIPAA compliant) 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 36 of 39
HRP-503 Revised: December 2, 2022
services. The server-side app is built on a scalable architecture using 
Ruby on Rails, MySQL, Redis (for caching) and Node.js for 
analytics functions. Ayogo’s infrastructure has been certified as 
secure and private by IT departments of large pharmaceutical 
companies and hospital systems. Ayogo uses storage encryption for 
personal identifying information, SSL and VPN for transport 
encryption, firewalls to protect access to unapproved ports and IP 
addresses, centralized key management for system administrator 
authorization and authentication, hourly backups, and regular 
HIPAA and security training for all employees. Ayogo's client side 
architecture is web based and runs on the Web and on Mobile 
devices. The client-side app uses Typescript, ES6 modules, 
Webpack, npm, and Apache Cordova. The client communicates 
securely with the server over HTTPS. Ayogo's client side 
infrastructure has undergone security and privacy certifications with 
NowSecure. Analytics de-identification occurs within the Ayogo 
VPC and outputs de-identified data to make analytics reports with 
software such as Tableau. De-identified data is stored in Ayogo’s 
data warehouse; users are identified by a 36-character unique 
identifier linked to a study ID via a secure access code.
17.5  N/A
18.0
Provisions to Monitor the Data to Ensure the Safety of Subjects.
18.1 Site research staff must first follow both the FSU IRB and their own 
IRB’s procedure for reporting and managing untoward effects.
There are three types of untoward effects to be identified: 1) those 
related to the participant, 2) those related to the study staff, and 3) 
those related to the neighborhood/community (if applicable).
First, the study will catalogue any untoward effect related to the 
participant. Reporting is required for occurrences including social 
harms, psychological distress, and serious life-threatening events 
such as suicide attempts. These may be immediately apparent to the 
study staff, such as the participant’s emotional upset state requiring 
referral for counseling; or they may be delayed and reported later to 
study staff, such as physical harm to an individual for having 
participated in the study. Study staff will record these untoward 
effects in the site study log, notify the protocol team, and complete 
appropriate CRFs to document adverse events and social harms. 
Study staff will be briefed during the training on the scope of 
possible untoward effects and instructed to report events.
Second, study staff may encounter untoward events during sessions 
that personally affect them. Training and guidance will seek to 
minimize this risk. Nonetheless, an assessment of the cost of 
conducting this study must include cataloguing these events as well. 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 37 of 39
HRP-503 Revised: December 2, 2022
The protocol chairs should be notified of these events so that they 
may be immediately addressed, evaluated, and guidance modified or 
expanded to minimize similar risk to other study staff.
Third, a critically important area any community-based study intends 
to evaluate is the impact, including untoward effects, of the project 
on the community. This will be done informally for this protocol 
with untoward events being reported to the protocol team.
All untoward effects/adverse events/unanticipated problems will also 
need to be reported to the FSU IRB if they meet all three of the 
following criteria:
“Unanticipated problems involving risks to subjects or others” 
(UPIRSO) refers to any incident, experience, or outcome that: is 
unexpected (in terms of nature, severity, or frequency) given (a) the 
research procedures that are described in the protocol-related 
documents, such as the IRB-approved research protocol and 
informed consent document; and (b) the characteristics of the subject 
population being studied; is related or possibly related to a subject’s 
participation in the research; and 
Suggests that the research places subjects or others at a greater risk 
of harm (including physical, psychological, economic, or social 
harm) than was previously known or recognized. 
Events that meet the criteria for an UPIRSO and are also serious 
adverse events should be reported to the FSU IRB within one (1) 
week of the investigator becoming aware of the event. Any other 
events that meet the criteria for a UPIRSO should be reported to the 
IRB within two (2) weeks of the investigator becoming aware of the 
problem. 
If the report cannot be completed in its entirety within the required 
time period, a preliminary report should be submitted. The report 
should be amended once the event is resolved and/or more 
information becomes available.
19.0
Provisions to Protect the Privacy Interests of Subjects
19.1 N/A
19.2 To minimize the risk of participants feeling uncomfortable about 
answering personal questions, we used Computer Assisted Self 
Interview (CASI) methods for the study's assessments. In CASI, 
participants read assessment questions on a laptop computer or 
mobile phone and use a combination of mouse click and 
keyboard/touchscreen entry to input the answers themselves. Study 
staff were available to assist participants with questions or technical 
difficulties on the CASI. Participants were also able to refuse to 
answer any question that makes them uncomfortable. In-depth 
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 38 of 39
HRP-503 Revised: December 2, 2022
interviews were conducted face-to-face or online depending on the 
research project. Descriptions of procedures were provided in each 
study projects’ human subjects sections.
19.3 Participant-related study information will be identified through a 
study ID number (SID) and participant code on all participant CRFs, 
audio files, transcripts, lab specimens, and CASI files. Participant 
names or other personally-identifying information will not be used 
on any study documents other than the Contact Information 
Worksheet (stored in double-locked office separate from other study 
information only accessible by designated study staff) and will be 
redacted from field trial interview transcripts.
19.4 HIPAA Waiver: Clinical SRVs asked participants to sign a HIPAA 
waiver to access their medical records at the clinical SRV to check 
for viral load test results in the eligible time period prior to each 
study visit. SRVs also asked participants to sign medical record 
releases for any external clinical sites where participants report 
having had a viral load test in the past month or plan to have a viral 
load test during the 6-month intervention period. Participants were 
asked to sign a paper or online HIPAA waiver. Qualtrics was used 
for the online HIPAA waiver.
CFR Parts 160 and 164 Standards for Privacy of Individually 
Identifiable Health Information ("Privacy Rule" Pursuant to the 
Health Insurance Portability and Accountability Act - HIPAA)
Each site was responsible for adherence to their individual 
institution’s HIPAA policies and procedures.
20.0
Compensation for Research-Related Injury
20.1 N/A 
20.2 N/A
20.3 N/A
21.0
Economic Burden to Subjects
21.1 There are no costs to participants for completion of analysis of data 
collected during the study. 
22.0
Consent Process
22.1 This protocol is for data analysis only, no new/additional participants 
will be enrolled in the study. 
23.0
Process to Document Consent in Writing
23.1 All data analyzed under this protocol has been collected using UNC-
CH IRB approved documentation and procedures.
23.2 N/A
STUDY00003843
FSU IRB Approved
1 February 2023

PROTOCOL TITLE: Piloting a Sequential Multiple-Assignment Randomization Trial to 
Evaluate AllyQuest
Page 39 of 39
HRP-503 Revised: December 2, 2022
23.3 N/A
24.0
Setting
24.1 Analysis of study data will be completed by research staff employed 
by Florida State University.
25.0
Resources Available
25.1 All proposed study staff have participated in the required trainings in 
participation and conduct of studies that involve human subjects, and 
any future study staff will do so upon hiring. Research staff at 
individual SRVs who interact with AllyQuest participants at 
assessments do not need to be clinicians. A research assistant (RA) 
level position should be sufficient to verify eligibility (obtain 
informed consent, be available for questions during the CASI, and 
explain the AllyQuest intervention. Research staff at SRVs will be 
trained via videoconferencing on the intervention components and 
will be given a script and checklist to review with participants.
STUDY00003843
FSU IRB Approved
1 February 2023