MyPEEPS (Male Youth Pursuing Empowerment, Education, and Prevention Around Sexuality) Mobile for Young Transgender Men
Phase: NA
Conditions: HIV
Sponsor: Columbia University

1 
Protocol Number: AAAT8624 
Version 1.5 11/09/2023
NCT05424718 
MyPEEPS YTM: To Advance HIV Prevention 
Sponsored by: 
Division of AIDS (DAIDS) United States (US) 
National Institute of Mental Health (NIMH) 
US National Institutes of Health (NIH) 
Multiple Principal Investigators (MPIs): 
Rebecca Schnall 
Robert Garofalo 
Co-Investigators (Co-Is) 
Lisa M. Kuhns 
Asa Radix 
Final Version 1.5 
November 09, 2023 

2 
Protocol Number: AAAT8624 
Version 1.5  
11/09/2023 
 
 
 
 
 
TABLE OF CONTENTS 
PROTOCOL TEAM ROSTER .............................................................................................................. 4 
Co-Investigators .................................................................................................................................. 4 
LIST OF ABBREVIATIONS .................................................................................................................. 5 
MyPEEPS YTM: To AdvanceHIV Prevention ....................................................................................... 6 
1.3 
Specific Aims ...................................................................................................................... 8 
1.4 
Preliminary Research ......................................................................................................... 8 
2.1 
STUDY DESIGN .............................................................................................................. 10 
2.2 
Theoretical Framework ..................................................................................................... 10 
3.1 
STUDY POPULATION ....................................................................................................... 11 
3.2.2 
Inclusion of Children ........................................................................................................... 11 
3.3 
Exclusion Criteria for All Aims ........................................................................................... 11 
3.4 
Recruitment Procedures ................................................................................................... 12 
3.5 
Screening Procedures ...................................................................................................... 15 
3.6 
Informed Assent and Consent ........................................................................................... 15 
4.0 
STUDY PROCEDURES ..................................................................................................... 16 
4.2 
Locator Form/Contact Information ..................................................................................... 16 
4.3 
Specific Aim 1 Procedures ................................................................................................ 16 
4.4 
Specific Aim 2 Procedures ................................................................................................ 18 
4.5 
Specific Aim 3 Procedures ................................................................................................ 25 
5.0 
DATA COLLECTION AND SITE MONITORING ................................................................. 27 
5.2 
Data Records ................................................................................................................... 27 
5.3 
Data Collection ................................................................................................................. 27 
5.4 
Data Submission .............................................................................................................. 28 
5.5 
Data Quality Control and Quality Assurance...................................................................... 29 
5.6 
Study Site Monitoring and Record Availability ................................................................... 29 
6.1 
PARTICIPANT TRACKING & CLINICAL MANAGEMNT .................................................... 29 
6.3 
Intervening on “Social Harm” ............................................................................................ 29 
6.4 
Criteria for Permanent Intervention Discontinuation for an Individual Participant .................. 30 

3 
Protocol Number: AAAT8624 
Version 1.5  
11/09/2023 
 
 
6.5 
Criteria for Premature Study Discontinuation for an Individual Participant ........................... 30 
7.1 
DATA MANGEMENT, SECURITY & ANALYSIS ................................................................ 31 
7.3 
Data and Safety Monitoring ................................................................................................ 31 
7.4 
Adverse events .................................................................................................................. 33 
7.5 
Data Security and Privacy .................................................................................................. 34 
7.6 
Data Analysis Plan for Aim 2 .............................................................................................. 35 
7.7 
Missing Data ...................................................................................................................... 39 
8.1 
HUMAN SUBJECTS PROTECTIONS ................................................................................ 39 
8.2 
Waiver of Parental Written Consent .................................................................................... 40 
8.3 
Vulnerable Subjects:Protecting Against/Minimizing Potential Risk ..................................... 41 
8.4 
Risks .................................................................................................................................. 41 
8.5 
Social Impact Events .......................................................................................................... 42 
8.6 
Benefits .............................................................................................................................. 42 
8.7 
Participant Privacy and Confidentiality ............................................................................... 43 
8.8 
Certificate of Confidentiality ................................................................................................ 44 
8.9 
Unexpected and Serious Adverse Event Reporting ............................................................ 45 
8.10 
ClinicalTrials.gov ................................................................................................................ 45 
9.0 PUBLICATION POLICY ........................................................................................................................ 45 

4 
 
 
Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
PROTOCOL TEAM ROSTER 
 
Multiple Principal Investigators 
 
Rebecca Schnall, PhD, MPH, RN-BC 
Columbia University School of Nursing 
212-342-6886 
rb897@columbia.edu 
School of Nursing 
Columbia University 
560 West 168th Street 
New York, NY 10032 
 
Robert Garofalo, MD, MPH 
Ann & Robert H. Lurie Children’s Hospital Chicago 
312-227-6110 
RGarofalo@luriechildrens.org 
Center for Gender, Sexuality and HIV Prevention 
4711 N. Broadway 
Chicago, Illinois 60640 
 
 
Co-Investigators 
 
Lisa M. Kuhns, PhD (Core Team Member) 
Northwestern University Feinberg School of Medicine 
312-227-6119 
lkuhns@luriechildrens.org 
Ann & Robert H. Lurie Children's Hospital of Chicago 
225 E Chicago Avenue, BOX 161 
Chicago Illinois 60611 
 
Asa Radix, MD (Core Team Member) 
Callen-Lorde Community Health Center 
212-271-7200 
ARadix@callen-lorde.org 
356 West 18th Street 
New York, NY 10011 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
11 
 
 
 
LIST OF ABBREVIATIONS 
 
 
AE 
Adverse Event 
AES/AES256 
Advanced Encryption Standard 256-bit algorithm 
ARBA 
AIDS-Risk Behavior Assessment 
CASI 
Computer Assisted Self Interviewing 
CDC 
Centers for Disease Control 
CFR 
Code of Federal Regulations 
Co-I 
Co-Investigator 
CRF 
Case Report Forms 
CUIMC 
Columbia University Irving Medical Center 
DAIDS 
Division of Aids 
DSMB 
Data Safety and Monitoring Board 
E2E 
End-to-End 
EC/IRB 
Ethics Committee/Institutional Review Board 
HAL 
Hair Analytical Laboratory 
HIPAA 
Health Insurance Portability and Accountability Act 
HIV 
Human Immunodeficiency Virus 
ID 
Identification card 
IP Address 
Internet Protocol Address 
IRB 
Institutional Review Board 
ISO 
Information Security Office 
ITT 
Intent-to-treat 
MCAR 
Missing completely at random 
mHealth 
Mobile Health 
MI 
Multiple imputation 
MPIs 
Multiple Principal Investigators 
MSM 
Men who have sex with men 
MyPEEPS 
Male youth Pursuing Empowerment, Education, and 
Prevention around Sexuality 
NIMH 
National Institute of Mental Health 
NIH 
National Institutes of Health 
nPEP 
Non-Occupation Post-Exposure Prophylaxis 
PHI 
Protected Health Information 
PI 
Principal Investigator 
PID 
Participant Identification Number 
PrEP 
Pre-Exposure Prophylaxis 
PRS 
Prevention Research Synthesis 
PSSUQ 
Post-Study System Usability Questionnaire 
RCT 
Randomized Control Trial 
REDCap 
Research Electronic Data Capture 
SAE 
Serious adverse event 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
 
SEM 
Social Ecological Model 
SEM 
Structural Equation Modeling 
SID 
Study ID number 
SOC 
Standard of care 
STI 
Sexually Transmitted Infection 
TFV 
Tenofovir 
US 
United States 
YMSM 
Young men who have sex with men 
YTM 
Young transmasculine men 
 
MyPEEPS YTM: To Advance HIV Prevention 
 
 
1.1 
INTRODUCTION 
 
1.2 
Background Information 
Transgender men (TM) are at high-risk for acquiring HIV. Growing research on TM 
clearly demonstrates increased HIV risk and burden, although not as much risk in comparison to 
transgender women among whom risk and vulnerability to HIV/STIs has been better described, but 
considerably higher than the general US population. A recent review was among the first to 
systematically estimate HIV risk and burden in TM in the US. In this review, there was a high 
prevalence of HIV (3.2%; lab confirmed) and high rates of HIV-risk behaviors, such as sex work 
(13.1%) and unprotected sexual intercourse (24.5%), which have been strongly associated with 
HIV acquisition(1). Although the prevalence of HIV is lower in TM as compared to transgender 
women, having mainly female sexual partners might partially explain the comparatively lower HIV 
prevalence among TM. Previously undescribed HIV risk behaviors and contextual factorspoint 
to the immediate need for the development and testing of HIV prevention interventions 
among YTM. More specifically, the overall estimate for STI diagnoses for TM was 28.7%. Co- 
morbid infection with bacterial STIs increases the risk of transmission and acquisition of HIV(2, 3), 
and it has been well established that STIs play an important role in HIV infections among cisgender 
YMSM(4). Further, prevalence of drug use was high with 38.1% of TM reporting illicit 
drug use. 
Importantly, in our community-based sample of 565 TM in NYC, HIV prevalence among TM 
who have sex with cisgender men only was 11.8% as compared to 3.5% for TM who have sex with 
cisgender men and other partners, and 2.1% for TM who have sex with cisgender women only. 
These data suggest the potential for alarmingly high rates of HIV in TM who have sex with 
cisgender men in urban settings, which has not been previously described and speaks directly to 
the public health significance of the proposed research. 
Stigma and discrimination have contributed to disparities in access to healthcare 
services for TM. Studies show TM experience a high burden of discrimination in multiple settings, 
including healthcare settings(5, 6). In a large study of TM adults, 33% of the sample delayed 
needed medical care when sick or injured, and 39% delayed routine preventive care. Stigma and 
discrimination among TM make it less likely for them to seek sexual healthcare(7). Low knowledge 
of PrEP(1), low rates of PrEP access(8), and low uptake of PrEP(9) among TM suggests the strong 
need for the development of health prevention interventions delivered outside of traditional 
 
 
11 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
11 
 
 
healthcare settings. 
Despite biomedical advances in HIV prevention, there remains a dearth of evidence- 
based, sexual health HIV prevention interventions for TM. More than four decades into the HIV 
epidemic, the current Centers for Disease Control and Prevention (CDC) compendium of 
evidence- based interventions (EBIs) for HIV prevention has no evidence-based interventions for  
YTM. This underscores the need for behavioral HIV/STI prevention interventions targeted to YTM 
to avert new infections given their increased risk in comparison to the general population. 
Therefore, there is a critical need to develop and test the efficacy of HIV prevention interventions 
for YTM, especially those under 18 years of age. We comprehensively looked at the existing EBIs 
and considered adaptation of those developed for adolescent females, those with family based- 
approaches, those designed to target racial/ethnic groups, or those developed for MSM; however, 
we decided on using the MyPEEPS intervention framework because we have preliminary data 
suggesting that YTM find the intervention to be very usable. Specific areas that are needed for 
developing this intervention for YTM have been identified that can be achieved through Aim 1 of 
this proposed study. 
MyPEEPS (Male Youth Pursuing Empowerment, Education, and Prevention around 
Sexuality) is, to our knowledge, the only theoretically-driven intervention in the published 
literature which has been tested in YMSM under age 18 years and developed with strong 
formative work among a diverse group of YMSM. MyPEEPS (R34MH079707; PI: Garofalo) was 
originally developed as a manualized curriculum consisting of 6 interactive group sessions (2 hours 
each), delivered twice weekly for 3 weeks. MyPEEPS was developed and tested among diverse 
YMSM ages 16-20 years, including Black, Latino, and White YMSM. The MyPEEPS intervention  
targets social-cognitive and cognitive-behavioral factors based on best practices for behavior 
change, emotion regulation, and HIV intervention (e.g., knowledge, self-efficacy, and behavioral 
skills) within YMSM-specific social contexts. The MyPEEPS scenarios include, for example, those 
involving emotionally activating and cognitively complex situations involving for partner-specific 
factors (e.g., older partners), experiences of social stigmatization (e.g., by race and/or sexual 
orientation), and sexualized contexts (e.g., online sexual partner interaction, under the influence of 
alcohol or drugs) and specifically address emotional regulation and minority stress which are 
salient issues facing adolescents at risk for acquiring HIV. Building on this work, our study team 
translated the MyPEEPS intervention onto a mobile platform for a slightly younger age group, ages 
13-18 years, with content delivered through YMSM avatars (e.g., caricature or graphic identity) 
whose profiles and problems are based on the formative research of the original pilot trial and who 
manage their sexual health against a backdrop of personal, family-based, and relational 
challenges. We are now testing MyPEEPS Mobile in a national randomized trial of 761 cisgender 
YMSM (U01MD011279; MPIs: Schnall and Garofalo) with interim analysis (n=350) supporting 
preliminary efficacy (see Preliminary Studies). 
 
Benefits of mHealth in HIV prevention interventions. As part of a comprehensive strategy 
across the continuum of HIV prevention and care, behavioral interventions remain an important 
tool in the fight against HIV(10). While many HIV prevention interventions have been delivered 
face-to-face, the emergence of eHealth as a platform for health behavior change provides new 
opportunities for developing HIV prevention strategies(11). eHealth is a generic term that applies to 
an increasingly large number of electronically delivered interventions and can include web-based 
tools including videos, games, chat rooms, and social networking sites as well as text/SMS and e- 
mail messaging (12). Schnall et al. conducted a review of eHealth HIV prevention interventions in 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
11 
 
 
MSM and found preliminary evidence that eHealth (e.g., SMS, web-based education modules) 
improves HIV prevention behaviors in MSM but has not been widely developed for or tested in 
adolescents(13). Studies have also shown that eHealth HIV prevention interventions developed for 
adult MSM(14-17) are appealing because of their privacy feature and convenience of use(18). 
Therefore, delivery of MyPEEPS using mobile technology has the potential to remove many 
of the barriers to engagement in YTM and improve scalability(19). We will adopt recent 
recommendations for planning of scale up of efficacious mHealth HIV prevention interventions, 
including formation of a scale-up working group, which will meet regularly to formulate 
recommendations for future scale up should the intervention proveefficacious. 
In summary, this application proposes to intervene on a population, YTM, among whom evidence 
of HIV risk is emerging and increasingly well-documented, at an age (15-25 years) of increasing 
vulnerability by building on our existing MyPEEPS mobile intervention and developing an 
intervention specific to the needs of YTM. The proposed mHealth approach is well-suited to both 
reach this relatively hidden population and, if efficacious, scale widely. 
 
1.3 
Specific Aims 
 
 
1) Using qualitative methodology, expert feedback, and usability assessments, develop 
MyPEEPS Mobile for YTM. 
2) Conduct a pilot randomized controlled trial to examine the feasibility, acceptability, and 
preliminary efficacy of the revised MyPEEPS Mobile App in a sample of 80 YTM (15-25 years) 
and refine the study methods for a future efficacy trial. 
3) Assess predisposing, enabling, and reinforcing factors for MyPEEPS Mobile among YTM 
through theoretically-guided in-depth interviews. 
 
1.4 
Preliminary Research 
 
a) Development and Pilot Testing of the MyPEEPS Intervention (R34MH079707; PI: 
Garofalo). MyPEEPS is a social and behavioral theory-driven HIV prevention intervention 
for diverse YMSM, developed by members of our research team, (Garofalo [MPI], Kuhns 
[Co-I]), using a multi-stage, mixed-method approach. MyPEEPS was based on the Social- 
Personal Framework (Figure 1) (20) and added important psychosocial and contextual risk 
factors to YMSM risk-taking. The MyPEEPS pilot study evaluated the initial efficacy, 
feasibility, and acceptability in an ethnically diverse sample (N=101), including MSM under 
age 18 years, using a randomized controlled design with an active, time-matched control 
group. There were no significant differences between arms with regard to demographic 
characteristics. Sexual risk and social cognitive outcomes were assessed at baseline, 6-, 
and 12-weeks post-intervention. Over the entire follow-up period, intervention participants 
were less likely than controls to engage in any sexual behavior while under the influence of 
substances (p<.05), and a decreasing trend in unprotected anal sex while under the 
influence of substances was also observed in this group (p = .08), which is an important 
risk factor for acquiring HIV(21). Thus, the MyPEEPS intervention, a 6-session behavioral 
intervention tailored to YMSM ages 16-20 years, was shown to be feasible, acceptable, and 
demonstrated evidence of preliminary efficacy in reducing sexual risk, specifically sexual 
risk while under the influence of substances(22). 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
11 
 
 
b) The MyPEEPS Mobile Trial (U01MD011279; MPIs: Schnall and Garofalo). Our current 
trial of 700 YMSM has shown promising preliminary results. First, we have successfully 
reached our recruitment goals. Second, we conducted an interim analysis for the purposes 
of report to our Data and Safety Monitoring Board and evaluated the preliminary outcomes 
in the first half (N=350) of our study sample from baseline to 3-month follow-up. Higher self- 
efficacy for HIV prevention behaviors (p<.001) and more recent HIV tests in the past 3 
months (p=0.02) were reported by the intervention group compared to control. The number 
of condomless anal sex acts was lower among the intervention group for both insertive anal 
sex acts (p=0.03) and receptive anal sex acts (p=0.0001). These preliminary findings were 
reported to our Data and Safety Monitoring Board and are very promising for supporting the 
use of this intervention for reducing HIV risk behaviors in YTM. Finally, and of great 
relevance to this proposed study, over 800 YTM screened into our study because of their 
interest, sexual risk, and behavioral vulnerability but were ineligible based on birth sex 
criteria. We will use the list of transgender men who we screened as part of their larger 
efficacy trial, as part of our recruitment efforts for the proposed R34 study. Further, we 
published the findings from our formative work in several peer-reviewed publications and 
presentations(23-27). 
c) In preparation for this application, we conducted formative focus groups with YTM. 
We conducted 6 focus group sessions with 49 YTM from 4 sites (8 Birmingham, AL, 17 
Chicago, IL, 12 New York, 11 Seattle, WA). In our sample of 49 YTM, 25 had ever had sex 
with a cisgender guy and 11 had sex with a transgender female (e.g., sex designated male 
at birth). Over half (27 of 49) feared disclosing their status as a YTM to a doctor or 
healthcare provider, demonstrating a key barrier to health seeking. Only 24 (49%) had 
been tested for HIV in their lifetime. Prior to the focus group session, all participants 
completed all the MyPEEPS Mobile activities on their mobile phones (e.g., the modules 
designed for 
the cisgender YMSM in our current trial). Findings from the focus group 
sessions demonstrated the perceived usefulness of the intervention for YTM, but also   
highlighted important limitations. One participant stated, “But yes, it was super realisticand 
just made it less scary just knowing that you could honestly talk to someone like that  
where it’s totally protected and private.” Another youth described the usefulness of the App 
and said, “I wished I had found out about that sooner, because I had a STD scare about a 
year ago, and I had to learn about it in my doctor’s office with my father there, which was 
absolutely terrifying.” We conducted a follow-up survey after the participants used the App 
to assess feasibility and acceptability; 82% (N=40) agreed that “Using MyPEEPS willmake 
it easier to make safer decisions about their sexual health” and “MyPEEPS gave them 
information and skills needed to avoid situations that make them uncomfortable and put 
their sexual health at risk of HIV and other STIs.” However salient, limitations to the current 
App content highlighted the need for the specific tailoring we propose herein, targeting 
mechanisms of risk specific to YTM. The specific areas that needed to be addressed were: 
1) content related to family planning, 2) sex while transitioning, 3) body parts, 4) body type, 
5) gender inclusivity, 6) addition of TM and gender non-binary avatars, 7) social challenges 
such as stigma, violence, mis-gendering, and transphobia, 8) sexual/relational power 
dynamics, and 
9) educating their partners on their gender identity. 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
11 
 
 
2.1 
STUDY DESIGN 
 
We propose to develop a mobile intervention for YTM building on MyPEEPS Mobile, which was 
originally developed for very young sexual minority men. Central to this intervention is the premise 
that it will be delivered to youth during a developmental period that precedes or coincides with 
sexual debut, an important time for intervention, prior to or concurrent with initiation of high-risk 
behaviors. Given both the gap in HIV prevention science and the lack of current targeted 
interventions, and building upon our multidisciplinary team’s extensive experience in HIV 
prevention, mHealth, behavioral interventions, randomized controlled trials, and transgender 
health, we propose the following specific aims: 1) Using qualitative methodology, expert feedback, 
and usability assessments, develop MyPEEPS Mobile for YTM, 2) Conduct a pilot randomized 
controlled trial to examine the feasibility, acceptability, and preliminary efficacy of the revised 
MyPEEPS Mobile App in a sample of 80 YTM (15-25 years) and refine the study methods for a 
future efficacy trial, and 3) Assess predisposing, enabling, and reinforcing factors for MyPEEPS 
Mobile among YTM through theoretically-guided in-depth interviews. The proposed MyPEEPS 
intervention for YTM is a novel and evidence-driven intervention using mobile technology to deliver 
HIV prevention information to high-risk youth. This will be the first study to develop and pilot a 
scaled-up, mobile HIV prevention intervention designed by, and piloted for, a diverse group of YTM. 
The final product of this study will be the basis for an R01 application to conduct a large- scale 
efficacy study for this population. The proposed MyPEEPS Mobile intervention for YTM is a novel 
and evidence-driven intervention using mobile technology to deliver HIV prevention information 
specifically developed for YTM. Consequently, we are confident that the proposed intervention will 
be of high impact intervention for improving HIV prevention behaviors in YTM and have long-term 
implications for overall improvement in the public’s health. 
 
2.2 
Theoretical Framework 
 
Theoretical Framework. The Social-Personal Framework (Figure 1) was the theoretical 
framework that guided the two prior NIH-funded MyPEEPS studies. The framework has been used 
broadly for HIV prevention in YMSM, and young women at high-risk for HIV and we now propose 
to use an enhanced model for guiding our development of MyPEEPS Mobile for YTM. We updated 
Figure 1. Social-Personal Framework of HIV and Substance Use Risk 
the model based on our formativefocus 
groups with YTM (described below and 
highlighted in yellow in Figure 1). The 
barriers to sexual health promotion for 
YTM that are presented in the pilot data 
below are now included (highlighted in 
Figure 1) in the theoretical framework for 
adapting the MyPEEPS intervention for 
transmasculine young people. This 
framework highlights the role of broader 
factors including: psychological well- 
being/emotional regulation, peer and 
partner relationships, sexual context 
variables, and stress related to an 
emerging sexual minority identity (sexual 
Psychological Well -Being/Emo೦onalRegula೦on 
Externalizing Symptoms, Internalizing Symptoms
(Depression, anxiety, loneliness, self-esteem 
Seeking professional help and interpersonal support for
dealing with diﬃcult experiences 
Sexual Minority Stress 
Homophobia 
Discrimina೦on
Violence 
HIV and Substance Use, 
Substance Useinﬂuenced
by gender aﬃrming
therapy,
Knowledge,A೦tudes,
Beliefs 
Peer and Partner 
Rela೦onships 
Peer norms/Socializing 
 
Partner Pressure 
Sexual Context Variables 
unhealthy age -power 
Internet use 
Public sex 
Older Partners
Discussions of sexual
consent 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
21 
 
 
minority stress). This model guided the development of the content for MyPEEPS Mobile. The model 
provides clear direction for HIV prevention programming related to each of these constructs which are 
described in Table 3: developing positive peer norms, encouraging associations with non-risk-taking 
peers and adults, reducing psychological distress, improving emotion regulation, teaching assertive 
partner communication and safer sex behaviors (e.g., condom use), and developing a healthy personal 
identity. 
For example, the current MyPEEPS Mobile has content related to sexual minority stress and is 
presented with 4 ways to manage stigma (see Table 3, activity 17). In MyPEEPS Mobile for YTM, we 
will update the content to provide information on dealing with stigma specific to YTM and coping with 
victimization and violence. Guided by this framework, we plan to update the sexual context content to 
specifically focus on discussions of sexual consent, which was a noteworthy issue amongst the YTM in 
our focus group sessions. As a final example, our current MyPEEPS 
Mobile has extensive information on substance use (see Table 3, activity 7 “Goin’ Downhill Fast). For 
MyPEEPS Mobile for YTM, we will include additional information about the interaction between 
substance use and gender affirming therapy. Each activity will be reviewed, and we will seek user- 
informed input from a diverse group of YTM for adaptation as described in the research plan for Aim 1 
below. 
 
3.1 
STUDY POPULATION 
 
3.2 
Inclusion Criteria for All Aims 
 
3.2.1 
To participate in any aspect of the study, participants must be: 1) between 15 and 25 years 
of age; 2) female sex assigned at birth; 3) identify as a transgender man or along the transmasculine 
spectrum (including a transmasculine non-binary gender; e.g., male, trans male, transmasculine gender 
non-binary); 4) understand and read English; 5) live within the US; 6) own a smartphone; 7) self-report 
condomless receptive anal or vaginal penile sex with either a cisgender male or transgender woman 
(e.g., individual designated or assigned male at birth) in the past year; and 8) self-report HIV-negative or 
unknown status. Note: Despite being uncommonly reported in the epidemiological literature on sexual 
risk among TM, and after careful consideration, we decided to include having had condomless receptive 
anal or vaginal sex witha transgender woman based on the following considerations: (1) the anatomical 
mechanics of receptive penile intercourse being the highest risk sexual act for HIV transmission; (2) the 
high HIV prevalence among transgender women, making that population high risk sexual partners; and 
(3) our focus groups with YTM reported that penile sex with a transgender woman was not uncommon 
among the proposed target population. The eligibility criteria was expanded to include transmasculine 
gender nonbinary individuals (1) approximately 30% of transgender individuals identify as nonbinary or 
gender diverse(28), (2) at this age youth identify across the spectrum and are fluid in their identity (3) 
nonbinary individuals experience similar barriers to healthcare and exclusion from HIV prevention 
interventions.(29, 30). 
 
3.2.2 Inclusion of Children 
Participants will include children ages 15-17, thus children will be well represented. The investigative team has 
extensive experience conducting research with youth in the proposed age range. Involvement of children in this 
study will be in compliance with all applicable subparts of 45 CFR Part 46 as well as other pertinent Federal and 
State laws/regulations. This project meets standards for inclusion of children. 
 
3.3 
Exclusion Criteria for All Aims 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
21 
 
 
 
Participant exclusion criteria for all Aims. Youth are ineligible to participate in the trial if: 1) they are 
HIV positive; 2) they are unable to provide informed consent due to severe mental or physical illness or 
substance intoxication at the time of enrollment; 3) they are concurrently enrolled in another HIV 
prevention study. Note: Given that (a) 72% of our current same-age sample of YMSM age 15-18 
reported condomless anal sex in the past year and (b) >800 YTM screened ineligible in our current 
protocol based on birth-assigned sex alone, but reported considerable behavioral risk, we believe that 
even with this stringent inclusion/exclusion criteria we will be able to successfully recruit our proposed 
sample in the timeline proposed. Although these criteria may require us to screen a larger number of 
participants, we will still be able to reach the recruitment goals, and this is important for assessing the 
feasibility of this inclusion criteria for a larger R01 trial. 
 
3.4 
Recruitment Procedures 
 
3.4.1 
Recruitment Targets 
 
To address all study aims, we will recruit a total of 105 participants. For the heuristic evaluation, we will 
recruit 5 informaticians. For the usability testing, RCT, and in-depth semi-structured interviews, we will 
recruit 100high-risk racially diverse YTM in the U.S. using community informed advertisements and 
messages through electronic methods (e.g., Twitch, Cameo, TikTok, Instagram, Twitter, Facebook, 
Snapchat partnering with YouTube and Instagram influencers) that we have successfully used to recruit 
diverse YMSM across several prior NIH-funded projects using social ecological frameworks. We will use 
these electronic methods to target high HIV prevalence geographic contexts (known as “hot spots”) and 
EHE jurisdictions. Per the RFA, we will recruit at least 50% of our study participants from the EHE 
priority jurisdictions. Since 41% of people acquiring HIV annually in the U.S. live outside of EHE priority 
jurisdictions and are often understudied, we will recruit 50% of our participants from across the broader 
U.S (non-EHE priority jurisdictions). Additionally, RFA-IA-21-018 calls for research to include 50% racial 
minorities, which is integrated into our recruitment strategy (30% Black Non-Hispanic and 30% Latinx (all 
races) YTM). Deliberately including YTM of color is critical, as it is estimated that 50% of Black MSM will 
acquire HIV in their lifetimes compared to 1 in 11 White MSM.(31) 
 
3.4.2 
Targeted Online National Campaign Led by the Study Team 
 
In our own campaigns, we will use race- and age-specific as well as population-based advertising 
strategies, including psychographic- and geo-targeting (in-depth, publicly available consumer data such 
as interests; bulk upload by region, city, zip code, or Designated Market Areas (DMA)) to advertise only 
to potentially eligible individuals. an especially strong following among YMSM. We will also consumer- 
centered marketing strategies advertise through Twitter, Snapchat, TikTok, Cameo, Twitch and partner 
with YouTube and Instagram influencers to serve as ‘study ambassadors.’ We will also advertise through 
popular dating sites (Grindr, Scruff, Grizzly, Hornet, etc.) to recruit study participants who are 18+ years. 
We will target our recruitment advertising to YMSM of color.(32) Sample ads from our past studies which 
were developed by our staff with our community-based participants and have been successfully used to 
recruit Black and Latino YMSM are in Table 1. 
 
 
Table 1. Sample Study Advertisement to Recruit Eligible Participants on Social Media 
a. Instagram Profile 
b. ‘Holding Hands’ 
c. ‘Text Messaging’ 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
21 
 
 
Instagram profile receives on 
average 1,200 profile visits a 
week. With 990 followers, 
interested volunteers visit our 
active Instagram profile to 
learn about the study project, 
watch our story and scroll 
through content posted daily. 
In the span of 15 months, the 
Featuring two diverse young 
guys holding hands with a 
rainbow background, this 
popular ad was promoted on 
the feeds and stories of 
Instagram users using a 
targeted demographic criterion. 
In the span of 6 months, 690 
eligible volunteers screened 
This ad represents the 
widespread use of text 
messaging to communicate 
between adolescent MSM. 
Over the course of our 
MyPEEPS RCT, this ad has 
recruited 338 eligible 
volunteers of which 63 have 
successfully enrolled. 
study recruited 270 eligible 
volunteers who screened 
using the link provided on our 
profile; 64 of those people 
enrolled in the MyPEEPS 
Mobile RCT. 
into the study from this ad and 
96 have successfully enrolled. 
d. ‘Paw-some Dads!’ 
e. ‘Flowers and Laughter’ 
f. ‘Getting a Match while at 
Gym’ 
 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
21 
 
 
This ad for the mLab App trial 
features two YMSM sitting 
outside a coffee shop having 
breakfast, while one is 
attending to their dog, the 
other one is taking a picture of 
their food for Instagram. This 
ad affirms the idea that YMSM 
can in be healthy, happy 
relationships partaking in 
activities that in interest them. 
Our study staff develops 
relatable LGBTQ study 
advertisements so potential 
participants can see 
themselves represented and 
know that this study is looking 
to enroll individuals like them. 
Our study staff has developed 
creatives to represent 
Black/African American YMSM 
and highlight the opportunityfor 
interested volunteers to 
participate in the mLab App 
trial, which has been widely 
advertised across social media 
platforms as a way to 
‘participate in LGBTQ 
research.’ Over the years, our 
study staff has learned that 
potential participants are very 
interested in giving back to the 
LGBTQ community and are 
looking for ways to activelyhelp 
in addressing health issuesthat 
most concern their peers. 
In developing ads for social 
media campaigns, we 
intentionally combine common 
interests of YMSM into one 
impactful image to grab the 
attention of recruits for the 
mLab App trial. The ad below 
depicts a Latino YMSM bench 
pressing at the gym, whose 
phone received a notification 
about a ‘match’ in an LGBTQ 
dating app. Although 
advertising platforms limit 
targeting to age, gender, and 
geographic location, our ads 
showcase real-world behaviors 
that YMSM are participating in 
to find eligible men who are 
having sex with men, that is, 
letting social media users know 
who we are looking for. 
To the most paw-some 
dads we know! 
 
 
3.4.3 
Targeted Online National Campaign Led by Commando LLC 
 
Additionally, we will work with our partner, Commando LLC to develop, launch, and execute a national 
ad campaign. Commando has experience with running successful advertising campaigns and will be 
working with us to develop social media strategy (including creatives and content such as In-Feed 
Videos, Reels, Facebook/Instagram Lives, Stories), providing technical support (landing page 
assistance), and analytics reporting (DMA analysis that provide powerful insights, such as which DMAs 
have higher conversion rates, engagement, and return on ads spends). They will be advertising on top 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
21 
 
 
LGBTQ dating applications (e.g., Grindr, Growlr, Adam4Adam, Scruff, Jackd,) with whom they’ve been 
very successful in reaching YMSM for other studies. 
3.4.4 
Recruitment Limitations, Anticipated Problems and Alternative Solutions Consistent with 
multi-prong approaches designed to reduce recruitment bias, we will employ the 
aforementioned recruitment methods. We will ward off potential recruitment problems by carefully 
monitoring each approach to prevent recruitment problems. This will involve weekly review of recruitment 
data with staff to assess efforts and reports provided by Commando LLC. In this way, the recruitment 
process will be a dynamic process involving actionable, data-driven insights to help us improve targeting, 
segmentation, and personalization to improve the recruitment experience for YTM. 
 
3.5 
Screening Procedures 
 
3.5.1 
Eligibility Screening Procedures 
 
Potential participants will be electronically screened through REDCap(33) for eligibility using the 
full 
screening instrument. If a potential participant is screened and eligible and willing to participate, 
then they will voluntarily provide e-assent/consent. If it is determined during enrollment that a person is 
concurrently enrolled in another HIV prevention study, they will be deemed ineligible. 
 
3.5.2 
Verification of Identity and Initial Screening Criteria 
 
We will use the following procedures to verify the race, sex, and age of participants, and ensure that 
participants are discrete, not duplicated, and to document the country of residence, for example, 
individuals living in the U.S. or its territories): 1) Prior to signing the electronic assent/consent form for 
trial participation, participants are asked to do a confirmatory screening visit via videoconferencing 
during their baseline phone call to confirm race, sex, and age. If participants become ineligible, we let 
them know that, unfortunately, we cannot continue with the visit because eligibility has changed; 2) To 
cross-check age, participants are asked their date of birth; 3) Participants are required to show any form 
of photo ID during the initial video conference so we can verify identity. If a participant does not have a 
government or school issued ID, we will ask them to furnish a report card/transcript with their legal 
name, age, and sex; 4) We will use 2 methods to verify participants’ residence: a) We will ask 
participants to share a form of ID which includes an address, and b) we will cross-check the address on 
the ID with the IP address that we collect through REDCap software. If there is a discrepancy, we will 
review with the study participant to better understand if there is a rationale (e.g., participant is in 
college,). Using procedures previously piloted, we will determine co-enrollment. 
 
3.6 
Informed Assent and Consent 
 
At the enrollment visit we will collect written informed e-consent (18 to 25 year olds; assent for 15 to 17 
year olds) (see Appendix A) for study trial participation, which will detail purpose of trial, study 
procedures, compensation for contributed time, risks, benefits, site contact information, confidentiality 
and voluntary participation. The consent process also details the trial and study compensation. YMSM 
will then be randomized to the intervention arm (MyPEEPS Mobile) or control (delayed intervention) arm. 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
21 
 
 
4.1 
STUDY PROCEDURES 
 
4.2 
Enrollment Procedures 
 
When a participant screens eligible, a study team member will schedule a visit through a 
videoconference call using Zoom. These procedures ensure the integrity and success of the study 
because: 1) We are able to eliminate fraud by verifying participants’ identities via videoconference, fraud 
being a potential problem in online research,(34, 35) and 2) We establish rapport with our study 
participants and have seen very high retention rates related to this rapport building between our staff 
and study participants, which will be augmented by electronic retention strategies in this study. 
 
4.3 
Locator Form/Contact Information 
 
At enrollment, participants will be asked to provide contact information to contact them throughout the 
duration of study for follow-up assessments. This information will be captured in REDCap may be updated 
during follow-up assessment and/or whenever the participant needs to update their contact information on 
file. We will collect each participant’s cell phone number, email address, as well as encourage them to 
share their social media handles (e.g., Snapchat, Instagram, Twitter Facebook, WhatsApp, and/or Skype 
usernames). Participants will be asked if it is okay to mention the name of the project and method(s) of 
study communication preferred when receiving automated reminders (e.g., text messaging, email, phone 
call, leave voicemail). Study staff will not send messages or leave voicemail messages unless expressly 
permitted to do so by the participant. If permission is given to leave voice messages, site staff will assure 
participants that messages left will not include any protected health information or information related to 
study participation. If permission is given to send text messages and/or if study staff is unable to get in 
contact through email, staff will send text messages using an IRB-approved messaging script to contact 
participants throughout the duration of the study. Contact information will be maintained using the same 
confidential data management practices used for all study data. 
 
4.4 
Specific Aim 1 Procedures. 
 
 
 
MyPEEPS was built on considerable formative work with multi-racial and ethnic groups 
(R34MH079707; PI: Garofalo and U01MD011279; MPIs: Schnall and Garofalo). Findings from the 
U01 have been peer-reviewed and published.(27, 36-40) In preparation for this R34, we conducted 
extensive preliminary work to identify the common themes, concerns, and HIV risk and protective 
factors in YTM to expand MyPEEPS and ensure its acceptability for this group, particularly for YTM 
who are multi-racial or come from communities of color. This will be done using the focus group data 
collected from 49 YTM in our preliminary work for this application, and through the feedback by our  
Expert Review Panel and usability testing with YTM. 
Expert Review Panel. The goal of this panel is to reach censuses on common concerns, risk, and 
protective factors for YTM. 
Procedures. The expert and youth advisory panels provided feedback remotely by reviewing the 
curriculum and providing written comments. 
Specific Aim 1: Using qualitative methodology, expert feedback, and usability assessments, 
develop MyPEEPS Mobile for YTM. 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
21 
 
 
Development of mobile delivery technology. Following the expert panel sessions and building on our 
findings from our formative focus groups, we will develop mock-ups of a MyPEEPS Mobile App with our 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
21 
 
 
partners at One Cow Standing, a software development company (see Letter of Support). The mockup 
will have partial functionality of the system and enable us to test the design. MyPEEPS Mobile for YTM 
will be novel, innovative, scientifically sound, scalable, translatable, and likely to have a strong public 
health impact, as detailed in the Innovation section. Dr. Radix will oversee all updates to the content of 
the MyPEEPS Mobile App, with support from MPIs Drs. 
Schnall and Garofalo. The final product of this work will be a Beta version of the app for usability testing. 
Refinement of the MyPEEPS Mobile. Following the development of MyPEEPS Mobile for YTM, we will 
conduct usability testing. Usability Testing. The goal of usability testing is to improve the design and 
increase the likelihood of technology acceptance. We will evaluate the user interface and system 
functions of the MyPEEPS Mobile App and assess whether they are consistent with the end-users’ 
needs. We will conduct two types of usability assessments: A) Heuristic Evaluation and B) End-User 
Usability Testing. 
 
Heuristic Evaluation. Sample: Five informaticians with training in human-computer interaction and who 
have published in the field of informatics will be recruited as usability experts. Nielsen recommends using 
three to five evaluators since one gains little additional information by using larger numbers(41). We will 
recruit them through direct contact with the Informatics Departments at Columbia University and Weill 
Cornell Medical College, both of which have a large cadre of informatics researchers. Dr. Schnall will 
send a message to potential evaluators via email using the MyPEEPS Heuristic Email Recruitment Script. 
Procedures: The heuristic evaluator will be consented using the Heuristic Evaluation Consent form. Once 
consented,the heuristic evaluators will assess a web-based Beta version of the MyPEEPS Mobile App 
remotely via Zoom software. Evaluator screen movements and audio will be recorded via Zoom. 
Identifiable characteristics, such as facial features and names, will not be recorded. Similar to procedures 
that we have used in ourprior work(42-44), 
each evaluator will be asked to evaluate the system using 
the Heuristic Evaluation Checklist and to think-aloud while performing the usability testing(45). 
Participants will be asked to say aloud what they are thinking, seeing, and trying to do while they are 
performing the tasks required for the scenarios. When a user finds errors or the researchers find critical 
incidents that are characterized by comments, silence, or looks of puzzlement, the researcher will record 
the users’ activities. 
Recording the users’ interactions and vocalizations provides additional feedback that can highlight 
problems that would not be identified with static screen shots(46). Each evaluator will walk through the 
use case scenarios (see “MyPEEPS Heuristic UseCaseScenarios Procedure 11.16”) and a study team 
member will navigate through the app per evaluator instructions to complete the assigned tasks. Once 
the walkthrough is complete, the evaluator will fill out the Qualtrics survey titled, “Heuristic Evaluation 
Survey.” After the heuristic evaluation, participants will be asked to rate the prototype’s perceived ease of 
use and perceived potential usefulness using a standardized instrument. Instrument: Nielsen(47) 
proposed a list of ten recommended heuristics for a usable interface design. Each heuristic will be 
evaluated by one or more items, and the overall severity of the identified heuristic violations will be 
rated.(48) Evaluators will also complete the Health Information Technology (IT) Usability Evaluation 
Scale (Health-ITUES) to rate usability. This tool varies from most traditional measurement scales in that it 
is designed to support customization at the item level to match the specific task/expectation and health IT 
system while retaining standardization at the construct level. The Health-ITUES supports evaluation of 
three levels of task/expectation: user-system, user-system-task, and user-system- taskenvironment. 
Schnall (MPI) has published on the usefulness of the Health-ITUES for evaluating the usability of mHealth 
technology(53). Data Analysis: The frequencies of usability issues will be calculated according to the 
heuristic principles adapted from Nielsen’s checklist. 
Mean severity scores will be calculated for each heuristic principle. Evaluators’ comments about usability 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
21 
 
 
problems on the evaluation form and the recording will be grouped and content analyzed according to 
the usability factors of Nielsen’s heuristics.(49) Recordings will be kept until data analysis is complete 
and findings have been disseminated. Heuristic evaluators will be compensated a $150 Amazon gift code 
for their time and will receive the gift code via email using the Heuristic Evaluator / Usability 
Compensation Email Template. 
Usability Testing with YTM. We will conduct usability testing with YTM to identify violations of 
usability principles and any potential obstacles to their effective use and content of intervention 
adaptation of MyPEEPS Mobile with members of our advisory board of transmasculine youth. 
This is an iterative process that involves testing the system and then using the results to change 
it to better meet users' needs. Sample: We will recruit 20 YTM (15-25 years old) who will 
participate remotely in the formative evaluation of the prototype user interface screens. The 
rationale for 20 participants is based on past usability research, which has indicated that 95% of 
usability problems can be identified with 20 users(50). These youth will serve as community 
voices, specifically youth as advisors. We will purposely sample our usability testing participants 
to ensure diversity regarding race, ethnicity, and age. We will be recruiting by reaching out to 
participants in our REDCap database who previously expressed interest in the study or 
participants who are enrolled in another one of our studies. When we reach out to these 
participants in REDCap, we will be using the MyPEEPS Participant Outreach Script. We will also 
be using flyers for recruitment, once approved by the IRB. Participants will be screened for 
eligibility using the Usability Screener in REDCap. Procedures: We are requesting a waiver of  
documentation of consent for online screening procedures. The screening information will only be kept 
electronically in REDCap, which is a secure, CUMC approved system. If participants screen eligible,  
study staff will go through the informed consent process with participants. If participants consent  
to participate, they will complete the Assent/Consent form for Usability Testing. Participants will 
complete a brief demographic background and computer usage survey. 
Participants will use a web-based beta version of the MyPEEPS Mobile with screen movements and 
audio recorded via Zoom software. Identifiable characteristics, such as facial features and names, will 
not be recorded. After the usability evaluation, participants will be asked to rate the prototype’s 
perceived ease of use and perceived potential usefulness using a standardized instrument. Instrument: 
We will measure self-reported ease of use and usability with the Health Information Technology (IT) 
Usability Evaluation Scale (Health-ITUES)(51, 52). Data Analysis: The analysis will be based on the 
recordings of user sessions, transcriptions, notes, and the user surveys, and mean task performance 
time will be calculated. Dr. Schnall will search for critical incidents which will be characterized by 
comments, silence, and repetitive actions. Dr. Schnall will review these incidents in detail using Zoom 
software. The incidents will be identified, and the users’ written comments summarized. Content 
analysis, a technique for making replicative and valid inferences from data, will be performed by the 
research assistant under Schnall’s supervision. The comments will be categorized according to the 
positive characteristics, negative characteristics, and recommendations made by the end- users. Results 
from the standardized surveys will be analyzed using SAS 9.4 (SAS Institute, Cary NC) to calculate the 
descriptive statistics to complement the findings from the usability assessment. Using the findings from 
these activities, we will refine MyPEEPS for use in Aim 2. Recordings will be kept until data analysis is 
complete and findings have been disseminated. Participants will be compensated for their time in the 
form of a $40 Amazon gift code and will receive the gift code via email using the Heuristic Evaluator / 
Usability Compensation Email Template. 
 
4.5 
Specific Aim 2 Procedures. 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
21 
 
 
 
 
Overview. We will conduct a 6-month randomized controlled pilot study with 80 YTM. The goals of the 
pilot study are to: a) Gain direct feedback from participants about whether and to what degree the 
MyPEEPS Mobile App worked as intended; b) Assess the acceptability of MyPEEPS Mobile dosing and 
content for YTM; and c) Observe the flow of procedures, including assessment at baseline, retention 
efforts, data management, and follow-up assessments. The recruitment plan, screening, and 
consent/assent processes are described in the Protection of Human Subjects and Clinical Trials 
Information sections. Our primary outcome for this study will be: change in number of condomless 
receptive anal or vaginal sex acts with sex partners assigned male at birth. Sampling Approach to 
Successfully Enroll YTM in this Project. We will recruit 80 participants from an online national sample 
and ensure that at least half of our sample is from racial/ethnic minority groups. Using active and passive 
recruitment methods, the on-site project coordinators will oversee and participate in recruitment efforts. 
The study staff hired for this project, on-site coordinator, and the peer-to-peer networking efforts will be 
used to recruit study participants. Our pilot study will comprise a convenience sample through active 
online recruitment through Instagram, Twitter, Snapchat, and Facebook advertisements. We will use 
psychographic targeting (in-depth, publicly available consumer data such as interests, city) to advertise 
only to potentially eligible individuals. We have successfully used this approach to recruit our sample of 
YMSM in the MyPEEPS project. Through our advertisements, and as previously mentioned, we have 
screened 
>800 YTM who would be eligible for the proposed MyPEEPS study, and we will plan to recruit them as 
well as others into the proposed study. We are confident in our ability to enroll the proposed target 
sample. 
Sample Size and Power Calculation. This study plan is to enroll 80 participants with 1:1 random 
assignment to the intervention arm and the control arm (i.e., 40 in each arm). Compared to a large- 
scale randomized trial, we do not expect that this pilot trial will have power to detect many of the 
 
effects that would be of scientific or have policy significance. This is consistent with the aims of this 
pilot grant mechanism, where budgetary limitations necessitate small sample sizes. Group means 
on continuous variables typically begin to stabilize by 20-30 subjects. Thus, a sample size of 40 
participants per group should provide relatively stable group means for the intervention and control 
conditions, even with some sample attrition. 
Potential Problems/Alternative Solutions Regarding Recruitment. Consistent with the multi- 
prong recruitment approach designed to reduce recruitment bias(54, 55), our team will carefully 
monitor each approach to prevent recruitment problems. This will involve weekly review of 
recruitment data with staff to assess efforts. In this way, recruitment will be a dynamic process and 
will reach the proposed diverse group of YTM. 
Eligibility Screening and Informed Assent/Consent. See Protection of Human Subjects.  
Study Enrollment. A baseline visit will be conducted involving a behavioral assessment and will 
last approximately 1.5-2 hours. YTM will then be randomized to the intervention or delayed 
intervention arm. An enrollment project will be created in REDCap to collect and manage data for 
enrolled participants. The longitudinal study module, automatic survey invitations, and secure file 
upload capabilities can be used in conjunction with customizable reports to ensure streamlined 
operations and efficient data management. Eligible volunteers will be scheduled for a one-time 
Specific Aim 2: Conduct a pilot randomized controlled trial to examine the feasibility, 
acceptability, and preliminary efficacy of the MyPEEPS Mobile App for YTM in a sample of 80 
YTM (15-25 years) and refine the study methods for a future efficacy trial. 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
21 
 
 
face-to-face video call (e.g., Zoom), with a study team member to enroll. The enrollment visit is 
conducted via video to validate identification and age (view identification documents) and build 
rapport to support retention over time. Study staff will create a record for the volunteer, and 
REDCap will send the volunteer a customized interview assessment via e-mail beginning with the 
consent/assent process with electronic signature capability. The longitudinal module of REDCap 
will automate follow-up data collection by generating a unique study visit schedule for every 
participant, anchored to their enrollment date. Paired with automated survey invitations, 
participants will be automatically invited via e-mail to complete follow-up surveys based on their 
unique study visit schedule. These invitations include up to 5 automated reminders sent in pre- 
defined intervals. Staff monitor the completion of follow-up survey data at 3 and 6 months, which 
include the primary and secondary outcome assessments, via the reports, which query the 
completion status in real time. 
Table 3. Overview of MyPEEPS Content and Activity Type 
Module 
Number/Name/Screenshot 
from App Module 
Activity 
Number 
& Name 
Activity Content 
Activity 
Type 
Module 1. Introduction 
1. Welcome 
to 
MyPEEPS 
User inputs name, telephone number, e- 
mail address, and how they prefer to get 
notifications. 
Questions 
2. BottomLine User is asked the furthest they will go with 
a one-time hookup in several sexual 
scenarios. 
Multiple 
Choice 
3. Underwear 
Personality 
Quiz 
User completes personality quiz and is 
introduced to characters in the app. 
Multiple 
Choice 
4. My Bulls-I 
User is asked to input the ways they refer 
to their body parts after seeing an 
example from one of the app characters. 
Open- 
Ended 
Questions 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
22 
 
 
Module 2. Serious Talk 
 
5. P’s On- 
Again, Off- 
Again 
BottomLine 
Video animation of text messaging 
conversation between two characters, P 
and Nico, about P’s new relationship that 
has led him to ignore his BottomLine. The 
user is asked to complete questions about 
why P should be concerned about his 
BottomLine. 
Video and 
Multiple 
Choice 
6. Sexy 
Settings 
User is presented several settings in which 
sex takes place and potential threats to 
BottomLine and asked to match each 
setting to correct threat. 
Matching 
7. Going 
Downhill 
Fast 
User is presented with information about 
effects of alcohol and common illicit or 
misused drugs. Resources for additional 
information on each substance are 
provided via external web links. After 
reading through information, users match 
substances to potential threats. 
Matching 
8. Move Up, 
Move 
Back 
User is asked series of questions related to their own life experience. 
User is introduced to personal identities and characteristics that may 
place them at a societal advantage or disadvantage, termed “VIP 
(privileged)/General admission (non-privileged)” status 
Yes/No 
Questions 
9. HIV 
True/False 
User completes a series of True/False questions related to HIV, with 
detailed fact-based information provided for each response. 
True/False 
10. Jeopar-T 
User completes a series of True/False questions related to 
hormones, reproductive health, and surgeries to win points. 
True/False 
11. Checking 
In On Your 
BottomLine 
User is given opportunity to review and make changes to their 
BottomLine, taking into consideration any information they learned from 
completed activities. 
Multiple 
Choice 
Module 3. Safety First 
12. P 
Learns 
About Safer 
Sex 
User is presented with a scenario about P 
trying to make his way to clinic for HIV 
testing on public transportation. P 
experiences difficulties and rude behavior 
on the bus and user is presented with 
recommendations for managing anger and 
frustration. 
Video 
13. Tackling 
Testing 
User watches a video animation about 
what to expect during Tommy’s first 
experience being tested for HIV and P’s 
experience seeking testing and 
reproductive healthcare. Video  
presents clinic scenario and discussion 
with a health provider. 
Video 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
23 
 
 
14. 
Spread 
Out 
User completes an activity matching a 
given sexual act with its corresponding 
level of risk (no risk, low, medium, high), to 
apply lessons learned in prior activities 
specific to HIV/STI transmission risk. 
Matching 
15. Steps to 
User is presented with 12 steps for 
effective condom use and must correctly 
Ordering 
Question 

23 
Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
 
 
 
 
 
ching 
 
 
ultiple 
hoice 
Effective 
Condom Use 
order the steps by selecting them 
sequentially from list of all steps. 
16. Safer Injection 
User is given the opportunity to review the safe needle injection  Mat 
steps. User then completes an activity matching 
safer injection steps to images. 
17. Checking In On 
Your Bottom Line 
Again 
User is again given the opportunity to review and make 
     M 
changes to their BottomLine, taking into consideration any C 
information learned in prior activities. 
Module 4. Making 
Tough Situations 
Safer 
18. 
Disclosure 
and Safety 
User is presented with emotional coping 
Multiple 
strategies to manage disclosure and 
Choice 
responses from potential partners while 
dating. 
19. Red 
Flag, Green 
Flag 
User is presented with dating app profiles 
 Yes/No 
and learn about online safety as they 
Questions 
decide whether or not they would engage 
the profile further. 
20. Peep in 
Love 
User is presented scene where P is 
Multiple 
confronted with a “swirl of emotions” 
Choice 
related to a sexual encounter and 
presented with management techniques to 
stick to their BottomLine. 
21.Healthy 
Relationshi 
ps 
User is presented with a dating scenario 
between two characters. They are asked to 
identify characteristics of the relationship as 
healthy or unhealthy. 
22. 4 Ways to 
Manage 
Stigma 
User is presented with four different 
Matching 
strategies to manage stigma. They are 
then asked to match those strategies to 
scenarios presented as comic panels. 
23. Get a 
Clue! 
User is presented with a “slot machine” 
Multiple 
activity in which combinations of feelings, Choice partner 
characteristics, and settings are 
presented, and they are asked what sexual 
decision they would make in each 
scenario, keeping their BottomLine and 
communication strategies in mind. 
24. Last Time 
Checking In 
On Your 
Bottom 
Line 
User is given final opportunity to review 
Multiple 
and make changes to their BottomLine, 
Choice 
taking into consideration any 
information learned from completing 
prior activities 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
25 
 
 
25. 
BottomLine 
Overview 
User is presented with chronology of how 
their BottomLine changed 
throughout the app and encouraged to 
continue to stick to their goals for sexual 
safety. 
Summary 
Randomization to Intervention and Control (Delayed Intervention) Arms. Using the 
randomization module in REDCap and the approach described below, there will be 
random or minimally biased assignment of subjects to study arms. We will use block 
randomization with randomly permutated blocks(56) to reduce opportunities for 
selection bias. The advantage of the permutated block design is that treatment 
assignment is pre-determined before the trial begins and then assignment remains 
static throughout the course of the trial(57). Blinding and random assignment will be 
maintained through continuous supervision by key members of the research team. All 
staff and participants will be kept blinded to outcome measurements during data 
collection(58). This technique will maintain complete randomness of the assignment of 
a subject to a group. Participants will be randomized based on the use of a password 
protected computer- generated random numbers at baseline to avoid the possibility of 
study staff subverting randomization as has been noted in previous studies(59). 
Following completion of the informed assent/consent and baseline assessment, 
participants will be randomly assigned to one of the two trial arms using the 
randomization module in REDCap(57). This willminimize the biased assignment of 
study subjects(60). 
Intervention Delivery. If the study participant is randomized to the intervention arm, they will have 
access to the MyPEEPS Mobile App for the next three months. Participants randomized to the 
control (delayed intervention) arm will be given access to MyPEEPS Mobile at the 3-month study 
visit. Through our dashboard, we can monitor when participants login to MyPEEPS Mobile, time 
spent on each activity, and whether they are actively using the intervention or just logged in. 
Participants will have 3 months to complete the 4 MyPEEPS Mobile modules, which includes 25 
embedded mobile activities (See Table 3). 
Participants in the intervention group will be able to access the MyPEEPS intervention from 
baseline until the end of month 3 and the control arm (delayed intervention) will have access to the 
MyPEEPS App from months 4-6 (see Figure 2). Overview of data collection time points. We  
will conduct simultaneous assessments for both study arms at baseline, 3 months, and 6 months. 
The delayed intervention design is essentially a 
cross-over design in which data are collected in 
parallel from both study arms through the primary 
endpoint - 3-months post-randomization (Figure 2). 
The arms are then crossed over, which has multiple 
benefits, including: 1) for ethical reasons per our 
Figure 2. Assessment Time Points and Intervention Delivery for 
Each Study Arm 
experience with HIV prevention interventions with 
transgender populations, the delayed intervention 
group receives the potential benefit of the 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
26 
 
 
intervention at the point of cross-over; 2) at the 6-month follow-up, it provides for a test of the 
durability of the intervention effect in the intervention group (i.e., 3-months post-intervention, pre- 
post); 3) it provides an additional secondary test of the intervention effect in the delayed 
intervention arm (pre-post); and 4) it provides additional time points (and power) in the intervention 
group to test moderators of the trajectory of sexual risk over time. We successfully used this study 
design in an NIH-funded R34 study(61). 
MyPEEPS avatars represent “regular guys” like the study participants who will walk the participants 
through the curriculum. Following the selection of an avatar, the study participant will navigate 
through each of the study activities. Participants do not need to complete an activity in a single 
sitting but can stop and come back to complete the activities at any time during the time frame 
before the 3-month follow-up. Study staff will have access to a dashboard to monitor module 
completion. In addition, participants will receive weekly text messages and/or e-mails (participants 
will select) to remind them to complete the MyPEEPS module. If 2 weeks have passed and a 
participant has not completed any modules, then the app will send out notification reminders to 
remind them to complete their modules. 
 
4.6 
Specific Aim 3 Procedures. 
 
Overview. Post-intervention, we will conduct individual, in-depth, semi-structured interviews with 
participants who completed the intervention. 
Sample. We will purposively select participants to represent a diverse set of experiences (e.g., 
participants who completed App Use, high vs. low sexual activity). Although we will continue to 
recruit participants until reaching thematic saturation, we anticipate conducting about 20 
interviews 
Procedures. The in-depth interviews will be 45-60 minutes in length. Following completion of the 
informed consent process, all interviews will be audio-recorded. The interview guide will be 
informed by the predisposing, reinforcing, and Enabling Constructs in Evaluation (PRECEDE) 
portion of the PRECEDE-PROCEED Model of health program planning and evaluation(62). In 
addition to the evaluation of public health programs, this framework has been applied to the 
evaluation of health information technology and has been proposed by several authors as a 
strategy for assessing predisposing, enabling, and 
reinforcing factors for use and acceptance of health 
information technology tools(63-65). Structured 
interview questions are listed in Table 4. 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
27 
 
 
Ensuring Rigor. The team will adhere to qualitative research processes to ensure the credibility, 
confirmability, dependability, and transferability of the qualitative data from these analyses.(66) 
To support the credibility of the data, we will conduct peer debriefing and triangulate findings 
across multiple data sources (surveys, interview data). In addition, we will use “member checks,” 
i.e., sharing of initial data interpretations with participants to ensure accurate interpretations. 
Triangulation of findings, along with reflexivity, will enhance the confirmability of the 
interpretations. The investigators will carefully record an audit trail and keep extensive field notes 
to facilitate transferability of study findings into other contexts. Table 
Data Analysis. All in-depth interviews will be transcribed verbatim and then coded. Recordings 
will 
be kept until data analysis is complete and findings have been disseminated. The 
development and application of a coding scheme is an integral component of the data analysis 
process. It enables the systematic examination and interpretation of the data related to the primary 
analytic foci. The coding scheme is conceptualized as a multilevel structure. At the highest level are 
the primary analytic foci coded as headings. Specific aspects or dimensions of the headings are 
assigned core codes. Specific aspects or dimensions of the core codes are assigned sub-codes. 
We will use NVivo™ (QSR International, Victoria, Australia), a software program for qualitative 
analysis, to facilitate the analysis. The following 7 steps will be used to develop the coding scheme: 
Step 1: Identify the principal issues discussed by participants; Step 2: Construct definitions of the 
primary analytic themes; Step 3: Develop and apply core codes and sub-codes to the initial set of 
interviews; Step 4: Develop a provisional coding scheme; Step 5: Test and refine the provisional 
coding scheme; Step 6: Reconcile coding differences and construct an updated and final coding 
scheme; Step 7: Apply the coding scheme to the full data set and assess inter-coder reliability. 
After all transcripts have been coded, we will extract and examine the content of text segments 
linked to core codes and sub- codes relevant to understanding barriers and facilitators to the use of 
the MyPEEPS App. Based on the coded data, we will propose ways in which certain themes are 
analytically related. A careful examination of the coded text will reveal the associations among 
these themes and may lead to more refined data searches. Once we establish patterns of 
relationships among themes and issues, we will identify participants’ accounts that support or refute 
these patterns. Identifying and accounting for cases that deviate from an interpretative pattern 
enables us to test and confirm the pattern’s validity and robustness. 
Triangulation of Findings. We will examine changes in HIV risk behaviors and its relationship to 
factors including but not limited to changes in substance use, alcohol use, stigma, victimization. 
We will triangulate the findings from our quantitative outcome measures (Aim 2) with the findings 
from our qualitative data (Aim 3) to better understand participants’ decisions for their HIV risk 
behaviors. 
Anticipated Problems and Alternative Study Design Considerations. The study team thought 
very carefully about alternative design considerations. (1) We carefully considered a more 
traditional RCT design and control group. However, in prior HIV prevention interventions projects 
targeting transgender populations we received considerable negative pushback from the 
community regarding this approach(67). (2) We carefully considered less stringent sexual risk 
inclusion criteria, but firmly believe that we can recruit the proposed sample using condomless 
sex as our outcome variable. (3) If needed for recruitment to maximize participants who report 
condomless sex, we can expand the age range to 25 years based on age range of the original 
MyPEEPS trial. 
Summary. The final product of this study will be the basis for an R01 application to conduct a 
large-scale efficacy study for this population. The proposed MyPEEPS Mobile intervention for 
YTM is a novel and evidence-driven intervention using mobile technology to deliver HIV 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
 
prevention information specifically developed for YTM. Consequently, we are confident that 
proposed intervention will be of high impact intervention for improving HIV prevention behaviors 
in YTM and have long-term implications for overall improvement in the public’s health. 
 
5.1 DATA COLLECTION AND SITE MONITORING 
 
5.2 
Development of Protocol and Case Report Forms 
 
The Protocol Team in collaboration is responsible for the development of this protocol as well as 
the Case Report Forms (CRFs) needed to collect the information required to implement this 
protocol. 
 
5.3 
Data Records 
 
Participant-related study information will be identified through a Participant ID (PID) on all 
participant CRF and Computer Assisted Self Interviewing (CASI) files. Participant names or other 
personally-identifying information will not be used on any study documents and will be redacted 
from interview transcripts. Participant names and their PID will be stored separately from other 
study information in REDCap or secure University servers, accessible only to designated study 
staff, site monitors, and representatives from the NIH. PIDs will not be entered into the 
MyPEEPS Mobile app; instead, a unique username will be assigned to each participant when 
they create an account for the MyPEEPS Mobile app. These unique usernames will be provided 
by the study team. Original source documents for individual participants will be maintained at 
the respective study site and will be accessible only to the study staff. Data from original source 
documents will be transcribed on CRFs as applicable. 
 
5.4 
Data Collection 
 
 
5.4.1 
CASI Data 
 
Data collected using a web-based CASI method will be on a portable computer, tablet, or mobile 
phone (what the participant chooses) via an internet-based application. All survey data will be 
collected using REDCap. Data will remain confidential. The participant’s unique PID # will be 
used in order to link the responses to the participant. 
 
5.4.2 
Zoom Platform Description 
 
Zoom will be used to schedule videoconference with participants remotely for screening and 
enrollment procedures. Zoom is compatible on PCs, tablets, and smartphones; as well as 
maintains the option to conduct an audio conference without the video component. Zoom is 
responsible for keeping all patient information secure and report any breaches of protected 
health information (PHI). 
 
End-to-end encryption. Zoom encrypts all presentation content at the application layer using the 
Advanced Encryption Standard (AES) 256-bit algorithm. Zoom end-to-end (E2E) chat encryption 
28 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
 
allows for a secured communication where only the intended recipient can read the secured 
message. Zoom uses public and private keys to encrypt the chat session with Advance 
Encryption Standard (AES256), and session keys are generated with device unique hardware ID 
to avoid data being read from other devices. This ensures that the session cannot be 
eavesdropped or tampered with. 
 
Cloud Control Infrastructure. A distributed network of low-latency multimedia routers (software) 
resides on Zoom’s communications infrastructure. With these low-latency multimedia routers, all 
session data originating from the host’s device and arriving at the participants’ devices is 
dynamically switched — never stored persistently through the Zoom communications 
infrastructure. Zoom’s communications infrastructure for real-time video, audio, and data 
communications resides on Zoom dedicated servers, which are housed in SSAE 16 SOC2 
compliant datacenters on opposite sides of the US. Zoom sessions are completely temporary 
and operate analogously to the popular mobile conversation over the public mobile network. In 
addition to unique security benefits, Zoom’s communications infrastructure also enables an 
extremely scalable and highly available meeting infrastructure unrestricted by the limitations of 
physical data centers. 
 
The Zoom client communicates with the multimedia router to establish a reliable and secure 
connection. At the time of instantiation, the Zoom client will determine the best method for 
communication, attempting to connect automatically using udp and tcp port 8801, 8802 and 8804 
or HTTPS (port 443/TLS). 
 
5.4 
Data Submission 
 
 
5.4.1 
CASI Data Transmission 
 
Only authorized users will be able to access and open the survey through REDCap. To ensure 
data privacy, as soon as data is entered (in real-time), it will be encrypted during transmission to 
REDCap. The data will routinely be downloaded and stored in a secure database on University 
servers. 
 
5.4.2 
Retention Data 
 
The study will use a Health Insurance Portability and Accountability Act (HIPAA)-compliant web- 
based platform entitled REDCap, which aids studies with various aspects of participant 
recruitment, study implementation, and retention. The application has the ability to securely 
manage participant information across multiple studies and customers simultaneously, stratifying 
participant information by study and site. REDCap provides study staff the ability to conduct data 
entry for longitudinal participant tracking from screening to study completion and to use secure 
messaging, study calendar management, secure photo uploads. The ability to designate specific 
roles with certain levels of access to every REDCap user allows for greater control around 
permissions and accessibility to participant information. Users can even be limited to a reporting 
only role, which allows for study oversight through real time aggregate reporting, but no access 
to PHI. 
 
29 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
30 
 
 
 
5.5 
Data Quality Control and Quality Assurance 
 
Investigators receiving federal funding must adhere to the Code of Federal Regulations (CFR) to 
protect research participants and produce reliable study information. The study coordinators will 
monitor data entry and will have an internal quality assurance plan that will identify problems and 
correct errors in research study records. 
 
5.6 
Study Site Monitoring and Record Availability 
 
Site monitors under contract to the National Institute of Allergy and Infectious Diseases (NIAID) will 
visit participating clinical research sites to review participants records, including consent forms, 
CRFs, medical records (e.g., physicians’ progress notes, nurses’ notes, individuals’ hospital charts), 
and laboratory records to ensure protection of study participants, compliance with the IRB approved 
protocol/amendments, and accuracy and completeness of records. The monitors will inspect sites’ 
regulatory files to ensure that local regulatory requirements, in addition to U.S. Federal regulations, 
are being followed.  They will also inspect  sites’ pharmacies to review product storage and 
management. 
 
6.1 PARTICIPANT TRACKING & CLINICAL MANAGEMNT 
 
6.2 
Retention Efforts 
 
Study staff will track retention, which will be reviewed weekly by the PIs. As has been successful 
in previous studies with this population, including the MyPEEPS Mobile efficacy trial, we will: 1) 
obtain, at baseline, participants’ email addresses, phone numbers, social media handles, as well 
as the addresses and contact information of people participants believe could help us locate them 
in the future; 2) allow for any time of day, and all days of the week, for participants to complete 
online assessment visits; 3) provide participants with numerous methods for contacting study 
staff, including providing study phone numbers, email addresses, and social media handles and 
if agreed to by the participant, having them program our site numbers into their mobile phone 
during the enrollment visit; 4) routinely verify participants’ locator information at every contact 
point; 5) if correspondence is returned indicating a participant no longer has a given email 
address, study staff will telephone to obtain their new email address; 6) if the phone number is 
disconnected, we will contact them via their social media accounts and/or individuals listed in 
participants’ records; and 7) if participants are not responsive after 5 email correspondence 
attempts and/or have given permission to message them via phone number, we will attempt to 
make contact using participants’ phone numbers using IRB-approved scripts. We will make every 
effort to maintain contact with participants, even if they move or discontinue the intervention, for 
those still willing to complete follow-up assessment visits. Given the older study population 
proposed in this trial, we are confident we will retain at  least 80% of the study population for this 
study proposed in this trial, we are confident we will retain at least 80% of the study population 
for this study. 
 
6.3 
Intervening on “Social Harm” 
 
All sites have specific policies governing the treatment of human participants. These policies 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
31 
 
 
specify that medical and psychological assistance will be available in the immediate environment 
in the event a participant should experience any adverse reactions resulting from study 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
32 
 
 
procedures. 
 
While participants will be informed that they may refuse to answer any question at any time, 
responses or reactions to certain questions may indicate distress on the part of the participants. If 
at any time during the study, a participant divulges that he is at risk for harm, including but not 
limited to being abused or experiencing violence, if harm is suspected or likely, or if the 
participant states he is suicidal/homicidal, measures will be taken to ensure his or her safety. 
Reporting will be done as appropriate to the situation and the legal statutes, including reporting 
to child protection agencies or other appropriate agencies and referrals will be provided to 
appropriate support, counseling, or treatment resources. 
 
 
6.4 
Criteria for Permanent Intervention Discontinuation for an Individual Participant 
 
The criteria for permanent discontinuation of further study intervention for an individual 
participant are: 
• Study product/intervention-related adverse event 
• Reaching a defined clinical endpoint 
• Completion of study intervention as defined in the protocol 
• Request by participant to terminate study intervention 
• Clinical conditions, which in the best judgment of the investigator are believed to be harmful 
or potentially life-threatening to the participant, even if not addressed in the AE Management 
section of the protocol 
 
Participants may end their participation in the study at any time. No further data collection will 
occur from the date the decision is made to permanently discontinue the subject from the study. 
Participants who experience distress during the survey can access our list of community 
referrals, which can be viewed on our study's website, or contact the research staff using the 
information provided to the participant within the consent/assent process. Any unexpected 
adverse events will be immediately reported to the Columbia University IRB, if applicable. All 
study activities will halt pending Columbia University IRB review and recommendations if 
necessary. If a participant withdraws or is removed from the study, the site study coordinator 
will complete the necessary documentation regarding the participants off study status. 
 
 
6.5 
Criteria for Premature Study Discontinuation for an Individual Participant 
 
The criteria for premature discontinuation from the study for an individual participant are: 
• Lost to follow up as evidenced by failure by the participant to attend specified 
consecutive clinic visits, at the discretion of the site investigator; 
• Participant repeatedly non-compliant with study intervention as prescribed; 
• Request by participant to withdraw assent/consent; 
• The study is cancelled by the NIH; 
• The study is cancelled for other administrative reasons; 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
33 
 
 
• The participant becomes incarcerated or placed in detention during thestudy; 
• The participant reports an HIV positive diagnoses; 
• Participant judged by the investigator to be at significant risk of failing to comply with the 
provisions of the protocol as to cause harm to self or seriously interfere with the validity of 
study results; or 
• Death of the participant 
 
7.1 
DATA MANGEMENT, SECURITY & ANALYSIS 
 
7.2 
Power Considerations and Sample Size Calculation (Aim2) 
 
Sample Size and Power Calculation. The study plan is to enroll 80 participants with 1:1 random 
assignment to the intervention arm and the control arm (i.e., 40 in each arm). Compared to a 
large- scale randomized trial, we do not expect that this pilot trial will have power to detect many 
of the effects that would be of scientific or policy significance. This is consistent with the aims of 
this pilot grant mechanism, where budgetary limitations necessitate small sample sizes. Group 
means on continuous variables typically begin to stabilize by 20-30 participants. Thus, a sample 
size of 40 participants per group should provide relatively stable group means for the intervention 
and control conditions, even with some sample attrition. 
 
Study Design. Participants in the intervention group will be able to access the MyPEEPS 
intervention from baseline until the end of month 3, and the control arm (delayed intervention) 
will have access to the MyPEEPS App from months 4-6. 
 
Overview of Data Collection Time Points. We will conduct simultaneous assessments for both 
study arms at baseline, 3 months, and 6 months. We will also collect an immediate post- 
intervention assessment for the delayed intervention arm. The delayed intervention design is 
essentially a cross-over design in which data are collected in parallel from both study arms 
through the primary endpoint - 3 months post-randomization. The arms are then crossed over, 
which has multiple benefits, including: 1) for ethical reasons per our experience with transgender 
populations, the delayed intervention group receives the potential benefit of the intervention at 
the point of cross-over; 2) at the 6-month follow-up, it provides for a test of the durability of the 
intervention effect in the intervention group (i.e., 3 months post-intervention, pre-post); 3) it 
provides an additional secondary test of the intervention effect in the delayed intervention arm 
(pre-post); and 
4) it provides additional time points (and power) in the intervention group, to test moderators of 
the trajectory of sexual risk over time. 
 
7.3 
Data and Safety Monitoring 
 
 
7.3.1 
Training on Human Subjects and Data Safety and Monitoring. 
 
All proposed staff have participated in the NIH required trainings for conduct of studies that 
involve human subjects and any future study staff will do so upon hiring. Training for all staff 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
 
includes (but is not limited to) Protection of Human Subjects, Informed Consent, Good Clinical 
Practice, Quality Management, Confidentiality, and Reporting of Adverse Events, REDCap data 
entry and management. If any study staff discovers any untreated condition (e.g., onset of 
physical or mental health condition), they will refer participants to appropriate treatment 
immediately. 
 
7.3.2 
Data Management and Data Quality 
 
Columbia University will be responsible for computerized survey programming, data capture, 
management, and analysis. All study information will be identified through the Participant 
Identification Number (PID) on all forms and computerized files. Data files will be exported from 
the ACASI program and imported into the SPSS database for storage. Computer data files never 
have any identifying information and are encrypted prior to transfer between study sites. Data 
entered into the SPSS database will not include any identifying information. Only authorized 
users with a login name and password will be able to open the computerized survey, and only 
those with administrative privileges will be able to access data. The study research assistant 
(RA) will use a login name and password to gain access to the software to administer it to a 
participant, but they will have no ability to access the saved data. The database, data structure, 
and data quality will be routinely reviewed by Dr. Schnall, and the Data Manager will set up the 
database, check data quality, and prepare the data for use. The Multiple PIs will work closely to 
plan analyses for various purposes, including for hypothesis testing and manuscript development 
and reports. Data quality will be examined before statistical analyses are conducted, including 
examination of missing data, 
assessment of distributional assumptions, and identification of outliers. In addition to data quality, 
the comparability between intervention and control groups will be carefully examined, including 
baseline balance and differential attritions at all waves of follow-up. While differences are not 
expected from the randomized design, it is prudent to plan for this contingency so that sufficient 
follow-up data and appropriate statistical methods can be used for intent-to-treat analyses. The 
psychometric properties of instruments will also be examined, as will the patterns of missing 
data. 
 
7.3.3 
Data Monitoring 
 
For the study, weekly reports for the sites will be created by the data manager to review relevant 
app engagement data, barriers with recruitment/enrollment and retention, urine sample 
collection, compliance with the protocol, and accuracy and completeness of the records. The 
investigative team will hold reoccurring conference calls, and these reports will be briefly 
reviewed by the team at these meetings. These regular reviews will ensure close 
communication between the research assistants, quickly identify missing data points, and 
ensure consistent management of any issues with the protocol between sites. 
 
Data quality will be examined before statistical analyses are conducted, including examination of 
missing data, assessment of distributional assumptions, and identification of outliers. In addition 
to data quality, the comparability between intervention and control groups will be carefully 
examined, including baseline balance and differential attritions at all waves of follow-up. While 
differences are not expected from the randomized design, it is prudent to plan for this 
 
33 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
 
contingency so that sufficient follow-up data and appropriate statistical methods can be used for 
intent-to-treat analyses. 
 
7.3.4 
Data Safety and Monitoring Board 
 
The contact Principal Investigator (PI) will enlist a Data and Safety Monitoring Board (DSMB) to 
oversee the data, safety elements, and overall integrity of the study. The DSMB will: 1) monitor 
recruitment, enrollment, and adherence of study participants; 2) formulate criteria for modifying or 
discontinuing the intervention of individual subjects; and 3) review serious adverse events (SAEs). 
The objectives of the DSMB will be to assess the safety of the intervention trial and to assure the 
highest degree of subject safety. We will work with the NIMH to identify individuals to serve in this 
role. Monitoring board membership will be reviewed and approved by NIMH. Should there be any 
questions regarding the independence of the monitoring board it will be addressed and corrected 
if necessary. 
 
The biostatistician with clinical trials expertise and the data manager will prepare monitoring 
reports in advance of every DSMB meeting. The DSMB will remain blinded to randomization 
status unless, for safety reasons, the DSMB, in consultation with the NIMH Program 
Administrator, decides it is important to unmask the data. 
 
The DSMB will: 1) review the protocol as funded and make suggestions for any changes 
(especially safety related); 2) assess the endpoints suggested by the investigators; 3) review 
study progress by reviewing recruitment, retention, and compliance of participants and data 
quality; 4) determine formatting for data reports; 5) review endpoints for safety; 6) submit written 
reports and 
suggestions to the team; and 7) add to or modify this list of objectives. The Project Director will 
document the minutes of the meetings and distribute to the Chair of the DSMB for final approval 
or revision. 
 
7.4 
Adverse events 
 
7.4.1 
Adverse event assessment 
 
We anticipate that the Data Safety and Monitoring Board will define study-specific serious adverse 
events (SAE)s. While we do not anticipate any SAEs, we will suggest looking for imbalance among 
treatment arms with regard to the following criteria: safety and data security and study drop-out 
 
7.4.2 
Adverse event reporting 
 
We will follow the guidelines that require investigators to promptly notify the IRB (within days of 
the occurrence) when SAEs occur. SAEs will be defined as death, life threatening illness, 
hospitalization or prolongation of hospitalization, and persistent/significant disability. The IRB 
requires that any SAE that is unexpected and related or possibly related to the research 
intervention must be reported. SAEs that are unrelated to the research intervention do not have 
to be reported to the IRB (however, we will report these to the monitoring entity and NIH). Risks 
that are described in the protocol and consent form do not have to be reported as SAEs unless 
34 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
 
the expected SAE occurs more frequently or is more serious than expected. One exception to 
this rule is in the case of a death. All deaths must be reported, whether or not the death was 
related to the research. 
 
7.5 
Data Security and Privacy 
 
Beginning with the development process and throughout the research project, we will follow the 
privacy and security principles set forth at healthhit.gov. Our team is familiar with the importance 
of the privacy and security of personal health information to engender individual trust in the use 
of health IT applications. We have expertise and experience in this domain as we have 
developed a number of health IT systems funded through NIH and the Agency for Healthcare 
Research and Quality for persons living with HIV whose personal health information is usually 
held to higher security standards than traditional patients, as HIV has historically been a 
stigmatized disease. 
We built the MyPEEPS App (NIMHD U01 MD011279) which is housed on the CUIMC IT 
servers. The CUIMC servers are located in a secure datacenter, with necessary redundancies. 
Currently the network can be accessed remotely via Virtual Private Network with a Citrix 
solution being developed. All servers have HIPAA compliant security. 
 
CUIMC has an Information Security Office (ISO) that facilitates all aspects of information security 
risk management at CUIMC, with a particular focus on threat management and HIPAA 
compliance. This includes administration and enforcement of information security policies on 
campus. ISO also provides guidance to CUIMC schools and departments regarding any 
information security concerns they may have. The ISO collaborates with the entire CUIMC 
community to protect the confidentiality, integrity, and availability of critical information and 
computer resources. The ISO strives to implement secure computing infrastructure and 
practices with sensitivity to CUIMC's educational and research environment. Columbia 
University has an information security charter which is the foundation of all of the work carried 
out by Dr. Schnall and her research team. In specific, Dr. Schnall will work with the CUIMC IT 
server group and the information security office to protect the confidentiality, integrity, and 
availability of participants’ data. Confidentiality means that information is only accessible to 
authorized users. Integrity means safeguarding the accuracy and completeness of data and 
processing methods. Availability means ensuring that authorized users, such as research 
participants, have access to data and associated information resources when required. 
 
Prior to assent and consent, study participants will be informed as to what data the 
applications will collect. Data will be encrypted and stored securely on the CUIMC IT 
servers. As a starting point for ensuring privacy and security, all smartphones will be 
password–protected. In addition, there will be an additional password for the App so that 
only study subjects will be able to open the App. 
 
7.5.1 
Procedures to Ensure Compliance with Monitoring Plan and ReportingRequirements 
 
Biweekly reports for the sites will be created by the study statistician to review relevant app 
engagement data, barriers with recruitment/enrollment and retention, laboratory and medical 
records, compliance with the protocol, and accuracy and completeness of the records. The 
 
35 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
36 
 
 
investigative team will schedule biweekly conference calls, and these reports will be briefly 
reviewed by the team. These regular reviews will ensure close communication between the 
research assistants, quickly identify missing data points, and ensure consistent management 
of any issues with the protocol between sites. 
 
7.6 
Data Analysis Plan for Aim 2 
 
Assessment of feasibility and acceptability of the intervention (Table 5). We have 
specified the following measures of feasibility and acceptability: Retention Rates, 
Compliance Rates, Dose, Eligibility Criteria, Recruitment and Enrollment, Missing Data, 
Study Measures(68-70). 
Table 5. Measures of Feasibility and Acceptability of MyPEEPS Mobile for YTM 
Constructs 
Measures 
Threshold (if applicable) 
Retention 
Rates 
-How many enrollees remain in the study? 
-Are they about the same in both arms? 
At least 80% will remain in the study. 
Compliance 
Rates 
What is the completion rate of app 
sessions by participants? 
At least 80% will have completed at 
least 75% of the intervention content. 
Dose 
Is the dose of the intervention e.g., 
number of sessions adequate 
(assessment of paradata). 
Does the dose need to be adjusted? 
Eligibility 
Criteria 
Are eligibility criteria acceptable, or do 
they need modification because they are 
too stringent? 
Eligibility rates (number of individuals 
eligible among those approached) 
Recruitment 
and 
Enrollment 
What is the pace of recruitment? Barriers 
to recruitment and enrollment 
• Rates of refusal (numberof refusals 
vs. enrolled among those 
 
approached) 
• Rates of no shows 
Missing 
Data 
What is the extent and patterns of missing 
The survey questions (Number and 
data? 
Type) may need to be adjusted 
Measures 
Length of time and burden to the 
respondents 
Study Outcomes. The pilot study will be used to assess the preliminary efficacy of 
MyPEEPS Mobile for YTM. Our primary outcome will be the number of condomless 
receptive anal and/or vaginal sex acts with either a cisgender male or transgender woman. 
To estimate the effect size for a larger clinical trial, we will focus on the primary outcome 
of: Condomless receptive anal or vaginal sex acts. The measures listed in Table 6 have 
been used in our MyPEEPS trialand piloted 
with 49 YTM as part of our formative 
work for this application. We will collect all measures at baseline, 3 and 6 months, other 
than demographic variables which will only be collected at baseline. We also ask 
participants to report on their own knowledge of their most recent partners’ use of PrEP or 
ART medications. Given the unreliability of perceptions of partner-related risk in 
particular(71), we will use these data for a secondary sensitivity analysis (see below). This 
study is not powered for efficacy or tests of moderation or mediation; nonetheless, the 
purpose of the inclusion of these factors is to measure the impact of the intervention on 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
37 
 
 
them as intermediate targets given their potential role in behavior change. 
 
Table 6. Outcome Measures 
Demographics 
Sociodemographic (e.g., age, race/ethnicity level of education, housing status) 
Newest Vital Sign(72) 
Short Test of Functional Health Literacy (S-TOFHLA)(73) 
Transgender Congruence Scale(74) 
Measures of Hormone Use 
Primary Outcome Measures 
Condomless receptive anal and/or vaginal sex (adapted from AIDS-Risk Behavior Assessment)(68-70) 
Intermediate Outcome Measures 
Self-efficacy for safer sex and situational temptation for unsafe sex(75) 
Condom Errors(76) 
Health Protective Communication Scale(77) 
HIV-Knowledge Questionnaire – HIV-KQ-18(78) 
Alcohol and Substance Use(79) 
Sexual Behavior Index(80) 
Secondary Outcomes 
nPEP and PrEP Use 
Self-reported HIV and STI Testing 
Partner(s) PrEP Use, PrEP Adherence or Viral Suppression (if partner is HIV+) 
Program Evaluation 
Knowledge of and beliefs about the content of the MyPEEPS Mobile intervention 
MyPEEPS intervention acceptability and tolerability(81) 
Paradata 
 
Data Analysis. All multivariate analyses will be preceded by standard descriptive bivariate 
analyses to describe key variables and relationships among key variables. These analyses will 
include means, frequency tables, histograms, and examination of distributions. All statistical 
tests will be two-sided tests with the level of significance at 0.05(60). 
Outcomes: This study will examine whether the intervention will reduce the number of condomless receptive anal  
or vaginal penile sex with either a cisgender male or transgender woman; the primary outcome is a count 
measure (Table 6 - Outcome Measures). We assume that all count outcome variables follow 
Poisson or negative binomial (NB) distribution, y~Poi(λ) or y~NB(λ, κ) and all percentage outcomes 
follow binomial distribution, y~bin(n, p). Let the expected value of outcome, E(y) = φ, so φ = λ for 
count oua tcomes and φ = p for binary outcomes. We propose the following generalized linear 
mixed model (GLMM) to analyze all outcomes. The level of analysis 
will be assessed at each data point. For person i and data assess wave W (i.e., baseline, 3, and 6 
months): 
h೦φi,w೦                 =                   β0                    +                   β1Arm                    +                   β2W                    +                  β3INT                  +                  β4𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂𝐂                +                  β5𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗 
+ μi, 
where h(. ) Is the link function for the GLMM. We will 
use log-links (Poisson or NB regression) for count 
Table 7. Codes for design variable s 
Delayed 
Months 
Intervention 
Arm 
Intervention 
Arm 
Arm W INT Arm W INT 
Baseline 
0 
0 
0 
1 
0 
0 
3 mos 
0 
1 
1 
1 
1 
0 
6 mos 
0 
2 
1 
1 
2 
1 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
38 
 
 
outcomes and logit-links (logistic regression) for binary 
outcomes. μi iid~N(0, σj ) are personal level random 
effects for the interceptor. Variables Arm, W, and INT 
are design variables. The codes for the design variables are presented in Table 7. Variable Arm is 
the 
indicator for intervention arms (0=intervention arm and 1=delayed intervention arm); variable W is 
data assess wave indicator (0=baseline, 1=3 months, 2=6 months); and variable INT is the 
intervention indicator (0=pre-intervention, and 1=post-intervention). COV is a vector of personal 
level covariates at the baseline, such as age, sex, etc., and 𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗𝐗 is a vector of time-dependent 
covariates assessed at each wave of data for each subject controlling for potential bias between 
two arms. This model will include both baseline personal level covariates as well as personal-wave 
level covariates to control for different types of potential confounders. Regression parameter β3 is 
the mean value change for the outcome variables between pre- and post-intervention and 
measures the impact of the intervention on each outcome variable. For the Poisson or NB models, 
exp (β3) is the ratio of mean counts (i.e., risk ratio) between post and pre-intervention. For the 
logistic models, exp (β3) is the odds ratio between pre- and post-intervention. If the count outcome 
has excess 0s (for example, much greater than expected number of people who had 0 receptive 
condomless anal or vaginal sex act as comparison to number of people who had 1,2,..,etc. number 
of receptive condomless anal or vaginal sex acts), the above Poisson or negative binomial GLMMs 
can be extended to zero-inflated Poisson or negative binomial models(82). 
Sensitivity Analysis: To address whether PrEP use by the partner or the viral suppression status 
of a HIV+ partner affects the outcome, we will ask participants if their partner(s) use PrEP, use 
PrEP regularly or if their partner is HIV+ are virally suppressed and we will conduct a 
counterfactual sensitivity analysis(83, 84), which includes the following steps: 1) We will 
examine if there is any difference in partners’ PrEP use and/or HIV status between pre- and 
post- intervention. This comparison can be done using the same GLMM for the primary 
outcome, treating partners’ PrEP use and/or HIV status as the outcome. 2) We will examine the 
impact (or association) of partners’ PrEP use and/or HIV status on condomless anal sex based 
on a GLMM or LMM. 3) If there are difference in step 1) AND significant effects in step 2), our 
estimation of the impact of intervention on main outcome will be biased. We will correct 
estimates by estimating the true causal effect of impact on mainoutcome. 
Secondary Outcomes: We will use GLMM to analyze secondary outcomes with appropriate choice 
of link functions according to the outcomes. All analyses will be done on the final data set 
(baseline n=80). As an objective of this R34, we will quantify intervention effect sizes of program 
impact on risky sexual behavior and substance use. To gauge effect sizes, we will use Cohen’s d 
(85) and Hedge’s g corrected for sample size(86), both of which express the average amount of 
individual change between experimental and control groups in terms of units of standard 
deviation. In addition, we will also estimate the correlations within outcomes across timepoints as 
the magnitude of this correlation can have significant impact on power to detect effects in 
repeated measures analyses. These estimates of effect size will enable us to accurately 
determine the sample size necessary to achieve adequate statistical power in future 
investigations of program efficacy within this population. We will use the following outcomes to 
estimate the statistical power for a larger study: (1) number of receptive condomless anal or 
vaginal sex acts with male partners (e.g., cisgender males and transgender women) during the 
past 3 months, and (2) number of HIV tests during the past 3 months. We will estimate the 
statistical power to examine overall effect with the total subjects to conduct stratified analysis to 
examine the effects in some subgroups (such as by sex and/or for some racial/ethnicity 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
39 
 
 
subgroups). All power estimations will be based on α=0.05 and 2-sided tests. We will base our 
power estimates on the retention rate at the 3-month and 6-month follow up assessment for each 
study arm, the intra-cluster correlation. 
Given these parameters, if the findings from the pilot study indicate that at post-intervention 
the number of receptive condomless anal or vaginal sex acts during the past 3 months 
decreased by 45% or a relative risk (RR) of 0.55, then for a future R01 study, we will have at 
least 99% power to detect such difference for a total sample size of 320. Secondly, for the 
stratified analyses, we will have 87% power to detect a relative risk of 0.55 in subgroups with a 
sample size of 64. For the HIV testing outcome, if the number of HIV tests during the past 3 
months increases by 75% (i.e., RR 
=1.75), a sample size of 320 participants will have an 84% power to detect such difference.  
Paradata: we will assess participants’ use of the App over time through the collection of 
paradata(87), specifically the page accessed, time stamp and device type. Paradata is considered 
“free” in that it does not require any additional effort from the user(88) Data Analysis: At the 
individual level, we will understand participants’ use of the App over time through the collection of 
paradata(87) which is “free” in that it does not require any additional effort from the user(88). To 
explore barriers and facilitators to widespread implementation of MyPEEPS Mobile for YTM, we 
will collect data during intervention, implementation, and after the trial has ended. The primary 
paradata to be collected are page accessed, time stamp, and device type. From these data, we 
will derive the following use of data for each session: duration on each page, page progression 
through the application, time from login to result, and total time from login to logout. We will 
analyze the data at the individual-level (i.e., user-level), application-level, page-level, and 
session- 
level and assess how these differ by demographic characteristics, technology use, 
and outcome measures. Additionally, we will measure the amount in bytes of user data 
transmitted. Importantly, longitudinal analysis will determine if the user experience changes with 
repeated use. The paradata collected from each page will be analyzed to generate a “heatmap” 
of user-interaction 
(i.e., the distribution of activity for each link/button) that will inform 
user duration on each page of the App and user interaction with App content, contact pages, and 
the help page. We will explore usability issues with consideration for how many times users 
accessed help and what page of the App referred them to the help, implying the need for 
clarification. We will analyze differences in 
aggregated data by demographic group (e.g., age) to better understand engagement with the 
intervention and potential facilitators and barriers to App use. 
 
7.6.1 
Data Analysis for Aim 3 
 
All in-depth interviews will be transcribed verbatim and then coded. The development and 
application of a coding scheme is an integral component of the data analysis process. It enables 
the systematic examination and interpretation of the data related to the primary analytic foci. The 
coding scheme is conceptualized as a multilevel structure. At the highest level are the primary 
analytic foci coded as headings. Specific aspects or dimensions of the headings are assigned 
core codes. Specific aspects or dimensions of the core codes are assigned sub-codes. We will 
use NVivo™ (QSR International, Victoria, Australia), a software program for qualitative analysis, 
to facilitate the analysis. 
The following 7 steps will be used to develop the coding scheme: Step 1: Identify the principal 
issues discussed by participants; Step 2: Construct definitions of the primary analytic themes; 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
49 
 
 
Step 3: Develop and apply core codes and sub-codes to the initial set of interviews; Step 4: 
Develop a provisional coding scheme; Step 5: Test and refine the provisional coding scheme; 
Step 6: Reconcile coding differences and construct an updated and final coding scheme; Step 7: 
Apply the coding scheme to the full data set and assess inter-coder reliability. After all transcripts 
have been coded, we will extract and examine the content of text segments linked to core codes 
and sub- codes relevant to understanding barriers and facilitators to the use of the MyPEEPS 
App. Based on the coded data, we will propose ways in which certain themes are analytically 
related. A careful examination of the coded text will reveal the associations among these themes 
and may lead to more refined data searches. Once we establish patterns of relationships among 
themes and issues, we will identify participants’ accounts that support or refute these patterns. 
Identifying and accounting for cases that deviate from an interpretative pattern enables us to test 
and confirm the pattern’s validity and robustness. 
Triangulation of Findings. We will examine changes in HIV risk behaviors and its relationship to 
factors including but not limited to changes in substance use, alcohol use, stigma, victimization. 
We will triangulate the findings from our quantitative outcome measures (Aim 2) with the findings 
from our qualitative data (Aim 3) to better understand participants’ decisions for their HIV risk 
behaviors. 
 
7.7 
Missing Data 
 
Missing data may occur in the proposed study in several ways. First, it may be due to item non- 
response. In these cases, we will prorate total scores for a measure by taking an average score 
on the measure and multiplying it by the total number of items in the scale. Attrition can result in 
missing data due to missed assessments or dropout from the study. Prior to performing any 
outcome analyses, we will evaluate the amount, reasons, and patterns of missing data. Missing 
data unrelated to the outcome of interest will be considered missing completely at random 
(MCAR), and complete case analysis will still generate unbiased estimates.(89) For the missing 
values at the baseline or partial baseline collected data, we will use a multiple imputation 
approach.(89) We will conduct sensitivity analyses to compare estimates of treatment effects with 
and without multiple imputation to assess the effect of missing data on statistical inference. 
 
8.1 
HUMAN SUBJECTS PROTECTIONS 
 
8.2 
Informed Consent and Assent 
 
The study plan, advertisements, or recruitment letters, lay description of the study, and all 
consent forms will be submitted to the IRB following proposal acceptance and prior to study 
initiation. MPI, Dr. Schnall, will be responsible for obtaining IRB approval for this study. 
Recruitment for study participation will occur following IRB approval. The MPIs will delegate 
study task to research staff including determining eligibility for inclusion, explaining the purpose 
of the study, answering any questions, and obtaining e-assent/consent from the participants. 
Individuals who agree to participate will sign an e-assent/consent form. Potential risks and 
strategies for risk management will be carefully explained as part of informed e-assent/consent 
procedures. All HIPAA requirements will be applied to this study. 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
41 
 
 
8.2.1 
Consent and Assent Process 
Written informed e-assent will obtained from study participants under age 18. Participants at age 
17 will be considered children for purposes of study activities. As such, they will complete 
informed assent for these portions of the study. Study participants ages 18 and above will 
provide written informed consent. If a respondent asks any questions about the study, an on-site 
research assistant will clarify or answer questions. The participant’s ability to not respond to any 
study questions and/or terminate participation at any time will be stated clearly in consent forms. 
 
8.2.2 
Parental consent 
Consistent with our prior studies in this population, we will not require parental consent for study 
enrollment. Parental consent may decrease participation rates because some youth (17-year- 
olds) will fear that they may be “outed” as a result of participation. Disclosure of sexual 
orientation or gender identity may place participating youth at risk for parental harassment, 
abuse, or expulsion from the home. The nature and scope of the proposed research do not pose 
more than “minimal risk” to participants, or that “the probability and magnitude of harm or 
discomfort anticipated in the research are not greater in and of themselves than those ordinarily 
encountered in daily life or during the performance of routine physical or psychological 
examinations or tests” (45 CFR Part 46.102). Study measures are standard in this population, as 
are waivers of parental permission for survey and interview studies. 
 
8.2 
Waiver of Parental Written Consent and Screening Procedure Written Consent 
 
Each study site has experience conducting HIV prevention research with minors. We will be 
applying for a waiver of written informed consent from the parents of study participants because 
the involvement of children in this research meets the criteria for waiving parent permission (45 
CFR 46.116(d)). The study qualifies for a waiver of parental consent as the following criteria are 
met in this study: 
• The research involves no more than minimal risk to the subjects - The study does not 
place the patient at any additional risk than using their smartphones during their everyday 
lives. 
The only other risk to participants is a breach of their protected health information, and we 
have put systems in place to protect their identity. 
• The waiver or alteration will not adversely affect the rights and welfare of the subjects - 
If parents are required to consent their children then this may present a breach of 
confidentiality related to participants’ sexuality and sexual behaviors. 
• The research could not practicably be carried out without the waiver or alteration - If 
consent is required from parents then this may jeopardize the confidentiality of our study 
participants’ health behaviors and may place study participants at increased risk of 
adverse outcomes associated with disclosure of their sexual orientation to their parents. 
Past research has shown a higher risk of familial childhood maltreatment among lesbian, 
gay, and bisexual individuals than among heterosexualindividuals. 
• Whenever appropriate, the subjects will be provided with additional pertinent information 
after participation - All study subjects will be given a copy of the written assent form 
which will include contact information of our study team as well as referral information. 
To compensate for the waiver of parental consent, participants receive the described formal 
individual assessment of capacity to consent (above) to ensure their understanding of study 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
42 
 
 
goals, procedures, and risks from disclosure of sensitive information. Consistent with national 
policy recommendations from the Society for Adolescent Medicine, requiring parental 
permission for the proposed study would have a number of possible negative effects, including: 
(1) reducing the validity of the findings by effectively eliminating potential participants unwilling to 
share permission forms with their parents/guardians; (2) increasing risk to some youth whose 
parents have a negative response to the material in the permission forms that would (correctly) 
suggest their child has a minority or alternative sexual orientation; and (3) contributing little to 
actual subject protection, given the minimal risk of study participation. 
 
We are also requesting a waiver of documentation of consent for online screening procedures. 
The screening information will only be kept electronically in REDCap, which is a secure, CUMC 
approved system. 
 
8.3 
Vulnerable Subjects: Protecting Against/Minimizing Potential Risk 
 
8.3.1 
Procedures in the event of a reactive HIV test. 
In the event a participant has a reactive HIV test, study team members will counsel the participant 
regarding the meaning of their result. Specifically, the study team will reiterate with the participant 
that: 
• Their result is a preliminary positive, and it is likely that they are infected with HIV, 
• The preliminary positive result needs to be verified with confirmatory testing, 
• They need to return for the results of their confirmatory testing,and 
• Resources for counseling and referral to care are available to them. 
 
The participant will be referred for follow-up testing and care. We have one physician and one 
nurse as part of our investigator team. 
 
8.4 
Risks 
 
8.4.1 
Overview of Potential Risks 
 
There is no more than a minimal risk associated with any of the proposed study activities. The 
study activities meet the general definition found in Subpart A (46.102) that the probability and 
magnitude of harm or discomfort anticipated in the research are not greater in and of themselves 
than those ordinarily encountered in daily life or during the performance of routine physical or 
psychological assessments or tests. The risks of participating in this study are few. Potential 
risks are those related to venipuncture, discomfort with study interview questions, and potential 
breaches of confidentiality. It is possible that certain questions on the survey may make 
participants feel uncomfortable, but participants are free to decline to answer any questions. 
Additionally, clinical study sites have co-located mental health and/or counseling services that 
may be consulted should a participant enter crisis. Participation in research can involve loss of 
privacy. All study data will be maintained on Columbia University Irving Medical Center (CUIMC) 
and Lurie Children’s Hospital servers that are secure and HIPAA compliant. All signed consent 
forms, study data, and payment receipts used in this study will be kept in locked files at both 
sites which only the investigators can access. We will also receive a Federal Certificate of 
Confidentiality which will protect against attempts by law enforcement or other government 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
 
agencies to access our data. 
 
8.4.2 
General Risks 
There may be risks or discomforts in participating in this study. Participants may feel 
uncomfortable with the HIV prevention information that is provided and completing some 
questions in the survey. Participants may skip any HIV information or questions that may make 
them feel uncomfortable or stop the research procedure. People around may observe 
participants using the MyPEEPS Mobile application. If participants are concerned about people 
seeing them use of MyPEEPS Mobile, it is important that they access the application in private 
location. 
 
8.4.3 
Loss of confidentiality 
 
A risk of taking part in this study is the possibility of a loss of confidentiality or privacy. Loss of 
confidentiality or privacy means having personal information shared with someone who is not 
on the study team and was not supposed to see or know about your information. All study 
data will be stored in password protected computers or file cabinets in locked offices. All 
research team members will pass the protection of human subjects and research HIPAA 
exams and sign a protocol-specific conflict of interest. All procedures have been designed to 
protect each participant's privacy and allow for anonymous participation. All study data will be 
maintained in a completely secure and HIPAA compliant environment. All CUIMC servers 
have HIPAA compliant security 
 
8.5 
Social Impact Events 
 
Individuals enrolled in this study may experience personal problems resulting from the study 
participation. Such problems are termed social impact events. Although study sites will make 
every effort to protect participant privacy and confidentiality, it is possible that participants' 
involvement in the study could become known to others, and that participants may 
experience stigmatization or discrimination as a result of being perceived as being HIV 
infected or at risk for HIV infection. For example, participants could be treated unfairly, or 
could have problems being accepted by their families and/or communities. Problems may 
also occur in circumstances in which study participation is not disclosed, such as impact on 
employment related to time taken for study visits. 
In the event that a participant reports a social impact event, every effort will be made by study 
staff to provide appropriate assistance, and/or referrals to appropriate resources. Social 
impact events are documented and reviewed on a scheduled basis by the protocol team 
leadership with the goal of reducing their incidence and enhancing the ability of study staff to 
mitigate them when possible. 
Social impact events that are judged by the MPIs/designee to be serious, unexpected, or more 
severe or frequent than anticipated, will be reported to the responsible site’s Ethics 
Committee/Institutional Review Board (EC/IRB) promptly, or otherwise in accordance withthe 
EC/IRB’s requirements. 
 
8.6 
Benefits 
 
The potential benefits to an individual participant in the study are not known. The potential 
benefits of the study to others could be considerable. If our hypotheses are true, this study will 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
 
make a significant contribution towards improving HIV incidence in YTM. This study has not 
been designed for the direct benefit of its participants; however, there are a number of ways in 
which they may derive benefit, such as awareness of HIV status, increased access to HIV 
prevention tools (e.g., testing, PrEP) and HIV services, if a participant sero-converts. The 
proposed research will inform the delivery of HIV prevention messages. The knowledge gained 
will contribute to the body of knowledge regarding the use of health information technology for 
improving the lives of YTM at risk for HIV. The avoidance of HIV throughout study participation 
will be a significant personal benefit to participants. 
 
8.7 
Participant Privacy and Confidentiality 
 
Access to individually identifiable private information about human subjects. Access to 
individually identified private information about human subjects will be limited to research team 
members who collect and manage the data, the Project Director, and the MPIs. De-identified 
data will be accessible to all members of the research team involved in the data analysis. 
 
Our study team is extremely prudent in keeping subject data secure and confidential. All 
laboratory specimens, evaluation forms, reports, and other records will be identified by a unique 
coded number to maintain participant confidentiality. The material, records, and data obtained 
through participation in the study will be specifically for research purposes. Existing health 
records may be used with the permission of the participants. Materials will be obtained by 
trained clinical staff at each study site. Data will be stored using Research Electronic Data 
Capture (REDCap) at each respective performance site, and then the completely deidentified 
data will be merged at CUIMC. All laboratory specimens will be identified only by the 
identification number. The code linking the participant identification number to subject identifying 
information (name, address, etc.) is maintained at the clinical sites through REDCap, and only 
authorized site personnel have access to the code. Limited individually identifiable private 
information is collected that is essential for processing participant payments and for analysis 
purposes. 
 
All study data will be stored in password-protected computers or file cabinets in locked offices. 
All research team members will pass the protection of human subjects and HIPAA research 
exams and sign a protocol-specific conflict of interest. Risks will be minimized by not including 
personal 
identifying information on the forms, when possible, and by conducting interviews and collection of 
personal information in a private setting. All data will be collected using unique patient 
identification codes. All laboratory specimens, evaluation forms, reports, and other records will be 
identified by a coded number to maintain participant confidentiality. All records will be stored in a 
locked file cabinet. Study data from both sites will be collected and managed using REDCap. 
REDCap is a secure web application designed to support data capture for research studies, 
providing user- friendly web-based case report forms, real-time data entry validation (e.g., for 
data types and range checks), audit trails, and a de-identified data export mechanism to 
common statistical packages (SPSS, SAS, Stata, R/S-Plus). REDCap data collection projects 
rely on a thorough study-specific data dictionary defined in an iterative self-documenting process 
by all members of the research team. This iterative development and testing process results in a 
well-planned data collection strategy for individual studies. REDCap also includes a powerful tool 
for building and managing online surveys. The research team can create and design surveys ina 
web browser and engage potential respondents using a variety of notification methods. REDCap 
is flexible enough to be used for a variety of types of research and provides an intuitive user 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
 
interface for database and survey design and data entry. Lastly, clinical information will not be 
released without written permission of the participant, except as necessary for monitoring by the 
IRB or the NIH. 
 
8.7.1 
Confidentiality of Study Data 
 
We have developed systematic protocols for data handling and storage over multiple cohort 
studies. To protect the integrity of the participant’s data, the RA will assign each participant a 
unique PID at study enrollment. This code number will be used for all study data. We will maintain 
a list of participants with links between identifying information and code numbers for the sole 
purpose of avoiding any duplication in completion of the survey. Only the PIs, RAs, and Project 
Director will have access to these lists, which will be kept behind double-locks or on a secure 
server with password protected access. Computer files consist of the tracking database 
(REDCap), and study data files downloaded from the database. Tracking files are maintained in a 
highly secure scheduling and monitoring database which both sites can access. This database 
contains all contact information and is used to schedule and track study visits; it is completely 
password protected and user access and privileges will be managed by the data manager and 
Project Director. The tracking database stores contact preferences including information the 
participant agreed to provide for the purposes of tracking and communication; all 
communications strictly follow the participants’ contact limitations. Data files are exported from 
REDCap and imported into the SPSS database for storage and analysis. Computer data files 
never have any identifying information and are encrypted for transfer between study sites. Data 
files do not include information that could be used to identify the participant from the data file 
alone. Columbia University will provide a standards-compliant (HIPAA, HHS Cybersecure) private 
cloud server for the hosting of application content and user data for the duration of the application 
deployment. This is a secure system and will be further protected by login credentials for limited 
access to protect participant confidentiality. Participants will also be encouraged to choose a 
private location to program a password for the application and complete the intervention modules 
for additional security. They will be prompted to log-out of the website after completing or 
progressing through any given module. 
 
8.8 
Certificate of Confidentiality 
 
This research specifically targets a vulnerable population, children - YTM age 17. We will take 
every available step to minimize the risk of identifying/linking data being subpoenaed, stolen, or 
inadvertently released. First, per Section 2012 of the 21st Century Cures Act as implemented in 
the 2017 NIH Certificates of Confidentiality Policy, all ongoing or new research funded by NIH as 
of December 13, 2016 that is collecting or using identifiable, sensitive information is automatically 
issued a CoC. As noted on the NIH website, 
(http://grants.nih.gov/grants/policy/COC/faqs.htm#187), a Certificate of Confidentiality will help the 
research team “...avoid compelled ‘involuntary disclosure’ (e.g., subpoenas) of names and other 
identifying information about any individual who participates as a research participant (i.e., about 
46 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
47 
 
 
whom the investigator maintains identifying information) during any time the Certificate is in 
effect.” Second, all research staff members are required to complete ethical clearance 
certification regarding protection of human’s subjects through their relevant IRBs. Third, all 
studies will have documented procedures to safeguard against the risk of the linking information 
being stolen by keeping such information in locked spaces to which only essential study 
personnel who have completed CITI certification for human subjects research ethics training 
(http://citiprogram.org) will have access. 
 
8.9 
Unexpected and Serious Adverse Event Reporting 
 
A detailed monitoring plan will be included as part of the study protocol, submitted to the IRB, and 
reviewed and approved by the NIMH before the study begins. Prior to initiation of the study, 
agreement about the data safety monitoring plan will be confirmed to ensure the safety of 
subjects and the validity and integrity of the data. The research coordinator at each site will 
report serious adverse events (SAEs) that are unexpected and study-related immediately to a 
study physician who will convey this information to the study team, IRB, and the NIH. All AEs and 
SAEs will be captured, reports will be completed, and information will be entered into the study 
database. A safety report will detail all serious and unexpected AEs or other unanticipated 
problems that involve risk to study participants or others and whether these appeared to be 
related to the study- based interventions or research assessment protocols. All AEs will be 
reviewed every 6 months, or sooner, with the designated safety Data Safety and Monitoring 
Board. 
 
8.10 
ClinicalTrials.gov 
 
This trial and protocol will be registered in ClinicalTrials.gov. 
 
9.0 PUBLICATION POLICY 
 
Any presentation, abstract, or manuscript will be made available for review by the study 
sponsor(s) prior to submission. 
 
10.0 REFERENCES 
 
1. 
Becasen JS, Denard CL, Mullins MM, Higa DH, Sipe TA. Estimating the Prevalence of HIVand 
Sexual Behaviors Among the US Transgender Population: A Systematic Review and Meta-Analysis, 
2006– 2017. American journal of public health. 2018;109(1):e1-e8. doi: 10.2105/AJPH.2018.304727. 
2. 
Bernstein KT, Marcus JL, Nieri G, Philip SS, Klausner JD. Rectal gonorrhea andchlamydia 
reinfection is associated with increased risk of HIV seroconversion. JAIDS Journal of Acquired 
Immune Deficiency Syndromes. 2010;53(4):537-43. 
3. 
Fleming DT, Wasserheit JN. From epidemiological synergy to public health policy and practice: 
the contribution of other sexually transmitted diseases to sexual transmission of HIV infection. Sexually 
transmitted infections. 1999;75(1):3-17. 
4. 
Beck EC, Birkett M, Armbruster B, Mustanski B. A data-driven simulation of HIV spread among 
young men who have sex with men: the role of age and race mixing, and STIs. Journal of acquired 
immune deficiency syndromes (1999). 2015;70(2):186. 
5. 
Bockting WO, Miner MH, Swinburne Romine RE, Hamilton A, Coleman E.Stigma, mental 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
48 
 
 
health, and resilience in an online sample of the US transgender population. American journal of public 
health. 2013;103(5):943-51. 
6. 
Reisner SL, Gamarel KE, Dunham E, Hopwood R, Hwahng S. Female-to-male transmasculine adult 
health: a mixed-methods community-based needs assessment. Journal of the American Psychiatric Nurses 
Association. 2013;19(5):293-303. 
7. 
Reisner SL, Pardo ST, Gamarel KE, White Hughto JM, Pardee DJ, Keo-Meier CL. Substance use 
to cope with stigma in healthcare among U.S. female-to-male trans masculine adults. LGBT Health. 
2015;2(4):324-32. doi: 10.1089/lgbt.2015.0001. 
8. 
Golub SA, Fikslin RA, Starbuck L, Klein A. High Rates of PrEP Eligibility but Low Rates of 
PrEP Access Among a National Sample of Transmasculine Individuals. J Acquir Immune Defic Syndr. 
2019;82(1):e1-e7. Epub 2019/06/25. doi: 10.1097/QAI.0000000000002116. PubMed PMID: 
31232834; PMCID: PMC6692190. 
9. 
Reisner SL, Moore CS, Asquith A, Pardee DJ, Sarvet A, Mayer G, Mayer KH. High risk andlow 
uptake of pre-exposure prophylaxis to prevent HIV acquisition in a national online sample of 
transgender men who have sex with men in the United States. J Int AIDS Soc. 2019;22(9):e25391. 
Epub 2019/09/20. doi: 10.1002/jia2.25391. PubMed PMID: 31536171; PMCID: PMC6752156. 
10. 
Herbst JH, Beeker C, Mathew A, McNally T, Passin WF, Kay LS, Crepaz N, Lyles CM, Briss P, 
Chattopadhyay S, Johnson RL. The Effectiveness of Individual-, Group-, and Community-Level HIV 
Behavioral Risk-Reduction Interventions for Adult Men Who Have Sex with Men: A Systematic 
Review. American Journal of Preventive Medicine. 2007;32(4, Supplement):38-67. doi:  
http://dx.doi.org/10.1016/j.amepre.2006.12.006. 
11. 
Schnall R, Cimino JJ, Bakken S. Development of a prototype continuity of care record with context- 
specific links to meet the information needs of case managers for persons living with HIV. International 
journal of medical informatics. 2012. Epub 2012/05/29. doi: 10.1016/j.ijmedinf.2012.05.002. PubMed 
PMID: 22632821. 
12. 
Bennett GG, Glasgow RE. The delivery of public health interventions via the Internet: 
actualizing their potential. Annual review of public health. 2009;30:273-92. doi: 
10.1146/annurev.publhealth.031308.100235. PubMed PMID: 19296777. 
13. 
Schnall R, Wantland D, Velez O, Cato K, Jia H. Feasibility testing of a web-based symptom 
self- management system for persons living with HIV. J Assoc Nurses AIDS Care. 2014;25(4):364-71. 
doi: 10.1016/j.jana.2013.09.002. PubMed PMID: 24434198; PMCID: 4057945. 
14. 
Moskowitz D, Melton, D., & Owczarzak, J. , 77(1), 20-26. . PowerON: the use of instant 
message counseling and the Internet to facilitate HIV/STD education and prevention. Patient Educ 
Couns. 2009;77(1):20-6. doi: 10.1016/j.pec.2009.01.002. 
15. 
Wei C, Lim S, Guadamuz T, Koe S. Virtual Versus Physical Spaces: Which Facilitates Greater 
HIV Risk Taking Among Men Who Have Sex with Men in East and South-East Asia? AIDS and Behavior. 
2013:1-8. doi: 10.1007/s10461-013-0628-4. 
16. 
Williams M, Bowen A, Ei S. An evaluation of the experiences of rural MSM who accessed an 
online HIV/AIDS health promotion intervention. Health promotion practice. 2010;11(4):474-82. doi: 
10.1177/1524839908324783. PubMed PMID: 19116419; PMCID: 2916959. 
17. 
Hightow-Weidman L, Pike, E., Fowler, B., Matthews, D., Kibe, J., McCoy, R., & Adimora, A. . 
HealthMpowerment.org: Feasibility and acceptability of delivering an Internet intervention to young 
Black men who have sex with men. AIDS Care. 2012;24(7):910-20. doi: 
10.1080/09540121.2011.647677. 
18. 
Williams ML, Bowen AM, Horvath KJ. The social/sexual environment of gay men residing in a 
rural frontier state: implications for the development of HIV prevention programs. The Journal of rural 
health : official journal of the American Rural Health Association and the National Rural Health Care 
Association. 2005;21(1):48-55. PubMed PMID: 15667009; PMCID: 2614671. 
19. 
Paul ME, Castillo M, Emmanuel P, Bauermeister JA, Mena LA, Sullivan PS, Hightow-Weidman 
LB. Scale up mHealth HIV interventions: site and public health perspectives and lessons learned from 
P3. Mhealth. 2021;7:38. Epub 2021/04/27. doi: 10.21037/mhealth-20-65. PubMed PMID:33898607; 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
49 
 
 
PMCID: PMC8063006. 
20. 
Donenberg GR, Pao M. Youths and HIV/AIDS: Psychiatry's Role in a Changing Epidemic. Journal 
of the American Academy of Child & Adolescent Psychiatry. 2005;44(8):728-47. doi:  
http://dx.doi.org/10.1097/01.chi.0000166381.68392.02. 
21. 
Balaji A, Bowles K, Le B, et al. High HIV incidence and prevalence and associated risk 
factors among young MSM, 2008. AIDS. 2013;27:269-78. 
22. 
Hidalgo M, Kuhns L, Hotton A, Johnson A, Mustanski B, Garofalo R. The MyPEEPS Randomized 
Controlled Trial: A Pilot of Preliminary Efficacy, Feasibility, and Acceptability of a Group-Level, HIV Risk 
Reduction Intervention for Young Men Who Have Sex with Men. Arch Sex Behav. 2015;44(2):475-85. 
doi: 10.1007/s10508-014-0347-6. 
23. 
Gordian-Arroyo A, Garofalo R, Kuhns LM, Pearson C, Bruce J, Batey DS, Radix A, Belkind U, 
Hidalgo MA, Hirshfield S, Schrimshaw EW, Schnall R. Awareness, Willingness, and Perceived Efficacy 
of Pre-exposure Prophylaxis among Adolescent Sexual Minority Males. J Urban Health. 2020. Epub 
2020/08/14. doi: 10.1007/s11524-020-00447-5. PubMed PMID: 32789625. 
24. 
Ignacio M, Garofalo R, Pearson C, Kuhns LM, Bruce J, Scott Batey D, Radix A, Belkind U, 
Hidalgo MA, Hirshfield S, Schnall R. Pilot feasibility trial of the MyPEEPS mobile app to reduce sexual 
risk among young men in 4 cities. JAMIA Open. 2019;2(2):272-9. Epub 2019/07/12. doi: 
10.1093/jamiaopen/ooz008. PubMed PMID: 31294422; PMCID: PMC6603441. 
25. 
Kuhns LM, Garofalo R, Hidalgo M, Hirshfield S, Pearson C, Bruce J, Batey DS, Radix A, BelkindU, 
Jia H, Schnall R. A randomized controlled efficacy trial of an mHealth HIV prevention intervention for 
sexual minority young men: MyPEEPS mobile study protocol. BMC Public Health. 2020;20(1):65. Epub 
2020/01/17. doi: 10.1186/s12889-020-8180-4. PubMed PMID: 31941475; PMCID: PMC6964028. 
26. 
Schnall R, Kuhns L, Pearson C, Bruce J, Batey DS, Radix A, Belkind U, Hidalgo MA, Hirshfield 
S, Ganzhorn S, Garofalo R. Preliminary Results from a Pragmatic Clinical Trial of MyPEEPS Mobile to 
Improve HIV Prevention Behaviors in Young Men. Stud Health Technol Inform. 2020;270:1365-6. Epub 
2020/06/24. doi: 10.3233/SHTI200444. PubMed PMID: 32570661. 
27. 
Schnall R, Kuhns LM, Hidalgo MA, Powell D, Thai J, Hirshfield S, Pearson C, Ignacio M, Bruce 
J, Batey DS. Adaptation of a group-based HIV risk reduction intervention to a mobile app for young 
sexual minority men. AIDS Education and Prevention. 2018;30(6):449-62. 
28. 
James SE, Herman, J.L., Rankin, S., Keisling, M., Mottet, L., Anafi, M. The Report of the2015 
U.S. Transgender Survey Washington, D.C.: National Center for Transgender Equality; 2015. Available 
from:      https://transequality.org/sites/default/files/docs/usts/USTS-Full-Report-Dec17.pdf. 
29. 
Gass BV, Horvath KJ, Marrow E, Rood BA, Pantalone DW. Associations Between Social 
Support Availability and HIV Risk and Protective Factors in a U.S. Sample of Adults with Diverse 
Transgender 
Identities. LGBT Health. 2021;8(1):60-7. Epub 2020/12/29. doi: 10.1089/lgbt.2020.0035. PubMed PMID: 
33370219. 
30. 
Todd K, Peitzmeier SM, Kattari SK, Miller-Peruse M, Sharma A, Stephenson R. Demographic 
and Behavioral Profiles of Nonbinary and Binary Transgender Youth. Transgend Health. 2019;4(1):254- 
61. Epub 2019/10/24. doi: 10.1089/trgh.2018.0068. PubMed PMID: 31641692; PMCID:PMC6802728. 
31. 
Centers for Disease Control and Prevention. Lifetime Risk of HIV Diagnosis 2016 [updated 
February 2323 April 2018]. Available from: https://www.cdc.gov/nchhstp/newsroom/2016/croi-press-   
release-risk.html. 
32. 
Fauci AS, Redfield RR, Sigounas G, Weahkee MD, Giroir BP. Ending the HIV Epidemic: A Plan 
for the United States. JAMA. 2019;321(9):844-5. doi: 10.1001/jama.2019.1343. 
33. 
Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data 
capture (REDCap)—a metadata-driven methodology and workflow process for providing translational 
research informatics support. Journal of biomedical informatics. 2009;42(2):377-81. 
34. 
Ballard AM, Cardwell T, Young AM. Fraud detection protocol for web-based research among 
men who have sex with men: development and descriptive evaluation. JMIR Public Health Surveill. 
2019;5(1):e12344. 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
50 
 
 
35. 
Teitcher JE, Bockting WO, Bauermeister JA, Hoefer CJ, Miner MH, Klitzman RL. Detecting, 
preventing, and responding to “fraudsters” in internet research: ethics and tradeoffs. The Journal of 
Law, Medicine & Ethics. 2015;43(1):116-33. 
36. 
Ignacio M, Garofalo R, Pearson C, Kuhns LM, Bruce J, D Batey S, Radix A, Belkind U, Hidalgo 
MA, Hirshfield S. Pilot feasibility trial of the MyPEEPS mobile app to reduce sexual risk among young 
men in 4 cities. JAMIA Open. 2019. 
37. 
Schnall R, Kuhns L, Hidalgo M, Hirshfield S, Pearson C, Radix A, Belkind U, Bruce J, Batey DS, 
Garofalo R. Development of MyPEEPS Mobile: A Behavioral Health Intervention for Young Men. Studies 
in health technology and informatics. 2018;250:31-. 
38. 
Schnall R, Kuhns L, Pearson C, Bruce J, Batey DS, Radix A, Belkind U, Garibay-Rodriguez R, 
Grimball R, Hidalgo M, editors. Comparison of Recruitment Strategies for Young Men’s Participation in a 
HIV Prevention Trial. APHA's 2019 Annual Meeting and Expo (Nov 2-Nov 6); 2019: American Public 
Health Association. 
39. 
Cho H, Powell D, Pichon A, Thai J, Bruce J, Kuhns LM, Garofalo R, Schnall R. A mobile health 
intervention for HIV prevention among racially and ethnically diverse young men: usability evaluation. 
JMIR mHealth and uHealth. 2018;6(9):e11450. 
40. 
Cho H, Powell D, Pichon A, Kuhns LM, Garofalo R, Schnall R. Eye-tracking retrospective think- 
aloud as a novel approach for a usability evaluation. International journal of medical informatics. 
2019;129:366-73. 
41. 
Nielsen J, Molich R. Heuristic evaluation of user interfaces. Proceedings of the SIGCHI 
Conference on Human Factors in Computing Systems; Seattle, Washington, USA. 97281: ACM; 1990.p. 
249-56. 
42. 
Sheehan B, Lee Y-J, Rodriguez M, Tiase V, Schnall R. A Comparison of Usability Factors of 
Four Mobile Devices for Accessing Healthcare Information by Adolescents. Appl Clin Infor. 
2012;3(4):356-66. 
43. 
Lee Y, Johnson SB, Trembowelski S, Schnall R, Sheehan B, Bakken S. Cognitive 
walkthrough evaluation of an investigator profiles directory. . Proc AMIA Annu Fall Symp. 2011. 
44. 
Schnall R, Kaufman D, Hyun S, Bakken S, editors. Cognitive Evaluation of Case Managers’ Use of 
a Continuity of Care Document for Persons Living with HIV AMIA Annual Symposium; 2009; San 
Francisco, CA. 
45. 
Bertini E, Gabrielli S, Kimani S, editors. Appropriating and assessing heuristics for 
mobile computing. Advanced Visual Interfaces International Working Conference; 2006; 
Venezia, Italy. 
46. 
Gibson L, Gaffey E, Spendlove J, editors. The Usability Factor: More than just a Pretty Interface. 
8th International Congress in Nursing Informatics; 2003; Rio de Janeiro, Brazil. 
47. 
Nielson J. Ten Usability Heuristics 2005 [updated 2005]. Available 
from: http://www.useit.com/papers/heuristic/heuristic_list.html. 
48. 
Bright T, Bakken S, Johnson S. Heuristic evaluation of eNote: an electronic notes system. 
New York2006. 
49. 
Nielsen J. Heuristic evaluation. In: Nielsen J, Mack RL, editors. Usability Inspection Methods. 
New York, NY: John Wiley & Sons; 1994. 
50. 
Faulkner L. Beyond the five-user assumption: benefits of increased sample sizes in usability 
testing. Behav Res Methods Instrum Comput. 2003;35(3):379-83. Epub 2003/11/01. PubMed PMID: 
14587545. 
51. 
Yen PY, Wantland D, Bakken S. Development of a Customizable Health IT Usability Evaluation 
Scale. AMIA Annual Symposium proceedings / AMIA Symposium AMIA Symposium. 2010;2010:917- 
21. PubMed PMID: 21347112; PMCID: 3041285. 
52. 
Davis F. Perceived usefulness, perceived ease of use, and user acceptance of 
information technology. MIS Quarterly. 1989;13(3):319-40. 
53. 
Brown W, 3rd, Yen PY, Rojas M, Schnall R. Assessment of the Health IT Usability Evaluation 
Model (Health-ITUEM) for evaluating mobile health (mHealth) technology. Journal of biomedical 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
51 
 
 
informatics. 2013;46(6):1080-7. doi: 10.1016/j.jbi.2013.08.001. PubMed PMID: 23973872; PMCID: 
3844064. 
54. 
Lloyd CE, Johnson MR, Mughal S, Sturt JA, Collins GS, Roy T, Bibi R, Barnett AH. Securing 
recruitment and obtaining informed consent in minority ethnic groups in the UK. BMC Health Serv 
Res. 2008;8:68. Epub 2008/04/01. doi: 1472-6963-8-68 [pii] 
10.1186/1472-6963-8-68 [doi]. PubMed PMID: 18373876; PMCID: 2311303. 
55. 
Swanson GM, Ward AJ. Recruiting minorities into clinical trials: toward a participant-friendly 
system. J Natl Cancer Inst. 1995;87(23):1747-59. Epub 1995/12/06. PubMed PMID: 7473831. 
56. 
Altman DG, Bland JM. Treatment allocation in controlled trials: why randomise? 
BMJ. 1999;318(7192):1209-. 
57. 
Hedden Sl, Woolsen RF, Malcolm RJ. Randomization in substance abuse clinical trials. 
Substance Abuse Treatment, Prevention, and Policy. 2006;1:6. 
58. 
Schulz KF, Altman DG, Moher D, for the CONSORT Group. CONSORT 2010 Statement: 
updated guidelines for reporting parallel group randomised trials. J Clin Epi 2010;. BMC Medicine 
2010;8(18). Epub 24 march 2010. 
59. 
Vickers AJ. How to randomize. Journal of the Society for Integrative Oncology. 2006;4(4):194- 
8. PubMed PMID: 17022927; PMCID: 2596474. 
60. 
Lyles CM, Crepaz N, Herbst JH, Kay LS. Evidence–Based HIV Behavioral Prevention From the 
Perspective of the CDC's HIV/AIDS Prevention Research Synthesis Team. AIDS Education and 
Prevention. 2006;18(supp):21-31. doi: 10.1521/aeap.2006.18.supp.21. 
61. 
Garofalo R, Kuhns LM, Hotton A, Johnson A, Muldoon A, Rice D. A Randomized Controlled Trial 
of Personalized Text Message Reminders to Promote Medication Adherence Among HIV-Positive 
Adolescents and Young Adults. AIDS Behav. 2016;20(5):1049-59. Epub 2015/09/13. doi: 
10.1007/s10461- 015-1192-x. PubMed PMID: 26362167; PMCID: PMC4788595. 
62. 
Green L, Kreuter, M., Deeds, S., Partridge, K. Health education planning: Adiagnostic 
approach. Mountain View, CA: Mayfield; 1980. 
63. 
Kukafka R, Johnson SB, Linfante A, Allegrante JP. Grounding a new information technology 
implementation framework in behavioral science: a systematic analysis of the literature on IT use. Journal 
of biomedical informatics. 2003;36(3):218-27. Epub 2003/11/15. doi: 10.1016/j.jbi.2003.09.002. PubMed 
PMID: 14615230. 
64. 
Odlum M, Gordon P, Camhi E, Schnall R, Teixeira P, Bakken S. Perceptions of 
predisposing, enabling, and reinforcing factors influencing the use of a continuity of caredocument 
in special needs PLWH. J Health Care Poor Underserved. 2012;23(4):1457-76. Epub 2012/01/01. 
doi: 10.1353/hpu.2012.0172. PubMed PMID: 23698661. 
65. 
Schnall R, Gordon P, Camhi E, Bakken S. Perceptions of factors influencing use of an electronic 
record for case management of persons living with HIV. AIDS Care. 2011;23(3):357-65. Epub 
2011/02/25. doi: 10.1080/09540121.2010.507745. PubMed PMID: 21347899; PMCID: PMC3129034. 
66. 
Guba EG. Criteria for assessing the trustworthiness of naturalistic inquiries. 
Educational Communication and Technology Journal1981;29(2):75-91. 
67. 
Garofalo R, Kuhns LM, Reisner SL, Biello K, Mimiaga MJ. Efficacy of an Empowerment- 
Based, Group-Delivered HIV Prevention Intervention for Young Transgender Women: The Project 
LifeSkills 
Randomized Clinical Trial. JAMA Pediatr. 2018;172(10):916-23. doi: 10.1001/jamapediatrics.2018.1799. 
PubMed PMID: 30105381. 
68. 
Donenberg GR, Emerson E, Bryant FB, Wilson H, Weber-Shifrin E. Understanding AIDS-Risk 
Behavior Among Adolescents in Psychiatric Care: Links to Psychopathology and Peer Relationships. 
Journal of the American Academy of Child and Adolescent Psychiatry. 2001;40(6):642-53. PubMed 
PMID: PMC1201503. 
69. 
Garofalo R, Mustanski BS, McKirnan DJ, Herrick A, Donenberg GR. Methamphetamine and 
Young Men who have Sex with Men: Patterns, Correlates and Consequences of Use. Archives of 
pediatrics & adolescent medicine. 2007;161(6):591-6. doi: 10.1001/archpedi.161.6.591. PubMed PMID: 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
52 
 
 
PMC3985401. 
70. 
Garofalo R, Herrick A, Mustanski BS, Donenberg GR. Tip of the Iceberg: Young Men Who Have 
Sex With Men, the Internet, and HIV Risk. American journal of public health. 2007;97(6):1113-7. doi: 
10.2105/AJPH.2005.075630. PubMed PMID: PMC1874202. 
71. 
Gallagher T, Link L, Ramos M, Bottger E, Aberg J, Daskalakis D. Self-Perception of HIV Risk 
and Candidacy for Pre-Exposure Prophylaxis Among Men Who Have Sex with Men Testing for HIV at 
Commercial Sex Venues in New York City. LGBT Health. 2014;1(3):218-24. Epub 2014/09/01. doi: 
10.1089/lgbt.2013.0046. PubMed PMID: 26789715. 
72. 
Weiss BD, Mays MZ, Martz W, Castro KM, DeWalt DA, Pignone MP, Mockbee J, Hale FA. 
Quick Assessment of Literacy in Primary Care: The Newest Vital Sign. The Annals of Family Medicine. 
2005;3(6):514-22. doi: 10.1370/afm.405. 
73. 
Baker DW, Williams MV, Parker RM, Gazmararian JA, Nurss J. Development of a brief test 
to measure functional health literacy. Patient Education and Counseling. 1999;38(1):33-42. doi:    
http://dx.doi.org/10.1016/S0738-3991(98)00116-5. 
74. 
Kozee HB, Tylka TL, Bauerband LA. Measuring transgender individuals’ comfort with gender 
identity and appearance: Development and validation of the Transgender Congruence Scale. Psychology 
of Women Quarterly. 2012;36(2):179-96. 
75. 
Parsons JT, Halkitis PN, Bimbi D, Borkowski T. Perceptions of the benefits and costs associated 
with condom use and unprotected sex among late adolescent college students. Journal of Adolescence. 
2000;23(4):377-91. doi: http://dx.doi.org/10.1006/jado.2000.0326. 
76. 
CROSBY RA, SANDERS SA, YARBER WL, GRAHAM CA, DODGE B. Condom Use Errors 
and Problems Among College Men. Sexually Transmitted Diseases. 2002;29(9):552-7. PubMed 
PMID: 00007435-200209000-00010. 
77. 
Catania JA. The health protective communication scale. Davis CM, Yarber WL, Bauserman 
R, Schreer G, Davis SL, editors. Thousand Oaks, CA: Sage Publications;1998. 
78. 
Carey MP, Schroder KE. Development and psychometric evaluation of the brief HIV Knowledge 
Questionnaire. AIDS Educ Prev. 2002;14(2):172-82. Epub 2002/05/10. PubMed PMID: 12000234; 
PMCID: 2423729. 
79. 
National Institutes of Drug Abuse. NIDA-Modified Assist 2015. Availablefrom: 
file:///C:/Users/rb897/Desktop/APA_DSM5_Level-2-Substance-Use-Child-Age-11-to- 
17.pdf. 
80. 
Hennessy M, Bleakley A, Fishbein M, Jordan A. Validating an Index of Adolescent Sexual 
Behavior Using Psychosocial Theory and Social Trait Correlates. AIDS and behavior. 2008;12(2):321. 
doi: 10.1007/s10461-007-9272-1. PubMed PMID: PMC2819112. 
81. 
Tarnowski KJ, Simonian SJ. Assessing treatment acceptance: The abbreviated acceptability 
rating profile. Journal of Behavior Therapy and Experimental Psychiatry. 1992;23(2):101-6. doi:  
http://dx.doi.org/10.1016/0005-7916(92)90007-6. 
82. 
Min Y, Agresti A. Random effect models for repeated measures of zero-inflated count 
data. Statistical modelling. 2005;5(1):1-19. 
83. 
VanderWeele TJ, Chiba Y. Sensitivity analysis for direct and indirect effects in the presence of 
exposure-induced mediator-outcome confounders. Epidemiol Biostat Public Health. 2014;11(2):e9027. 
doi: 10.2427/9027. PubMed PMID: 25580387. 
84. 
VanderWeele TJ. Mediation Analysis: A Practitioner's Guide. Annual Review of Public 
Health. 2016;37(1):17-32. doi: 10.1146/annurev-publhealth-032315-021402. PubMed PMID: 
26653405. 
85. 
Cohen J. Statistical power analysis for the behavioral sciences: Jacob Cohen. J Am Stat Assoc. 
1988;84(363):19-74. 
86. 
Hedges LV. Distribution theory for Glass's estimator of effect size and related estimators. journal of 
Educational Statistics. 1981;6(2):107-28. 
87. 
Bauermeister JA, Golinkoff JM, Muessig KE, Horvath KJ, Hightow-Weidman LB. Addressing 
engagement in technology-based behavioural HIV interventions through paradata metrics. Current opinion 

Protocol Number: AAAT8624 
Version 1.5 
11/09/2023 
53 
 
 
in HIV and AIDS. 2017;12(5):442-6. doi: 10.1097/COH.0000000000000396. PubMed PMID: 28617711; 
PMCID: PMC5637536. 
88. 
Kreuter F. Improving surveys with paradata: Analytic uses of process information: John Wiley & 
Sons; 2013. 
89. 
Little RJA, Rubin DB. Statistical analysis with missing data. New York: Wiley; 1987.