Efficacy of Internet-based HIV Prevention
Phase: NA
Conditions: HIV, Gonorrhea, Chlamydia
Sponsor: Northwestern University

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 1 of 50
PROTOCOL DOCUMENT
Project Title:
Efficacy of Internet-based HIV Prevention
Principal Investigator:
Brian Mustanski, PhD
Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University
Co-Investigators:
George J. Greene, PhD
Northwestern University
Brent Johnson, PhD
Emory University
Jeffery Parsons, PhD
Hunter College
Patrick Sullivan, PhD
Emory University
Sponsor:
National Institute on Drug Abuse (1R01DA035145)
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 2 of 50
ABSTRACT
SIGNIFICANCE
Men who have sex with men (MSM) are the largest HIV transmission group in the US and infections are 
increasing most among young MSM (18-24 years), however, few HIV prevention programs have been 
developed and proven effective with this vulnerable population. To address these HIV prevention concerns, we 
collaborated with community-based organizations (CBOs) in the context of an NIH R34 grant to develop and 
pilot test, Keep It Up! (KIU!), an online HIV prevention program tailored to ethnically diverse YMSM. Our small-
scale pilot randomized clinical trial (RCT) demonstrated the feasibility and acceptability of the KIU! intervention: 
we enrolled an ethnically diverse sample of YMSM; achieved excellent retention through 3-month follow-up 
(89%); maintained robust intervention engagement; and produced a significant 44% decrease in unprotected 
anal sex relative to an active control. 
SCIENTIFIC OBJECTIVE
The overarching goal of the current proposal is to advance scientific knowledge of technology-based, 
behavioral HIV prevention, as well as improve public health by establishing the efficacy of an innovative 
eHealth prevention program for YMSM. 
SPECIFIC AIMS
We will accomplish these goals with three specific aims. First, we will integrate the KIU! intervention into a 
widely-used health technology platform to increase its scalability, adaptability, and potential for broad 
implementation. Second, we will test the efficacy of the KIU! eHealth intervention in a multisite RCT by: a) 
enrolling ethnically diverse YMSM (N = 900; > 65% racial/ethnic minorities) who tested HIV negative in the 
clinics of our community partners in Chicago, New York, and Atlanta. We anticipate that approximately 4500 
YMSM will have to be screened to enroll this number of YMSM; b) randomizing participants to either the KIU! 
intervention or an HIV knowledge control condition; and c) measuring intervention outcomes at baseline and 
follow-up assessments at 3, 6, and 12 months. The primary behavioral outcome will be the count of 
unprotected anal sex acts and the primary biomedical outcome will be STI incidence. And third, we will explore 
if the KIU! intervention has differential efficacy across important sub-groups of YMSM based on race/ethnicity, 
relationship status (serious versus casual relationships), and gay/bisexual identity. 
STUDY POPULATION
Ethnically diverse YMSM (N = 4532; 18-29 years; > 65% racial/ethnic minorities) who tested HIV negative in 
the clinics of our community partners in Chicago, New York and Atlanta. (This total sample includes 12 YMSM 
who will participate in acceptability and usability testing prior to implementation, and the 20 YMSM who will 
complete timed post-release testing.)  About 20% of these young men (n = 900) are anticipated to enroll into 
and complete the study. 
STUDY DESIGN
This is a two-group, active-control, RCT of the KIU! an online intervention.  Participants will be randomized to 
two groups in equal proportions (KIU! condition n=450; control arm n=450).  The KIU! intervention involves 7 
modules completed across three sessions, done at least 24 hours apart (i.e., at least 3 days), and totaling ~2 
hours to content. Across these modules, the KIU! intervention uses diverse delivery methods (e.g., videos, 
animation, games) to address HIV knowledge, motivate safer behaviors, teach behavioral skills, and instill self-
efficacy for preventive behaviors.  The control condition contains the same number of modules as the KIU! 
condition, with the same requirement to participate across three sessions.  It reflects HIV information that is 
currently available on many websites so as to understand how the KIU! intervention improves upon what is 
currently available online. Booster sessions will be delivered at 3 and 6 months for each arm.
PROCEDURES
Participants will be recruited (1) from the HIV testing clinics and mobile testing units of our partner CBOs in 
Chicago, New York, and Atlanta, (2) from HIV testing at Hunter College and Emory University, (3) from at-
home HIV testing, (4) from street outreach efforts of university staff and interns, (5) using print, online, and 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 3 of 50
recorded ads, (6) by referral from ongoing studies and IRB-approved research participant registries at the 
university sites, (7) by referral from community partners who provide HIV testing, and (8) from Chicago 
Department of Public Health (CDPH) clinics. Eligible participants across all study sites will be offered the 
opportunity to visit Emory University, Hunter College, or Northwestern University to complete the baseline 
assessment and complete their first set of self-administered STI test kits. Regardless of recruitment source, 
prior to beginning the intervention, participants will still have the contact with an HIV testing counselor or study 
team member that is important for recruitment and retention. They will be randomly assigned into one of two 
study groups, either the KIU! intervention or an HIV knowledge control condition.  They will be asked to 
complete online questionnaires (five questionnaires total, approximately 20 - 40 minutes each), complete an 
online intervention within 3 weeks, and maintain consistent online contact throughout the course of the study (a 
total of 12 months after enrolling).  This research includes at-home urine tests and at-home self-administered 
rectal swabs for STIs. Both STI tests will be collected at pre- and post-intervention (12 month follow-up).  
Participants who test positive at baseline will be tested at 3 month and 6 month follow-up in addition to at the 
12 month follow-up.
  
A. SIGNIFICANCE
A.1. The burdens of HIV/AIDS and STIs fall disproportionately on YMSM. Among all U.S. youth, YMSM 
accounted for 68% of HIV/AIDS diagnoses in 2008—approximately 3.5 times as many infections as among 
young women in all risk categories14. The CDC’s 2008 NHBS survey of MSM in 21 U.S. cities found the HIV 
prevalence to be 7% in 18 to 19 year-olds and 12% in 20 to 24 year-olds, and much higher among black 
YMSM (9% and 20%, respectively)15. HIV infections among YMSM are almost entirely transmitted via 
unprotected sex16. While less is known about the epidemiology of other sexually transmitted infections (STIs) 
among YMSM, evidence suggests the existence of health disparities17. For example, 20% of YMSM in the 
CDC’s Young Men’s Survey reported a prior STI diagnosis, which is higher than other groups of youth17.
As alarming as these epidemiological data are, there has not been a commensurate prevention 
response. The vast majority of HIV prevention programs in the published literature are focused on 
heterosexual adults and youth18,19 and older MSM20.   A recent meta-analysis of HIV behavioral interventions 
targeting MSM did not report a single RCT where the mean age was less than 2320 and there is no CDC best 
evidence intervention for YMSM13.  Given the paucity of HIV prevention research focused on YMSM, there is 
an urgent need to establish scientifically sound prevention programs for these high-risk young men.  
A.2. eHealth HIV prevention- A focus on impact. eHeath is the use of electronic technologies in health, 
health care, and public health21. The promise of eHealth interventions lie in their potential for intervention 
impact22,23, defined as the product of efficacy times reach (% of population receiving)24,25. While ideally an 
intervention will be both potent and widely implemented, the current arsenal of HIV prevention in the US 
includes a variety of individual and small group interventions that have proven efficacy for reducing HIV risk 
behaviors in adult MSM26-28 and heterosexual youth28-31 (but not YMSM), but whose reach has been limited by 
economic and structural barriers to implementation32-37.  Meta-analyses have reported significant effects of 
computer-based programs on HIV risk and protective behaviors that compared favorably to interventions 
delivered by human facilitators38-40.  And while computer- and Internet-based interventions have significant 
potential for overcoming barriers to cost-effective implementation37,41,42, the reality is that this potential has not 
yet  been realized by dissemination and implementation23.  A critically important barrier to dissemination is that 
fact that most prior eHealth HIV interventions have been built into an IT system that is idiosyncratic to the 
researcher’s environment, thereby limiting intervention portability, scalability, tailorability, and the ability to 
update and refresh as new technology emerges8.  To meet the bar of potential intervention impact, it is critical 
that eHealth interventions are designed with consideration of reach and future dissemination.  
Of particular concern is the ability of to reach the highest need groups, such as racial/ethnic minorities with 
online HIV prevention interventions43.  KIU! was designed for delivery upon receipt of a negative HIV test 
result; this innovative approach of linking an eHealth intervention to a clinical encounter was designed to 
increase reach to diverse and high risk YMSM because:
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 4 of 50
(a) Most urban high risk YMSM have recently received HIV testing (77% in prior 12 months; 58% in 
prior 6 months16) and this does not differ by race/ethnicity44.  HIV testing comprises a large proportion of the 
CDC’s prevention budget45; therefore, linking to this common biomedical service increases the potential for 
sustained intervention reach and integration into standard care, compared to programs that require their own 
recruitment infrastructure. 
(b) Linking to face-to-face HIV testing can maximize engagement of racial/ethnic minority YMSM who 
are at increased risk46. While frequent Internet use is the norm among young adults of all racial/ethnic 
groups47, there is evidence that Black YMSM are less likely to seek out HIV information online48 and with only a 
few exceptions49,50, prior Internet-based HIV behavioral studies of MSM have substantially under-enrolled 
racial/ethnic minorities43,51-55.  At the time we were developing the R34 intervention development grant for KIU!, 
our prior experience in conducting research with minority YMSM suggested to us that face-to-face recruitment 
by trusted HIV clinic staff could facilitate enrollment, even when the intervention was delivered online. Our 
preliminary data from the pilot RCT and mixed-methods research on this topic suggests that indeed this face-
to-face recruitment approach was successful in facilitating racial/ethnic minority YMSM participation46. In the 
course of the proposed study, we will establish the efficacy of this approach for implementing eHealth 
interventions that reach racial/ethnic minorities at the highest risk for infection.
(c) In addition to increasing intervention impact, linking to HIV testing fulfills an unmet need for greater 
prevention in the context of testing. Research on the effectiveness of voluntary counseling and testing (VCT) in 
reducing risk behaviors in those who test negative has been mixed56,57. Repeat negative test results are 
correlated with increased HIV risk behaviors in MSM58, perhaps because of perceived invulnerability to HIV 
enhanced by feelings of “dodging the bullet” after multiple negative results. This pattern of increased risk taking 
parallels the concept of “risk compensation” that has been a concern with biomedical prevention approaches59-
62. There is also evidence that prevention counseling is not routinely provided across settings63 and is highly 
variable in quality56. This is a missed opportunity for prevention with high risk YMSM and our goal is to 
transform this into an opportunity for health promotion by “capturing” YMSM in a “teachable moment” via 
technology that is accessible, engaging, and behavior changing. This study will also demonstrate a novel 
approach for linking an eHealth behavioral prevention intervention to a clinical encounter, and this knowledge 
can then be generalized to future integration of eHealth and biomedical prevention with other emerging 
technologies (e.g. PrEP).   
A.3. YMSM have unique HIV prevention needs. HIV prevention programs for YMSM should not be simple 
modifications of those developed for heterosexual emerging adults, as YMSM are impacted by unique cultural, 
contextual, and developmental factors18,64-67. YMSM share some common risk/protective factors with other 
young people, but also have unique experiences (e.g. family support is protective against HIV risk 68-70, but 
“coming out,” is unique to sexual minority youth).  The role of sociosexual contexts are also dramatically 
different for YMSM than their heterosexual peers (e.g. online/smartphone hook up sites)19,71.  Similarly, 
programs designed for older adult MSM may not generalize to the developmental context of emerging 
adulthood, the unique period during which individuals are “too old” to be adolescents (>18), but not “old enough” 
to be adults (~24)72-74. Psychosocial capacities that are dependent on brain development and which moderate 
risk taking—such as impulse control and resistance to peer influence—continue to mature through emerging 
adulthood75. These developmental factors are nested within sociocultural factors and relational factors, like 
having older partners76-78. Not surprisingly, risk factors such as alcohol consumption have been found to have 
differential effects across developmental stages in MSM79.  
A.4. The need for an effective intervention across subgroups of YMSM. YMSM are not a homogenous 
group and there is significant need for HIV prevention programming that works across subgroups. At the same 
time, it is important to consider that subgroups may not all respond the same to an intervention. We consider 
three such important dimensions: (a) Compared to their White peers in the U.S. in 2009, more than twice as 
many Black YMSM were diagnosed with HIV80 and Black and Hispanic MSM were more likely to become 
infected at younger ages4. While Black YMSM are most affected by HIV, there is significant need for 
prevention programming with White and Latino YMSM as these group of young men receive more HIV positive 
diagnoses than young women across all racial groups81.  While the causes of these racial disparities are poorly 
understood82-86, cultural factors may lead racial/ethnic groups to be differentially responsive to the content and 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 5 of 50
style of particular intervention71,87-89. (b) MSM is a behavioral grouping, not an identity90,91. Some HIV risk 
processes have been found to differ between gay and bisexual MSM92-94 and these differing identities may be 
linked to the acceptability and effectiveness of an HIV prevention program’s style and content. (c) About 30% 
of sexually active YMSM report being in a “serious” relationship.  Intriguing new findings from Mustanski and 
Sullivan show that among YMSM the majority of HIV transmissions and unprotected sex occur in “serious 
relationships”95,96.  Romantic relationships have only been addressed in minimal ways in most MSM-focused 
HIV prevention programs; therefore, the extent to which interventions work equally well across YMSM in the 
context of serious or casual partnerships is largely unknown. The KIU! intervention was designed to be 
responsive across these three dimensions of heterogeneity among YMSM, and in addition, an important aim of 
this grant is to explore moderating effects of these differences on intervention efficacy. If significant differences 
are found, this information can then be used to target the intervention to subgroups where it is most effective 
and indicate the need for future tailoring to meet the needs of other subgroups.  
B. APPROACH
B.1. Investigative Team. The interdisciplinary investigative team has complementary experience with eHealth 
research, developing and testing HIV interventions, distance testing for HIV and STIs, and research with 
YMSM. The team of investigators also has a track record of collaborating on prior studies and initiatives108-117, 
which will enable their distance collaborations (see also C.7). To prepare this application they held two in-
person meetings and multiple conference calls.  They have assembled a cadre of CBOs that perform high 
volume HIV testing with diverse YMSM that have agreed to serve as recruitment.
The NIH Social and Behavioral HIV Prevention Think Tank committee on technologies stressed the 
importance of behavioral scientists collaborating with computer scientists to maximize innovation and effective 
use of technology8.  This proposal benefits greatly from the collaboration of an internal Health Information 
Technology (HIT) group within the PI’s home department.  The HIT group is a team of computer scientists with 
expertise in software architecture, software development, systems management, and user support.  The HIT 
team created the Assessment Center (AC) platform in the context of an NIH roadmap initiative, and it has been 
used by over 3,000 primarily NIH-funded researchers (see  C.8.). By collaborating with this group the proposal 
will benefit from cutting-edge HIT expertise throughout the life cycle of the project, including development, 
implementation, and ongoing upgrades and system refinements.  
B.2. Formative Research. 
(a) Project Q2 (PI: Mustanski) is the longest-running longitudinal study of LGBT youth. At baseline, 
participants were ages 16-20 and the sample is extremely diverse (86% racial minorities). Using retention 
procedures similar to those in the proposed grant, we achieved 91% retention at the one year follow-up 
assessment82,95,118,119 and currently have 88% retention in the 3.5 year follow-up. We have reported on mental 
health119,120, alarmingly high frequency of condom errors (96% of condom users made at least one error in the 
past 6 months)121, family factors122, substance use118,123, racial disparities in HIV risk82, and romantic relationship 
factors95,124. Drs. Mustanski and Greene were recently awarded an NIMH grant (R21MH095413) to study the 
role of romantic relationships in HIV risk across development in this sample. 
(b) Through a William T. Grant Scholars Award to Dr. Mustanski, we have been conducting mixed 
methods research on the role of the Internet in the development of sexual health among LGBT adolescents. 
Qualitative results125,126 have been used to inform the design of the proposed booster session. This study has 
helped to identify racial minority barriers to accessing online HIV prevention content and suggests means for 
overcoming these barriers, which we have incorporated into the proposed study125,127. 
(c) Crew 450 (PIs: Mustanski & Garofalo; R01DA025548) is following an RDS-recruited128 cohort of 450 
ethnically-diverse YMSM (ages 16-20) over 2 years to characterize trajectories of syndemic development. 
Retention in the study has been excellent (85% at 1 year).  Data are available on urine sample tests for N. 
Gonorrhea (NG) and Chlamydia T. (CT) at enrollment and 1-year follow up, yielding estimates of STI 
prevalence = 10.2% and annual incidence = 8.9%.  HIV prevalence at enrollment was 7.8% (58% of these were 
newly diagnosed in the context of the study) and annual incidence is estimated at 2.2%. 
 
(e) The InvolveMENt study (PI: Sullivan; R01MH85600) is designed to identify reasons for Black/White 
disparities in HIV incidence and prevalence.  The study is enrolling MSM aged 18-39 from community venues in 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 6 of 50
Atlanta and is following them over 2 years with repeated STI and HIV testing.  High rates of annual incidence 
infections with chlamydia (11.9%) and gonorrhea (3.8%) have been documented among Black participants 
aged < 25 years.  Incidence estimates from this study and Crew 450 are used for power calculations in the 
current proposal. 
(d) Sullivan’s Checking In study (RC1MD004370) aimed to overcome barriers to online HIV prevention 
research with MSM, and utilized at-home HIV testing to accrue biological endpoints relevant to HIV prevention 
studies.  Formative research indicated a high level (83%) of hypothetical willingness among MSM to participate 
in at-home HIV testing129; however, some potential barriers were identified and these were mitigated in the 
design. This protocol resulted in 84% of men (N = 896) who agreed to receive an at-home test kit returning the 
dried blood spot specimen130. For men who tested HIV positive, results were available to men through the 
Home Access medical information center, which included a live telephone counselor and referrals to care (86% 
visited a provider within 3 months). For STI testing in the current proposal, the process of mailing a kit and 
having participants return a specimen by mail is similar to that successfully used for at-home HIV testing, but 
with a less invasive specimen collection process, so we anticipate comparably high rates of specimen return.
(e) The MSM Health and Behavior Survey (PI: Sullivan; Emory CFAR, P30 AI050409) is a periodic 
survey of internet-using MSM designed to measure internationally comparable indicators of recent sexual risk 
and use of prevention services among MSM in 7 countries. To understand willingness to test for an STI at 
home for the purpose of this proposal, a new survey was created to target US men on Facebook (aged 18-24 
years) who indicated they were interested in men. A total of 565 YMSM (97% identified as gay) consented and 
enrolled in the online survey.  The majority of the sample was White (72%) and 15% were Black or Latino. 31% 
of the sample had never tested for STIs.  Participants were asked about willingness to take a confidential, 
urine-based, at-home STI test as part of a university study at varying levels of hypothetical incentive;  75% of 
participants who were offered no incentive reported being somewhat likely or very likely to participate.  With 
incentives increasing from $10 to $40, willingness also increased from 81% to 86% (differences not 
significant).  This high level of acceptability parallels results for the at home HIV testing in the Checking In 
Study and support the feasibility of our proposed approach to distance testing for STIs.    
 
(g) Parsons’ Young Men’s Health Project (R01DA20366), is an RCT testing a 4-session motivational 
interviewing risk reduction intervention for non-treatment seeking HIV- YMSM in NYC with excellent retention 
(85% at 12-month follow-up). Baseline analyses have indicated that several domains addressed via KIU! 
modules are critical factors in predicting risky sex among YMSM. A desire for increased intimacy was shown to 
play a pivotal role in shaping negative attitudes about condoms and unprotected anal sex131. Identification with 
the gay community protects against HIV risk, in that YMSM who feel more connected report increased condom 
use132. Finally, analyses show that substance use and favorable attitudes towards unprotected sex predict 
sexual risk, and mental health moderated the relationship between partner type and condom use. 
(c) Dr. Parsons is PI of Project Click (R34MH08633), which is currently developing a culturally appropriate 
Internet-based intervention for bisexually active Black men. Through this project, Parsons and his team have 
developed strong collaborative relationships with the four community-based sites which have agreed to 
participate in the proposed study.
 (e) Drs. Sullivan and Mustanski are respectively leading the Atlanta and Chicago sites for a CDC-funded 
multisite study to develop and test an eHealth solution to recruit and educate MSM to self-administer rapid 
HIV tests. The solution includes components for recruitment, assessment, multimedia education about how to 
self-administer a rapid HIV test, randomization, and referral to care for those who test positive. 
B.3. The KIU! intervention was developed and pilot tested in the context of an NIMH R34.  The first year 
was dedicated to conducting mixed-methods research with ethnically-diverse YMSM to inform the intervention 
design and content48 and receive feedback from the target population. An initial screening of 951 diverse 
young men (52% racial/ethnic minority) in HIV testing clinics showed that among the 633 who had sex with a 
man in the last 6 months, 81% identified as gay, 17% bisexual, and less than 1% as heterosexual48, thereby 
establishing our intervention focus on YMSM that identify as gay or bisexual. Most YMSM had used the 
Internet to find HIV/AIDS information (88%). Subsequent qualitative interviews informed the development of 
content (e.g. identification of myths about HIV transmission) and refinement of messaging that was appealing 
across racial/ethnic groups107. Content was developed in collaboration with YMSM-serving CBOs. 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 7 of 50
The second year was dedicated to designing the online intervention and conducting usability testing with 
diverse YMSM. Several meta-analyses have identified key features of HIV primary prevention programs that 
predict efficacy among MSM20,133 which informed the KIU! design, including interpersonal/communication skills 
training, varied presentation formats, intervention delivery across more than one session, avoidance of formats 
that allow participants to meet offline, and use of a count outcome (e.g. number of unprotected sex acts). 
Finally, the literature supports the importance of using theory of health behavior change as the foundation for 
intervention20,134. KIU! was developed based on IMB model of HIV risk behavior change6,7, which has been 
used among YMSM105, and relapse prevention135, which focuses on identifying triggers for risk and the use of 
cognitive-behavioral strategies to break the link from triggers to risk20,136.   
The KIU! intervention involved 7 modules completed across three sessions, done at least 24 hours apart (i.e. 
at least 3 days), and totaling ~2 hours to content. An innovative aspect of KIU! was that each module was 
based on a particular setting  or situation relevant to the lives of YMSM, with developmentally appropriate 
health behavior change content embedded within each of these settings (see Appendix 1). Module content is 
designed to be sensitive and responsive to user bandwidth.  Across these modules, the KIU! intervention uses 
diverse delivery methods (e.g. videos, animation, games) to address gaps in HIV knowledge, motivate safer 
behaviors, teach behavioral skills, and instill self-efficacy for preventive behaviors.   
Module
Style
Content
1. Healthy and Whole 
Person
Diverse peer 
videos
Welcomes and engages participants in the intervention. Discusses connections 
to family, community, and romantic partners and setting positive peer norms for 
condom use and obtaining support from family of origin and choice68,69.
2. Hooking up Online
  
Stylized 
animation 
with 3 
scenarios
Follow three YMSM chatting online with a focus on identifying triggers for 
unprotected sex. Embedded content focuses on the effects of mood on 
risk137,138, negotiating correct condom use, consequences of drug and alcohol 
on decision making19, and facts about STI symptoms and prevention.
3. Sex in the City
Scripted 
soap opera 
style video
A diverse cast of YMSM highlight the risks in making assumptions about a 
partner’s HIV status or monogamy, the limits of serosorting in HIV negative 
YMSM when only ~20% of HIV+ YMSM know their status139, the importance of 
regular testing, and skills for negotiating condom use within relationships.
4. The Club Game
Virtual 
reality game 
Through interactive game play, participants address pros/cons of condom use, 
steps to correct condom use, consequences of excessive alcohol consumption 
or drug use, issues with presuming HIV status in others, and effects of sexual 
arousal on decision making19.
5. Dating 
(an Older Partner)
 
Flash 
animation 
quiz style
Explores the power dynamics between an older and younger man in a dating 
relationship, and how YMSM can assert healthy behaviors95. Embedded in the 
module is identification of a continuum of safer sex behaviors and strategies for 
implementing them.
6. A Serious 
Relationship
Illustrated 
story with 
flash and 
scripted 
scenarios
An illustrated story about dating and considers ways to get sexual, emotional, 
and health needs met in relationships and how ongoing condom use can be an 
important aspect of that. The module also includes a video of a YMSM who 
receives an HIV diagnosis while in a relationship. It wraps up with a video with 
actors portraying examples of good and bad communication about condom use.
7. Setting Risk 
Reduction Goals
Prevention 
goals 
worksheet
Participants develop a realistic and practical HIV and STI prevention plan.  
Suggested goals are tailored to risks reported in participants’ baseline 
assessment data. The purpose is to plan to prevent previous risky behaviors 
and to troubleshoot obstacles to successful implementation of the plan.
3-month Booster
Scripted, 
flash, and 
animation 
games 
scenarios
A chance to revisit goals, receive tailored feedback to troubleshoot obstacles, 
and set new goals or re-affirm existing ones. Included is a series of videos that 
follow a new character after he has a HIV scare and learns the importance of 
regular HIV testing and condom use. Also included is video follow-up from one 
of the characters from the Module 3 soap opera who, like the participants, 
received an HIV negative test result in the recent past and is working to 
maintain his risk reduction as some of the fear accompanying the test has 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 8 of 50
Table 1: Effects of KIU! on HIV risk behaviors at 3 months 
post-intervention
KIU! Control Adj. Effect Size
Unprotected Anal Sex Acts 
3.70
6.20
ERR = .56*
* p < .05
subsided (delivered at 1.5 months in pilot KIU! trial, now at 3 months). 
6-month Booster
Loosely 
scripted 
videos, 
animation, 
and game 
scenarios
A chance to revisit goals, receive tailored feedback to troubleshoot obstacles, 
and set new goals or re-affirm existing ones. Included is a series of videos that 
follow the character introduced in the 3 month booster as he navigates the 
dating scene and becomes involved in a serious relationship. The character 
learns the importance of having a partner who meets his emotional, physical, 
and health needs. Also included is information on stopping and restarting 
condom use in a relationship. They are links to review content from the original 
intervention modules.
The control condition contains the same number of modules as the KIU! condition, with the same 
requirement to participate across three sessions.  It reflects HIV information that is currently available on many 
websites so as to understand how the KIU! intervention improves upon what is currently available online. It is 
didactic, not tailored to YMSM, non-interactive, and focused on HIV/STI knowledge.  Using this approach as a 
control condition will ensure that both groups have equivalent access to the Internet for HIV-related content.  
In the third year, we conducted a pilot, single-site, RCT to demonstrate feasibility, acceptability, and 
intervention effect size. The pilot RCT documented our ability to recruit an ethnically diverse sample of 
YMSM from HIV testing clinics. A comparison to clinic records indicates nearly 100% compliance by 
indigenous testing staff with recruitment procedures.103 (85% of those invited) YMSM enrolled and were 
randomized (74% minorities). Our block randomization procedure was successfully automated by the 
computerized tracking system, with no significant differences in baseline demographic and behavioral variables 
across conditions.
Analytics related to engagement found high fidelity to intervention and control conditions (completion of all 
modules, mean time in intervention modules matched expected duration). Retention was excellent through 
the 3-month follow-up (87%).  Participant acceptability and interest in the intervention was very high. 
Ratings on the Intervention Acceptability Scale140 were very high. One participant said in an open-ended 
response, “this is the future of gay sex-ed.”  Participants connected to the characters in the intervention 
modules, with one saying, “I wasn't completely aware of my behavior until I judged a character's behavior and 
then compared the same behavior to my own.” At each login, participants were asked about the setting they 
were in; on average they rated the setting as very private and minimally distracting.  The majority of 
participants completed some or all of the intervention from home (98%). At each assessment, participants were 
asked to report any adverse experiences resulting from participation and none were reported. 
In addition to the primary goal of providing evidence of acceptability and feasibility, analyses of the RCT pilot 
data found positive intervention effects. As shown in Table 1, controlling for baseline levels, KIU! showed a 
significant 44% reduction in the rate of unprotected anal sex, compared to the control arm, among sexually 
active YMSM. This effect size is larger than the mean 
effect reported in a recent meta-analysis of offline HIV 
prevention programs delivered to adult MSM26. Promising 
intervention effects were also found for the behavioral 
outcomes of condom errors and failures, which previously 
have been reported to be very high among YMSM121 and associated with STI infection141.  HIV knowledge, 
health protective communication skills, and intentions to use condoms increased in both the KIU! and control 
conditions, reflecting the fact that the control condition was an active HIV knowledge arm. This pattern 
suggests that both arms had positive effects on attitudinal and knowledge outcomes, but KIU! was significantly 
better in terms of behavioral outcomes. 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 9 of 50
Our data on recruitment of a diverse sample of YMSM, successful retention, high engagement and 
acceptability, and positive intervention effects highlight the promise of KIU! and the value in evaluating it in a 
larger scale efficacy trial. We have also been successful in disseminating findings of the R34 grant, which 
already include two published papers46,48, one paper under review142, and 11 academic presentations127,143-152. 
B.4. This application requests funds to conduct a multi-site randomized clinical trial (RCT) of KIU!  To 
achieve aim 1, months 1-6 will be spent programming new booster sessions (for 3 and 6 months post-
intervention) and conducting technological upgrades in order to bring it to scale in the Assessment Center 
platform.  To achieve aim 2, in months 7-54 we will recruit ethnically diverse young MSM (N=4500; >65% 
racial/ethnic minorities) that test HIV negative from the clinics of our community partners in Chicago, New York, 
and Atlanta. About 20% of these young men (n = 900) are anticipated to enroll into and complete the study. 
We will randomize participants to either KIU! or the existing HIV knowledge control  condition similar to existing 
web-based HIV educational materials. We will measure self-report intervention outcomes at baseline and 
follow-up assessments at 3, 6, and 12 months and STIs at baseline and 12 months. Participants who test 
positive at baseline will also be tested at 3 month and 6 month follow-ups in addition to the 12 month follow-up. 
Months 55-60 will be dedicated to analyses and dissemination. We will test for dose effects and decay in 
intervention effects across the 12 months of follow-up data. To achieve aim 3, we will explore moderators of 
intervention effects, including race/ethnicity, relationship status, and sexual identity.  Our proposed project 
seeks to advance public health by testing the efficacy of an innovative HIV prevention program for a high risk 
and underserved population (YMSM) and advance scientific understanding of how to utilize eHealth solutions 
to reduce HIV/STI risk behaviors and link eHealth intervention to clinical encounters. 
B.5. Justification for a multisite RCT.  The completed single-site, pilot RCT of the KIU! intervention provides 
preliminary evidence of the potential of the program. Conducting a multi-city trial of KIU! is a logical next step 
because: (a) recruitment across three cities allows for sufficiently rapid enrollment of diverse YMSM during the 
project timeframe; (b) greater generalizability of findings to YMSM in distinct geographic areas (Midwest, 
Southeast, and East Coast); (c) collaborations with diverse CBOs allows for greater variability in context, which 
can lead to identification of possible obstacles to implementation across settings that can be mitigated prior to 
future dissemination; (d) the size and epidemiologic trends of the HIV/AIDS epidemic in Chicago and New York 
makes them good candidates for conducting a RCT among YMSM.  Both are sites for the National HIV 
Behavioral Surveillance System, which targets metropolitan areas hit hardest by the disease.  In both cities, 
male-to-male sexual contact is the primary mode of transmission153,154 and youth are increasingly bearing the 
brunt of the epidemic154.  These cities represent distinct geographical regions with notable differences in 
population size and culture, but both are highly ethnically diverse (City of Chicago: 2.7M, 35% Black, 28% 
Latino; New York City: 8.3M, 25% Black, 28% Latino)155. 
B.6. Justification for STI incidence as a biological endpoint. STIs are important risk factors in the 
transmission and acquisition of HIV infection due to increased biological susceptibility156,157. Randomized trials 
suggest that STI control may have beneficial effects on HIV incidence in populations with high STI rates and 
growing HIV epidemics156,158 such as with YMSM19 (with limited effects in generalized epidemics158,159).  As 
such, there has been a call to incorporate STI testing and treatment into HIV prevention programs at the 
individual and community levels160-162.  At the same time, STI infections can serve as sensitive biomarkers in 
efficacy trials of sexual risk reduction interventions, particularly when HIV infection rates are too low to allow 
sufficient power with attainable sample sizes163-167. In the current study, HIV incidence is not viable as a 
biomedical endpoint because participants are recruited upon receiving an HIV negative test result and, given 
the annual incidence of infections among tested MSM168,169, a sample of >4,500 would be required to identify a 
moderate intervention effect, which would exceed budgetary allowances. 
We propose to test for urethral and rectal Gonorrhea (NG) and Chlamydia (CT) and refer for treatment in both 
arms of the trial. Joint NG/CT incidence will serve as a biomedical endpoint for establishing intervention 
efficacy and as a means for determining the feasibility of incorporating an innovative approach to STI testing 
into an online HIV prevention solution. In addition to urethral NG/CT, we will test for rectal NG/CT because 
recent research has shown rectal infection to be just as common, if not more, than urethral infections. Study 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 10 of 50
Co-I, Patrick Sullivan, Ph.D., is part of an Atlanta team that recently analyzed STI incidence of local MSM. 
Black YMSM had a urethral chlamydia incidence rate of 10.5 per 100 PYs but a rate of 22.9 per 100 PYs for 
rectal chlamydia; their incidence rate of urethral gonorrhea was 4.4 per 100 PYs but 20.2 per 100 PYs for 
rectal gonorrhea. White YMSM had a urethral chlamydia incidence rate of 4.5 per 100 PYs but a rate of 6.7 per 
100 PYs for rectal chlamydia; their incidence rate of urethral gonorrhea was 1.5 per 100 PYs but 4.4 per 100 
PYs for rectal gonorrhea215. We focus on NG/CT because: (1) they are the most common STIs among young 
adults in the US81; (2) they are the principal STIs that amplify HIV infection among MSM in the US170; and (3) 
there are established solutions for distance screening that are reliable and acceptable to users171.
B.7. Subjects: We will recruit an ethnically diverse sample of YMSM between the ages of 18 and 29 (N= 
4532). ). (This total sample includes 12 YMSM who will participate in acceptability and usability testing, and the 
20 YMSM who will complete timed post-release testing.) About 20% of these young men (n = 900) are 
anticipated to enroll into and complete the study.  
Participants will represent an equal proportion of Black, Latino, and White YMSM, consistent with both high-
risk YMSM populations in each city as well as the demographics of the population seeking services at each 
recruitment site. Given city-specific testing patterns, we will readily achieve representation across groups, and 
we will stratify enrollment to equal 1/3 for each group. 
Study inclusion criteria include: (1) MSM, defined as a birth male who identifies as male and who self-
reports having sexual contact with another male in the past year; (2) 18-29 years old; (3) report unprotected 
anal sex with another male in the last 6 months. Consistent with recommendations for assuring sufficient event 
rates in RCTs172, this will enhance ability to detect intervention effects; and (4) are able to read English at a 
minimum 8th grade level, because the measures and intervention are delivered in English.  YMSM will be 
excluded from participation if they: (1) do not have an e-mail address that can be used to contact them for 
retention purposes; (2)  are currently in a behaviorally monogamous relationship lasting longer than 6 months 
(regardless of partner’s perceived HIV status). We use this criterion because we believe safer sex interventions 
for MSM in long-term monogamous relationships will require substantial tailoring from those who have casual 
sex or in the early phases of a romantic relationship.  Our prior research with YMSM indicates high rates of 
unprotected sex in the early months of a romantic relationship and we will explore relationship status (casual 
versus early stage serious relationship) as an effect modifier95; or (3) Are HIV+. We have elected to exclude 
HIV+ individuals because we believe that secondary prevention messages need to be different than primary 
prevention messages173.  If a participant seroconverts during the follow-up waves, he will be allowed to 
continue participating and be referred to care (see Human Subjects).
B.8. Plan for long distance collaboration and project management.  Our experience conducting several 
multisite research projects has taught us the importance of regular live- and tele-communication, a clear 
division of roles, and well-developed protocols. As described in detail in the budget justification, the PI and/or 
Project Director will travel to each performance city once per year and there will be annual investigator 
meetings (in rotating cities). We will make extensive use of technology (i.e., e-mail, shared server storage, 
videoconferencing, performance dashboards) to allow for project management and collaboration across sites.  
All tracking and retention activities will be centrally managed in Chicago where the tracking technology is 
based. The research study staff will provide local referrals to treatment and the Atlanta and New York based 
project coordinators will assist with retention of difficult-to-track participants. Drs. Mustanski, Parsons, and 
Sullivan have been working collaboratively for 6 years, and thus have a long history of effective distance 
communication.
B.9. Phase 1: Technological upgrades to the KIU! intervention and tracking interface in Months 1-6. 
The OAR Social and Behavioral HIV Prevention Research Think Tank highlighted the ongoing issue in eHealth 
HIV prevention research that the majority of interventions are built into a static IT system that is idiosyncratic to 
the researcher, thereby limiting intervention portability, scalability, tailorability, and the ability to update and 
refresh as new technology emerges8. Our proposal is highly responsive to this concern as we propose to 
integrate the KIU! intervention into the widely used and supported Assessment Center (AC) platform designed 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 11 of 50
for measuring patient reported outcomes in health research and care11,12.  AC is built upon a technologically 
rich, reusable software platform that can be accessed, managed, and shared by researchers. A rigorous 
software development process174-176 was utilized in order to create a system that would meet end-user needs 
and have a high potential for being adopted by the clinical research community. NIH has invested heavily in 
projects that use the AC platform (PROMIS - $80 million; NIH Toolbox $25 million; NeuroQOL $7 million, plus 
dozens of smaller projects) and it is being utilized by thousands of NIH funded researchers, eight foreign 
countries have user groups, and it is being adopted in a growing number of academic medical centers (e.g. 
Cleveland Clinic, Northwestern Medicine), and community clinics. The AC platform offers a number of features 
that make it desirable for eHealth interventions, including a participant enrollment/tracking system, a feature 
rich participant data collection engine, and interface that makes it easy to update module content.  We already 
piloted the integration of several KIU! modules into the AC platform to assure functionality.  Ongoing 
development (e.g. linkage to electronic health records; text messaging), extensive support, and ongoing 
investment from NIH ensure that it will be the platform to use not just today, but also well into the future. 
After already reviewing each KIU! session with Atlanta’s and New York’s diverse investigative teams (which 
includes YMSM staff) and community site collaborators, the consensus was to adapt the first module to include 
local videos to strengthen relevance across settings (Midwest, Southeast, East Coast) and we will implement 
this as a technological solution that could readily be done in future locations to increase local cultural 
relevance.  During month 2, new videos will be shot in New York and Atlanta that are similar in content to 
current KIU! videos using the existing “man on the street” style. In month 3, the videos will be edited by internal 
production staff.  All other content was deemed by local investigative and community teams to be relevant 
across geographical location. 
During months 2-5 the investigative team will program new booster content for 3 and 6 months post 
intervention. This will supplement the booster already implemented for 3 months post-intervention. Many HIV 
prevention investigators177-179 have suggested the importance of providing boosters over time to strengthen the 
impact of the original intervention. However, boosters have been rarely implemented in offline interventions, 
likely due to the prohibitive costs of tracking participants and delivering intervention content.  As evidence of 
this, a meta-analysis of 43 unique HIV prevention trials among MSM reported only 2 with identified boosters26.  
Boosters can be cost-effectively delivered in eHealth interventions, and a recent online HIV prevention trial 
suggested the need for boosters as intervention effects degraded by 3 months without them 180. 
Formative research for KIU! and subsequent interviews with diverse YMSM in the WT Grant study (see 
Preliminary Studies) were used to inform the content of the 6-month booster. The IMB model7 guided our 
development of this session. The boosters provide tailored feedback on achievement of risk reduction goals, 
enhance motivation for risk reduction, and opportunities to strengthen HIV prevention skills. The 3 month 
booster also focuses on the importance of repeat HIV testing following the CDC’s recommendation of twice 
annual HIV testing among MSM 181, which is consistent with our aim to examine effects of the intervention on 
follow-up HIV testing. The 6 month booster focuses on healthy relationships. 
We have created additional booster control sessions to match the number of modules in the updated 
intervention. Quickly-assembled prototypes of boosters will be generated initially to test technical and 
intervention assumptions.  Once module development is complete, a tabletop review (or “Beta version”) will be 
led by professional quality assurance staff to ensure that the intervention functionality operates properly and 
that the content is accurate, complete and clear.  We will also conduct usability and acceptability testing with 
12 ethnically diverse YMSM (4 Caucasian, 4 Black, 4 Latino).  An established usability testing protocol will 
assess usability in practice (i.e., across different monitors, bandwidths, etc.) and allow participants to “think out 
loud” as they go through the modules in order to identify navigation and usability issues. Further revisions will 
be performed as necessary.
We will implement the revised KIU! intervention and take the modules live during month 6.  End-user technical 
support will go into effect.  The modules will be ready for participant enrollment. During month 6 we will host a 
multi-site investigative team meeting to train project staff on all recruitment, screening, and enrollment 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 12 of 50
procedures. FAQs, flowcharts, and web-based videos will be distributed for training of indigenous CBO clinic 
staff who will perform enrollment activities. 
B.10. Phase 2: A two arm multi-site RCT will be conducted in Months 7 - 54.  
(a) Recruitment. The team in each city has assembled a cadre of CBOs whose indigenous staff will 
recruit YMSM upon a negative HIV test result (see letters of support from all community partners). We have 
partnered with CBOs for recruitment in order to maximize our ability to test the intervention under conditions in 
which it would eventually be disseminated.  Based on our prior data collection and the need to stay within 
budgetary caps, we project enrollment of 22 YMSM per month. Based on our pilot data and reported 
enrollment rates of other recent similar HIV prevention trials varying from 15 - 45%182-187, we conservatively 
project the RCT enrollment rate to be at least 20% of YMSM screened (inclusive of eligibility requirements). 
Therefore we anticipate needing to screen ~4500 YMSM across all years.  More than this number of tests were 
performed annually across these clinics, but many of these visits represent YMSM repeating testing. Factoring 
this in, along with our plan for equal representation by racial/ethnic groups and equal enrollment across cities, 
we project we will be able to readily achieve recruitment goals.
(a2) CBO Recruitment. At the conclusion of their post-test appointment or rapid-test, YMSM who test HIV 
negative will be given a tablet computer by the staff member who conducted the HIV test (with paper-and-
pencil backup as necessary).  In many cases this will be a nurse, medical assistant, or other individual certified 
to conduct HIV testing. Clients can also be screened online on the Assessment Center platform or offline on 
the iFormBuilder app. The staff member will be trained in appropriate and ethical methods of recruiting 
participants in medical settings. To minimize the risk for coercion, the staff member and the study information 
sheet/screen emphasize the optional nature of participation and that it will not affect, in any way, their access 
to healthcare services. These recruitment procedures were previously approved by the IRBs at the University 
of Illinois at Chicago, implemented in the KIU! pilot study, and no adverse events were reported.
  
The tablet (or paper form) will display information about the study, ask the potential participant whether or not 
he wants to consider participation, and ask demographic questions for screening purposes.  If the participant is 
not interested, the tablet (or paper form) will ask questions assessing reasons for non-participation.  This will 
allow us to test for potential biases in enrollment. The recruiter will not be asking these questions and the tablet 
(or paper form) will indicate that answering these questions are voluntary. As such, the potential participant 
may simply choose to skip these questions.  For participants who are determined to be eligible, the device will 
ask for contact information.  For participants who are determined to be not eligible, the device will display 
ineligibility information and ask the participant if he would like to consent to enter his contact information for 
consideration in future studies. When paper and pencil are used, participants will provide their contact 
information if they are interested in participating in Keep It Up! or other studies; they will not be aware of their 
eligibility at the time. The tablet (and paper form) includes a question in the contact information section asking 
for consent to add participants to the IMPACT database. All subjects will have to consent to being added to the 
IMPACT database before research staff approaches them about participating in other studies. Information 
collected on the tablet through Assessment Center will be securely uploaded to the tracking system.  
Information collected on the tablet using the iFormBuilder app will be exported to a CSV file and either the 
information will be uploaded to Assessment Center or manually entered by research study staff.  Information 
collected with paper and pencil will be manually entered by research study staff into Assessment Center within 
3 days of receipt. This procedure allows for rigorous tracking of recruitment and participation rates in a timely 
manner. YMSM interested in participating will be instructed to expect an e-mail with more information about the 
study and offered local sources of private Internet access if they do not have their own.  The Project 
Coordinators based at Northwestern, Hunter, and Emory will be on hand to provide ongoing training and 
support to testing staff to assure easy compliance with this recruitment procedure. Brief videos, FAQs, and flow 
charts will be provided to testing staff to train them on the recruitment procedures.
Potential participants will be sent a recruitment e-mail to the address entered into the tablet. Based on 
research by Co-I Sullivan on successful recruitment and retention of racial minorities in online HIV behavioral 
research, we will specifically request email addresses that are used to manage finances and/or checked daily 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 13 of 50
188.  Non-response to the recruitment e-mail after 4 days will trigger the retention protocol (see attached 
Contact and Retention Protocol) that will include further e-mails, phone calls if a valid phone number was 
provided, and a text message (if consent provided). Following recommendations for Internet-based research 97, 
inclusion criteria will not be posted, but instead measured directly, in order to reduce potential for participant 
deception in order to gain incentives.
(a3) Expanded Recruitment. Recruitment from the CBOs will be supplemented by ads, street outreach, 
referrals of participants from ongoing studies and participants who have consented to being added to IRB 
approved research participant registries at the university sites, referrals from the Chicago Department of Public 
Health (CDPH) clinics, and referrals of participants from community partners who provide HIV testing. In 
addition to the CBO sites, Emory University and Hunter College will offer HIV testing to participants recruited 
through these expanded recruitment methods. Participants who visit these university sites will be tested by 
trained research staff and will join the study in the same way they would if they were testing at a CBO.
Staff members across all testing sites will be trained in appropriate and ethical methods of recruiting 
participants in medical settings. To minimize the risk for coercion, the staff member and the study information 
sheet/screen emphasize the optional nature of participation and that it will not affect, in any way, their access 
to healthcare services. These recruitment procedures were previously approved by the IRBs at the University 
of Illinois at Chicago, implemented in the KIU! pilot study, and no adverse events were reported.
Northwestern University and Emory University will also offer at-home HIV testing to participants recruited 
through these expanded recruitment methods. Staff members across all testing sites will use the skills taught 
during their HIV testing and counseling training to help participants cope with any distress related to their test 
results. When necessary, staff members will work with the participant to link them to care and will be available 
to provide counseling support following established organizational procedures.  
1.
Recruitment through online, print, and telephone ads. The KIU! research assistants will 
place online recruitment ads on popular sites such as Facebook, Grindr, and Craigslist. 
Additionally, RAs will send private messages to young men on Facebook who “like” or “share” 
the ads that we post on Facebook. These messages will encourage the young men to click the 
link in the ad and complete our online screener if they have not already done so. Samples of the 
messages we will use are included in the “Recruitment-Online & Telephone Ads samples 
5.1.14” document. The RAs will also place recorded recruitment ads on phone party lines like 
Talkee’s Chicago Blade line where YMSM can chat real time in a group or one-on-one or leave 
personal ads for being contacted privately. Print ads will be created by the study RAs and 
distributed to partner CBOs as well as handed out at events where the study target population is 
in attendance. Participants recruited through these ads will be directed to a study webpage for 
detailed project information and a brief eligibility screener. Interested participants will provide 
contact information (name, phone, email address) and complete the screener. 
For those who screen ineligible, the online screener will ask the participant if he would like to 
consent to enter his contact information for consideration in future studies. All subjects will have 
to consent to being added to the IMPACT database before research staff approaches them 
about participating in other studies. Information collected from the online screener will be 
exported to a CSV file and either the information will be uploaded to Assessment Center or 
manually entered by research study staff within 3 days of receipt. 
For those who screen eligible, the participant will be asked to confirm his HIV negative status by 
(1) taking an HIV test at a partner CBO testing site or university site (Emory and Hunter) or (2) 
taking an at-home HIV test provided by a university site (Emory and Northwestern).
Participants that agree to take an at-home HIV test will be mailed an FDA approved at-home 
test kit. Participants will be offered the option of testing with Oraquick (oral swab) or Home 
Access (finger stick). To join the study with an Oraquick at-home HIV test, participants will be 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 14 of 50
required to interpret their test result by comparing their test stick to the pictures and descriptions 
on test kit directions. After determining their test result, the participant will need to call study 
staff and report their test result. 
To join the study with a Home Access at-home specimen collection kit, participants will be 
required to mail back their blood specimen to Home Access for testing. Participants must use 
the pre-paid and pre-addressed envelope provided by Home Access to return their blood 
sample. After shipping their blood specimen, participants will call a toll-free number after seven 
days to receive their test results and post-test counseling. Participants will also be asked, during 
the online screener, to consent to allowing Home Access to share their results with research 
staff. HIV results will be shared between Home Access and Northwestern University via SFTP.  
University staff will use this information to confirm participants’ eligibility for the study. 
Participants who agree to take either at-home HIV test kit will also be mailed at-home urethral 
and rectal STI test kits.
 
For individuals with HIV negative test results. 
(a) At CBO sites: The testing counselor will provide the participant with a tablet (or paper form) 
to complete the full screener and confirm eligibility. If eligible, an enrollment email with a link to 
the study to be sent to the participant’s email address. 
If ineligible, the tablet (or paper form) will ask the participant if he would like to consent to enter 
his contact information for consideration in future studies. When paper and pencil are used, 
participants will provide their contact information if they are interested in participating in Keep It 
Up! or other studies; they will not be aware of their eligibility at the time. The tablet (and paper 
form) includes a question in the contact information section asking for consent to add 
participants to the IMPACT database. All subjects will have to consent to being added to the 
IMPACT database before research staff approaches them about participating in other studies. 
Information collected on the tablet through Assessment Center will be securely uploaded to the 
tracking system.  Information collected with paper and pencil will be manually entered by 
research study staff into Assessment Center within 3 days of receipt.
(b) At university sites (Emory and Hunter): The testing counselor will provide the participant with 
a tablet (or paper form) to complete the full screener and confirm eligibility. If eligible, an 
enrollment email with a link to the study to be sent to the participant’s email address. The testing 
counselor will encourage the participant to complete the baseline assessment and first set of 
self-administered STI test kits at the university site. Upon completion, the participant will be 
compensated $50 in cash. 
If ineligible, the testing counselor will also let the participant know that he is ineligible for KIU! 
but may be eligible for other studies. If interested in other studies, university staff will collect 
additional contact info using the tablet (or paper form) to enter into the IMPACT database. The 
participant will also receive $10 in cash for his travel to the university site.
(c) At-home HIV testing: If the participant tests negative, the project coordinator will call the 
participant to do a full screening over the phone and confirm eligibility. If eligible, an enrollment 
email with a link to the study to be sent to the participant’s email address. Upon completion of 
the baseline assessment, the participant will be compensated $30 in Citi Visa gift cards. 
If ineligible, the project coordinator will let the participant know that he is ineligible for KIU! but 
may be eligible for other studies. If the participant is interested in other studies, the project 
coordinator will collect additional contact info and to enter it into the IMPACT database. All 
subjects will have to consent to being added to the IMPACT database.
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 15 of 50
For individuals with HIV positive test results. 
(a) At CBO sites: The testing counselor will work with the participant to link them to care and will 
be available to provide counseling support following established organizational procedures.  
Linkage-to-care will include referring the participant to a clinic that will conduct a free 
confirmatory HIV test, and to someone that will work with them receive treatment if their 
confirmatory test returns positive. The testing counselor will also let the participant know that the 
KIU! research assistant will follow-up with them about their eligibility for the study. 
At the conclusion of the appointment or rapid-test, the testing counselor will call the KIU! 
research assistant and confirm that the participant has tested positive. The KIU! research 
assistant will email the participant to let him know that he is ineligible for KIU! but may be 
eligible for other studies. If the participant is interested in other studies and consents to being 
added to the IMPACT database, the KIU! research assistant will collect additional contact 
information from him.  
(b) At university sites: 
Emory – The testing counselor will use the skills taught during their HIV testing and counseling 
training to help participants cope with distress after receiving a positive test result. The testing 
counselor will work with the participant to link them to care and will be available to provide 
counseling support following established organizational procedures.  Linkage-to-care will 
include referring the participant to a clinic that will conduct a free confirmatory HIV test, and to 
someone that will work with them receive treatment if their confirmatory test returns positive. 
Participants under 24 years old will be referred to the Ponce de Leon center at Grady while 
those who are 24 years and older will be referred to their county health department.  After the 
participant has been linked to care, the testing counselor will give the participant $50 in cash for 
travel and time spent being counseled and linked to care. The testing counselor will also let the 
participant know that he is ineligible for KIU! but may be eligible for other studies at Emory. 
Hunter – The testing counselor will use the skills taught during their HIV testing and counseling 
training to help participants cope with distress after receiving a positive test result. The testing 
counselor will work with the participant to link them to care and will be available to provide 
counseling support following established organizational procedures.  Linkage-to-care will 
include referring the participant to a clinic that will conduct a free confirmatory HIV test, and to 
someone that will work with them receive treatment if their confirmatory test returns positive. 
Participants may be referred to the Mount Sinai Comprehensive Health Program, Callen-Lorde 
Community Health Center, or Gay Men’s Health Crisis Prevention Center. The testing counselor 
and a senior staff member will walk or take a cab with the participant to the HIV confirmatory 
testing clinic.  Before taking the trip to the confirmatory clinic, the testing counselor will give the 
participant $50 in cash for travel and time spent being counseled and linked to care. The testing 
counselor will also let the participant know that he is ineligible for KIU! but may be eligible for 
other studies at Hunter. 
(c) At-home HIV testing: If the participant tests positive using an Oraquick test kit, the project 
coordinator will work with the participant to link them to care and will be available to provide 
counseling support.  The project coordinator will use established organizational linkage-to-care 
procedures that have been used in other IRB approved studies. Linkage-to-care will include 
referring the participant to a clinic that will conduct a free confirmatory HIV test, and to someone 
that will work with them to receive treatment if their confirmatory test returns positive.  In Atlanta, 
participants under 24 years old will be referred to the Ponce de Leon center at Grady while 
those who are 24 years and older will be referred to their county health department.  In Chicago, 
participants will be referred to a linkage-to-care specialist at Center on Halsted. In New York, 
participants may be referred to the Mount Sinai Comprehensive Health Program, Callen-Lorde 
Community Health Center, or Gay Men’s Health Crisis Prevention Center. For participants who 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 16 of 50
live outside of Atlanta, Chicago, and New York, study staff will use the AIDS.gov service locator 
to make referrals to the nearest health care provider.
If the participant tests positive using a Home Access specimen collection kit, trained counselors 
employed by Home Access will provide counseling support and linkage to care. Trained study 
staff at the university sites can provide additional support and linkage to care as necessary. 
University staff will be responsible for immediately reporting positive HIV test results to the 
appropriate health department. 
2.
Recruitment through street outreach. Prior to recruiting through street outreach, the research 
team will obtain permission from management or owners of venues to recruit for study 
participants. Once permission is granted, university staff and interns will approach potential 
participants in various venues that cater to YMSM (e.g. bars, clubs, bathhouses, and festivals), 
briefly explain the purposes of the study, and invite them to answer questions that screen for 
preliminary eligibility. Individuals will answer questions privately on a tablet (or paper form if a 
tablet is not available). 
Those deemed eligible after completing the screener will be asked for their contact information, 
and will be contacted over the phone or email shortly after initial contact. University staff will 
offer information about the study over the phone and options for confirming their HIV negative 
status. If a participant wants to visit a partner CBO testing site or university site (Emory and 
Hunter), university staff will schedule an appointment for HIV testing. If a participant wants to 
take an at-home HIV test, university staff will mail out a FDA approved at-home HIV test. 
After taking an at-home HIV test or testing at a partner CBO testing site or university site (Emory 
and Hunter), participants will follow the same procedure as described under “Recruitment 
through online, print, and telephone ads.” 
For those who screen ineligible, the device will display ineligibility information and ask the 
participant if he would like to consent to enter his contact information for consideration in future 
studies. When paper and pencil are used, participants will provide their contact information if 
they are interested in participating in Keep It Up! or other studies; they will not be aware of their 
eligibility at the time. The tablet (and paper form) includes a question in the contact information 
section asking for consent to add participants to the IMPACT database. All subjects will have to 
consent to being added to the IMPACT database before research staff approaches them about 
participating in other studies. Information collected on the tablet through Assessment Center will 
be securely uploaded to the tracking system. Information collected with paper and pencil will be 
manually entered by research study staff into Assessment Center within 3 days of receipt.
3.
Recruitment through community partner referrals. Participants recruited through referrals 
from community partners will be given a business card with a pre-generated screener code and 
a link to an online screener after testing HIV negative. The screener codes on these business 
cards will only be valid for three months from the date that they are given to community 
partners. New batches of business cards with valid screener codes will be regularly distributed 
to partner sites. Once the screener code expires, the participant will no longer be able to use it 
as verification that they have recently tested HIV negative. 
The participants will use the link to access a study webpage for detailed project information and 
a brief eligibility screener. Interested participants will provide contact information (name, phone, 
email address) and complete the screener, including the screener code as proof that they have 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 17 of 50
recently tested negative at a community partner site. If eligible, they will be sent an automatic 
enrollment email with a link to the KIU! study.
For those who screen ineligible, the online screener will ask the participant if he would like to 
consent to enter his contact information for consideration in future studies. All subjects will have 
to consent to being added to the IMPACT database before research staff approaches them 
about participating in other studies. Information collected from the online screener will be 
exported to a CSV file and either the information will be uploaded to Assessment Center or 
manually entered by research study staff within 3 days of receipt. 
4.
Recruitment through referrals from ongoing studies and research participant registries. 
Participants who become interested in the study through referrals from ongoing studies or IRB 
approved research participant registries at the university sites will be directed to a study 
webpage for detailed project information and a brief eligibility screener. Interested participants 
will follow the same procedure as described under “Recruitment through online, print, and 
telephone ads.” 
5.
Recruitment through CDPH clinics.  At the conclusion of an HIV testing appointment, the 
CDPH staff member will give male clients who have tested HIV negative, and appear to be 
between the ages of 18 and 29, a brief description of the study. They will mention to the client 
that CDPH is working with Northwestern University researchers on the Keep It Up! study as a 
way to learn more about the health behaviors and outcomes of young men who have sex with 
men (YMSM). The CDPH staff member will briefly mention features of the study, such as it 
being a paid, online HIV prevention research study. To minimize the risk for coercion, the CDPH 
staff member will emphasize the optional nature of participation and that it will not affect, in any 
way, their access to healthcare services. The CDPH staff member will ask the client if he is 
interested in hearing more details about the study and possibly screening to see if he is eligible. 
If the client indicates that he is interested, the CDPH staff member will walk the client over to a 
Keep It Up! staff member who is stationed in another private room at the clinic. CDPH staff 
contact with participants will be limited to the referral of participants to the study and will not 
include performing informed consent or enrolling participants into the study.
The Keep It Up! staff member will give more details about the study (e.g. randomization, 
timeline, and study components). To minimize the risk for coercion, the staff member will 
reiterate the optional nature of participation and that it will not affect, in any way, their access to 
healthcare services at CDPH. If the client still appears interested, the Keep It Up! staff member 
will invite them to screen for the study. 
The client will complete the online screener using a computer or tablet (or paper form if 
necessary) in the private room. If eligible, an enrollment email with a link to the study to be sent 
to the participant’s email address. The Keep It Up!  staff member will inform the participant of 
the opportunity to complete the baseline assessment and STI testing while on-site at the clinic. 
If a participant prefers to complete the baseline assessment and STI testing at home, before 
starting the baseline assessment he will complete an online consent form that outlines the at-
home STI testing process. If a participant prefers to complete the baseline assessment and STI 
testing while on-site at the clinic, before starting the baseline assessment, he will complete a 
paper consent form that outlines the STI testing process during clinic visits. 
Most clients who test for HIV at a CDPH clinic will also have already been tested for urethral 
and rectal STIs as a part of their clinic visit. STI test samples are shipped to University of Illinois 
at Chicago (UIC) lab for analysis. Participants who have already completed STI testing will be 
asked to consent to the clinic’s UIC lab sharing their results with study staff via fax.  It typically 
takes 3 – 7 days for the return of results. CDPH staff will be responsible for providing these 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 18 of 50
participants with treatment and reporting positive results to appropriate authorities. Upon 
completing the baseline assessment, the participant will be compensated $50 cash and will be 
emailed a link to the first session of the intervention. 
Clients who have not completed STI testing as indicated by the study (i.e., urethral and rectal 
testing for chlamydia and gonorrhea) during their clinic visit or decline to consent to the clinic’s 
UIC lab sharing their test results will have the option to take the test kits home, be mailed the 
test kits by university staff, or self-administer the test kits while still on site at the CDPH clinic. 
Clients who take the kits home or have the kits mailed to their home will also have to complete 
the baseline assessment at home. These clients will complete the online consent form before 
beginning the baseline assessment, self-administer the STI tests, and mail the completed kits to 
the CDC for analysis. Upon receipt of their test kits by the CDC, they will be compensated $30 
in Visa gift cards and emailed a link to the first session of the intervention. 
For participants who choose to self-administer the STI tests on-site at the CDPH clinic, the 
completed kits will be mailed by university staff to the CDC lab for analysis. Samples that are 
not mailed out the same day will be stored in a locked storage cabinet by university staff for no 
more than 7 days before being mailed to the CDC. Upon completing the baseline assessment 
and STI testing, the participant will be compensated $50 cash and will be emailed a link to the 
first session of the intervention. 
Keep It Up! staff will be trained in appropriate and ethical methods of recruiting participants in 
medical settings. All Keep It Up! staff will complete Human Subjects Research CITI training 
before taking part in any recruitment, consenting, or enrollment activities. CDPH staff contact 
with participants will be limited to referring participants to the study and will not include 
performing tasks, such as collecting informed consent, for which Northwestern University 
requires Human Subjects CITI Training. CDPH staff will be trained by Keep It Up! staff through 
PowerPoint presentation and handouts on the background, rationale, and set up of the study. 
CDPH staff will also be given a verbal script with basic study information that they can use to 
refer participants to the study. The importance of emphasizing the optional nature of 
participation and that it will not affect, in any way, clients’ access to healthcare services will be 
stressed during CDPH staff trainings. The CDPH supervising physician, who will also act as the 
study liaison, has also completed Human Subjects Research CITI training and will be able to 
offer ongoing guidance to CDPH staff on appropriate ways to refer clients to the study.
Eligible participants across all study sites will be offered the opportunity to visit Emory University, Hunter 
College, or Northwestern University to complete the baseline assessment and complete their first set of self-
administered STI test kits. 
(b) Intensive efforts will be put into tracking and retaining participants across waves using 
procedures proven effective in our previous online and offline longitudinal studies. Retention has been 
identified as one of the greatest challenges to Internet-based HIV prevention research, with early studies 
reporting poor retention rates (30-40%)101. Recent studies have improved retention into the 80% range, even 
across one year189, by optimizing contact with E-raffles and providing financial incentives for survey completion 
101.  Using these approaches in our R34 resulted in 89% retention at 3-months. In Project Q2 and Crew 450, 
we used similar approaches and obtained >80% retention of YMSM over several years. Building on this 
experience we will assure a high retention rate by: (a) maintaining up-to-date contact information by contacting 
each participant monthly for an E-raffle (i.e. link that must be clicked within 48 hours to be entered in raffle to 
win $50 pre-loaded Citi VISA card). This requires minimal participant effort and quickly identifies inactive 
contact information.  (b) utilize our established contact trace protocol (see attached Contact and Retention 
Protocol) for inactive contact information using ancillary contact information.  (c) Participant incentives will be 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 19 of 50
picked up in person or mailed, providing further reason for participants to provide reliable contact information. 
(d) The tracking system for the project will be customized to record all contacts with participants and provide 
alerts when more intense follow-up is required.
(b1) We will respectfully contact participants using their approved methods of communication and 
contact information.  Participants will be contacted for the duration of the study unless they express 
their intention to withdraw from the study.  Setting limits to the number of attempted contacts is not 
helpful for recruiting and retaining difficult to reach populations in research, such as YMSM.   We have 
used these procedures in our previous studies with no complaints from study participants.
(b2) Overview of the Contact and Retention Protocol for recruitment/enrollment:
1.
If a client is eligible and interested in participating in the study, he will automatically be sent an 
email from Assessment Center with a link to the baseline assessment. 
2.
If a client does not click on the link or respond to the email within 4 days, Assessment Center 
will be triggered to automatically send him a reminder email and, if permission is given, a text 
message reminder. If a client has not responded  after the first reminder email and text 
message, Assessment Center will send out another reminder email at the following intervals: 13 
days, 9 days, 5 days, and 1 day before the assessment closes. The project coordinator will also 
call the client to remind him to begin the baseline assessment. If the project coordinator is 
unable to reach the client with one call, s/he may make additional phone call attempts.
3.
If a client has not responded after these attempts at contact by email (supplemented by texts 
and phone calls), the project coordinator, at his/her discretion, may send out additional 
reminders using the client’s preferred contact method every 3 days.  
(b3) Overview of the Contact and Retention Protocol for STI test kits
1.
Participants will be sent at home STI test kits at baseline and 12 month follow-up. Participants 
who test positive at baseline will also be tested at 3 month and 6 month follow-ups in addition to 
the 12 month follow-up. If a participant does not return an STI test kit by the due date it’s to be 
received, the project coordinator will email and, if permitted, text the participant the next day to 
remind him to return the test kit. If a participant has not returned his test kit after 7 days of its 
due date, the project coordinator will send another reminder using the participant’s preferred 
method of contact. The project coordinator will call the participant if he has not returned his kit 
14 days after the due date. If the participant does not return his kit or make arrangements to do 
so after this third attempt at contact, the project coordinator, at his/her discretion may make 
additional calls every 7 days.
2.
If a participant does not access his STI results within 14 days of the results being made 
available, the project coordinator will remind the participant, using his preferred method of 
contact, to access his results.  If the participant does not access his results after this reminder, 
the project coordinator, at his/her discretion may call or give additional reminders using the 
participant’s preferred method of contact every 7 days. 
3.
A minimum of three attempts at contact should be made for both the reminders to return kits 
and access test results. If a participant does not respond to these attempts, the project 
coordinator can make additional attempts for the duration of the study unless the participant 
explicitly asks to no longer be contacted.
 
(b4) Overview of the Contact and Retention Protocol for monthly raffles:
1.
Each month participants will be sent an email with a link to enroll in a $50 raffle. These monthly 
raffles will serve to confirm that a participant’s email address is up to date and in use. 
2.
If a participant does not click the raffle link within 48 hours, the project coordinator will make at 
least three attempts to call the participant to confirm that the email address is up to date and in 
use. If the participant is not reached within a week, the project coordinator will contact approved 
alternative contacts (e.g., parent, friend).  In addition, project coordinators will utilize Internet, 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 20 of 50
public database systems, and private services to locate any individual who is unresponsive to 
contact efforts.
3.
Participants will be contacted for the duration of the study unless they express their intention to 
withdraw from the study.
(c) Intervention delivery. Upon enrollment, participants will randomly assigned by the online program 
to receive KIU! or the HIV knowledge control arm. Randomization will be stratified by race and CBO at baseline 
by computerized system.  Stratifying by race will assure sufficient representations by race/ethnicity in each 
condition to address Aim 3—exploring potential racial/ethnic differences in outcome effects.  Stratification by 
CBO will prevent imbalance in factors that may influence intervention responsiveness across cities and 
clinics190. After the pre-test assessment and remote STI testing, participants will receive the intervention 
content across three sessions across a minimum of 3 days and a maximum of 3 weeks, based on principles of 
effective HIV interventions and high acceptability in the pilot phase (see preliminary studies). The existing 
booster session will be delivered at 3 months, and the newly created booster session content at 3 and 6 
months. 
(d) Self-Reported Measures. Participants will complete self-report assessments at baseline, 
immediately post-intervention, and 3-, 6-, and 12- months post-intervention. Immediately after the interventions 
are complete, an assessment of intervention acceptability and tolerability will take place along with an 
assessment of HIV knowledge (which will not be repeated to reduce practice effects). We have elected to 
conduct an immediate post-test assessment for 4 reasons: (1) It will allow us to make a payment to participants 
after completion of the intervention modules, which will increase retention and verification of contact 
information; (2) it will allow us to assess acceptability and tolerability of the intervention immediately post-
completion, while the content and style are still fresh to participants; (3) it will allow us to test immediate gains 
in knowledge to evaluate how well intervention material was learned using the online format; and (4) these 
procedures are consistent with our pilot study and were found to be acceptable to YMSM and likely aided in 
retention. The next assessments will occur at 3, 6, and 12-month post-intervention, allowing us to collect 
outcome data, provide a participant payment that will aid in retention, and to deliver booster sessions focused 
on reviewing and refining risk reduction goals and consolidating and reinforcing intervention messages.  In all 
cases assessment will occur prior to boosters to prevent contagion from proximal booster material. We follow 
participants to 12 months in order to assess behavioral outcomes far enough post-intervention to allow for the 
potential occurrence of risk behaviors and HIV testing. This assessment plan also allows us to model possible 
degradation of treatment effects over time and to assess outcomes 6 months from the final booster session, 
which meets criteria for being classified as a best-evidence (tier I) HIV prevention program by the CDC13.
Similar procedures and measures successfully used in the developmental study will be employed. 
Assessments will be completed via the Internet using a Computer-Assisted Self Interview (CASI). We will 
administer knowledge/information based measures immediately post-intervention along with the measures of 
intervention acceptability and tolerability.  Motivation, skills (i.e. partner sexual communication, correct condom 
use), and behavioral outcomes (i.e. number of insertive and receptive unprotected anal sex acts, condom 
errors) will be measured at all follow-up points. Whenever possible, we selected measures: (1) designed for 
high-risk YMSM to maximize sensitivity, (2) previously tested with diverse populations to minimize cultural bias, 
and (3) to maximize comparability to other studies in order to compare findings.  All measures and a table 
depicting what is assessed at each time-point can be found in Appendix 2.  Participants will be paid for 
completing assessment waves (~30 minute duration; See Budget Justification). As we have done successfully 
in prior online studies, participants will be given the option of receiving payment in person or via mail. 
Construct 
Administration Points    .    
Measure
Unprotected sex acts and 
HIV testing
BL
3
6
12
The AIDS-Risk Behavior Assessment (ARBA)191,192  has been used with YMSM82,102,191 and 
assesses behaviors on a partner-by-partner level, starting with recent partners, as well as in the 
aggregate.  Partners are classified as serious or casual and relationship duration is measured95.  
Questions differentiate between insertive and receptive anal sex. The ARBA will also assess 
secondary outcomes of HIV testing behaviors. 
Primary 
Outcomes
STI incidence
CT and GC testing of urine and rectal samples using the FDA cleared Gen-Probe APTIMA 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 21 of 50
BL
3
6
12
Combo 2 Assay. All participants will be tested at baseline and 12 month follow-up. Participants 
who test positive at baseline will also be tested at 3 month and 6 month follow-ups.
Secondary Outcomes
HIV Information 
BL
3
6
12
_________________
Motivation &Behavioral 
Skills
BL
3
6
12
*The brief HIV/AIDS Knowledge questionnaire  assess knowledge of transmission and prevention 
193. It has strong internal consistency, test-retest stability193, and has been used successfully with 
young adults194.  A total correct score will be used in analyses.   
*HIV/AIDS Motivation and Behavioral Skills195 assesses: Motivation (e.g. intentions to use 
condoms, perceived threat of HIV or STI infection, motivation to become safer), Social Norms 
(e.g., partners, friends, or family members opinions about condom use), and Behavioral Skills 
(e.g. self-efficacy, negotiating condom use). Internal reliability alphas range from .73 to .94 and 
the measure has been used and developed for MSM. *Behavioral skills also include: the Condom 
Errors Questionnaire196, that assesses condom use errors within the last 3 months (e.g. using oil-
based lubricant). The scale has been found to be predictive of condom breakage and 
slippage197,198 and has been used with YMSM199,200.
*The Health Protective Communication Scale   measures how respondents discuss health 
protection with their sex partners.  It has been used with diverse adolescent and young adult 
samples (alpha = .84 in a national sample201).  
Intervention acceptability 
and tolerability
Post
Developed by Tarnowski and Simonian140, it includes a combination of open-ended questions 
(e.g. “What aspect of the program did you like the least?”) and closed-ended Likert-style 
questions that form a scale of intervention acceptability (alpha = .87).  
Dose, engagement, and 
process
During Intervention
Intervention dose will be defined as the number of module and booster sessions completed. 
Although time logged into the intervention will be tracked, it is a poor proxy for engagement as 
individuals may stay logged in even when not engaged and a “time out” function only partially 
rectifies this. To measure engagement we will track key activities within modules (e.g. 
interactions within games) and develop a score based on participant deviations from sample 
average engagement. We will include measures related to the process of the intervention, such 
as the environment where participants complete the modules (i.e. location, level of privacy, etc.) 
and reasons for non-participation in the study; and at the end of each assessment we will ask 
participants if they have experienced any adverse events as a result of their participation in the 
study (see Human Subjects).
Other Measures
Demographics and reading 
ability
BL
We will use standard measures of age, ethnicity, education, and socio-economic status. YMSM-
specific items include gender identity, sexual orientation identity, and anatomic sex at birth. 
Reading ability at or above an 8th grade level will be assessed during the screening questionnaire 
using an adaptation of portions of the Peabody Picture Vocabulary Test-Revised used 
successfully in the R34 202.  
(e) Exit Interviews: We have created a short exit interview script to collect qualitative data from 
participants after they complete their final 12 month follow-up assessment. We will conduct 20 exit interviews 
by phone to hear participants’ thoughts on the intervention and its effect on their behavior. Ten participants will 
be randomly selected from a pool of intervention completers who responded well to the intervention (defined as 
a reduction in condomless anal sex acts at 12-month follow-up) and ten participants will be randomly selected 
from a pool of intervention completers  who did not respond well to the intervention (defined as no change or 
an increase in condomless anal sex acts at 12-month follow-up). By including people who did versus did not 
respond well to the interview, we will be able to better understand differences in participants’ experiences that 
may have contributed to whether or not the intervention changed their behavior. We will record the interviews 
by putting the call on speakerphone mode and using a handheld digital recording device. Before beginning the 
exit interviews, the study team will ask participants for verbal consent that they agree to complete the exit 
interview and have the conversation recorded. Participants will be reminded that all of the information they 
provide during the exit interview is confidential and that they can refuse to answer any questions that they do 
not feel comfortable answering. The study team will transcribe the audio and review responses for feedback on 
the intervention. The information gained from the qualitative interviews will be used to continue to refine the 
intervention in future iterations.  It will also help the study team to understand whether or not participants find it 
acceptable to complete and return biological samples for STI testing, which will inform future research efforts. 
The interview will take 10 – 15 minutes to complete and participants will be compensated with a $5 gift card as 
thanks for their time and participation. 
(f) Repeat STI Interviews: Preliminary data analysis indicates that a subset of KIU participants tested 
positive for rectal STIs more than once during the study. We plan to continue refining the KIU program in future 
research, so we are interested in getting a better understanding of what factors are contributing to repeat STIs 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 22 of 50
in this subset of participants.  This information will allows us to add content to future iterations of the 
intervention to address these issues and hopefully reduce the occurrence of repeat STIs in this population. In 
order to gain this important information, we have created a brief interview script to collect qualitative data from 
20 randomly selected participants from a pool of participants who tested positive for rectal STIs more than 
once during the study. We will conduct interviews by phone to learn about participants’ experiences getting 
tested for and seeking treatment for rectal STIs, including their reactions to testing positive for a rectal STI, 
how it influenced their thinking and behavior, and their experiences related to seeking treatment (e.g., barriers 
to seeking treatment, doctors’ reactions). We will record the interviews by putting the call on speakerphone 
mode and using a handheld digital recording device. Before beginning the interviews, the study team will 
describe the potential risks and benefits associated with participating in the interview, describe how we handle 
privacy and confidentiality, and ask participants for verbal consent to participate in the interview and to have it 
audio recorded. Participants will be reminded that all of the information they provide during the interview is 
confidential and that they can refuse to answer any questions that they do not feel comfortable answering. The 
study team will transcribe the audio and conduct qualitative data analyses in order to identify themes in 
responses. The information gained from the qualitative interviews will help to broaden our understanding of risk 
for repeat rectal STIs, including identifying factors that may be barriers to accessing treatment. This information 
has the potential to inform future development of HIV/STI prevention interventions for this population. The 
interview will take 10-15 minutes to complete and participants will be compensated with a $10 gift card as 
thanks for their time and participation. Participants who are difficult to reach (defined as non-responsive to 3 
contact attempts) will be offered a $20 gift card as thanks for their time and participation. Participants who are 
recruited for these interviews come from a relatively small subset of participants who tested positive for rectal 
STIs more than once during the study, so we hope increasing the incentive for difficult to reach participants will 
help us reach our goal of 20 interviews. Increased incentives like this have been shown to increase 
participation among difficult to reach and non-responsive participants in other research studies without 
perceptions of coercion. 
(g) Timed Post-Release Testing: We will use time spent on intervention and booster sessions as a proxy 
for participant engagement and as a means to refine future implementations of KIU!. In order to determine a 
baseline measurement for the amount of time it takes to complete each intervention and booster session, we 
will recruit 20 participants via online social media and application ads. We are recruiting participants who meet 
the same eligibility requirements as the 900 participants who were enrolled into the KIU! intervention. Potential 
participants will click the ad, fill out a screener, and, if eligible, provide contact information. They will then be 
contacted by a KIU! study staff who will describe KIU! in greater detail. If they are still interested in participating 
in the timed post-release testing, a research assistant will email potential participants a link to an online 
consent form. Once the participant has consented, a research assistant will email them a unique link to the first 
intervention session. Participants will be given ten days to complete the three intervention sessions and 3- and 
6- month booster sessions. Each session takes about 30 minutes to complete. After participants complete the 
6 month booster session, study staff will call participants to complete a 10 – 15 minute exit interview. 
Participants will be reminded that the conversation will be recorded, all of the information they provide during 
the exit interview is confidential, and that they can refuse to answer any questions that they do not feel 
comfortable answering. The study team will transcribe the audio and review responses for feedback on the 
intervention. The information gained from the qualitative exit interviews will be used to continue to refine the 
intervention in future iterations. Overall, participants will spend approximately 2 hours and 45 minutes 
completing study activities (2 ½ hours for the online intervention and booster sessions, and 10 – 15 minutes for 
the phone exit interview). Participants must complete the intervention, two booster sessions, and phone exit 
interview to be mailed a physical $50 Visa gift card.
(h) STI biomarkers: Our approach to the distance STI testing follows Sullivan’s effective protocol for 
the Checking In Study (see C.2.d).  To test for STIs, we utilize at-home urine collection kits and at-home rectal 
swab test kits which both use the FDA approved Gen-Probe APTIMA Combo 2® Assay204,  a target 
amplification nucleic acid probe test that utilizes target capture for the in vitro qualitative detection and 
differentiation of ribosomal RNA from CT and NG. Gen-Probe reports sensitivity of 98 – 100% and specificity of 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 23 of 50
98%- 100%  for urethral CT and NG.  Urine and rectal samples will be tested in the Centers for Disease 
Control and Prevention (CDC) Division of STD Prevention Laboratory. 
STI testing protocol at baseline will vary based on enrollment site.  Participants who enroll off-site (e.g. at home 
or at a library) will be shipped both test kits in a nondescript box. Easy-to-understand instructions for collecting 
and returning the samples will be provided with the kits. In addition to the written instructions provided with the 
rectal STI kit, a private video with instructions for properly collecting the rectal samples will be shared with 
participants. The sample collection kits are identified by study ID only.  After the samples are returned through 
the mail and processed by the CDC lab, results will be sent on at least a biweekly basis to Northwestern 
University.    If a specimen is not received within 3 weeks of mailing, an electronic reminder will be sent 
through the medium chosen by the participant (text message or email) or phone if no preferred medium was 
selected by the participant.  Repeat reminders will be sent in subsequent weeks if the specimen remains 
unreturned and, eventually, telephone follow-up will be used to support participants whose specimens were 
overdue by 4 weeks from time of mailing.  The STI testing protocol at all follow-ups will be the same for these 
participants. 
Participants who enroll on-site at Emory University, Hunter College, or Northwestern University will collect 
samples for baseline STI testing while at the university site. Easy-to-understand instructions for collecting and 
returning the samples will be provided with the kits. In addition to the written instructions, KIU! staff will be on 
hand to address any concerns or questions that participants have about self-administering the test kits. The 
sample collection kits are identified by study ID only. KIU! staff will mail the STI test samples to the CDC.  After 
the samples are returned through the mail and processed by the CDC lab, results will be sent on at least a 
biweekly basis to Northwestern University. For these participants, the STI testing protocol at all follow-ups will 
be the same one used by participants who completed their baseline STI testing off-site.
In most cases, participants who are recruited on-site at a CDPH clinic will have already completed STI testing 
as a part of their clinic visit. CDPH clinics also test for STIs using the FDA approved APTIMA assay. 
Participants who have already completed STI testing will be asked to consent to the clinic’s UIC lab sharing 
their results with study staff via fax. KIU! staff will call the lab client services and provide the patient name, date 
of birth, and date of specimen collection in order to verify that correct results are sent. In addition, KIU! staff will 
verify the fax number to which the results should be sent. The fax machine to which results will be sent is 
located in the mail room of Northwestern’s Department of Medical Social Sciences (MSS). The MMS office is 
not open to the general public. The office is only accessible to staff with ID cards and invited guests who are 
buzzed into the office. Guests are typically escorted through the office by staff. In general, only MSS staff 
access the fax machine. To protect participant’s confidentiality, KIU! staff will ask the lab to confirm the time 
that the fax will be sent so that the results are immediately retrieved from the fax machine. All faxes of results 
will also include a fax cover sheet that describes the confidential nature of the contents and has instructions to 
notify the sender to report any faxes received in error and destroy the misdirected documents. The cover letter 
also warns that any mishandling of the documents, such as copying or dissemination, is illegal. It typically 
takes 3 – 7 days for the return of results. CDPH staff will be responsible for providing these participants with 
treatment and reporting positive results to appropriate authorities. During instances in which a client has not 
already completed STI testing or declines to consent to the clinic’s UIC lab sharing their test results, university 
staff will offer the client the option to take the test kits home, be mailed the test kits by university staff, or self-
administer the test kits while still on-site at the CDPH clinic. Participants who self-administer at home will be 
responsible for mailing the kits back to the CDC for analysis. Participants who self-administer on-site at the 
CDPH clinic will have their kits mailed by university staff to the CDC for analysis. 
All STI results will be manually entered into the study’s REDCap tracking database. In order to avoid data entry 
errors and sharing incorrect results with participants, the study team will do comprehensive checks on logged 
results and the PDFs of results sent to participants (see “C.2.d. Protection against a Breach of Confidentiality” 
for more details). 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 24 of 50
If a participant’s specimen is positive, the Physician of Record at the CDC reviews the results before releasing 
to participants.  We will deliver test results to participants using a secure, encrypted email. To open the email 
and access their results, participants must enter the study ID that was provided to them with their test kit. They 
will be able to download and print out a hard copy of their results.  Participants may speak to research staff if 
they wish. With positive test results for chlamydia or gonorrhea, research study staff will: (1) encourage 
participants to notify current and past sex partners; (2) provide local referrals for free or low cost treatment; and 
(3) make a legally required confidential report to the appropriate health department. Health department 
professionals (or disease information specialists) may contact and interview the participant to obtain names of 
his sex partners to notify and encourage them to seek evaluation and treatment. STI testing and results will 
also be self-reported at behavioral assessment time points throughout the follow-up period to determine if a 
participant tested positive and was treated prior to the 12-month STI test. 
B.11. RCT Analytic Plan. This is a two-group, active-control, RCT of an online intervention. The study’s 
hypothesis concerns between-group differences in the change in HIV/STI risk behavior and STI incidence from 
pre-intervention to later follow-ups. There are 4 assessment points of RCT outcomes: baseline, 3, 6, and 12 
months post-intervention. Prior to conducting full analyses, standard descriptive analyses will be conducted to 
check for invalid values, outliers, and logical consistency between variables. We will assess the comparability 
of the experimental and control groups at baseline. If any variables differ significantly at baseline, they will be 
included as covariates in the adjusted outcome analyses along with variables known from prior research (e.g. 
age) to be correlated with the outcome under analysis.  Following extensive data description and analysis at 
separate follow-up points, the primary analysis will utilize all follow-up points by fitting mixed effect regression 
models. Also known as hierarchical linear models (HLM) 205-207, these models allow for the presence of missing 
data, time-varying or invariant covariates, and subject measurement at different time points.  HLM models 
appropriate to the characteristics of the distribution (i.e. normal, Poisson) will be fit using SAS. 
The primary behavioral outcome measure of treatment success will be the count of unprotected anal sex acts 
and the biomedical endpoint will be NG/CT incidence (Aim 2). Secondary outcomes include: condom errors, IMB 
factors, and receipt of an HIV test. We will also explore intervention effects separately for receptive and insertive 
anal sex. The primary independent variable of interest is the indicator for the intervention group. The baseline 
value of the outcome and other required covariates at baseline will be included as additional independent 
variables. The intervention will recruit participants from multiple sites (multiple clinics in three cities); however it 
is not delivered in groups so site will be adjusted for as a fixed rather than a random independent variable. A 
random intercept term will be included to account for individual average levels of the outcome. The effect of 
time will first be estimated as a categorical variable in order to describe and adjust for time trends as observed. 
A second strategy will start with estimating a single slope reflecting average change across all 4 time points, 
then add a quadratic term if it contributes significantly to the model. Random effects for time variables, 
reflecting individual variations in time trends, will be included in addition to random intercepts, if significant. A 
single effect of group will test the intervention effect averaged across all time points, adjusting for all included 
covariates and underlying time trends. Potential moderators (race/ethnicity, relationships status, and 
gay/bisexual identity) will be added as independent variables and their interactions with intervention group will be 
tested (Aim 3). Addition of interaction terms between time and intervention group will provide a test for 
comparing time trends in change scores between groups.
Analysis of all available measurements among all participants regardless of dose is consistent with an “intent-to-
treat” perspective172. It is desirable to also perform an analysis which considers the “dose” and engagement in 
the intervention. We will use Rubin’s causal modeling to consider dose-effect when making comparisons across 
the two groups208,209. Further, in our HLM models, missing data is assumed to be missing at random conditional 
on the independent variables in the model; therefore we are in effect assuming that participants with incomplete 
measurements would have been similar to others in their group (their same covariate values) with complete 
measurements.
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 25 of 50
B.12. Power and Sample Size. All power calculations were performed in R version 2.14.0 and using common 
sample size formula for repeated measures210 with nominal Type I error rates (i.e. α=.05) and effect estimates 
based on our preliminary studies. We will screen approximately 4500 YMSM, enroll 900 YMSM, and assume 
80% retention based on our preliminary studies.  Assuming participants are randomized to two groups in equal 
proportions, we will have 97% power to detect a 11% reduction in the rate of unprotected sex (ERR = 0.89) 
due to the intervention, assuming the correlation between repeated measures is 0.8.  The correlation among 
repeated measures plays an important role in the calculation; in this case, our power calculation is 
conservative if the correlation among repeated measures is, in fact, weaker than 0.8.  The smallest effect size 
with which we will have 80% power to detect is a 6.5% reduction in the rate of unprotected sex. Technically 
speaking, our study will have greater power than what is reported here after we adjust for our blocked design. 
In conclusion, we will have 80% power to detect an effect size equal to half of what we observed in our pilot 
trial and less than half of the lower bound of effect sizes seen in other HIV prevention programs with MSM26.  
For our primary biomedical endpoint of STI incidence, we assume the annual incidence based on our 
longitudinal cohort studies in the range of 10 to 15%.  If the incidence rate in the control group is 10%, we will 
have 81%, 70%, and 56% power to detect an incidence reduction of 7%, 6%, and 5%, respectively.  If the 
incidence rate in the control group is higher, say 15%, then the power reduces to 71%, 60%, and 47%, 
respectively.  The objective of aim 3 is to estimate the intervention effect among subgroups and test for 
significant differences.  In data analyses, this will be achieved by testing 2-way interactions, treatment by 
subgroup.  For a conservative power calculation, we repeated the power calculations as per aim 1 with a 
reduced sample size. Assuming N=200 per group (say, for racial/ethnic subgroups), an ERR=0.89 as above, 
and strong correlation among repeated measures (i.e. 0.8), we will have 91%, 85%, and 77% power to detect a 
10%, 9%, and 8% reduction in the rate of unprotected sex among YMSM, respectively, due to KIU! 
intervention.  The power will be less for effect sizes more subtle than 8% and less for smaller subgroups.  In 
addition to racial/ethnic subgroups, other subgroups of interest include sexual orientation identity groups (81% 
gay, 17% bisexual identified) and relationship groupings (30% recent serious relationships). The power for 
each subgroup comparison will be similar to the power described above.  
B.13. There are several alternatives to the design proposed here that deserve consideration. First, is the 
linkage of the online intervention to HIV testing, instead of recruiting YMSM online (e.g. banner ads). Online 
recruitment has been used extensively for other online HIV behavioral research projects with MSM41,101, but 
with a few important exceptions these studies have had difficulties with recruitment of racial/ethnic minorities 
and with longitudinal retention. We believe that linking to HIV testing is an innovative method to help overcome 
these issues while simultaneously addressing the increasing lack of prevention counseling in this context. 
Furthermore, it advances the science of linking eHealth intervention into clinical encounters, which can then be 
applied to the inclusion of eHealth prevention in future biomedical prevention strategies (e.g. PrEP). Second, 
we considered using HIV status as a trial outcome. We ruled out this alternative for two reasons: a) by 
recruiting YMSM that just tested negative, seroconversions across one year would be a rare event and 
therefore require a cost prohibitive sample size (N > 4,500),  and b) conducting HIV testing as part of the study 
would not allow us to include elective testing as a trial outcome. 
C. HUMAN SUBJECTS
C.1. RISKS TO HUMAN SUBJECTS
C.1.a. Human Subjects Involvement, Characteristics, and Design.  We will screen 4532 ethnically diverse 
18–29 year-old young men who have sex with men (YMSM) following an HIV negative test result from 
community clinics in Chicago, New York, and Atlanta or at a university site (Emory and Hunter). (This total 
sample includes 12 YMSM who will participate in acceptability and usability testing prior to implementation.)  
About 20% of these young men (n = 900) are anticipated to enroll into and complete the study. Investigators 
with a history of successfully conducting HIV behavioral research will be supervising research in these cities. 
Three academic institutions (Northwestern University, Hunter College, and Emory University) with exemplary 
human subjects protection programs will be involved in participant recruitment. The investigative team has 
extensive experience conducting research and providing services to YMSM.  The Principal Investigator has 
been a member of the Institutional Review Board (IRB) and has expertise in ethical issues related to 
conducting research with sexual minority youth211.
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 26 of 50
MSM will range in age from 18-29 years, consistent with findings regarding emerging adulthood as a period of 
heightened risk for STIs/HIV among MSM15,212-214. We will recruit a racially and ethnically diverse sample, 
representing both high-risk YMSM in Chicago, Atlanta, and New York, as well as the demographics of the 
population seeking services at each recruitment site. As described in the research design, the size and 
epidemiologic trends of the HIV/AIDS epidemic in each city make them good candidates for conducting an 
RCT among YMSM.  These cities represent distinct geographical regions (i.e. Midwest, Southeast, East Coast) 
with notable differences in population demographics and culture. This diversity will allow examination of 
whether the intervention works equally well in a variety of urban settings, thus increasing our ability to draw 
conclusions about the generalizability of findings. Given site-specific recruitment patterns, we will readily 
achieve representation across White, Black, and Latino groups, but we will also stratify enrollment to equal 1/3 
for each group). Each site will screen approximately 1250 YMSM and enroll approximately 250 YMSM.  
Inclusion and exclusion criteria: We will include participants in the study if they: (1) MSM, defined as a birth 
male who identifies as male and who self-reports having sexual contact with another male in the past year; (2) 
received an HIV negative test result from a participating clinic/recruitment site; (3) are 18-29 years old; and (4) 
had unprotected anal sex with another male in the last 6 months. YMSM will not be enrolled in the study if they: 
(1) do not have an e-mail address that can be used to contact them for retention purposes; (2) are currently in 
a behaviorally monogamous relationship lasting longer than 6 months. We use this criterion because we 
believe safer sex interventions for MSM in long-term monogamous relationships will require substantial 
tailoring from those who are single or in the early phases of a relationship; (3) did not have unprotected anal 
sex with another male in the last 6 months. Consistent with recommendations for assuring sufficient event 
rates in RCTs 172, this will ensure that sufficient rates of sexual behavior are occurring at pre-test to allow us to 
detect intervention effects on behavior at follow-up; (4) HIV positive serostatus at time of enrollment. We have 
elected to exclude HIV+ individuals because we believe that secondary prevention messages may need to be 
different than primary prevention messages.  Nevertheless, if a participant seroconverts during the course of 
the study, he will be allowed to continue participating (with appropriate adjustment in the analyses). We will 
also refer him for HIV services (see below); and (5) since the measures and intervention are delivered in 
English, participants must be able to read English at a minimum 8th grade level.  Our developmental R34 KIU! 
data, and data on client flow at our recruitment sites, suggest that despite having many exclusion criteria we 
will be able to meet our target recruitment goals. 
YMSM will participate in either usability and acceptability tests in phase 1 or the RCT in phase 2. 
In phase 1 (months 1-6), we will perform necessary technological upgrades to the tracking system, program 
additional booster modules, conduct usability testing, and prepare for the RCT. Twelve ethnically diverse 
YMSM (4 Caucasian, 4 African American, 4 Latino) will participate in usability and acceptability testing of new 
booster session and the existing modules that will be updated with video interviews conducted in New York 
and Atlanta. A testing protocol established by the Northwestern University Center for Behavioral Intervention 
Technologies (CBIT) will be used to assess usability across standards (i.e. across different monitors, 
bandwidths, etc.) and in practice with target participants who will “think out loud” and be observed as they go 
through the modules (i.e. ability to navigate, cultural appropriateness).  Participants in usability testing will be 
reimbursed $35 in cash for approximately 90 minutes of their time and for travel expenses.   
In phase 2 (months 7 – 48), 4500 YMSM will participate in the RCT phase of the study. About 20% of these 
young men (n = 900) are anticipated to enroll into and complete the study.  Participants will be automatically 
randomized into the experimental intervention (KIU!) or the control condition (HIV Knowledge) after verification 
of participant eligibility and completion of informed consent online. Randomization will be stratified by race and 
recruitment clinic.  In both arms of the trial we will test for Gonorrhea (NG) and Chlamydia (CT), using the at-
home rectal swab test kits and at-home urine test kits, and refer for treatment as necessary at baseline and 
post-intervention. In the experimental condition, participants will have already received HIV testing and 
counseling, and in the context of the proposed study will receive the highly-interactive and YMSM-tailored 
Internet-based prevention program called Keep It Up! (KIU!). Consistent with meta-analytic data on effective 
HIV interventions, the content will be delivered over three sessions across a three week period (see Appendix 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 27 of 50
1 for intervention overview).  Participants in the KIU! arm will also receive booster sessions at 3- and 6--months 
from completion of the intervention.  These sessions continue to provide HIV prevention motivation and 
behavioral skills training.
In the control condition, participants will also have already received HIV testing and counseling. In the context 
of this study they will receive the HIV-information modules that focus on facts about HIV and STIs.  There are 
similarly 7 modules in this informational curriculum that, just like the KIU! arm, must be completed across three 
sessions.  YMSM in this control arm will receive booster sessions at 3- and 6-months that provide further facts 
about HIV and STIs.
We acknowledge the importance of considering the ethical ramifications of delivering an HIV information-only 
control condition to a group at high risk for HIV infection. Ethical objections to randomization with a control 
group generally center around situations where an existing effective intervention exists172.  In the case of online 
HIV prevention with YMSM no such proven-effective intervention exists.  We highlight that in our pilot RCT the 
information control condition showed increased HIV knowledge as a product of their participation, and 
therefore the control group does receive a benefit from participation. We take the view that properly designed 
and conducted RCTs are ethical when they answer important public health questions without impairing the 
welfare of participants who provide informed consent.
Participants in both conditions will complete all assessment materials.  Similar procedures and measures 
successfully used in the developmental R34 study will be employed. Assessments will be completed via the 
Internet using an Internet-based Computer-Assisted Self Interview (CASI). Assessments will occur at baseline, 
3-,6-, and12-month post intervention. At each log-in to the intervention we will ask about the physical setting 
where the participant is located. We will assess levels of privacy, distractions, and the type of location. At each 
assessment point participants will be asked if they have experienced any adverse events as a result of their 
participation in the program. Endorsement of adverse events will trigger an email alert to the principal 
investigator who will respond expeditiously.  Participants will also be given a contact number to call if they need 
more immediate assistance in regards to the adverse situation. Note that in the pilot trial no participants 
reported adverse events during the trial.
Each assessment will take less than 30 minutes. As we have done successfully in prior online studies, 
participants will be given the option of receiving payment in person or via mail. Payments will be made after 
each of the following tasks are completed:
1.
Baseline payment will vary based on enrollment site.
a.
For participants who enroll off-site (e.g. at home or at a library), a $30 pre-loaded Citi VISA card 
will be given after participants complete the baseline assessment and the CDC lab receives the 
first STI test kit sample.
b.
For participants who enroll on-site at Emory University, Hunter College, or Northwestern 
University, $50 in cash will be given after participants complete the baseline assessment and 
STI testing. We will give these participants a consent form upon arrival describing the payment 
they will receive for completing the baseline assessment and STI testing on-site a university 
site.. University staff will mail the STI test samples to the CDC.
c.
 For participants who enroll on-site at a CDPH clinic, $50 in cash will be given after participants 
complete the baseline assessment and STI testing (or consent to the clinic’s UIC lab sharing 
their STI results). We will give these participants a paper consent form upon arrival describing 
the payment they receive for completing the baseline assessment and STI testing on-site at a 
CDPH clinic. 
2.
$20 pre-loaded Citi VISA card will be given after participants complete last intervention module and 
post-intervention assessment.  
3.
$20 or $30 pre-loaded Citi VISA card will be given after the participant completes the 3-month booster 
session and assessment. Participants who only complete the 3 month booster session and assessment 
will receive a $20 pre-loaded Citi VISA card. Participants who take at-home STI tests, in addition to 
completing the 3 month booster session and assessment, will receive a $30 pre-loaded Citi VISA card.
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 28 of 50
4.
$20 or $30 pre-loaded Citi VISA card will be given after the participant completes the 6-month booster 
session and assessment. Participants who only complete the 6 month booster session and assessment 
will receive a $20 pre-loaded Citi VISA card. Participants who take at-home STI tests, in addition to 
completing the 6 month booster session and assessment, will receive a $30 pre-loaded Citi VISA card.
5.
$30 or $50 pre-loaded Citi VISA card will be given after participants complete the 12-month 
assessment and the CDC lab receives the second STI test kit sample. The standard payment for this 
follow-up is a $30 pre-loaded Citi Visa card. Participants who are difficult to reach and are out of 
window for completing their final assessment and STI test kit will be offered a $50 pre-loaded Citi Visa 
card. These participants typically do not respond to contact attempts and do not complete one or more 
assessments. Non-responsive participants and those who did not complete previous assessments will 
also be offered a free movie ticket to AMC Theaters. Increased incentives like this have been shown to 
increase participation and retention among hard to reach and non-responsive participants in other 
research studies without perceptions of coercion. Younger participants (18 – 24 years old) also make 
up a larger portion of participants who are hard to reach or non-responsive. Latino and African 
American YMSM make up more than 60% of participants who are hard to reach or non-responsive. As 
these groups are most at risk for HIV, it is very importance to collect outcome data from them as a 
means of seeing how useful the intervention is in reducing risks among groups that are most in need of 
effective HIV prevention interventions.
Participants who come on-site to Emory, Hunter, or Northwestern to complete an assessment and/or STI 
testing will be given the option to receive payment in cash.
There are six different total amounts of incentives that a participant can receive. The different amounts are 
determined by enrollment site and STI status at baseline. Across all possible combinations of enrollment site 
and STI status, the maximum payment amount is $160 for participants who enroll on-site and test positive for 
STIs at baseline.  See the table below for a detailed description of the six tier incentive structure:
Payment Schedule
Baseline 
Intervention
3 Month 
Follow-up
6 Month 
Follow-up
12 Month 
Follow-up
Total
Participants who…

Enroll on-site

Test positive for STIs at 
baseline and repeat 
testing at all follow-ups
$50
$20
$30
$30
$30
$160

Enroll on-site

Test negative for STIs at 
baseline and repeat 
testing at 12 month 
follow-up
$50
$20
$20
$20
$30
$140

Enroll off-site

Test positive for STIs at 
baseline and repeat 
testing at all follow-ups
$30
$20
$30
$30
$30
$140

Enroll off-site

Test negative for STIs at 
baseline and repeat 
testing at 12 month 
follow-up
$30
$20
$20
$20
$30
$120
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 29 of 50

Enrolled on-site

One or more incomplete 
assessments

Completes 12 month 
assessment and test kit
$50
-
-
-
$50
  $100 - 
$180

Enrolled off-site

One or more incomplete 
assessments

Completes 12 month 
assessment and test kit
$30
-
-
-
$50
 $80 - 
$160
In our online and telephone recruitment ads, as well as the print ads that are displayed at our CBO partner 
sites, we will advertise the $120 incentive. We anticipate that most participants will enroll off-site and only 
complete STI testing at baseline and 12-month follow-up. At our university sites and CDPH sites where 
participants are able to enroll by completing the baseline assessment and STI testing, we will post print ads 
that advertise the $140 incentive. We also anticipate that most of these participants will only complete STI 
testing at baseline and 12-month follow-up.  
We will randomly select 20 participants to complete a short phone exit interview after their 12 month follow-up 
assessment. Participants will be compensated with a $5 gift card as thanks for their time and participation.
We will also randomly select 20 participants who tested positive for rectal STIs more than once during the 
study to complete a brief phone interview focused on their experiences getting tested for and seeking 
treatment for rectal STIs. Participants will be compensated with a $10 gift card as thanks for their time and 
participation. Participants who are difficult to reach (defined as non-responsive to 3 contact attempts) will be 
offered a $20 gift card as thanks for their time and participation. 
We will maintain up-to-date contact information by contacting each participant monthly using their preferred 
mode of communication. E-raffles will take place at monthly intervals.  Participants will be sent an e-mail that 
includes a link, that when clicked enters them in a raffle to win a $50 pre-loaded Citi VISA card. Based on 
projected recruitment of 264 participants per year, we estimate the odds of winning a raffle are 1 in 264 (or 
3.78%).  Participants will be entered in 11 raffles during their study enrollment.
The E-raffle link must be clicked within 48 hours. Failure to click the link will initiate a contact trace protocol. As 
described previously in this document, and in the Contact and Retention Protocol, participants will be 
contacted for the duration of the study unless they express their intention to withdraw from the study. As a 
component of study consent, participants will agree to provide us with extensive contact information, including 
home address, telephone / mobile numbers, e-mail, and two contacts that can reliably reach them.  
Participants will also consent to regularly updating locator information.  We will be sensitive to privacy and 
confidentiality and ask participants if we can cite the study in letters, phone messages, or when speaking to 
other contacts.  Our Tracking Coordinator (based at Northwestern University) will be responsible for most 
participant communication. He will introduce himself to participants via e-mail at the beginning of the study, 
provide technical support when needed, and provide participation reminders during the intervention and follow-
up phases.  If necessary, other research staff and CBO partners may help with contacting difficult to reach 
participants. As always, we will fully respect the right of a participant to choose to remove themselves from 
participation. 
We will recruit 20 participants via online ads in order to determine a baseline measurement for the amount of 
time it takes to complete each intervention and booster session. Participants who are eligible and consent to 
participation will be mailed a physical $50 Visa gift card following the completion of the intervention sessions, 
the 3 month and 6 month boosters, and exit interview.
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 30 of 50
C.1.b. Sources of Research Materials. We have used all of the proposed measures with YMSM either in the 
developmental R34 or previous studies with sexual minority young adults.  Whenever possible, we selected 
measures: (1) designed for high-risk YMSM to maximize sensitivity, (2) previously tested with diverse 
populations to minimize cultural bias, and (3) to maximize comparability to other studies in order to compare 
findings.  All measures and a Table depicting what is assessed at each time-point can be found in Appendix 2.  
Assessments will be completed via the Internet using an Internet-based Computer-Assisted Self Interview 
(CASI). Only the PI and Project Coordinators at Northwestern will have access to individually identifiable data 
and data will be collected and stored on secure servers maintained by Assessment Center (see below). 
Questionnaire measures will assess: HIV/STI risk behavior, alcohol and drug use, HIV/AIDS-related  
knowledge, attitudes, and motivation, condom use errors, safer sex communication and self-efficacy, 
intervention acceptability, tolerability, and process (e.g., where intervention was completed, reasons for non-
participation, and whether they experienced any adverse events as a result of participation). In addition, 
participants will report on their age, ethnicity, education, socio-economic status, and specific YMSM indices of 
gender identity, sexual orientation, and anatomic sex at birth. Social desirability will also be assessed to serve 
as a validity check for self-report responses and reading ability will also be assessed. 
STIs will be tested for using at home urine collection kits and at-home rectal swab test kits which both use the 
Gen-Probe APTIMA Combo 2® Assay which detects Chlamydia trachomatis (CT) and Neisseria gonorrhoeae 
(NG) to aid in the diagnosis of chlamydial and/or gonococcal urogenital disease.  For urethral CT and NG, 
Gen-Probe reports sensitivity of 98 – 100% and  specificity of 98 – 100%.  STI testing will be conducted in the 
Centers for Disease Control and Prevention (CDC) Division of STD Prevention Laboratory.
C.1.c. Potential Risks. Potential risks consist of the actual assessment and recruitment process (e.g., 
potential coercion, discomfort or distress during the intervention or assessments), breaches of confidentiality, 
and HIV seroconversion or STI diagnosis during the course of the study. It is possible that certain assessment 
items (e.g. sexual or drug behavior; STI and HIV testing results) may make participants feel uncomfortable.  
There is a small possibility that a participant’s information could be intercepted while in transit over the Internet 
or that data could be hacked from our servers. It is also possible that participants will experience a loss of 
confidentiality if they complete the assessments or intervention modules in a public space. Finally, a participant 
could test positive during the course of the study, however we have established procedures set up at each 
clinic linking participants to services and care (see below). 
C.1.d. Exit Interview Risks. It is possible that certain questions asked during the interview (e.g., questions 
about the acceptability of rectal STI testing) may make participants feel uncomfortable.  Participants will be 
reminded that they can refuse to answer any questions that they are uncomfortable answering.  There is a 
small possibility that a participant’s interview audio and transcription could be hacked from our servers where 
the files will be saved. It is also possible that participants will experience a loss of confidentiality if they 
complete the exit interview in a public space. We have established procedures to protect against these 
potential risks (see Section C.2.a.8).
C.1.e. Repeat STI Interview Risks. It is possible that certain questions asked during the interview (e.g., 
questions about STIs) may make participants feel uncomfortable.  Participants will be reminded that they can 
refuse to answer any questions that they are uncomfortable answering.  There is a small possibility that a 
participant’s interview audio and transcription could be hacked from our servers where the files will be saved. It 
is also possible that participants will experience a loss of confidentiality if they complete the exit interview in a 
public space. We have established procedures to protect against these potential risks (see Section C.2.a.9).
C.1.f. Timed Post-Release Testing Risks. It is possible that some of the content in the intervention modules 
and booster sessions may make participants feel uncomfortable. Participants will be reminded that they can 
call a research staff member about anything that can come up during the study. We will also refer participants 
to counseling services if they want. They will be reminded that they can stop participating at any time. It is also 
possible that participants my find some questions unpleasant or hard to answer. Participants will be reminded 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 31 of 50
that they can stop the exit interview at any time and withdraw from the completing the timed post-release 
testing.
C.1.g. At-Home HIV Testing Risks. The risks of participating in at-home HIV testing include those associated 
with loss of confidentiality, fingersticks to draw blood, and returning test results. For participants who complete 
at-home testing using Home Access specimen collection kits, fingersticks may cause temporary discomfort and 
rarely, infection. Some persons may pass out at the sight of blood, but this is rare. Disclosure of a preliminary 
positive result from a rapid HIV test or confirmed reactive HIV test result may cause distress and even 
substantial psychological trauma. Participants will be offered the number of a suicide prevention or other crisis 
counseling hotline in case of distress about preliminary positive or confirmed positive HIV test results. 
Individuals who agree to participate in at-home HIV testing will receive post-test counseling and, if appropriate, 
be linked to care. The project coordinator will use established organizational linkage-to-care procedures that 
have been used in other IRB approved studies. Linkage-to-care will include referring the participant to a clinic 
that will conduct a free confirmatory HIV test, and to someone that will work with them receive treatment if their 
confirmatory test returns positive. In Atlanta, participants under 24 years old will be referred to the Ponce de 
Leon center at Grady while those who are 24 years and older will be referred to their county health department.  
In Chicago, participants will be referred to a linkage-to-care specialist at Center on Halsted. In New York, 
participants may be referred to the Mount Sinai Comprehensive Health Program, Callen-Lorde Community 
Health Center, or Gay Men’s Health Crisis Prevention Center. For participants who live outside of Atlanta, 
Chicago, and New York, study staff will use the AIDS.gov service locator to make referrals to the nearest 
health care provider.
C.2. ADEQUACY OF PROTECTION AGAINST RISKS
C.2.a. Recruitment and Informed Consent. YMSM who tested HIV negative from our clinic partners will be 
recruited. Co-Investigators in each city have assembled a cadre of CBOs that will recruit YMSM upon a 
negative HIV test result (see letters of support). We have partnered with CBOs for recruitment in order to 
maximize our ability to test the intervention under conditions in which it would eventually be disseminated.  We 
will apply the following recruitment procedures used successfully in our developmental R34; however, we will 
streamline the process and utilize tablet PC devices (with paper-and-pencil option if preferred) to recruit and 
screen participants. 
The consent process will differ depending on the source from which a participant is recruited. 
(C.2.a.1) CBO Recruitment. At the conclusion of their post-test appointment or rapid-test, YMSM who test HIV 
negative will be given a tablet computer by the staff member who conducted the HIV test (with paper-and-
pencil backup as necessary).  Clients can be screened online on the Assessment Center platform or offline on 
the iFormBuilder app. The tablets (or paper form) will display information about the study, ask the potential 
participant demographic questions, and whether or not he wants to be contacted by study staff. If so, the 
device (or paper form) will ask for contact information.  If not, the tablet (or paper form) will ask questions 
assessing reasons for nonparticipation.  When paper and pencil are used, participants will provide their contact 
information if they are interested in participating in Keep It Up! or other studies; they will not be aware of their 
eligibility at the time. An alteration of consent and waiver of documentation will be requested from the IRBs for 
this screener. Information collected on the tablet will be encrypted on the device and securely transmitted to 
the Assessment Center server by staff. Device log-ins will prevent unauthorized access to the encrypted 
information on the device. Information collected on the tablet using the iFormBuilder app will be exported to a 
CSV file and either the information will be uploaded to Assessment Center or manually entered by research 
study staff. The CSV files will be password protected. Information collected with paper and pencil will be 
manually entered by research study staff into Assessment Center within 3 days of receipt. Paper forms will be 
stored in locked cabinets or other similar storage units. 
YMSM interested in participating will be instructed to expect an e-mail with more information about the study 
and told that they can use Internet-enabled computers at their local recruitment site for free.  The research 
coordinator based in Chicago, Atlanta, and New York will be on hand to provide training and support to testing 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 32 of 50
staff to assure easy compliance with this recruitment procedure. Potential participants will be sent a 
recruitment e-mail to the address entered into the tablet (or on paper form); we will solicit email addresses that 
are checked regularly. The participant will be given the option to immediately be considered for participation, 
defer participation to a specified later date within 4 weeks (e.g. a vacation preclude immediate enrollment), or 
have their information removed from the contact database. Non-response to the recruitment e-mail after 4 days 
will trigger the retention protocol that will include further e-mails, phone calls if a valid phone number was 
provided, a text message (if consent provided).  Individuals who wish to be considered for participation will 
complete a brief online questionnaire that will assess inclusion criteria, and if eligible will be enrolled and 
directed to the baseline survey.
The only data collected prior to informed consent will be screening data used to assure eligibility and reasons 
for non-participation.  Consent will be conducted online using procedures that have previously been IRB 
approved and used successfully in our developmental R34 (see Preliminary Studies). The consent document 
will be written to Federal OHRP standards with 8th grade or below reading level.  The consent statements will 
have the required elements of informed consent, including (a) a description of the study and the amount of time 
required for participation; (b) acknowledgment that participation is voluntary and that no negative 
consequences will occur for those who choose not to participate; (c) acknowledgment of the participant’s right 
to withdraw at any time; (d) guarantee of confidentiality unless any person is in danger of injury; (e) a 
statement of payment, including amount and circumstances under which the respondent would not be paid 
(including early withdrawal from the study); and (f) contact information for relevant investigators and IRB 
members. 
(C.2.a.2) Recruitment through online, print, and telephone ads. Participants recruited through these ads 
will be directed to a study webpage for detailed project information and a brief eligibility screener. The 
participant is then asked if he is willing to complete the brief Eligibility Screener to determine if he is eligible to 
participate.  
If he consents by clicking yes, his consent will be saved in a CSV file and later manually entered and saved 
into Assessment Center database as a data point. Once consent is given, the online Eligibility Screener form 
asks questions about demographics, including age, sex, sexual orientation, relationship status, and sexual 
behaviors. If the participant appears to be eligible, he will be asked to provide contact information so that study 
staff can follow-up and help him schedule an HIV test at a testing site or mail him an at-home HIV test kit to 
confirm his HIV negative status. 
Participants who choose to receive an at-home HIV test kit will be presented with an online consent addendum 
in the brief eligibility online screener. Participants who choose to be mailed a Home Access at-home specimen 
collection kit will be asked to consent to allowing Home Access to share their results with university staff. HIV 
results will be shared between Home Access and Northwestern University via SFTP.  Participants who choose 
to be mailed an Oraquick at-home HIV test will be asked to call university staff to report their results. University 
staff will use this information to confirm participants’ eligibility for the study. If a participant does not consent to 
share his results with university staff, he will be told that he is unable to participate in the study without 
confirming his HIV status with university staff. 
For participants who visit testing site. 
After the participant takes an HIV test and confirms his HIV negative status, the testing counselor will provide 
the participant with a tablet (or paper form) to complete the full screener and confirm eligibility. If eligible, an 
email with a study link to be sent to the participant. After clicking this link, the participant will provide online 
consent before beginning the study.  Participants read through our online consent form and click either "I 
Agree" or "Decline". Their responses are collected and stored in the Assessment Center database.
If the participant is ineligible, the online screener (or paper form) will ask if the participant would like to be 
contacted for other studies. All subjects will have to consent to being added to the IMPACT database before 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 33 of 50
research staff approaches them about participating in other studies. When consent is collected it will be saved 
in a CSV file and later manually entered and saved into the Assessment Center database.
For participants who take at-home test (Home Access) 
After the participant collects his blood specimen, he will mail it back to Home Access for analysis. The 
participant must use the pre-paid and pre-addressed envelope provided by Home Access to return their blood 
sample. After shipping his blood specimen, the participant will call a toll-free number after seven days to 
receive his test results and post-test counseling. Home Access will also share the results with Northwestern 
University via SFTP. If the participant tests negative, the project coordinator will call the participant to do a full 
screening over the phone and confirm eligibility. If eligible, an enrollment email with a link to the study to be 
sent to the participant’s email address. After clicking this link, the participant will provide online consent before 
beginning the study.  Participants read through our online consent form and click either "I Agree" or "Decline". 
Their responses are collected and stored in the Assessment Center database.
If ineligible, the project coordinator will let the participant know that he is ineligible for KIU! but may be eligible 
for other studies. If the participant is interested in other studies, the project coordinator will collect additional 
contact info and enter it into the IMPACT database. All subjects will have to consent to being added to the 
IMPACT database.
For participants who take at-home test (OraQuick) 
After the participant takes an at-home HIV test, he will be required to interpret his test result by comparing his 
test stick to the pictures and descriptions on test kit directions. After determining his test result, the participant 
will need to call study staff and report his test result. If the test result is negative, the project coordinator will do 
a full screening with the participant over the phone and confirm eligibility. If eligible, an enrollment email with a 
link to the study to be sent to the participant’s email address. After clicking this link, the participant will provide 
online consent before beginning the study.  Participants read through our online consent form and click either 
"I Agree" or "Decline". Their responses are collected and stored in the Assessment Center database.
If ineligible, the project coordinator will let the participant know that he is ineligible for KIU! but may be eligible 
for other studies. If the participant is interested in other studies, the project coordinator will collect additional 
contact info and to enter it into the IMPACT database. All subjects will have to consent to being added to the 
IMPACT database.
(C.2.a.3) Recruitment through street outreach. University staff and interns will approach potential 
participants in various venues that cater to YMSM (e.g. bars, clubs, bathhouses, and festivals) and request the 
participant’s permission to tell him about the study. If the participant consents, university staff and interns will 
explain the purposes of the study and invite him to answer questions that screen for preliminary eligibility. If the 
participant is interested, he will answer questions privately on a tablet (or paper form if a tablet is not available).
Those deemed eligible after completing the screener will be asked for their contact information, and will be 
contacted over the phone or email shortly after initial contact. University staff will offer information about the 
study and options for confirming their HIV negative status. If a participant wants to visit a partner CBO testing 
site or university site (Emory and Hunter), university staff will schedule an appointment for HIV testing. If a 
participant wants to take an at-home HIV test, university staff will mail out a FDA approved at-home HIV test. 
After taking an at-home HIV test or testing at a partner CBO testing site or university site (Emory and Hunter), 
participants will follow the same consent procedure as described under “Recruitment through online, print, and 
telephone ads.” 
For those who screen ineligible, the tablet will display ineligibility information and ask the participant if he would 
like to consent to enter his contact information for consideration in future studies. When paper and pencil are 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 34 of 50
used, participants will provide their contact information if they are interested in participating in Keep It Up! or 
other studies; they will not be aware of their eligibility at the time. The tablet (and paper form) includes a 
question in the contact information section asking for consent to add participants to the IMPACT database. All 
subjects will have to consent to being added to the IMPACT database before research staff approaches them 
about participating in other studies. Information collected on the tablet through Assessment Center will be 
securely uploaded to the tracking system. Information collected with paper and pencil will be manually entered 
by research study staff into Assessment Center within 3 days of receipt.
(C.2.a.4) Recruitment through community partner referrals. Participants recruited through referrals from 
community partners that do not perform direct, full recruitment for KIU! will be given a business card with a pre-
generated screener code and a link an online screener after testing HIV negative. The screener codes on 
these business cards will only be valid for three months from the date that they are given to community 
partners. New batches of business cards with valid screener codes will be regularly distributed to partner sites. 
Once the screener code expires, the participant will no longer be able to use it as verification that they have 
recently tested HIV negative. 
The participants will use the link to access a study webpage for detailed project information and a brief 
eligibility screener. The participant is then asked if he is willing to complete the brief Eligibility Screener to 
determine if he is eligible to participate.  
If he consents by clicking yes, his consent will be saved in a CSV file and later manually entered and saved 
into Assessment Center database as a data point. Once consent is given, the online Eligibility Screener form 
asks questions about demographics, including age, sex, sexual orientation, relationship status, and sexual 
behaviors. The participant will also be prompted to enter the screener code from the KIU! business card to 
verify that he has recently tested HIV negative. If the participant appears to be eligible, an automatic email  
with a study link will be triggered to be sent to the participant. After clicking this link, the participant will provide 
online consent before beginning the study.  Participants read through our online consent form and click either 
"I Agree" or "Decline". Their responses are collected and stored in the Assessment Center database.
For those who screen ineligible, the online screener will ask the participant if he would like to consent to enter 
his contact information for consideration in future studies. All subjects will have to consent to being added to 
the IMPACT database before research staff approaches them about participating in other studies. Information 
collected from the online screener will be exported to a CSV file and either the information will be uploaded to 
Assessment Center or manually entered by research study staff within 3 days of receipt.
(C.2.a.5) Recruitment through referrals from studies and research participant registries. Participants 
who become interested in the study through referrals from past and ongoing studies or IRB approved research 
participant registries at the university sites will be directed to a study webpage for detailed project information 
and a brief eligibility screener. Interested participants will follow the same consent procedure as described 
under “(C.2.a.2) Recruitment through online, print, and telephone ads.”
(C.2.a.6) Recruitment through CDPH clinics.
At the conclusion of an HIV testing appointment, the CDPH staff member will give male clients who have 
tested HIV negative, and appear to be between the ages of 18 and 29, a brief description of the study. They 
will mention to the client that CDPH is working with Northwestern University researchers on the Keep It Up! 
study as a way to learn more about the health behaviors and outcomes of young men who have sex with men 
(YMSM). The CDPH staff member will briefly mention features of the study, such as it being a paid, online HIV 
prevention research study. To minimize the risk for coercion, the CDPH staff member will emphasize the 
optional nature of participation and that it will not affect, in any way, their access to healthcare services. The 
CDPH staff member will ask the client if he is interested in hearing more details about the study and possibly 
screening to see if he is eligible. If the client indicates that he is interested, the CDPH staff member will walk 
the client over to a Keep It Up! staff member who is stationed in another private room at the clinic. CDPH staff 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 35 of 50
contact with participants will be limited to the referral of participants to the study and will not include performing 
informed consent or enrolling participants into the study.
Prior to administering the eligibility screener, Keep It Up! staff will request the client’s permission to tell him 
more about the study using a computer or tablet (or using pencil and paper, as necessary). The Keep It Up! 
staff member will give more details about the study (e.g. randomization, timeline, and study components). To 
minimize the risk for coercion, the staff member will reiterate the optional nature of participation and that it will 
not affect, in any way, their access to healthcare services at CDPH. If the client still appears interested, the 
Keep It Up! staff member will invite them to screen for the study. If the client consents to this, he will be given 
detailed information about the study on the computer or tablet (or paper form). The participant is then asked if 
he is willing to complete the brief online eligibility screener to determine if he is eligible to participate.
If he consents by clicking yes (or checking yes on the paper form), this consent will be documented as a data 
point in the Assessment Center database (if paper form used is used, the participant’s consent will be manually 
entered and saved into Assessment Center database as a data point). Once consent is given, the online 
eligibility screener form (or paper form) asks questions about demographics, including age, sex, sexual 
orientation, relationship status, and sexual behaviors.
If the participant is eligible, he will be asked to provide contact information so that study staff can email him the 
study link and to maintain contact throughout the course of the study. The Keep It Up! staff member will inform 
the participant of the opportunity to complete the baseline assessment and STI testing while on-site at the 
clinic. If the participant agrees, the Keep It Up! staff member will present him with a paper consent form that 
gives more information about the STI testing process on site. Participants who completed STI testing during 
their clinic visit will also be asked to consent to the clinic’s UIC lab sharing their STI results with study staff.  
After consenting to the on-site baseline and STI testing process, the participant will access the study using the 
link that we send via email. If the participant prefers to complete the baseline assessment and STI testing at 
home, before beginning the baseline assessment, he will be given an online consent form that outlines the at-
home STI testing process.
If the participant is ineligible or not interested in taking part in Keep It Up, the computer (or paper form) asks if 
the participant is interested in other studies. If the participant would like to be contacted for other studies, the 
tablet (or paper form) prompts the participant to give his contact information. All subjects will have to consent to 
being added to the IMPACT database before research staff approaches them about participating in other 
studies. When consent is collected on the tablet, it will be captured and documented in Assessment Center. 
When consent is collected on paper, it will be manually entered and saved into the Assessment Center 
database.
(C.2.a.7) Participants who complete the baseline assessment and STI testing on-site at Emory 
University, Hunter College, or Northwestern University. Across all study sites, participants who have 
screened eligible will be offered the opportunity to visit Emory University, Hunter College, or Northwestern 
University to complete the baseline assessment and complete their first set of self-administered STI test kits. 
Upon arrival at the university site, participants will be given a paper consent form by KIU! study staff that gives 
more information about the STI testing process on site.
(C.2.a.8) Participants who complete an exit interview.  Prior to administering phone exit interviews, study 
staff will give participants a brief description of what the interview entails (e.g. reasons for interview, time 
commitment, and incentive). After providing this information, the study team will ask the participant to give 
verbal consent that they would like to take part in the interview and agree to be recorded. If the participant is 
not interested in taking part in the interview or being recorded, study staff will thank them for their time and end 
the phone call. 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 36 of 50
(C.2.a.9) Participants who complete the repeat STI interview.  Prior to administering repeat STI phone 
interviews, study staff will give participants a brief description of what the interview entails (e.g. reasons for 
interview, time commitment, and incentive). After providing this information, the study team will ask the 
participant to give verbal consent that they would like to take part in the interview and agree to be recorded. If 
the participant is not interested in taking part in the interview or being recorded, study staff will thank them for 
their time and end the phone call. 
(C.2.a.10) Participants who complete the Timed Post-Release Testing. After potential participants screen 
as eligible, study staff who contact them to describe KIU! in greater detail. If they are still interested in 
participating in the timed post-release testing, a research assistant will email potential participants a link to an 
online consent form. Once the participant has consented, a research assistant will email them a unique link to 
the first intervention session. If a potential participant is not interested in completing the post-release testing, 
study staff will thank them for their time and end communication. 
C.2.b. Protection Against Potential Coercion.  Any study that recruits participants from a healthcare setting 
has the potential for participants to feel coerced into participating. The investigators have extensive experience 
confronting the ethical issues of participant recruitment in clinical settings. We will work with all involved IRBs 
to assure compliance with ethical and HIPPA standards.  Specifically, , testing staff will not be given incentives 
for recruiting participants, and we will make it clear to participants that their decision regarding participation will 
not affect, in any way, their access to healthcare services. 
C.2.c. Protection Against Discomfort. Multiple IRB-monitored studies at Northwestern University and our 
collaborating sites have used these or closely related measures with no known adverse effects.  Participants 
will be told that they do not have to answer any question they do not wish to.  Participants will be instructed 
that they can take a break and return to the online assessments or modules at a later time if they are 
experiencing mild distress. In the unlikely event that a participant experiences considerable distress they will 
be instructed to contact the Tracking Coordinator, who will refer them as appropriate to either the PI (who is a 
Licensed Clinical Psychologist) or the local project coordinators who can assist in making appropriate local 
referrals. 
C.2.d. Protection Against a Breach of Confidentiality. We will take several steps to prevent a breach of 
confidentiality. (1) We have developed systematic protocols for data handling and storage over longitudinal 
online studies.  Assessment Center has multiple mechanisms for providing data security in the transfer of data 
from client machines to Assessment Center servers.  These webservers use Secure Socket Layer (SSL or 
https) technology to encrypt data exchanged between the client and the server.  SSL will be implemented in all 
systems built for the project to ensure the encryption of data during transmission from the participants’ 
workstations to Assessment Center servers.  All project data will be stored on secured, password-protected 
database servers behind passcode-protected doors.  All components of the data system will be accessible only 
through a login and password unique to each assigned user on the study team.  The security access levels for 
these login accounts will be tiered based on specific roles and responsibilities for the project.  The features and 
privileges associated with each role and subsequent access level will be determined based on the level of 
access to confidential data needed by each team member.  For example, only project staff who maintain 
personal contact with participants will be given access to personal identifying information.  Tracking staff will 
not be given access to survey data so they cannot link identifying information to participant data.  All other 
team members will only have access to data that are tied to a random ID number that cannot be used to 
personally identify the participant.  Database administrators in the Health Information Technology (HIT) group 
in the PI’s Department at Northwestern who manage the Assessment Center technology will only access the 
database with names and contact information when absolutely necessary for maintaining the database. 
Staying consistent with information provided to participants in the consent form, data sources will be purged 
two years after the end of the study in 2017. De-identified data will be stored indefinitely on secure shared 
drives housed on Northwestern’s servers.  (2) All study staff, including HIT staff, will be trained in Human 
Subjects research and will sign confidentiality agreements prior to participation in the project. (3) The HIT 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 37 of 50
technical staff will also work with study collaborators to develop standard protocols for maintaining a secure 
environment for data submission from the clinic to the Assessment Center servers, including potentially 
implementing a Virtual Private Network (VPN) connection that will be used to enter identifying information, to 
provide an additional layer of encryption for that sensitive data. (4) We will employ the latest in encryption 
technology and server security (e.g. SSL encryption technology, firewalls, removing identifying information 
from servers accessible to the public) to minimize this possibility. (5)  All names and contact information for the 
follow-up of participants will be stored in a database that is separate from the participant’s responses to the 
screener and the assessment.  Only a random study ID will be stored with the participant’s data for linkage to 
subsequent data later. (6) Participants recruited through ads and referrals will complete an online screener 
using Limesurvey. The LimeSurvey software is hosted on the kiu2.org domain on a virtual private server with 
an SSL encryption certificate. Only the PI and IMPACT Program webmaster have the passwords to the root 
folder of the virtual private server. All data collected by LimeSurvey will be saved as CSV files in password 
protected folders on a secure university shared drive. (7) As we have done in over five prior studies, we will 
obtain a Certificate of Confidentiality issued by the federal government and the certificate will be maintained 
through the life of the research project. This protects the data from subpoena and thus prevents the data from 
being used against the participant. (8) We will inform study participants of the limits of the certificate of 
confidentiality during the consent process. (9) The majority of our messages to participants are automatically 
sent to participants through Assessment Center and other survey programs to reduce change of human error. 
Most messages not automatically sent are individualized messages sent to one participant at a time. Our 
protocol also instructs study staff to never send group emails without using blind carbon copying (bcc). We will 
use mail merge software for emails that need to go to more than one participant rather than sending group 
emails. Mail merge only sends one email to each participant. No participants will be carbon copied on an email 
sent via mail merge. If, for any reason, it is ever preferable to send a group email using bcc, the email will be 
reviewed by at least two staff members before it is sent out. (10) When texting our participants, we will 
frequently use a service, Red Oxygen, that allows us to send texts from our secure Northwestern Outlook 
email accounts. Red Oxygen users can send messages to specific participants as well as bulk messages to 
lists of participants.  Red Oxygen’s Outlook add-on allows users to email individual text messages to 
participants and also receive and follow-up on responses through their Outlook account. Red Oxygen also 
offers standalone bulk texting software (Bulk SMS) that gives users the ability to email large text message 
blasts through their Outlook account. When bulk messages are sent, each recipient will only see the 
designated Keep It Up! phone number tied to our Red Oxygen account. They will not see the phone numbers 
of other participants. As with the individual messages, all responses to bulk messages are routed to KIU! staff’s 
Northwestern Outlook email account. Red Oxygen follows the National Privacy Principles as set out in the 
Privacy Act 1988 (Cth) and the Privacy Amendment (Private Sector) Act 2000. Red Oxygen is not allowed to 
use or sell any of the data from text messages that are sent through their systems. Text messages are 
considered phone calls and the use or sell of these messages would be considered wiretapping. More of Red 
Oxygen’s privacy policy can be reviewed at http://redoxygen.com/privacy-policy/. 
To help protect participants’ confidentiality while they complete the Internet-based assessments and 
intervention modules, we will: (1) remind participants each time they log on that they will be answering 
personal questions; (2) encourage them to complete the modules and questionnaires in a private environment; 
and (3) suggest spaces with greater privacy (e.g. in their dorm room with the door closed, at a coffee shop in 
an area where no one can look over their shoulder, etc). 
To help protect participants’ confidentiality while they complete the exit interviews as well as the repeat STI 
interviews, we will: (1) administer and record the interview in a private office space; (2) encourage participants 
to complete the interview in a private setting; (3) immediately transfer the recording of the interview from the 
digital recorder to a password protected folder on a secure university shared drive that only the study team can 
assess; (4) delete the interview from the digital recorder as soon as the audio file is uploaded to the secure 
university shared drive; and (5) save transcriptions of the audio recording to the secure university shared drive 
that only the study team can assess.
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 38 of 50
We have also developed procedures with the STI testing laboratory at the CDC to prevent a breach of 
confidentiality:  (1) Participants will be given unique user IDs and passwords for all STI testing materials and 
for accessing STI test results.  (2) Electronic communications with the CDC lab will be conducted via secure 
emails.  This exporting de-identified STI test results to research study staff.  (3) At the CDC lab, all STI test 
results are stored on a secure, password-protected server and the results will only be provided to the research 
team with evidence of an IRB-approved research protocol. (4) For participants who collect the urine and rectal 
samples on-site at Emory University, Hunter College, or Northwestern University, samples that are not mailed 
out the same day will be stored in a locked storage cabinet by KIU! study staff for no more than 7 days before 
being mailed to the CDC. The APTIMA Combo 2 Assay collection and transport tubes contain solution that 
protects the samples from degradation during storage. Urine samples can be stored for up to 30 days and 
rectal swabs can be stored for up to 60 days without refrigeration. Clinical specimens collected for this study 
will be tested for Chlamydia trachomatis and Neisseria gonorrhoeae. Any remaining specimen material will be 
unlinked from patient identifiers and stored by the CDC, up to 20 years, for possible future bacterial or viral 
research. Any future studies with these specimens will not involve analysis of human nucleic acid.
Participants who are recruited from CDPH clinic sites and complete STI testing as a part of their clinic visit will 
be asked to authorize the clinic’s UIC lab sharing their STI results with the study team. Participants will be 
asked to read and sign a consent form that includes a section on HIPAA Authorization to Use or Disclose 
Health Information that describes which personal health information will be shared and how this health 
information will be shared. To help protect participants against a breach of confidentiality, all results will be 
shared via fax as the HIPAA Privacy Rule allows protected health information to be shared in this manner. KIU! 
staff will call the lab client services and provide the patient name, date of birth, and date of specimen collection 
in order to verify that correct results are sent. In addition, KIU! staff will verify the fax number to which the 
results should be sent. The fax machine to which results will be sent is located in the mail room of 
Northwestern’s Department of Medical Social Sciences (MSS). The MMS office is not open to the general 
public. The office is only accessible to staff with ID cards and invited guests who are buzzed into the office. 
Guests are typically escorted through the office by staff. In general, only MSS staff access the fax machine.  
KIU! staff will ask the lab to confirm the time that the fax will be sent so that the results are immediately 
retrieved from the fax machine. All faxes of results will also include a fax cover sheet that describes the 
confidential nature of the contents and has instructions to notify the sender to report any faxes received in error 
and destroy the misdirected documents. The cover letter also warns that any mishandling of the documents, 
such as copying or dissemination, is illegal.
In order to avoid data entry errors and sharing incorrect results with participants, the study team will do 
comprehensive checks on logged results and the PDFs of results sent to participants. First, in order to get a list 
of new STI results to be entered into REDCap and shared with participants, one of the two research assistants 
will apply filters to the running spreadsheet of results received from the CDC. Second, the Project Director and 
other research assistant will double check that the list is accurate. Once the list is confirmed as accurate, a 
research assistant will log them in REDCap and prepare PDFs of results to send to participants. Finally, the 
Project Director will check the results logged in REDCap as well as the PDFs before they are sent to 
participants. For individual results received from CDPH, the Project Director will compare the lab report to the 
results logged in REDCap and the PDF draft of results. In addition to this system of checks, the study Data 
Manager will perform quality assurance every two weeks on all STI data entered in REDCap.  
We have also developed procedures with Home Access Health Corporation to prevent a breach of 
confidentiality:  (1) Participants will be given a unique 11 digit code number for all specimen collection kit 
materials and for accessing HIV test results. (2) Electronic communications with Home Access will be 
conducted via SFTP.  HIV test results linked to unique 11 digit code number will be shared with research study 
staff through the SFTP. (3) At Home Access, all HIV test results are stored on a secure, password-protected 
server and the results will only be provided to the research team with evidence of an IRB-approved research 
protocol. (4) Blood specimens collected for this study will be tested for HIV-1. Any remaining specimen material 
will stored by Home Access for up to 3 months before being disposed. The storage of blood samples for up to 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 39 of 50
3 months is part of the Home Access standard testing protocol approved by the FDA in 1996. The samples are 
not retested or used for any other purposes while stored.
Participants who complete at-home HIV testing using an Oraquick test kit will be given a unique five digit code 
for all HIV testing materials. We will ask that participants only share their results over the phone with a 
university staff member. Participants who use Oraquick test kits will be responsible for disposing their oral 
swabs. 
C.2.e. HIV Seroconversion and STI diagnosis during the course of the study.  After initial enrollment and 
intervention completion, if a participant reports an HIV seroconversion on the questionnaires he will still be 
eligible to participate in subsequent assessments.  Participants who report seroconversion during the study will 
be referred for care at one of our community partners or appropriately referred elsewhere if he requests.  Each 
site coordinator based at Northwestern University, Emory University, and Hunter College will maintain lists of 
referrals and will assist in making linkages to care. 
With regard to STI testing and diagnosis:

All participants will be tested for Gonorrhea and Chlamydia at baseline and post-intervention (12 month 
follow-up). Participants who test positive at baseline will be tested at 3 month and 6 month follow-up in 
addition to at the 12 month follow-up. 

For participants who complete STI testing at a CDPH clinic, CDPH clinic staff will contact participants 
with their STI test results. For those who are positive for Chlamydia or Gonorrhea, CDPH clinic staff will 
report the results to the state or local health department. Clinic staff will also make sure that participants 
receive appropriate medical attention. 

Participants who complete STI testing at home, at a university site, or at a CDPH clinic will be sent an 
encrypted email that they can only open using their unique study ID to receive their STI test results. 
Participants may speak to Keep It Up! study staff if they have any questions or concerns about their 
results. For those who are positive for Chlamydia or Gonorrhoeae, Keep It Up! study staff are required 
to report the results to state or local health department. Research study staff will follow up with the 
client to be sure they receive appropriate medical attention.  Research staff will: (1) encourage 
participants to notify current and past sex partners; (2) provide local referrals for free or low cost 
treatment; and (3) make a legally required confidential report to the appropriate health department. 
Health department professionals (or disease information specialists) may contact and interview the 
participant to obtain names of his sex partners to notify and encourage them to seek evaluation and 
treatment. The ability to link YMSM to needed clinical services is a distinct strength of this proposal as 
the multi-disciplinary investigative has considerable experience doing research with high-risk youth 
populations and the program of research is embedded in a larger network of CBOs designed to meet 
the needs of LGBT youth and YMSM.
C.3. POTENTIAL BENEFITS OF THE STUDY TO SUBJECTS AND OTHERS
There are several potential benefits to this study. YMSM may benefit by reducing their risky sexual behavior, 
accessing HIV testing services, and decreasing their risk of HIV/STI infection as a result of program 
participation.  Collaborating CBOs may benefit from the collected evaluation data by improving existing HIV 
testing services and delivering prevention programs to YMSM.  Society will benefit from the creation of an 
intervention that reduces risk behavior among YMSM, limiting the spread of HIV/AIDS, and addressing health 
disparities. Possible risks are outweighed by the potential to reduce HIV/STI-risk among YMSM. 
In addition to these general study benefits, the exit interviews and the repeat STI interviews may provide 
information that can be used to refine future iterations of the intervention and better inform research efforts 
towards comprehensive at-home STI testing. As with the current intervention, these future iterations may 
benefit society by reducing risk behaviors among YMSM and limiting the spread of HIV and STIs. 
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 40 of 50
C.4. IMPORTANCE OF THE KNOWLEDGE TO BE GAINED 
Rising HIV/AIDS infection rates among YMSM and lack of existing evidence-based prevention programs 
warrant research to develop and test tailored HIV prevention programs. The purpose of this study is to 
systematically test an online, highly engaging and promising HIV prevention program tailored to YMSM. Such 
an intervention could play an important role in providing accessible prevention to YMSM, in particular YMSM 
seeking HIV testing. Currently, most testing clinics have limited time and resources to provide prevention. This 
approach represents an opportunity to develop a cost-effective and easy-to-use intervention that will engage 
participants and motivate and teach risk reduction behaviors. 
C.5. DATA SAFETY AND MONITORING PLAN (DSMP)
Please see attached DSMP submitted to NIDA on June 4, 2012.
C.6. CLINICAL TRIAL REGISTRATION
The trial will be registered at the ClinicalTrials.gov Protocol Registration System Information Website by the 
Principal Investigator after Northwestern University, Hunter College, Emory University receive IRB approval for 
this study.
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 41 of 50
REFERENCES CITED
1.
White House Office of National AIDS Policy. National HIV/AIDS strategy for the united States. 
Washington, DC2010.
2.
Centers for Disease Control and Prevention. HIV Surveillance in Adolescents and Young Adults. 2010; 
http://www.cdc.gov/hiv/topics/surveillance/resources/slides/adolescents/index.htm.
3.
Centers for Disease Control and Prevention. Trends in HIV/AIDS diagnoses among men who have sex 
with men- 33 states, 2001- 2006. MMWR Morb Mortal Wkly Rep. 2008;57:681-686.
4.
Prejean J, Song R, Hernandez A, et al. Estimated HIV Incidence in the United States, 2006-2009. PLoS 
ONE. 2011;6(8):e17502.
5.
Mustanski B, Lyons T, Garcia SC. Internet Use and Sexual Health of Young Men Who Have Sex with 
Men: A Mixed-Methods Study. Arch Sex Behav. Feb 25 2010.
6.
Fisher JD, Fisher WA, Williams SS, Malloy TE. Empirical tests of an information-motivation-behavioral 
skills model of AIDS-preventive behavior with gay men and heterosexual university students. Health 
Psychol. May 1994;13(3):238-250.
7.
Fisher JD, Fisher WA. The information-motivation-behavioral skills model. In: DiClemente RJ, Crosby 
RA, Kegler MC, eds. Emerging theories in health promotion practice and research: Strategies for 
improving public health. San Francisco: Jossey-Bass; 2002:40-70.
8.
NIH Office of AIDS Research. Recommendations from the OAR Social and Behavioral HIV Prevention 
Research Think Tank. unpublished report. 2010.
9.
Cella D, Yount S, Rothrock N, et al. The Patient-Reported Outcomes Measurement Information System 
(PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Med. Care. May 
2007;45(5 Suppl 1):S3-S11.
10.
Cella D, Riley W, Stone A, et al. The Patient-Reported Outcomes Measurement Information System 
(PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-
2008. J. Clin. Epidemiol. Nov 2010;63(11):1179-1194.
11.
Gershon RC, Rothrock N, Hanrahan R, Bass M, Cella D. The use of PROMIS and assessment center 
to deliver patient-reported outcome measures in clinical research. J. Appl. Meas. 2010;11(3):304-314.
12.
Gershon R, Rothrock NE, Hanrahan RT, Jansky LJ, Harniss M, Riley W. The development of a clinical 
outcomes survey research application: Assessment Center. Qual. Life Res. Jun 2010;19(5):677-685.
13.
CDC. Compendium of Evidence-Based HIV Prevention Interventions. 2011; 
http://www.cdc.gov/hiv/topics/research/prs/evidence-based-interventions.htm. Accessed 9/10/11.
14.
CDC. HIV Surveillance in Adolescents and Young Adults. 2010; 
http://www.cdc.gov/hiv/topics/surveillance/resources/slides/adolescents/index.htm. Accessed 
12/2/2010.
15.
CDC. Prevalence and awareness of HIV infection among men who have sex with men --- 21 cities, 
United States, 2008. MMWR Morb. Mortal. Wkly. Rep. Sep 24 2010;59(37):1201-1207.
16.
CDC. National HIV Behavioral Surveillance: HIV risk and testing behaviors among young MSM. Atlanta: 
US DHHS; 2009.
17.
Steele BC, Melendez-Morales L, Campoluci R, DeLuca N, Dean HD. Health disparities in HIV/AIDS, 
viral hepatitis, sexually transmitted dieseases, and Turberculosis: Issues, burden, and response, a 
retrospective review 2000-2004. Atlanta: Center for Disease Control and Prevention;2007.
18.
Harper GW. Sex isn't that simple: Culture and context in HIV prevention interventions for gay and 
bisexual male adolescents. American Psychologist. 2007;62:806-813.
19.
Mustanski BS, Newcomb ME, Du Bois SN, Garcia SC, Grov C. HIV in Young Men Who Have Sex with 
Men: A Review of Epidemiology, Risk and Protective Factors, and Interventions. Annual Review of Sex 
Research Special Issue J. Sex Res. Mar 2011;48(2):218-253.
20.
Herbst JH, Sherba RT, Crepaz N, et al. A meta-analytic review of HIV behavioral interventions for 
reducing sexual risk behavior of men who have sex with men. J. Acquir. Immune Defic. Syndr. Jun 1 
2005;39(2):228-241.
21.
Oh H, Rizo C, Enkin M, Jadad A. What is eHealth (3): a systematic review of published definitions. J. 
Med. Internet Res. 2005;7(1):e1.
22.
Chiasson MA, Hirshfield S, Rietmeijer C. HIV prevention and care in the digital age. J. Acquir. Immune 
Defic. Syndr. Dec 15 2010;55 Suppl 2:S94-97.
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 42 of 50
23.
Bennett GG, Glasgow RE. The delivery of public health interventions via the Internet: actualizing their 
potential. Annu. Rev. Public Health. Apr 29 2009;30:273-292.
24.
Abrams DB, Orleans CT, Niaura RS, Goldstein MG, Prochaska JO, Velicer W. Integrating individual 
and public health perspectives for treatment of tobacco dependence under managed health care: a 
combined stepped-care and matching model. Ann. Behav. Med. Fall 1996;18(4):290-304.
25.
Glasgow RE, Klesges LM, Dzewaltowski DA, Estabrooks PA, Vogt TM. Evaluating the impact of health 
promotion programs: using the RE-AIM framework to form summary measures for decision making 
involving complex issues. Health Educ. Res. Oct 2006;21(5):688-694.
26.
Johnson WD, Diaz RM, Flanders WD, et al. Behavioral interventions to reduce risk for sexual 
transmission of HIV among men who have sex with men. Cochrane Database Syst. Rev. 
2008(3):CD001230.
27.
Herbst JH, Beeker C, Mathew A, et al. The effectiveness of individual-, group-, and community-level 
HIV behavioral risk-reduction interventions for adult men who have sex with men: a systematic review. 
Am. J. Prev. Med. Apr 2007;32(4 Suppl):S38-67.
28.
Lyles CM, Kay LS, Crepaz N, et al. Best-evidence interventions: findings from a systematic review of 
HIV behavioral interventions for US populations at high risk, 2000-2004. Am. J. Public Health. Jan 
2007;97(1):133-143.
29.
Mullen PD, Ramirez G, Strouse D, Hedges LV, Sogolow E. Meta-analysis of the effects of behavioral 
HIV prevention interventions on the sexual risk behavior of sexually experienced adolescents in 
controlled studies in the United States. J. Acquir. Immune Defic. Syndr. Jul 1 2002;30 Suppl 1:S94-
S105.
30.
Pedlow CT, Carey MP. Developmentally appropriate sexual risk reduction interventions for 
adolescents: rationale, review of interventions, and recommendations for research and practice. Ann. 
Behav. Med. Jun 2004;27(3):172-184.
31.
Jemmott JB, 3rd, Jemmott LS. HIV risk reduction behavioral interventions with heterosexual 
adolescents. AIDS. Sep 2000;14 Suppl 2:S40-52.
32.
Kelly JA, Spielberg F, McAuliffe TL. Defining, designing, implementing, and evaluating phase 4 HIV 
prevention effectiveness trials for vulnerable populations. J. Acquir. Immune Defic. Syndr. Mar 1 
2008;47 Suppl 1:S28-33.
33.
Rietmeijer CA. Risk reduction counselling for prevention of sexually transmitted infections: how it works 
and how to make it work. Sex. Transm. Infect. Feb 2007;83(1):2-9.
34.
Solomon J, Card JJ, Malow RM. Adapting efficacious interventions: advancing translational research in 
HIV prevention. Eval. Health Prof. Jun 2006;29(2):162-194.
35.
Bell SG, Newcomer SF, Bachrach C, et al. Challenges in replicating interventions. J. Adolesc. Health. 
Jun 2007;40(6):514-520.
36.
Neumann MS, Sogolow ED. Replicating effective programs: HIV/AIDS prevention technology transfer. 
AIDS Educ. Prev. 2000;12(5 Suppl):35-48.
37.
Swendeman D, Rotheram-Borus MJ. Innovation in sexually transmitted disease and HIV prevention: 
internet and mobile phone delivery vehicles for global diffusion. Curr. Opin. Psychiatry. Mar 
2010;23(2):139-144.
38.
Noar SM, Black HG, Pierce LB. Efficacy of computer technology-based HIV prevention interventions: a 
meta-analysis. AIDS. Jan 2 2009;23(1):107-115.
39.
Bailey JV, Murray E, Rait G, et al. Interactive computer-based interventions for sexual health 
promotion. Cochrane Database Syst. Rev. 2010(9):CD006483.
40.
Noar SM. Computer technology-based interventions in HIV prevention: state of the evidence and future 
directions for research. AIDS Care. May 2011;23(5):525-533.
41.
Ybarra ML, Bull SS. Current trends in Internet- and cell phone-based HIV prevention and intervention 
programs. Curr. HIV/AIDS Rep. Dec 2007;4(4):201-207.
42.
Lewis MA, Uhrig JD, Ayala G, Stryker J. Reaching men who have sex with men for hiv prevention 
messaging with new media: Recommendations from an expert consultation. Ann Forum Collab HIV 
Res. 2011;13:10-16.
43.
Sullivan PS, Khosropour CM, Luisi N, et al. Bias in online recruitment and retention of racial and ethnic 
minority men who have sex with men. J. Med. Internet Res. 2011;13(2):e38.
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 43 of 50
44.
Finlayson TJ, Le B, Smith A, et al. HIV risk, prevention, and testing behaviors among men who have 
sex with men--National HIV Behavioral Surveillance System, 21 U.S. cities, United States, 2008. 
MMWR Surveill. Summ. Oct 28 2011;60(14):1-34.
45.
Ruiz MS, Institute of Medicine (U.S.). Committee on HIV Prevention Strategies in the United States. No 
time to lose : getting more from HIV prevention. Washington, D.C.: National Academy Press; 2001.
46.
Du Bois SN, Johnson SE, Mustanski B. Examining Racial and Ethnic Minority Differences Among 
YMSM During Recruitment for an Online HIV Prevention Intervention Study. AIDS Behav. Oct 11 2011.
47.
Pew Internet & American Life Project. Trend Data. 2009; www.pewinternet.org. Accessed 4/9/09.
48.
Mustanski B, Lyons T, Garcia SC. Internet use and sexual health of young men who have sex with 
men: A mixed-methods study. Arch. Sex. Behav. 2010;in press.
49.
Rosser BR, Oakes JM, Horvath KJ, Konstan JA, Danilenko GP, Peterson JL. HIV sexual risk behavior 
by men who use the Internet to seek sex with men: results of the Men's INTernet Sex Study-II (MINTS-
II). AIDS Behav. Jun 2009;13(3):488-498.
50.
Rosser BR, Miner MH, Bockting WO, et al. HIV risk and the internet: results of the Men's INTernet Sex 
(MINTS) Study. AIDS Behav. Aug 2009;13(4):746-756.
51.
Raymond HF, Rebchook G, Curotto A, et al. Comparing internet-based and venue-based methods to 
sample MSM in the San Francisco Bay Area. AIDS Behav. Feb 2010;14(1):218-224.
52.
Bull SS, Lloyd L, Rietmeijer C, McFarlane M. Recruitment and retention of an online sample for an HIV 
prevention intervention targeting men who have sex with men: the Smart Sex Quest Project. AIDS 
Care. Nov 2004;16(8):931-943.
53.
Chiasson MA, Hirshfield S, Remien RH, Humberstone M, Wong T, Wolitski RJ. A comparison of on-line 
and off-line sexual risk in men who have sex with men: An event-based on-line survey. J. Acquir. 
Immune Defic. Syndr. Feb 1 2007;44(2):235-243.
54.
Hirshfield S, Remien RH, Walavalkar I, Chiasson MA. Crystal methamphetamine use predicts incident 
STD infection among men who have sex with men recruited online: a nested case-control study. J. 
Med. Internet Res. Nov 29 2004;6(4):e41.
55.
Halkitis PN, Parsons JT. Intentional unsafe sex (barebacking) among HIV-positive gay men who seek 
sexual partners on the internet. AIDS Care. Jun 2003;15(3):367-378.
56.
Weinhardt LS, Carey MP, Johnson BT, Bickham NL. Effects of HIV counseling and testing on sexual 
risk behavior: a meta-analytic review of published research, 1985-1997. Am. J. Public Health. Sep 
1999;89(9):1397-1405.
57.
Kamb ML, Fishbein M, Douglas JM, Jr., et al. Efficacy of risk-reduction counseling to prevent human 
immunodeficiency virus and sexually transmitted diseases: a randomized controlled trial. Project 
RESPECT Study Group. JAMA. Oct 7 1998;280(13):1161-1167.
58.
Fernyak SE, Page-Shafer K, Kellogg TA, McFarland W, Katz MH. Risk behaviors and HIV incidence 
among repeat testers at publicly funded HIV testing sites in San Francisco. J. Acquir. Immune Defic. 
Syndr. Sep 1 2002;31(1):63-70.
59.
Cassell MM, Halperin DT, Shelton JD, Stanton D. Risk compensation: the Achilles' heel of innovations 
in HIV prevention? BMJ. Mar 11 2006;332(7541):605-607.
60.
Eaton LA, Kalichman S. Risk compensation in HIV prevention: implications for vaccines, microbicides, 
and other biomedical HIV prevention technologies. Curr. HIV/AIDS Rep. Dec 2007;4(4):165-172.
61.
Kalichman S, Eaton L, Pinkerton S. Circumcision for HIV prevention: failure to fully account for 
behavioral risk compensation. PLoS Med. Mar 27 2007;4(3):e138; author reply e146.
62.
Pinkerton SD. Sexual risk compensation and HIV/STD transmission: empirical evidence and theoretical 
considerations. Risk Anal. Aug 2001;21(4):727-736.
63.
Silvestre AJ, Gehl MB, Encandela J, Schelzel G. A participant observation study using actors at 30 
publicly funded HIV counseling and testing sites in Pennsylvania. Am. J. Public Health. Jul 
2000;90(7):1096-1099.
64.
Garofalo R, Harper GW. Not all adolescents are the same: addressing the unique needs of gay and 
bisexual male youth. Adolesc. Med. Oct 2003;14(3):595-611, vi.
65.
Rosario M, Schrimshaw EW, Hunter J. A model of sexual risk behaviors among young gay and 
bisexual men: Longitudinal associations of mental health, substance abuse, sexual abuse, and the 
coming-out process. AIDS Educ. Prev. Oct 2006;18(5):444-460.
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 44 of 50
66.
Coker TR, Austin SB, Schuster MA. The health and health care of lesbian, gay, and bisexual 
adolescents. Annu. Rev. Public Health. Apr 21;31:457-477.
67.
Mustanski B, Newcomb M, DuBois S, Garcia SC, Grov C. HIV in young men who have sex with men: A 
review of epidemiology, risk and protective factors, and interventions. Annual Review of Sex Research 
(imprint of Journal of Sex Research). 2011;48(2-3):218-253.
68.
Garofalo R, Mustanski B, Donenberg G. Parents know and parents matter; is it time to develop family-
based HIV prevention programs for young men who have sex with men? J. Adolesc. Health. Aug 
2008;43(2):201-204.
69.
Ryan C, Huebner D, Diaz RM, Sanchez J. Family rejection as a predictor of negative health outcomes 
in white and Latino lesbian, gay, and bisexual young adults. Pediatrics. Jan 2009;123(1):346-352.
70.
LaSala MC. Parental influence, gay youths, and safer sex. Health Soc. Work. Feb 2007;32(1):49-55.
71.
Harper GW. Sex isn't that simple: culture and context in HIV prevention interventions for gay and 
bisexual male adolescents. Am. Psychol. Nov 2007;62(8):803-819.
72.
Arnett JJ. Emerging adulthood. A theory of development from the late teens through the twenties. Am. 
Psychol. May 2000;55(5):469-480.
73.
Arnett JJ. Conceptions of the transition to adulthood: Perspectives from adolescence through midlfe. 
Journal of Adult Development. 2001;8:133-143.
74.
Arnett JJ. Conceptions of the transition to adulthood among emerging adults in American ethnic groups. 
New Dir. Child Adolesc. Dev. Summer 2003(100):63-75.
75.
Steinberg L. A Social Neuroscience Perspective on Adolescent Risk-Taking. Dev. Rev. Mar 
2008;28(1):78-106.
76.
Bingham TA, Harawa NT, Johnson DF, Secura GM, MacKellar DA, Valleroy LA. The effect of partner 
characteristics on HIV infection among African American men who have sex with men in the Young 
Men's Survey, Los Angeles, 1999-2000. AIDS Educ. Prev. Feb 2003;15(1 Suppl A):39-52.
77.
Hurt CB, Matthews DD, Calabria MS, et al. Sex with Older Partners Is Associated With Primary HIV 
Infection Among Men Who Have Sex With Men in North Carolina. J. Acquir. Immune Defic. Syndr. Jan 
7 2010;54:185-190.
78.
Mustanski B, Newcomb M. Longitudinal effects of partner and relationship characteristics on sexual risk 
taking in ethnically-diverse young men who have sex with men. under review.
79.
Mustanski B. Moderating Effects of Age on the Alcohol and Sexual Risk Taking Association: An Online 
Daily Diary Study of Men Who have Sex with Men. AIDS Behav. Jan 2008;12(1):118-126.
80.
CDC. Estimates of new HIV infections in the United States, 2006–2009: U.S Department of Health and 
Human Services, Center for Disease Control and Prevention;2011.
81.
CDC. Surveillance Summaries: Sexual and Reproductive Health of Persons Aged 10–24 Years — 
United States, 2002–2007. WWMR. 2009;58.
82.
Clerkin EM, Newcomb ME, Mustanski B. Unpacking the racial disparity in HIV rates: the effect of race 
on risky sexual behavior among Black young men who have sex with men (YMSM). J. Behav. Med. 
Aug 2011;34(4):237-243.
83.
Garofalo R, Mustanski B, Johnson A, Emerson E. Exploring Factors That Underlie Racial/Ethnic 
Disparities in HIV Risk among Young Men Who Have Sex with Men. Journal of Urban Health-Bulletin of 
the New York Academy of Medicine. Mar 2010;87(2):318-323.
84.
Harawa NT, Greenland S, Bingham TA, et al. Associations of race/ethnicity with HIV prevalence and 
HIV-related behaviors among young men who have sex with men in 7 urban centers in the United 
States. J. Acquir. Immune Defic. Syndr. Apr 15 2004;35(5):526-536.
85.
Mustanski B, Stauffer A, Garofalo R. At the intersection of HIV/AIDS Disparities: Young African 
American Men who Have Sex with Men. In: Johnson W, ed. Social Work and Social Welfare 
Responses to African American Males: What Have We Seen with Our Own Eyes. Oxford, UK: Oxford 
Press; 2009.
86.
Millett GA, Flores SA, Peterson JL, Bakeman R. Explaining disparities in HIV infection among black and 
white men who have sex with men: a meta-analysis of HIV risk behaviors. AIDS. Oct 1 
2007;21(15):2083-2091.
87.
Raj A, Amaro H. Culturally tailoring HIV/AIDS prevention programs: Why, when, and how? In: Kazarian 
SS, Evans DR, eds. Handbook of cultural health psychology. San Diego, Calif.: Academic Press; 2001.
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 45 of 50
88.
Wilson BD, Miller RL. Examining strategies for culturally grounded HIV prevention: a review. AIDS 
Educ. Prev. Apr 2003;15(2):184-202.
89.
Resnicow K, DiIorio C, Davis R. Culture and the development of HIV prevention and treatment 
programs. In: Edgar T, Noar SM, Freimuth VS, eds. Communication perspectives on HIV/AIDS for the 
21st century. New York: Lawrence Ealbaum Associates; 2008.
90.
Savin-Williams RC. Who's Gay? Does It Matter? Current Directions in Psychological Science. 
2006;15(1):40-44.
91.
Young RM, Meyer IH. The Trouble With "MSM" and "WSW": Erasure of the Sexual-Minority Person in 
Public Health Discourse. Am. J. Public Health. July 1, 2005 2005;95(7):1144-1149.
92.
Wheeler DP, Lauby JL, Liu KL, Van Sluytman LG, Murrill C. A Comparative Analysis of Sexual Risk 
Characteristics of Black Men Who Have Sex with Men or with Men and Women. Archives of sexual 
behavior. May 29 2008.
93.
Lauby JL, Millett GA, Lapollo AB, Bond L, Murrill CS, Marks G. Sexual Risk Behaviors of HIV-Positive, 
HIV-Negative, and Serostatus-Unknown Black Men Who Have Sex with Men and Women. Archives of 
sexual behavior. Jun 3 2008.
94.
Flores SA, Mansergh G, Marks G, Guzman R, Colfax G. Gay identity-related factors and sexual risk 
among men who have sex with men in San Francisco. AIDS Educ. Prev. Apr 2009;21(2):91-103.
95.
Mustanski B, Newcomb ME, Clerkin EM. Relationship characteristics and sexual risk-taking in young 
men who have sex with men. Health Psychol. Sep 2011;30(5):597-605.
96.
Sullivan PS, Salazar L, Buchbinder S, Sanchez TH. Estimating the proportion of HIV transmissions 
from main sex partners among men who have sex with men in five US cities. AIDS. Jun 1 
2009;23(9):1153-1162.
97.
Mustanski BS. Getting Wired: Exploiting the Internet for the collection of valid sexuality data. Journal of 
Sex Research. 2001;38:292-301.
98.
Mustanski BS. Methodological issues in online sexual health research. In: Liamputtong P, ed. Health 
research in cyberspace: Methodological, practical and personal experiences in doing health and social 
science researching online.: Nova Science Publishers, Inc.; 2006:Chapter 7.
99.
Konstan JA, Rosser BRS, Ross MW, Stanton J, Edwards WM. The story of Subject Naught: A 
cautionary but optimistic tale of internet survey research. . Journal of Computer-Mediated 
Communication. 2005;10.
100.
Danaher BG, Jazdzewski SA, McKay HG, Hudson CR. Bandwidth constraints to using video and other 
rich media in behavior change websites. J. Med. Internet Res. Sep 16 2005;7(4):e49.
101.
Pequegnat W, Rosser BR, Bowen AM, et al. Conducting Internet-Based HIV/STD Prevention Survey 
Research: Considerations in Design and Evaluation. AIDS Behav. Oct 20 2006;11:505-521.
102.
Mustanski B, Garofalo R, Herrick A, Donenberg G. Psychosocial health problems increase risk for HIV 
among urban young men who have sex with men: preliminary evidence of a syndemic in need of 
attention. Ann. Behav. Med. Aug 2007;34(1):37-45.
103.
Fisher JD, Fisher WA, Williams SS, Malloy TE. Empirical tests of an information-motivation-behavioral 
skills model of AIDS-preventive behavior with gay men and heterosexual university students. Health 
Psychol. 1994;13(3):238-250.
104.
Kalichman SC, Picciano JF, Roffman RA. Motivation to reduce HIV risk behaviors in the context of the 
Information, Motivation and Behavioral Skills (IMB) model of HIV prevention. J Health Psychol. 
2008;13(5):680-689.
105.
Fisher CM. Are Information, Motivation, and Behavioral skills linked with HIV-related sexual risk among 
young men who have sex with men? J HIV AIDS Soc Ser. 2011;10(1):5-21.
106.
Clark RC, Mayer RE. E-Learning and the science of instruction : Proven guidelines for consumers and 
designers of multimedia learning. San Francisco, CA: Jossey-Bass/Pfeiffer; 2003.
107.
Mustanski B, Lyons T, Garcia SC. Internet use and sexual health of young men who have sex with 
men: a mixed-methods study. Arch. Sex. Behav. Apr 2011;40(2):289-300.
108.
Grov C, Golub SA, Mustanski B, Parsons JT. Sexual compulsivity, state affect, and sexual risk behavior 
in a daily diary study of gay and bisexual men. Psychol. Addict. Behav. Sep 2010;24(3):487-497.
109.
Parsons JT, Rosof E, Mustanski B. Patient-related factors predicting HIV medication adherence among 
men and women with alcohol problems. J Health Psychol. Mar 2007;12(2):357-370.
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 46 of 50
110.
Parsons JT, Rosof E, Mustanski B. The temporal relationship between alcohol consumption and HIV-
medication adherence: a multilevel model of direct and moderating effects. Health Psychol. Sep 
2008;27(5):628-637.
111.
Parsons JT, Rosof E, Mustanski B. Medication adherence mediates the relationship between 
adherence self-efficacy and biological assessments of HIV health among those with alcohol use 
disorders. AIDS Behav. Jan 2008;12(1):95-103.
112.
Mustanski B. The Internet as a setting for sexual health development among LGB youth: William T 
Grant Foundation; 2008-2013.
113.
Mustanski B. Internet-Based HIV/STI Prevention for Young MSM Receiving HIV Testing: National 
Institute of Mental Health; 2007 - 2010.
114.
Parsons J. Compulsive behavior, mental health, and HIV risk.  NIH R01MH087714: National institute of 
Mental Health; 2010-2015.
115.
Mustanski B, Parsons J. International Invited Meeting on Gay Men's Sexual Health: A Focus on 
Couples.  Organized by Drs. Jeffrey Parsons and Brian Mustanski. San Juan, PR2012.
116.
Mustanski B, Parsons J, Sullivan P. Invited Plenary: Male Couples and HIV Prevention. Society for 
Prevention Research. Washington, DC. 2012.
117.
Tregear SJP, Sullivan PLIaPoAS, Mustanski BPoCS, Chiasson MAPoNCS. Evaluation of Rapid HIV 
Self-Testing among MSM in High Prevalence Cities. 2011-N-13417: Center for Disease Control and 
Prevention; 2011-2015.
118.
Newcomb ME, Clerkin EM, Mustanski B. Sensation Seeking Moderates the Effects of Alcohol and Drug 
Use Prior to Sex on Sexual Risk in Young Men Who Have Sex with Men. AIDS Behav. Oct 20 
2011;15:565-575.
119.
Liu R, Mustanski B. Risk factors for suicidal ideation and self-harm in LGBT youth. Am. J. Prev. Med. 
2011;In Press.
120.
Mustanski BS, Garofalo R, Emerson EM. Mental health disorders, psychological distress, and 
suicidality in a diverse sample of lesbian, gay, bisexual, and transgender youths. Am. J. Public Health. 
Dec 2010;100(12):2426-2432.
121.
DuBois SN, Emerson E, Mustanski B. Condom errors, failures, and erection problems among a diverse 
sample of young men who have sex with men. AIDS & Behavior. 2010;in press.
122.
Mustanski B, Hunter J. Parents as agents of HIV prevention for gay, lesbian, and bisexual youth. In: 
Pequegnat W, Bell C, eds. Family and HIV/aids : cultural and contextual issues in prevention and 
treatment. New York: Springer; 2011:Chapter 12.
123.
Newcomb ME, Clerkin EM, Mustanski B. Sensation Seeking Moderates the Effects of Alcohol and Drug 
Use Prior to Sex on Sexual Risk in Young Men Who Have Sex with Men. AIDS Behav. Oct 20 2010;In 
Press.
124.
Mustanski B, Newcomb M. Relationships and risk: Longitudinal analysis of relationship status and 
unprotected sex in young gay and bisexual men. Society for the Scientific Study of Sexuality. Las 
Vegas, NA2010.
125.
Magee JC, Bigelow L, Dehaan S, Mustanski BS. Sexual Health Information Seeking Online: A Mixed-
Methods Study Among Lesbian, Gay, Bisexual, and Transgender Young People. Health Educ. Behav. 
Apr 13 2011.
126.
DeHaan S, Kuper LE, Magee JC, Bigelow L, Mustanski BS. Interplay Between Online and Offline 
Sexual Health Resources among LGBT Youth under review.
127.
Mustanski B, Greene G, Birkett M. Mixed Method Analyses of the Role of the Internet in Sexual Health 
Information Seeking William T Grant Foundation Workshop on Mixed-Methods Research. Washington, 
DC2011.
128.
Heckathorn DD. Respondent-driven sampling: A new approach to the study of hidden populations. Soc. 
Probl. 1997;44:174-200.
129.
Sharma A, Sullivan PS, Khosropour CM. Willingness to Take a Free Home HIV Test and Associated 
Factors among Internet-Using Men Who Have Sex with Men. J Int Assoc Physicians AIDS Care (Chic). 
Apr 28 2011.
130.
Khosropour CM, Sullivan PS. At-home HIV testing of MSM Enrolled in an Online HIV Behavioral Risk 
Study. Paper presented at: National HIV Prevention Confernece2011; Atlanta, GA.
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 47 of 50
131.
Golub SA, Starks TJ, Payton G, Parsons JT. The Critical Role of Intimacy in the Sexual Risk Behaviors 
of Gay and Bisexual Men. AIDS Behav. Jun 1 2011.
132.
Lelutiu-Weinberger C, Pachankis JE, Golub SA, Walker JJ, Bamonte AJ, Parsons JT. Age Cohort 
Differences in the Effects of Gay-Related Stigma, Anxiety and Identification with the Gay Community on 
Sexual Risk and Substance Use. AIDS Behav. Oct 30 2011.
133.
Johnson WD, Holtgrave DR, McClellan WM, Flanders WD, Hill AN, Goodman M. HIV intervention 
research for men who have sex with men: a 7-year update. AIDS Educ. Prev. Dec 2005;17(6):568-589.
134.
Rotheram-Borus MJ, Swendeman D, Flannery D, Rice E, Adamson DM, Ingram B. Common factors in 
effective HIV prevention programs. AIDS Behav. Jun 2009;13(3):399-408.
135.
Marlatt GA, Donovan DM. Relapse prevention : maintenance strategies in the treatment of addictive 
behaviors. 2nd ed. New York: Guilford Press; 2005.
136.
Albarracin D, Gillette JC, Earl AN, Glasman LR, Durantini MR, Ho MH. A Test of Major Assumptions 
About Behavior Change: A Comprehensive Look at the Effects of Passive and Active HIV-Prevention 
Interventions Since the Beginning of the Epidemic. Psychol. Bull. Nov 2005;131(6):856-897.
137.
Mustanski B. The influence of state and trait affect on HIV risk behaviors: A daily diary study of MSM. 
Health Psychol. Sep 2007;26(5):618-626.
138.
Crepaz N, Marks G. Are negative affective states associated with HIV sexual risk behaviors? A meta-
analytic review. Health Psychol. 2001;20:291-299.
139.
Valleroy LA, MacKellar DA, Karon JM. HIV prevalence and associated risks in young men who have 
sex with men: Young Men's Survey Study Group. Journal of the American Medical Association. 
2000;284:198-204.
140.
Tarnowski KJ, Simonian SJ. Accessing treatment acceptability: The abbreviated acceptability rating 
profile. J. Behav. Ther. Exp. Psychiatry. 1992;23:101-106.
141.
Warner L, Newman DR, Kamb ML, et al. Problems with condom use among patients attending sexually 
transmitted disease clinics: prevalence, predictors, and relation to incident gonorrhea and chlamydia. 
Am. J. Epidemiol. Feb 1 2008;167(3):341-349.
142.
Mustanski B, Garofalo R, Monahan C, Gratzer B, Andrews K. Feasibility, acceptability, and preliminary 
efficacy of an online HIV prevention program for diverse young men who have sex with men: the Keep 
It Up! intervention. Under Review. 2011.
143.
Mustanski B, Garofalo R, Garcia S, Lyons T. The role of the Internet in the sexual health of ethnically 
diverse young men who have sex with men. Society of the Scientific Study of Sexuality. San Juan, 
PR2008.
144.
Mustanski B, Newcomb M. Keep It Up!  An NIH funded project to prevent HIV among young MSM. 
Sex::Tech: Harnessing technology to promote healthy teen sexuality. San Francisco2008.
145.
Mustanski B. The Internet as a tool for health research and promotion with youth. Cook County Hospital 
and Rush University Pediatrics Grand Rounds. Chicago, IL 2009.
146.
Mustanski B. Recruitment of Young MSM in Chicago: RDS, Internet, and Clinic-Based Approaches. 
National Institute of Drug Abuse Meeting on Recruiting Drug-Using Men Who have Sex with Men 
(MSM) for HIV Prevention Research. Bethesda, MA2009.
147.
Mustanski B. Promoting the sexual health of minority youth: The role of the Internet. Northwestern 
University Cells-to-Society Colloquim Series. Evanston, IL2009.
148.
Mustanski B, Garcia S, Lyons T. I never hooked up online, except that one time:” A mixed methods 
study of the Internet and sexual health among young MSM. Sex::Tech: Harnessing technology to 
promote healthy teen sexuality. San Francisco2009.
149.
Serrano PA, Mustanski B. Gay and Bisexual adolescents and the Internet: Functions, Processes, and 
Predictors. Illinois African American and Latino Higher Education Alliance Student Research Forum 
Chicago, IL2009.
150.
Mustanski B, Garofalo R, Garcia SC, Gratzer B, Hinds M, Monahan C. Testing negative and staying 
negative: Internet-based HIV prevention program for ethnically-diverse young MSM. . Society for 
Prevention Research. Denver, CO. 2010.
151.
Mustanski B. LGBT youth, technology, and sexual health. University of Virgninia Youth-Nex Center to 
Promote Effective Youth Development Inaugural Conference: Forward Thinking: Preparing Our Youth 
for the Coming World. Charlottesville, VA2011.
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 48 of 50
152.
Mustanski B, Garofalo R, Monahan C. Results of a randomized clinical trial of an Internet-based HIV 
prevention program for young MSM. AIDS Impact. Santa Fe, NM2011.
153.
New York City Department of Health and Mental Hygiene: HIV Epidemiology and Field Services Unit. 
New York City Annual HIV/AIDS Surveillance Statistics. New York, NY2008.
154.
Chicago Department of Public Health. STI/HIV Surveillance Report. Chicago, IL: City of Chicago;2011.
155.
U.S. Census Bureau. American FactFinder. 2010; http://factfinder.census.gov. Accessed 25 April 2010.
156.
Steen R, Wi TE, Kamali A, Ndowa F. Control of sexually transmitted infections and prevention of HIV 
transmission: mending a fractured paradigm. Bull. World Health Organ. Nov 2009;87(11):858-865.
157.
Cohen MS, Hoffman IF, Royce RA, et al. Reduction of concentration of HIV-1 in semen after treatment 
of urethritis: implications for prevention of sexual transmission of HIV-1. AIDSCAP Malawi Research 
Group. Lancet. Jun 28 1997;349(9069):1868-1873.
158.
Korenromp EL, White RG, Orroth KK, et al. Determinants of the impact of sexually transmitted infection 
treatment on prevention of HIV infection: a synthesis of evidence from the Mwanza, Rakai, and Masaka 
intervention trials. J. Infect. Dis. Feb 1 2005;191 Suppl 1:S168-178.
159.
Potts M, Halperin DT, Kirby D, et al. Public health. Reassessing HIV prevention. Science. May 9 
2008;320(5877):749-750.
160.
Hayes R, Watson-Jones D, Celum C, van de Wijgert J, Wasserheit J. Treatment of sexually transmitted 
infections for HIV prevention: end of the road or new beginning? AIDS. Oct 2010;24 Suppl 4:S15-26.
161.
Fleming DT, Wasserheit JN. From epidemiological synergy to public health policy and practice: The 
contribution of other sexually transmitted diseases to sexual transmission of HIV infection. Sex. 
Transm. Infect. 1999;75:3-17.
162.
White R, Celum C, Wasserheit J, Aral S, Hayes R. Control of sexually transmitted infections for HIV 
prevention. Lancet. Oct 11 2008;372(9646):1297; author reply 1297-1298.
163.
Pinkerton SD, Layde PM. Using sexually transmitted disease incidence as a surrogate marker for HIV 
incidence in prevention trials: a modeling study. Sex. Transm. Dis. May 2002;29(5):298-307.
164.
Schachter J. Biologic versus behavioral endpoints--the duet continues. Sex. Transm. Dis. Sep 
2000;27(8):456-457.
165.
Peterman TA, Lin LS, Newman DR, et al. Does measured behavior reflect STD risk? An analysis of 
data from a randomized controlled behavioral intervention study. Project RESPECT Study Group. Sex. 
Transm. Dis. Sep 2000;27(8):446-451.
166.
Fishbein M, Pequegnat W. Evaluating AIDS prevention interventions using behavioral and biological 
outcome measures. Sex. Transm. Dis. Feb 2000;27(2):101-110.
167.
Quinn TC. Association of sexually transmitted diseases and infection with the human immunodeficiency 
virus: biological cofactors and markers of behavioural interventions. Int. J. STD AIDS. 1996;7 Suppl 
2:17-24.
168.
Scheer S, Kellogg T, Klausner JD, et al. HIV is hyperendemic among men who have sex with men in 
San Francisco: 10-year trends in HIV incidence, HIV prevalence, sexually transmitted infections and 
sexual risk behaviour. Sex. Transm. Infect. Nov 2008;84(6):493-498.
169.
Donnell D, Hughes JP, Fleming TR. Challenges in the design of HIV prevention trials in the United 
States. J. Acquir. Immune Defic. Syndr. Dec 2010;55 Suppl 2:S136-140.
170.
Huhn GD, McIntyre AF, Broad JM, et al. Factors associated with newly diagnosed HIV among persons 
with concomitant sexually transmitted diseases. Sex. Transm. Dis. Aug 2008;35(8):731-737.
171.
Chai SJ, Aumakhan B, Barnes M, et al. Internet-based screening for sexually transmitted infections to 
reach nonclinic populations in the community: risk factors for infection in men. Sex. Transm. Dis. Dec 
2010;37(12):756-763.
172.
Friedman LM, Furberg C, DeMets DL. Fundamentals of clinical trials. 3rd ed. New York: Springer; 
1998.
173.
Janssen RS, Holtgrave DR, Valdiserri RO, Shepherd M, Gayle HD, De Cock KM. The Serostatus 
Approach to Fighting the HIV Epidemic: prevention strategies for infected individuals. Am. J. Public 
Health. Jul 2001;91(7):1019-1024.
174.
Cockburn A. Agile software development. Boston: Addison-Wesley; 2002.
175.
Highsmith JA. Agile project management: creating innovative products. Boston: Addison-Wesley; 2004.
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 49 of 50
176.
Schwaber K, Beedle M. Agile software development with Scrum. Upper Saddle River, NJ: Prentice Hall; 
2002.
177.
Ross MW, Williams ML. Effective targeted and community HIV/STD prevention programs. J Sex Res. 
Feb 2002;39(1):58-62.
178.
Kelly JA, Murphy DA, Sikkema KJ, et al. Randomised, controlled, community-level HIV-prevention 
intervention for sexual-risk behaviour among homosexual men in US cities. Community HIV Prevention 
Research Collaborative. Lancet. Nov 22 1997;350(9090):1500-1505.
179.
Rotheram-Borus MJ, Duan N. Next generation of preventive interventions. J. Am. Acad. Child Adolesc. 
Psychiatry. May 2003;42(5):518-526.
180.
Rosser BS, Oakes JM, Konstan J, et al. Reducing HIV risk behavior of men who have sex with men 
through persuasive computing: results of the Men's INTernet Study-II. AIDS. Jul 1 2010;In Press.
181.
CDC. Sexually transmitted diseases treatment guidelines, 2006. MMWR Recomm. Rep. Aug 4 
2006;55(RR-11):1-94.
182.
El-Bassel N, Jemmott JB, Landis JR, et al. National Institute of Mental Health Multisite Eban HIV/STD 
Prevention Intervention for African American HIV Serodiscordant Couples: A Cluster Randomized Trial. 
Arch. Intern. Med. Jul 12 2010.
183.
Mansergh G, Koblin BA, McKirnan DJ, et al. An Intervention to Reduce HIV Risk Behavior of 
Substance-Using Men Who Have Sex with Men: A Two-Group Randomized Trial with a 
Nonrandomized Third Group. PLoS Med. 08/24 2010;7(8):e1000329.
184.
Dilley JW, Woods WJ, Sabatino J, et al. Changing sexual behavior among gay male repeat testers for 
HIV: a randomized, controlled trial of a single-session intervention. J. Acquir. Immune Defic. Syndr. Jun 
1 2002;30(2):177-186.
185.
Dilley JW, Woods WJ, Loeb L, et al. Brief cognitive counseling with HIV testing to reduce sexual risk 
among men who have sex with men: results from a randomized controlled trial using paraprofessional 
counselors. J. Acquir. Immune Defic. Syndr. Apr 15 2007;44(5):569-577.
186.
DiClemente RJ, Wingood GM, Harrington KF, et al. Efficacy of an HIV prevention intervention for 
African American adolescent girls: a randomized controlled trial. JAMA. Jul 14 2004;292(2):171-179.
187.
Wilton L, Herbst JH, Coury-Doniger P, et al. Efficacy of an HIV/STI prevention intervention for black 
men who have sex with men: findings from the Many Men, Many Voices (3MV) project. AIDS Behav. 
Jun 2009;13(3):532-544.
188.
Sullivan PS, Khosropour CM, Luisi N. The Barriers to Online Prevention Research (BOPR) Study: 
Methods to improve the enrollment of racial/ethnic minority MSM in online HIV prevention research. 
Sex::Tech. San Francisco, CA.2010.
189.
Rosser BR, Oakes JM, Konstan J, et al. Reducing HIV Risk Behavior of MSM through Persuasive 
Computing: Results of the Men’s INTernet Study (MINTS-II). under review. 2010.
190.
Kernan WN, Viscoli CM, Makuch RW, Brass LM, Horwitz RI. Stratified randomization for clinical trials. 
J. Clin. Epidemiol. Jan 1999;52(1):19-26.
191.
Donenberg GR, Emerson E, Bryant FB, Wilson H, Weber-Shifrin E. Understanding AIDS-risk behavior 
among adolescents in psychiatric care: Links to psychopathology and peer relationships. J. Am. Acad. 
Child Adolesc. Psychiatry. Jun 2001;40(6):642-653.
192.
Mustanski B, Emerson E, Garofalo R, Donenberg G. AIDS Risk Behavior Assessment (ARBA)- LGBT 
youth version. Chicago: Institute for Juvenile Research; 2007.
193.
Carey MP, Schroder KE. Development and psychometric evaluation of the brief HIV Knowledge 
Questionnaire. AIDS Educ. Prev. Apr 2002;14(2):172-182.
194.
Jaworski BC, Carey MP. Development and psychometric evaluation of a self-administered 
questionnaire to measure knowledge of Sexually Transmitted Diseases. AIDS Behav. 2007;11:557-
574.
195.
Kalichman SC, Picciano JF, Roffman RA. Motivation to reduce HIV risk behaviors in the context of the 
Information, Motivation and Behavioral Skills (IMB) model of HIV prevention. J Health Psychol. Jul 
2008;13(5):680-689.
196.
Crosby RA, Sanders SA, Yarber WL, Graham CA, Dodge B. Condom use errors and problems among 
college men. Sex. Transm. Dis. Sep 2002;29(9):552-557.
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.

Study Title: Efficacy of Internet-based HIV Prevention (PI: Mustanski);
Research Protocol STU00073433, Version Date: 12/12/16; Page 50 of 50
197.
Yarber WL, Graham CA, Sanders SA, Crosby RA. Correlates of condom breakage and slippage among 
university undergraduates. Int. J. STD AIDS. Jul 2004;15(7):467-472.
198.
Sanders SA, Graham CA, Yarber WL, Crosby RA. Condom use errors and problems among young 
women who put condoms on their male partners. J Am Med Womens Assoc. Spring 2003;58(2):95-98.
199.
DuBois SN, Mustanski B, Garofalo R. Condom errors and failures among young men who have sex 
with men: an analysis of frequency, differences by race, and sources of hiv-related information. Society 
for the Scientific Study of Sexuality. Puerto Vallarta, Mexico2009.
200.
DuBois SN, Mustanski B, Emerson E. Condom errors, failures, and erection problems among a diverse 
sample of young men who have sex with men. under review. 2010.
201.
Catania JA. The health protective communication scale. In: Davis CM, Yarber WL, Bauserman R, 
Schreer G, Davis SL, eds. Handbook of Sexuality-Related Measures. Thousand Oaks, CA: Sage 
Publications; 1998:544-546.
202.
Dunn LM, Dunn LM. Manual for the Peabody picture vocabulary test-revised. Circle Pinces, MN: 
American Guidance Services; 1981.
203.
Identigene. Identigene STD Test Collection Kit. 2011; http://www.identigene.com/std-testing/.
204.
Gene-Probe. APTIMA COMBO 2® Assay. 2011; http://www.gen-probe.com/products-
services/aptima.aspx.
205.
Raudenbush SW, Bryk AS. Hierarchical linear models : applications and data analysis methods. 2nd 
ed. Thousand Oaks: Sage Publications; 2002.
206.
Zeger SL, Liang KY, Albert PS. Models for longitudinal data: a generalized estimating equation 
approach. Biometrics. Dec 1988;44(4):1049-1060.
207.
Gibbons RD, Hedeker D, Waternaux C, Davis JM. Random regression models: a comprehensive 
approach to the analysis of longitudinal psychiatric data. Psychopharmacol. Bull. 1988;24(3):438-443.
208.
Rubin DB. Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. 
Psychol. 1974;66(5):688-701.
209.
Rubin DB. Bayesian inference for causal effects: The role of randomization. The Annals of Statistics. 
1978;6(1):34-58.
210.
Diggle P, Liang K-Y, Zeger SL. Analysis of longitudinal data. OxfordNew York: Clarendon Press ;
Oxford University Press; 1994.
211.
Mustanski B. Ethical and regulatory issues with conducting sexuality research with LGBT adolescents: 
A call to action for a scientifically informed approach. Arch. Sex. Behav. Aug 2011;40(4):673-686.
212.
CDC. HIV/AIDS and young men who have sex with men. Atlanta, GA.: US Department of Health and 
Human Services, Center for Disease Control and Prevention. ; 2009.
213.
CDC. HIV and AIDS among Gay and Bisexual Men. Atlanta: US Department of Health and Human 
Services; 2010.
214.
MacKellar DA, Valleroy LA, Secura GM, et al. Unrecognized HIV infection, risk behaviors, and 
perceptions of risk among young men who have sex with men: opportunities for advancing HIV 
prevention in the third decade of HIV/AIDS. J. Acquir. Immune Defic. Syndr. Apr 15 2005;38(5):603-
614.
215. 
O’Hara BM, Kelley CF, Rosenberg ES, Sanchez TH, Frew PM, Peterson JL, Norton JC, del Rio C, 
Sullivan PS. Chlamydia, Gonorrhea and Syphilis Incidence among Black and White HIV-Negative Men 
who Have Sex with Men (MSM) in Atlanta, GA. Poster presented at: XIX International AIDS Conference 
(AIDS 2012); July 2012; Washington DC.
IRB #: STU00073433 Approved by NU IRB for use on or after 8/22/2017 through 8/21/2018.