mLab App for Improving Uptake of Rapid HIV Self-testing and Linking Youth to Care
Phase: NA
Conditions: HIV Infections
Sponsor: Rebecca Schnall, RN, MPH, PhD

03-20-24 
1 
 
 
Stand Alone Protocol: mLab App – IRB-AAAR8760 
 
mLab App for Improving Uptake of Rapid HIV Self-testing and Linking Youth to Care 
NCT03803683 
 
 
PI: Dr. Rebecca Schnall 

03-20-24 
2 
 
 
 
 
 
 
 
Abstract: 
The number of youth living with HIV continues to rise, and they are disproportionately represented at each 
stage of the care continuum. Most relevant to this application, it is estimated that less than half of HIV-infected 
youth in the United States (US) have been diagnosed with HIV, and AIDS-related deaths among youth have 
increased over the past decade despite decreased death rates among all other age groups, pointing to the urgent 
need for increased testing among youth. Black and Latino youth are at increased risk of poor HIV-related 
outcomes and have disparate testing rates as compared to White youth. Mobile Health (mHealth) technology is 
a powerful and relevant tool which represents a promising approach for improving outcomes among youth 
living with HIV. Youth are avid adopters and heavy users of smartphones and digital technologies, and these 
technologies offer opportunities to tailor interventions to developmental stages and personal needs. Importantly, 
these technologies are capable of delivering interventions in real-time and in ecologic settings. This creates an 
opportunity to remotely reach youth through mobile and connected health approaches to strengthen their HIV 
care continuum engagement and treatment outcomes. In response, our study team has developed the mLab App, 
an innovative mobile and connected technology that combines HIV prevention information with push 
notifications/reminders to complete HIV testing and an automated image processing feature to provide 
accessible, objective, secure, and real-time feedback on home-based OraQuick (lateral flow assay) HIV test 
results. The mLab App also contains an innovative automated data collection and a results reporting feature. 
Findings from our preliminary work in New York City indicate that youth perceive the mLab App as useful, 
easy to use, and effective at improving health outcomes and intend to use the technology. Findings from our 
preliminary work in Africa support the sensitivity and specificity of the imaging algorithm for interpreting 
lateral flow assay results. Theoretically-guided by the Health Information Technology Usability Evaluation 
Model (Health-ITUEM), the proposed project will refine and test a next-generation diagnostic intervention 
delivered on a mobile platform to improve HIV testing and linkage-to-care outcomes among youth living with 
and at-risk for HIV. Well in line with the objectives of RFA-MH-18-606, we will conduct a careful, iterative 
process of technology refinement based on input from end users, experts, and our youth advisory board. We will 
then enroll 525 high-risk youth (age 18-29 years) in a 12-month RCT to assess differences in HIV testing rates 
and linkage to care between arm1-intervention (mLab App), arm 2- Standard of care-HIV information control 
arm, and arm 3-HIV home test only. Finally, we will analyze paradata, defined as auxiliary data that capture 
details about the process of interaction with the technology, to understand the effect of user engagement of the 
mLab App on improving HIV testing rates and linkage to care. Interventions delivered through mHealth 
technology represent a promising approach for improving outcomes among youth. Given the pervasiveness, low 
cost, and convenience of mobile technology, the mLab App holds promise to help achieve the goals of the 
National HIV/AIDS Strategy in the US by increasing the number of persons living with HIV who know their 
serostatus, decreasing HIV-related disparities, and ultimately reducing the risk of HIV transmission and 
acquisition. 

03-20-24 
3 
 
 
Study Aims: 
The number of youth living with HIV in the United States (US) continues to rise, and the epidemic is 
exacerbated in racial, ethnic, and sexual minority youth who bear a disproportionate burden of the HIV 
epidemic. Estimates of ~ 80% of new infections occur in young men who have sex with men (YMSM) and 
young transgender women (YTGW), and the epidemic is further magnified in Black and Latino youth.1-11 There 
are large disparities in HIV testing rates in youth and more specifically in our target study population, Latino 
and Black YMSM and YTGW. The low uptake of HIV testing in youth can be explained by a number of 
factors. Developmentally, youth often perceive themselves to be at low-risk for acquiring HIV as well as having 
an inaccurate perception of its impact.12 There are also social and healthcare system factors that make YMSM 
and YTGW more vulnerable to becoming infected with HIV and less likely to be tested for HIV. Social factors 
include stigma, homophobia, and racism. These factors may cause YMSM and YTGW to feel rejected and 
isolated,9, 12-15 and as a result they may not disclose their sexual orientation or gender identities16 or seek HIV 
prevention and testing services.12 Healthcare system factors include limited and inadequate access to youth- 
centered HIV testing services.9, 17 As a result, many YMSM and YTGW do not seek HIV testing services and 
are therefore less likely to be aware that they are infected.9, 11, 16, 18 HIV+ youth who do not know they are 
infected are not engaged in lifesaving treatment and care and are at risk of infecting others,9 pointing to the 
urgent need for interventions to increase the uptake of HIV testing in Black and Latino YMSM and YTGW. 
Mobile health (mHealth) technology is a powerful platform for the delivery of HIV prevention interventions, 
including HIV testing,19, 20 and is especially relevant for racial and ethnic minority youth.21 Approaches using 
mHealth have the advantage of a simple interface for users, accessibility anywhere cell signals/Wi-Fi are 
available, relative affordability, and have been successfully promoted to reach stigmatized and disenfranchised 
populations.22, 23 While preliminary evidence suggests that mHealth technology (e.g., smartphone apps) is 
feasible, attractive, and effective for promoting HIV prevention and care outcomes among youth, many apps 
have not been designed by end-users and, of those that do exist in the marketplace, none have not been well- 
evaluated with YMSM or YTGW.24 Furthermore, the need is even greater in Black and Latino YMSM and 
YTGW, due to socioeconomic factors, cultural norms, stigma, homophobia, and discrimination.25-34 Building on 
our extensive preliminary work,35, 36 our multi-disciplinary team (public health scientists, clinicians, and 
engineers) has developed the mLab App, an innovative mobile and connected technology that combines HIV 
prevention information with push notifications for testing, a mobile phone imaging feature of a lateral flow 
assay (the OraQuick rapid home HIV self-test), and automated image processing to provide accessible, 
objective, secure, and real-time feedback on HIV test results. The mLab App also contains an automated data 
collection and results reporting feature, which relays test results back to the research team, and the study 
participant triggering messages to encourage future repeat testing for a non-reactive test or linkage to 
confirmatory testing and treatment for a reactive test. Our preliminary work in Africa supports the specificity 
and sensitivity of the image processing algorithm compared to visual interpretation of lateral flow tests37 and 
has been successfully used with lateral flow assays for malaria testing in non-clinical settings. Preliminary work 
in New York City (NYC) indicates strong acceptability and feasibility of the mLab App in YMSM and 
YTGW.38 Given our robust preliminary data, the innovativeness of the mLab App, and the public health need 
for the development of efficacious interventions to increase the uptake of HIV testing and linkage to care 
among high-risk youth, our study aims are to: 
Aim 1: Refine the user interface of the mLab App – a mobile and connected health intervention for increasing 
the uptake of rapid home HIV self-testing (OraQuick) and linkage to care – by employing rapid cycle iterative 
design methods with feedback from human computer interaction experts (N=5), end-users (N=20), and youth 
advisory board members (N=10). 
Aim 2: Evaluate the efficacy of the mLab App in improving both HIV testing rates and linkage to care versus 
standard preventive care in 525 Black, Latino and other young men and transgender women (inclusive of 
YMSM and YTGW age 18-29 years) in a 12-month multi-site trial, with randomization to three groups 
(intervention, standard of care-HIV information control arm, HIV home tests) at each site. 
Hypothesis 1. Youth randomized to the mLab App arm (arm 1), as compared to youth in the standard of care 
HIV information control arm, will have: a) increased rates of HIV testing within the first 6 months of the trial; 
and b) increased rates of re-testing in months 7-12 of the trial. 

03-20-24 
4 
 
 
Hypothesis 2. The total number of youth who test HIV positive and link to care in the mLab App arm will be 
greater as compared to the standard of care HIV information control arm. 
Hypothesis 3. The image processing in the mLab App will: 1) have no statistically significant difference in 
sensitivity or specificity when compared to confirmatory testing, 2) have greater sensitivity and specificity when 
compared to the user-submitted self-analyses. 
Aim 3: Monitor and analyze paradata to understand the effect of user engagement of the mLab App on 
improving HIV testing rates and linkage to care. 
Impact: The proposed study addresses significant scientific questions and public health concerns through a 
rigorous and systematic effort to refine and test the mLab App intervention and better understand its most useful 
components to promote HIV testing and linkage to care among Black, Latino and other YMSM and YTGW – 
those most at risk for acquiring HIV but also the least likely to have been tested. Given the novelty and promise 
of next generation mHealth diagnostics and the high rates of undiagnosed HIV in youth (specifically YMSM 
and YTGW), the mLab App has the potential to significantly curb the HIV epidemic by improving detection 
and enabling youth to promptly seek confirmatory testing and follow-up care.39 Building on our earlier user- 
centered design work,24, 40 and integrating a real-time diagnostic device, this work has the potential to improve 
outcomes across the HIV care continuum for high-risk youth. 
 
Background. 
The number of youth living with HIV continues to rise, and they are disproportionately represented at each 
stage of the care continuum. Most relevant to this application, it is estimated that less than half of HIV-infected 
youth in the US have been diagnosed with HIV, and AIDS-related deaths among youth have increased over the 
past decade despite decreased death rates among all other age groups. Simply stated – youth unaware they are 
HIV+ cannot get the treatment they need to stay healthy and may infect others without knowing it. Thus, 
increasing access to HIV testing is a critical component to engaging and identifying YMSM and YTGW with 
undiagnosed HIV, linking them to care, and lowering forward HIV transmission.20, 6, 8, 41-46 
YMSM and YTGW, and specifically Blacks and Latinos, are disproportionately infected with HIV. To 
illustrate, in 2015, youth comprised 22% of all new cases of HIV.9 Of these youth, 81% of infections occurred 
among YMSM.9 Among YTGW under the age of 29, limited data exists, but community-based samples suggest 
an HIV prevalence from 5% to 20% in this population.47 These numbers are exacerbated in racial/ethnic 
minorities. Black men who have sex with men (MSM) have more HIV diagnoses than any other racial/ethnic group 
of MSM (38%) and Black YMSM comprise 39% of these HIV diagnoses.7 Latino MSM comprise 27% of HIV 
diagnoses among MSM7 and 7 out of 10 new HIV diagnoses among Latinos.48 YTGW have also been 
disproportionately affected by HIV10 with the highest percentage of HIV+ test results of any gender category.11 
There are a number of behavioral and social factors that likely account for the high rates of new and 
undiagnosed HIV infections among youth, and specifically YMSM and YTGW. Engaging in receptive anal 
intercourse49 and a higher likelihood of having partners who may be are at increased risk for HIV are some of 
the behavioral factors that potentiate the HIV epidemic in youth.50-53 Moreover, having never witnessed the 
devastating effect of HIV/AIDS in the early years of the epidemic, youth may perceive themselves to be at 
lower risk of HIV.12, 54 Social factors including stigma, homophobia, and racism may compound those factors; 
many YMSM and YTGW feel rejected, isolated, and/or lack social support.9, 12-15 Healthcare system factors also 
contribute to the low HIV testing rates in youth. Many youth avoid contact with providers who offer HIV 
testing and care due to lack of health insurance,55 discomfort with facilities and services,56 fear of 
stigmatization,57 and concerns about confidentiality.58, 59 For these reasons, many YMSM and YTGW avoid 
HIV testing services,10, 12 making them unaware that they may be infected with HIV.9, 16, 18 
Outreach is needed among YMSM and YTGW to engage them in HIV testing, which remains an important tool 
in the fight against HIV.60, 61 There are large disparities in HIV testing rates in youth and ethnic and racial 
minorities. Among those HIV-infected, only 49% of YMSM aged 18-24 years compared to 66% of adults62 
knew of their infection, highlighting the need for improved outreach for testing among high-risk youth.19,27 
Among all MSM, 54% of Black/African American men knew of their infection, compared with 63% of 
Hispanic/Latino men and 86% of white men.5, 54, 63 – reflecting huge racial and ethnic disparities and the need 
for our proposed study that targets enrollment of toward Black and Latino YMSM and YTGW (NIMHD 
priority area).45, 64, 65 Transgender women (TGW) of all ages are also immensely burdened by HIV. In one 

03-20-24 
5 
 
 
Figure 2. Home Screen of mLab App 
study, TGW were shown to have a lower prevalence of ever having been tested (35.6%) or having been tested 
in the past year (10.0%) for HIV compared to cisgender gay and bisexual men (61.8% ever tested; 21.6% tested 
in past year).11 TGW have consistently low HIV testing rates66 and a resultant high percentage of undiagnosed 
infection in comparison to the general population,67 pointing to the need for interventions to increase the uptake 
of HIV testing in this population.68 YMSM and YTGW do not have adequate access to HIV prevention and 
testing 9, 17 and they have poorer access to healthcare, in general12. For example, Koblin (DSMB member) and 
colleagues found that Black MSM in particular were less likely to have a usual place for healthcare or to have 
visited a healthcare provider recently69, thus providing scientific premise for the need to provide easier access to 
HIV testing services for this most at-risk group. The healthcare system is failing to test youth, contributing to 
the high percentage of youth with undiagnosed HIV; this is especially true for YMSM and YTGW who are 
often overlooked by the current healthcare system.70, 71 As a result, more Black and Latino men end up being 
tested in non-clinical settings than White men,72 pointing to the need for expanding non-clinical options, such as 
self-testing, especially among racial and ethnic minorities.71, 73 
Scientific Premise to Support the Use of HIV Self-Testing. Concerns about the HIV self-test have focused on its 
potential to reduce contact with care providers or in healthcare settings where more sensitive tests may be 
warranted and other prevention approaches, such as pre-exposure and post-exposure prophylaxis, might be 
delivered.74 Another concern frequently cited about the HIV self-test is that individuals who receive a reactive 
or preliminary positive test result may be less likely to seek or to receive a confirmatory test and be linked to 
appropriate care. Findings from a recent RCT suggests that when supported through a helpline, individuals 
identified as HIV+ through self-testing were adequately and appropriately linked to care.75 Perhaps most 
importantly, there were no serious adverse events described in this study or in other HIV self-testing studies.76 
While HIV self-testing kits can be purchased over the counter, we acknowledge that there has been low uptake 
of HIV self-testing among YMSM in the US,77 pointing to the need for technologies, such as the one proposed 
in this study, to promote the uptake of the HIV self-test (OraQuick). To our knowledge, no prior studies have 
looked at uptake among YTGW. Barriers are not only limited to cost; our research indicates that there is an 
overall lack of understanding of how to use the test correctly.78 To overcome these challenges, we will use 
careful iterative feedback from youth and experts to refine the mLab App, a mobile and connected technology 
to improve HIV testing, diagnosis, and linkage to care. Mobile Health (mHealth) technology is a powerful 
platform for the delivery of HIV prevention strategies, including HIV testing.19, 20 There is already extremely 
high mobile phone penetration in the US,79 especially among racial and ethnic minority groups21 and youth.21 
This creates the opportunity for health interventions in a portable format with enhanced privacy. mHealth is a 
powerful and relevant tool which can be developed to meet the needs of its end-users and has great potential to 
transform how healthcare is provided and consumed.80, 81 
In response to the need for interventions to increase HIV testing in 
youth and in direct response to 
RFA-MH-18-606, our study team 
developed the mLab App, which 
affords advantages over existing 
self-test options to support the 
potential for higher uptake of the 
HIV self-test. The mLab App is a 
mobile app on your phone that is 
accessible using a login name and 
password. The app will provide 
HIV prevention information, push 
notification reminders for testing, 
step-by-step instructions for using 
the OraQuick HIV tests, an image 
upload function so you can send an 
image of your OraQuick HIV test to 
Figure 1. Diagnostic Strategy workflow and image- 
processing algorithm 
 
 
the study team. You will enter the results of the OraQuick test in the app. By 

03-20-24 
6 
 
 
using the app, we are asking you to help test the app's ability to interpret the 

03-20-24 
7 
 
 
results of the OraQuick tests. You will not be able see the results of this interpretation. If you test HIV positive 
by the OraQuick HIV test, the mLab app will provide information on how to set up follow-up confirmatory 
testing within the following 24 hours. All information that you provide within the app will be stored on a 
secured server. 
 
The mLab App is derived from extensive participatory based research (focus groups, design sessions, 
usability testing) with young men (CDC U01 PS003715) (described in detail in APPROACH: preliminary study 
#1).35, 82 Building on this extensive user-centered design work and the engineering work of Dr. Scherr (Co-I; see 
APPROACH: preliminary study #3), who developed the automated image processing algorithm (Figure 1) to 
provide real-time interpretation of smartphone camera images of a lateral flow assays for malaria.37 The mLab 
App extends the algorithm to interpret the HIV self-test (OraQuick). 
The workflow of the app is provided as a powerpoint attachment entitled “20180912_IDE 
_R8760_Schnall_mLab Workflow_Updated Octob.” 
The mLab App addresses many of the current barriers to self-testing kits through the integration with a 
smartphone to overcome ambiguous test interpretations, provide immediate results reporting, and help support 
linkage to care. In addition, the mLab user interface promotes a holistic diagnostic experience because it 
provides step by step error-checking with clear picture directions. While in principle, rapid tests, such as the 
OraQuick, seem simple to interpret83 with weak positive bands or weak control lines, it is all-too-often difficult 
for users to accurately interpret the test results, which we found in our own research on the OraQuick home 
test.84 Beyond diagnosis, the mLab App provides information facilitating linkage to care for those who test 
HIV-positive and educates users on the importance of follow-up testing and prevention services for those who 
test HIV-negative. 
Paradata, defined as auxiliary data that capture details about the process of interaction with the technology, will 
drive our understanding of user engagement with the mLab App and of subsequent behavior change. Outcomes 
from HIV prevention and care studies delivered via a mHealth platform have primarily relied on analyses by 
group assignment and assume that participants’ engagement is equitable within the intervention arm and stable 
for the duration of the trial. The use of paradata in technology interventions involving HIV prevention and care 
is an understudied area of interest.85 While a number of technology-based HIV interventions have previously 
suggested that online behavioral interventions are acceptable and efficacious in promoting behavioral change,86 
a recent review of this literature identified very limited reports of participant engagement, measured through 
paradata, with a technology-based intervention.87 In our own recently published work on the effect of a mHealth 
app for improving symptom burden in PLWH, we monitored app usage throughout the trial and reported this as 
part of the final outcomes of our study.88 Given our prior work, we will examine paradata since this provides an 
opportunity to monitor and analyze aspects of the mLab App that may play a role in changes in HIV testing 
rates and linkage to care. By monitoring and analyzing paradata, components of the mLab App, such as the 
timing, location, and frequency of the rapid images of HIV tests taken by a user and access to other prevention 
information, will be instrumental in understanding what components of the intervention mediate behavior 
change. Engagement with the mLab App will furnish a greater understanding of user satisfaction and retention 
of our intervention. mLab App paradata will have implications for the scale-up and dissemination of this 
intervention as well as for the development of future technology-based HIV interventions with targeted 
behavioral outcomes. 
Theoretical Framework. This study is guided by the Health Information Technology Usability Evaluation 
Model (Health-ITUEM), a technology acceptance theoretical framework that guides usability evaluations for 
assessing mHealth technologies.89 This technology acceptance model was developed in response to the current 
gaps in the extant usability literature90 and was validated by Schnall et al. for use with mHealth technology and 
has been widely used (cited 89 times since 2013). The Health-ITUEM integrates multiple concepts of usability 
from the Technology Acceptance Model91 and the International Organization for Standardization standard 
9241-11.92 Guided by this model, we will carefully and iteratively refine the mLab App to ensure that the 
content and approach are engaging and usable since the two underlying tenets of technology acceptance are 
usefulness and ease of use.93 
Our proposed study, based on our preliminary work, uses a careful iterative process guided by the Health- 
ITUEM to refine and test a mHealth intervention for improving outcomes for youth across the HIV care 

03-20-24 
8 
 
 
continuum. The project will develop and test the mLab App, an innovative mobile and connected technology 
that combines HIV prevention information, with push notification reminders for testing, mobile phone imaging 
of a lateral flow diagnostic assay (OraQuick) and automated image processing to provide accessible, objective, 
secure, and real-time feedback on test results. The mLab App also contains an automated data collection and 
results reporting feature. The reporting feature relays test results back to the research team, triggering messages 
to encourage future repeat testing for a non-reactive test or for a reactive test, linkage to confirmatory testing 
and treatment. The proposed study is highly significant as summarized below: 
• Mobile and connected point-of-care diagnostics are promising tools to increase the uptake of HIV self- 
testing with the goal to improve diagnosis of previously undiagnosed individuals and link youth to care. 
• This project focuses on increasing HIV testing in one the highest risk populations–Black and Latino YMSM 
and YTGW (18-29 years). 
• The study is designed to answer the most important question that will drive this technology forward: Will 
the mLab App improve the uptake of the HIV self-test in previously undiagnosed youth? 
Preliminary Data:We have conducted four preliminary studies to support the use of this intervention for 
youth. 
1) Dr. Schnall led a CDC cooperative agreement (U01 PS004975) using critical iterative end-user feedback to 
design a mobile app for promoting HIV prevention behaviors in high-risk MSM.35 The methodological details, 
associated findings, and final Design Document have been widely disseminated.94 Findings from this study 
guided the content of the mLab App, which is being refined in this proposed study. 
2) Dr. Schnall conducted a study using in-depth interviews, observations, and a think-aloud protocol to 
understand high-risk young adults' use of the rapid (HIV) self-test. Our study incorporated a performance 
record to carefully identify competency in self-administration of the test.84 This study provided evidence of the 
perceived usefulness of the self-test by young adults, especially in light of their concerns about lack of privacy 
in medical settings. Notably, only one (of 21) participants followed all of the instructions for using the test. The 
policy implications of this finding are important since the FDA requirements for labelling and packaging are 
critical for the safe use of devices, but at the same time, end-users’ abilities to understand and use these package 
inserts, especially in stressful situations, must be better considered.84 To address this need, we have developed 
the mLab App which provides step by step instructions on the smartphone screen and also an imaging algorithm 
for interpretation of test results so that the participants can be less burdened by the interpretation of fuzzy red 
lines, a common and well-known limitation to the self-test. 
3) Dr. Scherr’s team published its work on smartphone integration of a number of existing technologies as an 
attractive tool for standardized detection and reporting of infectious diseases. Dr. Scherr demonstrated that 
using an unmodified mobile phone to photograph rapid detection lateral flow assays is superior to visual 
interpretation by inexperienced users. In short, the photo imaging algorithm has been successfully used with 
lateral flow assay tests for malaria in with untrained users in non-clinical settings.37 Dr. Scherr’s work 
demonstrated that an automated image processing algorithm has an improved limit of detection over a 
commercially available lateral flow reader and reduced reporting errors inherent in visual test interpretation. 
4) To understand high-risk YMSM and YTGW’s plans for using the mLab App, barriers to use, and feasibility 
of using the imaging algorithm (the mLab), Drs. Schnall and Scherr conducted a mixed methods observational 
study among 18 YMSM and YTGW (mean age 24) who have all engaged in high-risk sexual behavior 
(unprotected anal sex) in the past 3 months. Participants used the mLab App and then completed a follow-up 
survey and an in-depth interview. Participants completed the Health-ITUES survey and rated the mLab App as 
Impact on health (4.3 out of 5), Useful (4.4 out of 5) and Easy to Use (4.4 out of 5). We collected paradata as 
part of our pilot study and collected time stamps, pages accessed, test image, type of Internet browser, operating 
system, and the smartphone device. This paradata mirrors the data that will be captured in Aim 3 and analyzed 
alongside our primary outcome data. Following the survey, we conducted in-depth interviews to understand 
high-risk youth’s plans for using the mLab App and barriers to use. All of the participants reported this would 
be a very useful tool for high-risk youth and thought most youth would want to use the mLab App. They also all 
indicated they would seek follow-up care if they themselves tested positive and saw the principal advantages of 
the mLab App as being “convenient and portable” and an enabler for promoting uptake of HIV testing. In 
particular, youth noted that the mLab would be especially helpful to youth who did not have a close relationship 

03-20-24 
9 
 
 
with a provider or were concerned of potential stigma from a provider regarding their sexual behavior. One 
participant described “the anxiety he feels during the week that he’s waiting for his results and that the mLab 
App would offer him ‘peace of mind’ as the results are delivered faster.” He specifically mentioned the mLab 
App could have a significant impact among the Latino MSM, as getting tested at the doctor’s office is 
associated with issues such as “stigma, fear, discrimination, and homophobia.” Another respondent described 
the usefulness of the app and stated “mLab App would be great for young MSM because of their smartphone 
use.” He also thought the app would be of great use for individuals without a clinic where they can regularly get 
tested or for those who simply do not like to get tested. A participant who is a public health student and very 
knowledgeable about HIV testing and prevention explained that he “liked the mLab App and that there was no 
other tech like it”. Another participant reported that he liked “that you can enter your zip code in the ‘STI’ 
resources page to find the nearest clinic to get tested and that this is especially important if users would like to 
get a follow-up test.” Participants also provided useful feedback on the user interface with suggestions such as 
that words (i.e., “positive”) appear inside the box with the visualization of the result (i.e. “+”) and avoiding 
red/green to reduce stigmatizing implications. In summary, given our preliminary data supporting the feasibility 
and acceptability of the mLab App, we believe the proposed research formally testing the efficacy of the mLab 
App is the next logical step in research designed to improve HIV testing and linkage to care outcomes in 
YMSM and YTGW. 
 
 
Study Procedures 
Overall approach. We will first use the design specifications from our preliminary work to refine the user 
interface of a functional app which incorporates automated image processing of HIV test results (mLab App) 
and assess its usability. We will then enroll 525 Black and Latino youth (inclusive of YMSM and YTGW age 
18-29 years) in a 12 month RCT to assess differences in HIV testing rates and linkage to care between the 
intervention (mLab App)-arm 1;standard of care HIV information control-arm 2; HIV home tests-arm 3. We 
will analyze paradata to understand the effect of user engagement of the mLab App on improving HIV testing 
rates and linkage to care. 
Scientific Premise. YMSM and YTGW currently face barriers to HIV testing in clinics (e.g., socioeconomic 
 
 
Overview of Study Aims, Design and Methods. Table 1 presents an overview of the design, participants, and 
data analysis plan for 
each aim. 
Scientific Rigor. We 
will use a careful 
iterative process to 
refine the usability and 
test the efficacy of the 
mLab App for 
increasing the uptake of 
rapid home HIV self- 
testing. To increase 
scientific rigor we will: 
1) have an image of the 
participants’ OraQuick 
test result ensuring the 
validity of self-reported 
testing collected via 
surveys; 2) use validated measures; 3) use a randomized design to reduce bias in study arm assignment; 4) 
exercise extensive retention efforts to achieve high retention and reduce selection bias; 5) publish our design 
and analysis plan on clinicaltrials.gov before we commence data collection; 6) publish anonymized public use 
data sets after primary analyses are reported. Consideration of sex as a biological variable. We do not aim to 
Table 1. Overview of Design, Methods, Participants, and Data Analysis 
Aim Design/Methods Participants 
Data Analysis 
1 
Usability 
Evaluation 
Heuristic evaluation: 
human-computer 
interaction experts (N=5); 
End-User testing: YMSM 
and YTGW (N=20); Youth 
Advisory Board (N=10) 
Quantitative and qualitative 
summary of heuristic 
violations; thematic 
analysis of think-aloud 
protocol, quantitative 
summary of mouse clicks, 
time, etc. 
2 
Randomized 
controlled 
design 
High-risk YMSM and 
YTGW 
(N=525; arm 1 n=210, arm 
2 n=210, arm 3 n=105) 
Descriptive statistics; 
Linear regression model 
3 
Analysis of Para 
Data 
Data Collected through 
Participant use of the 
mLab App (N=210) 

03-20-24 
10 
 
 
evaluate sex differences; the justification for only including assignment of male sex at birth is that the 
epidemiologic data demonstrate a clear, disproportionate impact of HIV on young racial and ethnic minority 
men. 
 
Aim 1: Refine the mLab App – a mobile and connected health intervention for increasing the uptake of 
rapid home HIV self-testing and linkage to care – by employing rapid cycle iterative design methods with 
feedback from human computer interaction experts (N=5), end-users (N=20), and youth advisory board 
members (N=10). There are three steps with refinements to the mLab App following each step. Step 1. 
Heuristic Evaluation with experts, Step 2. End-User Usability Evaluation. Step 3. Review by YAB. Note the 
Youth Advisory Board members will not be self-testing. They will only review the content and function of the 
app. 
 
Usability Testing. The goal of the usability evaluation is to refine the mLab App User interface. Usability 
Testing will promote improving the quality of the user’s interaction with and perceptions of the mLab App. 
Step 1. Heuristic Evaluation. We will conduct a Heuristic Evaluation with five informaticians who will 
participate as usability experts and must have training in human-computer interaction and have published in the 
field of informatics. 
Procedures. 
Heuristic Evaluators will be asked to complete a short demographic survey. Following completion of the 
survey, the usability experts will be asked to use the Oraquick test and also the mLab App. 
 
The mLab App is mobile app that is accessible using a login name and password. The app will provide HIV 
prevention information, push notification reminders for testing, step-by-step instructions for using the OraQuick 
HIV tests, an image upload function to send an image of your OraQuick HIV test to the study team. 
 
Participants will: 
a) be asked to manually enter the Oraquick test results; and 
b) take a picture of the test results for the app to analyze the results; and 
c) receive the app’s interpretation of the Oraquick test results. 
 
By using the app, we are asking participants to help test the app's ability to interpret the results of the OraQuick 
tests. If you test HIV positive by the OraQuick HIV test and/or the mLab app will provide information on how 
to set up follow-up confirmatory testing within the following 24 hours. All information within the app will be 
stored on a secured server at Vanderbilt University. 
 
While Morae Software is recording their actions, participants will be asked to use the following use cases while 
using the app. 
 
First Login 
1) Taking A Test 
a. If it’s your first time logging in, it’s time for a test, click on the “New Test” tab 
i. Watch the video 
ii. Read the instructions and make sure you are clear on how to complete your OraQuick test 
(If you need additional help please click the “Help” tab) 
iii. Do the Oraquick test 
iv. Click on the start timer 
v. Do a task you find relaxing for the duration of the timer 
vi. Read your results and enter what you think they are 
vii. Get your results on the results page 
2) Other tasks: 

03-20-24 
11 
 
 
a. Check for nearby services 
b. Contact us to ask a question 
c. Check history 
d. Look at the HIV facts on the homepage and save the ones you like 
e. Read and watch videos on PEP and PrEP 
f. Check your inbox for messages 
g. Look at your timeline on the homepage for the date of your next test 
3) Logout 
 
Second Login 
 
1) Check the Testing Timeline on the Homepage to see when your next test should be 
2) Check history under the Test History tab after getting back the results 
3) Check for nearby services 
4) Contact us to ask a question 
5) Check history under the Test History tab 
6) If it’s not time for a test, you can: 
a. Look at your timeline on the homepage for the date of your next test 
b. Check your inbox for messages 
7) Logout 
 
Heuristic evaluators will be asked to evaluate the mLab App interface using the Heuristic Evaluation Checklist 
and to think-aloud while they perform the usability testing.95 The process will be recorded using Morae 
software,96 which allows the researcher to record and analyze the audio recording and screen shots that are 
captured during the heuristic evaluation. Instrument. Nielsen97 proposed a list of ten recommended heuristics for 
a usable interface design. Bright et al.98 developed a Heuristic Evaluation Checklist based on Nielsen’s ten 
heuristics. For the purpose of this study, the checklist will be further modified to apply to mobile devices and 
will be adapted based on Nielsen’s heuristics for a mobile device.95 Each heuristic will be evaluated and the 
overall severity of the identified heuristic violations will be rated. Data analysis. The frequencies of usability 
issues will be calculated according to the heuristic principles adapted from Nielsen.99 Mean severity scores will 
be calculated for each heuristic principle. Evaluators’ comments about usability problems on the evaluation 
form and the Morae recording will be grouped and content analyzed according to the usability factors of 
Nielsen’s heuristics.99 Dr. Schnall will review the final list of recommended changes with Drs. Garofalo and 
Kuhns. Finally, Dr. Scherr will review and implement the final list of recommended changes. Schnall has used 
this methodology in numerous mobile intervention studies and has published widely on the findings.35, 89, 100-106 
 
Step 2. End-User Usability Evaluation. Study Population. Our study participants will be young men and 
transgender women (inclusive of YMSM and YTGW age 18-29 years) who have engaged in recent high-risk 
sexual behavior with a man. Inclusion criteria. (1) 18-29 years of age (see justification below); (2) assigned 
male sex at birth of any current gender identification; (3) understand and read English; (4) sexually active and 
at risk for HIV infection per CDC guidance (e.g., YMSM or YTGW and recent anal sex with men)107(5) 
smartphone ownership; and (6) self-report being HIV-negative or unknown status.. Exclusion Criteria. (1) 
persons who have a known diagnosis of HIV; and (2) persons for whom the investigators determine that 
participation may be detrimental to the participant or to the study (e.g., severe cognitive deficit). 
Justification for Key Inclusion/Exclusion Criteria. The proposed inclusion ages of 18-29 captures the upper 
range of the greatest increase in new infections,54 and the youngest age is the average age when sexual initiation 
begins108, 109 (see Inclusion of Children section). We also carefully considered inclusion of participants who are 
taking PrEP and will exclude participants who report taking it at baseline. While there is a growing body of 
literature on poor PrEP adherence, 110-112 particularly among Blacks and Latinos (NIMHD funding priority) as 
these are the two racial and ethnic minority groups who are most heavily burdened by HIV, 4, 6-8, 113 including 
those on PrEP at baseline would attenuate the intervention effect due to PrEP treatment guidelines, which 

03-20-24 
12 
 
 
include HIV testing at 3-month intervals. We will be collecting data on PrEP use and self-reported adherence to 
PrEP as part of the trial. Individuals on PrEP at follow-up visits will be encouraged to follow their current 
treatment protocol with their PrEP provider, including HIV/STI testing requirements and medical visits. We 
will control for PrEP use at follow-up in all analyses. 
YMSM and YTGW. We will be including BOTH YMSM who have sex with men and YTGW who have sex 
with men. We believe that targeting of those youth most at-risk of acquiring HIV is the most efficient strategy 
for implementation in practice. Based on the research to date, barriers to HIV testing for YMSM and YTGW 
have significant overlap,114, 115 suggesting that the mLab App which seeks to overcome these barriers is likely to 
increase HIV testing uptake in both groups. Nevertheless, we will test potential differences in intervention 
response by race/ethnicity, sexual and gender minority status (i.e., YMSM vs. YTGW), and co-morbid 
conditions (i.e., depression and anxiety symptoms, history of trauma) in order to inform future tailoring or 
targeting of the mLab App. 
Settings. This will be a multi-site study in NYC and Chicago. NYC, NY (Schnall, MPI).116 In 2013, there were 
117,618 persons living with HIV in NYC. Of these, 43,940 (over 37%) were infected through male to male 
sexual contact. Our main recruitment site will be through the Columbia University Medical Center Community 
Partnership for Health. In addition, we have a number of close community partners in NYC from where we will 
be recruiting. We work closely the LGBT Center in lower Manhattan and Ali Forney, which provides services 
for homeless young MSM and TGW and particularly Black and Latino youth. Chicago, IL (Garofalo, MPI). 
The 2013 rate of HIV infection diagnoses in Chicago (40.4 per 100,000) is approximately 2.5 times higher than 
the national rate, and the prevalence rate for Chicago (827.9/100,000) is nearly 3 times the national rate. Our 
main recruitment site will be through The Center for Gender, Sexuality and HIV Prevention at Lurie Children’s 
Hospital within the Division of Adolescent Medicine, directed by Dr. Garofalo. Additionally, we will advertise 
the study to the entire hospital system in the Chicago area to recruit youth and collaborate with community 
partners who serve youth. 
All of our study activities will be conducted in a private conference room at the Columbia University School of 
Nursing or at the Lurie Children’s Hospital. 
Sample. We will recruit 20 youth from both Chicago and NYC to participate in the evaluation of the mLab App 
interface screens. According to Faulkner,117 20 users are sufficient to identify 99% of usability problems. Each 
participant will be given $40 for participating in the usability evaluation, which is expected to take 60-90 
minutes. 
Participant Recruitment. We will use a multi-modal recruitment strategy. The research team has extensive 
experience recruiting YMSM and YTGW of color into research studies, including the pilot study of the mLab 
App, in which 12/18 participants (67%) were non-white and the preliminary study in which 70% were non- 
white (CDC U01 PS004975). 
We have used a variety of recruitment venues for other similar studies and maintain strong working 
relationships with online advertising vendors and local community based organizations. Although online venues 
are constantly evolving, in the past major categories of recruitment have included social network sites (e.g., 
Facebook, Instagram, Twitter); online sexual networking apps (e.g., Grindr, Scruff); and banner advertisements 
on other websites frequented by MSM (e.g., POZ).We will recruit youth through posting flyers and promoting 
the study through community partners (e.g., community-based organizations – see letters of support). Those 
recruited through these flyers will be directed to an online web survey (e.g., REDCap) for eligibility screening. 
Facebook: We will conduct targeted recruitment via Facebook. This involves targeting advertisements to all 
adults in the two study cities based on several factors, including interests expressed in profiles and through 
expressed "likes" of other posts or pages with gay and trans themes. Those who click on the Facebook ads will 
be referred to a brief introduction script and a self-administered online screening form with questions identical 
to those to be administered in venues. Grindr. We will recruit on Grindr using banner or pop-up ads that target 
YMSM and YTGW from NYC and Chicago. Grindr has the ability to target members who provide their 
race/ethnicity in their profile. Those who click on the banner ad will be directed to the brief online consent and 
screener to determine eligibility. BGCLive (Black Gay Chat). We will also recruit YMSM and YTGW on 
BGCLive using banner ads. Based on our previous recruitment, almost 90% of men recruited from their website 
self-identified as primarily Black. Media Sites. The study will set up an Instagram profile to promote the study, 
and we will also advertise through Tumbler, Yikyak, Periscope, and Snapchat. Craigslist Advertisements. We 

03-20-24 
13 
 
 
will post free daily ads on Craigslist, in both the Volunteers and Seeking Men sections. 
Those who click on the banner ad will be directed to the brief online consent and screener to determine 
eligibility. 
Alternatives and Limitations. We decided against using respondent driven sampling (RDS) due to risk of bias 
because there are known social networks in our study population.118 Although it has been reported that if certain 
conditions are met and if the appropriate estimation procedures are used, bias can be minimized,119 in our 
previous study, “Crew 450” with YMSM (ages 16-20) (R01 DA025548; MPI: Garofalo), RDS resulted in an 
inefficient rate of recruitment, and the technique was ineffective in generating a representative sample.120, 121 
 
Procedures for usability testing with 20 end users. 
Prior to the start of usability testing, participants will be asked to complete a demographic and behavioral 
survey entitled mLab usability demographics. 
 
Following their completion of the survey, participants will be given an Oraquick test and access to the mLab 
App. The mLab App is mobile app that is accessible using a login name and password. The app will provide HIV 
prevention information, push notification reminders for testing, step-by-step instructions for using the OraQuick HIV 
tests, an image upload function to send an image of your OraQuick HIV test to the study team. 
 
By using the app, we are asking participants to help test the app's ability to interpret the results of the OraQuick 
tests. If you test HIV positive by the OraQuick HIV test and/or the mLab app will provide information on how 
to set up follow-up confirmatory testing within the following 24 hours. All information within the app will be 
stored on a secured server at Vanderbilt University. 
 
Each participant will use the mLab App at the study site. The mLab App is mobile app on your phone that is 
accessible using a login name and password. The app will provide HIV prevention information, push 
notification reminders for testing, step-by-step instructions for using the OraQuick HIV tests, an image upload 
function to send an image of your OraQuick HIV test to the study team. 
 
Participants will be asked to: 
a) manually enter the Oraquick test results; and 
b) take a picture of the test results for the app to analyze the results 
c) you will receive the app’s interpretation of the Oraquick test results. 
 
By using the app, we are asking participants to help test the app's ability to interpret the results of the OraQuick 
tests. If you test HIV positive by the OraQuick HIV test and/or the mLab app will provide information on how 
to set up follow-up confirmatory testing within the following 24 hours. All information that you provide within 
the app will be stored on a secured server at Vanderbilt University 
 
We will use Morae software™ and a think-aloud protocol that asks participants to say whatever they are 
looking at, thinking, doing, and feeling as they go about their task.122-124 Before we start the task of using the 
mLab App, we will first explain the think-aloud protocol to the participants and ask them to do the simple 
practice task of counting the windows in their house/apartment while thinking aloud. The skills demonstration 
session will be videotaped so that the research team can assess the participant’s performance, similar to the one 
that we created in our work with the HIV OraQuick Test.78, 84 
 
During the think-aloud procedures participants will be asked to complete the following use cases: 
mLab Use Cases 
First Login 

03-20-24 
14 
 
 
4) Taking A Test 
a. If it’s your first time logging in, it’s time for a test, click on the “New Test” tab 
i. Watch the video 
ii. Read the instructions and make sure you are clear on how to complete your OraQuick test 
(If you need additional help please click the “Help” tab) 
iii. Do the Oraquick test 
iv. Click on the start timer 
v. Do a task you find relaxing for the duration of the timer 
vi. Read your results and enter what you think they are 
vii. Get your results on the results page 
5) Other tasks: 
a. Check for nearby services 
b. Contact us to ask a question 
c. Check history 
d. Look at the HIV facts on the homepage and save the ones you like 
e. Read and watch videos on PEP and PrEP 
f. Check your inbox for messages 
g. Look at your timeline on the homepage for the date of your next test 
6) Logout 
 
Second Login 
 
8) Check the Testing Timeline on the Homepage to see when your next test should be 
9) Check history under the Test History tab after getting back the results 
10) Check for nearby services 
11) Contact us to ask a question 
12) Check history under the Test History tab 
13) If it’s not time for a test, you can: 
a. Look at your timeline on the homepage for the date of your next test 
b. Check your inbox for messages 
14) Logout 
 
If a participant is HIV positive, the study team member will refer them to the HIV clinic (Dr. Garofalo [MPI] in 
Chicago; Dr. Olender [Co-I] in New York) for further evaluation and treatment (See Protection of Human 
Subjects Section). Participants will upload images of their test results through the mLab App. 
We will also ask participants to answer the following open-ended questions. 
 
Thinking back the mLab app, how would you apply this information, lessons, or activities in your own 
life? 
Probes: 
 
-How well does the app address the things that concern you most right now? 
 
-How do you believe the material from the app has context to “real life”? 
 
-What particular information or activities were most useful to you personally? 
 
-How well could you apply the information from the app to a situation(s) that you encounter on a day-to-day 
basis? 
How do you perceive this app would be of relevance to other young adult MSM and TGW? 

03-20-24 
15 
 
 
Potential Probes: 
 
-How well does the app address the things that concern your friends most right now? 
 
-What particular elements do you believe are most useful to the young adult MSM community more generally? 
 
-How age-appropriate were the content and activities? 
 
Following completion of the think-aloud protocol and interview questions, participants will complete a usability 
questionnaire comprised of the Health Information Technology (IT) Usability Evaluation Scale (Health- 
ITUES).93, 125 and the PSSUQ. 
 
Data Analysis. Think-aloud data from the skills assessment will be coded by graduate research assistants under 
the guidance of Dr. Schnall. The analysis will be based on the Morae recordings of user sessions. The mean 
task performance time will be calculated. The researcher will search for critical incidents which will be 
characterized by comments (i.e., Please help; difficulties using the system), silence, and repetitive actions. 
Through the observation of the use of the mLab App and review of the videos by the PI and graduate research 
assistants, we will be able to capture rich data about participants’ problems using the mLab. The analysis will be 
based on the Morae recordings of user sessions, transcriptions, field notes, and the user surveys. Content 
analysis, a technique for making replicative and valid inferences from data, will be performed. 
Step 3. Review with YAB. Drs. Schnall and Garofalo will facilitate the review of the mLab App user 
interface, content, and function with the YAB. Through our review, we will ensure that the mLab App is 
culturally appropriate for both YMSM and YTGW. Dr. Schnall will insert changes directly into the mLab App 
mock-ups viewed on a screen, supplemented by note-taking by study staff and audio-recorded. This allows us 
to confirm meaning, make revisions as necessary, and reach consensus with the YAB. Dr. Garofalo will probe 
for further discussion on feasible ways forward and to ensure that the mLab App meets the current prevention 
guidelines. We will incorporate the recommended changes. As a final step, we will have members of the YAB 
test the final mLab App after the meeting in their own settings (e.g., Home) and return any comments or 
additional edits within 2 weeks. Final recommendations for the user interface will be shared with Dr. Scherr, 
who will work with his team and the graphic design team at Little Green Software (see letter of support) to 
make the final updates to the app. 
Aim 2: Evaluate the efficacy of the mLab App in improving both HIV testing rates and linkage to care 
versus standard preventive care in 525 Black, Latino and other young men and transgender women 
(inclusive of YMSM and YTGW age 18-29 years) in a 12-month multi-site trial, with randomization at 
each site to three groups arm 1: intervention (mLab App) (N=210); arm 2: standard of care HIV 
information control arm (N=210); arm 3: OraQuick home test kits only (N=105). 
Hypothesis 1. Youth randomized to the mLab App arm, as compared to youth in the standard of care HIV 
information control arm, will have: a) increased rates of HIV testing within the first 6 months of the trial; and 
b) increased rates of re-testing in months 7-12 of the trial. 
Hypothesis 2. The total number of youth who test HIV positive and link to care in the mLab App arm will be 
greater as compared to the standard of care HIV information control arm. 
Hypothesis 3. The image processing in the mLab App will: 1) have no statistically significant difference in 
sensitivity or specificity when compared to confirmatory testing, 2) have greater sensitivity and specificity when 
compared to the user-submitted self-analyses. 
Design Overview. A randomized controlled trial will be conducted with 525 young men over 12 months. 
Participants will be randomly assigned to the mLab App arm 1 (intervention) or a control arm 2 (standard of 
care HIV testing information arm) or arm 3 (OraQuick HIV home tests).127 Participants in arm 1 will receive 
access to the mLab app and HIV home test kits, those in arm 2 will receive standard of care HIV information 
only. The primary comparison will be between arms 1 and 2. Participants in arm 3 will receive HIV home tests 
kits, but not access to the mLab app. Participants in all arms will receive standard preventive care at baseline. 
Recruitment. Our recruitment activities will mirror those described in Aim 1. Step 2. We will recruit youth 
from both Chicago and NYC as well as the entire state of Illinois and neighboring states of Wisconsin, 

03-20-24 
16 
 
 
Michigan and Indiana in the Midwest; and all of the state of New York and neighboring states of Connecticut 
and New Jersey. We will recruit on Grindr and other social media using banner or pop-up ads that target 
YMSM and YTGW from these areas. 
Sample. Our sample will include 525 Black,  Latino, and other young men and transgender women (inclusive 
of YMSM and YTGW aged 18-29 years), with Chicago: N=180 and New York: N =345; representing the 
population size of each city.128 Statistical power and sample size calculation. This study will use a 2:2:1 RCT 
design. We plan to recruit total 525 subjects (210, 210, and 105 will be in arm 1, 2, and 3, respectively). The 
power and sample size calculations are based on the comparison of HIV testing rates in the mLab App arm 
compared to the standard of care HIV information control arm for Hypothesis 1 (i.e., compare HIV testing at 6 
months and re-testing during 7 to 12 months). We calculated the statistical power to detect a difference across 
all participants (YMSM and YTGW). Our study is not powered to detect differences in outcomes in YMSM and 
YTGW separately but we will analyze differences in effect by gender identity (see Data Analysis Section 
below). The effect size is calculated based on the findings from the FORTH RCT75 in which self-testing 
resulted in a two times increase in frequency of HIV testing in high-risk MSM, and a nearly four times increase 
in non-recent testers compared with standard care, without reducing the frequency of facility-based HIV testing. 
Using a Poisson distribution, we estimated a 40% HIV testing rate over 6 months for the control group and 
approximately 60% HIV testing rate for the intervention group. We are using a more conservative effect size 
than the one reported previously by assuming 60% testing rate for the intervention as compared to 85% in the 
FORTH RCT study.75 This 20% difference is equivalent to a medium effect size. All power calculations are 
based on alpha=0.05 and two-sided tests of the primary comparison and an attrition rate of 20% at 6 and 12 
months. We will have at least 80% power for Hypothesis 1, to conduct two tests simultaneously (i.e., to 
compare HIV testing rate during the first 6 months and to compare re-testing rate during 7 to 12 months). 
Eligibility criteria. In addition to the eligibility criteria listed in Aim 1 – End-User Usability Testing, we will 
also include the following eligibility criteria: 1) understand the limitations of the OraQuick test and the mLab 
App (e.g., a confirmatory test is needed – see Protection of Human Subjects); 2) not having been tested for HIV 
in the past 6 months (e.g., therefore being somewhat outside of the current CDC testing recommendations for 
high-risk populations – see below). 3) not currently taking PrEP and 4) receive a non-reactive result on the 
rapid HIV test at the baseline visit 
Alternatives and Potential Limitations. We carefully considered the frequency of testing for our study 
participants and have agreed that participants should test at least every 3-6 months based on the following 2 
considerations: 1) Current CDC guidelines recommend annual testing and more frequent (every 3-6 months) for 
sexually active MSM at high-risk for HIV infection based on individual risk factors.129, 130 2) Since all of our 
participants meet the criteria for use of PrEP107 and based on our clinical experience for interpretation of these 
guidelines, we have specified that our study participants should test a minimum of every 3-6 months.27,28  As 
part of the intervention, participants in arm 1  will  be reminded to use the OraQuick tests at 3 month, 6 month, 
9 month, and 12 month time points through the mLab app. 
Study Enrollment. Study participants will come in to our study site in NYC or Chicago to provide informed 
consent, enroll, and complete a self-administered baseline survey on an iPad. Prior to enrollment into the RCT 
participants will be consented to screen and administered a rapid HIV test to determine eligibility. If a 
participant is found to be preliminary positive in the screening procedures they will be linked to care at NYP in 
New York or Lurie Children’s Hospital in Chicago or a healthcare provider of their choice. If they are non- 
reactive they will be randomized and enrolled into the trial. The baseline survey will include questions on 
demographic characteristics; health literacy131; sexual risk behaviors including number of men (and women) 
they engaged in anal or oral sex with, condomless anal intercourse, as well as their HIV testing history and 
opinions regarding HIV testing. The baseline survey will also include questions on PEP/PrEP use and 
adherence, drug and alcohol use,132 and the HIV Risk Index.133 After completing the baseline survey, study 
participants in all three study arms will receive condoms, an information sheet on HIV prevention, PrEP 
assessment, referral information for clinics that provide PrEP, and a card with the study team’s contact 
information and the NYP/Columbia Comprehensive Care Clinic Phone/Text Number (NYC) and 
Northwestern/Lurie Children’s Hospital (Chicago). 
Randomization. After providing informed consent, youth will be randomized to study arms in a 2:2:1 ratio of : 
arm 1-mLab App (2), arm 2-Standard of Care HIV information arm (2), arm 3-HIV home tests (1). We will use 

03-20-24 
17 
 
 
stratified randomization134 to ensure equal representation between treatment groups at each study site (Chicago 
vs. NYC).135 To reduce opportunities for selection bias, we will use a variable permuted randomization block 
design136 where the block size itself is randomly selected (i.e., blocks of five to ten). The advantage of the 
permuted block design is that treatment assignment is pre-determined before the trial begins and then assignment 
remains static throughout the course of the trial.137 Within each city stratum, a computer program will randomly 
assign each participant to the next treatment allocation from a random-permuted block randomized sequence. 
All three arms will receive standard-of-care HIV/STI testing-related risk reduction counseling, a box of 
condoms, PrEP assessment, and referral information for clinics that provide PrEP. 
Alternatives and Potential Limitations. Given that participants must be HIV negative to participate in the study, 
HIV testing will be included as the final screening component to verify study status prior to enrolling. This will 
help ensure that participants are not falsifying information on their screeners to enroll in the study. Importantly, 
participants are expected to continue normal healthcare activities during the course of the study and to be 
offered HIV testing referral information and on-site testing through existing service providers. 
Arm 2- Standard of care HIV information control arm. Youth will receive standard-of-care HIV/STI testing- 
related risk reduction counseling, a box of condoms, PrEP assessment, and referral information for clinics that 
provide PrEP during your first visit. Youth randomized to arm 2, standard of care control, will be sent an email 
or text with links to mobile-optimized online prevention information, including PrEP and HIV testing 
information that is found on the CDC website. 127 They will also receive a study information card listing the 
Columbia University School of Nursing / Lurie Children’s study teams’ contact information. Once youth are 
randomized, they will be analyzed according to their original assignment (i.e., intent-to-treat). 
Arm 3-HIV home testing arm. Youth randomized to arm 3 will be provided with the 2 OraQuick tests 
(including the package insert), and a box of condoms to take home at baseline. They will receive 2 more 
OraQuick tests at the 6 month visit. Participants may come in person to receive their tests or their tests may be 
mailed to them at their verified address if they are unable to complete an in-person 6 month visit. OraQuick 
tests will be sent via tracked shipping through USPS, FedEX, UPS. At their baseline appointment, youth will 
also be sent an email or text with links to mobile-optimized online prevention information, including PrEP and 
HIV testing information that is found on the CDC website. They will also receive a study information card 
listing the Columbia University School of Nursing / Lurie Children’s study teams’ contact information 
Arm 1-Intervention Arm (mLab App + HIV home test). Youth randomized to the intervention arm will be 
provided with the mLab App, 2 OraQuick tests (including the package insert), and a box of condoms to take 
home at baseline. They will receive 2 more OraQuick tests at their 6 month visit. Participants may come in 
person to receive their tests or their tests may be mailed to them at their verified address if they are unable to 
complete the in-person visit. The OraQuick tests will be sent via tracked shipping through USPS, FedEx, and 
UPS. At their baseline appointment, youth will also be sent an email or text with links to mobile-optimized 
online prevention information, including PrEP and HIV testing information that is found on the CDC website. 
They will also receive a study information card listing the Columbia University School of Nursing / Lurie 
Children’s study teams’ contact information. The goal of the mLab App is not for partner testing. We will 
advise participants upon study enrollment and consent (see Protection of Human Subjects) that the technology 
is NOT to be used for this purpose in this study. Given this limitation, we will suggest that participants who 
have partners who are interested in testing should contact the study team in each respective city for more 
information on HIV testing. We will also reinforce with participants that the mLab App is considered a 
screening device and they should seek confirmatory testing (for both negative and positive test results). The 
mLab App is mobile app on your phone that is accessible using a login name and password. The app will 
provide HIV prevention information, push notification reminders for testing, step-by-step instructions for using 
the OraQuick HIV tests, an image upload function to send an image of your OraQuick HIV test to the study 
team. 
 
Participants will: 
a) be asked to manually enter the Oraquick test results; and 
b) take a picture of the test results for the app to analyze the results; and 

03-20-24 
18 
 
 
c) receive the app’s interpretation of the Oraquick test results. 
 
By using the app, we are asking participants to help test the app's ability to interpret the results of the OraQuick 
tests. If you test HIV positive by the OraQuick HIV test and/or the mLab app will provide information on how 
to set up follow-up confirmatory testing within the following 24 hours. All information that you provide within 
the app will be stored on a secured server at Vanderbilt University 
Reminder notifications. Participants in the intervention arm will be instructed to use the mLab App and the HIV 
self-test kit at 3 months, 6 months, 9 months, and 12 months after enrollment. Participants will receive a 
reminder push notification 2 weeks prior to their scheduled test date and 1 week after their scheduled test 
date.138 In the mLab app, if a participant indicates they have received a preliminary positive HIV test using the 
home HIV test, participants will be directed to another screen on their smartphone directing them to call the 
study team or the warm line at the NYP or Lurie clinic (depending on geographic location) to receive follow-up 
confirmatory testing [and if needed, linkage to care]. Participants will be told that this result does not 
necessarily indicate that they are HIV positive (see false positive in Protection of Human Subjects), that further 
follow-up testing is needed, and they should contact the clinic or the study team within 24 hours. The mLab 
team will return the call within normal business hours. The number and nature of calls from participants to the 
study team (e.g., related to mLab App use, test interpretation, distress at reactive results) will all be recorded 
and will be reviewed yearly or at a date determined by the DSMB with our DSMB (see letters of support). We 
will track linkage to follow-up testing and care. Our DSMB will carefully monitor our study activities and data 
– see Data Safety and Monitoring Plan.Sound Retention Efforts. Participants will be asked after study 
screening what would be the best way for us to remind them of the appointment (voice phone, text, e-mail). 
Participant retention during the intervention will beenhanced in several ways. First, participants will be 
compensated for their time to complete the baseline assessment and completing the follow-up assessments. In 
addition, during the follow-up survey (6 months), participants will be asked to update their locator information. 
Using these extensive procedures promotes participant attendance at the follow-up visits, which has been shown 
to be successful in previous trials.139, 140 We have 4 levels of outcomes: primary, secondary, intermediate, and 
program evaluation. The survey items are included in the Qualtrics attachment. The primary outcome for Aim 2 
/ study trial (Uptake of rapid HIV 
self-testing) is the proportion of participants who self-report being tested for HIV in the past 6 months. The 
secondary goal is to collect and analyze data on the app’s test interpretation performance. For the intervention 
arm (arm 1), we will also have data to confirm their self-report since we will have a photo image of their 
OraQuick test. We will assess and describe any bias in self-report of HIV testing in the intervention condition. 
Follow-Up Surveys. All participants will receive a follow-up survey at 6 and 12 months after the baseline visit. 
The follow-up survey will be administered either in person on an iPad or, if they are unable to complete an in- 
person visit, remotely through a Qualtrics survey distributed via email. In total there will be three survey time 
points for all study participants (baseline, 6 and 12 months). Study participants will receive $40 at baseline, $55 
at 6-month follow up, and $75 at 12-month follow up for the time to complete the surveys. 
Data Analysis. Groups 1, 2, and 3 will be described with respect to baseline characteristics (e.g., means, 
standard deviations, ranges, and proportions). Before beginning formal analyses, we will examine the patterns 
of missing data, paying special attention to the balance of missing data in the study arms. From our prior trials, 
we anticipate all participants will provide baseline data, 90% will provide immediate post-intervention data, and 
approximately 80% will complete the 6 and 12 month post-enrollment assessment. Analysis of Intervention 
Effects. All multivariate analyses will be preceded by standard descriptive bivariate analyses to describe the key 
variables and relationships among them. These analyses will include means, frequency tables, histograms, and 
examination of distributions. Frequencies and rates of HIV tests, as well as corresponding confidence intervals, 
will be calculated for each arm. All statistical tests will be two-sided tests with the level of significance at 0.05. 
Hypotheses testing will be based on logistics models to compare (1) HIV testing during the first 6 months; and 
(2) HIV re-testing at 7-12 months between the mLab App arm and the control (Hypothesis 1). We will conduct 
stratified analyses to examine the differences in testing uptake in subgroups (i.e., racial/ethnic, age, YMSM vs. 
YTGW, and risk level). Effect sizes will also be compared between arms 1 and 3 to describe differences in 
testing and linkage behavior attributable to the mLab app with home test distribution versus home test 
distribution only. Linkage to care will be measured by the percent of study participants who tested positive and 

03-20-24 
19 
 
 
attended a first HIV care appointment at NYP/Columbia or Lurie or who provide documentation of care at 
another clinic and the time to link to care after positive result. Follow-up surveys for HIV positive participants 
will also include questions regarding positive diagnosis, such as disclosure and seeking care. For participants 
who test HIV positive, we will compare the rates of linkage to care between the mLab App arm and the control 
arm using Fisher’s exact test (Hypothesis 2). However, because the number of positive (HIV) tests is expected 
to be approximately 25-40, this part of the study will not have power to detect a significant statistical difference 
between groups. 
mLab Image-Processing Evaluation Methods. During use of the mLab App, participants will submit images 
of their Oraquick self-test to the app for analysis by an automated image-processing algorithm. Prior to 
submitting this image for analysis, participants will be asked to interpret the results of the test using the 
instructions that are provided standard with the OraQuick test. This self-reported result will be recorded and 
uploaded with the user-submitted image of their rapid test. Study team members with expertise in rapid test use 
and analysis will evaluate the uploaded images of each rapid test, and provide an additional assessment. The 
user-submitted result and the study team assessment will not be used by mLab in its automated image- 
processing. All users that submit tests that are identified as positive, either by the automated image-processing 
in mLab, the study team personnel’s inspection of the image of the rapid test, or by the study participant 
themselves, will be referred for confirmatory testing. 
(Hypothesis 3).mLab Image-Processing Data Analysis Plan. Including the confirmatory test, which will be 
treated as the gold standard to determine whether a test result is truly false or positive, there will be results from 
four analysis methods collected on the same set of tests (1. User-submitted results, 2. Study team interpretation 
of uploaded images, 3. Automated mLab App results, 4. Confirmatory testing). The sensitivity and specificity 
for each of these methods will be compared using McNemar’s test. This will result in the following statistical 
comparisons for both sensitivity and specificity: 1) user-submitted analysis to study team analysis, 2) user- 
submitted analysis to mLab analysis, 3) study team analysis of uploaded images, 4) user-submitted analysis to 
confirmatory testing, 5) study team analysis to confirmatory testing, 6) mLab analysis to confirmatory testing. 
Missing Data. Multiple imputation (MI) methods will be applied to address missing values under the missing at 
random (MAR) assumption (i.e., missingness can be fully accounted for by variables with complete 
information).141 These analyses will be complemented with assessment of how sensitive the inferences are to 
the MAR assumptions. Sensitivity analysis will be performed based on selection models for dropout.142-144 
Intention-to-Treat (ITT) Analysis: All analyses will use the ITT principle,145 which requires subjects’ data to be 
analyzed as randomized, regardless of whether they used the mLab App or not. 
 
Aim 3: Monitor and analyze paradata to understand the effect of user engagement of the mLab App on 
improving HIV testing rates and linkage to care. 
To understand barriers and/or facilitators of its use, the mLab app will automatically 
collect paradata for each user interaction throughout the study. This paradata is 
considered “free” in that it does not require any additional effort from the user.146 
This data will be analyzed for each user, as well as aggregated to understand what 
component of the intervention lead to behavior change. We will specifically 
examine who uses the mLab App and how frequently it is used over time. 
Methods. We will capitalize on the use of the extant mLab software which is written 
as a mobile-friendly web application in hypertext markup language (HTML), 
hypertext preprocessor (PHP), JavaScript, and Python (for computer vision), and 
integrates with REDCap for backend data storage. On each page of the mLab app, a 
PHP script logs every user interaction before the page of the application is displayed to the user. Therefore, any 
event that would refresh or send the user to a new page (e.g., HTML links) initiates an event-driven paradata 
capture. The PHP script transmits the data to REDCap through the secure REDCap API. 
Data Analysis. The primary paradata that will be collected is shown in Table 4. From this data, we will derive 
the following use data for each session: duration on each page, page progression through the application, time 
from login to result, number of rapid test pictures previewed before sent for result, and total time prior to logout. 
We will analyze the data on multiple scales and perspectives: individual-level (i.e. user-level), application-level, 
page-level, session-level, and how these differ by demographic characteristics, HIV testing history, technology 
Table. Paradata 
Collected for Each 
mLab Event 
1. Unique Code 
2. Page accessed 
3. Time stamp 
4. Rapid test image 
5. Internet browser type 
6. Operating system 
7. Device 

03-20-24 
20 
 
 
use, and outcomes measures. At the app-level, we will analyze timestamp data including time of day and day of 
the week that HIV self-testing is performed. We will also analyze the time-to-HIV self-testing result from App 
login -- a primary indicator of usability. Efficiency is a major construct of the Health-ITUEM model, the 
guiding theoretical framework for our study (see Figure 4). Additionally, we will measure the amount in bytes 
of user data transmitted. Importantly, longitudinal analysis will determine if user-experience changes with 
repeated use. 
The paradata collected from each page will be analyzed to generate a “heatmap” of user-interaction (i.e., the 
distribution of activity for each link/button). Use of the heatmap will inform user duration on each page of the 
application user interaction with the HIV prevention content, the contact pages, and the help page. We will 
explore usability issues with consideration for how many times users accessed help, and what page of the mLab 
app referred them to the help, implying the need for clarification. Similarly, contact page paradata will be 
analyzed to determine at what point in the use of the imaging software (the mLab) users initiated contact with 
the study team and/or the clinic. Specifically, referring pages to the contact page will be analyzed (i.e., do users 
engage with contact resources only after a diagnosis is provided?). Finally, we will use paradata to understand 
when users disengage with the mLab app (e.g., logout after received diagnosis, logout prior to diagnosis, 
session timeout). 
Since the mLab App provides an image-based diagnosis of HIV rapid tests, paradata will be used to 
understand the process of capturing a photograph for analysis. To this end, we will count the number of images 
previewed prior to submission of a final image for mLab analysis. Within each image, we will perform digital 
image analysis to determine orientation, luminosity, and major colors within an image – all factors that would 
indicate successful analysis in mLab. Importantly, this analysis will be used to improve the computer vision 
algorithm. 
We will analyze difference in the aggregated data by demographic group (age, race, and risk level) to allow 
us to better understand engagement with the intervention and potential facilitators and barriers to mLab App 
use. We will evaluate if grouping by HIV status affects these results in any way (e.g., do HIV+ users engage 
with features differently than HIV- users?). Finally, as a web application, there is potential for slightly varied 
experiences on different devices or web browsers, and we will evaluate if there are differences in mLab 
paradata across devices and browsers. 

03-20-24 
21 
 
 
Risks, Benefits & Monitoring 
Potential Risks 
 
The risks of participating in this study are: 
 
We will discuss each of these risks below and describe the protection again these risk under “Adequacy of 
Protection Against Risks.” 
General Risks: 
1. A risk of taking part in this study is the possibility of a loss of confidentiality or privacy, including that 
associated with the app on the participants’ phone. Loss of privacy means having their personal 
information shared with individuals outside of the study team who were not supposed to see or know 
about their information. The study team plans to protect the privacy of participants. Their plans for 
keeping your information private are described in the Privacy section of Protection Against Risk. 
 
Risks Specific to the Use of OraQuick: 
1. Subjects must follow package instructions on the OraQuick test and always read the test results 
from the OraQuick test themselves given that the accuracy of the mLab application has not been 
proven., Confirmatory HIV testing is still needed. 
2. Participants may mistakenly believe that a negative result of the OraQuick Test means that they are not 
infected with HIV. There is a chance that the test does not detect an HIV infection. To offset this risk, 
participants will be repeatedly told that a negative result in the OraQuick test does not mean that they are 
not HIV infected, since infections that occurred within the prior three months may not be detected by the 
test, and that condoms should still be worn to avoid the risk of HIV and other STIs. 
3. Participants may experience distress if they get a positive result in the OraQuick HIV test. If at any point 
during their participation in this study they request a referral for a confirmatory test, we will refer them to 
an HIV/STI clinic for a confirmatory test and referrals for treatment and care if needed, including 
psychological assistance. We will assist participants in scheduling an appointment for a first consultation. 
4. There is a risk of receiving a false positive test result, meaning participants have tested HIV positive, but 
are in fact HIV negative. Such participants will be instructed on the limitations of the OraQuick Test, 
including that false positive results have occurred, which again stresses the need for confirmatory testing 
since results may change. Limitations of the OraQuick Test: A potential risk of the study is that 
participants might mistakenly believe that the test kit protects them against HIV and/or other STIs. During 
the consent process, participants will be reminded that self-tests will not protect them against HIV or 
other sexually transmitted diseases or infections, and that they must use protection such as condoms when 
engaging in sexual intercourse if they want to avoid infection. Participants will also be reminded that HIV 
testing does not include testing for STIs. There is a small risk that a positive result is a mistake. Fewer 
than 1 in 100 people who get tested may receive an incorrect result. Therefore, if a participant tests 
positive, a second test is needed to confirm the result. If the participant receives a HIV positive result 
while testing during this study, we will provide confirmatory testing. Additional HIV testing will be made 
available to individuals who test negative and who request it. 
 
Risks Specific to the Use of the mLab App: 
1. The mLab App is not FDA-approved for this indication. Results from the mLab app are investigational 
and the manual read of the OraQuick In-Home Test is the result of record. 
2. There may be false positive results related to the performance of the mLab App, integration with REDCap 
and potential problems with the usability of the overall system. 
3. There may be false negative or positive results caused by issues with the imaging algorithm (e.g. lighting, 
geometries, and appropriately set processing thresholds). 
Adequacy of Protection against Risks 

03-20-24 
22 
 
 
a. Recruitment and Informed Consent 
 
Recruitment for study participation will occur following approval by the CUMC IRB. The PI will determine 
eligibility for inclusion, explain the purpose of the study, answer any questions, and obtain written consent from 
the participants. 
Patients who agree to participate will sign a consent form. Recruitment strategies will be developed in 
collaboration with clinics and community centers. Potential risks and strategies for risk management will be 
carefully explained as part of informed consent procedures. All HIPAA requirements will be applied to this 
study. The study plan, advertisements or recruitment letters, lay description of the study, and all consent forms 
will be submitted to the IRB following proposal acceptance and prior to study initiation. The study PIs will be 
responsible for obtaining IRB approval for this study. 
We will inform the study participants upon enrollment that the goal of the mLab App is for self-testing and 
NOT for partner testing/sero-sorting. We will advise participants upon study enrollment and consent that the 
technology is NOT to be used for their partners during the study. Given this limitation, we will suggest that 
participants who have partners who are interested in testing should contact the study team for more information 
on HIV testing. 
We will also remind participants that the mLab App is not an approved diagnostic test and recommend seeking 
confirmatory testing (for both negative and positive test results). Upon receiving a positive HIV test result, 
study participants will then be directed to another screen on their smartphone with the phone number of the 
Comprehensive Care Program Warm Line. Participants will be told that this result does not necessarily indicate 
that they are HIV-positive but that further follow-up testing is needed. Participants will also be advised to 
contact the Clinic or the study team within 24 hours of receiving a positive test result. The number and nature of 
calls from participants to the study team (e.g., related to mLab App use, test interpretation, distress at reactive 
results) will all be recorded. All participants will be advised to test quarterly, and will receive the referral 
number to the Comprehensive Care Clinic. 
 
 
b. Protections against Risks 
 
This study will be submitted to the IRB before starting the study. We will be careful to ensure that no coercion 
occurs during the recruitment periods by ensuring the voluntary participation of both healthcare providers and 
patients. All participants will be screened to assess for study eligibility. Non-participation will not affect 
patients’ medical care in any way. 
 
 
1) Confidentiality: The smartphone that is connected to the mLab App will require a password. All study 
data, including mLab App results will be encrypted and stored on secure HIPAA-compliant servers at 
the Columbia University Medical Center campus. All study data will be kept in password-protected 
computers or file cabinets in locked offices. All study data will be maintained in a completely secure and 
HIPAA-compliant environment. All servers have HIPAA-compliant security. Nonetheless, there is 
always the risk of a data breach, so we will make our study participants aware of this risk upon 
enrollment. 
2) False Negative Results: The mLab App has the potential for providing a false negative result, especially 
during the period of acute HIV infection in which a newly infected individual may be most contagious. 

03-20-24 
23 
 
 
We will clarify these issues before giving the mLab App to participants, test their understanding of the 
limitations of the HIV Home Test with a structured instrument (Appendix B), provide a study 
information card that contains information about test limitations, and recommend use of condoms 
regardless of the test outcome. Upon enrollment, we will explain to participants that the mLab App 
cannot determine their status with 100% accuracy. The mLab App may not detect acute HIV infection. 
3) Risk of receiving a false positive HIV home test result: Participants will be informed of the limitations 
of the mLab App, including the occurrence of false positive results. Again, the study team will stress the 
need for confirmatory testing. 
4) Distress a participant may experience if s/he receives a positive HIV result at home: If a research subject 
gets a positive result in the mLab App at any point during participation in this study, this result will 
require confirmation. The study team will provide the participant with a confirmatory test and referrals 
for treatment and care if needed, including psychological assistance. 
5) Window Period for Interpretation of Results: In the mLab App, we have added in 2 points of instruction 
related to the 20-40 minute window period for the Oraquick results – see Appendix L, slides 7 and 11. 
Second the images will be time stamped and we will not return algorithm results if the time from 
reporting in screen 7 and the time to take a picture in screen 11 has been less than 20 minutes or more 
than 40 minutes. 
6) Invalid Test Results: If the participant reports an invalid result – the software will send the following 
message back to the study participant: You reported that your results are invalid. An Invalid test result 
means that there was a problem running the test, either related to the specimen or to the Test Device. An 
Invalid test will not be interpreted by the mLab App. Repeat the test with a new Divided Pouch and a 
new oral fluid. Please contact the study team if you are unable to get a valid test result upon repeat 
testing. If the participant reports an invalid test then we will turn off the ability for the mLab App to 
return the algorithm results to the participants. However, invalid images will still be stored in REDCap 
and reviewed by the study team. 
7) Imaging Algorithm: The parameter values in the mLab App’s image processing algorithm have been 
validated and selected to primarily optimize sensitivity and specificity. However, it is possible for the 
mLab App to return both false positives and false negatives. The study team will monitor the images that 
are sent using the mLab App, and will contact study participants in the event of either: 1) the mLab App 
returns an incorrect result, and 2) a discrepancy between the participant’s manually reported result and 
the mLab App result. 
 
c. Additional Planned Procedures for Protecting Against/Minimizing Potential Risk/Additional 
Protections for Children 
 
 
Procedures in the Event of a Reactive HIV Test 
 
In the event a participant has a reactive HIV test, study team members will counsel the participant regarding the 
meaning of his result. Specifically, the study team will reiterate with the participant that: 
 
 
1) His result is a preliminary positive and it is likely that he is infected with HIV; 
2) The preliminary positive result needs to be verified with confirmatory testing; 
3) He needs to return for the results of his confirmatory testing; and 
4) Resources for counseling and referral to care are available to him. 
 
The participant will be referred for follow-up testing and care at a clinic at Columbia University Medical Center 
or Lurie Children’s Hospital. We have clinicians on staff at the clinic who are part of our study team. 

03-20-24 
24 
 
 
 
 
We detail below the risks and procedures undertaken to minimize them for the intervention arm. 
 
 
 
1. Participants must follow package instructions on the OraQuick test. They must always read the 
test results from the OraQuick test themselves, given that these results are more reliable than the mLab 
App results and the accuracy of the mLab app results have yet to be proven. Participants will be 
required to report the results from the manual real and enter it into the mobile app and will be 
contacted either by phone or by the app’s messaging portal. In addition, confirmatory HIV testing will 
be required if an individual provides positive test results from the manual read of OraQuick. 
2. Participants may mistakenly believe that a negative result of the Oraquick Test means that they are not 
infected with HIV. There is a chance that the test does not detect an HIV infection. To offset this risk, 
study staff will remind participants upon enrollment that a negative result in the OraQuick test does 
not mean that they are not HIV infected, since infections that occurred within the prior three months 
may not be detected by the test, and that condoms should still be worn to avoid the risk of HIV and 
other STIs. Study staff will ask participants to explain their understanding of the meaning of a negative 
Oraquick test result. 
3. Participants may experience distress if they get a positive result in the OraQuick HIV test. If a 
participant tests positive, he/she will be referred to a clinic for confirmatory testing without a request. 
If a participant tests negative, then we will refer only if he/she requests additional testing. If at any 
point during their participation in this study they request a referral for a confirmatory test, the study 
team will refer them to an HIV/STI clinic for a confirmatory test and referrals for treatment and care if 
needed, including psychological assistance. We will assist them in scheduling an appointment for a 
first consultation. 
4. There is a risk of receiving a false positive test result, meaning they have tested HIV positive, but are 
in fact HIV negative. They will be instructed on the limitations of the OraQuick Test, including that 
false positive results have occurred, which again stresses the need for confirmatory testing since 
results may change. Limitations of the OraQuick Test: A potential risk of the study is that the 
participant might mistakenly believe that the test kit protects them against HIV and/or other STIs. It is 
important that they know that self-tests will not protect them against HIV or other sexually 
transmitted diseases or infections, and they must use protection such as condoms when engaging in 
sexual intercourse if they want to avoid infection. HIV testing also does not include testing for STIs. 
A positive result from the OraQuick Test generally indicates that HIV antibodies have been detected 
in their body. There is a small risk that a positive result is a mistake. Fewer than 1 in 100 people who 
get tested may receive an incorrect result. Therefore, if they test positive, a second test is needed to 
confirm the result. If participants receive a HIV positive result while testing during this study, the 
study team will provide them with confirmatory testing. A negative result in the OraQuick test 
generally means that HIV has not been detected in their body and that they are HIV uninfected. A 
negative result could also mean that the participant may have gotten HIV too recently for the test to 
detect it. Additional HIV testing will be made available to all subjects. 
5. A risk of taking part in this study is the possibility of a loss of confidentiality or privacy, including that 
associated with the mLab app on their phone. Loss of privacy means having their personal information 
shared with someone who is not on the study team and was not supposed to see or know about their 
information. The study team plans to protect the participants’ privacy by storing all data on locked 
computers in locked facilities on secure servers hosted by Columbia University Medical Center. The 
servers are HIPAA compliance and set to conform to the current industry standards for storing 
Personal Health Information (PHI) 

03-20-24 
25 
 
 
BENEFITS: 
 
Potential Benefits of the Proposed Research to Human Subjects and Others 
 
This study has not been designed for the direct benefit of its participants; however, there are a number of ways 
in which they may derive benefit. The proposed research will inform the delivery of HIV prevention messages. 
The knowledge gained will contribute to the body of knowledge regarding the use of health information 
technology for improving the lives of MSM at risk for HIV. The avoidance of HIV through study participation 
will be a significant personal benefit to participants. 
Participants in the mLab App arm will be referred to standard-of-care services either at the time of a reactive 
HIV test result (if sero-positive) or the end of the study (if sero-negative). Standard-of-care services for HIV- 
negative MSM, regardless of risk level, includes provision of HIV/STI testing (based on the local HIV testing 
standard procedure) as described below. Standard-of-care services for HIV-positive MSM include provision of 
linkage to HIV care services based on the standard-of-care services offered in the local project site community. 
 
 
Importance of the Knowledge to be Gained 
 
Each new case of HIV constitutes a burden to the individual, the health care system of our country, and to 
society at large. Our study is expected to result in benefits for society since it will provide a knowledge base for 
HIV and STI prevention strategies. Point-of-care tests are used worldwide to screen individuals for antibodies 
against HIV. The risks to research participants enumerated above are reasonable in relation to the anticipated 
increase in knowledge on the potential use of self-testing and behavioral economics for improving outcomes 
across the HIV care continuum. 
 
ALTERNATIVES: 
The alternative is not to participate in the research. 
 
DATA SAFETY MONITORING PLAN: 
 
This study will have a data safety monitoring board (DSMB) which will meet yearly or on a schedule 
determined by the DSMB. The DSMB will consistof individuals who are not otherwise associated with the 
study who will review study progress, safety events and make recommendations to the investigative team. The 
DSMB will be asked to commit to 5-10 hours of time per year for meeting preparation, attendance, and report 
writing. They will be asked to elect a Chair who will lead the effort of the group, facilitate meetings and be 
responsible for report preparation. In the annual meetings,the DSMB members will review the evidence of study 
progress, including targeted versus actual recruitment, retention, and interim analysis of efficacy as well as 
review of safety events. Based on this review, the group will draft a report of recommendations to the 
investigators regarding current procedures and/or trial suspension in the case of safety events. The four 
members of the DSMB include Drs. Donenberg, Koblin, Pearson and Scrimshaw. 
 
 
Geri Donenberg, PhD has extensive experience conducting NIH-funded research nationally and internationally. 
As Principal Investigator/Co-Principal Investigator of nine federally-funded studies, and Co-Investigator of an 
additional 11, Dr. Donenberg brings vast expertise in basic longitudinal research, prevention and intervention 
development and adaptation, evidence-based prevention program implementation, and the conduct of 
randomized controlled trials with primarily low-income, ethnic minority youth and families. She has published 

03-20-24 
26 
 
 
over 80 peer-reviewed manuscripts underscoring the impact of family factors, individual attitudes and beliefs, 
and peer and partner characteristics related to adolescent sexual behavior and substance use, particularly youth 
with mental health problems and teens in the juvenile justice system. Dr. Donenberg has also published widely 
on youth outcomes related to HIV prevention interventions. Dr. Donenberg has been involved in numerous 
multi-site collaborative studies in the US and Africa, and she oversees two large research and service 
university-based programs designed to address HIV prevention, testing, counseling, treatment and linkage to 
care for underserved indigent youth and adults. 
 
 
Beryl Koblin, PhD, is the Head of the Laboratory of Infectious Disease Prevention at the New York Blood 
Center. She has led a research group for 25 years focused on HIV epidemiology in high-risk populations and 
conducted numerous behavioral and biomedical intervention trials. I have designed, conducted, and analyzed 
numerous epidemiologic studies and intervention trials involving MSM and have been Principal Investigator 
over 25 grants from NIH, CDC, and private foundations. I have significant experience with conducting research 
with young MSM through implementing the CDC-funded Young Men’s Surveys in New York City and the 
National HIV Behavioral Surveillance MSM survey. I led the development and implementation of HPTN 061, a 
feasibility study of a multicomponent intervention to reduce HIV infection largest cohort of Black MSM in the 
US. I am currently the Principal Investigator of an NIH-funded R01 study that seeks to develop an intervention 
utilizing technology to match young Black MSM to their optimal HIV testing approach 
 
 
Cynthia Pearson, PhD, is a Research Associate Professor School of Social Work and Associate Director at 
the Indigenous Wellness Research Institute Research Core, University of Washington. Her research 
incorporates the complexities of HIV/AIDS risks such as sexual behavior, substance use, and trauma; and 
identifies strengths and protective factors that support indigenous women’s wellness. Dr. Pearson reaches 
community members and identifies resources to create innovative sustainable interventions. Dr. Pearson has 
worked with communities in Mozambique, China, and indigenous communities of the Pacific Northwest. Dr. 
Pearson is the Principle Investigator of ETHICS: Ethics Training for Health in Indigenous Communities Study 
(R01HD082181). She is also a former fellow with Fordham University HIV and Drug Abuse Prevention 
Research Ethics Training Institute (RETI) and is a member of the Advancing Indigenous Research Ethics in 
Practice and Policy Committee at the University of Washington. Her expertise is in designing tribally-based 
health studies from an ecological perspective that emphasize social, economic, political, environmental, and 
historical determinants of health. Specifically, Dr. Pearson’s research focuses on the intersecting risk of 
substance use, historical and lifetime trauma, and HIV risk and how culture, place and community serve as 
protective factors (R34DA034529). Dr. Pearson meets community members where they are and identifies 
community ways of knowing and resources to create innovative sustainable interventions. She is the principle 
investigator and co-investigator on multiple federally-funded grants using a community-engaged approach, 
closely collaborating with tribal communities across the US in the promotion of American Indians and Alaska 
Natives wellness. 
 
 
Eric Schrimshaw PhD, is a social/health psychologist whose research focuses on the role of interpersonal 
relationships on health and well-being. His work and interests are focused on three aspects of social 
relationships. First, much of Dr. Schrimshaw's early work (including his dissertation) was focused on the 
beneficial role of supportive relationships and the negative impact of stigma, conflict, and rejection on mental 
and behavioral health outcomes. Second, Dr. Schrimshaw's work has addressed the health implications of 
concealing stigmatized identities. Specifically this work has focused on how self-disclosure or the 
communication of personal information with others has beneficial role in health and well-being, how 
concealment can have negative implications for health, and how non-disclosure can impede access to care and 

03-20-24 
27 
 
 
support. Finally, most recently, Dr. Schrimshaw's work has focused on how different social environments where 
sexual relationships are formed may impede communication and facilitate sexual risk. Of particular interest are 
the use of the Internet and smartphone technologies for meeting sexual partners, the influence of these 
technologies on communication, and whether these technologies could contribute to sexual risk. Employing a 
mixed-methods approach that involves both qualitative interviewing and quantitative survey methods, his work 
documents the importance of interpersonal relationships for understanding mental health, substance use, and 
sexual risk behavior. He has addressed these issues within several populations including adults living with 
HIV/AIDS, gay/lesbian/bisexual adolescents, gay/bisexual men, and bisexual men "on the down low." Dr. 
Schrimshaw has published over 50 journal articles addressing the role of interpersonal relationships and health. 
 
 
b. Procedures for Monitoring Data Management and Integrity 
 
b.1. To protect the integrity of participants’ data, the Project Coordinator will assign an ID number to all 
participants. The participant ID number will be used to identify data collected. Since the study has repeated 
follow-up visits, we will maintain a list of participants which link identifying information to study ID numbers. 
Only a limited number of staff members will have access to this list, which will be kept in locked files and in a 
password protected computer file. Other study personnel will have access on an as needed basis to perform their 
duties. 
 
 
b.2. All research personnel will complete extensive training before they are granted access to this 
identifying information. Study personnel will complete training in the protection of human subjects which 
complies with federal guidelines delineated in 45 CFR Part 46. Personnel will also sign confidentiality 
statements that specify that if the participants’ confidentiality is breached unintentionally that personnel will 
follow the procedures for reporting this breach to the Principal Investigator (PI). The confidentiality statement 
also states that unintentional or deliberate violations of participants’ confidentiality may result in demotion or 
termination depending upon the severity of the event. Personnel will also participate in training with the 
Investigators and/or Project Coordinator regarding data safety, confidentiality of participants, limits of 
confidentiality, and proper administration of the study protocol. 
 
 
b.3. We will store all personally identifiable data in locked cabinets. All forms containing personally 
identifiable information, including participant locator forms (i.e., containing contact information) will be 
maintained in a password-protected computer files. Data that are entered into computer files will be de- 
identified and maintained on a subdirectory of an institutional server with password protection. The PI will 
review all requests, current and future, to use the data, and any data files that are provided to other individuals 
will be de-identified. Data files will contain code numbers in order to make reference to particular cases across 
multiple assessment waves. 
 
 
b.4. We will apply for a Federal Certificate of Confidentiality, and the certificate will be maintained 
throughout the life of the research project. We will inform study participants of the above procedures and the 
limits of confidentiality during the assent/consent process, prior to data collection. Specifically, we will warn 
participants that state law mandates reporting of abuse and/or neglect of children, and that threat of harm to self 
or others requires intervention by clinical staff. We will inform participants that criminal behavior (i.e., drug 
use) is not reported to authorities, and that the security of this information is protected by the Federal Certificate 
of Confidentiality. 

03-20-24 
28 
 
 
 
 
b.5. Plan for Privacy and Security Protections. Beginning with the development process and throughout 
the research project, we will follow the privacy and security principles set forth at healthhit.gov. Our team is 
familiar with the importance of the privacy and security of personal health information to engender individual 
trust in the use of health information technology (HIT) applications.  We  have expertise  and  experience in 
this domain as we have developed a number of HIT systems  for persons  from high-risk  populations, 
including those who live with HIV, whose personal health information is usually held to higher security 
standards than traditional patients. Most recently, we built the web-based MyPEEPS system which is housed 
on the Columbia University Medical Center (CUMC) IT servers. All servers are located in a secure datacenter, 
with necessary redundancies. Currently the network can be accessed remotely via VPN with a Citrix solution 
being developed. All servers have HIPAA compliant security. 
 
 
c. 
Reporting Procedures 
 
All adverse events will be reported per the requirements of the CUMC IRB. Should any physical or 
psychological manifestation be exhibited at any time during the study, Dr. Schnall will consult with her co- 
investigators regarding clinical situations as they arise. If there are any serious adverse events, the PI will 
complete an adverse event form and report to the CUMC IRB within 48-72 hours. Once the PI notifies the 
CUMC IRB, they will be required to report any of the following to the NIH in a timely fashion: 
 
a. 
Unanticipated problems or unexpected serious adverse events that may be related to the study protocol 
(within 48-72 hours) 
 
b. 
IRB-approved revisions to the study protocol that indicate a change in risk for participants 
 
c. 
A summary of recommendations made by the DSMB or other monitoring entity and the action plan for 
response 
 
d. 
Notice of any actions taken by the IRB or regulatory bodies regarding the research and any responses to 
those actions 
 
 
To further ensure protection of human subjects, we have devised our Data Safety and Monitoring Plan so that it 
follows policies and procedures in place at Columbia University Medical Center. 

03-20-24 
29 
 
 
REFERENCES 
1. 
Shrestha RK, Sansom SL, Schulden JD, Song B, Smith LC, Ramirez R, Mares-DelGrasso A, Heffelfinger JD. Costs 
And Effectiveness of Finding New HIV Diagnoses by Using Rapid Testing In Transgender Communities. AIDS Education 
and Prevention. 2011;23(3_supplement):49-57. doi: 10.1521/aeap.2011.23.3_supp.49. 
2. 
Garofalo R, Kuhns LM, Reisner SL, Mimiaga MJ. Behavioral Interventions to Prevent HIV Transmission and 
Acquisition for Transgender Women: A Critical Review. Journal of Acquired Immune Deficiency Syndromes (1999). 
2016;72(Suppl 3):S220-S5. doi: 10.1097/QAI.0000000000001084. PubMed PMID: PMC4969058. 
3. 
Hall HI, Song R, Rhodes P, Prejean J, An Q, Lee LM, Karon J, Brookmeyer R, Kaplan EH, McKenna MT, Janssen RS, 
Group HIVIS. Estimation of HIV incidence in the United States. JAMA : the journal of the American Medical Association. 
2008;300(5):520-9. doi: 10.1001/jama.300.5.520. PubMed PMID: 18677024; PMCID: 2919237. 
4. 
Center for Disease Control and Prevention. HIV among African Americans 2017. Available from: 
https://www.cdc.gov/hiv/group/racialethnic/africanamericans/index.html. 
5. 
Center for Disease Control. HIV Among Gay and Bisexual Men 2017. Available from: 
https://www.cdc.gov/hiv/group/msm/index.html. 
6. 
Chen M, Rhodes PH, Hall IH, Kilmarx PH, Branson BM, Valleroy LA, Prevention CfDCa. Prevalence of undiagnosed 
HIV infection among persons aged ≥ 13 years--National HIV Surveillance System, United States, 2005-2008. MMWR 
Supplements. 2012;61(2):57-64. PubMed PMID: 22695465. 
7. 
Center for Disease Control and Prevention. HIV Among African American Gay and Bisexual Men 2017. Available 
from: http://www.cdc.gov/hiv/group/msm/bmsm.html. 
8. 
Center for Disease Control and Prevention. HIV among gay, bisexual, and other men who have sex with 
men.2017. Available from: https://www.cdc.gov/hiv/group/msm/index.html. 
9. 
Centers for Disease Control and Prevention. HIV among youth 2017. Available from: 
http://www.cdc.gov/hiv/pdf/library_factsheet_HIV_amongYouth.pdf. 
10. 
Mayer KH, Grinsztejn B, El-Sadr WM. Transgender People and HIV Prevention: What We Know and What We 
Need to Know, a Call to Action. Journal of Acquired Immune Deficiency Syndromes (1999). 2016;72(Suppl 3):S207-S9. 
doi: 10.1097/QAI.0000000000001086. PubMed PMID: PMC4969053. 
11. 
Pitasi MA, Oraka E, Clark H, Town M, DiNenno EA. HIV Testing Among Transgender Women and Men - 27 States 
and Guam, 2014-2015. MMWR Morbidity and mortality weekly report. 2017;66(33):883-7. Epub 2017/08/25. doi: 
10.15585/mmwr.mm6633a3. PubMed PMID: 28837547. 
12. 
Centers for Disease Control and Prevention. Highlights of CDC activites addressign HIV prevention among African 
American, gay, bisexual, and other men who have sex with men. 2011. 
13. 
Resnick MD, Bearman PS, Blum RW, Bauman KE, Harris KM, Jones J, Tabor J, Beuhring T, Sieving RE, Shew M, 
Ireland M, Bearinger LH, Udry JR. Protecting adolescents from harm: findings from the National Longitudinal Study on 
Adolescent Health. Journal of the American Medical Association. 1997;278(20):823-32. 
14. 
Garafolo R, Wolf RC, Kessel S, Palfrey J, DuRant RH. The association between health risk behaviors and sexual 
orientation among a school-based sample of adolescents. Pediatrics. 1998;101(895-902). 
15. 
Coalition JtF. Just the facts about sexual orientation and youth: a primer for principals, educators, and school 
personnel. Washington, D.C.: American Psychological Association, 2008. 
16. 
Centers for Disease Control and Prevention. HIV/STD risks in young men who have sex with men who do not 
disclose their sexual orientation—six US cities, 1994–2000. Morbidity and Mortality Weekly Report. 2003;52:81-5. 
17. 
Blake S, Ledsky R, Lehman T, Goodenow C, Sawyer R, Hack T. Preventing sexual risk behaviors among gay, 
lesbian, and bisexual adolescents: The benefits of gay-sensitive HIV instruction in schools. American Journal of Public 
Health 2001;91(6):940-6. 
18. 
Centers for Disease Control and Prevention. Prevalence and awareness of HIV Infection among men who have 
sex with men—21 cities, United States, 2008. Morbidity and Mortality Weekly Report 2010;59(37):1201-7. 
19. 
Sullivan P, Jones J, Kishore N, Stephenson R. The Roles of Technology in Primary HIV Prevention for Men Who 
Have Sex with Men. Curr HIV/AIDS Rep. 2015;12(4):481-8. doi: 10.1007/s11904-015-0293-5. 
20. 
Kirk GD, Himelhoch SS, Westergaard RP, Beckwith CG. Using Mobile Health Technology to Improve HIV Care for 
Persons Living with HIV and Substance Abuse. AIDS research and treatment. 2013;2013. 
21. 
Center PR. Internet Project Survey. 2014 January 9-12, 2014. Report No. 
22. 
Catalani C, Philbrick W, Fraser H, Mechael P, Israelski DM. mHealth for HIV Treatment & Prevention: A 
Systematic Review of the Literature. The open AIDS journal. 2013;7:17-41. Epub 2013/10/18. doi: 
10.2174/1874613620130812003. PubMed PMID: 24133558; PMCID: Pmc3795408. 

03-20-24 
30 
 
 
23. 
Devi BR, Syed-Abdul S, Kumar A, Iqbal U, Nguyen PA, Li YC, Jian WS. mHealth: An updated systematic review 
with a focus on HIV/AIDS and tuberculosis long term management using mobile phones. Computer methods and 
programs in biomedicine. 2015;122(2):257-65. Epub 2015/08/26. doi: 10.1016/j.cmpb.2015.08.003. PubMed PMID: 
26304621. 
24. 
Schnall R, Higgins T, Brown W, Carballo-Dieguez A, Bakken S. Trust, Perceived Risk, Perceived Ease of Use and 
Perceived Usefulness as Factors Related to mHealth Technology Use. Studies in Health Technology and Informatics. 
2015;216:467-71. doi: 10.3233/978-1-61499-564-7-467; PMCID: PMC5588863. 
25. 
Millett GA, Peterson JL, Flores SA, Hart TA, Jeffries WL, Wilson PA, Rourke SB, Heilig CM, Elford J, Fenton KA. 
Comparisons of disparities and risks of HIV infection in black and other men who have sex with men in Canada, UK, and 
USA: a meta-analysis. The Lancet. 2012;380(9839):341-8. 
26. 
Ayala G, Bingham T, Kim J, Wheeler DP, Millett GA. Modeling the impact of social discrimination and financial 
hardship on the sexual risk of HIV among Latino and Black men who have sex with men. Am J Public Health. 2012;102 
Suppl 2:S242-9. Epub 2012/03/10. doi: 10.2105/ajph.2011.300641. PubMed PMID: 22401516; PMCID: PMC3477911. 
27. 
Brito AM, Kendall C, Kerr L, Mota RM, Guimaraes MD, Dourado I, Pinho AA, Benzaken AS, Brignol S, Reingold AL. 
Factors Associated with Low Levels of HIV Testing among Men Who Have Sex with Men (MSM) in Brazil. PLoS One. 
2015;10(6):e0130445. Epub 2015/06/23. doi: 10.1371/journal.pone.0130445. PubMed PMID: 26098559; PMCID: 
PMC4476690. 
28. 
Malebranche DJ, Peterson JL, Fullilove RE, Stackhouse RW. Race and sexual identity: perceptions about medical 
culture and healthcare among Black men who have sex with men. Journal of the National Medical Association. 
2004;96(1):97. 
29. 
Levy ME, Wilton L, Phillips G, Glick SN, Kuo I, Brewer RA, Elliott A, Watson C, Magnus M. Understanding 
Structural Barriers to Accessing HIV Testing and Prevention Services Among Black Men Who Have Sex with Men (BMSM) 
in the United States. AIDS and behavior. 2014;18(5):972-96. doi: 10.1007/s10461-014-0719-x. PubMed PMID: 
PMC4509742. 
30. 
Solorio R, Forehand M, Simoni J. Attitudes towards and beliefs about HIV testing among Latino immigrant MSM: 
a comparison of testers and nontesters. AIDS research and treatment. 2013;2013. 
31. 
Kinsler JJ, Lee S-J, Sayles JN, Newman PA, Diamant A, Cunningham W. The impact of acculturation on utilization 
of HIV prevention services and access to care among an at-risk Hispanic population. Journal of Health Care for the Poor 
and Underserved. 2009;20(4):996. 
32. 
Spadafino JT, Martinez O, Levine EC, Dodge B, Muñoz-Laboy M, Fernandez MI. Correlates of HIV and STI testing 
among Latino men who have sex with men in New York City. AIDS care. 2016;28(6):695-8. 
33. 
Joseph HA, Belcher L, O’Donnell L, Fernandez MI, Spikes PS, Flores SA. HIV testing among sexually active 
Hispanic/Latino MSM in Miami-Dade County and New York City opportunities for increasing acceptance and frequency 
of testing. Health promotion practice. 2014;15(6):867-80. 
34. 
Oster AM, Russell K, Wiegand RE, Valverde E, Forrest DW, Cribbin M, Le BC, Paz-Bailey G. HIV infection and 
testing among Latino men who have sex with men in the United States: the role of location of birth and other social 
determinants. PloS one. 2013;8(9):e73779. 
35. 
Brown W, Carballo-Diéguez A, John RM, Schnall R. Information, Motivation, and Behavioral Skills of High-risk 
Young Adults to Use the HIV Self-Test. AIDS and behavior. 2016;20(9):2000-9. doi: 10.1007/s10461-016-1309-x. PubMed 
PMID: PMC4988939. 
36. 
Aliabadi N, Carballo-Dieguez A, Bakken S, Rojas M, Brown W, Carry M, Mosley JP, Gelaude D, Schnall R. Using the 
Information-Motivation-Behavioral Skills Model to Guide the Development of an HIV Prevention Smartphone 
Application for High-Risk MSM. AIDS education and prevention : official publication of the International Society for AIDS 
Education. 2015;27(6):522-37. doi: 10.1521/aeap.2015.27.6.522. PubMed PMID: PMC4868035. 
37. 
Scherr TF, Gupta S, Wright DW, Haselton FR. Mobile phone imaging and cloud-based analysis for standardized 
malaria detection and reporting2016;6:28645. doi: 10.1038/srep28645 
 
https://www.nature.com/articles/srep28645#supplementary-information. 
38. 
Schnall R, Pichon A, Gradilla M, Scherr TF, editors. Feasibility and Acceptabilty of the mLab App for Improving 
Uptake of HIV Testing in High-Risk Youth. AMIA Annual Symposium; 2018; San Franscisco, CA. 
39. 
Mandell BF. Home testing: The metamorphosis of attitudes about HIV infection. Cleveland Clinic Journal of 
Medicine. 2012;79(10):669, 74. doi: 10.3949/ccjm.79b.12010. 

03-20-24 
31 
 
 
40. 
Schnall R, Rojas M, Travers J, Brown W, Bakken S, editors. Use of Design Science for Informing the Development 
of a Mobile App for Persons Living with HIV. AMIA Annual Symposium; 2014; Washington, DC. 
41. 
Skarbinski J, Rosenberg E, Paz-Bailey G, Hall HI, Rose CE, Viall AH, Fagan JL, Lansky A, Mermin JH. Human 
immunodeficiency virus transmission at each step of the care continuum in the United States. JAMA Internal Medicine. 
2015;175(4):588-96. doi: 10.1001/jamainternmed.2014.8180. PubMed PMID: 25706928. 
42. 
Campsmith ML, Rhodes PH, Hall HI, Green TA. Undiagnosed HIV prevalence among adults and adolescents in the 
United States at the end of 2006. Journal of Acquired Immune Deficiency Syndrome. 2010;53(5):619-24. doi: 
10.1097/QAI.0b013e3181bf1c45. PubMed PMID: 19838124. 
43. 
Zanoni BC, Mayer KH. The adolescent and young adult HIV cascade of care in the United States: exaggerated 
health disparities. AIDS Patient Care and STDS. 2014;28(3):128-35. doi: 10.1089/apc.2013.0345. PubMed PMID: 
24601734; PMCID: PMC3948479. 
44. 
Hall HI, Holtgrave DR, Maulsby C. HIV transmission rates from persons living with HIV who are aware and 
unaware of their infection. AIDS. 2012;26(7):893-6. doi: 10.1097/QAD.0b013e328351f73f. PubMed PMID: 22313960. 
45. 
Agugo PC. HIV & AIDS: Protection form infection, related illnesses and AIDS: lulu.com; 2013. 
46. 
Centers for Disease Control and Prevention. CDC Analysis Provides New Look at Disproportionate Impact of HIV 
and Syphilis Among U.S. Gay and Bisexual Men. Atlanta, GA: 2010. 
47. 
Reisner SL, Jadwin-Cakmak L, White Hughto JM, Martinez M, Salomon L, Harper GW. Characterizing the HIV 
Prevention and Care Continua in a Sample of Transgender Youth in the U.S. AIDS and Behavior. 2017;21(12):3312-27. 
doi: 10.1007/s10461-017-1938-8. 
48. 
Center for Disease Control and Prevention. HIV Among Hispanics/Latinos Atlanta, GA2017. Available from: 
https://www.cdc.gov/hiv/group/racialethnic/hispaniclatinos/index.html. 
49. 
Vitinghoff E, Douglas J, Judon F, McKiman D, MacQueen K, Buchinder SP. Per-contact risk of human 
immunodificiency virus tramnsmision between male sexual partners. American journal of epidemiology. 
1999;150(3):306-11. 
50. 
Operario D, Soma T, Underhill K. Sex Work and HIV Status Among Transgender Women: Systematic Review and 
Meta-Analysis. JAIDS Journal of Acquired Immune Deficiency Syndromes. 2008;48(1):97-103. doi: 
10.1097/QAI.0b013e31816e3971. PubMed PMID: 00126334-200805010-00013. 
51. 
Clements-Nolle K, Marx R, Guzman R, Katz M. HIV prevalence, risk behaviors, health care use, and mental health 
status of transgender persons: implications for public health intervention. American Journal of Public Health. 
2001;91(6):915-21. PubMed PMID: PMC1446468. 
52. 
Hotton AL, Garofalo R, Kuhns LM, Johnson AK. Substance Use as a Mediator of the Relationship Between Life 
Stress and Sexual Risk Among Young Transgender Women. AIDS Education and Prevention. 2013;25(1):62-71. doi: 
10.1521/aeap.2013.25.1.62. 
53. 
Brennan J, Kuhns LM, Johnson AK, Belzer M, Wilson EC, Garofalo R, the Adolescent Medicine Trials Network for 
HIVAI. Syndemic Theory and HIV-Related Risk Among Young Transgender Women: The Role of Multiple, Co-Occurring 
Health Problems and Social Marginalization. American Journal of Public Health. 2012;102(9):1751-7. doi: 
10.2105/AJPH.2011.300433. PubMed PMID: PMC3416048. 
54. 
Center for Disease Control and Prevention. HIV and AIDS among gay and bisexual men. 2013. 
55. 
Dorell CG, Sutton MY, Oster AM, Hardnett F, Thomas PE, Gaul ZJ, Mena LA, Heffelfinger JD. Missed opportunities 
for HIV testing in health care settings among young African American men who have sex with men: implications for the 
HIV epidemic. AIDS Patient Care STDS. 2011;25(11):657-64. Epub 2011/09/20. doi: 10.1089/apc.2011.0203. PubMed 
PMID: 21923415. 
56. 
Pharr JR, Lough NL, Ezeanolue EE. Barriers to HIV Testing Among Young Men Who Have Sex With Men (MSM): 
Experiences from Clark County, Nevada. Global journal of health science. 2015;8(7):9-17. Epub 2015/01/01. doi: 
10.5539/gjhs.v8n7p9. PubMed PMID: 26925893; PMCID: PMC4965686. 
57. 
Veinot TC, Caldwell E, Loveluck J, Arnold MP, Bauermeister J. HIV Testing Behavior and Social Network 
Characteristics and Functions Among Young Men Who have Sex with Men (YMSM) in Metropolitan Detroit. AIDS and 
behavior. 2016;20(11):2739-61. doi: 10.1007/s10461-016-1296-y. PubMed PMID: PMC5234266. 
58. 
Bhoobun S, Jetty A, Koroma MA, Kamara MJ, Kabia M, Coulson R, Ansumana R, Jacobsen KH. Facilitators and 
Barriers Related to Voluntary Counseling and Testing for HIV Among Young Adults in Bo, Sierra Leone. J Community 
Health. 2014;39(3):514-20. doi: 10.1007/s10900-013-9788-4. PubMed PMID: WOS:000335392600015. 

03-20-24 
32 
 
 
59. 
Phillips G, Ybarra ML, Prescott TL, Parsons JT, Mustanski B. Low Rates of HIV Testing among Adolescent Gay, 
Bisexual, and Queer. The Journal of adolescent health : official publication of the Society for Adolescent Medicine. 
2015;57(4):407-12. doi: 10.1016/j.jadohealth.2015.06.014. PubMed PMID: PMC4583790. 
60. 
White House Office of National AIDS Policy. National HIV/AIDS Strategy for The United States: Updated to 2020. 
2015. 
61. 
Kurth AE, Lally MA, Choko AT, Inwani IW, Fortenberry JD. HIV testing and linkage to services for youth. Journal of 
the International AIDS Society. 2015;18(2Suppl 1):19433. doi: 10.7448/IAS.18.2.19433. PubMed PMID: PMC4344538. 
62. 
Center for Disease Control and Prevention. HIV testing and risk behaviors among gay, bisexual, and other men 
who have sex with men: United States. Morbidity and Mortality Weekly Report. 2013;62(47):958-62. 
63. 
Centers for Disease Control and Prevention. HIV surveillance in men who have sex with men (MSM). 2015. 
64. 
HIV/AIDS JUNPo. Report on the Global AIDS Epidemic. Geneva, Switzerland: UNAIDS, 2010. 
65. 
Centers for Disease Control and Prevention. HIV and AIDS among gay and bisexual men. 2013. 
66. 
Poteat T, Reisner SL, Radix A. HIV epidemics among transgender women. Current Opinion in HIV and AIDS. 
2014;9(2):168-73. doi: 10.1097/coh.0000000000000030. PubMed PMID: 01222929-201403000-00010. 
67. 
Schulden JD, Song B, Barros A, Mares-DelGrasso A, Martin CW, Ramirez R, Smith LC, Wheeler DP, Oster AM, 
Sullivan PS, Heffelfinger JD. Rapid HIV Testing in Transgender Communities by Community-Based Organizations in Three 
Cities. Public Health Reports. 2008;123(Suppl 3):101-14. PubMed PMID: PMC2567010. 
68. 
Herbst JH, Jacobs ED, Finlayson TJ, McKleroy VS, Neumann MS, Crepaz N. Estimating HIV prevalence and risk 
behaviors of transgender persons in the United States: a systematic review. AIDS Behav. 2008;12(1):1-17. Epub 
2007/08/19. doi: 10.1007/s10461-007-9299-3. PubMed PMID: 17694429. 
69. 
Koblin B, Mayer K, Eshleman S, Wang L, Mannheimer S, et al. Correlates of HIV acquisition in a cohort of black 
men who have sex with men in the United States: HIV prevention trials network (HPTN) 061. PLoS ONE. 2013; 
8(7)(7):e70413 
70. 
Center for Disease Control and Prevention (CDC). Sexually Transmitted Diseases Treatment Guidelines 2010. 
Available from: http://origin.glb.cdc.gov/mmWR/preview/mmwrhtml/rr5912a1.htm. 
71. 
D’Angelo LJ. When will routine testing for human immunodeficiency virus infection be the routine for 
adolescents? Archives of Pediatrics & Adolescent Medicine. 2012;166(4):385-6. doi: 10.1001/archpediatrics.2011.1555. 
72. 
Centers for Disease Control and Prevention. HIV Surveillance Report. 2014. 
73. 
Centers for Disease Control and Prevention. HIV testing and risk behaviors among gay, bisexual, and other men 
who have sex with men-United States. MMWR Morbidity and mortality weekly report. 2013;62(47):958. 
74. 
Stevens DR, Vrana CJ, Dlin RE, Korte JE. A Global Review of HIV Self-testing: Themes and Implications. AIDS and 
Behavior. 2017:1-16. doi: 10.1007/s10461-017-1707-8. 
75. 
Jamil MS, Prestage G, Fairley CK, Grulich AE, Smith KS, Chen M, Holt M, McNulty AM, Bavinton BR, Conway DP, 
Wand H, Keen P, Bradley J, Kolstee J, Batrouney C, Russell D, Law M, Kaldor JM, Guy RJ. Effect of availability of HIV self- 
testing on HIV testing frequency in gay and bisexual men at high risk of infection (FORTH): a waiting-list randomised 
controlled trial. The Lancet HIV. doi: https://doi.org/10.1016/S2352-3018(17)30023-1. 
76. 
Martinez O, Carballo-Diéguez A, Ibitoye M, Frasca T, Brown W, Balan I. Anticipated and Actual Reactions to 
Receiving HIV Positive Results Through Self-Testing Among Gay and Bisexual Men. AIDS and Behavior. 2014;18(12):2485- 
95. doi: 10.1007/s10461-014-0790-3. 
77. 
Eaton L, Driffin D, Smith H, Conway-Washington C, White D, Cherry C. Black men who have sex with men, sexual 
risk-taking, and willingness to use rapid home HIV tests. Prevention Science. 2015;16:321-9. 
78. 
Schnall R, John RM, Carballo-Dieguez A. Do High-Risk Young Adults Use the HIV Self-Test Appropriately? 
Observations from a Think-Aloud Study. AIDS Behav. 2016;20(4):939-48. doi: 10.1007/s10461-015-1240-6. PubMed 
PMID: 26518679; PMCID: PMC4799728. 
79. 
CTIA-The Wireless Association. Wirless Quick Facts 2014. Available from: http://www.ctia.org/your-wireless- 
life/how-wireless-works/wireless-quick-facts. 
80. 
Cole-Lewis H, Kershaw T. Text Messaging as a Tool for Behavior Change in Disease Prevention and Management. 
Epidemiologic Reviews. 2010;32(1):56-69. doi: 10.1093/epirev/mxq004. 
81. 
Guse K, Levine D, Martins S, Lira A, Gaarde J, Westmorland W, Gilliam M. Interventions using new digital media 
to improve adolescent sexual health: a systematic review. The Journal of adolescent health : official publication of the 
Society for Adolescent Medicine. 2012;51(6):535-43. doi: 10.1016/j.jadohealth.2012.03.014. PubMed PMID: 23174462. 
82. 
Aliabadi N, Carballo-Dieguez A, Bakken S, Rojas M, Brown W, 3rd, Carry M, Mosley JP, Gelaude D, Schnall R. 
Using the Information-Motivation-Behavioral Skills Model to Guide the Development of an HIV Prevention Smartphone 

03-20-24 
33 
 
 
Application for High-Risk MSM. AIDS Educ Prev. 2015;27(6):522-37. Epub 2015/11/26. doi: 10.1521/aeap.2015.27.6.522. 
PubMed PMID: 26595265; PMCID: PMC4868035. 
83. 
Paltiel AD, Walensky RP. Home HIV testing: good news but not a game changer. Ann Intern Med. 
2012;157(10):744-6. Epub 2012/10/10. doi: 1377315 [pii] 
10.7326/0003-4819-157-10-201211200-00545. PubMed PMID: 23044643. 
84. 
Schnall R, John R, Carballo-Dieguez A. Do High-Risk Young Adults Use the HIV Self-Test Appropriately? 
Observations from a Think-Aloud Study. AIDS and Behavior. 2015:1-10. doi: 10.1007/s10461-015-1240-6; PMCID: 
PMC4799728 
85. 
Bauermeister JA, Golinkoff JM, Muessig KE, Horvath KJ, Hightow-Weidman LB. Addressing engagement in 
technology-based behavioural HIV interventions through paradata metrics. Curr Opin HIV AIDS. 2017;12(5):442-6. Epub 
2017/06/16. doi: 10.1097/coh.0000000000000396. PubMed PMID: 28617711; PMCID: PMC5637536. 
86. 
Hightow-Weidman LB, Muessig KE, Bauermeister J, Zhang C, LeGrand S. Youth, Technology, and HIV: Recent 
Advances and Future Directions. Curr HIV/AIDS Rep. 2015;12(4):500-15. Epub 2015/09/20. doi: 10.1007/s11904-015- 
0280-x. PubMed PMID: 26385582; PMCID: PMC4643403. 
87. 
Baltierra NB, Muessig KE, Pike EC, LeGrand S, Bull SS, Hightow-Weidman LB. More than just tracking time: 
complex measures of user engagement with an internet-based health promotion intervention. Journal of biomedical 
informatics. 2016;59:299-307. doi: 10.1016/j.jbi.2015.12.015. PubMed PMID: PMC4792652. 
88. 
Schnall R, Cho H, Mangone A, Pichon A, Jia H. Mobile Health Technology for Improving Symptom Management in 
Low Income Persons Living with HIV. AIDS and behavior. 2018. doi: 10.1007/s10461-017-2014-0. 
89. 
Brown W, 3rd, Yen PY, Rojas M, Schnall R. Assessment of the Health IT Usability Evaluation Model (Health- 
ITUEM) for evaluating mobile health (mHealth) technology. J Biomed Inform. 2013;46(6):1080-7. doi: 
10.1016/j.jbi.2013.08.001. PubMed PMID: 23973872; PMCID: PMC3844064. 
90. 
Yen P-Y, Bakken S. Review of health information technology usability study methodologies. Journal of the 
American Medical Informatics Association. 2011;19(3):413-22. 
91. 
Davis FD, Bagozzi RP, Warshaw PR. User acceptance of computer technology: a comparison of two theoretical 
models. Management science. 1989;35(8):982-1003. 
92. 
Standard I. Ergonomic requirements for office work with visual display terminals (vdts)–part 11: Guidance on 
usability. ISO Standard 9241-11: 1998. International Organization for Standardization. 1998. 
93. 
Davis F. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS 
Quarterly. 1989;13(3):319-40. 
94. 
Schnall R, Rojas M, Bakken S, Brown W, Carballo-Dieguez A, Carry M, Gelaude D, Mosley JP, Travers J. A user- 
centered model for designing consumer mobile health (mHealth) applications (apps). Journal of Biomedical Informatics. 
2016;60:243-51. doi: http://dx.doi.org/10.1016/j.jbi.2016.02.002. 
95. 
Bertini E, Gabrielli S, Kimani S, editors. Appropriating and assessing heuristics for mobile computing. Advanced 
Visual Interfaces International Working Conference; 2006; Venezia, Italy. 
96. 
TechSmith. Morae Usability and Web Site Testing: TechSmith Corporation; 1995. Available from: 
http://www.techsmith.com/products/morae/default.asp. 
97. 
Nielson J. Ten Usability Heuristics 2005 [updated 2005]. Available from: 
http://www.useit.com/papers/heuristic/heuristic_list.html. 
98. 
Bright T, Bakken S, Johnson S. Heuristic evaluation of eNote: an electronic notes system. New York2006. 
99. 
Nielsen J. Heuristic evaluation. In: Nielsen J, Mack RL, editors. Usability Inspection Methods. New York, NY: John 
Wiley & Sons; 1994. 
100. 
Schnall R, Liu N, Sperling J, Green R, Clark S, Vawdrey D. An electronic alert for HIV screening in the emergency 
department increases screening but not the diagnosis of HIV. Appl Clin Inform. 2014;5(1):299-312. doi: 10.4338/ACI- 
2013-09-RA-0075. PubMed PMID: 24734140; PMCID: PMC3974262. 
101. 
Sheehan B, Lee Y, Rodriguez M, Tiase V, Schnall R. A comparison of usability factors of four mobile devices for 
accessing healthcare information by adolescents. Appl Clin Inform. 2012;3(4):356-66. doi: 10.4338/ACI-2012-06-RA- 
0021. PubMed PMID: 23227134; PMCID: PMC3517216. 
102. 
Schnall R, Cimino JJ, Bakken S. Development of a prototype continuity of care record with context-specific links 
to meet the information needs of case managers for persons living with HIV. Int J Med Inform. 2012;81(8):549-55. doi: 
10.1016/j.ijmedinf.2012.05.002. PubMed PMID: 22632821; PMCID: PMC3515778. 

03-20-24 
34 
 
 
103. 
Schnall R, Cho H, Liu J. Validation of the Health Information Technology Usability Evaluation Scale (Health-ITUES) 
for Usability Assessment of Mobile Health Technology. JMIR Mhealth Uhealth. 2018;6(1):e4. 
104. 
Schnall R, Mosley JP, Iribarren SJ, Bakken S, Carballo-Dieguez A, Brown Iii W. Comparison of a User-Centered 
Design, Self-Management App to Existing mHealth Apps for Persons Living With HIV. JMIR Mhealth Uhealth. 
2015;3(3):e91. doi: 10.2196/mhealth.4882. PubMed PMID: 26385783; PMCID: PMC4704937. 
105. 
Salz T, McCabe MS, Oeffinger KC, Corcoran S, Vickers AJ, Salner AL, Dornelas E, Schnall R, Raghunathan NJ, 
Fortier E, Baxi SS. A head and neck cancer intervention for use in survivorship clinics: a protocol for a feasibility study. 
Pilot Feasibility Stud. 2016;2:23. doi: 10.1186/s40814-016-0061-3. PubMed PMID: 27965842; PMCID: PMC5153865. 
106. 
Schnall R, Bakken S, Brown Iii W, Carballo-Dieguez A, Iribarren S. Usabilty Evaluation of a Prototype Mobile App 
for Health Management for Persons Living with HIV. Stud Health Technol Inform. 2016;225:481-5. PubMed PMID: 
27332247. 
107. 
Center for Disease Control. Pre-exposure Prophylaxis (PrEP) for HIV Prevention Atlanta, GA2014 [cited 2017 
December 21]. Available from: https://www.cdc.gov/hiv/pdf/PrEP_fact_sheet_final.pdf. 
108. 
Finer LB, Philbin JM. Trends in Ages at Key Reproductive Transitions in the United States, 1951&#x2013;2010. 
Women's Health Issues.24(3):e271-e9. doi: 10.1016/j.whi.2014.02.002. 
109. 
Guttmacher Institute. American Teens’ Sexual and Reproductive Health 2017 [cited 2017 May 21]. Available 
from: https://www.guttmacher.org/fact-sheet/american-teens-sexual-and-reproductive-health. 
110. 
Mutua G, Sanders E, Mugo P, Anzala O, Haberer JE, Bangsberg D, Barin B, Rooney JF, Mark D, Chetty P, Fast P, 
Priddy FH. Safety and Adherence to Intermittent Pre-Exposure Prophylaxis (PrEP) for HIV-1 in African Men Who Have Sex 
with Men and Female Sex Workers. PLoS ONE. 2012;7(4):e33103. doi: 10.1371/journal.pone.0033103. 
111. 
van der Straten A, Stadler J, Luecke E, Laborde N, Hartmann M, Montgomery ET, on behalf of the V-CST. 
Perspectives on use of oral and vaginal antiretrovirals for HIV prevention: the VOICE-C qualitative study in Johannesburg, 
South Africa. Journal of the International AIDS Society. 2014;17(3Suppl 2):19146. doi: 10.7448/IAS.17.3.19146. PubMed 
PMID: PMC4163995. 
112. 
van der Straten A, Stadler J, Montgomery E, Hartmann M, Magazi B, Mathebula F, Schwartz K, Laborde N, Soto- 
Torres L. Women’s Experiences with Oral and Vaginal Pre-Exposure Prophylaxis: The VOICE-C Qualitative Study in 
Johannesburg, South Africa. PloS one. 2014;9(2):e89118. doi: 10.1371/journal.pone.0089118. 
113. 
Millett GA, Peterson JL, Flores SA, Hart TA, Jeffries WLt, Wilson PA, Rourke SB, Heilig CM, Elford J, Fenton KA, 
Remis RS. Comparisons of disparities and risks of HIV infection in black and other men who have sex with men in 
Canada, UK, and USA: a meta-analysis. Lancet (London, England). 2012;380(9839):341-8. Epub 2012/07/24. doi: 
10.1016/s0140-6736(12)60899-x. PubMed PMID: 22819656. 
114. 
Golub SA, Gamarel KE. The impact of anticipated HIV stigma on delays in HIV testing behaviors: findings from a 
community-based sample of men who have sex with men and transgender women in New York City. AIDS patient care 
and STDs. 2013;27(11):621-7. 
115. 
Wei C, Herrick A, Raymond HF, Anglemyer A, Gerbase A, Noar SM. Social marketing interventions to increase 
HIV/STI testing uptake among men who have sex with men and male-to-female transgender women. Cochrane Database 
of Systematic Reviews. 2011(9). doi: 10.1002/14651858.CD009337. PubMed PMID: CD009337. 
116. 
United States Census Bureau. Quick Facts: Seattle (city), New York 2015. Available from: 
http://www.census.gov/quickfacts/table/PST045214/3651000,00. 
117. 
Faulkner L. Beyond the five-user assumption: benefits of increased sample sizes in usability testing. Behav Res 
Methods Instrum Comput. 2003;35(3):379-83. Epub 2003/11/01. PubMed PMID: 14587545. 
118. 
Hickson DA, Mena LA, Wilton L, Tieu H-V, Koblin BA, Cummings V, Latkin C, Mayer KH. Sexual Networks, Dyadic 
Characteristics, and HIV Acquisition and Transmission Behaviors Among Black Men Who Have Sex With Men in 6 US 
Cities. American Journal of Epidemiology. 2017;185(9):786-800. doi: 10.1093/aje/kww144. 
119. 
Salganik MJ, Heckathorn DD. Sampling and Estimation in Hidden Populations Using Respondent-Driven 
Sampling. Sociological Methodology. 2004;34:193-239. doi: 10.2307/3649374. 
120. 
Gile KJ, Handcock MS. RESPONDENT-DRIVEN SAMPLING: AN ASSESSMENT OF CURRENT METHODOLOGY. 
Sociological Methodology. 2010;40(1):285-327. doi: 10.1111/j.1467-9531.2010.01223.x. 
121. 
Kuhns LM, Kwon S, Ryan DT, Garofalo R, Phillips G, 2nd, Mustanski BS. Evaluation of respondent-driven sampling 
in a study of urban young men who have sex with men. J Urban Health. 2015;92(1):151-67. doi: 10.1007/s11524-014- 
9897-0. PubMed PMID: 25128301; PMCID: PMC4338125. 
122. 
Ericsson KA, Simon HA. Verbal Reports as Data. Psychological Review. 1980;87(3):215-51. PubMed PMID: 
EJ231273. 

03-20-24 
35 
 
 
123. 
van Someren MW, Barnard YF, Sandberg JAC. The Think Aloud Method: A practical guide to modelling cognitive 
processes. London: Academic Press; 1994. 
124. 
Ward L, Traweek D. Application of a metacognitive strategy to assessment, intervention, and consultation: A 
think-aloud technique. Journal of School Psychology. 1993;31(4):469-85. doi: http://dx.doi.org/10.1016/0022- 
4405(93)90032-E. 
125. 
Yen PY, Wantland D, Bakken S. Development of a Customizable Health IT Usability Evaluation Scale. AMIA Annu 
Symp Proc. 2010;2010:917-21. PubMed PMID: 21347112; PMCID: 3041285. 
126. 
Schnall R, Cho H, Liu J. Validation of the Health Information Technology Usability Evaluation Scale (Health-ITUES) 
for Usability Assessment of Mobile Health Technology. JMIR mHealth and uHealth. In Press. 
127. 
Center for Disease Control and Prevention (CDC). HIV Testing 101 2017 [cited 2017 December 22]. Available 
from: https://www.cdc.gov/hiv/pdf/library/factsheets/hiv-testing-101-info-sheet.pdf. 
128. 
United States Census Bureau. U.S. Census Bureau: State and County QuickFacts. 2010. Available from: 
http://quickfacts.census.gov/qfd/states/17/1714000.html?cssp=SERP. 
129. 
DiNenno EA. Recommendations for HIV screening of gay, bisexual, and other men who have sex with men— 
United States, 2017. MMWR Morbidity and mortality weekly report. 2017;66. 
130. 
Center for Disease Control and Prevention (CDC). HIV/AIDS Testing -- Should I get tested for HIV? 2017 [cited 
2017 December 22]. Available from: https://www.cdc.gov/hiv/basics/testing.html#get_tested. 
131. 
Baker DW, Williams MV, Parker RM, Gazmararian JA, Nurss J. Development of a brief test to measure functional 
health literacy. Patient education and counseling. 1999;38(1):33-42. 
132. 
National Institutes of Drug Abuse. NIDA-Modified Assist 2015. Available from: 
file:///C:/Users/rb897/Desktop/APA_DSM5_Level-2-Substance-Use-Child-Age-11-to-17.pdf. 
133. 
Wilton J, Kain T, Fowler S, Hart TA, Grennan T, Maxwell J, Tan DHS. Use of an HIV-risk screening tool to identify 
optimal candidates for PrEP scale-up among men who have sex with men in Toronto, Canada: disconnect between 
objective and subjective HIV risk. Journal of the International AIDS Society. 2016;19(1):20777. doi: 
10.7448/IAS.19.1.20777. PubMed PMID: PMC4911732. 
134. 
Kernan WN, Viscoli CM, Makuch RW, Brass LM, Horwitz RI. Stratified Randomization for Clinical Trials. Journal of 
Clinical Epidemiology. 1999;52(1):19-26. doi: https://doi.org/10.1016/S0895-4356(98)00138-3. 
135. 
Smith DK, Pals SL, Herbst JH, Shinde S, Carey JW. Development of a Clinical Screening Index Predictive of Incident 
HIV Infection Among Men Who Have Sex With Men in the United States. JAIDS Journal of Acquired Immune Deficiency 
Syndromes. 2012;60(4). 
136. 
Altman DG, Bland JM. Treatment allocation in controlled trials: why randomise? BMJ. 1999;318(7192):1209-. 
137. 
Hedden Sl, Woolsen RF, Malcolm RJ. Randomization in substance abuse clinical trials. Substance Abuse 
Treatment, Prevention, and Policy. 2006;1:6. 
138. 
Bourne C, Knight V, Guy R, Wand H, Lu H, McNulty A. Short message service reminder intervention doubles 
sexually transmitted infection/HIV re-testing rates among men who have sex with men. Sexually Transmitted Infections. 
2011;87(3):229. 
139. 
Mansergh G, Koblin BA, McKirnan DJ, Hudson SM, Flores SA, Wiegand RE, Purcell DW, Colfax GN, for the Project 
MIXST. An Intervention to Reduce HIV Risk Behavior of Substance-Using Men Who Have Sex with Men: A Two-Group 
Randomized Trial with a Nonrandomized Third Group. PLoS Med. 2010;7(8):e1000329. doi: 
10.1371/journal.pmed.1000329. 
140. 
Lyles CM, Crepaz N, Herbst JH, Kay LS. Evidence–Based HIV Behavioral Prevention From the Perspective of the 
CDC's HIV/AIDS Prevention Research Synthesis Team. AIDS Education and Prevention. 2006;18(supp):21-31. doi: 
10.1521/aeap.2006.18.supp.21. 
141. 
Little RJ, Rubin DB. Statistical Analysis with Missing Data. New York, NY John Wiley & Sons; 2002. 
142. 
Diggle P, Kenward MG. Informative Drop-Out in Longitudinal Data Analysis. Journal of the Royal Statistical 
Society Series C (Applied Statistics). 1994;43(1):49-93. doi: 10.2307/2986113. 
143. 
Liu X, Waternaux C, Petkova E. Influence of human immunodeficiency virus infection on neurological 
impairment: an analysis of longitudinal binary data with informative drop-out. Journal of the Royal Statistical Society: 
Series C (Applied Statistics). 1999;48(1):103-15. doi: 10.1111/1467-9876.00143. 
144. 
Carpenter J, Pocock S, Lamm CJ. Coping with missing data in clinical trials: a model-based approach applied to 
asthma trials. Statistics in medicine. 2002;21(8):1043-66. Epub 2002/04/05. PubMed PMID: 11933033. 
145. 
Lachin JM. Statistical considerations in the intent-to-treat principle. Controlled clinical trials. 2000;21(3):167-89. 
Epub 2000/05/24. PubMed PMID: 10822117. 

03-20-24 
36 
 
 
146. 
Kreuter F. Improving surveys with paradata: Analytic uses of process information: John Wiley & Sons; 2013.