Yunnan-ADARC HIV Prevention Program: Developing and Testing a Model to Implement and Sustain PrEP Delivery in China
Phase: NA
Conditions: HIV/AIDS
Sponsor: Columbia University

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Yunnan-ADARC HIV Prevention Program: Developing and 
Testing a Model to Implement and Sustain PrEP Delivery in China 
 
NCT03992274 
 
 
Principal Investigator: Kathrine Meyers, DrPH, MPP, MSc 
 
 
Effective: 08/31/2023 (IRB Approved) 
 
 

Research Aims & Abstracts 
 
B-HAPPY is a multisite randomized stepped-wedge study to compare the 
effectiveness of two PrEP implementation strategy bundles: 1) Standard 
Implementation: comprising the strategies that the Yunnan CDC would usually use to 
introduce HIV prevention innovations in the absence of the proposed study; and 2) 
Enhanced Implementation: standard implementation enhanced by an approach guided 
by the Stages of Implementation Completion (SIC) framework to plan and implement 
PrEP. 
We hypothesize that adding SIC-guided implementation onto standard health 
system implementation of PrEP will improve outcomes and lead to greater likelihood of 
a) implementation success and b) sustainment. Successful implementation will be 
defined as a site achieving Five-80% targets and maintaining them three months in a 
row: 1) Awareness of PrEP among 80% of MSM clients; 2) PrEP eligibility assessment 
for 80% of MSM clients; 3) PrEP linkage for 80% of MSM clients who wish to start and 
are medically-eligible; 4) PrEP initiation for 80% of MSM who linked to PrEP services; 
and 5) six-month PrEP continuation for 80% of those who remain PrEP-eligible. 
Sustainment will be defined as achieving the Five-80s continuously over 6-months. 
To examine the implementation process and efficiency of the Enhanced versus 
Standard implementation approaches we will use SIC and COINS tools to monitor sites 
for implementation process and resource use. Outcomes will include proportion of 
implementation activities completed, duration of each implementation phase, and cost 
and resource allocation for completion of each implementation phase. Based on robust 
studies that assess implementation using the SIC, we hypothesize that time and 

resources spent in Pre-Implementation will be associated with higher odds of successful 
implementation and sustainment 
Scientific Abstract: Pre-exposure prophylaxis (PrEP), a key biomedical HIV 
prevention strategy, can significantly reduce HIV incidence in real-world settings. 
However, introduction and global scale-up of PrEP have been slow. Part of the 
challenge is that PrEP is typically introduced into existing health systems through a hit-
or-miss process that is rarely studied. We propose the Biomedical HIV/AIDS Prevention 
Program, Yunnan (B-HAPPY), an implementation science study of the process of 
introduction and implementation of PrEP in a specific international health system facing 
high rates of HIV incidence among men who have sex with men. 
B-HAPPY will introduce and evaluate the implementation of PrEP among over 
1000 men who have sex with men, in eight sites in Yunnan, China. The study is guided 
by the Stages of Implementation Completion Framework (SIC) and will use SIC-
associated assessment tools to measure progression across eight pre-defined stages of 
implementation, including documenting costs and required resources. 
The study has two specific aims: 1) To evaluate whether the addition of 
Enhanced implementation strategies to Standard health system PrEP delivery improves 
client-services outcomes; 2) To compare the time, costs, and resources required for 
Standard versus Enhanced PrEP implementation. Enhanced PrEP implementation will 
rely on a centralized technical assistance strategy to deliver PrEP-related knowledge, 
skills, and best practices to clinics delivering PrEP. The outcome of the study will be an 
implementation blueprint that may be used across China and in other low- and middle-
income countries to guide the introduction of PrEP into existing health systems. 

Lay Abstract: B-HAPPY is an implementation science research project to study 
the process by which preexposure prophylaxis (PrEP) is introduced and integrated into 
a specific international health system facing high rates of new HIV infections among 
men who have sex with men. The study aims to (1) evaluate whether the addition of 
Enhanced implementation strategies to Standard health system PrEP delivery improves 
client-services outcomes; (2) compare the time, costs, and resources required for 
Standard versus Enhanced PrEP implementation. Enhanced PrEP implementation will 
rely on a systems analysis and improvement approach guided by the Stages of 
Implementation Completion (SIC) framework. 
 
Study Purpose and Rationale 
 
Pre-exposure prophylaxis (PrEP) is a key biomedical HIV prevention strategy 
that can significantly reduce HIV incidence in real-world settings. Yet globally, PrEP 
introduction and scale up have been slow. PrEP is typically integrated into existing 
health systems through a hit-or-miss process that is rarely studied, leaving a gap in 
understanding of implementation processes, and how sociopolitical and cultural 
contexts influence implementation. A scientific study of the process of introduction and 
implementation of PrEP in a specific international health system has the potential to 
produce actionable knowledge, and more efficient and effective implementation that will 
both reduce HIV incidence and costs within the existing system and accelerate access 
to effective prevention in similar health systems. PrEP implementation in the US and 
globally has taught us that clinical guidance is necessary but not sufficient for large 
scale implementation. Effective implementation has multiple components. Health 
systems must 1) build awareness and generate demand; 2) identify potential 

beneficiaries; 3) offer and link people to services; 4) get pills to those who need them; 
and 5) support users to adhere both to pills and to regular check-ups. These five 
components comprise the PrEP Care Continuum (Kelly et al., 2015; Nunn et al., 2017). 
Studies are testing strategies for each of these components: for example, comparing 
methods for screening (Kelly et al., 2015), evaluating the impact of same-day initiation 
(Wendel, 2017) and testing different retention strategies (Amico & Stirratt, 2014; 
Holtzman et al, 2015; Fuchs et al., 2018). While knowledge generated in each of these 
studies is critical, not one is holistic enough to inform planning for large, state-run health 
systems seeking to integrate PrEP into services. What is needed are protocols to detail 
how to implement what the clinical protocol specifies. 
HIV prevalence among Chinese MSM is estimated between 5% and 15% 
depending on region (Wu et al., 2013; Zhong et al., 2014; Xu et al, 2016; Chen et al., 
2018; Ning et al., 2018). Estimated incidence —around 5 per 100 person-years 
nationally (Wang et al., 2014; Xu et al. 2016; Mao et al., 2018) and in Yunnan (Yunnan 
CDC, 2018)—is almost ten times the incidence among MSM in the US (Shang & Zhang, 
2015; Rosenberg et al., 2016; Ma et al., 2017). There are an estimated 6-10 million 
MSM aged between 15-49 years old living in China (Wong et al., 2009) and the PI’s 
recent modeling paper estimated 2.5 million PrEP-eligible MSM living in China (Zhang 
et al., 2018). By extending these assumptions to Yunnan, we estimate 80,000 PrEP-
eligible MSM in that province alone. Under relatively conservative assumptions, our 
model estimated that hundreds of thousands of HIV infections could be averted over a 
twenty-year horizon and that delaying implementation by five years would reduce cost-
effectiveness by 36% and lead to ~90,000 additional infections among MSM. 

Implementation of PrEP in more effective ways than the standard approach of issuing 
clinical guidance, without robust implementation guidance, is urgently needed. 
The Stages of Implementation Completion (SIC) Framework was derived from an 
8-stage SIC assessment tool that consistently identifies key implementation activities 
needed to achieve sustainable outcomes. The SIC tool was originally developed for a 
large-scale randomized trial comparing two strategies to deliver a child-welfare 
intervention across two state systems (Chamberlain et al., 2011). Mixed-methods 
research suggests the utility of SIC as a general Implementation Framework: 
organizational stakeholders and leaders used it to guide planning for the adoption of a 
new health intervention (Palinkas et al., 2018). The SIC emphasizes identifying 
implementation activities most likely to help organizations plan for successful 
implementation and anticipate barriers that hinder success. SIC findings consistently 
demonstrate that challenges with implementation can be mitigated through completion 
of pre-implementation activities such as Readiness Planning (Palinkas et al., 2018). 
This study will use the SIC Framework to guide and study the implementation of PrEP in 
Yunnan China. 
We propose to study this process in Yunnan Province, China, a key geography in 
the country’s HIV initiatives. Yunnan has made great progress towards attaining 90-90-
90 treatment targets, but there continue to be between 8,000~11,000 HIV diagnoses per 
year, and an estimated 5% annual incidence among men who have sex with men 
(MSM) (Yunnan CDC, 2018). Given the existing relationship of Aaron Diamond AIDS 
Research Center (ADARC) with the Yunnan government, we were approached to help 
design a study to identify an optimal implementation approach for PrEP. In collaboration 

with the Yunnan CDC, we developed the Yunnan-ADARC HIV Prevention Program. 
This real-world study provides a unique opportunity to evaluate the use of supportive 
implementation strategies and frameworks to increase the chance for successful and 
sustained implementation. In preparation for this study, the Yunnan CDC secured a 
pledge for a donation of 2128 person-years of TDF/FTC from Gilead Sciences. By 
removing barriers to PrEP access for a period, lessons for health system integration can 
be identified and directly applied once PrEP can be procured when generic TDF/FTC 
becomes available by the third year of the proposed project. 
B-HAPPY has two specific aims: 1) To evaluate whether the addition of 
Enhanced implementation strategies to Standard health system PrEP delivery improves 
client-services outcomes; 2) To compare the time, costs, and resources required for 
Standard versus Enhanced PrEP implementation. Enhanced PrEP implementation will 
rely on a systems analysis improvement approach (Sherr et al., 2014; Gimbel et al., 
2016) to deliver PrEP-related knowledge, skills, and best practices to clinics delivering 
PrEP. The outcome of the study will be an implementation blueprint that may be used 
across China and in other low- and middle-income countries to guide the introduction of 
PrEP into existing health systems. 
Study Procedures 
 
All study procedures will be conducted at international site (Yunnan, China). All 
study procedures have been reviewed and approved by Yunnan CDC IRB. Columbia 
personnel might be involved in data collection (obtaining content, conducting interviews 
with clients and program staff) at international sites. 

The overall study uses a stepped-wedge design to understand PrEP 
implementation at eight study sites in Yunnan, China. Each study site will go through 
four study periods. Sites will start with a six-month pre-baseline period, where PrEP is 
not offered but HIV prevention utilization data will be collected among MSM client 
population. HIV prevention utilization data will be captured by the PrEP M&E data 
system created by Yunnan CDC. Examples of indicators to be collected include number 
and frequency of HIV and STI tests, diagnoses, and treatment status. These metrics will 
be collected throughout all study periods. Each site will then have a different length of 
Baseline Period of standard PrEP implementation followed by a one-year Experiment 
Period with enhanced PrEP implementation. 
The study procedures involve audio recording of research subjects. All study 
procedures will be performed at eight study sites by research staff who have completed 
human subject protection training after obtaining informed consent from participants. 
This study will not use previously collected biospecimens, data or records. Detailed 
study procedures are listed below: 
Audio and/or video recording of research subjects: This study will conduct in-
depth interviews with (1) MSM clients, and (2) program staff to understand their 
experience with PrEP implementation. Interview guides are submitted in English only as 
these are documents to guide the English-Chinese bilingual interviewers. Therefore, no 
Chinese version of the interview guide is attached. We will audio-record interviews with 
participants who provide permission for recording. Audio-recording is a very widely used 
technique in qualitative research. It allows researchers to fully engage and concentrate 
on the interview rather than writing notes, which may be a distraction for both the 

interviewer and the participants. Audio-recording will help researchers keep accurate 
records of qualitative data and capture non-verbal expressions during the interview. 
Informed consent will be obtained before any study procedures are initiated. During the 
consent process, the interviewer will ask participant’s permission for audio-recording. 
For detailed process of obtaining consent, please see section “Informed Consent 
Process”. After informed consent is obtained, the interviewer will announce that the 
interview will begin and turn on audiorecording equipment. The interviewer will begin the 
interview by noting the time, date, and unique ID number for the interview. S/he will 
proceed to conduct the interview guided by the interview guides. The interview will be 
in-person or remotely one-on-one and is expected to last 45 to 60 minutes. All data will 
be confidential and protected. Compensation of 100 Yuan (approximately USD $16) will 
be provided upon completion of the interview. De-identified data will be sent to 
contractors external to Columbia University to support data analysis. 
 
 
Study Design 
 
B-HAPPY is a multisite randomized stepped-wedge trial to compare the 
effectiveness of two implementation strategy bundles: 1) Standard Implementation: 
comprising the strategies that the Yunnan CDC would usually use to introduce HIV 
prevention innovations in the absence of the proposed study; and 2) Enhanced 
Implementation: standard implementation enhanced by a SIC-guided approach to plan 
and implement PrEP. Eight sites will receive identical Standard Implementation during a 
six-month baseline period; additional strategies will be introduced to blocks of 2 sites in 
six-month increments. Importantly, the role of the SIC will shift between Baseline when 

it will be used as an observational assessment tool only, and Enhanced, when it will be 
used both to guide implementation and to assess it. During Enhanced Implementation 
each site will receive Implementation Support to plan and introduce PrEP by following 
the eight stages of the PrEP-adapted SIC. Implementation Support will involve a 
centralized TA system developed using US-based PrEP experience. The support will 
utilize a SIC-guided systems analysis and improvement approach that focuses on 
addressing implementation issues identified through real-time analysis of client, 
provider/site, and implementation data. During the study, sites will participate in four 
study periods: a Pre-Baseline Period (Routine service with no PrEP implementation), a 
Baseline Period (Standard Implementation of PrEP), an Experiment Period (Enhanced 
Implementation of PrEP), and a Post-Experiment-Sustainment Period. Sites are 
randomized to the timing of the intervention phase—the intervention condition changes 
over time and the intervention effect is the difference in outcomes between the Baseline 
and Experiment Periods, and the Post-Experiment and Baseline Periods. 
We hypothesize that adding SIC-guided implementation onto standard health 
system implementation of PrEP will improve outcomes and lead to greater likelihood of 
a) implementation success and b) sustainment. Successful implementation will be 
defined as a site achieving "Five-80%" targets and maintaining them three months in a 
row (Aim 1): 1) Awareness of PrEP among 80% of MSM clients; 2) PrEP eligibility 
assessment for 80% of MSM clients; 3) PrEP linkage for 80% of MSM clients who wish 
to start and are medically-eligible; 4) PrEP initiation for 80% of MSM who are linked to 
PrEP services; and 5) six-month PrEP continuation for 80% of those who remain PrEP-
eligible. Sustainment will be defined as achieving the Five-80% targets continuously 

over 6 months. The study team set 80% targets for two reasons: 1) 80% targets are 
commonly used in observational facilitated training in implementation processes; 2) the 
PI’s recently-published PrEP modeling analysis clearly showed the differential 
epidemiological impact a high real-world PrEP effectiveness would have compared to a 
low real world effectiveness: with 80% effectiveness, PrEP would lead to 345,000 
infections averted over a 20-year horizon compared to 219,000 with 50% effectiveness 
and 87,900 with 20% effectiveness (Zhang et al., 2018), spurring the team to choose 
ambitious targets that could drive down infections among MSM. To examine the 
implementation process and efficiency of the Enhanced versus Standard 
implementation approaches we will use SIC and COINS tools to monitor sites for 
implementation process and resource use. Outcomes will include proportion of 
implementation activities completed, duration of each implementation phase, and cost 
and resource allocation for completion of each implementation phase (Aim 2). Based on 
robust studies that assess implementation using the SIC, we hypothesize that time and 
resources spent in Pre-Implementation will be associated with higher odds of successful 
implementation and sustainment. 
 
Recruitment 
 
All recruitment activities will take place at our international site (Yunnan, China) 
and will be conducted in Mandarin Chinese, the primary language of the region, by 
Chinese-speaking research staff. All research staff will complete the required human 
subject training before conducting any recruitment activities. All procedures related to 
recruitment have been reviewed and approved by Yunnan CDC IRB. Columbia 
personnel will not be involved in the recruitment of participants. 

Two types of participants will be recruited for in-depth interviews in our study: (1) 
MSM clients of participating facilities; (2) program staff who are involved in PrEP 
implementation. We will conduct the interviews with clients and program staff in each 
study period, as a result, a total of four rounds of interviews will be conducted. 
1. MSM clients. Overview: In each study period, we estimate prescreening over 
96 MSM clients to achieve the enrollment goal of 48 evaluable study participants (6 
clients per site, 8 sites). We estimate a 50% response rate based on our experience 
conducting qualitative research among MSM in China, and our consultation with local 
collaborators. Given the qualitative nature of the study, we may conduct fewer 
interviews with clients if data saturation is reached (i.e. adding more participants into the 
study does not result in new findings). The total number of participants for the client 
interview will be 192 (48 per study period * 4 study periods). 
Two methods will be used for recruiting MSM clients: person-to-person, and 
handout/flyer. 
Research staff will conduct recruitment activities through person-to-person 
methods at facilities that are participating in the PrEP implementation, e.g. community-
based organizations (CBOs), STI clinic, HIV testing center, and HIV treatment clinic. In 
the waiting room of the facilities, staff will approach to male clients and explain the 
purpose of the interview study. It will be made clear that participation in the interview is 
totally voluntary; declining to participate in the interview will not have any negative 
impacts on access to HIV prevention services and PrEP care at sites. 
If a client shows interest, staff will administer a set of screening questions to 
determine their eligibility to participate in the study. If eligible, research staff will conduct 

a qualitative interview with the client directly in a private room in the facility, or schedule 
a time for interview in the future. 
In addition to the in-person recruitment approach, we will also print handouts 
about the interview study and distribute them in the waiting room of the facilities. The 
handout will include a brief introduction of the study purpose and procedures, and 
contact information of site research staff. Clients who are interested in the interview 
study will contact research staff for screening. 
2. Program staff. Overview: In each study period, we estimate prescreening over 
60 program staff to achieve the enrollment goal of 40 evaluable study participants (5 per 
site, 8 sites). We estimate a 67% response rate based on our experience conducting 
qualitative research among health workers in China, and our consultation with local 
collaborators. The total number of participants for the client interview will be 160 (40 per 
study period * 4 study periods). 
Given the qualitative nature of the study, we may conduct fewer interviews with 
program staff if data saturation is reached (i.e. adding more participants into the study 
does not result in new findings). We will recruit program staff through email. 
Recruitment strategy for interviews with program staff will be developed in 
collaboration local site coordinating agency and will target staff at study sites who have 
been employed for at least 3 months. We will work with coordinating agency at each site 
and send an email to all program staff introducing the study and inviting them to 
participate in an in-person or phone interview. Program staff who are interested in the 
interview study will contact researchers for screening. 

In the recruitment email, it will be made clear that participation in the interview is 
totally voluntary. This will be particularly emphasized for program staff at each site to 
ensure that they do not feel coerced into participation due to a belief that they are 
required to do so as part of their job. 
In addition, a purposeful, role-based sampling strategy will be used to recruit 
program staff due to their role in PrEP implementation or because other interviews 
identified them as having an important perspective. For program staff who were 
identified through this process, we will use identical recruitment procedures through an 
email invitation as outlined above. 
 
Informed Consent Process 
 
The only record linking the subject and the research would be the consent 
document and the principal risk would be potential harm resulting from a breach of 
confidentiality. Each subject, or parent/LAR if applicable, will be asked whether the 
subject wants documentation linking the subject with the research, and the subject's 
wishes will govern. 
We will not collect any confidential information in our interviews with clients and 
program staff. The only record linking their identity and the research would be the 
consent document, as a result, we request a waiver of written documentation of 
consent. 
Informed consent process will take place at our international site (Yunnan, China) 
and will be conducted in Mandarin Chinese, the primary language of the region, by 
Chinese-speaking research staff. All research staff will complete required human 
subject training and additional training on obtaining informed consent before conducting 

any consent activities. All procedures related to informed consent have been reviewed 
and approved by Yunnan CDC IRB (see attached approval letter). Columbia personnel 
may also be involved in obtaining informed consent from potential participants. 
This study will use electronic consenting (“e-consenting”) for potential study 
participants. Given that potential participants will be recruited in person at study sites 
and remotely, both in person e-consent and remote e-consent will be used, with an 
information sheet that describes study procedures and without a signature. All e-
consent process will occur online using the China-based survey platform WenJuanXing 
(WJX, https://www.wjx.cn/), which our team has previously used for conducting e-
consent and data collection. 
1. Consent process for client interview. Clients will be recruited through two 
ways: in-person recruitment in the waiting room, and through information on the 
recruitment handout. 
(1) For clients who are recruited through an in-person mechanism in the waiting 
room, in-person e-consent will be conducted after they indicated their interest in the 
study and are screened eligible. Research staff will bring the potential participant to a 
private room. Potential participants will be given ample time to read an information 
sheet (see submitted document) on an electronic tablet. They will also be given a 
chance to ask questions and express concerns. 
Additionally, research staff who have been designated to consent will discuss the 
specifics of the study including but not limited to the purpose of the research, 
procedures, time commitment, required tasks, benefits, risks, confidentiality etc. in a 
comprehensible (nonscientific) manner, using language readily understandable by the 

participant. Participants will be told that the participation is voluntary and that, if they do 
not consent, they will not be penalized. The consent process will include a request for 
permission to audio-record the interview for transcription and analysis. The person 
consenting will assure the voluntariness of the consent and will assess comprehension 
of the informed consent form through a teach-back method. 
If clients wish to be in the study and are ready to give consent, they will click an “I 
Agree” icon to consent to participate in the interview study. The research staff 
consenting will offer the participant a copy of the information sheet via email. If 
participants choose to receive a copy, research staff will remind participants that it is 
possible that the information sheet might be seen by others and therefore their 
confidentiality might be breached, so it is important that they properly store the copy. 
(2) For clients who wish to conduct the interview via telephone, research staff will 
email clients a link to an information sheet describing the purpose, procedures, potential 
risks and benefits of participating in this interview study. The consent process will 
include a request for permission to audio-record the interview for transcription and 
analysis. The email will also include information on how questions from clients may be 
asked and answered from a remote location, e.g. scheduling an in-person interview with 
research staff who are designated to recruitment and obtaining informed consent at 
study site, or scheduling a time for phone calls or electronic chatting.  
If clients wish to be in the study and are ready to give consent, they will click an “I 
Agree” icon to consent to participate in the interview study. Once the client has given 
the consent, they will see a reminder that asks them to print or email a copy of this 
information sheet for their own records. They will also be reminded that it is possible 

that the information sheet might be seen by others and therefore their confidentiality 
might be breached, so it is very important that they properly store the copy. 
2. Consent process for program staff interview. We will use identical consent 
procedures as listed above. 
 
 
 
Confidentiality of study data 
 
All data will be in electronic format and we will not have any hard copy of study 
data. In general, we will use Secure Sockets Layer (SSL) encryption to securely transfer 
the study data that are in electronic format. We will store study data on password-
protected, encrypted computers, and will restrict access to the data to key study 
personnel. 
This study will analyze data from five different sources. Here is a detailed 
description of how we will ensure confidentiality and safely stored data from five 
different sources: 
1. SIC and COINS data 
Description: The PrEP-SIC data includes a log of implementation activities 
completed by each site along with the dates of activity completion (e.g.: date written 
PrEP implementation plan completed, date first PrEP user retained in care for six 
months). The COINS tool assesses direct and indirect expenses, including the number 
of hours per role expected in completing each implementation activity listed in the SIC. 
SIC and COINS data will be collected by our Yunnan-based research coordinator 
through phone call and in-person site visits at two to four-week intervals and entered 

into the SIC website. SIC and COINS tool measures implementation process and does 
NOT contain personal identifiable information (PII) or protected health information (PHI). 
2. PrEP M&E data 
Description: Yunnan CDC (our primary collaborator in China and a subaward 
recipient on this grant) is upgrading their HIV prevention monitoring and evaluation 
(M&E) system to incorporate PrEP service delivery indicators. We expect this PrEP 
M&E data system to start working in May 2020, and it will document data including 
routine HIV utilization, PrEP service delivery and process measures for all facilities 
involved in PrEP implementation across the province and including at our study site. 
Yunnan CDC staff will extract data and aggregated at site level (e.g. number of HIV 
testing done per site, percentage of MSM clients receiving sexual health needs 
assessment, number of PrEP initiations). Only this site-level, aggregated data will be 
shared with the B-HAPPY study team. 
Data transfer and storage: The HIV prevention M&E system was created by 
Yunnan CDC in compliance with local ethical review and human subject protection 
regulations. Data that are extracted from the system will be saved in encrypted, 
password-protected database. Secure Sockets Layer (SSL) encryption will be used to 
securely transfer the extracted data from the system to our study team. We will store the 
database in password-protected, encrypted computers, and will restrict access to the 
data to key study personnel. 
3. Client checklist 
Description: To evaluate the process of HIV prevention and PrEP care delivery at 
each site, MSM clients at all facilities participating in PrEP implementation will be asked 

to anonymously fill out a 7-question yes/no checklist at visit exit. Sample questions 
include: "were you aware of PrEP prior to your visit today? (Yes / No)”. The checklist will 
be answered anonymously and will not collect any personal identifiable information (PII) 
or protected health information (PHI). Study sites will program the questions on an 
online survey platform, and use a tablet to collect this data. 
Data transfer and storage: Dedicated program staff will download checklist data 
from the online survey platform regularly and save them in encrypted, password-
protected database. Secure Sockets Layer (SSL) encryption will be used to securely 
transfer the data. We will store the database in password-protected, encrypted 
computers, and will restrict access to the data to key study personnel. 
4. Client interview 
Description: our study will conduct interviews with MSM clients to understand 
their experience accessing HIV prevention service and PrEP care at study sites, and 
factors associated with their decisions of PrEP use and regimen choice. See "Study 
procedures" section for details of the interview. 
Storage: For interviews with participants who give permission to audio-record, 
interviews will be recorded on a device that is not connected to the Internet. After the 
interview, the recording will be uploaded through a Virtual Private Network (VPN) to be 
stored in a password-protected, encrypted computer. Dedicated research team staff will 
transcribe and/or conduct rapid analysis of qualitative data using memo writing or 
debriefing techniques. For interviews that are transcribed, audio recordings will be 
deleted after we review transcripts for accurate translation. Any identifying information 
(e.g. name, home address, date of birth) will be redacted in the transcripts. Computers 

containing electronic data will be password protected with encryption software. This 
data will be coded and CUMC research staff will not have access to the key to link the 
direct identifiers. 
5. Program staff interview 
Description: Our team will conduct interviews with program staff at eight sites to 
understand their experience with PrEP implementation, discuss environmental context 
and resource-related issues as well as components of delivery and challenges in real 
time to inform the refinement of implementation process. See "Study Procedures" 
section for details of the interview. 
Storage: Identical procedures will be used for the client interview. 
Risks, Benefits & Monitoring 
 
Potential risks 
This is a study with minimal risk. Study procedures involving human subjects (i.e. 
in-depth interviews with clients and program staff) pose no more risk than expected in 
daily life. Participants (both MSM clients in PrEP implementation and study site staff 
who are enrolled in survey or focus groups) in this study might experience potential 
risks including the following: 
Sensitive topics. The goal of client interview is to understand their experience 
accessing sexual health services, including HIV and STI prevention, at study sites. 
Some participants may feel slightly uncomfortable or irritated with the questions 
discussing their experience with sexual health services. To address this risk, 
participants are encouraged but not required to answer all interview questions. 
Participants may skip or terminate the interview at any time. For program staff, the 

interview might include questions related to their organization culture, their perceptions 
of and roles in PrEP implementation. Some program staff may experience anxiety or 
discomfort caused by some of the questions. To address this risk, program staff are 
encouraged but not required to answer all interview questions. They may terminate the 
interview at any time. 
Loss of confidentiality. We will not collect any personal identifiable information 
(e.g. name, address, place and date of birth, private health information) in our interview 
studies. We will make every effort to keep information confidential, but no system for 
protecting confidentiality can be completely secure. Given that the only record linking 
the subject and the research would be the consent document with their signature and 
the principal risk would be potential harm resulting from a breach of confidentiality, we 
requested a waiver of documentation of informed consent, and decided to use 
information sheet for consent. We will minimize risk of a breach of confidentiality by 
anonymizing in-depth interviews with a unique identifier, deleting audio recordings after 
we review transcripts for accurate transcription, and redacting any identifying 
information from transcripts of interviews. Participants will also have an option to decline 
to answer any or all questions. Study participants are encouraged but not required to 
answer all interview questions. They may terminate the interview at any time. More 
details on how we plan to ensure confidentiality of study data are described in the 
“Confidentiality of study data” section. 
 
Potential Benefits 
Client interview: There is no direct benefit for MSM clients to participate in the 
study. However, knowledge gathered in the study may help researchers better 

understand PrEP implementation for MSM in China. This may help prevent HIV and 
might be beneficial to MSM clients in Yunnan and other places. 
Program staff interview: There is no direct benefit for program staff to participate 
in the study. However, knowledge gathered in the study may help researchers better 
understand program staff’s perspectives of PrEP implementation in China. This may 
help identify facilitators and barriers to PrEP service delivery, and might be beneficial to 
program staff to carry out HIV prevention services and PrEP care at their workplace. 
 
Available alternative interventions 
This is a study with minimal risk. The alternative to this study is not to participate. 
For MSM clients, non-participation in the study will not have a negative impact on their 
access to regular HIV and STI services at study sites. For site program staff, non-
participation in the interviews will not negatively impact their employment nor their 
eligibility to participate PrEP implementation or ongoing capacity building activities. 
 
Data and Safety Monitoring 
This study entails minimal risk to study participants and does not require a Data 
and Safety Monitoring Board (DSMB). The PI (Dr. Meyers), study site investigators in 
Yunnan CDC, and other designated personnel will conduct regular internal monitoring to 
assure the safe and proper conduct of the protocol, following the general principles of 
quality management. Internal monitoring of eligibility documentation and information 
sheet will be conducted by the research team shortly after enrollment begins. Internal 
monitoring activities will be documented by logs, and meeting minutes. 
 
Subjects 

 
Target enrollment: 352 
 
The proposed study will be conducted outside of the US. All sites are located in 
Yunnan, China. Based on race and ethnicity data of Chinese population, we expect all 
potential participants to be Asian, and non-Hispanic or Latino. This study will not 
exclude certain racial or ethnic group members from participating in the research. This 
study will include two types of research participants: 1.MSM clients: our study is 
exclusively focused on men who have sex with men (MSM) population with the goal of 
developing and testing a Model to Implement and Sustain PrEP Delivery for MSM in 
China. MSM population have been disproportionally impacted by the HIV/AIDS 
epidemic in China (see Background section for more details). The planned distribution 
of this type of study participants will be 100% male sex (at birth), Asian and non-
Hispanic or Latino. 2.Program staff: we estimate the gender distribution of this type of 
participants will be 50% male and 50% female. 
Upon completion of the interview, participants will be compensated 100 RMB 
(approximately USD $16) for their time. The compensation is determined based on our 
discussion with local collaborators and our previous experience conducting research in 
China. 
 
Statistical Analysis Plan 
 
Traditional designs—whether client-randomized or site-randomized—were not 
viable due to statistical power, contamination risk, and the dual focus on intervention 
and implementation outcomes. For instance, randomizing clients would risk 
contamination within sites, and randomizing sites would drastically restrict power. This 

is addressed by the proposed stepped-wedge design,25-27 with all sites participating in a 
baseline period (Standard Implementation), experiment period (Enhanced 
Implementation), and post-experiment period (Observation). Blocks of sites are 
randomized to the timing of the experiment period. Because intervention condition 
changes over time, the intervention effects reflect within-block changes (e.g., between 
the Enhanced and Standard periods). The design is highly flexible and maximizes 
power, and the proposed analyses straightforwardly accommodate different data 
structures for client-services outcomes (Aim 1) and implementation process outcomes 
(Aim 2), clients that span periods, and outcomes that are both research- and site-
collected. 
Data structure. For Aim 1, the client-services outcomes are structured with 
repeated visits (level-1) that are nested within clients (level-2) who are nested within 
periods (level-3) that are nested within sites (level-4). However, for analysis, the 
outcomes will reflect a single status for each client (e.g., initiation of PrEP), reducing the 
data to a three-level structure. For Aim 2, the implementation process outcomes are 
structured with three repeated measurements (i.e., across the baseline, enhanced, and 
observation periods; level-1) that are nested within sites (level-2). In each case, the 
small number of sites will be addressed. Of note, across aims, supplementary time-to-
event analyses28, 29 (e.g., the effect of Standard Implementation on time to client 
awareness) are possible by reformulating the data to client- and site-period structures. 
Likewise, clients will be associated with the period in which the outcome status is 
finalized. 

Model building, estimation, and significance testing. The nested data structures 
will be evaluated using mixed-effects regression models, but with 8 sites, specialized 
methods are required to estimate and validate the effects.29,30 Model estimation will 
utilize Bayesian Markov chain Monte Carlo (MCMC) methods.27 With this approach, the 
results are based on a large number of replications, providing an empirically-based 
sampling distribution—rather than a single point estimate—for each model parameter 
(e.g., the effect of Enhanced Implementation). Further, the conclusions will be validated 
using a frequentist, fixed effects approach (i.e., dummy-coded indicators) to control for 
site effects.31 To test for within-block changes across the three periods, two dummy-
coded indicators will be included to differentiate the Enhanced and Observation periods 
from the reference Standard period. Continuous, dichotomous, and ordered categorical 
outcomes will be evaluated according to Gaussian, binomial, and ordinal distributions. 
To evaluate the magnitude and precision of effects, 95% CIs will be computed, along 
with odds ratios (ORs) for non-linear outcomes. The Bayesian MCMC models will be 
implemented in MLwiN,27 and single-level regression models will be implemented in R.32 
Evaluation of Specific Aim 1: To evaluate whether the addition of SIC-guided 
PrEP delivery (Enhanced Implementation) to standard health system PrEP delivery 
improves services outcomes, including: awareness, screening, offer, initiation, and 
continuation at 6 months. Although the outcomes are repeatedly monitored, each 
reflects a client-level status. The three-level mixed-effects regression model, with clients 
(level-1) within periods (level-2) within sites (level-3), will include control variables for 
client characteristics (e.g. age, partnership status, ethnicity [Han vs other], sexual 
identity) and site characteristics (e.g. staffing, collaboration history between VCT and 

STI clinics, institutional leadership, localized epidemiology), along with indicators for 
study periods. Additionally, to control for temporal effects—natural improvements in 
PrEP implementation over time—for each client, the models will control for the time of 
study entry using linear and quadratic polynomials for the number of months from the 
start of the Standard period. With a single outcome status for each client, this controls 
each client’s outcomes for the timing of participating during the site’s PrEP 
implementation process. The results will indicate the average within-block change from 
Standard to Enhanced in the log-odds of awareness, screening, offer, initiation, and 
continuation. Likewise, to determine whether the effects are maintained, outcomes from 
the Sustainment period will be compared to the Standard and Enhanced periods. 
Statistical power for Aim 1. Multilevel power analysis typically assumes 
interventions are at the level of individuals or groups. However, here, the intervention is 
time-varying. As such, power for the primary client-services level outcomes was 
estimated using a multi-step approach recommended by Hox33: 
First, the total sample of observations was computed. For client-services level 
PrEP outcomes (ie, awareness, screening, offer, initiations, continuation), the sample 
size calculation reflects the size of the site client population and the availability of 
donated TDF/FTC (minimum of 1064 PrEP initiators). We anticipate ~2600 MSM clients 
per site per year and estimate ~133 PrEP initiators per site. However, this sample size 
is nested, introducing dependency and decreasing the level of statistical power. 
Second, to penalize the total sample for nesting, the design effect formula is 
reorganized (i.e. neff = n/[1+{nclus–1}) to provide the effective sample size of independent 
observations. Using a conservative nesting effect (i.e. =0.25 for phases/sites), the 

observations provide statistical power equivalent to 32 independent observations. 
Despite the large number of MSM clients each year, the effective sample reaches an 
upper limit of 32 due to (a) the small number of sites and (b) the large cluster size per 
site. However, the large cluster provides high reliability for estimated level-2 effects. 
Third, the effective sample size is used in a conventional power analysis. The 
results indicated that power is .80 to detect an effect of OR = 3.3 for the effect of 
Standard Implementation on client services outcomes. This is equivalent to a medium-
to-large standardized effect of d=0.66, or a between-period difference in probability of 
29%. For a less conservative nesting effect (=0.10), the sample is sufficient to detect 
OR=2.0, or a 16% between-period difference. 
Evaluation of Specific Aim 2: To compare the time, costs, and resources required 
for Standard versus Enhanced PrEP implementation. The expected opportunity cost of 
implementation for sites will be modeled using an estimator that will identify the “best 
practice” implementation costs. The theoretical model assumes that sites attempt to 
minimize costs of implementation in order to maximize excess revenue subject to a 
standard economic production function. This production function supports a dual 
cost function: where Cit represents the SIC-measured implementation costs for 
the ith program in the tth phase, Xit are characteristics of program i in phase t, is 
an i.i.d. error term, is a one-sided error term corresponding to the technical 
inefficiency for the ith practice in phase, and represents the coefficients of the 
best practice (most efficient) cost function. We assume the best-practice cost 
function is of the CobbDouglas form, such that the log-linear transformation 
yields. 

With estimates of the model parameters, we will predict the best practice 
implementation costs and technical inefficiency costs for each program. We will also 
explore which program characteristics and contextual factors are associated with higher 
or lower estimated inefficiencies. We will include measures of the “best practice” 
implementation costs, technical cost inefficiency and intervention costs in a model 
predicting successful implementation. The aggregated cost variables will be added to 
linear probability models (because cost measures require a prediction substitution 
model) as covariates alongside SIC Duration and Proportion scores to predict the binary 
outcome of whether a program sustained a minimum of 1-year post Stage 8 of the SIC 
(Competency). Differences in use of resources will be examined between sites that 
sustain versus those that do not. The assessment and analysis of implementation costs 
will be critical for the government as they develop their plan for widespread scale-up. 
Implementation outcomes. For Aim 2, with the focus on implementation process 
outcomes (rather than client services outcomes), the data are structured with three 
periods (level-1) within sites (level-2). The outcomes include the proportion of 
implementation activities completed, duration of activity completion, maximum 
implementation stage reached, and benchmarks on the Five-80s (3 consecutive months 
for implementation, 6 consecutive months for sustainment). The model will include key 
client and site characteristics, and dummy-coded indicators to differentiate the three 
periods. The results will indicate the difference between Enhanced and Standard in the 
level of each implementation outcome. As with Aim 1, the conclusions will be validated 
based on single-level models with fixed-effect control variables for site differences. 
Likewise, with the focus on achieving implementation process benchmarks, the model 

will be extended to a time-to-event framework,29 testing for differences between 
Enhanced and Standard in the time required to achieve benchmarks (adjusted for the 
duration of the Standard period). With the focus on implementation process outcomes, 
power for this aim is inherently limited. Results will focus on estimates of 95% CIs for 
the effect of Enhanced Implementation. 
Attrition & Missing Data. Following intention-to-treat principles, sites will be 
retained in research measurement independently of participation in Enhanced 
Implementation.34 Highly successful tracking and retention protocols will minimize 
missing data. However, some data will inevitably be missing. To address this, 
assumptions will be evaluated, and informed by the results, one of several approaches 
will be used35: i) For a small proportion of missing data with evidence of Missing at 
Random (MAR), the data will be analyzed using the estimation procedures described 
above. ii) For a non-trivial amount of missing data and evidence of MAR, multiple 
imputation for repeated measurements will be used.36 iii) For a non-random missing 
data mechanism, pattern mixture models will be used to identify and control the effect of 
the non-random mechanism.37, 38 
Data Analysis 
 
Data analysis was informed by the Consolidated Framework for Implementation 
Research (CFIR). CFIR consolidates holistic factors determining the implementation 
and delivery of an intervention/innovation. Evidence shows that utilizing CFIR can lead 
to a better PrEP implementation design and a smoother PrEP service delivery to the 
target audience. We developed an Interview Guide and conducted initial data analysis 
based on the CFIR V1.0. Following updates to the CFIR in 2022, we applied CFIR V2.0 

to develop an analytic matrix applied to the data to identify our key findings. We 
employed the Rapid Qualitative Analysis approach to analyze the data. Implementation 
studies show that rapid analysis can yield similar and consistent results while saving 
costs and resources. Data were first transcribed verbatim into Chinese and then 
translated verbatim to English for analysis. Four members of the research group split 
into teams of two, with each team reading the assigned transcripts and then 
summarizing the transcript into a two-page memo with a goal to reduce the data. An 
excel-based matrix template was developed with the CFIR domains, subdomains, 
constructs, and subconstructs listed in the rows and facilitators, barriers and solutions to 
overcoming barrier written as the column headers. After creating the matrix and arriving 
at consensus within the research team on the definition of each CFIR construct, text 
from transcripts was copied into the matrix. Teams met weekly to review matrix 
application and ensure consistency. Differences in application of matrix and 
interpretation of data were reconciled through discussion.  Finally, two teams came 
together to discuss and name the most relevant barriers, facilitators, and solutions to 
overcoming barriers within each CFIR constructs. 
 
 
 

 
References 
 
1. Nunn AS, Brinkley-Rubinstein L, Oldenburg CE, et al. Defining the HIV preexposure prophylaxis care 
continuum. Aids. Mar 13 2017;31(5):731-734. 
2. Kelley C, Kahle E, Siegler A, et al. Applying a PrEP continuum of care for men who have sex with men 
in Atlanta, Georgia. Clinical Infectious Diseases. 2015;61(10):1590-1597. 
3. Wendel K. Same Day PrEP in the STD Clinic: The Denver Public Health Experience. 2017. 
http://www.ncsddc.org/wp-content/uploads/2017/11/Wendel-Same-Day-PrEP.pdf. Accessed 
September 2nd, 2018. 
4. Fuchs JD, Stojanovski K, Vittinghoff E, et al. A mobile health strategy to support adherence to 
antiretroviral preexposure prophylaxis. AIDS Patient Care STDS. Mar 2018;32(3):104-111. 
5. Amico KR, Stirratt MJ. Adherence to preexposure prophylaxis: current, emerging, and anticipated 
bases of evidence. Clinical Infectious Diseases. Jul 2014;59 Suppl 1:S55-60. 
6. Holtzman C, Brady KA, Yehia BR. Retention in care and medication adherence: Current challenges to 
antiretroviral therapy success. Drugs. 2015;75(5):445-454. 
7. Wu Z, Xu J, Liu E, et al. HIV and syphilis prevalence among men who have sex with men: a cross-
sectional survey of 61 cities in China. Clinical infectious diseases. 2013;57(2):298-309. 
8. Zhong F, Liang B, Xu H, et al. Increasing HIV and decreasing syphilis prevalence in a context of 
persistently high unprotected anal intercourse, six consecutive annual surveys among men who have 
sex with men in Guangzhou, China, 2008 to 2013. PloS one. 2014;9(7):e103136.  
9. Xu J, Tang W, Zou H, et al. High HIV incidence epidemic among men who have sex with men in 
China: results from a multi-site crosssectional study. Infectious Diseases of Poverty. September 05 
2016;5(1):82. 
10. Chen Q, Sun Y, Sun W, et al. Trends of HIV incidence and prevalence among men who have sex with 
men in Beijing, China: Nine consecutive cross-sectional surveys, 2008- 2016. PloS one. 
2018;13(8):e0201953. 
11. Ning Z, Fu J, Zhuang M, et al. HIV and syphilis epidemic among MSM and non-MSM aged 50 and 
above in Shanghai, China: A yearly cross-sectional study, 2008–2014. Global public health. 2018:1-9. 
12. Wang QQ, Chen XS, Yin YP, et al. HIV prevalence, incidence and risk behaviours among men who 
have sex with men in Yangzhou and Guangzhou, China: a cohort study. Journal of the International 
AIDS Society. 2014;17(1):18849. 
13. Mao X, Wang Z, Hu Q, et al. HIV incidence is rapidly increasing with age among young men who have 
sex with men in China: a multicentre cross-sectional survey. HIV medicine. 2018 
14. Yunnan CDC. HIV Epidemiology in Yunnan China from 1994 - 2017. R01 Application Planning 
Meeting. Kunming, Yunnan, China. 2018 
15. Rosenberg E, Grey J, Sanchez T, Sullivan P. Rates of prevalent HIV infection, prevalent diagnoses, 
and new diagnoses among men who have sex with men in US states, metropolitan statistical areas, 
and counties, 2012-2013. JMIR Public Health and Surveillance. 2016;2(1):e22. 
16. Shang H, Zhang L. MSM and HIV-1 infection in China. National Science Review. 2015;2(4):388-391. 
17. Ma J, Li Y, Zhang R, Wang J, Pan S, Wang L. HIV incidence among men who have sex with men in 
Kunming city by a cohort study during 2012 to 2016. Chinese Journal of AIDS&STD. 2017;8:755- 757.  
18. Wong FY, Huang ZJ, Wang W, et al. STIs and HIV among men having sex with men in China: a 
ticking time bomb? AIDS Education and Prevention. Oct 2009;21(5):430-446.  
19. Zhang, L., Peng, P., Wu, Y., et al. Modelling the epidemiological impact and cost-effectiveness of 
PrEP for HIV transmission in MSM in China. AIDS and Behavior. 2018;23(2): 523-533. 

20. Chamberlain P, Brown H, Saldana L. Observational measure of implementation progress in 
community based settings: The stages of implementation completion (SIC). Implementation Science. 
2011;6:116-116.  
21. Palinkas LA, Campbell M, Saldana L. Agency Leaders' Assessments of Feasibility and Desirability of 
Implementation of Evidence-Based Practices in Youth-Serving Organizations Using the Stages of 
Implementation Completion. Front Public Health. 2018;6:161.  
22. Sherr, Kenneth, Sarah Gimbel, Alison Rustagi, Ruth Nduati, Fatima Cuembelo, Carey Farquhar, 
Judith Wasserheit, and Stephen Gloyd. "Systems analysis and improvement to optimize pMTCT 
(SAIA): a cluster randomized trial." Implementation Science 9, no. 1 (2014): 55. 
23. Gimbel, Sarah, Alison S. Rustagi, Julia Robinson, Seydou Kouyate, Joana Coutinho, Ruth Nduati, 
James Pfeiffer et al. "Evaluation of a systems analysis and improvement approach to optimize 
prevention of mother-to-child transmission of HIV using the consolidated framework for implementation 
research." Journal of acquired immune deficiency syndromes (1999) 72, no. Suppl 2 (2016): S108. 
24. Zhang, L., Peng, P., Wu, Y., et al. Modelling the epidemiological impact and cost-effectiveness of 
PrEP for HIV transmission in MSM in China. AIDS and Behavior. 2018;23(2): 523- 533. 
25. Fok CCT, Henry D, Allen J. Research designs for intervention research with small samples II: Stepped 
wedge and interrupted time-series designs. Prevention Science. 2015;16(7):967-977. 
26. Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. 
Contemporary Clinical Trials. 2007;28(2):182-191. 
27. MCMC estimation in MLwiN. University of Bristol: Centre for Multilevel Modelling. [computer program]. 
Version; 2005. 
28. Gibbons RD, Duan N, Meltzer D, et al. Waiting for organ transplantation: results of an analysis by an 
Institute of Medicine Committee. Biostatistics. 2003;4(2):207-222. 
29. Singer JD, Willett JB, Willett JB. Applied longitudinal data analysis: Modeling change and event 
occurrence: Oxford University Press; 2003. 
30. Maas CJ, Hox JJ. Sufficient sample sizes for multilevel modeling. Methodology. 2005;1(3):86-92. 
31. McNeish D, Stapleton LM. Modeling clustered data with very few clusters. Multivariate Behavioral 
Research. 2016;51(4):495-518. 
32. R: A language and environment for statistical computing [computer program]. Version. Vienna, Austria: 
R Foundation for Statistical Computing; 2018. 
33. Hox JJ, Moerbeek M, van de Schoot R. Multilevel analysis: techniques and applications 3ed. New 
York Routledge; 2018. 
34. Nich C, Carroll KM. ‘Intention-to-treat’ meets ‘missing data’: Implications of alternate strategies for 
analyzing clinical trials data. Drug and Alcohol Dependence. 2002;68(2):121-130. 
35. Schafer JL, Graham JW. Missing data: Our view of the state of the art. Psychological Methods. 
2002;7(2):147. 
36. REALCOM: Methodology for realistically complex multilevel modeling [computer software & manual]. 
Bristol, UK: Centre for Multilevel Modelling. [computer program]. Version; 2008. 
37. Hedeker D, Gibbons RD. Application of random-effects pattern-mixture models for missing data in 
longitudinal studies. Psychological Methods. 1997;2(1):64. 
38. Molenberghs G, Verbeke G. Models for discrete longitudinal data. New York: Springer; 2006.