Implementation, Evaluation, and Cost Effectiveness of a Data-to-Care Strategy to Improve HIV Continuum Outcomes for Out of Care PLWH in Ukraine
Phase: NA
Conditions: HIV Infections
Sponsor: Johns Hopkins Bloomberg School of Public Health

 
 
 
Implementation, Evaluation, and Cost Effectiveness of a Data-to-Care Strategy 
to Improve HIV Continuum Outcomes for Out of Care PLWH in Ukraine 
Sponsored by: 
 
US National Institutes of Health 
 
NCT number: NCT05821413 
 
 
Co-Principal Investigators 
Jill Owczarzak, PhD 
The Johns Hopkins University 
 
Kostyantyn Dumchev, MD, MPH 
Ukrainian Institute on Public Health Policy 
 
 
 
 
 
D2C Ukraine 
Version 1.1 
21-Mar-2023 
 
 

D2C Ukraine 
2 
21-Mar-23 
 
Version 1.1 
Contents 
Study overview 
3 
Background 
5 
Project Team and Relevant Experience 
7 
Approach 
8 
Specific Aim 1: Adapt a D2C implementation strategy for Ukraine 
9 
Specific Aim 2: To pilot a randomized clinical trial of a D2C strategy versus standard of care 
12 
Study Design 
12 
Sample size 
12 
Intervention Overview 
12 
Procedures 
13 
Patient selection and verification of inclusion criteria 
13 
Patient outreach and verification of exclusion criteria 
13 
First visit 
14 
Intervention Procedures 
15 
Control Condition 
16 
Data Collection 
16 
Intervention implementation 
17 
Statistical Analysis and Feasibility Assessment 
17 
Specific Aim 3a: To assess the feasibility, acceptability, and implementation-related processes and 
outcomes of the D2C strategy 
18 
Specific Aim 3b: Economic Analyses To assess the costs and cost-threshold of the D2C strategy 
20 
Adverse events (including suicide risk) 
21 
Data management 
22 
Human subjects 
22 
Ethical Review 
22 
Qualitative data collection 
22 
Informed Consent 
23 
Benefits 
23 
Compensation 
23 
Risks Associated with Loss of Confidentiality 
23 
Protection Against Loss of Confidentiality 
23 
Quantitative data collection 
24 
Informed Consent 
24 
Potential risks 
24 
Protection Against Risks 
24 
Compensation 
25 
Confidentiality 
25 
Protocol Deviations 
25 
Dissemination, notification and reporting of results 
25 
Notifying Participants of Study Findings 
25 
Dissemination of Study Findings 
25 
Bibliography 
27 
Appendix 1: Recruitment Scripts 
33 
Appendix 2: Recruitment Log (structure) 
35 
Appendix 3: Informed Consent Form for the main study 
36 
Appendix 4: Case management checklist 
41 
Appendix 5: Contact information form 
42 
Appendix 6: Data Elements for MIS data extraction 
43 
 
 

D2C Ukraine 
3 
21-Mar-23 
 
Version 1.1 
Study overview 
In Ukraine, use of ART among people living with HIV (PLWH) has not kept pace with the number of 
new HIV diagnoses over the past two decades.1 Of the approximately 250,000 estimated people living 
with HIV in Ukraine, only 136,105 (44%) are receiving ART.2 The majority of those who test positive 
for HIV never returned to a care provider for a full assessment and enrollment in treatment. The 
cascade shows significant gaps: about 34,000 PLWH are diagnosed but not on ART. Drop-offs in the 
HIV care continuum and diminished access to HIV care persist despite scale-up of linkage to care and 
comprehensive case management programs for PLWH in Ukraine.3–9 ART use rates are particularly low 
among HIV-positive people who inject drugs (PWID), with only 38% on ART and 28% virally 
suppressed.10 At least 50% of PLWH in Ukraine acquired HIV though intravenous drug use and are 
likely to be active people who inject drugs (PWID).  
Data-to-Care (D2C) is a high-impact public health strategy that integrates multiple sources of data 
such as clinical data from medical information systems, surveillance data, and ongoing case 
management assessments with clients to identify PLWH who are not in care, engage them in care, 
and manage the HIV care continuum.11 D2C strategies complement evidence-based practices for HIV 
care adherence by integrating clinical and case management data at multiple points along the HIV 
care continuum, using systematic assessments to identify unmet needs such as substance abuse 
treatment and make appropriate care referrals, and using data to inform practice change and improve 
linkage to and retention in care. D2C strategies have been effectively implemented in jurisdictions 
throughout the United States12 but are not standard of care in low- and middle-income countries 
(LMICs) such as Ukraine. How to effectively implement D2C strategies in these new contexts has not 
been studied. To address this gap, this R34 study will adapt and pilot a D2C strategy to improve HIV 
care outcomes among not-in-care PWID living with HIV in Ukraine.  
This study’s Specific Aims are: 
1) To adapt a D2C implementation strategy for Ukraine. We will use the Dynamic 
Adaptation Process13,14 to identify the individual, organizational, policy, and infrastructure 
factors to adapt an existing US-developed D2C strategy in Ukraine. 
2) To study preliminary effectiveness of a D2C strategy versus standard of care on 
primary outcomes of HIV care engagement, ART initiation or re-initiation, and viral suppression 
among not-in-care PLWH (n=160); and secondary outcomes of engagement or re-engagement 
in ancillary services (e.g., drug treatment) and quality of life using a cluster randomized trial 
design. 
3) To assess the feasibility, acceptability, and implementation-related processes and 
outcomes of the D2C strategy using the Consolidated Framework for Implementation 
Research adapted for Complex Systems (CFIR-CS)15 and practice improvement notes, in-depth 
interviews with D2C implementers and participants, and quantitative D2C usage data.  
a. Determine the cost of the D2C strategy and estimate the thresholds for the 
intervention to be cost-effective or cost-saving.  
Ukraine is well-positioned as a site to identify how D2C strategies can be implemented in LMICs. With 
the support of international stakeholders such as PEPFAR and USAID, the Ukrainian Ministry of Health 
is actively promoting the use of a medical information system (MIS) at the HIV clinic level. The MIS 
contains patient-level information on HIV care appointments kept, medication prescriptions, all 
diagnosed co-morbidities, and clinical and laboratory test results.16 This information can be mobilized 

D2C Ukraine 
4 
21-Mar-23 
 
Version 1.1 
in a D2C strategy that tracks patients through the care continuum, uses data to make decisions about 
patient care and improve case management practices, attends to psychosocial factors that affect 
medication adherence (e.g., substance use), and coordinates the provision of non-clinical social 
services.17 Our project is positioned to institute and test strategies for maximizing the benefits of this 
system to improve the HIV cascade for PLWH with history of drug use. 
This study addresses 3 NIDA HIV/AIDS research priority areas: developing interventions that influence 
organizational structure, climate, and culture to promote evidence-based practices; evaluation of 
adaptation processes and intervention effectiveness in real-world settings; and studies of structural 
approaches to improve quality and utilization of services. It will be one of the first studies to adapt 
and evaluate a D2C strategy in an LMIC context, which has implications for HIV policies in Ukraine 
and globally. The proposed research will increase understanding of the intervention, organizational, 
provider, and patient characteristics that facilitate successful implementation of a D2C strategy, and 
the key policy, infrastructure, and process factors necessary to support integration of D2C into 
existing clinical and case management practices. This study will also position the research team to 
conduct a larger RCT to determine the effectiveness of a D2C strategy in Ukraine. 
 
 

D2C Ukraine 
5 
21-Mar-23 
 
Version 1.1 
Background 
People living with HIV (PLWH), particularly those who use drugs, experience significant 
need for supportive services such as case management and substance use treatment.18 
Provision of case management, employment services, housing, and substance use treatment can link 
and retain HIV positive populations to HIV care and improve HIV related clinical outcomes.19–24 A 
client-centered approach,25 with the use of a multidisciplinary team26 and coordinated care,27,28 is the 
most effective strategy to providing ancillary services to HIV infected populations. For people who 
inject drugs (PWID), medication-assisted treatment (MAT) is highly effective in improving linkage and 
retention in HIV care29–31 and health outcomes.29,32–34 Connecting HIV-positive PWID to ancillary 
services such as MAT is particularly important in Ukraine, where injection drug use (primarily opiates) 
accounts for up to 50 percent of new HIV cases.17,35,36   
The national HIV care cascade in Ukraine reveals large gaps between the number of 
people diagnosed with HIV and the number in active care, on ART, and virally 
suppressed.1 Among the approximately 250,000 
estimated PLWH in Ukraine, only 136,105 (54%) are 
receiving ART as of 01/01/2020.37 The majority of those 
who test positive for HIV never returned to a care 
provider for a full assessment and enrollment in 
treatment. The cascade (see adjacent figure) shows 
significant gaps: about 34,000 PLWH are in care but not 
on ART. The latest national bio-behavioral survey data 
show that only 52% of PWID are in HIV care, 38% are 
on ART and 28% are virally suppressed.38  
Improving access to and use of clinical and service provision data can lead to better 
health outcomes for PLWH who are not in care. In the United States, recipients of Ryan White 
funding increasingly recognize the value of multi-directional information exchanges that use HIV 
surveillance data, in combination with medical record data, to identify PLWH who are not in care, and 
intervention opportunities such as referral to HIV specialty care or supportive services.11 A data-to-
care approach is a public health strategy that integrates multiple sources of data to identify HIV 
individuals not in care, engage them in care, and manage the HIV care continuum.39–41 A data-to-care 
(D2C) strategy identifies patterns in services, outcomes, and demographics that might necessitate 
changes to program activities in order to improve outcomes across the HIV care continuum.42 For 
example, medical information systems (MIS) can be used to track patients through the care 
continuum and identify points of service breaks. Provider-mediated data usage involves health 
department staff working with providers to re-engage patients; electronic linkages coordinate between 
surveillance databases and medical records databases; and in direct outreach, trained staff reach out 
to persons who are identified as not-in-care through the surveillance system.43  
Key features of D2C approaches are collaboration between specialists across multiple 
public health and health care sectors, and feedback mechanisms to monitor progress and 
inform practice change. D2C requires cooperation between disease investigators, epidemiologists 
and researchers, information technologists, and service providers (case managers, clinicians).42 
Reports can include information on patients’ viral suppression, types of prescribed ART, medical visits 
attended and missed, and engagement in MAT programs for PWID. These reports can then be 
compared to benchmarks and strategies to realign resources can be developed to respond to 
100.0
67.6%, 
54.2
50.9
56,264
67,341
55,162
0
100000
200000
300000
Estimated
PLWH
Diagnosed
On ART
Virally
suppressed
Figure. National HIV care cascade in 
Ukraine 
As of 01.01.2020

D2C Ukraine 
6 
21-Mar-23 
 
Version 1.1 
performance gaps, such as more targeted outreach efforts to reach specific populations or more 
intensive approaches to locating clients who missed appointments.  
International and Ukrainian institutions have made significant investments to increase 
data quality and use to improve HIV outcomes for key populations. The Public Health Center 
of the Ministry of Health of Ukraine (PHC) in collaboration with the Network of PLWH funded by 
PEPFAR actively promotes the use of a nationwide Medical Information System (MIS) at the clinic 
level. The MIS contains patient-level information on HIV care appointments kept, medication 
prescriptions, all diagnosed co-morbidities, and clinical and laboratory test results. It also includes 
information on new and previously detected HIV cases. The MIS is also used by NGOs that provide 
social and medical support for PLWH, allowing for tracking treatment and patient pathways across 
service sectors and institutions. MIS use has been extended to all regions of Ukraine.16 PHC created a 
division to provide support MIS users and developed webinars and other materials to train a wide 
range of service provider cadre to use the system.16 PHC is currently working to add new features 
(such as MAT and hepatitis treatment monitoring) and to expand the use of MIS to facilitate patient 
tracking and analysis of data on cascade and quality of care indicators at the clinic and oblast 
(regional) levels.17 The Ukrainian Institute on Public Health Policy (UIPHP—see Study Team, below) 
evaluated outcomes of national ART optimization (scale-up of dolutegravir-based regimens) using the 
MIS data (ART OPTI project). PEPFAR, other international donors, and the PHC recognize the role that 
effective use of data will play in achieving 90-90-90 targets. 
International and Ukrainian institutions have made significant commitments to increasing 
the quality of HIV/AIDS services, strengthening the capacity of Ukrainian institutions to 
improve the continuum of HIV services for key populations, and improving access to MAT. 
USAID and PEPFAR both currently support efforts to improve HIV case finding, primarily through 
increased HIV testing and initial linkage to care. Moreover, Ukraine has expanded availability of and 
access to MAT, a critical component of successful retention and care for PWID.20–23,44–48 Our proposed 
D2C strategy complements these efforts and leverages the expansion of MAT in Ukraine by identifying 
out-of-care HIV-positive PWID and linking them with HIV and other services, including MAT. The D2C 
intervention would be a valuable amendment that would increase the cost-benefit ratio of any case-
management intervention by timely identifying patients who need it most.  
A planning grant (R34) is the ideal mechanism for this developmental project. It will allow 
the research team to establish the feasibility of a D2C for PWID in Ukraine and develop the tools, 
protocols, and management structure for implementing a full-scale trial. It will also produce important 
planning information on the cost of implementing a D2C in Ukraine and allow us to determine under 
what conditions a D2C may be a reasonable investment in Ukraine. Finally, the proposed analytic 
strategy will inform sample size estimation for a larger trial and determine whether ordinal logistic 
regression, and more specifically mixed proportional odds models, may be suitable for modeling 
movement along the HIV care continuum among PWID. 
The scientific premise of this project is strong. Only 29% of HIV-infected PWID in Ukraine are 
virally suppressed; of PWID who are on ART, only about 74% are undetectable.49 Connecting PLWH 
who are diagnosed to comprehensive case management and ancillary services can improve outcomes 
along the HIV care continuum. Clinical and service provision data, including viral suppression, ART 
prescription, medical visits, and engagement in other services, can be used to improve health 
outcomes for HIV-positive PWID. This D2C intervention extends current efforts to improve the 
availability and use of data by assessing clients with standardized, validated assessments on a range 

D2C Ukraine 
7 
21-Mar-23 
 
Version 1.1 
of health and social factors that contribute to engagement in care and using this information to create 
individualized treatment plans and implement practice change.  
INNOVATION 
Maximize the utility of new data systems: This project leverages a resource with widespread 
support from policy makers, funders, and providers. We will institute and test strategies for 
maximizing the benefits of an MIS to improve the HIV cascade for PWID through data collection, data 
analysis, and program improvement. 
Link clinical and case management data: The proposed D2C strategy prioritizes active review of 
not-in-care lists and contacting not-in-care individuals to engage them in HIV care and case 
management. We will develop a mechanism within the MIS to actively monitor and identify individuals 
who have fallen out of care or at risk for falling out, such as missed visits (which can indicate factors 
in individuals’ lives that may interfere with their ability to remain in care and predict long-term 
mortality among persons initiating HIV care).50,51  
Systematically identify client needs: The proposed data-to-care intervention will integrate an 
acuity tool into case management. This tool will allow case managers to systematically assess client 
needs in multiple domains (substance use, mental health, housing, quality of life) over time, develop 
client-centered treatment plans, and monitor treatment goals tailored to the specific needs of HIV-
positive PWID.  
Transfer best practices through Implementation Science: D2C strategies are recognized by the 
Centers for Disease Control and Prevention as a key component of high impact prevention efforts. The 
WHO recognizes the potential role of digital health information health systems strengthening52 and 
calls for an increase in high-quality research on the integration of health data into health care 
systems, particularly in low and middle income countries.53 Rigorous Implementation Science research 
is necessary to guide the adaption of highly complex interventions such as D2C into new contexts, 
such as LMICs, for maximum effectiveness and sustainability. Our proposed use of the Dynamic 
Adaptation Process and Consolidated Framework for Implementation Science will yield critical 
information on the organizational, policy, provider, client, and structural factors necessary for 
successful D2C implementation.  
Project Team and Relevant Experience 
This project brings together a strong team of multidisciplinary researchers with expertise in HIV 
research in Ukraine, linkage to care programs, implementation science, and qualitative and 
quantitative methodologies. An existing partnership between Johns Hopkins School of Public 
Health (JHSPH) and the Ukrainian Institute on Public Health Policy (UIPHP) will allow 
efficient implementation by avoiding start-up delays common to de novo research programs. Our 
collaborative portfolio includes 2 NIH-funded studies that explore drug use, social networks, and HIV 
risk (R21 DA044807) and stigma among HIV-positive women who use drugs (R21TW011060). The 
study will be led by Principal Investigators Jill Owczarzak, PhD and Konstantin Dumchev, MD, 
MPH. Owczarzak is an Associate Professor in the Department of Health, Behavior, and Society 
(HBS) at JHSPH. She is an expert in qualitative research methods and implementation science and 
served as the PI of 4 Ukraine-based, NIH-funded studies related to gender and access to care 
(R21DA040969); drug use practices, social networks, and place (R21 DA044807), stigma, gender, and 
HIV care (R21TW011060); and the development of evidence-based interventions for PWID (R01 
DA033644).54–58 Dumchev is the Director of UIPHP. He conducts research on substance use and 

D2C Ukraine 
8 
21-Mar-23 
 
Version 1.1 
HIV/AIDS, including epidemiology, estimation, and program effectiveness. He served as a Site PI for 
the HPTN 074 trial (UM1AI068619) of an integrated case management intervention aimed at 
improving linkage to care and ART initiation among PWID.34,59 During his work as a Strategic 
Information Advisor at CDC, he managed the development of the concept and terms of reference for 
the MIS. He continues to consult the PHC on further MIS development. Dr Dumchev has developed 
the Simple Treatment Monitoring Application (STMA), an electronic medical records system. STMA was 
widely used by MAT and HIV clinics,2,36,60 and is now being gradually replaced by MIS. Relevant 
ongoing projects include IMPACT (R01DA043125), integrating MAT into primary care using tele-
education and Quality Improvement interventions, and MEDIUM (U01DA045384), implementing 
mental health services into MAT clinics. Brian Weir, PhD is an HIV prevention researcher with 
expertise in the design and analysis of quantitative research and in economic cost-effectiveness 
analysis.61–65 Christopher Hoffmann, MD brings expertise in implementation science research on 
the improvement of HIV and TB outcomes in low and middle income settings.66–69 Tetiana 
Kiriazova, PhD is a Senior Researcher at UIPHP and is an expert on linkage to HIV and addiction 
care for PWID, prisoners and other populations domestically9,70,71 and internationally,72 and stigma 
towards key populations.73  
All study key personnel have current Good Clinical Practice and Human Subject Protection trainings 
certificates. 
Approach 
Overview of a Data-to-Care Strategy D2C strategies incorporate surveillance and other data into 
multiple points along the HIV care cascade, including Patient Identification and Outreach, Case 
Management, and Health and Social Service Use.  Patient Identification and Outreach 
encompasses: (1) using data within the HIV-specific reporting systems (such as eHARS in the US) to 
identify those patients who meet the not-in-care criteria (e.g., patients registered as HIV-positive who 
had no HIV medical visits in a calendar year76); (2) generating regular reports that list patients who 
meet the criteria; and (3) providing clinic-based providers with this list (including patient contact 
information).39,40,43 This information is then shared with case managers, disease intervention 
specialists, or linkage-to-care coordinators (depending on the implementation context), who then use 
available contact information to establish contact with the client and initiate engagement in care 
activities. After contact is made between the client and service provider (linkage-to-care specialist, 
case manager), through Case Management, care coordinators use Acuity Tools and other needs 
assessments to identify unmet needs, potential barriers to care, and referral pathways. Acuity Tools 
complement rather than supplant ongoing case management programs to assess mental health, 
substance use, social support, intimate partner violence, financial stability, housing situation, legal 
issues, dental care, and sexual health. Acuity Scores, derived from the Acuity Tool, determine the 
frequency and intensity of case management engagement with clients and the types of service 
referrals clients need.41,42,77 [See table at right.] For example, a client that keeps appointments as 
scheduled, is stably housed, has been in drug recovery for a year, and has no history of mental illness 
would be categorized as Level 1: Self-Management and receive face-to-face contact with a case 
manager once every 6 months. Clients who decline to take medication, exhibit significant risk 
behaviors, do not follow up on referrals, and have multiple medical diagnoses would be categorized as 
Level 4: Intensive Case Management and would have monthly face-to-face and weekly phone contact 
with a case manager. Health and Social Service Data are tracked in data management systems, 
including clinical patient data (CD4, viral load, ART initiation and re-initiation, co-morbid diagnoses 

D2C Ukraine 
9 
21-Mar-23 
 
Version 1.1 
and medications); and process data (enrollment in services, HIV care appointments made and kept, 
referrals, contact attempts).  
Study Sites Consistent with PEPFAR priorities, 
we will partner with large HIV clinics in four 
high burden regions: Dnipro, Poltava, Odesa 
and Kyiv City. We will select 4 sites from the 
list of clinics that will match the following 
criteria: (a) at least current 150 patients 
registered with injecting drug use (IDU) mode 
of transmission; (b) current provision of case 
management (Steps Toward Health or other 
interventions, described below); (c) MIS is fully 
operational and all patients are entered; (d) 
administration approval for the site and its 
clinicians to participate. 
Specific Aim 1: Adapt a D2C implementation strategy for Ukraine 
We will use the Dynamic Adaptation Process (DAP)—phases I (Exploration) and 2 (Preparation)—to 
guide the adaptation of a D2C strategy for Ukraine.13 The DAP considers the multilevel context of 
service delivery and engages multiple stakeholders, which is appropriate for a complex structural 
intervention such as a D2C. Project activities for Aim 1 will include baseline in-depth interviews with 
service providers, clinic directors, PLWH who inject drugs, and other key stakeholders who will be 
involved in adapting and implementing the D2C strategy; MIS data module and extraction protocol 
development; and finalizing the D2C strategy. 
Provider Baseline Interviews We will conduct baseline in-depth interviews (n=32 with case 
managers, nurses, and agency directors who will implement the D2C intervention. Directors (n=4) 
may not have regular, direct contact with clients but can provide an overall view of various systems 
that may impact access to treatment services and implementation of the D2C strategy, including 
policies and how they are put into practice, admission requirements, and health care and treatment 
organization. They can provide key insight into institutional processes and organizations.78 Case 
Managers/nurses/social workers (n=28) have regular, direct contact with clients within the social 
service and health care systems, and can provide practical, detailed information about program and 
policy implementation.79,80 All participants will engage in a full consent process. 
Provider Inclusion criteria and screening: (1) 18 years of age or over; (2) being an employee of 
an organization that currently conducts case management among PLWH; (3) speak either Ukrainian or 
Russian; and (4) able to consent and willing to participate.  
Provider Sampling plan and recruitment: Interview participants will be all case managers and 
directors of D2C intervention implementation sites that have agreed to participate in this research 
project.  
Provider Baseline Interview Content: Interviews will focus on current practices with respect to 
providing and coordinating services to PLWH, existing communication efforts/strategies, and how data 
is currently collected and used within case management. All interviews will also explore: 
- 
current referral processes, patient prioritization, client load, and client profiles 
- 
client counseling and continuing care services; 

D2C Ukraine 
10 
21-Mar-23 
 
Version 1.1 
- 
policies that may create barriers to the delivery of the D2C intervention, patient care 
coordination and current related public health interventions;  
- 
existing programs or other innovations to improve services or outcomes for PLWH and 
PWID; 
- 
critical barriers to care for an D2C program to address and the feasibility of addressing 
them;81  
- 
perceived client satisfaction, barriers to treatment, and general environmental features that 
may facilitate or undermine clients’ efforts to access services. 
PLWH Baseline Interviews (n=20). In-depth, qualitative interviews will be used to understand 
HIV-positive PLWH access to, use of, and perceived needs for services related to their HIV, drug use, 
mental and physical health, and social needs, and inform the D2C strategy based on participant needs 
and available resources.  
PLWH Inclusion criteria: 1) being at least 18 years old, living in one of the study cities; 2) having 
received an HIV diagnosis; 3) having self-reported injecting drugs in the previous year; 4) speak 
either Ukrainian or Russian; and 5) able to consent and willing to participate.  
PLWH Sampling plan and recruitment: We will interview 10 participants from each study city 
(n=20). 10 interviews in each city will allow us to obtain a breadth of perspectives on PLWH access to 
health care and other services and reasons why people do not link to care and why they may 
disengage in care. Due to its formative nature, we will target HIV-positive PLWH both in and out of 
treatment and employ purposive sampling to obtain a broad range of possible experiences in terms 
various characteristics: 
- 
demographic (e.g., age, marital/partner status, socioeconomic status); 
- 
behavioral (e.g., type of drugs injected, number of years injecting drugs); and 
- 
health (e.g., time since HIV diagnosis, mental health, comorbid conditions, drug 
treatment history). 
Interview participants will initially be recruited from NGOs, drug treatment programs, and HIV clinics 
through direct recruitment and participant referral. Facilitated by outreach workers affiliated with each 
agency, research assistants will identify seed participants to serve as initial contact for recruiting from 
the target population. We will identify a heterogeneous set of seeds at each site to enable us to reach 
relevant subgroups, including those who are and are not in HIV care and other services. Participants 
who have been successfully recruited will be given information cards about the study to refer 
acquaintances and members of their social networks who may meet inclusion criteria.82 This chain 
referral approach has been successfully used elsewhere in the region to reach and recruit individuals 
who are not clients of an NGO.83  
PLWH Interview content: These interviews will ask about: 
- 
health status and mental health diagnoses, including histories with substance use and 
treatment; 
- 
family and relationship status, housing stability, and sources of support (financial, 
social, emotional); 
- 
knowledge of and experiences with organizations that assist PLWH and PWID; 

D2C Ukraine 
11 
21-Mar-23 
 
Version 1.1 
- 
experiences with these agencies, including how they learned about them, motivation 
for seeking help, eligibility criteria, and application procedures; 
- 
experiences of service denial, perceived or experienced stigma, and treatment by staff; 
- 
ideas about what services are still needed [SEE APPENDIX B]. 
Data analysis Qualitative data management and analysis will be conducted with MAXQDA software. 
All interview recordings will be transcribed, translated, and uploaded to MAXQDA for coding and 
analysis. We will code all documents by document type (e.g., provider or PLWH interview), 
demographic information (city/region, years since HIV diagnosis, gender, socioeconomic status), role 
(director, frontline staff, policy maker). Then, we will create and apply text codes that reflect key 
dimensions of the DAP framework, including provider factors (prior experience with incorporating data 
in service provision, attitudes toward the D2C strategy, educational and training backgrounds); 
organizational factors (leadership buy-in, organizational climate, training and resource capacity); 
system factors (funding, contracts, policies); and client characteristics (comorbidities, prior 
experiences in care, level and type of need). In-depth interviews will be analyzed within the context of 
DAP to identify the system level, organizational, provider, and patient roles and characteristics that 
will inform the adaptation and implementation of the D2C strategy at each site and contextual factors 
that will need to be addressed for successful integration of a D2C strategy into the clinical setting. For 
example, data analysis may determine that HIV-positive PLWH need specific services that 
implementing organizations currently do not offer or that providers lack technical skills to retrieve and 
interpret acuity data. We will address these issues in the finalization of the D2C strategy, including 
identifying referral organizations, and through training, capacity-building, technical assistance 
strategies. We will also use findings from Aim 1 data analysis to match sites for paired randomization, 
as described below in Aim 2.   
Finalize the d2c strategy The finalization activities will include: (1) refining data extraction 
protocols and developing a case management module within the MIS; (2) developing a report 
generation and dissemination protocol for each clinic and (3) specify clinic/implementation team 
structure, roles, and management plans based on findings from in-depth interviews. 
Develop data extraction protocols and create case management module within MIS  
The MIS is designed as an electronic medical record system, to be used by clinical providers for 
routine patient data management. It has centralized architecture, where all data are immediately 
uploaded to a national-level server. According to the feedback we received in our ART OPTI project, 
tracking and prioritization of out-of-care patients is one of the most requested features for the MIS. 
We will partner with the Public Health Center of the MoH, the government body responsible for HIV 
response and coordination of PEPFAR activities in Ukraine and owner of the MIS to create a module 
within the MIS that will: a) identify all HIV-positive PWID meeting the criteria of not-in-care [patients 
registered as HIV-positive who had no HIV medical visits in a calendar year]; b) provide a useful 
summary of all clinical information that will be needed for case managers [drug use, CD4, viral load, 
ART history, HIV care appointments made and kept, co-morbid diagnoses and medications, and MAT 
history]; and c) provide a portal for entry and export of case management data (referrals made, 
appointments kept, contact attempts, case management notes). 
 
 

D2C Ukraine 
12 
21-Mar-23 
 
Version 1.1 
Specific Aim 2: To pilot a randomized clinical trial of a D2C strategy versus 
standard of care  
Study Design  
This pilot study is designed to test the performance characteristics and capabilities of study design, 
measures, procedures, recruitment criteria, and operational strategies that are under consideration for 
use in a prospective RCT. We will also assess feasibility of the RCT by measuring the key parameters 
for sample size calculation, namely the effect size and coefficient of variation.  
D2C is a structural intervention designed to sustainably change the HIV care providers’ practices and 
facilitate information exchange at the facility level. To address the implementation science questions 
by measuring the provider-level outcomes and avoid contamination of the control condition due to 
unblinded allocation of intervention at the patient level, we propose a cluster-randomized trial design 
for this pilot study and the prospective RCT. We will select eligible HIV clinics (see “Study Sites,” 
above) and randomly assign them 1:1 either to the intervention or the control arms. To achieve 
greater comparability between the arms, we will match pairs of eligible sites based on organizational 
context factors derived from Aim 1 data analysis, including facility size and type (e.g., NGO vs. public 
clinic), service delivery structure (e.g., social workers vs. nurses), and existing services for HIV-
positive PWID (e.g. Steps Toward Health) and will randomize within the pairs. This will minimize the 
degree of between-cluster variation within the groups and increase power and precision. 
Randomization will be conducted by the data manager using a computer program. 
Sample size  
We used methodology from Hayes et al84 to determine the effect size that the intervention needs to 
demonstrate in a feasible matched-pair randomized trial. We assumed the cluster size of 75 patients 
(based on the optimal case manager workload) with 5 clusters per arm (potentially manageable 
number given the existing research infrastructure in Ukraine), coefficient of variation between clusters 
within matched pairs equal to 0.15 (empirically estimated based on the data from 56 clinics in the ART 
OPTI study, see Previous Studies), power of 80% and 0.05 significance level. Proportion of 
participants re-engaging into ART within 3 months after inclusion into the study was used as the key 
outcome. Under the assumptions, the relative difference between the study arms should be at least 
22% (e.g. 50% vs 72%). The adjacent chart shows the 
power and sample size of a pilot trial needed to detect 
this difference with 0.05 significance level. To achieve 
80% power, we will need to recruit 160 participants, or 
80 per arm. We propose to divide the sample equally 
between 4 sites (2 per arm) to test the matching 
procedures. We do not adjust for loss-to-follow-up 
because for our main outcomes (ART initiation, viral 
suppression), loss to follow-up is equal to not achieving 
the outcome.  
Intervention Overview  
Clinics randomized to the intervention arm will implement the D2C strategy as described above, 
including Health Data Information, Active Surveillance, and Case Management with the addition of the 
data component. The D2C strategy will involve a 5-step process: (1) identify not-in-care PLWH using 
the MIS, (2) verify eligibility criteria, (3) contact patients and invite to visit the clinic, (4) determine 
care status and reengage into care, and (5) provide case management services and confirm 
engagement in care.12  

D2C Ukraine 
13 
21-Mar-23 
 
Version 1.1 
Steps 1 and 2 will follow the same procedure at both intervention and control sites to ensure equal 
conditions of study enrollment. Step 3 will be conducted using Active Outreach at the intervention 
sites. Steps 4 and 5 will be achieved through use of an Acuity Tool and derived Acuity Score. The 
Acuity Tool will be used to identify barriers to HIV care engagement, including mental health and 
substance abuse, and to establish referrals to other services, as described above in Overview of Data 
to Care Strategies. 
Procedures 
Patient selection and verification of inclusion criteria 
A Recruiter (clinical staff authorized to access MIS at each site) will use the D2C module in the MIS to 
access the “Tracking Journal” which includes patients who have a confirmed HIV diagnosis, started 
ART, and missed their clinical appointments (medication pick-up or other). Going through the list 
(‘New cases’ or ‘In progress’ or ‘Not tracked’ sections), Recruiter reviews the MIS data (including in 
other modules) and identify patients who meet the following criteria: 
a) registered within the MIS as having received a positive HIV diagnosis at any time in the past 
b) missed a clinical visit (ART pick-up or other) more than 7 days ago 
c) registered with IDU as probable mode of HIV transmission, or history of IDU documented at 
any clinical visit 
d) not been contacted by other clinical staff after the current missed visit 
e) 18 years or older 
For each patient meeting these criteria, Recruiter puts an “E” letter in the beginning of the Comment 
field in the Tracking Result section (before any other text in this field, separated by a space). This 
letter indicates that the patient is eligible for recruitment into the study and filters the Tracking 
Journal to display only these patients. 
Patient outreach and verification of exclusion criteria 
Next, using the contact information available in the MIS and paper documentation, Recruiter attempts 
to contact the patients in the E-list by phone using the procedure described in the “Tracking SOP” 
developed by the Public Health Center. If the contact is successful, Recruiter first has to make sure 
that he is talking to the intended patient, without revealing that he is calling regarding HIV treatment. 
Once the patient (or a trustee) identity is confirmed, Recruiter asks about the reasons of the missed 
visit, and assesses the following exclusion criteria and takes corresponding actions: 
o Patient is already re-engaged in care and visited the clinic recently 
▪ 
In this case Recruiter verifies the MIS clinical records and updates if necessary 
o Patient has sufficient supply of medications (available to him for any reason) 
▪ 
Recruiter asks about the amount, and records the date of the next visit 
accordingly 
o Patient moved to another clinic, city, country, or penitentiary institution 
▪ 
Recruiter provides information about clinics where the patient may continue 
treatment and initiates the formal patient transfer procedure 
o Patient died 
▪ 
Recruiter initiates the de-registration procedure 
In these cases, Recruiter adds a “X” letter to the Comment field (to make “EX”). All phone contact 
details are recorded in the MIS. 

D2C Ukraine 
14 
21-Mar-23 
 
Version 1.1 
If the contact is successful and exclusion criteria are not met, Recruiter explains why it is important to 
continue treatment without interruptions and informs the patient that he or she is eligible to enroll 
into a research study, using a standard script (Appendix 1, two versions for the study arms). 
Regardless of the interest in the study, Recruiter schedules patient’s next visit at the earliest 
convenience, and records the date in the MIS. In these cases, Recruiter adds a “L” letter to the 
Comment field (to make “EL”).  
If the first contact is unsuccessful, Recruiter adds a “R” letter to the Comment field (to make “ER”). 
The following contact attempts are conducted according the “Tracking SOP” – minimum three 
additional attempts with 1-2 weeks apart and engaging social workers and other projects if available, 
with additional invitation to participate in the study using the script. If the patient is reached but 
refuses to come to the clinic during the first contact, the status is also updated to “ER”. Additional 
three attempts to contact and convince the patient to come are made following the same procedure. 
The status of the patient remains as “ER” until he agrees to come to the clinic, or the final (fourth) 
attempt is unsuccessful. After four unsuccessful attempts (with or without contact with the patient) 
the status is changed to “ERZ”. 
At the Intervention sites, Recruiter exports the ER-list and gives it to a Case Manager (through Data 
Manager). The list includes, if available, Syrex and Case++ codes (alphanumeric unique IDs used by 
prevention and care and support NGOs) to enable outreach through NGOs. Case Manager at the 
Intervention sites undertakes additional efforts to reach and recruit the patients using all available 
methods (phone calls, home visits [following the procedures outlined in the “Tracking SOP”], 
community and social networks, harm reduction and HIV-service NGOs). Up to three contact attempts 
(bi-weekly) will be made for each participant. The dates and results of the attempts are recorded in 
the online Recruitment Log (developed based on the ER-list, Annex 2). In case of a successful contact, 
Case Manager assesses the exclusion criteria described above, invites to participate in the study, and 
schedules the next visit. Recruiter monitors the Case Manager’s contact attempts in the Recruitment 
Log bi-weekly and transfer information to the MIS, including the updates to the E-status (from “ER” to 
“ERX” or “ERL”). After three unsuccessful attempts by Case Manager (four total including the initial 
attempt by Recruiter) the status is changed to “ERZ”. 
Additional contacts with the patients on the EL-list may be made by Recruiter and/or Case Manager on 
the day before the scheduled visit to confirm the appointment and ensure the patient remembers 
about the visit. 
First visit 
The patients on the EL-list should be instructed to notify Recruiter and/or Case Manager before 
coming to the clinic, to expedite the visit procedures. If a patient from the EL-list comes to the clinic 
without notifying Recruiter or Case Manager, a receptionist should notify Recruiter and direct the 
patient to see them. 
After meeting the patient, Recruiter conducts (or refers to) regular clinical visit procedures, refers the 
patient to Interviewer, and changes the E-status to “E(R)S”. Interviewer conducts the Informed 
Consent procedure according to the study-specific SOP and using the consent form (Annex 3), and in 
case of consent performs baseline interview (see below). At the Control sites, Interviewers fill out the 
Contact Information Form (Annex 5) to enable tracking for follow-up interviews. 
After the interview, the patient comes back to Recruiter, who makes referrals to available care and 
support services (Standard of Care sites) or introduces the patient to Case Manager (Intervention 

D2C Ukraine 
15 
21-Mar-23 
 
Version 1.1 
No contact 
Four 
unsuccessful 
attempts 
Exclusion criteria met 
sites). At the Intervention sites, Recruiter fills out the top section of the Case Management Checklist 
(Annex 4) and gives it to the Case Manager together with the patient. 
Case Manager may conduct the first intervention session on the same day or schedule a meeting with 
the patient at the earliest convenience. 
 
Figure. Patient status update workflow 
Intervention Procedures 
The case-management component will be implemented according to a Manual, developed by the 
study team. Briefly, the content and intensity of case management will be based on an overall Acuity 
Score across domains. Case Managers will assess clients every 3 months to determine the treatment 
plan and record this information in the Case Management forms to enable progress monitoring. Care 
teams of nurses, case managers, and outreach specialists will coordinate holistic wrap-around care to 
meet the social service and health needs of out-of-care patients, many of whom face substance abuse 
and mental health barriers to HIV care. Case managers will develop client-centered service plans 
based around “SMART” objectives (Specific, Measurable, Achievable, Relevant, Time-limited).  
Case Managers will schedule new clients for intake appointments and work with clients to develop 
treatment plans. Specific duties include:  
- 
use Acuity Tool to assess patient-identified financial, socioeconomic, and housing profiles 
and provide comprehensive internal or external referrals for supplemental or long-term 
resources;  
- 
develop and implement personalized treatment plans for all clients based on the acuity 
assessments through existing case management programs; 
- 
ensure accurate tracking and documentation of all interactions and activities with patients 
using the acuity tool and Tracking Module within the MIS;  
- 
establish, develop, and maintain professional relationships with diverse treatment and 
service providers, including AIDS care, primary care, substance use, mental health, and 
social services; 
- 
maintain current contact information using the Contact Information Form (Annex 0) and 
updates it at each visit. 
Additionally, Case Managers assist Interviewers to schedule follow-up interviews (see below) in 
conjunction with the Intervention sessions (at 6 and 12 months). 
 
E 
Potentially eligible 
E(R)L 
Eligibility confirmed, 
visit scheduled 
ER 
Recruitment in 
process 
E(R)X 
Excluded 
E(R)S 
Visit successful 
ERZ 
Recruitment 
unsuccessful 
Visit scheduled 

D2C Ukraine 
16 
21-Mar-23 
 
Version 1.1 
Control Condition  
Clinics randomized to the standard of care arm will continue to implement their client identification 
processes and case management as usual. One of the most common interventions with a focus on 
PWID is Steps Toward Health,85 a case management program developed and tested through the 
PEPFAR-funded RESPOND project.8,86 It is theory-driven and standardized in manuals that contain 
background information, logic models, client assessment worksheets, step-by-step instructions for 
each session, and monitoring and evaluation guidelines. This 5-session intervention includes modules 
on self-care, HIV transmission risk reduction, drug treatment, and ART adherence. It does not include 
active outreach, client tracking, or tailored treatment components. Other possible interventions 
include Community-based Retention Intervention, Patients’ School, and others. 
Provision of these interventions (the fact of enrollment) will be recorded in the MIS by Recruiter, but 
the actual process data will not be collected. 
Data Collection  
Data on primary outcomes will be obtained from clinical records in the MIS and absence of data will 
be equal to not achieving the outcome, therefore the effective sample size will be exactly 160 
patients. Data from the MIS will be extracted monthly (Annex 6). The primary outcomes are defined 
as: 
- 
Engagement in HIV care: at least 3 clinical visits over at least 6 months within 12 months after 
study enrollment.  
- 
ART (re-)initiation: initiating or re-initiating ART within 3 months after study enrollment. 
- 
ART adherence: the percentage of days in possession of medication (pharmacy refill gap 
method)87 
- 
HIV Viral Suppression: having at least one VL test with <200cp/ml and none with >=200cp/ml 
within 12 months after study enrollment.  
Data on secondary outcomes will be collected using the patient survey at baseline, 6 and 12 
months after enrollment. The secondary outcomes will include items in the following domains:  
- 
Substance Use: Substance use battery from the Ukrainian IBBS studies, AUDIT-C; DAST-10 
scale for addiction severity. 
- 
Trust in Physician Scale 
- 
Substance Use & HIV Stigma Mechanism Scales 
- 
Readiness for Drug Treatment Scale 
- 
Mental Health: PHQ-9  
- 
Risk behaviors: Risk Behavior Assessment (RBA), modified for the drug use patterns of PWID 
in Ukraine. The RBA assesses drug use and sex-related HIV risk behaviors.  
- 
Quality of life: Short Form-12 (SF-12), a global measure of patient quality of life; HRQoL, HIV-
related quality of life 
- 
Medical co-morbidities and engagement in ancillary care: (OAT, TB, STI and Hepatitis 
treatment) by self-report. 

D2C Ukraine 
17 
21-Mar-23 
 
Version 1.1 
- 
HIV Treatment Satisfaction Questionnaire 
- 
Patient-side cost questionnaire 
- 
Socio-demographic information, including age, housing status, sex/gender, family/marital 
status, income, education, employment status. 
The surveys will be administered by an experienced Interviewer, hired at each site. The questionnaire 
will be programmed using REDCap platform installed on a UIPHP server. 
Intervention implementation  
At all intervention sites, we will document all implementation processes through quarterly 
implementation site meetings. Each quarter, the implementing teams at each intervention site 
(consisting of outreach specialists, clinic managers, case managers, and other service providers) will 
generate benchmark reports using the MIS and review the comprehensive clinic specific Health 
Information. The report will include both absolute numbers around key metrics (e.g., number clients 
linked to care from those identified as out of care, client distribution by acuity level, missed appoints) 
and areas of strength and in need of improvement (e.g., increasing efforts to locate not-in-care 
patients, continuing referrals to mental health services). In consultation with the investigators, the 
implementing teams at each site will review the report, analyze gaps in service delivery, and identify 
specific strategies for quality improvement. Implementing teams will be tasked with making concrete 
plans to improve or modify practices and assessing and re-assessing their goals and progress toward 
these goals.96,97 Following presentation of the report, the team will discuss: the areas the agency 
wants to prioritize for improvement over the next quarter; strategies for strengthening or developing 
new workplans; identifying the key people to involve in order to implement the change; and 
developing an action plan. These consultations will be recorded on a standardized consultation form, 
which will be analyzed as part of the feasibility and acceptability evaluation within the CFIR-CS 
framework. These meetings also correspond to DAP phases 3 (Implementation) and 4 (Sustainment) 
in order to incorporate feedback processes into implementation of the D2C intervention.13  
Statistical Analysis and Feasibility Assessment  
We have powered this pilot study to evaluate feasibility of a prospective cluster RCT. The RCT will be 
considered feasible if the difference in the proportion of patients achieving the primary outcome 
(engagement in HIV care) will reach 22% between the study arms. We will estimate the impact of the 
intervention on the HIV care continuum using mixed effects models. For all outcomes, the data will 
have the same structure, with observations (t) nested within participants (i) nested within sites (j) 
nested within matched pairs (k). The correlations among observations will be addressed by using 
fixed-effects indicators for matched pair and normally distributed random intercepts for sites and for 
participants. Our mixed-effects models will include indicators for each of the two follow-up 
assessments, an indicator for site group assignment, and two group-assignment-by-follow-up 
interaction terms. We will also include sex/gender, age, and baseline covariates that have been 
identified in preliminary analyses as varying significantly across sites. A logit link function will be used 
for the primary outcome of ART (re-)initiation (defined above), and the coefficients for the two group-
assignment-by-follow-up interaction terms will provide the estimates of the subject-specific 
intervention effects at 6 and 12 months post-baseline. We will examine the effects of sex/gender, age, 
substance use severity, ART history at baseline and other covariates in the multivariable models.  
We will characterize patterns of behavior that impact HIV transmission (episodes of syringe sharing 
and episodes of unprotected intercourse in the prior 30 days without viral suppression) and we will 
use the same mixed-effects approach described above but with a log link function and negative 

D2C Ukraine 
18 
21-Mar-23 
 
Version 1.1 
binomial distribution. These analyses will provide parameters for estimating the rate of onward HIV 
transmission for the cost-threshold analysis. All analyses will include the use of the mixed effects 
“meqrlogit,” “meologit,” and “menbreg” procedures using Huber/White robust variance-covariance 
estimators. We will examine whether missingness is associated with observed data using standard 
methods.98–100 We will conduct a second set of outcome analyses using multiple imputation to 
evaluate the robustness of the primary outcome analyses to missingness and report estimates based 
on multiple imputation that would be valid based on the assumption that the data are missing at 
random (MAR). We will produce imputed datasets (m > 10) using chained equations,101 run outcome 
analyses on all sets, and derive point and standard error estimates for treatment effects and other 
parameters from the distribution of estimates from the outcome analyses with imputed datasets.  
We will also evaluate the use of mixed proportional odds regression102,103 for determining whether 
intervention exposure is associated with an individual’s location on five distinct stages of HIV care 
continuum: (1) not engaged in HIV care in the prior six months; (2) engaged in HIV care in the prior 
six months but no ART use in the prior 30 days; (3) ART use in the prior 30 days but no evidence of 
viral suppression in the prior 6 months; and (4) ART use in the prior 30 days with evidence of viral 
suppression in the prior 6 months. The HIV care continuum can be viewed as an ordinal categorical 
variable, and ordinal logistic regression will be used to evaluate if group assignment is associated with 
the location along the HIV care continuum. We will use the Score test to evaluate whether the 
proportional odds assumption of ordinal logistic regression.104  
These analyses will be essential for planning a full study. The findings will be used for sample size 
estimates, and data on HIV transmission risk will be used to determine whether the intervention could 
reasonably be cost-saving or cost-effective. This pilot analysis will provide important information on 
the feasibility of using ordinal logistic regression in a larger trial or in other analyses of the HIV care 
continuum. 
 
Specific Aim 3a: To assess the feasibility, acceptability, and implementation-
related processes and outcomes of the D2C strategy 
We will use the Consolidated Framework for Implementation Science adapted for complex 
systems (CFIR-CS) to guide implementation evaluation of the D2C strategy.15,105,106 Key aspects of 
the D2C intervention are that it covers multiple settings (i.e., patients must access and interact with 
providers in many different health care sectors), its success depends on interactions and 
communication between organizations (e.g., between case managers, addiction treatment specialists, 
and HIV treatment providers); and it uses multiple types and sources of data to identify patterns in 
services and outcomes that will be used to improve or modify practices. The CFIR-CS organizes 
implementation factors into 5 domains: intervention characteristics (accuracy of data, timeliness 
of data sharing with partner agencies, development of practice change action plans based on data), 
organizational characteristics (integrating D2C strategy into existing protocols, agency-wide buy-
in, commitment to D2C strategy), providers’ roles and characteristics (case managers’ previous 
experiences, attitude toward the use of data in care provision, relationships with other medical and 
social service providers, time and resources devoted to providing case management for PLWH), 
patients’ roles and characteristics (type of medical and social needs, level of need), and 
implementation processes (participating in the monitoring and practice improvement process, 
using MIS data to inform practice change). The implementation of the D2C strategy is affected by the 
external context in which these organizations and their staff operate (economic, social, and political 
factors that may affect implementation process such as MAT “slot” and psychosocial support service). 

D2C Ukraine 
19 
21-Mar-23 
 
Version 1.1 
Measures of implementation are the intermediary outcomes107 that describe how well the 
implementation was carried out and include use of the acuity tool with new and returning clients, and 
use of data generated from acuity tool and MIS to modify practices, documentation of patient visits 
and contacts. Implementation influences outcomes, which include patient outcomes (viral load, 
appointment kept); provider outcomes (uptake, consistent use, patient contact, outreach efforts, time 
spent with patients); and processes of care (recruitment rate, extent to which intervention tracks 
patients and follows up on and coordinates tests, referrals, and care across organizations, settings, 
and acuity tool domains). 
Acceptability is defined as the perception among implementers that the D2C strategy is agreeable and 
satisfactory.108 Feasibility is defined as the extent to which the D2C strategy can be successfully 
carried out in a given context.108 Information to assess the feasibility, acceptability, and processes of 
D2C implementation will come from 3 sources: D2C usage data (quantitative), implementation 
meetings (qualitative), and practice improvement strategies at intervention sites (qualitative), and in-
depth follow-up interviews (qualitative) with providers, other stakeholders, and PLHW.  
Data-to-Care Usage Quantitative and qualitative process outcomes measures include but are not 
limited to: 
MIS Usage: Measures of implementation and intervention uptake will include frequency of accessing 
MIS data, frequency of contact attempts, and efficiency of the contact (i.e., how many contact 
attempts were necessary before a participant meets with the case manager). Case managers will 
record this information in a “Sessions Notes” section of the case management module within the MIS.  
Outreach: As part of their responsibilities, case managers will take notes that summarize each 
session with clients, including techniques employed to contact and track participants, link participants 
to services, and retain participants in case management. These data will be included in the “Session 
Notes” section. 
Meeting notes and benchmark reports: Intervention site meeting notes (“Intervention 
Implementation,” above, will provide information about clinic-level barriers and solutions to 
implementing the D2C strategy. 
In-depth Interviews In-depth interviews will be conducted approximately 6 months after 
participants are contacted for case management, and within 1 month of the intervention conclusion 
for providers.  
In-depth interviews with D2C implementers Follow-up interviews with directors and case 
managers across study sites will explore the overall effectiveness of the D2C intervention in 
connecting not-in-care clients to necessary services, incorporation of the D2C program into workflows; 
experiences with the D2C program and how the program has changed the service delivery context, 
including how case managers liaise between providers and facilitate enrollment processes; perceived 
feasibility of the D2C program, including ease of implementation, practicality, and viability;108 and 
training and leadership required for successful implementation.109 Specific questions will be asked 
about usability and user satisfaction with data entry systems, including perceived complexity, ease of 
use, confidence using the system, added value to case management activities, and training and 
support required to use the system efficiently.110 
In-depth interviews with D2C participants We will interview up to 20 participants recruited 
through a purposive sampling strategy to obtain a maximum variation sample by strategy arm, 
linkage-to-care status, age group, and gender. We will attempt to interview participants who did not 

D2C Ukraine 
20 
21-Mar-23 
 
Version 1.1 
re-engage or re-engaged and dropped out, using recruitment strategies described under Aim 2. 
Interviews will explore: experiences of the care strategy they were assigned to, interactions with case 
managers and other service providers, family and relationship status, housing stability, and sources of 
support (financial, social, emotional), experiences with HIV and other medical and social service 
providers, ideas about how to improve the programs. 
Data Analysis Qualitative and quantitative process and outcomes data will be analyzed according to 
domains within the CFIR-CS to identify the case manager, organizational, and other contextual factors 
for successful implementation of the D2C strategy. Quantitative assessment of feasibility will explore 
MIS usage and results based on the 7 D2C steps (describe above, “Intervention Condition”), including 
use of the MIS to contact participants, extent to which case notes are recorded and updated with the 
MIS, and whether linkage/engagement status is entered into the surveillance system. Review of case 
manager use of the session notes tool within the MIS will assess the extent to which case managers 
record required information (e.g., outreach efforts). Errors (such as missing or incomplete data) will 
be used to inform future training efforts or changes to the module to improve usability (e.g., 
incorporation of task reminders or completion checklists). Qualitative assessment of feasibility and 
acceptability will take place through systematic analysis of implementing sites experiences with the 
intervention, including their benchmark reports. We will create and apply text codes that reflect key 
analytic concepts drawn from the CFIR-CS, including organizational climate, attitudes toward the use 
of data, time and resources devoted to providing case management, and engagement in and 
commitment to the PDSA process. We will create document codes that reflect each agency’s overall 
success with the intervention (based on primary and secondary patient outcomes) to facilitate cross-
site comparison and identification of barriers, facilitators, and processes that lead to intervention 
uptake and implementation. Qualitative analysis will focus on how case managers incorporated data 
into their overall workflows, the extent to which using the Acuity Tool facilitated identification of areas 
of need among clients and linkage to the appropriate services, barriers to entering data into the MIS 
portal and using this data, and systems of collaboration and referral between case managers and 
provider types. Analysis will also explore case managers’ experiences working with new clients 
identified through the active surveillance system and how this intervention transformed their practice, 
including how different clinics organized implementation teams, role differentiation, communication 
strategies, and workload. Findings will inform how the D2C strategy can be adapted for broader 
implementation in different clinical and organizational contexts (e.g., Regional AIDS Center, NGOs that 
provide case management and other services for PLWH) and what organizational, policy, and capacity 
structures and processes are necessary for implementation.  
Specific Aim 3b: Economic Analyses To assess the costs and cost-threshold of the 
D2C strategy 
In accordance with the recommendations of the 2nd U.S. Panel on Cost-Effectiveness in Heath and 
Medicine,111,112 we will: inventory and value the resources consumed in the intervention; estimate 
intervention effectiveness in regards to HIV infections averted; estimate treatment costs averted and 
QALYs saved; and evaluate conditions under which the intervention would be cost-saving or cost-
effective. 
Cost analysis: We will estimate the cost of delivering the program locally from a societal perspective 
that incorporates broader costs, including non-medical costs to participants and costs reflected in the 
recently recommended impact inventory.112 We will use micro-costing113,114 to directly enumerate 
every input used in the intervention such as staff time spent on each intervention activity, equipment, 
and materials. Any in-kind contributions will also be enumerated and costed. We will use both top-

D2C Ukraine 
21 
21-Mar-23 
 
Version 1.1 
down and bottom-up approaches,115 and sources of data will include project records, salaries, cost 
worksheets, project manager interviews, and a project manager survey. The program manager will 
complete a survey to quantify per unit costs for program resources, including procedure-specific 
resources, general resources, fixed resources, and variable resources. Participant time, travel, and any 
child or elder care will be measured and costed based on prevailing local wage. Cost data for each 
type of resource will include “best estimates” for cost and units consumed, and credible ranges to be 
used in sensitivity analyses will be established by asking program staff to provide upper and lower 
bounds for any uncertain cost estimates.  
Effectiveness, medical costs, and QALYs: Few HIV prevention studies are sufficiently powered to 
directly measure reductions in infections, and we will estimate HIV incidence among sexual and 
injection-drug-using partners under the standard of care scenario and under the intervention scenario 
using Bernoulli process models116 of injection drug use and sexual intercourse with behavioral patterns 
based on participant assessments and HIV transmission probabilities based on an updated review of 
the literature. The model will be based on participant data with a one-year timeframe for intervention 
costs and HIV transmission, and we will incorporate literature-based estimates of lifetime treatment 
costs and QALYs associated with new HIV infections in Ukraine, discounted at 3% per annum.111,117 
Differences between the scenarios in estimated HIV transmission are expected to be driven primarily 
by ART use, but we will also incorporate observed data on intercourse, condom use, injection drug 
use, syringe sharing, and partner characteristics. The primary model will be based on the point 
estimates of parameters, but we will also conduct sensitivity analyses where input parameters are 
sampled from distributions that capture the uncertainty in our estimates. Many modeling approaches 
are available for modeling the incremental cost-effectiveness of D2C; given the exploratory nature of 
the proposed research, we believe a Bernoulli process model is appropriate. 
Incremental cost-effectiveness and threshold analyses: The cost-utility ratio will be computed 
based on the estimation of net incremental cost (i.e., difference between the incremental intervention 
costs and the incremental HIV treatment costs averted) divided by the QALYs averted. For a given 
model run, intervention will be considered: cost-saving if the incremental intervention cost is less than 
the incremental treatment cost averted; cost-effective if the net incremental cost per QALY saved is 
less than society’s willingness to pay to save one QALY; and not cost-effective if net incremental cost 
per QALY is greater than society’s willingness to pay.111 The proportion of model runs that are cost-
effective or cost-saving will be used to characterize the probability that the intervention meets these 
thresholds. We will also use one-way sensitivity analyses to identify conditions under which the 
intervention would be 90% likely to be cost-effective or cost-saving, such as specific intervention cost 
thresholds and intervention effectiveness thresholds. 
Adverse events (including suicide risk) 
The question 9 in PHQ-9 instrument may indicate suicidal thoughts among participants (“Thoughts 
that you would be better off dead or of hurting yourself in some way”). In case the participant 
responds anything except “not at all”, Interviewer asks an additional question: “In the last two weeks, 
have you had any thoughts of hurting yourself in some way?” (not at all, several days, more than half 
the days, nearly every day). If patient responds “not at all” to this question, the patient denies active 
suicidal thoughts. He or she is considered “very low or no risk” and no action is triggered at this point. 
If any other response is received, Interviewer will escort the patient to a clinician in the clinic for a 
formal assessment of suicide risk according to the national clinical standards (includes an assessment 
using the Columbia Suicide Severity Rating Scale). For those with a plan, the guidance stipulates that 

D2C Ukraine 
22 
21-Mar-23 
 
Version 1.1 
patients will be transport emergently offsite to a psychiatric inpatient hospital for further evaluation 
and may be detained against their will for up to 72 hours. 
Data management 
All paper-based forms collected as part of study (informed consent, contact information forms, case 
management forms) will be stored at the local study sites in locked cabinets. After completion of data 
collection, all paper forms will be shipped to UIPHP and stored for three years, after which they will be 
destroyed. 
The questionnaire will be completed on laptops/tablets provided to the participant by the Interviewer. 
The questionnaire will be programmed on REDCap online platform, which has sufficient functionality 
for data quality assurance, monitoring, and protection. The REDCap employs various methods to 
protect against malicious users who may attempt to identify and exploit any security vulnerabilities in 
the system. REDCap users have access only to data and information that they are supposed to have 
within the application by identifying user privileges. Each user has their own account, and the user 
account will only have access to REDCap projects that they created themselves or to projects which 
other users have granted them access. REDCap contains an auto-logout setting, which is customizable 
(default auto-logout time is 30 minutes), and will automatically log a user out of the system if they 
have not had any activity (e.g. typing, moving the pointer) on their current web page for the set 
amount of time. This prevents someone else from accessing their account and project data if they 
leave a workstation without properly logging out or closing their browser window. REDCap maintains a 
built-in audit trail that logs all user activity and pages viewed by every user, including contextual 
information (e.g., the project or record being accessed). Therefore, the Data Manager will be able to 
monitor any activity in the project, e.g. entering data, exporting data, modifying a field, running a 
report, or add/modifying a user. 
All electronic information will be stored in password-protected computers with double-password 
protection for opening specified files, as compliant with the US Health Insurance Portability and 
Accountability Act (HIPAA). All study logs will be recorded with participants’ codes as electronic 
records only and maintained in corporate cloud services (Google Drive for Business). Programs and 
software for data collection will only be available to be opened by the investigators. Electronic 
databases and analytic tools will be maintained through password-protected computers and files and 
maintained by the UIPHP IT support. Access permissions for storage locations will be provided to 
authorized personnel only. The research data will be stored for three years after completion of the 
study. 
Human subjects 
Ethical Review 
All investigators will have research ethics training certification. The study protocol will be submitted 
for ethical review to the Institutional Review Board (IRB) at the Ukrainian Institute on Public Health 
Policy (Kyiv, Ukraine). 
The Investigator will make safety and progress reports to the IRB within three months of study 
termination or completion.  These reports may include the total number of participants enrolled in the 
study, the number of participants who completed the study, all changes in the research activity, and 
all unanticipated problems involving risks to human subjects or others. 
Qualitative data collection  

D2C Ukraine 
23 
21-Mar-23 
 
Version 1.1 
Informed Consent 
For each IDI, potential participant informed consent forms will be provided by the Interviewer and 
written informed consent will be obtained. All questions that candidate may have will be adequately 
clarified and explained to them. Potential participants will be informed that their participation in the 
study is completely voluntary and that they have a right to withdraw their consents and stop their 
participation in the study at any time. Refusal or withdrawal from the study at any time will have no 
effect on the participant’s employment or access to health facilities. Participants will be informed that 
any information that they disclose during the course of the study will be considered confidential (i.e., 
no personal identifiers will be used and only aggregated information across all participants will be 
reported). Participants will have the potential risks and benefits of the study explained to them as 
well. For all candidates, understanding of the essential parts of the document will be assured by 
asking questions to verify their knowledge of the study procedures.  
Benefits  
There is no immediate benefit to study subjects. However, knowledge obtained in this study can help 
improve programmatic objectives and benefit the community by helping to improve HIV services 
provision. 
Compensation 
IDI participants will receive compensation for their time in the amount of 300 Ukrainian Hryvnas 
(UAH).  
Risks Associated with Loss of Confidentiality 
As with all research, there are risks. As there are no treatment procedures or interventions in this 
protocol, the only risk is a potential breach of confidentiality. The participation in IDIs is considered a 
study procedure that is associated with minimal risk for participants, the only risk is of potential 
breach of confidentiality. The potential sites for breaches of confidentiality include the recruitment 
process, interview itself and/or at data management systems. While the questions in the IDI guides 
do not explicitly raise the specter of risk, Key informants may respond with potentially sensitive 
information and/or emerge as topics of the discussion during the IDIs, about various aspects of 
patient management practices, stigma and discrimination in medical care, or other sensitive topics.  
Protection Against Loss of Confidentiality 
Extensive measures will be taken to ensure confidentiality. After Informed Consent is obtained, IDI 
participants will be assigned a study code which is the only identifier in the study database. Key 
informants will be assigned Study Participant Code containing no identifiable information and 
consisting of geographical area abbreviation, professional degree and research serial number.  
If during IDIs sensitive topics emerge and there is further inquiry through targeted probes during the 
IDIs, every effort will be made to ensure that the level of risk to participants does not exceed minimal 
risk. We seek to reduce risk using several strategies often used in such settings – these include: a) 
asking participants never to use personal identifiers (e.g., someone’s name); b) to discuss 
observations that they “deem” sensitive or potentially risky for them in a general way, in the third 
person, in terms of their “observation of others” or “stories they have heard”, and refrain from 
providing any names or any other identifying information. We will clearly stipulate this as part of the 
Informed Consent, pre-amble to the IDI guide and will reinforce it during the IDI conduct.  In general, 
the risk for Key informants is minimal. We will take measures to reduce this risk and will carefully and 
frequently monitor this.  

D2C Ukraine 
24 
21-Mar-23 
 
Version 1.1 
Prior to the start of each IDI the interviewer will also explain that the information that is discussed will 
remain anonymous and that all the information and views shared by the participant will be kept 
confidential. The participants will be instructed that they may choose to withdraw from the study at 
any point and decide not to complete the IDI activities if they feel participation causes them any 
distress. All participants will be able to choose a pseudonym they would like to be called throughout 
the study and assigned an anonymous identification code.  
Quantitative data collection 
Informed Consent 
Written informed consent will be obtained from each study participant. One informed consent form 
(ICF) version will be used for both study arms. 
The ICF will be read aloud to each eligible participant by the Interviewer. All questions that 
respondents may have will be adequately clarified and explained to them. Potential participants will be 
informed that their participation in the intervention is completely voluntary and that they have a right 
to withdraw their consent forms and stop their participation in the study at any time. Refusal or 
withdrawal from the study at any time will have no effect on the participant’s access to health facilities 
or HIV-related care and treatment. Participants will be informed that any information that they 
disclose during the course of the study will be considered confidential (i.e., no personal identifiers will 
be used and only aggregated information across all participants will be reported). Participants will 
have the potential risks and benefits of the study explained to them as well. For all candidates, 
understanding of the essential parts of the document will be assured by asking questions to verify 
their knowledge of the study procedures. If a candidate agrees to participate in the study, the 
candidate, or a witness if a candidate is illiterate, will document provision of informed consent by 
signing on the consent form. Unsigned copy of consent forms will be offered to the participants. 
Potential risks 
Participation in HIV care and treatment in Ukraine is not associated with any punitive or social risks 
with the exception of common stigma. This study does not increase the risk of stigma associated with 
drug use or HIV for participants. 
Protection Against Risks 
Participation in the study is completely voluntary and confidential. The unique identifier code, linking 
all study records is based on information provided by participants verbally, and no documents are 
requested. It is not possible to identify a person using this code. Contact information will be requested 
to allow invitation for follow-up assessments, and this information will be kept strictly confidential (see 
below). Mobile phone numbers in Ukraine in most cases are not personally identifiable information, 
because prepaid numbers are not linked to any contract or personal registration data. 
The study personnel will explain to participants that the information collected in the study may be 
accessed by the law enforcement entities only in case of an individualized request with regard to an 
open criminal case. 
As part of the informed consent procedure, all potential participants will be instructed that they do not 
have to disclose personal information which they are uncomfortable sharing and that they can 
withdraw from the study at any time. 
Participants will receive names and contact information of the organizations conducting the study, as 
well as other study team members, on their copy of the ICF. It will be explained to the participant that 
they may contact anyone on the list if they have any questions or comments regarding the study, if 

D2C Ukraine 
25 
21-Mar-23 
 
Version 1.1 
they suffered in any way due to their participation/failure to participate in the study. The participants 
will be informed that they may refuse to answer any questions that make them feel uncomfortable 
and that participation in the intervention is voluntary. Absolutely no information on any participants of 
the study will be given to other participants. 
To minimize psychological risk, the recruitment into the study will be done by a trained social worker, 
and interview will be conducted by a trained interviewer. The interviews will be carried out face-to-
face and there will be no additional persons in the room during the interview except the interviewer 
and respondent. Study staff will also be equipped to refer participants who need additional counseling 
and assistance to community resources, including referrals for HIV care and treatment. 
Compensation 
Participants will be compensated for their time and effort in this study, and/or be reimbursed for 
travel to study visits and time away from work. The compensation will be 400 UAH for each fully 
completed questionnaire (at baseline, 6 and 12 months), which will make 1200 UAH total if all three 
are completed. The compensation will be provided after each interview. This information is also 
specified in the ICF. 
Confidentiality 
All study-related information will be stored securely at the study sites and in the main office. All 
participant information will be stored in locked file cabinets in areas with access limited to study staff. 
All laboratory specimens, reports, and study data-collection, process and administrative forms will be 
identified by a coded number to maintain participant confidentiality. All records that contain names or 
other personal identifiers, such as locator forms and informed consent forms, will be stored with 
restricted access according to local SOP. All local databases will be secured with password-protected 
access systems. 
Participant study information will not be released without the written permission of the participant, 
except as necessary for monitoring by the Sponsor, government and regulatory authorities, and/or 
site IRBs. 
Protocol Deviations 
All protocol deviations, new/unexpected findings and changes to the study environment will be 
documented and immediately reported to the in-country study team who will then notify the Sponsor. 
If necessary, a formal report will be sent to the appropriate IRBs. Reporting of such incidents will be 
the responsibility of the Principal Investigators of this study. Any discussions, issues, and complaints 
related to the study will be reviewed promptly to ensure close monitoring of the impact of the study 
on participants. Appropriate action will be taken to resolve or deal with all issues accordingly. 
Dissemination, notification and reporting of results 
Notifying Participants of Study Findings 
Written material summarizing the findings from this study will be made available to participants and 
study staff upon completion of the study. 
Dissemination of Study Findings 
In-country data and country-specific information will be made available to national policymakers, 
organizations, and implementing partners as soon as possible. Study staff will be notified of the 
findings upon their presentation or publication.  

D2C Ukraine 
26 
21-Mar-23 
 
Version 1.1 
The study findings will be disseminated through presentations and publications in peer reviewed 
journals and other publications. Reports will be disseminated to international audiences, as well as to 
national and local stakeholders to assist in HIV treatment program planning. Any formal presentations 
at conferences or scientific publications will follow JHU and UIPHP procedures for publications and 
presentations. 
 
 
 

D2C Ukraine 
27 
21-Mar-23 
 
Version 1.1 
Bibliography 
1.  
UNAIDS. Miles to Go - The Response to HIV in Eastern Europe and Central Asia.; 2018. 
http://www.unaids.org/en/resources/documents/2018/miles-to-go_eastern-europe-and-central-asia. 
2.  
Public Health Center of the MoH of Ukraine GI of E and ID. HIV Infection in Ukraine Informational Bulletin #50.; 
2019. http://phc.org.ua/pages/diseases/hiv_aids/monitoring/information-bulletins. 
3.  
Kiriazova T, Postnov O, Bingham T, et al. Patient and provider perspectives inform an intervention to improve 
linkage to care for HIV patients in Ukraine. BMC Health Serv Res. 2018;18(1):58. 
https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-018-2885-4. 
4.  
Neduzhko O, Postnov O, Bingham T, Myers JJ, Flanigan T, Kiriazova T. Feasibility and Acceptability of the 
Modified Antiretroviral Treatment Access Study (MARTAS) Intervention Based on a Pilot Study in Ukraine. J Int Assoc 
Provid AIDS Care. 2019;18:232595821882325. http://journals.sagepub.com/doi/10.1177/2325958218823257. 
5.  
Network 100 Percent Life. USAID RESPECT Project. 2017. 
6.  
Boyko A, Dmytryshyna N, Tiuleneva N. Steps towards health: Social support for PLHIV who inject drugs. 
Implementation Manual. Respond Improv HIV/AIDS Serv Key Popul Ukr. 2016. http://respond.org.ua/file/6u.pdf. 
Accessed November 1, 2018. 
7.  
RESPOND Project: Evaluation of Integrated Promising Interventions for PLWH: “Patient’s School” and “Studio of 
Opportunities.”; 2017. 
8.  
Kiriazova T, Sereda Y, Yorick R, Shvab I, Dvoryak S. Assessment of the behavioral intervention “Steps Toward 
Health” for HIV-positive persons who inject drugs in Ukraine. In: College on Problems of Drug Dependence 79th Annual 
Meeting. Montreal, Canada; 2017. http://www.uiphp.org.ua/media/k2/attachments/Kiriazova_CPDD_2017.pdf. 
9.  
Neduzhko O, Postnov O, Sereda Y, et al. Modified Antiretroviral Treatment Access Study (MARTAS): A 
Randomized Controlled Trial of the Efficacy of a Linkage‐to‐Care Intervention Among HIV-Positive Patients in Ukraine. 
AIDS Behav. 2020. http://link.springer.com/10.1007/s10461-020-02873-7. 
10.  
Sazonova Y, Saliuk T. Main Results of Bio-Behavioral Surveillance among Key Populations.; 2018. 
http://aph.org.ua/wp-content/uploads/2018/10/OSNOVNY-REZULTATY-A4-ENG-site-version-16.10.2018_red.pdf. 
11.  
Herwehe J, Wilbright W, Abrams A, et al. Implementation of an innovative, integrated electronic medical record 
(EMR) and public health information exchange for HIV/AIDS. J Am Med Informatics Assoc. 2012;19(3):448-452. 
https://academic.oup.com/jamia/article-lookup/doi/10.1136/amiajnl-2011-000412. 
12.  
Sweeney P, Hoyte T, Mulatu MS, et al. Implementing a Data to Care Strategy to Improve Health Outcomes for 
People With HIV: A Report From the Care and Prevention in the United States Demonstration Project. Public Health Rep. 
2018;133(2_suppl):60S-74S. http://journals.sagepub.com/doi/10.1177/0033354918805987. 
13.  
Aarons GA, Green AE, Palinkas LA, et al. Dynamic adaptation process to implement an evidence-based child 
maltreatment intervention. Implement Sci. 2012;7(1):32. 
http://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-7-32. 
14.  
Chambers DA, Norton WE. The Adaptome. Am J Prev Med. 2016;51(4):S124-S131. 
https://linkinghub.elsevier.com/retrieve/pii/S0749379716301817. 
15.  
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health 
services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 
2009;4(1):50. http://www.implementationscience.com/content/4/1/50. 
16.  
Kovtun O, Lysenko Y, Sokolenko O, et al. The Medical Information System “HIV Infection in Ukraine:” 
Development and Implementation. Public Heal Panor. 2019;5(1):80-86. 
https://www.euro.who.int/__data/assets/pdf_file/0003/398343/short-comm-HIV-Ukraine-eng.pdf. 
17.  
PEPFAR. Ukraine Country Operational Plan (COP) 2017: Strategic Direction Summary.; 2017. 
https://www.pepfar.gov/documents/organization/272024.pdf. 
18.  
Blair JM, Fagan JL, Frazier EL, et al. Behavioral and Clinical Characteristics of Persons Receiving Medical Care for 
HIV Infection - Medical Monitoring Project, United States, 2009. MMWR Surveill Summ. 2014;63 Suppl 5:1-22. 
pmid:24941443. 
19.  
Degenhardt L, Bucello C, Mathers B, et al. Mortality among regular or dependent users of heroin and other 
opioids: A systematic review and meta-analysis of cohort studies. Addiction. 2011;106:32-51. pmid:21054613. 
20.  
Gowing L, Farrell MF, Bornemann R, Sullivan LE, Ali R. Oral substitution treatment of injecting opioid users for 
prevention of HIV infection. Cochrane database Syst Rev. 2011;(8):CD004145. pmid:21833948. 
21.  
Beyrer C, Malinowska-Sempruch K, Kamarulzaman A, Kazatchkine M, Sidibe M, Strathdee SA. Time to act: A call 
for comprehensive responses to HIV in people who use drugs. Lancet. 2010;376:551-563. pmid:20650515. 
22.  
Knowlton AR, Arnsten JH, Eldred LJ, et al. Antiretroviral use among active injection-drug users: the role of 
patient-provider engagement and structural factors. AIDS Patient Care STDS. 2010;24(7):421-428. pmid:20578910. 

D2C Ukraine 
28 
21-Mar-23 
 
Version 1.1 
23.  
Spire B, Lucas GM, Carrieri MP. Adherence to HIV treatment among IDUs and the role of opioid substitution 
treatment (OST). Int J Drug Policy. 2007;18:262-270. pmid:17689374. 
24.  
Saag LA, Tamhane AR, Batey DS, Mugavero MJ, Eaton EF. Mental health service utilization is associated with 
retention in care among persons living with HIV at a university-affiliated HIV clinic. AIDS Res Ther. 2018;15(1):1. 
pmid:29338735. 
25.  
Magnus M, Schmidt N, Kirkhart K, et al. Association between ancillary services and clinical and behavioral 
outcomes among HIV-infected women. AIDS Patient Care STDS. 2001;15(3):137-145. pmid:11313026. 
26.  
Johnson BT, Scott-Sheldon LAJ, Smoak ND, LaCroix JM, Anderson JR, Carey MP. Behavioral Interventions for 
African Americans to Reduce Sexual Risk of HIV: A Meta-Analysis of Randomized Controlled Trials. JAIDS J Acquir 
Immune Defic Syndr. 2009;51(4):492-501 10.1097/QAI.0b013e3181a28121. 
http://journals.lww.com/jaids/Fulltext/2009/08010/Behavioral_Interventions_for_African_Americans_to.18.aspx. 
27.  
Robertson MM, Penrose K, Irvine MK, et al. Impact of an HIV Care Coordination Program on Durable Viral 
Suppression. JAIDS J Acquir Immune Defic Syndr. 2018:1. http://insights.ovid.com/crossref?an=00126334-900000000-
96552. 
28.  
Morozova O, Dvoriak S, Pykalo I, Altice FL. Primary healthcare-based integrated care with opioid agonist 
treatment: First experience from Ukraine. Drug Alcohol Depend. 2017;173:132-138. 
https://linkinghub.elsevier.com/retrieve/pii/S0376871617300698. 
29.  
Altice FL, Bruce RD, Lucas GM, et al. HIV Treatment Outcomes among HIV-Infected, Opioid-Dependent Patients 
Receiving Buprenorphine/Naloxone Treatment within HIV Clinical Care Settings: Results from a Multisite Study.; 2011. 
pmid:21317590. 
30.  
Lucas GM. Clinic-Based Treatment of Opioid-Dependent HIV-Infected Patients Versus Referral to an Opioid 
Treatment Program. Ann Intern Med. 2010;152(11):704. http://annals.org/article.aspx?doi=10.7326/0003-4819-152-11-
201006010-00003. 
31.  
Uhlmann S, Milloy M-J, Kerr T, et al. Methadone maintenance therapy promotes initiation of antiretroviral 
therapy among injection drug users. Addiction. 2010;105(5):907-913. http://doi.wiley.com/10.1111/j.1360-
0443.2010.02905.x. 
32.  
Palepu A, Tyndall MW, Joy R, et al. Antiretroviral adherence and HIV treatment outcomes among HIV/HCV co-
infected injection drug users: The role of methadone maintenance therapy. Drug Alcohol Depend. 2006;84(2):188-194. 
http://linkinghub.elsevier.com/retrieve/pii/S0376871606000603. 
33.  
Low AJ, Mburu G, Welton NJ, et al. Impact of Opioid Substitution Therapy on Antiretroviral Therapy Outcomes: 
A Systematic Review and Meta-Analysis. Clin Infect Dis. 2016;63(8):1094-1104. https://academic.oup.com/cid/article-
lookup/doi/10.1093/cid/ciw416. 
34.  
Mazhnaya A, al  et. Opioid Agonist Treatment and Improved Outcomes at Each Stage of the HIV Treatment 
Cascade in People Who Inject Drugs in Ukraine. JAIDS J Acquir Immune Defic Syndr. 2018;79:288-295. 
35.  
HIV Infection in Ukraine. Newsletter No. 50.; 2019. 
https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=2ahUKEwikoPP_z4jeA
hUDT98KHfuGD3MQFjAAegQICRAC&url=https%3A%2F%2Fphc.org.ua%2Fuploads%2Fdocuments%2Fc21991%2F8996
daa51af04118a9a24efbe09d1c49.pdf&usg=AOvVaw3oEzBv6dZpRaT0MFy. 
36.  
Dumchev K, Dvoryak S, Chernova O, Morozova O, Altice FL. Retention in medication-assisted treatment 
programs in Ukraine—Identifying factors contributing to a continuing HIV epidemic. Int J Drug Policy. 2017;48:44-53. 
https://linkinghub.elsevier.com/retrieve/pii/S0955395917301184. 
37.  
Public Health Center of the MoH of Ukraine GI of E and ID. HIV Infection in Ukraine Informational Bulletin #51.; 
2020. http://phc.org.ua/pages/diseases/hiv_aids/monitoring/information-bulletins. 
38.  
Sazonova Y, Kulchynska R, Sereda Y, et al. HIV treatment cascade among people who inject drugs in Ukraine. 
Parekh BS, ed. PLoS One. 2020;15(12):e0244572. https://dx.plos.org/10.1371/journal.pone.0244572. 
39.  
Padilla M, Mattson CL, Scheer S, et al. Locating People Diagnosed With HIV for Public Health Action: Utility of 
HIV Case Surveillance and Other Data Sources. Public Health Rep. 2018;133(2):147-154. 
http://journals.sagepub.com/doi/10.1177/0033354918754541. 
40.  
Hart-Malloy R, Brown S, Bogucki K, Tesoriero J. Implementing data-to-care initiatives for HIV in New York state: 
assessing the value of community health centers identifying persons out of care for health department follow-up. AIDS 
Care. 2018;30(3):391-396. https://www.tandfonline.com/doi/full/10.1080/09540121.2017.1363851. 
41.  
Tesoriero JM, Johnson BL, Hart-Malloy R, et al. Improving Retention in HIV Care Through New Yorkʼs Expanded 
Partner Services Data-to-Care Pilot. J Public Heal Manag Pract. 2017;23(3):255-263. 
http://insights.ovid.com/crossref?an=00124784-201705000-00004. 
42.  
US Department of Health and Human Services. Improving Health Outcomes Through Data Utilization: 2016 Ryan 
White HIV/AIDS Program Highlights. Rockville, MD; 2016. 

D2C Ukraine 
29 
21-Mar-23 
 
Version 1.1 
43.  
Evans D, Van Gorder D, Morin SF, Steward WT, Gaffney S, Charlebois ED. Acceptance of the Use of HIV 
Surveillance Data for Care Engagement. JAIDS J Acquir Immune Defic Syndr. 2015;69:S31-S36. 
http://content.wkhealth.com/linkback/openurl?sid=WKPTLP:landingpage&an=00126334-201505011-00005. 
44.  
Malta M, Magnanini MMF, Strathdee SA, Bastos FI. Adherence to antiretroviral therapy among HIV-infected drug 
users: a meta-analysis. AIDS Behav. 2010;14(4):731-747. pmid:19020970. 
45.  
Bachireddy C, Soule MC, Izenberg JM, Dvoryak S, Dumchev K, Altice FL. Integration of health services improves 
multiple healthcare outcomes among HIV-infected people who inject drugs in Ukraine. Drug Alcohol Depend. 
2014;134:106-114. pmid:24128379. 
46.  
MacArthur GJ, Minozzi S, Martin N, et al. Opiate substitution treatment and HIV transmission in people who 
inject drugs: systematic review and meta-analysis. BMJ. 2012;345(oct03 3):e5945-e5945. 
http://www.bmj.com/cgi/doi/10.1136/bmj.e5945. 
47.  
Metzger DS, Donnell D, Celentano DD, et al. Expanding Substance Use Treatment Options for HIV Prevention 
With Buprenorphine-Naloxone: HIV Prevention Trials Network 058. J Acquir Immune Defic Syndr. 2015;68(5):554-561. 
pmid:25564105. 
48.  
Korthuis PT, Fiellin DA, Fu R, et al. Improving adherence to HIV quality of care indicators in persons with opioid 
dependence: the role of buprenorphine. J Acquir Immune Defic Syndr. 2011;56 Suppl 1:S83-S90. pmid:21317600. 
49.  
Dumchev K, Novak Y, Saliuk T. Predictors of ART Initiation and Viral Suppression In a Large Cohort In Ukraine. 
In: CROI Abstract Number: 1047. Seattle, WA; 2019. http://www.croiconference.org/sessions/predictors-art-initiation-
and-viral-suppression-large-cohort-ukraine. 
50.  
Mugavero MJ. Elements of the HIV Care Continuum: Improving Engagement and Retention in Care. Top Antivir 
Med. 24(3):115-119. pmid:27841983. 
51.  
Horberg MA, Hurley LB, Silverberg MJ, Klein DB, Quesenberry CP, Mugavero MJ. Missed Office Visits and Risk of 
Mortality Among HIV-Infected Subjects in a Large Healthcare System in the United States. AIDS Patient Care STDS. 
2013;27(8):442-449. http://www.liebertpub.com/doi/10.1089/apc.2013.0073. 
52.  
WHO Website. WHO Developing Guidelines for Recommendations on Digital Health Interventions for RMNCAH 
and Health Systems Strengthening. WHO Website.; 2016. 
http://www.who.int/reproductivehealth/topics/mhealth/%0Adigital-health-interventions/en/. 
53.  
Labrique A, Vasudevan L, Weiss W, Wilson K. Establishing Standards to Evaluate the Impact of Integrating 
Digital Health into Health Systems. Glob Heal Sci Pract. 2018;6(Supplement):S5-S17. 
54.  
Owczarzak J, Phillips SD. Responsibility, Trust, and Transformation in Ukrainian Public Health. Perspect Eur. 
2014;44(2):34-42. 
55.  
Owczarzak J, Phillips SD, Filippova O, et al. A “Common Factors” Approach to Developing Culturally Tailored HIV 
Prevention Interventions. Heal Educ Behav. 2016;43(3):347-357. 
http://heb.sagepub.com/cgi/doi/10.1177/1090198115602665. 
56.  
Owczarzak J, Karelin M, Phillips SD. A View from the Frontlines in Slavyansk, Ukraine: HIV Prevention, Drug 
Treatment, and Help for People Who Use Drugs in a Conflict Zone. Int J Drug Policy. 2014. 
http://linkinghub.elsevier.com/retrieve/pii/S0955395914003338. Accessed December 1, 2014. 
57.  
Owczarzak J, Phillips SD, Cho W. ‘Pure’ drug users, commercial sex workers and ‘ordinary girls’: gendered 
narratives of HIV risk and prevention in post-Soviet Ukraine. Cult Health Sex. 2018:1-14. 
https://www.tandfonline.com/doi/full/10.1080/13691058.2017.1421708. 
58.  
Mazhnaya A, Tobin KE, Owczarzak J. Association between injection in public places and HIV/HCV risk behavior 
among people who use drugs in Ukraine. Drug Alcohol Depend. 2018;189:125-130. 
https://linkinghub.elsevier.com/retrieve/pii/S0376871618303181. 
59.  
Dumchev K, Miller W, Hanscom B et al. An integrated intervention to increase ART and MAT reduced mortality 
among PWID: Results from the HPTN 074 randomized trial. HIV Res Prev 2018 (HIVR4P 2018). 2018;Oral Abstr. 
60.  
Dumchev K, Novak Y, Shestakovskyi O, Saliuk T. Data from large electronic medical record dataset reveal 
significant improvements and gaps in HIV treatment cascade in Ukraine. 22nd Int AIDS Conf. 2018;Abstract #. 
61.  
Irwin A, Jozaghi E, Weir BW, Allen ST, Lindsay A, Sherman SG. Mitigating the heroin crisis in Baltimore, MD, 
USA: a cost-benefit analysis of a hypothetical supervised injection facility. Harm Reduct J. 2017;14(1):29. 
http://harmreductionjournal.biomedcentral.com/articles/10.1186/s12954-017-0153-2. 
62.  
Kasaie P, Weir B, Schnure M, et al. Integrated screening and treatment services for HIV, hypertension and 
diabetes in Kenya: assessing the epidemiological impact and cost-effectiveness from a national and regional perspective. 
J Int AIDS Soc. 2020;23 Suppl 1:e25499. pmid:32562353. 
63.  
Maulsby C, Jain KM, Weir BW, et al. The Cost and Threshold Analysis of Retention in Care (RiC): A Multi-Site 
National HIV Care Program. AIDS Behav. 2017;21(3):643-649. http://link.springer.com/10.1007/s10461-016-1623-3. 

D2C Ukraine 
30 
21-Mar-23 
 
Version 1.1 
64.  
Maulsby C, Jain KM, Weir BW, et al. Cost–Utility of Access to Care, a National HIV Linkage, Re-engagement and 
Retention in Care Program. AIDS Behav. 2018;22(11):3734-3741. http://link.springer.com/10.1007/s10461-017-2015-z. 
65.  
Wirtz AL, Weir BW, Mon SHH, et al. Testing the Effectiveness and Cost-Effectiveness of a Combination HIV 
Prevention Intervention Among Young Cisgender Men Who Have Sex With Men and Transgender Women Who Sell or 
Exchange Sex in Thailand: Protocol for the Combination Prevention Effectiveness. JMIR Res Protoc. 2020;9(1):e15354. 
https://www.researchprotocols.org/2020/1/e15354. 
66.  
Hoffmann CJ, Mabuto T, Ginindza S, et al. Strategies to Accelerate HIV Care and Antiretroviral Therapy Initiation 
After HIV Diagnosis: A Randomized Trial. J Acquir Immune Defic Syndr. 2017;75(5):540-547. pmid:28471840. 
67.  
Mabuto T, Latka MH, Kuwane B, Churchyard GJ, Charalambous S, Hoffmann CJ. Four models of HIV counseling 
and testing: utilization and test results in South Africa. PLoS One. 2014;9(7):e102267. pmid:25013938. 
68.  
Hoffmann CJ, Maritz J, van Zyl GU. CD4 count-based failure criteria combined with viral load monitoring may 
trigger worse switch decisions than viral load monitoring alone. Trop Med Int Health. 2016;21(2):219-223. 
pmid:26584666. 
69.  
Hoffmann CJ, Mabuto T, McCarthy K, Maulsby C, Holtgrave DR. A Framework to Inform Strategies to Improve 
the HIV Care Continuum in Low- and Middle-Income Countries. AIDS Educ Prev. 2016;28(4):351-364. pmid:27427929. 
70.  
Kiriazova T, Postnov O, Perehinets I, Neduzhko O. Association of injecting drug use and late enrolment in HIV 
medical care in Odessa Region, Ukraine. HIV Med. 2013;14:38-41. http://doi.wiley.com/10.1111/hiv.12059. 
71.  
Polonsky M, Azbel L, Wickersham JA, et al. Challenges to implementing opioid substitution therapy in Ukrainian 
prisons: Personnel attitudes toward addiction, treatment, and people with HIV/AIDS. Drug Alcohol Depend. 
2015;148:47-55. http://linkinghub.elsevier.com/retrieve/pii/S0376871614019875. 
72.  
Kiriazova T, Cheng DM, Coleman SM, et al. Factors Associated With Study Attrition Among HIV-Infected Risky 
Drinkers in St. Petersburg, Russia. HIV Clin Trials. 2014;15(3):116-125. 
http://www.tandfonline.com/doi/full/10.1310/hct1503-116. 
73.  
Kiriazova T, Lunze K, Raj A, et al. “It is easier for me to shoot up”: stigma, abandonment, and why HIV-positive 
drug users in Russia fail to link to HIV care. AIDS Care. 2017;29(5):559-563. 
https://www.tandfonline.com/doi/full/10.1080/09540121.2016.1259451. 
74.  
Miller WC, Hoffman IF, Hanscom BS, et al. A scalable, integrated intervention to engage people who inject drugs 
in HIV care and medication-assisted treatment (HPTN 074): a randomised, controlled phase 3 feasibility and efficacy 
study. Lancet. 2018;392(10149):747-759. https://linkinghub.elsevier.com/retrieve/pii/S0140673618314879. 
75.  
Gardner LI, Metsch LR, Anderson-Mahoney P, et al. Efficacy of a brief case management intervention to link 
recently diagnosed HIV-infected persons to care. AIDS. 2005;19(4):423-431. pmid:15750396. 
76.  
Yehia BR, Stephens-Shields AJ, Fleishman JA, et al. The HIV Care Continuum: Changes over Time in Retention 
in Care and Viral Suppression. Paraskevis D, ed. PLoS One. 2015;10(6):e0129376. 
https://dx.doi.org/10.1371/journal.pone.0129376. 
77.  
Ko N-Y, Liu H-Y, Lai Y-Y, Pai Y-H, Ko W-C. Case Management Interventions for HIV-Infected Individuals. Curr 
HIV/AIDS Rep. 2013;10(4):390-397. http://link.springer.com/10.1007/s11904-013-0183-7. 
78.  
Metzger ME, Alexander JA, Weiner BJ. The effects of leadership and governance processes on member 
participation in community health coalitions. Health Educ Behav. 2005;32:455-473. pmid:16009744. 
79.  
Magadzire BP, Budden A, Ward K, Jeffery R, Sanders D. Frontline health workers as brokers : provider 
perceptions , experiences and mitigating strategies to improve access to essential medicines in South Africa. 2014:1-10. 
80.  
Mignone J, Washington RG, Ramesh BM, Blanchard JF, Moses S. Formal and informal sector health providers in 
southern India : Role in the prevention and care of sexually transmitted infections , including HIV / AIDS. 
2007;19(February):152-159. 
81.  
Blase K, Fixsen D. ImpleMap: Exploring the Implementation Landscape. 2013. 
http://implementation.fpg.unc.edu/sites/implementation.fpg.unc.edu/files/resources/NIRN-ImpleMap.pdf. 
82.  
UNAIDS World AIDS Day Report. Geneva; 2011. 
http://www.unaids.org/sites/default/files/en/media/unaids/contentassets/documents/unaidspublication/2011/JC2216_W
orldAIDSday_report_2011_en.pdf. 
83.  
Rodriguez-Arenas MA, Jarrin I, del Amo J, et al. Delay in the initiation of HAART, poorer virological response, 
and higher mortality among HIV-infected injecting drug users in Spain. AIDS Res Hum Retroviruses. 2006;22(8):715-
723. pmid:16910826. 
84.  
Hayes RJ, Bennett S. Simple sample size calculation for cluster-randomized trials. Int J Epidemiol. 
1999;28(2):319-326. pmid:10342698. 
85.  
Boyko A, Dmitrishina N, Tyulenieva N. Інтегрована Проґрама з Формування Прихильності До АРТ у Кліентів 
ВІЛ-Сервісніх Орґанізацій Та Розвітку Їхньої Економічної Самостійності.; 2017. http://respond.org.ua/file/4u.pdf. 

D2C Ukraine 
31 
21-Mar-23 
 
Version 1.1 
86.  
RESPOND Project: Evaluation of Integrated Promising Interventions for PLWH: “Patient’s School” and “Studio of 
Opportunities.”; 2017. http://respond.org.ua/files/attachments/Integrated Intervention_IS Summary_2017_09_e.pdf. 
87.  
Orrell C, Cohen K, Leisegang R, Bangsberg DR, Wood R, Maartens G. Comparison of six methods to estimate 
adherence in an ART-naïve cohort in a resource-poor setting: which best predicts virological and resistance outcomes? 
AIDS Res Ther. 2017;14(1):20. http://aidsrestherapy.biomedcentral.com/articles/10.1186/s12981-017-0138-y. 
88.  
McLellan AT, Kushner H, Metzger D, et al. The fifth edition of the Addiction Severity Index. J Subst Abuse Treat. 
1992;9:199-213. pmid:1334156. 
89.  
Zanis DA, McLellan AT, Cnaan RA, Randall M. Reliability and validity of the Addiction Severity Index with a 
homeless sample. J Subst Abuse Treat. 1994;11:541-548. pmid:7884837. 
90.  
Radloff L. The CES-D scale: A self-report depression scale for research in the general population. Appl Psychol 
Meas. 1977;1(3):385-401. 
91.  
Latkin C, Davey-Rothwell M, Yang J, Crawford N. The Relationship between Drug User Stigma and Depression 
among Inner-City Drug Users in Baltimore, MD. J Urban Heal. 2013;90(1):147-156. 
http://link.springer.com/10.1007/s11524-012-9753-z. 
92.  
Costenbader E, Astone N, Latkin C. The dynamics of injection drug users’ personal networks and HIV risk 
behaviors. Addiction. 2006;101(7):1003-1013. 
93.  
Boyd J, Weissman M, Thompson W, Myers J. Screening for depression in a community sample. Understanding 
the discrepancies between depression symptom and diagnostic scales. Arch Gen Psychiatry. 1982;39(10):1195-1200. 
94.  
Booth RE, Kennedy J, Brewster JT, Semerik O. Drug Injectors and Dealers in Odessa, Ukraine. J Psychoactive 
Drugs. 2003;35:419-426. 
95.  
McLellan  Alterman AI, Cacciola J, Metzger D and O’Brien CP AT. A new measure of substance abuse treatment. 
Initial studies of the treatment services review. J Nerv Ment Dis. 1992;180(2):101-110. 
96.  
Langley G, Moen R, Nolan K, Nolan T, Norman C, Provost L. The Improvement Guide: A Practical Approach to 
Enhancing Organizational Performance, 2nd Edition. 2nd ed. John Wiley & Sons, Inc.; 2009. 
97.  
Izudi J, Mugenyi J, Mugabekazi M, et al. Retention of HIV-Positive Adolescents in Care: A Quality Improvement 
Intervention in Mid-Western Uganda. Biomed Res Int. 2018;2018:1-8. 
https://www.hindawi.com/journals/bmri/2018/1524016/. 
98.  
Enders C. Applying Missing Data Analysis. Guilford Press; 2010. 
99.  
Little R, Rubin D. Statistical Analysis with Missing Data. John Wiley & Sons, Inc.; 2014. 
100.  
Rubin DB. Multiple Imputation for Nonresponse in Surveys. New York: John Wiley & Sons, Inc.; 1987. 
101.  
White IR, Royston P, Wood AM. Multiple imputation using chained equations: Issues and guidance for practice. 
Stat Med. 2011;30(4):377-399. http://doi.wiley.com/10.1002/sim.4067. 
102.  
Hedeker D. Methods for Multilevel Ordinal Data in Prevention Research. Prev Sci. 2015;16(7):997-1006. 
http://link.springer.com/10.1007/s11121-014-0495-x. 
103.  
Hedeker D, Mermelstein RJ, Weeks KA. The thresholds of change model: An approach to analyzing stages of 
change data. Ann Behav Med. 1999;21(1):61-70. https://academic.oup.com/abm/article/21/1/61-70/4631412. 
104.  
Cohen J, Cohen P, West S, Aiken L. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. 
Routledge; 2013. 
105.  
Rojas S, Ashok M, Dy S, Wines R, Teixeira-Poit S. Contextual Frameworks for Research on the Implementation 
of Complex Systems Interventions: Methods Research Report. Rockville, MD; 2014. 
106.  
Means AR, Kemp CG, Gwayi-Chore M-C, et al. Evaluating and optimizing the consolidated framework for 
implementation research (CFIR) for use in low- and middle-income countries: a systematic review. Implement Sci. 
2020;15(1):17. https://implementationscience.biomedcentral.com/articles/10.1186/s13012-020-0977-0. 
107.  
Proctor E, Silmere H, Raghavan R, al  et. Outcomes for implementation research: conceptual distinctions, 
measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65-76. 
108.  
Weiner BJ, Lewis CC, Stanick C, et al. Psychometric assessment of three newly developed implementation 
outcome measures. Implement Sci. 2017;12(1):108. 
http://implementationscience.biomedcentral.com/articles/10.1186/s13012-017-0635-3. 
109.  
Aarons GA, Ehrhart MG, Farahnak LR, Hurlburt MS. Leadership and organizational change for implementation 
(LOCI): a randomized mixed method pilot study of a leadership and organization development intervention for evidence-
based practice implementation. Implement Sci. 2015;10(1):11. 
http://implementationscience.biomedcentral.com/articles/10.1186/s13012-014-0192-y. 
110.  
Kobak KA, Mundt JC, Kennard B. Integrating technology into cognitive behavior therapy for adolescent 
depression: a pilot study. Ann Gen Psychiatry. 2015;14(1):37. http://www.annals-general-
psychiatry.com/content/14/1/37. 

D2C Ukraine 
32 
21-Mar-23 
 
Version 1.1 
111.  
Neumann P, Sanders G, Russell L, Siegel J, Ganiats T. Cost Effectiveness in Health and Medicine. Second. 
Oxford, UK: Oxford University Press; 2017. 
112.  
Sanders GD, Neumann PJ, Basu A, et al. Recommendations for Conduct, Methodological Practices, and 
Reporting of Cost-effectiveness Analyses. JAMA. 2016;316(10):1093. 
http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.2016.12195. 
113.  
Frick KD. Microcosting Quantity Data Collection Methods. Med Care. 2009;47(Supplement):S76-S81. 
http://journals.lww.com/00005650-200907001-00013. 
114.  
Shrestha RK, Sansom SL, Farnham PG. Comparison of methods for estimating the cost of human 
immunodeficiency virus-testing interventions. J Public Health Manag Pract. 2012;18:259-267. pmid:22473119. 
115.  
Vassall A, Sweeney S, Kahn J, al  et. Reference Case for Estimating the Costs of Global Health Services and 
Interventions.; 2017. 
116.  
Lin F, Farnham PG, Shrestha RK, Mermin J, Sansom SL. Cost Effectiveness of HIV Prevention Interventions in 
the U.S. Am J Prev Med. 2016;50(6):699-708. https://linkinghub.elsevier.com/retrieve/pii/S0749379716000374. 
117.  
Holtgrave DR. Handbook of Economic Evaluation in HIV Prevention Programs. New York: Plenum Press; 1998. 
118.  
Hemming K, Haines TP, Chilton PJ, Girling AJ, Lilford RJ. The stepped wedge cluster randomised trial: rationale, 
design, analysis, and reporting. BMJ. 2015;350(feb06 1):h391-h391. http://www.bmj.com/cgi/doi/10.1136/bmj.h391. 
119.  
Vetter TR, Chou R. Clinical Trial Design Methodology for Pain Outcome Studies. In: Practical Management of 
Pain. Elsevier; 2014:1057-1065.e3. https://linkinghub.elsevier.com/retrieve/pii/B9780323083409000803. 
120.  
Donner A, Klar N. Pitfalls of and controversies in cluster randomization trials. Am J Public Health. 
2004;94(3):416-422. pmid:14998805. 
121.  
Jain KM, Holtgrave DR, Maulsby C, et al. Improving Access to HIV Care: Lessons from 5 U.S. Cities. Baltimore, 
MD: Johns Hopkins University Press; 2016. 
 
 

D2C Ukraine 
33 
21-Mar-23 
 
Version 1.1 
Appendix 1: Recruitment Scripts 
 
Client Recruitment Script – Control Sites 
 
Introduction 
 
My name is [STATE NAME], and I am a staff member here at [NAME OF AGENCY]. I would like to invite 
you to participate in a research study we are part of that is being conducted by Jill Owczarzak at Johns Hopkins 
University and Kostya Dumchev at the Ukrainian Institute on Public Health Policy. This study is trying to learn if 
a new program is effective in helping people remain in HIV care. The new program involves case managers, also 
known as social workers, who identify and address medical and social needs. Some clinics have been selected to 
implement the new program, whereas other clinics will continue their standard operations. This will allow our 
study team to determine the effectiveness of the new program.  
Statement of why he/she selected 
 
Your clinic, [NAME OF CLINIC], has been selected as one of the standard-of-care locations. You are 
being invited to join this research study because you are registered as a patient at [NAME OF CLINIC] and might 
be eligible to be part of this study. Your participation in this research will help us understand the impact of a 
new program to support people living with HIV. We are enrolling people living with HIV and have histories of 
substance use to understand how services can be adapted to meet their needs.  
Description of Procedures and Payment 
 
If you decide to join the study, you will receive the standard of care at [NAME OF CLINIC].  
You will also complete three surveys: one at the beginning, one after 6 months, and one at the final 12-month 
visit. Each survey will last about 1 hour and will ask you about your HIV, mental health, and substance use 
experiences.  
If you agree to participate, you will receive 400 hryvnia for completing each of the three surveys. 
Assessment of Interest in Participation 
 
Are you interested in hearing more details about the research study? 
● 
If not interested, thank the individual for his/her time. 
● 
If interested, then move to the consent form. 
 
 

D2C Ukraine 
34 
21-Mar-23 
 
Version 1.1 
  
Client Recruitment Script – Intervention Sites 
 
Introduction 
 
My name is [STATE NAME], and I am a staff member here at [NAME OF AGENCY]. I would like to invite 
you to participate in a research study we are part of that is being conducted by Jill Owczarzak at Johns Hopkins 
University and Kostya Dumchev at the Ukrainian Institute on Public Health Policy. This study is trying to learn if 
a new program is effective in helping people remain in HIV care. The new program involves case managers, also 
known as social workers, who identify and address medical and social needs. 
Statement of why he/she selected 
 
You are being invited to join this research study because you are registered as a patient at [NAME OF 
CLINIC] and might be eligible to be part of this study. Your participation in this research will help us understand 
the impact of a new program to support people living with HIV. We are enrolling people living with HIV and 
have histories of substance use to understand how services can be adapted to meet their needs.  
Description of Procedures and Payment 
 
If you decide to join the study, you will work with a case manager at the [NAME OF CLINIC] who will 
ask you questions about your health history, living situation, social support, and other basic needs. Based on 
your responses, the case manager will support you in staying connected to HIV care and treatment by working 
with you to address issues that may make it difficult to attend HIV care appointments. The frequency of 
meetings and check-in phone calls with the case manager will vary depending on your needs and preferences. 
The case manager will re-evaluate your situation every 3 months. These evaluations and planning sessions with 
the case manager will last approximately 45-60 minutes each. There will be a total of 5 such sessions during the 
study. 
You will also complete three surveys: one at the beginning, one after 6 months, and one at the final 12-month 
visit. Each survey will last about 1 hour and will ask you about your HIV, mental health, and substance use 
experiences.  
If you agree to participate, you will receive 400 hryvnia for completing each of the three surveys. 
Assessment of Interest in Participation 
 
Are you interested in hearing more details about the research study? 
● 
If not interested, thank the individual for his/her time. 
● 
If interested, then move to the consent form.  
 
 
 

D2C Ukraine 
35 
21-Mar-23 
 
Version 1.1 
Appendix 2: Recruitment Log (structure) 
 
Study Site ___ 
Responsible staff ___ 
Filled by the Data Manager: 
MIS ID 
Study 
ID 
Diagnosis 
date 
 
 
 
 
Filled by the Case Manager: 
Contact attempt 1: 
Date 
Mode 
Result 
Comment 
 
Phone; 
Clinic visit; 
Outreach; 
Other 
Visit scheduled; 
no contact; 
contact info incorrect; 
refusal; 
contact later 
other 
 
Contact attempt 2: 
Date 
Mode 
Result 
Comment 
 
 
 
 
Contact attempt 3: 
Date 
Mode 
Result 
Comment 
 
 
 
 
Contact result: 
Refusal 
reason 
Scheduled 
visit date 
Scheduled 
visit time 
No interest; 
No time; 
Cannot come; 
Other 
 
 
 
Filled by the Interviewer: 
Visit 
date 
Visit result 
 
Completed; Incomplete; No show; No consent