AIDS United Social Innovation Fund

Final Evaluation Report


	

May 2016

Updated 8/29/16 

Executive Summary 

AIDS United’s multi-sponsored Access to Care (A2C) initiative was conceived for people living with 

HIV/AIDS (PLWH) to increase their access to and retention in effective HIV healthcare and support 

services. The A2C subgrantees included twelve sites located throughout the U.S. that implemented 

innovative, evidence-based HIV linkage and retention-in-care programs. A2C programs shared common 

program models, including peer/patient navigation, community health workers, care coordination, 

intensive case management, and motivational interviewing, to serve the most vulnerable populations. 

The evaluation of A2C included impact, outcome, and implementation components. The impact 

evaluation assessed if individuals exposed to the BA2C intervention (Birmingham, Alabama: Lead agency 

University of Alabama at Birmingham in partnership with Birmingham AIDS Outreach) had greater 

improvements in health outcomes (CD4 count and viral load) at 12 months compared to individuals who 

did not receive the intervention. In addition, the impact evaluation constructed continuums of care for 

program participants for comparison with geographically local continuums of care. The outcome 

evaluation assessed trends in participant health outcomes (CD4 and viral load) at baseline and 12 

months. The outcome evaluation also included a network analysis, which described increases in

interagency collaboration among subgrantee partner agencies, as well as a cost analysis. The cost 

analysis estimated, for each subgrantee, the cost of delivering the program locally and the cost-saving 

threshold. An implementation research component tracked the number of clients who were enrolled in

the program, linked to HIV medical care, and retained in HIV medical care. In addition, the 

implementation evaluation qualitatively assessed barriers and facilitating factors of program 

implementation. 

Implementation evaluation: The A2C program served a total 4,704 participants. Across the cohort, 86% 

were linked to HIV medical care during the program. The majority of subgrantees reported HIV retention 

in care percentages above 75% at twelve months (range 45%–92%). Across the cohort, 63% of

participants (1570/2490) had current highly active antiretroviral treatment (HAART) prescriptions twelve 

months after enrollment. 

We conducted 90-minute interviews with staff at lead subgrantee and implementing partner agencies to 

better understand barriers and facilitators to program implementation. Several themes emerged across 

all program sites. Common barriers to program implementation included: the intensity of client needs;

recruiting and retaining participants; staffing and administrative hurdles; and challenges related to 

working with multiple partner organizations. The three themes that emerged as factors that facilitated 

SIF program implementation were strong relationships with partner organizations; the flexible and 

innovative nature of the SIF programs; and the expertise, dedication, and passion of staff. 

Outcome evaluation: When we compared mean CD4 counts and mean viral load at baseline and 12 

months for each subgrantee, we saw increases in mean CD4 from baseline to twelve months. These 

changes were significant (paired T-test) for participants enrolled in Boston, Chicago, New York, Los 

Angeles, San Diego, Indianapolis, Montgomery, St. Louis, Washington, DC, and Birmingham. Reductions 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 1 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

in viral load were observed across subgrantees. These reductions were significant for participants 

enrolled in the following program locations: Chicago, Los Angeles, Indianapolis, Montgomery, St. Louis, 

and Washington, DC. Increases in the percent of participants who were virally suppressed were seen 

across the cohort from baseline to twelve months, and were significant at eight locations. 

The structure of A2C programs included a lead agency and at least one implementing partner agency. 

The majority of lead agencies worked with a number of implementing partners (range 2–7) including 

clinics, health departments, AIDS service organizations, and social service providers. To better 

understand change at the network level, the national evaluation assessed changes in A2C partner 

network density six months prior to A2C and during implementation of A2C. For all subgrantee networks 

except for two, we saw increases in network density as well as increases in average node degree. 

Cost Analysis: The cost analysis assessed the cost program delivery as well as the cost-saving thresholds 

for the A2C programs, or the number of HIV infections that would need to be averted for a program to 

be cost-saving. Program costs varied considerably. Cost-per-client from the societal perspective ranged 

from $1,109.45 to $5,928.14 per client. Cost-saving thresholds ranged from .32 to 1.19, indicating that 

to be cost-saving the programs would need to avert one to two HIV infections (during the timeframe of

the analysis.) 

Impact evaluation: The primary outcome measure for the randomized control trail of the BA2C program 

was viral suppression. The study observed that intervention arm participants were more likely to be 

virally suppressed at twelve months compared to control arm participants but the results were not 

significant (OR 1.1 (0.4-2.7) p=0.80). The construction and comparison of HIV continuums of care found 

that the majority A2C program continuums of care exceeded local continuums of care. 

In summary, participants in the A2C programs saw improvements in engagement in HIV medical care 

along the HIV continuum of care, including viral suppression. Change occurred at the network level as 

well; interagency collaboration increased during program implementation. Finally, at the policy level,

the cost threshold analysis suggests that the A2C program models are a good use of resources. 

Program Background 

AIDS United’s multi-sponsored Access to Care (A2C) initiative was conceived for people living with 

HIV/AIDS (PLWH) to increase their access to and retention in effective HIV healthcare and support 

services. In particular, the initiative focused on those living in poverty who knew their HIV status but 

were not receiving HIV-specific care or support. Emphasis on hard-to-reach populations was integral in

AIDS United’s A2C work, which sought to identify the systemic and/or personal barriers to care PLWH 

may experience and support the development of systems and interventions to alleviate those barriers 

and implement innovative approaches to ensuring access to and consistent engagement in care. A2C 

was designed to reach the most difficult populations with refined outreach strategies and with 

appropriate resources that otherwise would not be available. All funded projects within the A2C 

portfolio created networks of care in their communities that collaborated to reduce barriers to care, 

provide innovative solutions to long-standing access problems, and to change the way that systems 

operate in their community. AIDS United is committed to supporting projects that not only focus on 

individual level solutions to barriers to care, but also systemic change will last long after AIDS United 

support is gone. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 2 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Social Innovation Fund (SIF) grant dollars allowed AIDS United to greatly expand the A2C initiative. 

Subgrantee sites were selected by a diverse External Review Committee consisting of eight experts in

the fields of HIV/AIDS, public health, program evaluation, and strategic philanthropy. The final

subgrantees were selected using criteria that included such factors as: 

 Serve low socio-economic-status areas that are highly affected by HIV

 Have a track record of using evidence-based practices 

 Have innovative ideas with at least preliminary evidence of effectiveness 

 Have the capacity to effectively manage a SIF project including evaluation and expansion/ 

replication of successful programs 

The subgrantee sites in the SIF cohort were: ActionAIDS, AIDS Action Committee of Massachusetts, AIDS 

Foundation of Chicago (AFC), AIDS Project Los Angeles (APLA), Amida Care, Christie’s Place, The Damien 

Center, Louisiana Public Health Institute, Medical AIDS Outreach, St. Louis Effort for AIDS, University of

Alabama at Birmingham, and Washington AIDS Partnership. Each subgrantee worked with multiple

partners (subcontractors) to implement the project. The SIF cohort was funded in two waves. 

Throughout this document, we refer to the first wave of subgrantees as the first cohort and the second 

wave of subgrantees as the second cohort1

. A brief description of the twelve SIF programs follows: 

ActionAIDS: Philadelphia, PA 

ActionAIDS, a Philadelphia-based HIV/AIDS service organization, implemented an intensive case 

management program designed to increase the rate of retention in health care for individuals who had 

been recently released from the Philadelphia Prison System. During the formative phase of this

program, ActionAIDS designed an acuity assessment instrument that identified clients at intake who 

presented the greatest risk of being lost to care. These clients then received intensive “Care Coach” case 

management services, expedited housing and behavioral health services, and related support to remain

engaged in care. ActionAIDS was joined by three other nonprofit service providers to help transform the 

delivery of services to recently incarcerated clients, as well as services to other clients who have mental

health or drug and alcohol issues and who lack stable, affordable housing. The primary goals of the 

project were to: 

 Identify sub-optimally engaged PLWH and link and retain them in care 

 Increase the number of formerly incarcerated PLWH who obtain/maintain safe and affordable

housing 

 Increase the number of formerly incarcerated PLWH who participated in back-to-work programs 

 Reduce recidivism for formerly incarcerated PLWH 

AIDS Action Committee (AAC): Boston, MA 

The primary goal of LEAP 2.0 was to improve the health of PLWH by providing comprehensive biopsycho-social support from crisis to stability and self-sufficiency. A collaborative effort between AIDS 

Action Committee of MA (AAC) and Partners in Health’s Prevention and Access to Care and Treatment 

(PACT) program, LEAP 2.0 (Linking, Educating, and Advocating with Peers) aimed to increase the number 

1 The first cohort includes: AIDS Action Committee of Massachusetts, AIDS Foundation Chicago, AIDS Project of Los 

Angeles, Amida Care, Christie’s Place, Medical AIDS Outreach, St. Louis Effort for AIDS, and Washington AIDS 

Partnership. The second A2C cohort includes: ActionAIDS, Damien Center, Louisiana Public Health Institute, and 

University of Alabama. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 3 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

of PLWH who were linked to and retained in high quality HIV medical care and supportive services. The 

LEAP 2.0 Integrated Advocacy Team minimally consisted of a Client Advocate, a Peer Leader, and a nontraditional Mental Health Specialist. Others may join the team depending on individual client needs. 

LEAP 2.0 priority areas were: 

 Linkage to and retention of PLWH in HIV medical care 

 Enhanced and expanded access and retention in supportive services (including non-traditional

mental health services and self-management support groups)

 Improved self-sufficiency of PLWH who are not engaged in care through participation in

benefits, treatment adherence, and workforce readiness workshops 

 Expanded continuum of care for PLWHA to better support long-term self-sufficiency 

AIDS Foundation of Chicago (AFC): Chicago, IL

The Connect2Care (C2C) project was a collaboration between AFC and two experienced HIV/AIDS 

organizations to create regional C2C hubs that conducted outreach and networking activities, creating 

seamless systems from HIV diagnosis to care as envisioned in the National HIV/AIDS Strategy. Focusing 

on PLWHA of color, these hubs worked with medical and support service providers in their regions to 

meet client needs and preferences and supported early and continuous care engagement. The 

overarching goal of the C2C project was to increase access to and consistent retention in HIV care for 

PLWH. The four primary intervention components for the project included: 

 System-level enhancements 

 Client-level outreach 

 Linkage to care 

 Client-level education 

Amida Care: New York, NY 

AmidaCONNECT utilized an Assertive Community Treatment (ACT) model in employing three field-based 

Mobile Engagement Teams (METs) to provide intensive outreach, care navigation, case management, 

and reengagement services to the most difficult to reach HIV-positive population—those dually and 

triply diagnosed with mental health and/or substance use issues and were unconnected to care (or at 

risk of becoming disconnected from care)—and connected them to long-term community service 

providers. The enrollees of AmidaCONNECT were not only living with HIV and un/under treated, but 

were often facing additional challenges that affect overall health and well-being, such as homelessness, 

poverty, lack of nutritious food, and legal issues. The METs engaged members in appropriate and 

ongoing primary and behavioral health care and substance use treatment in an effort to suppress HIV 

viral load, reduce health disparities, and generate positive health outcomes, while decreasing the 

utilization of costly hospitalizations and long term care that is often preventable. 

AIDS Project Los Angeles (APLA): Los Angeles, CA

The Care and Access Network represented a partnership among leading HIV/AIDS service organizations, 

APLA, Northeast Valley Health Corporation, and REACH LA, in Los Angeles County (LAC). All of the 

agencies provided a wide variety of clinical, support, and prevention services to a diverse client base 

across the vast geography of LAC. Each of the partners demonstrated a long history of engaging 

racial/ethnic and sexual minorities in the continuum of care. In addition, many of the agencies provided 

HIV counseling and testing services to high-risk populations and were connected to a broader network 

of service providers in their individual communities, inclusive of HIV primary medical care. Patient 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 4 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Navigators helped PLWH address individual barriers to care in order to increase access and retention in

medical care, access treatment, and participate in other supportive services (e.g., case management, 

housing, food and nutrition services, mental health, and HIV risk reduction programs) to enhance and 

maintain engagement in care, and to reduce the incidence of new HIV infections by reducing the 

community viral load. Strategies for reducing community viral load included Patient Navigators 

advocating for earlier initiation of ART as well as helping clients improve adherence. 

Christie’s Place (CP): San Diego, CA 

CHANGE 4 Women functioned as a network of care model. It was designed to improve timely entry, 

access to and retention in HIV care for women living with HIV in San Diego County, with an emphasis on 

women of color. The intervention had two levels of innovation—with individuals and among service 

agencies. The individual-level element focused on increasing the number and capacity of peer navigators 

and the CP peer coordinator to do outreach to and provide support for HIV-positive women. The agency 

innovations included CP’s “one-stop shop” approach to improving client knowledge of their health and 

social service situation and access to care via medical home electronic record access. CP was also a 

nationally recognized model for trauma-informed care and presented their model at numerous 

conferences as well as at the 2015 observance of National Women and Girls HIV/AIDS Awareness Day at 

the White House. Other systems-level innovations included increased formalization and coordination 

among the major HIV health care agencies in San Diego. The project goals were to: 

 Improve underserved women’s access to and retention in comprehensive HIV care 

 Improve utilization of HIV medical care and treatment for HIV-positive women 

 Strengthen health care and social services systems and community linkages through innovative 

and replicable interventions and new collaborations that improve engagement and retention in

care by HIV-positive women 

Damien Center: Indianapolis, IN

Damien CareLink provided coordinated HIV services and linkage to care for PLWH who were currently

not in care. According to the Centers for Disease Control and Prevention, timely linkage to care has a 

variety of benefits, including delaying disease progression and increasing positive health outcomes, 

reducing transmission and therefore preventing new infections, decreasing health care expenditures, 

and keeping community viral loads low. Numerous and complex barriers to care linkage exist and may 

include issues such as mental health and substance issues, fear and stigma, homelessness, lack of colocated facilities, and many others. Damien CareLink addressed these and other barriers for the target 

populations, which included PLWH who were tested confidentially and did not receive post-test 

counseling and those who had been lost to care, African Americans between the ages of 18-44, and 

Hispanics between the ages of 18-44. The priority areas to be addressed in accomplishing these goals

included: 

 Early HIV detection 

 Connecting those PLWH who are not in care, into care 

 Identifying those who are HIV positive, yet unaware of their HIV status 

 Referrals of HIV negative individuals to HIV prevention, health education, and risk reduction 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 5 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Louisiana Public Health Institute (LPHI): Baton Rouge, LA 

The overall goal of the Louisiana Reentry Initiative (LRI) was to link and retain formerly incarcerated 

individuals living with HIV infection in Louisiana with an initial focus in Baton Rouge and services also 

provided in New Orleans. The project goals were to: 

 Strengthen the current linkage and retention system for formerly incarcerated PLWH 

 Implement a social stabilization intervention to link/retain formerly incarcerated PLWH 

 Create broader statewide dialogue surrounding the impact of stigma on access and retention in 

care 

 Promote the expansion and sustainability of retention programs for formerly incarcerated PLWH 

 Improve the coordination between the Greater New Orleans Health Information Exchange 

(GNOHIE) and the electronic medical record in the Orleans Parish Prison, to enable more 

effective coordination of care for individuals living with HIV releasing from the prison system 

back to the community 

LPHI served as the lead agency and evaluator for the LRI. 

Medical AIDS Outreach: Montgomery, AL 

Medical AIDS Outreach, Inc. (MAO) is a private, non-profit, community-based AIDS service organization 

that was established in 1987. It has transitioned from a volunteer education and service organization to 

a full-time primary care facility. By installing telemedicine equipment in satellite clinics staffed with a 

nurse throughout their network of partnering agencies, HIV-positive individuals living in rural areas of

Alabama gained access to HIV specific medical care without the burden of driving to an HIV-specific

medical clinic. The telemedicine equipment allowed a physician or nurse practitioner in the main office 

to look at a client who may be miles/hours away. This coordinated system of care ensured that all HIV 

positive individuals living in Alabama received the same level and quality of care. 

St. Louis Effort for AIDS: St. Louis, MI

The Barrier Elimination and Care Navigation (BEACON) Project sought to locate and return to care PLWH 

in the Saint Louis region with no evidence of HIV primary care within the past year. The Project was a 

collaboration between the City Health Department, Saint Louis Effort for AIDS, and Washington 

University’s Project ARK. When PLWH who were out of care were identified, they received services from 

one of the Care Navigation Teams. Each team was made up of an Engagement Coordinator (who also 

serves as their Ryan White Case Manager), two HIV-positive Peer Advocates, and a shared Community 

Nurse. BEACON clients had: 

 Access to Peer Advocates to manage barriers of stigma, disclosure, and fear 

 Support from Peer Advocates and Engagement Coordinators who assist clients in navigating 

social service resources 

 Access to the Community Nurse for education and support in managing their treatment for HIV

and any other medical conditions 

Washington AIDS Partnership: Washington, DC 

Positive Pathways was a program in the District of Columbia that assisted HIV-positive African Americans 

living in Wards 5–8 to participate with HIV medical care and supportive safety net services, with a 

particular focus on HIV-positive women and their partners. Led by Washington AIDS Partnership, 

Positive Pathways was a collaboration between eight community partners and the DC Department of 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 6 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Health. Through a network of 12 trained peer Community Health Workers (CHWs) placed in community 

and primary care settings, Positive Pathways worked to support program participants to take full

advantage of HIV medical care and other community services to improve their health and quality of life. 

CHWs worked at their employing organizations and in the community to identify individuals who were 

HIV-positive and not receiving HIV medical care in order to build trust and inform them about living with 

HIV, to provide personalized assistance to help them enter medical care, and to support them 

throughout the early part of their care until they were fully involved. Peers utilized their unique position 

to address barriers to care that could present challenges for other medical professionals. These CHWs 

provided trust-based information and education; helped clients overcome fear, denial, and stigma;

conducted outreach and support deep in the community; took the time to walk clients through the 

often overwhelming healthcare system; and helped clients strategize to manage the logistics of caring 

for oneself in the context of a complicated life. 

University of Alabama at Birmingham (UAB): Birmingham, AL 

Birmingham Access to Care (BA2C) was a collaborative effort between UAB 1917 Clinic and Birmingham 

AIDS Outreach (BAO). HIV primary care and social service providers have, for years, recognized the 

profound public health implications of the "lost to follow-up," or fallen out of care, population. 

Improving engagement in HIV care for PLWH not currently seeking primary care treatment may have 

other beneficial effects for the public health. Reductions in risk transmission behaviors among 

individuals linked to care and the beneficial effects of ART in reducing HIV viral load and transmissibility 

may result in decreased secondary cases of HIV infection if the program is successful in improving 

linkage, retention, and re-engagement in HIV care. BA2C had a team of two Board-Licensed Social

Workers, data programmers, and project supervisors to focus on the following goals: 

 Increase re-engagement/linkage to HIV primary care among HIV-infected individuals lost to 

follow-up 

 Obtain information regarding individual-level facilitators and barriers to HIV primary care linkage 

 Increase the proportion of individuals lost to follow-up annually retained in comprehensive and 

coordinated outpatient medical services 

In collaboration with Johns Hopkins University (JHU), AIDS United conducted a single, multi-part 

evaluation of 12 subgrantee programs, all of which provided services to people living with HIV (PLWH) to 

improve access to care and health outcomes. The overall program, Access to Care (A2C), aimed to 

increase access to and retention in HIV healthcare and support services. SIF served PLWH who were at 

greatest need, including individuals who knew their HIV status but were not in care due to barriers such 

as poverty, homelessness, untreated mental health issues, and substance abuse. SIF was a national 

program with projects located in the East Coast, South, Midwest, and West Coast. While the specifics of

each SIF program were unique, all were evidence based. In addition, SIF programs shared common 

program models, including peer/patient navigation, community health workers, care coordination, 

intensive case management, and motivational interviewing. 

SIF served a total of 4,7042 participants. Details of the 12 programs, the populations they serve, the 

program model, program activities, and the number of enrolled participants are outlined below in Table

1. Because intervention models and populations served vary from site to site, numbers of participants 

served also vary. 

2 UAB Control arm (83 clients enrolled) not included in this total. The total enrolled, including UAB’s control arm, is

4,787. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 7 
’

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table 1: SIF subgrantees and number of participants served
	

Lead organization

and location 

Target population Program model Program activities # Participants

served 

ActionAIDS, 

Philadelphia 

Recently

incarcerated 

Intensive case 

management 

Case management,

expedited housing, back to

work, and behavioral

health services 

78 

AIDS Action 

Committee,

Boston, MA 

PLWH who are

non-US born

women, MSM or

injection drug users 

Integrated

advocacy team 

Peer support, health 

navigation, care 

coordination, motivational

interviewing 

342 

AIDS Foundation 

Chicago (AFC),

Chicago, IL 

PLWH with a focus 

on people of color 

Peer navigation 

and care

coordination 

System level

enhancements, client level

outreach, linkage to care, 

client level education 

639 

Amida Care: New 

York, NY 

Low-income PLWH Assertive 

community

treatment 

Mobile Engagement

Teams, community health 

workers, care

coordination, and 

motivational interviewing 

833 

AIDS Project Los

Angeles (APLA): Los 

Angeles, CA 

Low-income PLWH

who are MSM, 

women,

transgender, or

heterosexual men 

Strengths-based 

case 

management 

Peer navigation and 

motivational interviewing 

327 

Christie s Place 

(CP): San Diego, CA 

PLWH who are 

women of color 

with an emphasis

on Latinas 

Network of care Peer navigation, care 

coordination, motivational

interviewing, traumainformed care 

226 

Damien Center: 

Indianapolis, IN 

PLWH, 18-44, 

African American, 

Latino/a 

Peer navigation, 

care coordination 

Linkage to care,

coordinated services 

149 

Louisiana Public

Health Institute 

(LPHI): Baton

Rouge, LA 

PLWH, African 

American, Latino/a, 

25-44, recently

incarcerated 

Intensive case 

management 

Peer navigation, ARTAS, 

pre/post release case 

management; disease 

intervention specialists 

144 

Medical AIDS

Outreach (MAO), 

PLWH in the rural Telemedicine HIV medical care, nursing 

services, mental health 

244 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 8 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Lead organization

and location 

Target population Program model Program activities # Participants

served 

Montgomery, AL South and pharmacy services 

St. Louis Effort for 

AIDS: St. Louis, MI 

PLWH who are

ethnic minorities 

with a focus on 

African Americans 

Care navigation 

team 

Peer navigation and care 

coordination 

322 

Washington AIDS

Partnership (WAP):

Washington, DC 

Low-income, 

female, African 

American PLWH 

Peer navigation, 

care coordination 

Community health 

workers, linkage to care 

1313 

University of 

Alabama at 

Birmingham (UAB):

Birmingham, AL 

(intervention arm 

only) 

PLWH who have

fallen out of 

care/lost to follow 

up 

Strengths-based 

case 

management 

Care coordination, 

motivational interviewing 

87 

TOTAL 4,704 

Research Questions 

The evaluation of A2C included impact research questions, outcome research questions, and 

implementation research questions. All subgrantees participated in the outcome and implementation 

evaluation. Only one program site (lead grantee University of Alabama at Birmingham) conducted an 

impact evaluation. SIF research questions are outlined below by question type. 

Impact research questions (UAB)

Q1. Are participants who receive the BA2C intervention more likely to have a suppressed viral load at 12 

months than participants who receive standard of care? (Confirmatory) 

Outcome research questions (all subgrantees)

Individual level

Q2. What is the trend in mean CD4 from baseline to 12 months among SIF participants?
	

Q3. What is the trend in mean viral load from baseline to 12 months among SIF participants?
	

Q4. What is the trend in % of clients reporting an unsuppressed viral load from baseline to 12 months
	

among SIF participants? 

Q5. Compared to national statistics, does the SIF program have a higher percentage of participants 

linked to care, retained in care, on HAART, and with a suppressed viral load? 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 9 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Network level

Q6. Compared to the network six months prior to SIF, has the subcontractor network increased in

density?

Q7. Compared to the network six months prior to SIF, does the subcontractor network average node
	

degree increased?
	

Cost-analysis questions 

Q8. What is the cost saving threshold for SIF?
	

Q9. What is the cost effectiveness threshold for SIF?
	

Q10. What is the cost per quality adjusted life-year (QALY) saved by delivering services in each program
	

area?
	

Implementation research questions 

All subgrantees will contribute data to answer the implementation research questions except for Q13
	

and Q15, which will only be answered by UAB.
	

Q11. How many clients were served by SIF?
	

Q12. How many clients were linked to care?
	

Q13. Are participants in UAB’s BA2C intervention arm more likely to be linked to care than participants in

the standard-of-care arm? (UAB only)

Q14. How many clients were retained in care?
	

Q15. Are participants in UAB’s BA2C intervention arm more likely to be retained in care than participants
	

in the standard-of-care arm? (UAB only)?

Q.16 How many clients have a current prescription for ART? 

Q17. How have subgrantees implemented peer navigation models/strategies?3 

Q17a. What have been the biggest barriers to implementing peer navigation models/strategies? 

Q17b. What methods have been employed to overcome these barriers? 

Q17c. What has facilitated implementation of the peer navigation models/strategies? 

Q17d. What has fostered these facilitating factors? 

Q18. How have subgrantees implemented community health worker models/strategies?4 

Q18a. What have been the biggest barriers to implementing community health worker 

models/strategies?

Q18b. What methods have been employed to overcome these barriers? 

Q18c. What has facilitated implementation of community health worker models/strategies? 

Q18d. What has fostered these facilitating factors? 

Q19. How have subgrantees implemented care co-ordination models/strategies?5 

Q19a. What have been the biggest barriers to implementing care co-ordination 

models/strategies?

Q19b. What methods have been employed to overcome these barriers? 

Q19c. What has facilitated implementation of care co-ordination models/strategies? 

Q19d. What has fostered these facilitating factors? 

3 Sites will only be asked about the strategies/models that pertain to their programs. 4 Sites will only be asked about the strategies/models that pertain to their programs. 

5 Sites will only be asked about the strategies/models that pertain to their programs. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 10 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Q20a. How have subgrantees implemented motivational interviewing?6 

Q20b. What have been the biggest barriers to implementing motivational interviewing 

models/strategies?

Q20c. What methods have been employed to overcome these barriers? 

Q20d. What has facilitated implementation of motivational interviewing models/strategies? 

Q20e. What has fostered these facilitating factors? 

Evaluation Overview 

The evaluation of the SIF programs included three components: an impact assessment, an outcome 

evaluation, and an implementation evaluation. There were five main methods used: a randomized 

control trial, monitoring of patient-level data, economic analyses, case studies, and monitoring of subcontractor networks. Table 2 below outlines the methods used for each of the three components of the 

SIF evaluation (impact, outcome, and implementation). The randomized control trial of BA2C contributed 

to the implementation evaluation as well as the impact assessment. Monitoring of patient-level data 

tracked data for the implementation evaluation as well as the outcome evaluation. The economic 

analysis and the monitoring of sub-grantee networks both contributed to the outcome evaluation while

the case studies are a methodology used for the implementation evaluation. 

Table 2: Methods used for the SIF implementation, outcome, and impact evaluation 

Impact Assessment Outcome Evaluation Implementation Evaluation 

Method 

Randomized 

control trial 

Comparison of viral 

suppression in

intervention arm to 

control arm 

Monitoring of fidelity to program 

model, quality of program delivery, 

participant responsiveness, 

program differentiation, 

participant satisfaction. 

Comparison of linkage and 

retention in care in the 

intervention arm to the control 

arm 

Monitoring of 

patient-level

data 

Monitoring of trends 

in mean CD4, mean

viral load, and

undetectable viral

load 

Number of clients: served, linked 

to care, retained in care, with 

HAART prescription 

Economic 

analysis 

Cost-saving and costeffective thresholds;

cost per QALY 

Case studies Qualitative exploration of SIF 

program models and barriers to 

implementation 

Monitoring subgrantee

networks 

Comparison of

network density and

average node degree 

6 Sites will only be asked about the strategies/models that pertain to their programs. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 11 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

In the following sections, we outline the following for each of the three components of the A2C 

evaluation, starting with the implementation evaluation and followed by the outcome evaluation and 

the impact assessment: 

 Study design and procedures 

 Analysis methods 

 Findings 

 Lessons learned 

Implementation Evaluation 

Study design and procedures

Three methods were employed to conduct the implementation evaluation: monitoring of patient-level

data, case studies, and the randomized control trial (RCT) impact assessment. 

Monitoring of patient-level data: The SIF project aimed to improve health outcomes by increasing 

utilization of HIV primary care services. Specifically, the program linked PLWH who were out of care into 

HIV care and treatment services. Linkage to care is necessary for participants to be retained in care and 

for access to HIV medication. SIF programs target various populations and work in variety of locations 

(refer to Table 1). 

SIF programs do not have sampling frames and do not employ sampling strategies. Rather, they use a 

variety of methods to identify individuals in need of program services, and then recruit these individuals

into their programs using recruitment techniques such as in-reach and peer-led outreach. Table 3 

outlines enrollment and twelve-month follow-up by site and enrollment targets over the grant period. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 12 
’

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table 3: Number of SIF participants enrolled, 12-month follow-up, and target enrollment over the fiveyear grant period for the first cohort/three-year grant period for the second cohort. 

Lead organization and location Baseline n Twelve-month 

n 

Target 

enrollment 

ActionAIDS, Philadelphia 78 49 100 

AIDS Action Committee, Boston, MA 342 211 463 

AIDS Foundation Chicago (AFC), 

Chicago, IL 

639 334 680 

Amida Care: New York, NY 833 690 1068 

AIDS Project Los Angeles (APLA): Los 

Angeles, CA 

327 245 695 

Christie s Place (CP): San Diego, CA 226 161 200 

Damien Center: Indianapolis, IN 149 93 160 

Louisiana Public Health Institute 

(LPHI): Baton Rouge, LA 

144 75 130 

Medical AIDS Outreach (MAO),

Montgomery, AL 

244 189 225 

St. Louis Effort for AIDS: St. Louis, MI 322 213 360 

Washington AIDS Partnership (WAP): 

Washington, DC 

1313 310 2,314 

University of Alabama at Birmingham 

(UAB): Birmingham, AL 

170 119 250 

As noted below in the Lessons Learned section, AIDS United and our subgrantees have found through 

the course of this initiative that the effort required to enroll and retain people with multiples barriers 

into HIV care was much greater than initially anticipated. Because of this, as well as delays in initial IRB 

approval, enrollment numbers have been lower than initially projected. 

The national evaluation for SIF gathered data on three measures of implementation at the patient-level:

linkage to care, retention in care, and having a current prescription for HAART. Linkage to care was our 

primary process measure. Table 4 provides information on the measure statement, cite, indicator, and 

frequency of data collection for linkage to care, retention in care, and HAART. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 13 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table 4: Implementation measures collected by SIF subgrantees
	

Construct 

Linked to care 

Retained in care 

Measurement

statement 

One medical visit 

with a provider who 

has prescribing 

privileges within 30 

days of enrollment 

(in a medical care 

setting with the 

purpose of 

receiving HIV 

related care)

Two or more

medical visits with a 

provider with 

prescribing 

privileges in a HIV

care setting at least

two months apart

during the prior 

year 

Cite 

Adapted from NYC

State Department

of Health AIDS 

Institute, 2006 

Horberg, 2010 

Indicator 

% of SIF clients 

linked to care with 

30 days of 

enrollment 

% of clients 

retained in care 

When collected 

Once at baseline 

(+30 days) 

Baseline, 6 mo., 12 

mo. 

HAART Current 

prescription for 

HAART 

NA % of clients with a 

current prescription 

for HAART 

Baseline, 6 mo., 12 

mo. 

Table 5 provides an overview of data collection procedures—specifically, the source for the data, how 

the data were collected, and who collected or abstracted the data for the program. These procedures 

varied by site. While this might be a limitation to the study design, the SIF programs were designed with 

sustainability as a focus, and therefore subgrantees did not develop entirely new data collection systems 

(which would have been necessary for total standardization across sites), but rather were asked to 

refine and enhance established systems. While the sources and methods for data collection were not 

standardized across sites, as outlined above in Table 4, the measures used were standardized across 

sites. 

As outlined in Table 5, the majority of subgrantees collected data on implementation measures through 

data abstraction from clinical records or surveillance. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 14 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table 5: Implementation data collection procedures by subgrantee 

Site Linked to Care Retained pt on 

Source Means Individual Source Means Individual Source Means Individual

ActionAIDS:

Philadelphia 

CAREware Interviewer 

administered or data

abstracted from 

medical record 

Care coach or care

outreach specialist 

CAREware Interviewer 

administered or data

abstraction from 

medical record 

Care coach or care

outreach specialist 

CAREware Interviewer 

administered

or data

abstraction 

from medical

record 

Care coach or care

outreach specialist 

AIDS Action 

Committee (AAC):

Boston, MA 

CM 

verification of

medical 

appointments 

Interviewer 

administered with data 

abstracted from ETO

database and phone or

paper verification from 

clinics 

Case manager Self report in 

case

management

assessment

and/or case

management

verification of

medical 

appointments 

Interviewer 

administered with data 

abstracted from ETO

database and phone or

paper verification from 

clinics 

Case manager Case 

manager

assessment 

Interviewer 

administered 

with data

abstracted

from ETO

database 

Case manager 

AIDS Foundation 

Chicago (AFC):

Chicago, IL 

Client Medical

Record 

Data abstraction Regional care 

coordinators 

Client

Medical

Record 

Data abstraction Regional care 

coordinators, 

evaluation manager 

Client

medical

record,

survey 

Data

abstraction, 

interviewer 

administered 

Regional care 

coordinators 

AIDS Project Los 

Angeles (APLA):

Los Angeles, CA 

Clinic records Phone call Peer navigator Clinic records Phone call Peer navigator Intake

assessment,

lab slips or 

medical 

release

consent to 

clinics 

Interviewer 

administered

or data

abstraction 

Peer navigator 

Amida Care: New 

York, NY 

’

Pharmacy

data & MET 

client Survey 

Abstraction and 

interviewer 

administered 

Program manager,

case manager, peers 

Claims data Abstraction Program manager,

case manager, peers 

Pharmacy

data, MET

client survey 

Abstraction 

and

interviewer 

administered 

Program manager,

case manager, peers 

Christie s Place 

(CP): San Diego, 

CA 

Management

reports/client 

survey 

Data

abstraction/interviewer 

administered 

Project

coordinator/peer 

navigator 

Management

reports/client 

survey 

Data

abstraction/interviewer 

administered 

Project

coordinator/peer 

navigator 

Management

reports 

Data

abstraction 

Peer navigator 

The Damien 

Center (TDC):

Indianapolis, IN 

Management

report/client 

survey 

Data

abstraction/Interviewer 

administered 

Director quality 

assurance/linkage to 

care staff 

Management

report/client 

survey 

Data

abstraction/Interviewer 

administered 

Director quality 

assurance/linkage to 

care staff 

Management

reports 

Data

abstraction 

Director quality 

assurance 

Louisiana Public

Health Institute 

(LPHI): Baton

Rouge, LA 

Surveillance 

data 

Data abstraction from 

state surveillance data 

OPH data

management/analysis 

manager 

Surveillance 

data 

Data abstraction from 

state surveillance data 

OPH data

management/analysis 

manager 

NA NA NA 

Medical AIDS 

Outreach (MAO):

Montgomery, AL 

CAREware Data abstraction Registered

nurse/social worker 

CAREware Data abstraction Registered

nurse/social worker 

CAREware Data

abstraction 

Registered

nurse/social worker 

St. Louis Effort for SCOUT Data abstraction Lead navigator SCOUT Data abstraction Lead navigator NA NA NA 

in Care HAART Prescri i

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 15 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

te 

AIDS (EFA): St. 

Louis, MI

Washington AIDS 

Partnership

(WAP):

Washington, DC 

Source Means 

EMR/EHR at

clinic sites, lab 

reports and

self-reports at

CBOs,

community 

health

workers

(CHW) report

when CHWs

attend visits 

with clients 

Abstracted and

interviewer 

administered 

Individual 

CHW 

Source Means 

EMR/EHR at

clinic sites, 

lab reports

and selfreports at

CBOs 

Abstracted and

interviewer 

administered 

Individual 

CHW 

pt on 

Source Means 

Client

records,

client selfreport with 

prescription 

verification at 

CBOs 

Abstracted

and

interviewer 

administered 

Individual 

CHW and state 

University of

Alabama at 

Birmingham 

(UAB):

Birmingham, AL 

(intervention arm 

only) 

Medical

Records 

Data abstraction Data Analyst Medical

Records 

Data abstraction Data Analyst Medical

Records + 

survey 

Data

abstraction + 

interviewer 

administered 

Study coordinator

and research 

assistant 

Si Linked to Care Retained in Care HAART Prescri i

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 16 
 

 

 

 

 

 

 

 

 

 

 

Table 6: SIFsubgrantee, number of interviews, organizations interviewed and interview dates for 

qualitative data collecti on 

SIF subgrantee Number of interviews Organizations interviewed Interview dates 

ActionAIDS		 Admin / Service : 

Admin: 1 

Service: 1 

3 ActionAIDS

COMHAR

Gaudenzia

Pathways to Housing 

April – May 2015 

AIDS Action Committee Admin: 6 

Service: 6		

AAC

Fenway

PACT/JRI

BIDMC 

ICH 

February – March 2014 

AIDS Foundation Chicago 		 Admin: 5 

Service: 4		

AFC

MATEC

Chicago House 

Mercy Care Program 

July 2012 – October 2012 

Amida Care		 Admin: 4 

Service: 4		

Amida Care

Housing Works 

Help/PSI

Harlem United 

June 2013 – July 2013 

AIDS Project Los Angeles 		 Admin: 5 

Service: 6		

APLA

DHSP

MCA Clinic

THE Clinic

Rand Schrader 

Northeast Valley 

December – J anuary 2014 

Christie’s Place		 Admin: 4 

Service: 2		

Christie’s Place 

AVRC

MCAP

NCHS 

May – June 2012 

The Damien Center 		 Admin:2 

Service:7		

The Damien Center 

Brothers United 

Women in Motion 

Indiana Latino Institute 

Community North Hospital 

Bellflower 

Life Care 

July – August 2015 

Effort for AI DS		 Admin: 3 

Service: 3		

Washington University 

Infectious Disease Clinic 

Washington University 

BEACON Team 

St. Louis Effort for AI DS 

April 2014 

Louisiana Public Health 		 Admin:6 Louisiana Public Health June – August 2015 

Case studies: The case studies provide detailed descriptions of the program models as well as barriers 

and facilitators of implementation. This portion of the evaluation used an embedded case study design 

where subgrantees were embedded units of analysis within the larger program. Data were collected by 

JHU staff via one-on-one interviews. Interviews were semi-structured. We aimed to complete one case 

study for each SIF location, for a total of 12 case studies. We aimed to interview approximately two 

individuals at each sub-contracting organization–one individual who works at the administrative level

(such as a supervisor) and one person who works at the service level (such as a peer). Interviews were 

conducted either over the phone or in person depending on sub-grantee location. Interviews are tape 

recorded and transcribed. Below is a table outlining by subgrantee the number of interviews conducted, 

the organizations interviewed, the interview dates: 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016		 17 
 

SIF subgrantee Number of interviews Organizations interviewed Interview dates 

Institute Service:2		 Institute 

Capitol Area Reentry Program 

CresentCare

Women with a Vi sion 

Louisiana State Health 

Department 

Medical AIDS Outreach Admin: 4		

Service: 5		

MAO-Montgomery

MAO-Selma

MAO-Dothan

AIDS Action Coal ition 

Selma AIR

September – October 2014 

University of Alabama 		 Admin: 2 

Service:2 

Birmingham AIDS Outreach 

University of Alabama at 

Birmingham 

January – February 2016 

Washington AIDS 

Partnership 

Admin: 6 

Service: 5 

Washington AIDS Partnership 

Whitman-Walker Health 

Unity Health Care 

Chartered Health Plan 

Women’s Collective 

Institute for Public Health 

Innovation 

Family Medical Counseling 

Services 

October – November 2012 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Case study data were collected using a semi-structured interview guide. The interview guide included 

questions and probes about the program models, barriers to implementing the program, as well as 

factors that facilitated implementation of the program. Some example case study questions included:

“What have been your biggest barriers to doing this work?”; “What successful strategies has [your 

organization] used to overcome these barriers?” and “Can you describe any unsuccessful strategies 

[your organization] tried?” 

BA2C RCT impact study: The impact assessment took place at the University of Alabama at Birmingham’s

1917 Clinic. Participants were randomized to either a standard-of-care arm or an intervention arm that 

provided strengths-based case management and motivational interviewing from community social

workers. A component of the implementation evaluation is embedded within the RCT impact study. 

Specifically, the study assessed differences between the intervention and comparison arm in linkage to 

care and retention in care. Measures used to assess linkage and retention in care for the RCT were 

identical to those described above in Table 4. HIV primary care visit information was abstracted from 

1917 Clinic’s medical records by the study’s data analyst. Additionally, we attempted to obtain releases 

of information from participants who elect to attend HIV primary care at a location other than the UAB 

1917 Clinic. (Further details about data collection are outlined below in the Impact Study section.) 

The randomized control trial enrolled 170 participants (87 in the intervention arm and 83 in the control 

arm). 

Analysis methods 

Please find below a description of the analysis method for the implementation evaluation by research 

question. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016		 18 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Implementation Evaluation:

Q11. How many clients were served by SIF? Count of number of enrolled participants.
	

Q12. How many clients were linked to care? Count of number of participants with verification of a visit

for HIV medical care with a provider with prescribing privileges.
	

Q13. Are participants in UAB’s BA2C intervention arm more likely to be linked to care than participants in

the standard-of-care arm? (UAB only) We answered this question using unconditional univariate odds
	

ratios with a 95% confidence interval. (See Q1 for additional information on our analysis plan for
	

assessing associations.)
	

Q14. How many clients were retained in care? Count of number of participants with two visits at least
	

two months apart in the past twelve months.
	

Q15. Are participants in UAB’s BA2C intervention arm more likely to be retained in care than participants
	

in the standard-of-care arm? (UAB only) We answered this question using unconditional univariate odds
	

ratios with a 95% confidence interval. (See Q1 for additional information on our analysis plan for
	

assessing associations.)
	

Q16. How many clients have a current prescription for HAART? Count of number of participants with a
	

current prescription for HAART.
	

Q17-20. How have sub-grantee sites implemented the following program models/strategies: peer 

navigation, community health workers, care coordination, motivational interviewing? (case studies) 

Qualitative data were transcribed verbatim. We used directed content analysis to analyze our data. Our 

interview guide included open-ended questions as well as probes to explore participants’ experiences 

with predetermined categories such as collaboration, barriers, and relationships. Initially, we read 

through all transcripts without coding to allow for immersion. We then simultaneously coded the data 

and made notes of additional themes not captured in the initial coding schema. Our initial coding 

schema was created based on predefined research questions, the research literature, and our interview 

guide. The data were coded by the PI in this study (Dr. Maulsby) and five PhD student research 

assistants. Throughout the coding process, the team met regularly (weekly to bi-monthly) to discuss any 

discrepancies between coders as well as to discuss themes that emerged throughout the coding process. 

We discussed themes at both the site level and across sites. Codes were further defined and re-classified 

based on these discussions and resolution of coding discrepancies. 

Findings 

Monitoring of patient-level data: The implementation evaluation of SIF gathers data on enrollment, 

linkage to care, and retention in care. Findings are below in Table 7. 

The percentage of participants linked to care within 30 days varies across sites from 28% in Indianapolis, 

Indiana and Birmingham, Alabama to 100% in Los Angeles, California and Montgomery, Alabama. Across 

the cohort, 59% (2,779/4,704)7 of participants were linked to care within 30 days of enrollment and 86% 

(4022/4704) of participants were linked to care during the program. The majority of subgrantees reported 

HIV retention in care percentages above 75% at twelve months. For BA2C, as part of the additional 

7 UAB control excluded from the numerator and denominator (n=83). 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 19 
’

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

analyses of the data from the RCT, we also assessed retention in HIV care at 12 months with missing as 

failure (rather than missing=missing, which is presented below in Table 7). When missing was 

considered failure we found the 62.7% of the control arm were retained in care at 12 months (52/83)

compared to 59.8% (52/87) of the intervention arm. 

Across the cohort, 63% of participants (1570/2490) had current HAART prescriptions twelve months 

after enrollment. Two subgrantees (Los Angeles and Indianapolis) were not able to contribute to 

portions of this part of the implementation evaluation due to challenges in gaining access to data on 

HAART. 

Table 7: Linkage, retention, and HAART use among SIF participants by grantee 

Lead organization and location # 

participants

served 

Linked to 

care within

30 days

% (n/d) 

Linked to care

(without 30

days’ restriction)

% (n/d) 

Retained in 

care (at twelve 

months)

% (n/d) 

HAART 

prescription (at 

twelve months)

% (n/d)

ActionAIDS, Philadelphia 78 83% (65/78) 97% (76/78) 82% (40/49) 92% (45/49) 

AIDS Action Committee, Boston, 

MA 

342 31%

(106/342) 

82% (279/342) 85% (180/211) 68% (144/211)* 

AIDS Foundation Chicago (AFC), 

Chicago, IL 

639 57%

(366/639)* 

69% (440/639)* 45%

(150/334)* 

59% (198/334)* 

Amida Care: New York, NY 833 53%

(440/833) 

87% (722/833) 73% (506/690) 51% (349/690) 

AIDS Project Los Angeles (APLA): 

APLA arm 

245 100%

(245/245) 

100% (245/245) 65%

(128/197)* 

63% (125/197)* 

AIDS Project Los Angeles (APLA): 

DHSP arm 

82 83% (68/82) 95% (78/82) 92% (44/48) Not available8 

Christie s Place (CP): San Diego, 

CA 

226 91%

(205/226) 

93% (210/226) 74%

(119/161)* 

69% (111/161)* 

Damien Center: Indianapolis, IN 149 28%

(41/149)* 

79% (118/149)* 85% (79/93)* Not available 

Louisiana Public Health Institute 

(LPHI): Baton Rouge, LA9 

144 40% 83% (120/144) 60% (45/75)* 36% (27/75)* 

8 DHSP collected data on HAART use at baseline only.
	 9 Louisiana gets their visit and lab from state partners. There is a considerable delay in getting this data, usually

several months. This delay is anticipated and results in their clinic and lab data looking artificially low.
	

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 20 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Lead organization and location # 

participants

served 

Linked to 

care within

30 days

% (n/d) 

Linked to care

(without 30

days’ restriction)

% (n/d) 

Retained in 

care (at twelve 

months)

% (n/d) 

HAART 

prescription (at 

twelve months)

% (n/d)

(57/144) 

Medical AIDS Outreach (MAO),

Montgomery, AL 

244 100%

(244/244) 

100% (244/244) 95% (179/189) 96% (181/189) 

St. Louis Effort for AIDS: St. 

Louis, MI 

322 45%

(144/322)* 

89% (286/322) 90% (191/213) 73% (156/213)* 

Washington AIDS Partnership

(WAP): Washington, DC 

1313 59%

(774/1313) 

86% (1124/1313) 70% (216/310) 57% (178/310)* 

University of Alabama at 

Birmingham (UAB): Birmingham, 

AL (intervention arm only) 

87 28% (24/87) 93% (81/87) 85% (52/61) 92% (56/61) 

University of Alabama (UAB): 

Birmingham, AL (control arm 

only) 

83 24% (20/83) 92% (76/83) 90% (52/58) 90% (52/58) 

TOTAL (excluding UAB control 

arm) 

4704 59%

(2779/4704) 

86% (4022/4704) 73%

(1929/2631) 

63%

(1570/2490) 

*Greater than 10% missing. Interpret with caution. 

Case studies:10 

The case studies qualitatively assessed barriers and facilitators of program implementation. The sections 

below highlight cross-subgrantee findings. The accompanying report, “Social Innovations Fund Case 

Studies: Experiences with Linkage and Retention in HIV Care across Twelve Subgrantees and their

Partners,” provides a more detailed site-level analysis of program implementation, including a 

description of program models. 

Barriers to implementing linkage and retention in care programs 

Four themes that emerged across the SIF programs as barriers to program implementation were the 

intensity of client needs, recruiting and retaining participants, staffing and administrative hurdles, and 

challenges related to working with multiple partner organizations. These cross-cutting themes are 

discussed in more detail below. 

Theme 1: Intensity of client needs 

Across all 12 SIF subgrantees, the intensity of client needs posed a formidable barrier to program 

implementation. SIF participants had complex, interrelated needs that needed to be addressed before 

participants’ health care could become a priority. Specifically, untreated mental illness and substance 

10 Detailed descriptions of the sub-grantee program models and sub-grantee level findings are included in the 

accompanying report, Social Innovations Fund Case Studies: Experiences with Linkage and Retention in HIV Care 

across Twelve Subgrantees and their Partners 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 21 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

use were HIV comorbidities that were universally mentioned as unmet needs that prevented 

participants from successful re-engaging in care. All subgrantees described a lack of stable, affordable

housing. They explained that lack of housing posed the greatest barrier to linkage and retention in care. 

Subgrantees also found a severe dearth of access to resources and services to address the housing, 

mental health, and substance use needs of their clients. Additional needs that were also frequently

highlighted by subgrantees were employment, transportation, and dental care needs. 

To address clients’ needs, most SIF programs referred their participants to high-quality social services 

programs with which their staff had an ongoing relationship. These programs were sometimes housed 

within the lead SIF agency or a partner agency; other times, these programs were external to the SIF

network. SIF implementers had a deep knowledge of the program being referred to and a strong 

relationship with the individuals who provided social services, which led to a smooth referral process. 

Some SIF subgrantees referred directly to social services while others worked through existing case 

management programs. Regardless of the strategy used, however, program implementers emphasized 

severe shortages of services, especially for housing, mental health, substance use, and dental services. 

Theme 2: Recruitment and retention 

Most SIF subgrantees faced challenges with participant recruitment. The challenges varied considerably

depending on the recruitment strategy used. All implementers reported difficulties in communication 

with potential participants, enrolled participants, or other agencies working with PLWH. Many 

participants were geographically unstable, and subgrantees found it challenging to locate participants 

with out-of-date addresses and phone numbers. As a result, contact records, from sources such as 

medical records and case management records, were often outdated. Many subgrantees described 

locating participants as a time intensive and burdensome process with low yield. Successful participant 

recruitment and retention relied on reliable, ongoing communication between and within partner 

agencies about participants. Some agencies faced barriers to receiving and sharing participant-level

information with health care providers, community based organizations, health departments, and social 

organizations because of Health Insurance Portability Accountability Act (HIPAA) restraints. Other 

subgrantees, such as Amida Care, faced challenges with intra-agency communication systems that were 

not optimal. Subgrantees working with incarcerated populations faced unique communication 

challenges as their program participants transitioned out of jail or prison: incarcerated individuals often 

did not know their residence immediately following release and also lacked access to a stable phone 

number. 

Program implementers adopted a range of diverse and creative strategies to reach clients and indicated 

that it was important that programs have the flexibility to try a variety of approaches. Subgrantees 

facing challenges reaching participants on out-of-care lists made course corrections and relied more 

heavily on recruitment and referrals through partner organizations, and other avenues, such as a call-in

hotline. Email and social media, such as Facebook, also emerged as a successful way to keep in contact 

with participants, even those with many basic needs. One of the most important strategies for retaining 

participants was the quality of relationships with A2C staff. 

Theme 3: Staffing and administrative challenges 

The majority of subgrantees faced several challenges related to staffing. Many programs experienced 

high staff turnover. Turnover among staff who had direct participant contact, such as navigators and 

community health workers, was viewed as particularly problematic for linkage and retention in care 

programs. Respondents described that successful linkage and retention in HIV care relied heavily on 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 22 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

having strong rapport with participants, as well as strong relationships within the community. This cadre 

also needed intense, ongoing job training. Building necessary skills and relationships took time and 

commitment. Building relationships, in particular, compounded challenges when transitioning in new 

staff. A closely related theme was a high level of burnout among staff who worked directly with clients. 

While a small number of interviewees attributed this to an unsupportive work environment, others 

explained that staff burnout was a result of the demanding and highly personal nature of the work, 

particularly for peers. Peers, who were usually PLWH, often had similar background to participants and 

shared their own stories when relating with and motivating participants to engage in HIV medical care. 

Program implementers created opportunities, such as peer meetings and all program staff meetings, for 

direct service staff to share their experiences, lessons learned, and to problem solve challenges in 

program implementation. 

While all subgrantees that used peer models saw peers as vital to engaging participants, several

reported challenges around recruiting and training peers. Peers handled significant responsibility across 

a wide variety of areas and finding individuals with the unique and varied skillset needed was 

challenging. Respondents reported a need to conduct ongoing, supportive training in areas such as 

program implementation, data collection, professional conduct, and maintaining professional

boundaries with participants. Both direct service staff and supervisory staff reported that a close 

working relationship between direct service staff and their supervisors was critical for addressing some 

of the challenges faced by peers. 

A related administrative challenge was the burden of data collection, data management, and meeting 

reporting requirements. Several subgrantees indicated that that their program faced challenges meeting 

the reporting requirements of various funders, including SIF match funders and other federal funders. 

Respondents reported frustration over data collection systems and processes that were time intensive, 

duplicative, and incompatible across partner agencies. A few subgrantees reported that expectations or 

methods for data collection were not clear and changed midway through the program without 

explanation. To address these challenges, grantees adapted their data collection systems (for example

by switching from paper to electronic systems), provided additional training and supervision of staff, and 

developed creative ways to provide time for data related activities (such as designated meetings for 

data entry and management). 

Theme 4: Partnerships 

Although partnerships between implementing partners were seen as critical to successfully linking and 

retaining clients in care, significant time and effort was needed to harmonize these relationships. 

Through the Access to Care initiative, very different organizations had the opportunity to work together 

towards a common goal―and at times these differences were challenging. Respondents indicated that 

they needed to overcome cultural differences between organizations. Initially, some HIV medical 

providers were reluctant to work with direct service staff. There were also differences in the 

administrative cultures of organizations. While some partners were quite nimble, others faced 

considerable bureaucracy. Several sites reported challenges in obtaining buy-in from other local 

agencies that focused on PLWH. This barrier was explained as a result of a scarcity of resources, which 

fueled competition for clients and heightened concern over duplication of services. 

The primary strategy used to address these challenges was open communication to quell concerns and 

to build mutual understanding in an effort to develop even stronger working relationships across 

organizations. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 23 
“ '

“

“ ’

“

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table 8: Cross-cutting barriers to program implementations
	

Theme 

1. Intensity of client needs 

Illustrative quote 

...they re just trying to maintain life, and so when you get to a point where you 

don't know where you're staying that night, you don't know if you'll be safe where 

you're staying wherever you can find, if you'll be eating, that takes precedence 

over ‘Am I going to go to my doctor today? Did I get my script filled on time?’ And 

then also you have other issues coming into play, trauma, maladaptive coping 

skills like substance abuse that leads to addiction. All of that works as barriers 

against linkage to care and continuity of care.” (Louisiana)

2. Recruitment and 

retention challenges 

One of the first barriers, I think, has just been trying to locate clients that 

are considered lost to care. Sometimes locator information at the clinic

was poor and we tried out basically every other method available to us 

and we still were not able to find patients in the field, clients that have 

been lost to care. So that’s been a huge barrier to the program. I think 

more than 30 percent of the clients that we’ve been trying to locate we’ve 

just not been able to find at all. And so I think that’s been a lesson learned 

for our program.” (Los Angeles) 

3. Staffing and 

administrative 

challenges 

We don t want them [peers] to do straight up counseling with their 

supervisor, but they need an outlet and a regular one in order to survive 

doing the work because it is really demanding. A good supervisor can help

a peer with is even though the line’s a little more blurry for a peer than for 

a traditional staff person, we still need them to maintain some boundaries 

on a professional level and for their own well-being. So being able to 

develop a really trusting relationship with a clinically trained supervisor I

think is important to help anticipate what’s going to happen in this 

relationship.” (St. Louis) 

4.Harmonizing 

partnerships 

Originally, it was challenging or interesting in terms of getting the peer 

as a respected member of the team amongst medical providers, and I will

say that she has, partially because of who she is, but she’s been incredibly

successful in establishing herself as a professional, and that has impacted 

the way in which the medical providers even see other peer navigators at 

[organization name] within the community, and I believe that it’s 

positively impacted how well respected they are.” (San Diego) 

Cross-cutting facilitators to implementing linkage and retention in care programs 

Three themes emerged as factors that facilitated SIF program implementation were strong relationships 

with partner organizations; the flexible and innovative nature of the SIF programs; and the expertise, 

dedication, and passion of staff. These factors are described in detail below. 

Theme 1: Strong partnerships with organizations with expertise in a range of areas 

Across all sites, respondents reported that having strong partnerships with agencies that provided 

expertise in a range of areas was critical to program success. Specifically, subgrantees indicated the 

importance of having strong relationships with organizations providing social services and HIV primary 

care. For programs where the lead agency did not provide wraparound services, it was necessary to 

partner with organizations providing these services to meet the needs of participants. In particular, 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 24 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

housing, and mental health and substance use treatment was needed. Having strong partnerships with 

medical providers was also critically important, and many subgrantees recommended basing their staff

at medical providers’ offices as a way to strengthen these relationships. For example, one subgrantee 

had co-located case management, peer navigation, HIV primary care, and pharmacy services. Another 

subgrantee lauded working with small HIV primary care providers because small HIV primary care 

providers were viewed as having a keen grasp of community needs. A related theme was the 

importance of partnering with organizations that had expertise working with specific populations, 

including lesbian, gay, bisexual, transgender, and queer/questioning (LGBTQ) individuals and racial 

minorities who face unique challenges related to immigration or multiple forms of stigma. 

Strong relationships with organizations providing social services and with HIV primary care providers 

were useful for identifying potential participants and provided the benefit of a “warm” handoff. Because 

of these partnerships, SIF subgrantees were able to better serve their SIF clients―care was provided 

more quickly, a greater variety of services were provided to participants, and follow-up with clients who 

missed appointments was more achievable. In addition, strong relationships with HIV medical providers 

facilitated timely data collection and reporting for tracking participant health status and for meeting 

evaluation requirements. 

Respondents reported that having strong relationships at the onset of the project, having staff

dedicated to building and maintaining relationships, and frequent communication between partners 

facilitated the building of strong partnerships. Specifically, respondents recommended open 

communication from the onset of the program (even as early as the proposal writing stage) and 

maintaining strong communication through regular (e.g., monthly) partner meetings. During these 

meetings, subgrantees reported focusing on topics such as strategizing to address challenges, sharing 

lessons learned, handling issues related to caseloads (such as size, complexity, and specific cases),

trainings, supporting staff, and building new relationships with organizations. Several respondents also 

reported that a lead agency that was established and well-respected within the community was helpful

for building partnerships. 

Theme 2: Innovative and flexible program models

Several respondents indicated that innovations unique to their interventions facilitated success. These 

innovations were methodological as well as programmatic. For example, in Philadelphia, ActionAIDS 

developed an Acuity Scale that they used to assess participant vulnerability. The scale was developed 

over 18 months with staff from partner organizations. The Acuity Scale’s scoring system was used to 

determine clients’ needs and to tailor the intervention to the participant. Louisiana’s SIF program used 

process mapping to identify bottlenecks, improve efficiency, and identify optimal opportunities for 

meeting with participants within the jail. In Washington, DC, the SIF program worked with the University 

of the District of Columbia to develop an eight-month training for community health outreach workers, 

and in St. Louis, an emergency stabilization fund granted program participants access to a remedy for 

unexpected expenses. This fund helped BEACON participants to eliminate barriers outside the scope of

many other funding streams, such as Ryan White Funding. 

Trial and error are inherent to the iterative processes of program innovation. Respondents described the 

flexible nature of the SIF programs as being important for quick programmatic course corrections. 

Respondents indicated the utility of being able to adapt program strategies. This flexible approach 

facilitated the acknowledgment of implementation challenges and provided subgrantees with the 

opportunity to adapt program strategies accordingly, resulting in strong SIF program models. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 25 
“

“

“

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Theme 3: Characteristics of staff

Across all respondents, the passion and dedication of staff were universally recognized as being 

instrumental to program success. In particular, respondents lauded the connections between direct 

service staff and the individuals served by the program. Respondents noted the importance of hiring 

staff with whom the SIF program participants could relate. Peers were recognized for their unique ability 

to build trust and rapport with clients and to serve as role models in ways that other cadre of staff (such 

as nurses and case managers) was not. Supervisors of direct service staff played an important role by 

providing support, guidance, and supervision to direct service staff. Having clearly defined roles and 

responsibilities (for example between case management and direct services staff) helped to ensure fluid, 

seamless, and team-oriented relationships among staff. 

Table 9: Cross-cutting facilitators of program implementation 

Theme Illustrative quote 

1. Strong partnerships 

with organizations 

with expertise in a 

range of areas 

I think what has worked best for us...relationship, relationship, 

relationship. The old kind of old boys’ network that we talked about years 

ago. Things that happen in the back room. I think that [Organization D 

colleague] has worked really hard at developing very strong relationships 

with some of our key referral sources...we’ve got a few that we can really

kind of count on and rely on. And she’s just really worked hard at 

developing a trusting relationship with them, being incredibly reliable. 

They can count on her, when they call we will return the phone call, we 

will be there. We get information back to them so there’s a nice kind of 

collaborative process that goes on...” (Chicago) 

2. Characteristics of staff You know most times people make that first initial appointment, 

sometimes they don’t, but sometimes they make that first initial 

appointment and then make a decision not to go back to the doctor ‘I’m

not going to deal with this right now.’ But some form of magic takes place 

in the comfort that [linkage specialist] is able to offer to these people.” 

(Indianapolis)

3. Innovative and flexible

program models 

The creation of the acuity vulnerability scale under our ASSIST program is now is

being used across the board, both in our general prison program, and our agency 

program, in our CARE Coach model. Everybody who is incarcerated and we get a 

referral for, has an acuity vulnerability done on them from the intake on, and 

then every six months after that. So that is huge, that's one huge impact. It's 

really helped us to look at where our clients are. You know we talk

programmatically, we're going to meet clients where we are, but how do we 

know where they are if we don't gauge it, and we don't continue to gauge it? So, 

that's been a really, it's something that, you know, has changed the way that we 

do things.” (Philadelphia) 

BA2C randomized control trial: We observed that intervention arm participants were more likely to be 

linked to care, although these results were not statistically significant. Retention in care at T2 was more 

likely for participants in the control arm, although these results were also not statistically significant. 

These results may be explained by the nature of the intervention which provided intensive support for 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 26 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

individuals for the first six months of study enrollment, thus potentially providing impact on initial

linkage to care. Once the intensive phase of the intervention was complete, participants may have had 

more barriers to sustaining retention in medical care. 

Table 10: Linkage and retention in care among UAB Birmingham Access to Care (BA2C) participants 

Patient

characteristics, 

n (%) or median 

(IQR)

Linked to care within

30 days of contact 

Linked to care 

without 30-day

parameter 

Intervention 

n/N (%) 

24/87 27.6% 

81/87 93.1% 

Control 

n/N (%) 

20/83 24.1% 

76/83 91.6% 

OR (95% CI) 

1.2 (0.6-2.4) 

1.2 (0.4-3.9) 

p-value 

0.6 

0.7 

Retained in care at 

T211 52/61 85.3% 52/55 94.6% 0.3 (0.1-1.3) 0.1 

Lessons learned 

The implementation evaluation suggests that SIF grantees are successfully implementing HIV linkage and 

retention in care programs. 4,704 participants are enrolled in the evaluation (with additional clients 

being served through the program but not enrolled in evaluation). 

Monitoring of patient-level data: One of the key lessons learned through this initiative is the very high 

degree of effort that is required to engage people living with HIV who have fallen out of care back into 

care, particularly when they are experiencing multiple competing priorities such as homelessness or 

unstable housing, mental health issues, substance abuse, histories of trauma, other medical concerns, 

and employment and transportation challenges, among others. Many of our subgrantees proposed very 

high enrollment numbers in their initial proposals for funding and, even with subsequent reductions in 

enrollment goals upon funding award, have found that more effort than originally anticipated has been 

required to connect people with multiple barriers to care into HIV care. Accordingly, actual enrollment 

has been lower than initially projected. Fifty-nine percent of these participants were linked to care 

within thirty days of enrollment and 86% were linked to care at some point following enrollment. Across 

the entire cohort, among those with twelve-month data, 73% were retained in care and 63% had a 

current HAART prescription. Nationally among the 1.2 million estimated PLWH, the CDC estimates that 

40% are engaged in care12 and 37% are prescribed HAART (1). 

Case studies: The case studies assessed and analyzed barriers and facilitators of program 

implementation. Across sites, several cross-cutting barriers emerged. Universally, the high level of

participant need was a barrier to implementing linkage and retention in care programs. Staff reported a 

high acuity of unmet needs among participants and a dearth of local resources (such as housing 

programs) to address these needs. The most frequently mentioned needs among individuals

interviewed were housing and services for mental health issues and substance use. Staff reported a 

range of strategies to help participants address these needs (including referrals to partner agencies and 

11 Denominator limited to all participants with a HIV medical visit with a provider with prescribing privileges in the
	

past year.

12 Engaged in care defined by the CDC as having a HIV medical care visit during the Medical Monitoring Project’s
	

sampling period of Jan-April 2011.
	

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 27 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

helping participants to navigate social services). Having close relationships with partner organizations 

was seen as a vital to making sure that participant needs could be addressed. 

Recruitment and retention of program participants was another cross-cutting challenge to program 

implementation. Contact details of potential and enrolled participants were often out of date. 

Participants were highly transient, and this complicated locating participants for enrollment and 

services. Again, partnerships with community-based agencies that provided social services was an 

important strategy used to identify potential participants through referrals into the program and to keep 

in contact with existing participants. 

Staffing and administrative challenges also posed barriers to program implementation, in particular high 

staff turnover, a need for a frequent staff training, and difficulty finding individuals with the unique 

qualifications needed to successfully provide direct linkage and retention in care services. While strong 

partnerships with a diverse set of organizations was seen as a necessary element for successful program 

implementation, it also brought with it challenges. The most commonly mentioned challenges in this 

area were cultural differences between organizational partners and competition between agencies for 

funding and participants. 

In addition to strong partnerships, another facilitator of program implementation was the unique 

innovations of the SIF programs (such as ActionAIDS’s Acuity Scale, WAP’s CHW training, and St. Louis’s

emergency stabilization fund). Finally, the dedication and passion of the staff were universally

recognized as being instrumental to program success. In particular, peers were lauded for their unique 

ability to relate with participants, build rapport, and act as positive role models. 

BA2C Randomized Control Trial (RCT): Many of the lessons learned from the RCT mirror those of other 

SIF projects. Once out of care participants were identified and located, achieving re-engagement in, or 

linkage to, care within 30 days is extremely difficult, even for participants in the intervention arm who 

received additional support through a community social worker. Again, we believe that this is primarily

due to the extraordinary challenges in daily life that many in this population face. Anecdotally, we often 

spoke of “life chaos” as a reason that many participants were not immediately re-engaged; however, 

even our conceptualization of “life chaos” did not include the extreme socio-economic and psychological

barriers faced by individuals living in poverty, often in the remotest of locations. However, when given 

time and consistent support, the most vulnerable individuals can be located, re-engaged, and, 

ultimately, retained in HIV medical care. 

Outcome Evaluation 

Level of evidence, outcome evaluation 

The monitoring of patient-level data in the outcome evaluation has low internal validity and low to 

moderate external validity. This portion of the national evaluation used a pre-/post-test design, and 

therefore is subject to threats to internal validity. Pre-/post test designs are subject to all seven of the 

primary threats of internal validity (history, maturation, statistical regression, selection bias, 

experimental mortality, instrumentation and testing). This is primarily due to the fact that a pre-/post 

test design is nonexperimental and does not include a randomized control group (or other design 

features) to address these threats. In addition, this portion of the national evaluation used purposive 

sampling, which weakens external validity. However, it was conducted in a variety of settings and 

locations across the United States with diverse target populations. Similar findings across subgrantees 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 28 
’

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

could suggest a higher level of generalizability. The case studies were qualitative and were conducted in

a way as to achieve high to moderate validity including an analysis of disconfirming or deviant cases and 

respondent validation (2). 

Study design and procedures

Three methods will be employed to conduct the outcome evaluation: monitoring of patient-level data, 

economic analysis, and monitoring sub-grantee networks. 

Monitoring of patient-level data: The national evaluation of SIF gathered data on three outcome 

measures: mean CD4, mean viral load, and percent suppressed viral load. Baseline data were collected 

at enrollment or soon thereafter. Therefore, baseline measures aimed to represent clients’ health status 

prior to exposure to the program. Follow-up data were collected twelve months following enrollment. 

Outcome data were collected using a variety of methods and sources (Table 11). The majority of

outcome data were extracted from medical records, including existing electronic databases, or from 

paper records. If staff were not familiar with the databases or data collection at the start of the project, 

they received training in timely and complete data extraction techniques and continued to receive ongoing support as needed. Data were collected, monitored, and cleaned on an on-going basis by the 

subgrantees. Subgrantees reported aggregated data to JHU every six months via standardized excel

spreadsheets (data shells) that included a series of monitoring data checks. JHU checked the data shells

for data inconsistencies and analyzed the aggregate data for monitoring and evaluation purposes. 

Table 11: Outcomes data sources by subgrantee

Lead organization and

location 

Source for outcome data Outcome data collection

method 

Individual responsible

for outcome data 

collection 

ActionAIDS, Philadelphia CareWARE Interviewer administered 

or data extraction from 

medical record 

Care coach

or care outreach 

specialist 

AIDS Action Committee,

Boston, MA 

Clinics & case

managementassessment

(clients turn in lab

reports to CM, CM enter

into ETO. Or self report 

(esp. at baseline) 

Data extraction and 

interviewer administered 

Case manager 

AIDS Foundation Chicago 

(AFC), Chicago, IL 

Client medical record;

survey 

Data extraction, 

interviewer administered 

Regional care 

coordinators 

Amida Care: New York, 

NY 

Lab data & MET client 

Survey 

Extraction and 

interviewer administered 

Program manager, case

manager, peers 

AIDS Project Los Angeles 

(APLA): Los Angeles, CA 

Lab slips or fax medical 

release to consent to 

individual clinics 

Data extraction Peer navigator 

Christie s Place (CP): San 

Diego, CA 

Lab slips/

management reports 

Data extraction Peer navigator 

Damien Center: 

Indianapolis, IN 

CAREWare or

CaseManager database 

Data extraction Dir quality assurance 

Louisiana Public Health 

Institute (LPHI): Baton 

Surveillance data Data extraction from 

state surveillance data 

OPH Data Management /

Analysis manager 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 29 
’

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Lead organization and

location 

Source for outcome data Outcome data collection

method 

Individual responsible

for outcome data 

collection 

Rouge, LA 

Medical AIDS Outreach 

(MAO), Montgomery, AL 

St. Louis Effort for AIDS: 

St. Louis, MI 

CareWare 

SCOUT and state 

surveillance database 

Data extraction 

Data extraction 

Registered nurse/social

worker 

Lead navigator and state 

Washington AIDS

Partnership (WAP):

Washington, DC 

Client records and state 

surveillance database 

Extracted and

interviewer administered 

CHW and state 

University of Alabama at 

Birmingham (UAB):

Birmingham, AL 

Medical records Data extraction Study coordinator 

Table 12 below outlines study retention (monitoring of patient-level data) by subgrantee. The national 

evaluation monitors attrition using an ‘enrollment schematic’ form that asks subgrantees to indicate the 

total number of participants enrolled and the total number of participants with follow-up data. Of the 

total 4,787, enrolled 4,346 were eligible for twelve-month follow-up data collection. Of those eligible, 

data have been collected on 2,689 (62%). Among those without twelve-month data collected (2,098),

21% (441) have not been enrolled in the program long enough to reach twelve months, and the 

remaining 79% (1,657) are lost to the evaluation. 

Table 12: Retention at twelve months by subgrantee

Lead organization and location # 

enrolled 

# 12-month 

follow-up

completed13 

# not eligible

for 12-month

follow-up 

# without 12-month 

follow-up data (% 

of total enrolled)

ActionAIDS, Philadelphia 78 49 5 24 (31) 

AIDS Action Committee, Boston, MA 342 211 46 85 (25) 

AIDS Foundation Chicago (AFC), 

Chicago, IL 639 334 96 209 (33) 

Amida Care: New York, NY 833 690 36 107 (13) 

AIDS Project Los Angeles (APLA): Los 

Angeles, CA (APLA) 245 197 0 48 (20) 

AIDS Project Los Angeles (APLA): Los 

Angeles, CA (DHSP) 82 48 7 27 (33) 

Christie s Place (CP): San Diego, CA 226 161 2 63 (28) 

13 Subgrantee data came from multiple sources. The n for the follow-up data at 12 months indicates that a 

participant had a follow-up data point from any source. For example, a participant with lab data at 12 months but 

no survey data at 12 months was counted in the table above as having follow-up data at 12 months. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 30 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Lead organization and location # 

enrolled 

# 12-month 

follow-up

completed13 

# not eligible

for 12-month

follow-up 

# without 12-month 

follow-up data (% 

of total enrolled)

Damien Center: Indianapolis, IN 149 93 10 46 (34) 

Louisiana Public Health Institute 

(LPHI): Baton Rouge, LA 144 75 7 62 (43) 

Medical AIDS Outreach (MAO), 

Montgomery, AL 244 189 0 55 (23) 

St. Louis Effort for AIDS: St. Louis, MI 322 213 41 68 (21) 

Washington AIDS Partnership (WAP): 

Washington, DC14 1313 310 191 812 (62) 

University of Alabama at Birmingham 

(UAB): Birmingham, AL (intervention 

arm) 87 61 0 26 (30) 

University of Alabama at Birmingham 

(UAB): Birmingham, AL 83 58 0 25 (30) 

Total 4,787 2,689 441 1,657 (35) 

Currently, JHU lacks complete follow-up data on approximately 44% of the sample; as a result, these 

individuals were excluded from analyses that require follow-up data, such as assessment of trends in

health outcomes at baseline and twelve months. This exclusion could introduce bias, as the participants 

who were retained could be systematically different from those who are not. This bias is a threat to 

internal validity. Further analyses of missing data are addressed below in the analysis section. 

Subgrantees faced a variety of challenges in collecting complete data, especially complete follow-up 

data and complete clinic/lab data. Several grantees have described challenges in getting follow-up data 

from clients primarily due to the transient nature of the participant population. In addition, subgrantees 

reported challenges or significant delays in getting clinical data from partners, including medical 

provider’s offices and health departments. Increased communication, strengthening of partnerships, 

and formalizing relationships through MOUs helped to address some of these challenges. 

We conducted power analyses for detecting a difference in mean CD4 from baseline to twelve months. 

At alpha .05, a sample size of 22 per group would give us a power of .80. All subgrantees have follow-up 

data on more than 22 individuals. 

14 Washington AIDS Partnership faced considerable challenges in the first years of the program with follow-up data 

collection. As a result, twelve-month data will not be available for a substantial proportion of their participants 

(62%). A new evaluator for Washington, DC was hired in 2013 and the majority of the initial challenges faced by 

this subgrantee have been addressed 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 31 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Economic analysis: The national evaluation included a cost analysis and a threshold analysis. The cost 

analysis estimated the cost of delivering the program locally. The cost analysis was conducted from the 

payer perspective as well as from the societal perspective. The cost analysis from the societal

perspective included the payer costs plus the costs to the client (such as costs for travel, time and 

dependent care), while the cost from the payer perspective did not include costs to the client and only

include cost to the implementing agency. The threshold analyses calculated thresholds needed to make 

claims that programs are cost-saving or cost-effective. 

JHU provided training to each subgrantee via webinar on how to conduct cost and threshold analyses. 

The trainings provided detailed information on how to conduct cost and threshold analyses and also 

provided step-by-step instructions on how to fill out a cost analysis spreadsheet tool developed by JHU. 

The MS excel spreadsheet includes five steps and embedded formulas. The first step specifies the 

timeframe for the analysis. The second step describes the program and how participants interact with 

the program. The third step gathers information on the number of participants served, the number of

participant contacts, time participants spent in the program, and costs to the participant, such as travel

and dependent care. Step 4 gathers information on cost to the subgrantees to implement the programs 

and captures information on costs associated with staff and personnel, as well as materials and 

consumables. Staff and personnel costs include salary for all individuals who spend time working on the 

project. Materials and consumables include costs such as rent, utilities, promotional materials, and 

office supplies. Step 5 captures the overhead rate to gather data on any items not addressed in Step 4. 

In addition to the training, JHU also provided subgrantees with a protocol of line-by-line instructions for 

how to complete the cost spreadsheet as well as answers to frequently asked questions. Subgrantees 

were also provided with one-to-one technical assistance over the phone or via email to address any 

additional questions that arise while completing the cost analysis spread sheets. 

To complete the spreadsheets, the subgrantees extracted data from accounting forms, budgets, and 

administrative records. To assess the thresholds, JHU used several parameters from the literature. “T” is

the medical costs saved for each transmission averted. “W” the estimated price that society is willing to 

pay for a QALY and “was estimated to be 5.83 based on prior work by Holtgrave (3). “T” was estimated 

at $330,000 USD based on work by Farnham (4) and “W” was estimated using the commonly utilized 

standard of $100,000 USD per QALY saved (5-7). The cost-saving threshold was estimated using the 

formula C/T while the cost effectiveness threshold was estimated using the formula C/(T+WQ). 

Monitoring sub-contractor networks: The implementation evaluation of SIF included a network analysis. 

This portion of the evaluation gathered data on collaboration between sub-grantee organizations six

months prior to the start of SIF and during SIF. These data were used to develop visual representations 

of sub-contractor networks and to calculate network density and average node degree. The network 

analysis used a retrospective pre/post design. An online survey was used to obtain retrospective and 

current network data on the relationships between organizations. Network analysis data were collected 

from approximately two individuals at each sub-contracting organization. We gathered data from one 

individual who worked at the administrative level (such as a supervisor) and one person who worked at 

the service level (such as a peer). JHU worked with subgrantees to identify the appropriate individuals to 

survey. JHU then emailed these individuals to ask them to complete the on-line survey. A link to the online survey was embedded in the email. The survey was developed using Qualtrics. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 32 
 

 

 

 

 

 

 

 

 

 

 
	

Number of individual Organizations
	

SIF subgrantee surveys surveyed Survey dates 
	

ActionAIDS 5		 ActionAIDS April – May 2015 

COMHAR

Gaudenzia

Pathways to Housing 

AIDS Action Committee 12		 AAC February – April 2014 

Fenway

PACT/JRI

BIDMC 

ICH 

AIDS Foundation Chicago 10		 AFC March – November 2013 

MATEC

Chicago House 

Mercy Care Program 

Amida Care 8		 Amida Care June – November 2013 

Housing Works 

Help/PSI

Harlem United 

AIDS Project Los Angel es (DHSP 11		 APLA December 2013 – April 

arm)		 DHSP 2014 

MCA Clinic

THE Clinic

Rand Schrader 

Northeast Valley 

Christie’s Place 6		 Christie’s Place June – November 2013 

AVRC

MCAP

NCHS 

The Damien Center 9		 The Damien Center July – September 2015 

Brothers United 

Women in Motion 

Indiana Latino Institute 

Community North Hospital 

Bellflower 

Life Care 

Effort for AI DS 6		 Washington University March – May 2014 

Infectious Disease Clinic 

Washington University 

BEACON Team 

St. Louis Effort for AI DS 

Louisiana Public Health Institute 5		 Louisiana Public Health J une – September 2015 

Institute 

Capitol Area Reentry Program 

The network analysis calculates density and average node degree. These data were collected by asking 

“Does your organization collaborate with [insert name of organization] in the implementation of SIF” 

and “In the six months before SIF, did your organization work with any of the following organizations to 

link PLWH into HIV care and treatment?” Three sites (APLA, MAO, and UAB) did not participate in this

portion of the evaluation because their networks included only two organizations; as a result, the utility 

of the findings are limited. In addition, data on pre-implementation measures of agency 

interconnectivity are not available from WAP due to a survey collection error. Table 13 below provides 

detail on the number of individuals surveyed, the organizations surveyed, and the survey dates for the 

network analysis. 

Table 13: Number of individuals surveyed, organizations surveyed, and survey dates by subgrantee

AIDS United Social Innovation Fund Final Evaluation Report  May 2016		 33 
 

SIF subgrantee 

Number of 

surveys 

individual Organizations 

surveyed Survey dates 

CresentCare

Women with a Vi sion 

Washington AIDS Partnership 10 Washington AIDS Partnership

Whitman-Walker Health 

Unity Health Care 

Chartered Health Plan 

Women’s Collective 

Institute for Public Health 

Innovation 

Family Medical Counseling 

Services 

February – November 

2013 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Analysis

Analytical methods are outlined below by research question: 

Q2: What is the trend in mean CD4 from baseline to 12 months among participants? We calculated the 

mean for baseline and twelve months. T-tests were used to compare changes in mean CD4 from 

baseline to twelve months. 

Q3: What is the trend in mean viral load from baseline to 12 months among participants? We calculated 

the mean for each follow-up time point. T-tests were used to compare changes in mean viral load from 

baseline to twelve months. 

Q4: What is the trend in % of clients reporting a suppressed viral load from baseline to 12 months among 

participants exposed to SIF? We calculated percentages at baseline and twelve months. 

Q6. Compared to the network six months prior to SIF, has the subcontractor network increased in

density? We calculated density as the sum of ties divided by all possible ties at two time points (six

months prior to the start of the program and at least one year after program implementation). 

Q7. Compared to the network six months prior to SIF, does the subcontractor network average node 

degree increase? We calculated the average number of ties across nodes (organizations) at two time 

points (six months prior to the start of the program and at least one year from program 

implementation). 

Q8. What is the cost saving threshold for Access to Care? (cost analysis) We calculated the total cost of

each intervention and then divide the total cost of the intervention by lifetime cost of care and 

treatment ($330,000)(4). 

Q9. What is the cost effectiveness threshold for Access to Care? The cost effectiveness threshold is

C/(T+(W*Q) where C is the cost of the intervention, T is medical cost averted each time a HIV infection is

prevented ($330,000), Q is the number of quality adjusted life years (QALYs) saved for each HIV

transmission averted, and W is the price society appears willing to pay to “buy” a QALY. For Q, we used 

5.83 (3) and for W, we used $100,000 as the societal cost per QALY saved (5, 7, 8). 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 34 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Q10. What is the cost per QALY saved by delivering services in each program area? We used 

mathematical modeling to translate estimated transmission rates and observed program outcomes into 

an estimate the cost per QALYs saved. We used standard methods from the U.S. Panel on CostEffectiveness and Health (9, 10). The main formula for the analysis as R=(C-AT)/AQ where R is the costutility ratio which can be interpreted as the net cost per QALY saved. C, the cost of the program, and T 

the medical cost averted by the prevention of a HIV infection are defined above in Q9. Q (or QALY)

included both discounted QALYs saved through improvements in individual quality of life due to viral 

suppression (Q1) as well as QALYs saved through averted HIV transmissions (Q2). Based on the 

literature, the value for Q1 we used was 0.039 and the value we used for Q2 was 5.83 (3, 4). A is the 

number of transmission averted. To estimate A, we first estimated the number of individuals whose viral

loads went from not suppressed to suppressed over the course of the project using the available data 

from study participants. We did this by limiting the data to individual with data at enrollment and twelve 

months. We then estimated the proportion who became virally suppressed and applied that proportion 

to all participants enrolled who were eligible for twelve-month follow-up. This estimate of the net 

number of individuals who became suppressed was then multiplied by 0.046, a constant based on the 

literature that estimates the transmission rate of an individual with an unsuppressed viral load who is 

aware of their HIV status (11). The return on investment was calculated using the formula ROI=TA/C. 

Findings 

Monitoring of patient-level data: Tables 14-16 below outline changes from baseline to twelve months in

mean CD4, mean viral load and percent with a suppressed viral load. Analyses were limited to 

individuals with data at both baseline and twelve months to allow for testing of statistical differences. A 

paired T-test was used to assess differences in mean CD4 and mean viral load. All subgrantees saw 

increases in mean CD4 from baseline to twelve months among SIF participants. These changes were 

significant for participants enrolled in Boston, Chicago, New York, Los Angeles, San Diego, Indianapolis, 

Montgomery, St. Louis, Washington, DC, and Birmingham. 

Table 14: Mean CD4 by site at baseline and twelve months (limited to those with data at both baseline 

and twelve months)

Lead organization and location N Baseline

mean CD4

(SD) 

Twelve

month mean

CD4 (SD) 

p-value 

ActionAIDS, Philadelphia 41 523 (338) 555 (292) >.05 

AIDS Action Committee, Boston, MA 124 463 (320) 547 (324) <.05 

AIDS Foundation Chicago (AFC), 

Chicago, IL 

18915 456 (23) 528 (24) <.05 

Amida Care: New York, NY 342 514 (331) 548 (341) <.05 

AIDS Project Los Angeles (APLA): Los 

Angeles, CA (APLA arm) 

183 527 (294) 559 (285) <.05 

AIDS Project Los Angeles (APLA): Los 19 503 (383) 772 (1,089) >.05 

15 190 clients had CD4 data at both enrollment and 12 months. However, one outlier was removed. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 35 
’

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Lead organization and location N Baseline

mean CD4

(SD) 

Twelve

month mean

CD4 (SD) 

p-value 

Angeles, CA (DHSP arm) 

Christie s Place (CP): San Diego, CA 104 515 (288) 574 (303) <.05 

Damien Center: Indianapolis, IN 46 379 (245) 508 (281) <.05 

Louisiana Public Health Institute 

(LPHI): Baton Rouge, LA 

39 437 (335) 469 (346) >.05 

Medical AIDS Outreach (MAO),

Montgomery, AL 

153 527 (314) 591 (306) <.05 

St. Louis Effort for AIDS: St. Louis, MI 167 295 (255) 427 (269) <.05 

Washington AIDS Partnership (WAP): 

Washington, DC 

192 395 (253) 521 (310) <.05 

University of Alabama at Birmingham 

(UAB): Birmingham, AL (intervention 

arm) 

53 580 (365) 635 (385) <.05 

University of Alabama at Birmingham 

(UAB): Birmingham, AL (control arm) 

49 584 (366) 638 (387) <.05 

As outlined in Table 14, subgrantees saw reductions in viral load among participants from baseline to 

twelve months. These reductions are significant in the following program locations: Chicago, Los 

Angeles, Indianapolis, Montgomery, St. Louis, and Washington, DC. 

Table 15: Mean viral load by site at baseline and twelve months (limited to those with data at both 

baseline and twelve months)

Lead organization and location N Baseline mean 

viral load 

Twelve month 

mean viral load 

p-value 

ActionAIDS, Philadelphia 31 12,157 (32,889) 6,293 (22,420) >.05 

AIDS Action Committee, Boston, 

MA 

108 59,222 (398,940) 6,062 (33,440) >.05 

AIDS Foundation Chicago (AFC), 

Chicago, IL 

16516 48,403 (11,781) 19,972 (6,501) <.05 

Amida Care: New York, NY 420 25,671 (185,370) 9,734 (41,561) >.05 

AIDS Project Los Angeles (APLA): 

Los Angeles, CA (APLA arm) 

183 123,665 (925,383) 21,534

(105,988) 

>.05 

16 166 clients had viral load data at both enrollment and 12 months. However, one outlier was removed. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 36 
’

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Lead organization and location N Baseline mean 

viral load 

Twelve month 

mean viral load 

p-value 

AIDS Project Los Angeles (APLA): 

Los Angeles, CA (DHSP arm) 

34 90,037 (203,572) 4,017 (12,722) <.05 

Christie s Place (CP): San Diego, CA 91 6,287 (36,972) 1,539 (5,712) >.05 

Damien Center: Indianapolis, IN 51 81,953 (187,119) 15,446 (51,800) <.05 

Louisiana Public Health Institute 

(LPHI): Baton Rouge, LA 

52 119,857 (477,918) 23,586 (65,243) >.05 

Medical AIDS Outreach (MAO),

Montgomery, AL 

185 11,120 (51,782) 564 (3,796) <.05 

St. Louis Effort for AIDS: St. Louis, 

MO 

169 304,440

(1,119,948) 

19,446 (74,191) <.05 

Washington AIDS Partnership

(WAP): Washington, DC 

189 36,213 (138,168) 5,095 (15,584) <.05 

University of Alabama at 

Birmingham (UAB): Birmingham, 

AL (intervention arm) 

57 19,728 (72,891) 6,167 (17,869) >.05 

University of Alabama at 

Birmingham (UAB): Birmingham, 

AL (control arm) 

51 20,312 (73,989) 6,395 (18,159) >.05 

Across all subgrantees, we see an increase in percent suppressed viral load among participants with data 

available at baseline and twelve months. Confidence limits do not overlap in Boston, Chicago, New York, 

Los Angeles, Indianapolis, Montgomery, St. Louis, and Washington, DC. 

Table 16: Viral suppression by site at baseline and twelve months (limited to those with data at both 

baseline and twelve months)

Lead organization and location 

ActionAIDS, Philadelphia 

AIDS Action Committee, Boston, 

MA 

N 

31 

108 

Baseline % viral 

suppression 

58% (18/31) 

61% (66/108) 

Twelve month % 

viral suppression 

77% (24/31) 

83% (90/108) 

Confidence

interval 

overlap17 

Yes 

No 

AIDS Foundation Chicago (AFC), 

Chicago, IL 

166 53% (88/166) 68% (113/166) No 

17 95% Confidence intervals were computed by modified Wald method. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 37 
’

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Lead organization and location N Baseline % viral 

suppression 

Twelve month % 

viral suppression 

Confidence

interval 

overlap17 

Amida Care: New York, NY18 441 66% (291/441) 76% (334/441) No 

AIDS Project Los Angeles (APLA): 

Los Angeles, CA (APLA arm) 

183 51% (94/183) 67% (122/183) No 

AIDS Project Los Angeles (APLA): 

Los Angeles, CA (DSPH arm) 

34 44% (15/34) 74% (25/34) Yes 

Christie s Place (CP): San Diego, CA 91 73% (66/91) 85% (77/91) Yes 

Damien Center: Indianapolis, IN 51 20% (10/51) 75% (38/51) No 

Louisiana Public Health Institute 

(LPHI): Baton Rouge, LA 

52 52% (27/52) 60%(31/52) Yes 

Medical AIDS Outreach (MAO),

Montgomery, AL 

185 80% (148/185) 94% (174/185) No 

St. Louis Effort for AIDS: St. Louis, 

MI 

169 11% (18/169) 76% (128/169) No 

Washington AIDS Partnership

(WAP): Washington, DC 

189 49% (92/189) 78% (148/189) No 

University of Alabama at 

Birmingham (UAB): Birmingham, 

AL (intervention arm) 

57 79% (45/57) 79% (45/57) Yes 

University of Alabama at 

Birmingham (UAB): Birmingham, 

AL (control arm) 

51 75% (38/51) 76% (39/51) Yes 

To better understand missingness on our primary outcome of interest, viral suppression, we compared 

the demographic characteristics of participants with viral load data available at enrollment and twelve 

months to participants without those data available. Across all subgrantees, differences emerged. These 

differences are significant with regards to age, race and gender. Biases in attrition are important to 

understand when interpreting the outcomes from the SIF program. 

18 For Amida Care, undetectable VL and Suppressed VL has a larger N because in the survey, clients who do not 

remember a precise number of VL were provided an option to answer if VL was undetectable or not. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 38 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table 17: characteristics of individuals with data on viral suppression at baseline and 12 months 

*Chi-Square. Missing values excluded from Chi-Square analyses. 

19 One site (Medical AIDS Outreach) does not collect data on education. 

Had viral load data at both baseline and 12 months? All Sites

N=4377 

Yes 

n (%) 

No n

(%) 

P-value* 

TOTAL 1519 (35) 2858 (65) 

Age ≤ 24 years 97 (6) 257 (9) <0.05 

25 – 39 years 501 (33) 959 (34) 

40 years and above 920 (61) 1622 (57) 

Missing 1 (0) 20 (1) 

Race Black, non-Hispanic 1044 (69) 2095 (73) <0.05 

White, non-Hispanic 164 (11) 290 (10) 

Hispanic 231 (15) 312 (11) 

Other 72 (5) 111 (4) 

Missing 8 (1) 50 (2) 

Gender Male 949 (62) 1656 (58) <0.05 

Female 534 (35) 1072 (38) 

Transgender 35 (2) 105 (4) 

Other 0 (0) 1 (0) 

Missing 1 (0) 24 (1) 

Education19 Less than High School 411 (31) 903 (32) >0.05 

High School diploma or equivalent 495 (37) 992 (35) 

Some college or technical school 294 (22) 602 (22) 

College or higher education 119 (9) 207 (7) 

Other 0 (0) 1 (0) 

Missing 15 (1) 94 (3) 

(<200 copies/ml)

at baseline No 278 (18) 317 (11) 

Viral suppression Yes 1163 (77) 1136 (40) >0.05 

Missing 78 (5) 1400 (49) 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 39 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Lead agency name 

ActionAIDS 

Analysis time

period 

(months)

12 

Payer 

Overall

($) 

Per client 

($) 

276,049.22 4,678.80 

Societal 

Overall

($) 

Per client 

($) 

291,689.90 4,943.90 

AIDS Action Committee 6 295,035.90 2,565.53 330,465.98 2,873.62 

AIDS Foundation Chicago 

AIDS Project Los Angeles 

Division of HIV and STD 

Programs 

Amida Care 

6 

6 

6 

6 

377,375.62 

128,389.03 

170,865.69 

333,308.67 

5,169.53 

2,292.66 

7,428.94 

2,687.97 

385,318.85 

139,962.83 

174,858.49 

363,068.67 

5,278.33 

2,499.34 

7,602.54 

2,927.97 

Christie’s Place 6 266,882.00 5,446.57 290,478.79 5,928.14 

The Damien Center 12 342,155.05 4,562.07 364,991.71 4,866.56 

St. Louis Effort for AIDS 12 382,551.29 3,787.64 391,467.06 3,875.91 

Louisiana Public Health Institute 6 105,692.42 2,935.90 107,749.82 2,993.05 

Medical AIDS Outreach 6 102,313.65 1,088.44 104,288.12 1,109.45 

University of Alabama /

Birmingham AIDS Outreach 9 173,311.47 2,586.74 185,698.43 2,771.62 

Economic analysis: Table 18 presents the findings from the cost threshold analyses. Program costs are 

presented from both the payer perspective and the societal perspective. The table below displays 

overall costs as well as the costs per client. The payer costs ranged from $102,313.65 over six months to 

$382,551.29 over twelve months. The per client payer costs varied widely, ranging from $1088.44 to 

$7,428.94. The societal costs ranged from $104,288.12 over six months to $391,467.06 over twelve 

months. The per client societal costs ranged from $1,109.45 to $7,602.54. Table 18 displays the costsaving thresholds and the cost-effectiveness thresholds from the societal perspective. The cost-saving 

thresholds ranged from 0.32 to 1.19, and the cost-effectiveness thresholds ranged from 0.11 to 0.43. As 

discussed with CNCS program officer Parita Patel, the sensitivity analyses for the cost-utility analysis are 

on-going and final cost utility results will be completed in by July 1. 

Table 18: Payer and societal program costs – overall and per client 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 40 
Table 19: Cost-saving and cost-eff ective thresholds, by program 

Site name 

Cost-saving 

threshold 

C / T 

Cost effective 

threshold 

C / ( T + WQ) 

ActionAIDS 

AIDS Action Committee 

AIDS Foundation Chicago 

AIDS Project Los Angeles 

Division of HIV and STD Programs 

Amida Care 

Christie s Place ’

The Damien Center 

St. Louis Effort for A IDS 

Louisiana Public Health Institute 

Medical AIDS Outreach 

University of Alabama / Birmingham AIDS 

0.88 

1.00 

1.17 

0.39 

0.52 

1.10 

0.88 

1.04 

1.19 

0.33 

0.32 

0.56 

0.32 

0.36 

0.42 

0.14 

0.19 

0.40 

0.32 

0.37 

0.43 

0.12 

0.11 

0.20 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

C: the cost of the intervention; T: lifetime total cost of treatment ($330,000); Q: number of QALYs saved from each averted 

infection (5.83); W: the societally accepted cost per QALY ($100,000). Cost thresholds are calculated from the societal

perspective (e.g., using the societal costs) with the exception of AIDS Project Los Angeles and The Damien Center, which are 

from the payer perspective. 

Monitoring sub-contractor networks: Table 20 below highlights the findings from the network analysis. 

The network analysis assessed changes in interagency collaboration by looking at network density and 

average node degree six months prior to the implementation of SIF and then during the implementation 

of SIF. Network density is the proportion of all reported connections out of all possible connections 

between organizations. Across all sites but two we saw increases in network density. For Christie’s Place, 

network density decreased from 1.00 to 0.75. This decrease can be explained by a change in partnership

that occurred midway through the program. Similarly, we saw increases in node degree, or the average 

number of connections per organization in the network, across all organizations except for Christie’s

Place and APLA’s DHSP arm. This decrease can also be explained by changes in composition of the 

network during the implementation of the project. Visual depictions of the pre- and periimplementation SIF networks are displayed in the accompanying report, “Social Innovation Fund Case 

Studies: Experiences with Linkage and Retention in Care across Twelve Grantees and their Partners.” 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 41 
Table 20: 

’

Network density and average node degree, by program 

Network Density Average Node Degree 

Site name Pre Peri Pre Peri 

ActionAIDS 0.50 0.70 2.0 2.8 

AIDS Action Committee 0.45 0.60 1.8 2.4
	

AIDS Foundation Chicago 0.45 0.90 1.8 3.6
	

Division of HIV and STD Programs 0.70 0.60 3.3 2.8
	

Amida Care 0.50 0.83 1.5 2.5
	

Christie s Place 1.00 0.75 3.0 2.5
	

The Damien Center 0.26 0.64 1.6 3.9
	

St. Louis Effort for AIDS 0.50 1.00 1.0 2.0
	

Louisiana Public Health Institute 0.33 0.67 1.0 2.0
	

Washington AIDS Partnership NA 0.86 NA 6.0
	

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 42 
 

Lessons learned 

Across all subgrantees, mean CD4, mean viral load, and percent viral suppression trended in the 

hypothesized direction. For several sites (see tables 14-16 above) these trends are statistically

significant. All subgrantees participated in t he national evaluation, though some subgrantees faced 

significant challenges around data collection and reporting. Several grantees described challenges in

getting clinical data from medical providers’ offices, as well as connecting with clients for follow-up data 

collection. Sites developed systems (e.g., contracts with providers) to gain access to these data, sought 

the help of program staff accompanying clients to appointments, and asked clients to sign release forms 

that eased access to these data. 

The cost analysis found that program costs varied considerably, which is expected given the differences 

in program location, model, and population served. Cost per client from the societal perspective ranged 

from $1,109.45 per client to $5,928.14. Cost-saving thresholds ranged from .32 to 1.19, indicating that 

to be cost-saving the programs would need to avert from just over one to less than 1 HIV infection 

(during the timeframe of the analysis.) To be cost-effective, the programs would need to avert fewer 

than one HIV infection. These cost-saving and cost-effective thresholds appear quite achievable and 

compare favorably to cost analyses of other retention in care programs (12, 13). 

The network analysis found increases in density and average node degree across the SIF network. These 

findings suggest increases in interagency collaboration. It appears that targeted linkage and retention in 

care programs may increase collaboration among diverse organizations (public health departments, 

AIDS service organizations, social service organizations, and HIV primary care providers) serving people

living with HIV. Programs such as A2C that are working with highly underserved populations must 

address participant needs and barriers to HIV care (such as homelessness, unemployment, food 

insecurity, mental health challenges, substance use, and dental services) as part of providing HIV linkage 

and retention services. Often meeting such a range of client needs is b eyond the scope of a single 

organization. Increased collaboration and interaction between agencies has the potential to provide 

enhanced and streamlined services to PLWH with a hi gh acuity of need without building a new 

infrastructure. Thus, strong network collaboration among a variety of diverse organizations, such as the 

networks demonstrated by A2C, might help to provide timely and efficient “wrap around services” to 

PLWH in locations where “one-stop-shop” services are not yet available or are over taxed. 

Impact Evaluation 

The impact evaluation includes two components: (1) an RCT impact study; and (2) comparisons of local

treatment continuums of care. 

Level of evidence, impact evaluation 

The evaluation of SIF meets a “m oderate” level of evidence. The RCT impact study used a r andomized 

control design. This type of design should account for most threats to internal validity. The study could

not draw a random sample from a sampling frame as a traditional sampling frame was not available. 

Given this limitation, the study cannot make broad claims about generalizability. Thus, the external 

validity of the RCT impact study is moderate to low. However, to the best of our knowledge, there is

nothing unique about the setting and location for the RCT impact study (the 1917 Clinic in Birmingham, 

Alabama) that would cause the study findings to not be applicable in similar settings and locations, such 

as Southern cities in the United States. In addition, the RCT impact study tested the effect of an 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 43 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

evidence-based intervention and thus the study findings are contributing to a growing body of evidence 

to support the use of intensive case management and motivational interviewing. 

The all-subgrantee and grantee-specific continuums of care used data gathered from purposive samples. 

This limits generalizability and external validity. However, comparative continuums of care from all

grantees provides a valuable contribution to the understanding what linkage to care looks like for PLWH 

who are out of care across various settings (for example, urban vs. rural) and with diverse populations 

(for example, MSM and people under 40). 

Impact evaluation design 

BA2C RCT impact study: The impact assessment took place at the University of Alabama at Birmingham’s

(UAB’s) 1917 Clinic. The study used a randomized control group design. This study design was chosen 

because it controls for most threats to internal validity. Study participants were 1917 Clinic patients who 

met one of three definitions of out of care (failing to have a visit with a health care provider for twelve 

months, having multiple no-show primary care visits in the last twelve months, or having gaps in

continuous care of greater than six months in the past twelve months). To recruit RCT participants, a list 

of participants who are out of care that included contact information was routinely generated 

(quarterly) by the 1917 Clinic. Individuals on this list were recruited, consented, and then randomized to 

an intervention or standard-of-care arm. Thus, the unit of randomization was the individual. Study staff

used an iPad application designed by UAB Information Technology (IT) staff in collaboration with the 

subgrantee’s biostatistician to randomly assign participants to intervention or standard-of-care 

condition. Research staff hired exclusively to recruit and enroll participants into this study facilitated the 

randomization process. All research staff (research assistants and research technicians) received 

extensive training in study protocols, including the randomization process. At conclusion of the baseline 

assessment battery, research staff had the option to “submit” the participant’s responses to assessment 

instruments and proceed to randomization. The simple click of a “Randomization” button that appeared 

in the iPad application informed the research staff of which study arm the participant was assigned. 

Random assignment to study groups was predetermined using SAS programming and included in the 

study iPad application. The SAS program used a block randomization approach in which the first 250 

participants are randomly assigned to one of the two study arms. An additional command included the 

next 100 participants, assuring more evenly distributed assignment to both study groups. Only the study 

biostatistician and the IT development staff had access to this randomization schedule prior to 

participant randomization. Ideally, the randomization yields a balance for all covariates (measured and 

unmeasured). For measured covariates, we assessed statistical equivalence by testing if individuals in

the intervention and standard-of-care arm were similar at baseline on a range of demographic and 

health-related characteristics, including age, gender, race, retention, median CD4, and median viral load, 

as well as CD4 and viral load categorically. The intervention and control arms of the study were 

statistically equivalent for all covariates except for median number of days from first seropositive test. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 44 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table 21: Baseline socio-demographic and clinical characteristics of participants enrolled in Birmingham 

Access to Care (BA2C) 

I i Control

Characteristic 

ntervent on 

arm

n=87 

arm

n=83 

p-value 

Total 

n=170 

Age (years), mean(SD) 44.4 (10.9) 41.8 (10.5) 0.11a 43.1 (10.8) 

Race, n (%) 

White 

Black 

Other 

0.44b,c 

19 (21.8) 13 (15.7) 32 (18.8) 

64 (73.6) 68 (81.9) 132 (77.7) 

4 (4.6) 2 (2.4) 6 (3.5) 

Sex, n (%) 

Female 

Male

Transgender (M-F) 

0.75b,d 

30 (33.5) 27 (32.5) 57 (33.5) 

57 (65.5)

1 (1.2) 

55 (66.3)

0 (0.0) 

112 (65.9)

1 (0.6)

CD4 count (cells/uL),e 

median (min, max) 483 (257–778) 552 (393–865) 0.33f 531 (315–819) 

CD4 count (cells/uL), n (%) 

≤ 200 

> 200 

0.89g 

13 (16.9) 13 (16.1) 26 (16.5) 

64 (83.1) 68 (84.0) 132 (83.5) 

Viral Load (IU/mL),e 

median (min, max) 19 (19–851) 19 (19–4060) 0.93f 19 (19–1690) 

Viral Load (IU/mL), n (%) 

≤ 200

> 200 

0.87g 

57 (69.5)

25 (30.5) 

56 (68.3)

26 (31.7) 

113 (68.9)

51 (31.1)

Seropositivity (days),e 

median (min, max) 4464 (2052–7225) 2995 (1283–5233) <0.01f* 3585 (1609–6530) 

Retention, n (%) 

Yes

No 

54 (65.1)

29 (34.9) 

59 (67.8)

28 (32.2) 

0.70g 

113 (66.5)

57 (33.5) 

*Statistically significant at 0.05 level.

Max=maximum; Min=minimum; SD=standard deviation.
	

Missing data: CD4, n=12 (Intervention=10, Control=2); Viral load, n=6 (Intervention=5, Control=1);

Seropositivity, n=4 (Intervention=5, Control=1).
	 aCalculated by unpaired t-test.
	 bFisher’s exact test.
	 cAfter excluding the ‘Other’ category, the p-value was 0.27 (chi-square test for two proportions).

dAfter excluding the “Transgender” category, the p-value was 0.83 (chi-square test for two proportions).
	 eAs the variable was skewed, median (range) is reported.
	 f

Wilcoxon rank-sum test (non-parametric test).
	 gChi-square test for two proportions.
	

We did not use any matching prior to randomization. Enrollment into the RCT concluded in February 

2015 and the last participant provided follow-up data in December 2015 (the beginning of the 12-month 

follow-up window). 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 45 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

The standard-of-care arm received a passive referral. The passive referral included a brochure that 

informed the individual about the availability of HIV care and ancillary services at the 1917 Clinic and at 

the community partner site, Birmingham AIDS Outreach. The intervention arm received strengths-based 

case management and motivational interviewing from community social workers in 10–12 evidenceinformed sessions in a time-limited manner (within six months of enrollment). 

The design of the impact study assured protection against diffusion of program activities into the 

standard-of-care arm. Clear measures were taken to differentiate study controls from individuals 

assigned to the intervention. For example, study controls never have contact with the intervention staff

(community social workers). At baseline, study controls were provided standard of care passive referrals

for local HIV treatment and supportive services, including a flyer with contact information for both the 

HIV primary care clinic and AIDS service organization collaborating on this project. All study participants 

complete the same assessment battery at baseline, 6-month, and 12-month. Completed survey 

assessments of intervention arm participants have informed the individuals’ participation in the 

intervention; completed survey assessments for study controls are maintained for use study reporting 

and analysis only. Additionally, all participants receive lab tests at baseline and at 12 months to assess 

their CD4 cell count and their viral load. 

For control participants, these laboratory tests took place at an external provider in the participants’ 

community. For intervention participants, these tests occurred at the new clinic orientation visit as 

routine care for new clinic patients or in conjunction with another community HIV clinic that the 

intervention participant elects to attend. As the study intervention primarily consists of the assistance 

provided by the community social worker and ultimate re-engagement into HIV primary care, we 

assured that study controls were prevented from interaction with intervention staff, and we monitored 

control participants’ involvement with HIV primary care within the UAB system via system-wide medical

records. Community social workers were trained to complete intervention dosage data forms that 

document the type of interaction completed, date, amount of time, and resolution of the interaction, as 

well as the unique study identification number of the specific participant. From these data, we were able

to confirm any documented interaction between the intervention team and study controls. Differences 

between program participants and comparison group members in terms of access to program 

components and delivery of services were likewise be monitored through this documentation. All data 

for this research were collected using project-specific iPads and iPad applications develop specifically for 

this study. Paper documents were available and were used only in cases which use of the iPads was 

limited (e.g., areas with limited wireless access, times when the equipment is not functioning, etc.). 

Fidelity to the program design described above was assured and evaluated through a number of

activities. First, all aspects of the RCT were manualized; each project staff member—project director(s),

community social workers, research assistants, and community outreach workers—received intensive 

project orientation training that included role plays, problem-solving exercises, and introduction to 

study outcomes. Additionally, new staff members also received individualized project orientation. 

Second, the project director(s) and project evaluator conducted audits of staff activities, paperwork, and 

understanding to prevent drift. Third, regularly scheduled team meetings, led by project director(s),

were held. During these meetings, the team addressed implementation issues, such as linkage to care 

and retention in care among RCT participants, on an on-going basis. Finally, we convened monthly 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 46 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

“leadership” meetings that include study investigators, the project evaluator, project directors, and 

fiscal agent (among others) and during which implementation research questions were addressed, as 

well as issues relating to project sustainability and results dissemination. 

Comparison of local treatment continuums of care: To improve our understanding of the unmet HIV

care and treatment needs for individuals living with HIV, researchers have recently constructed 

continuums of care. The care and treatment continuum of care includes being HIV infected, diagnosed 

HIV positive, linked to HIV care, retained in HIV care, on ART, and having an undetectable viral load. The 

continuums of care estimate the number of PLWH in each category and allow researchers to estimate 

the percentage of PLWH who are linked to care and the percentage of PLWH who have suppressed viral

load. 

We used client-level data to create all-subgrantee and subgrantee-specific continuums of care to show 

the number of SIF participants who are enrolled, are linked to care, are retained in care, are on ART, and 

have a suppressed viral load. 

Measures

Randomized control trial: The primary outcome measure for this study is viral suppression, which is

defined as a viral load less than 200. Data on viral suppression are gathered at baseline and at 12 

months through a blood draw. Lab results are facilitated as part of study protocol or are collected as the 

participant’s routine care at their primary care provider appointment. For study labs, participants are 

transported to the UAB Clinical Research Unit (not affiliated with the 1917 Clinic) for blood draw; results 

are forwarded to the study coordinator. For labs collected as routine care (e.g., during hospitalization or 

a primary care provider visit), study staff abstract lab results from the participant’s electronic medical

record. All labwork is processed through the UAB Hospital Laboratories. UAB laboratories uses the 

Roche COBAS AmplePrep/COBAS TaqMan HIV-1 RNA v.2.0 to ascertain viral load. 

Comparison of local treatment continuums of care: the SIF evaluation developed an all subgrantee 

continuum of care (as well as subgrantee specific continuums of care). We aim to juxtapose the all

subgrantee continuum of care to the CDC’s national continuum of care. However, because SIF and the 

CDC are using different measures for the development of the continuum of care, direct comparisons 

may not be appropriate. Despite this, we feel that discussing the SIF continuum of care within the 

context of the CDC continuum of care is important for understanding our findings. Table 22 below 

outlines the measures used by JHU to form their continuum of care and the measures used by the CDC 

to create their continuum of care. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 47 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table 22: Continuum of care measures
	

CDC variable CDC calculation SIF variable SIF definition 

HIV diagnosed # PLWH from CDC HIV

Surveillance 

Enrolled in SIF # PLWH enrolled in SIF 

Linked to care # PLWH linked to care 

calculated as 77% of HIV

diagnosed based on a

meta-analysis by Marks 

(2010) that averaged % 

linked to care at 4,6 & 12

months & Torian (2011) 

Linked to care # PLWH with one visit 

with a medical care 

provider with prescribing 

privileges within a HIV

care setting 

Retained in Care # of PLWH retained in 

care calculated as 51% of

HIV diagnosed based on

Marks (2010) (averaged 

2 visits 6 mo.; 2 visits 12 

mo.; 3 visits 12 mo.), Hall

(2011), Tripathi (2011) 

Retained in care # PLWH who reported

two visits at least two

months apart in the past 

12 months (at 12 months 

follow-up) 

On ART # on ART calculated as 

88% of retained in care

based on the medical 

monitoring project 

On ART # of PLWH with a ART 

prescription (at 6 or 12 

months follow-up) 

Suppressed viral load # with a suppressed viral

load calculated as

number on ART times

88.8% based on the

medical monitoring 

project. Suppressed viral

load defined as <=200 

copies/mL 

# with suppressed viral 

load 

Defined as <=200 

copies/mL (at 6 or 12 

months follow-up) 

Data collection activities 

Potential study participants were identified using in-reach strategies that generate a list of PLWH who 

are out of care at the 1917 Clinic. All study participants were assigned a unique study identification 

number upon enrollment. All contacts (both for the intervention by the community social workers and 

for the research and follow-up) were documented using this unique number. Hence, at any given time, 

we could report a wealth of information regarding the projects interface with study participants (e.g., 

date and nature of contact). A local standard operating procedure was established to delineate the 

extent of efforts to initially contact potential participants for recruitment. This system included formal 

scripts which were used on initial and follow-up telephone calls with the potential participant, an 

emergency contact, a case manager, and a medical provider. The system provided direction about how 

many telephone calls to make, when to make them, when to send a recruitment letter, and how to send 

the letter (certified mail). The formal recruitment plan allowed attempts at initial contact for up to one 

year should an individual continue to be lost to follow-up. An on-site evaluation team worked closely

with the project staff—as well as the lead evaluation team at JHU—throughout the course of the study;

regular meetings were scheduled to discuss recruitment, retention, tracking, and other potentially 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 48 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

problematic issues. Standard reporting periods were established for providing up-to-date information to 

the lead evaluation team. Community outreach workers (COWs) were integral to project retention. Two 

COWs were hired for this project specifically to provide on-going communication with all study 

participants. COWs followed a standardized system for contacting study participants by telephone and 

letter after they have been enrolled and completed baseline activities. Additionally, the COWs were 

charged with providing transportation—a common barrier to retention in research studies at this site— 

to and from research study visits such as follow-up interviews (not intervention activities). Efforts to 

retain all participants in the study for its entirety were maintained until the study period ends. All study 

data were entered into the iPad application that was developed for use solely in this project. This single

system of data collection provided all collected data to the local project evaluator in real time; this 

allowed for random reviews of data collection activities. 

At initial contact and prior to baseline activities, we attempted to collect general demographic

information on all potential study participants (using a standardized form). Individuals were asked if

they were currently receiving HIV primary care and the date of their last appointment. For potential

participants whose information could not be confirmed in the system-wide UAB medical record, we 

asked individuals to provide us with the name of their most recent HIV primary care provider. After the 

participant provided informed consent to participate in the study, all data were collected and entered 

into an iPad application developed by the IT team at UAB specifically for this research. Additionally, 

study participants’ CD4 and viral load laboratory tests were either abstracted from the medical record 

(for intervention arm participants), received from external providers, or captured through study-specific

labs collected at baseline and 12-months. Medical record abstraction was used to obtain all participants’ 

HIV primary care appointment information during the study period. Locator information was collected 

on all study participants at baseline and regularly updated at study visits or other study contacts (e.g., 

telephone) in order to assist research staff in maintaining contact with participants and optimizing 

retention in the study. The information collected prior to enrollment and all baseline data was 

consistent between individuals in both study arms. All information collected prior to study enrollment 

and during baseline activities (and follow-up interviews) was collected by research staff (research 

assistants, research technicians, and/or research interns) during initial telephone contacts and semistructured face-to-face interviews. This same research staff was responsible for any necessary medical

record abstractions, as well. We concluded enrollment at this site in February 2015, and 170 individuals

were entered into the study and, therefore, have participated in all baseline activities. Enrollment 

activities were completed in December 2015. 

Immediately following completion of all baseline measures, the research staff facilitated the 

randomization process. Intervention participants were introduced to the community social worker and 

completed an initial session and were introduced to the community health outreach worker, who was 

responsible for maintaining contact with the participant throughout the study period. 

Intervention/Dosage data were collected beginning with this initial session with the community social

worker. Control participants immediately received passive referral to the local HIV clinic and ancillary 

services will be introduced to the COW. 

Participant demographic information was collected using a standard Patient Screening Form that had 

been used successfully by the site in other clinical trials. Specifically, this form included information 

regarding patient age, race, gender, ethnicity, education, annual income, and insurance status. All other 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 49 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

study measures were standardized and had been tested for reliability and validity for adult clinic 

populations. The local evaluation team was very familiar with each instrument and had extensively

conducted and published research with similar data. All study staff who accessed data via electronic

medical records at UAB participated in mandated training for use of the system. The local UAB staff 

completed all required trainings to access the electronic medical records systems. All study staff (at both 

the UAB and Birmingham AIDS Outreach sites) were adequately trained in use of the study-specific iPad 

application. 

Sample retention and missing data 

Randomized control trial: As a primary output for this study was retention in care, attrition was critically

evaluated. Herein lays the difficulty with missing data—poor engagement in care (or missing data) was, 

at least, an element of the study. Hence, missing data were not a nuisance; it was a major focus of the 

study. Additionally, linkage to and retention in care have been extensively studied at the UAB 

subgrantee site. For these studies, the subgrantee used three different strategies to account for missing 

data. For this analysis, one of these strategies was actually used—We considered missing data as missing 

(missing=missing). We plan to investigate a second strategy to consider missing data as failure 

(missing=failure). Results from this second strategy are complete for retention in care at 12 as discussed 

above in the implementation evaluation findings section of this report. 

Comparison of local treatment continuums of care: As discussed above in the outcome evaluation 

section, subgrantees faced challenges in collecting follow-up data, which resulted in missing data at 

follow-up. This introduces bias as the individuals with follow-up data are most likely systematically

different from those without follow-up data. When constructing the continuums of care, to be included 

in the final three bars (retention, HAART, viral suppression) a participant had to have follow-up data at 

either six or twelve months. Individuals without follow-up data were automatically coded as having a 

“no” response. For a study that focuses on retention in HIV care, missing visit, prescription, and lab data 

can be interpreted as an indicator of not being retained, on HAART or virally suppressed. However, this

is a conservative approach and, as a result, our estimates for the continuums of care (see impact 

assessment section) are most likely underestimates. Our approach is conservative because it is also 

possible that these individuals with missing data were not out of care but rather were getting care at 

another facility. Subgrantees tried to rule out this possibility, for example by calling around to differ local 

clinics and by checking to see if participants were incarcerated, but they faced considerable challenges 

in gathering this information (such as the intensity of time it took to gather this information and HIPAA 

constraints). 

Sample and supplemental power analysis

Randomized control trial: The proposed impact study does did not use a traditional sampling frame as 

national-, state-, or city-level lists of PLWH who are out of care are not available. This prohibits random 

sampling and the generalizability that random sampling affords. Therefore, the generalizability of study 

findings beyond patients who are out of care who attend the 1917 Clinic is unknown. However, there is

nothing unique to the study setting, location, or population that would cause the study findings to not 

be more broadly applicable. 

The target population for the study was PLWH in the Birmingham, Alabama area who were not engaged 

in HIV care and treatment services. It is unknown if the individuals who are out of care who attended 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 50 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

the 1917 Clinic are representative of PLWH who are out of care in Alabama or more broadly

representative of PLWH who are out of care in Southern cities or in the U.S. 

The randomized control trial enrolled 170 participants (87 in the intervention arm and 83 in the control 

arm). Assuming a reference proportion of 0.67, power of 0.80, and alpha of 0.05, the study has the 

power to assess a relative risk of 1.28. 

Statistical analysis of impacts 

BA2C randomized control trial: Q1: Are participants who receive the intervention more likely to have a 

suppressed viral load than individuals who receive standard of care? Initial exploration of the data 

included descriptive statistics of the variables (BL, T1, T2) reporting frequencies and percentages for 

categorical variables and mean (standard deviation) or median (quartiles/range) for continuous 

variables; the distribution of the continuous variables was assessed using histogram and QQ plots. The 

research arms (intervention versus control) were compared with regard to the socio-demographic and 

clinical variables examining baseline (at enrollment) differences informing whether “randomization” has 

worked or not. Furthermore, the research arms were compared with regard to the following outcomes 

of interest: viral suppression at T2 (12 months), change in viral load (BL to T2), linked to care within 30 

days or without 30-day parameter, and retained in care at T2 (12 months). Change in viral load from BL 

to T2 was examined by both as a continuous variable (copies/mL) and as a categorical variable (viral 

suppression at BL versusT2 within each arm). The differences were assessed comparing frequencies and 

percentages while association was examined using unconditional univariate odds ratio (OR) with 95% 

confidence interval (CI); for comparing viral load suppression at BL versus T2 within each arm, 

conditional ORs were reported owing to paired data. To maintain the original group allocation and 

minimizing confounding, “intention to treat” analysis was performed. This analysis is based only on 

“complete” data; sensitivity analysis and missing data imputation will be considered in future. 

Comparison of local treatment continuums of care: Q5. Compared to national and/or community-level

statistics, does the program have a higher percentage of participants linked to care, retained in care, on 

HAART, and with a suppressed viral load? We used client-level data to create across subgrantee and 

subgrantee-specific continuums of care of the number of SIF participants who: are enrolled, are linked 

to care, are retained in care, are on ART, and have a suppressed viral load. The all-subgrantee 

continuum of care are compared to national continuums of care developed by the CDC. The subgranteespecific continuums of care are compared to continuums of care for each location where available. 

Findings

Randomized control trial: We observed that intervention arm participants were more likely virally

suppressed at T2 (12 months), although these results were not statistically significant. As discussed 

above, the results may be explained by the nature of the intervention which provided intensive support 

for individuals for the first six months of study enrollment, thus potentially providing impact on linkage 

to care. Once the intensive phase of the intervention was complete, participants may have had more 

barriers to sustaining retention in medical care and ultimately viral suppression. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 51 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Table 23: Viral suppression among UAB Birmingham Access to Care (BA2C) participants
	

Patient

characteristics, 

n (%) or median (IQR) 

Intervention 

n/N (%) 

Control 

n/N (%) 

OR (95% CI) p-value 

Viral suppression 

(≤200) at 12 months 

(T2) 

46/58 79.3% 42/54 77.8% 1.1 (0.4-2.7) 0.8 

Table 23a: Change in Viral Load from Baseline (BL) to 12-Months (T2) by Study Arm for UAB Birmingham 

Access to Care (BA2C) participants 

Intervention Arm 

T2 Baseline 

Frequency

Percent

Row Pct

Col Pct 

Not Suppressed 

Not

Suppressed 

8

14.04

66.67

66.67 

Suppressed 

4

7.02

33.33

8.89 

Total 

12

21.05 

Suppressed 4

7.02

8.89

33.33 

41

71.93

91.11

91.11 

45

78.95 

Total 12

21.05 

45

78.95 

57

100.00 

Frequency Missing = 30 

OR=1.00 (95% CI: 0.19 – 5.37); p=1.00 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 52 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Control Arm 

T2 Baseline 

Frequency

Percent

Row Pct

Col Pct 

Not

Suppressed Suppressed Total 

Not Suppressed 9

17.65

75.00

69.23 

3

5.88

25.00

7.89 

12

23.53 

Suppressed 4

7.84

10.26

30.77 

35

68.63

89.74

92.11 

39

76.47 

Total 13

25.49 

38

74.51 

51

100.00 

Frequency Missing = 32 

Table 23b: Change in Viral Load from Baseline (BL) to 12-Months (T2) by Study Arm for UAB Birmingham 

Access to Care (BA2C) participants 

OR=1.33 (95% CI: 0.23 – 9.10); p=0.71 

Comparison of local treatment continuums of care: Table 24 presents the continuum of care for each 

site and for the A2C program overall. Overall, among A2C program participants, 86% were linked to care, 

69% were retained in care, 49% had a HAART prescription, and 62% were virally suppressed. (See 

Appendix 1 for links to the local cascades used in the table below). While direct comparisons with non 

SIF local continuums of care and the national continuum of care are not appropriate due to differences 

in methods, these non-SIF continuums of care do provide important context. The combined subgrantee 

SIF continuum of care exceeds the national continuum of care for each bar measured. The National HIV

AIDS Strategy sets out ambitious targets of 85% linked to care within 30 days, 90% retained in care, and 

80% virally suppressed. In order to achieve those goals, programs, such as those implemented by the SIF

subgrantees that engage the most underserved populations, will be are needed. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 53 
Table 24: HIV continuum of care among SIF participants by grantee 
	

Location, lead agency and number of 

participants enrolled (n) 

’

Linked to 

care

% (n) 

Retained in 

care

% (n) 

HAART 

prescription % 

(n) 

Virally suppressed 

(<200 copies/ml) %

(n) 

Philadelphia, 

PA 

A2C- ActionAIDS 

(78 clients served) 

97% (76/78) 67% (52/78) 91% (71/78) 68% (53/78)* 

Local 80% 42% Data not 

available 

46% 

Boston, MA A2C- AIDS Action 

Committee (AAC)

(342 clients served) 

82%

(279/342) 

77%

(265/342) 

64%

(220/342)* 

69% (237/342) 

Local 75% 59% Data not 

available 

65% 

Chicago, IL A2C- AIDS Foundation 

Chicago (AFC) 

(639 clients served) 

69%

(440/639)* 

48%

(305/639)* 

52%

(334/639)* 

52% (333/639)* 

Local 80% 54% 48% 40% 

New York City,

NY 

A2C- Amida Care 

(833 clients served) 

87%

(722/833) 

74%

(619/833) 

66% (551/833) 64% (536/833)* 

Local 73% 55% 51% 41% 

Los Angeles, 

CA 

A2C- AIDS Project Los 

Angeles (APLA): APLA 

arm

(245 clients served) 

100%

(245/245) 

76%

(186/245) 

71%

(175/245)* 

58% (141/245) 

Local 61% 51% Data not 

available 

50% 

Los Angeles, 

CA 

A2C- AIDS Project Los 

Angeles (APLA): DHSP 

arm

(82 clients served) 

95% (78/82) 84% (69/82) Data not 

available 

44% (36/82) 

Local 61% 51% Data not 

available 

50% 

San Diego, CA 
	 A2C- Christie s Place 
	

(226 clients served)
	

93%

(210/226) 

65%

(147/226)* 

66%

(150/226)* 

61% (137/226)* 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 54 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Location, lead agency and number of 

participants enrolled (n) 

Linked to 

care

% (n) 

Retained in 

care

% (n) 

HAART 

prescription % 

(n) 

Virally suppressed

(<200 copies/ml) %

(n) 

Local 54% 32% Data not 

available 

31% 

Indianapolis, 

IN 

A2C- Damien Center 

(149 clients served) 

79%

(118/149)* 

60%

(89/149)* 

Data not 

available 

49% (73/149)* 

Local 76% 48% 48% 81% 

Baton Rouge,

LA20 

A2C- Louisiana Public

Health Institute (LPHI)

(144 clients served) 

83%

(120/144) 

69%

(100/144)* 

37% (53/144)* 53% (77/144)* 

Local 69% 54% Data not 

available 

49% 

Montgomery,

AL 

A2C- Medical AIDS 

Outreach (MAO)

(244 clients served) 

100%

(244/244) 

88%

(215/244) 

90% (220/244) 96% (235/244) 

Local 82% 68% Data not 

available 

54% 

St. Louis, MI A2C- St. Louis Effort for 

AIDS (EFA)

(322 clients served) 

89%

(286/322) 

91%

(293/322) 

48%

(156/322)* 

65% (210/322)* 

Local 83% 53% Data not 

available 

54% 

Washington,

DC 

A2C- Washington AIDS

Partnership (WAP)

(1313 clients served) 

86%

(1124/1313) 

65%

(857/1313) 

Data not 

available 

63% (823/1313)* 

Local Percentage

not 

available21 

89% 59%22 63% 

 

 

 

 

 

 

20 Louisiana gets their visit and lab data from state partners. There is a considerable delay in getting this data, 

usually several months. This delay is anticipated and results in their clinic and lab data looking artificially low. 

21 The numerator was stated. 5,622 clients had one or more medical visits. No denominator was given. Total clients 

served (All HIV positives using any Ryan White care service from January 1- December 31, 2014) was not stated. 

22 One provider was unable to submit date and therefore was not included in this metric. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 55 
 

Location, lead agency and number of 

participants enrolled (n) 

Linked to 

care

% (n) 

Retained in 

care

% (n) 

HAART 

prescription % 

(n) 

Virally suppressed 

(<200 copies/ml) %

(n) 

Birmingham, 

AL

(intervention 

arm only) 

A2C- University of 

Alabama at 

Birmingham (UAB) 

(87 clients served) 

92% (80/87) 66% (57/87)* 64% (56/87)* 53% (46/87)* 

Local 74% 73% Data not 

available 

61% 

Birmingham, 

AL (control 

arm only) 

A2C- University of 

Alabama at 

Birmingham (UAB) 

(83 clients served) 

93% (77/83) 69% (57/83)* 73% (61/83)* 51% (42/83)* 

Local 74% 73% Data not 

available 

61% 

USA A2C- all sites 

(4704 clients served) 23 

86%

(4022/4704) 

69%

(3254/4704)* 

63%

(1986/3160)24* 

62% (2937/4704)* 

National 80% 40% 37% 30% 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

*Greater than 10% missing. Interpret with caution. 

Lessons Learned

BA2C RCT impact study: The UAB/BAO team learned many valuable lessons throughout the 

implementation of the RCT. The greatest challenges were study recruitment and retention. The 

complexity of attempting to locate BA2C target population was greater than expected. Waiting a full

year to adjudicate whether a patient was “out of care” meant that the study team was likely never to be 

able to contact that patient. UAB adjusted their eligibility criteria as a result. They also expanded 

recruiting hours to include some weekend and after-hours calling. Both of these efforts met with 

success. In addition, UAB promoted the study on radio, social media, Craigslist, and Grindr. These efforts 

produced many inquiries, but few eligible enrollees. They also implemented in-hospital recruitment, 

identifying eligible patients admitted to UAB hospital and approaching those patients in the hospital.

This effort was successful in securing the enrollment of a number of patients who they were previously

unable to contact. In addition, communication and locating participants turned out to be more difficult

expected. Great care, planning, and practice should be taken to ensure that timely and complete 

information can be shared quickly and easily between implementing organizations. UAB would

recommend having sufficient staff committed to the project from the outset, in order to locate and 

23 The control arm for University of Alabama not included.
	 24 This data does not include three sites- Damien Center, APLA_DHSP arm and Washington AIDS Partnership

(WAP).
	

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 56 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

transport participants. In addition, the challenges met highlight the importance of retention as a public

health practice. Our experience indicates that retaining participants is much easier that re-engaging out 

of care participants. 

Finally, prior to the grant award, we recommend beginning detailed discussions about the regulations 

inherent in the grant award and implementation process. To help prevent contract complications, 

include AIDS United leadership, local entity decision-makers, local department leadership, and a 

financial representative from each of these, in these in-depth discussions. 

Comparison of local treatment continuums of care: Most subgrantees’ A2C continuums of care 

compared favorably to the corresponding geographically local continuums of care and surpassed their

local continuums of care for all or most bars measured. In addition, the A2C all subgrantee continuum of

care surpassed the national HIV continuum of care for all bars measured. However, it is very important 

to note that direct comparisons between non-A2C cascades and A2C cascades may not be appropriate 

due to differences in methods used to construct the cascades. Therefore, the non-A2C cascades should

be used only to provide context and to better understand what gaps exist locally and how A2C programs 

might have served to address those gaps. In addition, one important lesson learned was that many sites 

struggled with getting data on whether or not participants had a current HAART prescription. This

challenge was not anticipated given that HAART data can be reliably gotten from pharmacy records, 

medical record, and self-report (although with some limitation). For future programs, we would

recommend either strengthening partnerships with pharmacies or relying on self-report data for HAART 

measures. 

Summary and Conclusions from Evaluation Findings 

This report presents findings from the national evaluation of the A2C program. Findings from the singlegroup assessment of participant health outcomes at baseline and 12 months suggest that A2C program 

participants successfully moved along the continuum of HIV care, as evidenced by significant increases 

over time in viral suppression. A2C subgrantee continuums of HIV care also compared favorably to 

location-specific HIV continuums of care. Additionally, the continuum of care for all A2C participants 

exceeded the national continuum of HIV care for all bars measured. However, it is important to note 

that while the RCT of one of the 12 A2C programs, BA2C, saw increases in viral suppression from baseline 

to 12 months, these increases were not statistically significant. 

At the network level, the evaluation of A2C assessed change in interagency collaboration as measured 

by network density and average node degree. Increases at most subgrantee locations in network density 

and average node degree suggest that interagency collaboration increased during the project. The 

findings also suggest that this collaboration was not simply due to the lead agency working with each 

implementing partner agency in a silo, but rather that collaboration increased between implementing 

partner agencies. Including at least one implementing partner agency was a mandate for the A2C 

program, and the evaluation suggests that, during A2C, collaboration between various types of 

organizations, including health departments, AIDS service organizations, HIV medical providers, and 

social service providers increased. All subgrantees acknowledged that the close partnerships with 

implementing partners was one of the key factors that helped their programs to succeed, especially

when working with highly vulnerable populations with a high acuity of need. Other linkage and 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 57 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

retention-in -care programs might want to consider adopting a similar model to A2C of lead agency and 

a network of implementing partners. 

The economic analysis of A2C assessed the cost of program delivery and estimated the cost-saving 

threshold, or the number of HIV infections each program would need to avert in order to be cost-saving. 

Program costs were estimated from the societal perspective, meaning that they took into consideration 

costs accrued by participants (such as costs related to transportation, lost wages, and dependent care).

Six-month program costs ranged from $104,288.12–$385,318.85. A range in costs was expected given 

differences in program models, locations, and target audience served. As such, cost results from one 

A2C program should not be directly compared to cost from another A2C program. The cost-saving 

threshold for the A2C programs ranged from 1–2. These cost-saving thresholds indicate that the 

programs would need to avert from 1–2 HIV infections to be cost-saving. The A2C program thresholds 

are highly achievable and compare favorably with other, similar programs. (14) Overall, these findings 

suggest that HIV linkage and retention in care programs are an efficient use of resources. 

The implementation evaluation sought to document barriers and facilitators to program implementation 

in order to inform the implementation of future programs. Across all 12 subgrantees, several themes 

emerged. Common barriers to program implementation included: the intensity of client needs;

recruiting and retaining participants; staffing and administrative hurdles; and challenges related to 

working with multiple partner organizations. The high level of need among individuals who are not 

engaged in HIV medical care has been documented previously, in particular the need for housing. (15) In

addition, the literature corroborates administrative challenges to implementing HIV linkage and 

retention-in-care programs, particularly those which employ peers. (16, 17) The three themes that 

emerged as factors that facilitated SIF program implementation were strong relationships with partner 

organizations; the flexible and innovative nature of the SIF programs; and the expertise, dedication, and 

passion of staff. These findings support prior research which has documented the facilitating factors at 

the organizational level, such as a prior history of collaboration between implementing agencies, strong 

leadership, formalized relationships such as MOUs, and clearly defined staff and implementing partner 

roles. (17) 

Human subjects protection 

BA2C RCT impact study: UAB was the lead agency for this study and served as the oversight investigator 

for all activities that involve human subjects. All UAB and Birmingham AIDS Outreach staff completed 

human subjects training required by the UAB Institutional Review Board (IRB). The original human 

subjects protocol was submitted to the UAB IRB for a full review in July 2013, and subsequently received 

approval. Annual investigator progress reports have been submitted as required. There have been no 

notable issues with the study’s IRB approval. 

JHU was a study partner but at no point did JHU collect or receive client-level data. JHU contacted the 

JHU IRB and explained JHU’s role in the study as a technical advisor. JHU’s IRB determined that JHU’s 

role constitutes non-human subjects research. 

Outcome and implementation evaluation: The Johns Hopkins IRB has determined the evaluation 

activities conducted by JHU to be non-human subjects research (including the work JHU does with 

regards to monitoring of participant-level outcomes, network analysis, case studies, and economic 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 58 
’

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

analysis). At the local level, each subgrantee site applied for IRB approval to carry out their proposed 

evaluation activities, including collection of patient-level data for the national evaluation. Table 25 

below outlines the date each subgrantee site obtained IRB approval, the final determination, and a brief 

overview of consent procedures. 

Table 25: IRB approval, final determination, and consent procedures by grantee 

Subgrantee Date of IRB approval Final determination Consent procedures 

ActionAIDS 4/1/2013 Project does not

constitute human 

subject research and

does not require IRB 

oversight 

NA 

AIDS Action Committee 

of Massachusetts 

9/26/2011 Minimal risk study At enrollment staff 

reviews consent form 

with participants who 

then sign and date. All

case managers have

completed Human 

Subject Training. 

AIDS Foundation 

Chicago 

9/2011 Human subjects

research 

At enrollment staff 

reviews consent form 

with participants who 

then sign and date. 

AIDS Project Los

Angeles 

7/2012 CBO Arm (Los

Angeles County

Department of Public

Health)

1/2013 Clinical Arm 

(University of Southern 

California (USC)) 

Human subjects

research 

At enrollment staff 

reviews informed

consent, California

Medical Bill of Rights, 

and the Medical

Release forms with 

participants. Signed and 

dated consent is

accepted for each

document. All staff

have been trained to

conduct human

subjects research and 

received certificates 

from USC and LA

County Department of

Public Health. 

Amida Care 11/30/2011 Non-research All data used for the

national evaluation are 

de-identified. 

Christie s Place 4/2011

9/11/2012 

Human subjects

research 

At enrollment staff 

reviews consent form 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 59 
“

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Subgrantee Date of IRB approval Final determination Consent procedures 

**this 2012 date was 

for a project revision 

with participants who 

then sign and date. 

Damien Center 6/26/2013 not necessary/not 

reviewable” 

Informed Participation 

Agreement signed by 

client 

Louisiana Public Health 

Institute 

8/6/ 2013 Human subjects

research 

Consent signed by 

participant 

Medical AIDS Outreach 9/19/2013 Human subjects

research 

Consent signed by 

participant 

St. Louis Effort for AIDS 5/2012 Human subjects

research 

Consent waived 

because all data are deidentified 

University of Alabama, 

Birmingham 

8/27/2013 Human subjects

research 

Performed by Research

Assistant prior to 

baseline activities;

baseline activities are 

scheduled to occur in

the community (at 

Birmingham AIDS 

Outreach)

Washington AIDS

Partnership, Positive 

Pathways 

6/17/2011 Exempt All data received by 

evaluation staff are deidentified. 

Timeline 

BA2C RCT impact study: UAB sent study progress reports to JHU and AU every six months, in June and 

December. The progress reports include information such as: changes to personnel or major duties of 

existing personnel, tasks completed and in progress, data collected, unforeseen problems that occurred, 

and a narrative of important events that occurred. Study enrollment began in August of 2013 and ran 

through January of 2015. Twelve-month follow-up data collection started in August 2014 and continued 

through February 2016. 

Monitoring of patient-level data: Subgrantees reported data to JHU every six months in June and 

December of each year. The final data was submitted to JHU in December of 2015. JHU cleaned and 

conducted final analyses through May 2016. 

Case studies and network analyses: Data were collected for the case studies and the network analyses 

approximately 18–24 months from the start of program implementation. There were two cohorts of SIF

subgrantees, and thus there are varying dates on data collection for case studies based on when 

implementation began Table 5 above highlights the dates for data collection for the case studies. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 60 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Evaluator/subgrantee role and involvement 

AIDS United and the JHU evaluation team have worked closely and collaboratively on the national 

evaluation of the SIF initiative. We participate jointly on calls with subgrantees sites following each data 

pull to discuss the evaluation process, any outstanding questions or issues, and next steps. AIDS United 

and JHU staff conducted two joint subgrantee site visits during 2014 and coordinate on communications 

and site visits whenever feasible. Additionally, we have collaborated closely on plans for the final year of

SIF funding and timelines for ending enrollment into the evaluation and completing follow up data 

collection cycles. 

Evaluation staff has remained consistent with Dr. Cathy Maulsby continuing to serve as the Principal

Investigator (PI) on the national evaluation and Dr. David Holtgrave continuing to serve as the Co-PI.

Blessing Enobun, MPH, has joined the Johns Hopkins evaluation team and has played an important role

in communication with subgrantees on data collection and analysis. At University of Alabama at 

Birmingham, Dr. James Raper serves as the PI, with Drs. Mugavero and Batey serving as Co-PIs for the 

randomized control trial of BA2C. 

References 

1. Bradley H, Hall HI, Wolitski RJ, Van Handel MM, Stone AE, LaFlam M, et al. Vital Signs: HIV

diagnosis, care, and treatment among persons living with HIV--United States, 2011. MMWR Morb 

Mortal Wkly Rep. 2014;63(47):1113-7. Epub 2014/11/27. doi: mm6347a5 [pii]. PubMed PMID:

25426654.

2. Green J, Thorogood N. Qualitative Methods for Health Research. Silverman D, editor. London:

Sage Publications; 2004. 

3. Holtgrave DR, Hall HI, Wehrmeyer L, Maulsby C. Costs, consequences and feasibility of strategies 

for achieving the goals of the National HIV/AIDS strategy in the United States: a closing window for 

success? AIDS Behav. 2012;16(6):1365-72. Epub 2012/05/23. doi: 10.1007/s10461-012-0207-0. PubMed 

PMID: 22610372. 

4. Farnham PG, Holtgrave DR, Gopalappa C, Hutchinson AB, Sansom SL. Lifetime costs and qualityadjusted life years saved from HIV prevention in the test and treat era. J Acquir Immune Defic Syndr. 

2013;64(2):e15-8. Epub 2013/09/21. doi: 10.1097/QAI.0b013e3182a5c8d4 

00126334-201310010-00022 [pii]. PubMed PMID: 24047975. 

5. Braithwaite RS, Meltzer DO, King JT, Jr., Leslie D, Roberts MS. What does the value of modern 

medicine say about the $50,000 per quality-adjusted life-year decision rule? Med Care. 2008;46(4):349-

56. Epub 2008/03/26. doi: 10.1097/MLR.0b013e31815c31a7 

00005650-200804000-00003 [pii]. PubMed PMID: 18362813. 

6. Walensky RP, Freedberg KA, Weinstein MC, Paltiel AD. Cost-effectiveness of HIV testing and 

treatment in the United States. Clin Infect Dis. 2007;45 Suppl 4:S248-54. Epub 2008/02/08. doi:

10.1086/522546. PubMed PMID: 18190295; PubMed Central PMCID: PMC2365915. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 61 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

7. Weinstein MC, Siegel JE, Gold MR, Kamlet MS, Russell LB. Recommendations of the Panel on 

Cost-effectiveness in Health and Medicine. JAMA. 1996;276(15):1253-8. Epub 1996/10/16. PubMed 

PMID: 8849754. 

8. Jain KM, Maulsby C, Brantley M, Kim JJ, Zulliger R, Riordan M, et al. Cost and cost threshold 

analyses for 12 innovative US HIV linkage and retention in care programs. AIDS Care. 2016:1-6. Epub 

2016/03/29. doi: 10.1080/09540121.2016.1164294. PubMed PMID: 27017972. 

9. Gold M, Siegel, J, Russell, L, Weinstein, M. Cost-effectiveness in health and medicine: report of

the panel on cost-effectiveness in heathl and medicine. New York: Oxford University Press; 1996. 

10. Holtgrave D. Handbook of economic evaluation of HIV prevention programs. New York: Plenum 

Publishing Corporation; 1998. 

11. Hall HI, Holtgrave DR, Tang T, Rhodes P. HIV transmission in the United States: considerations of 

viral load, risk behavior, and health disparities. AIDS Behav. 2013;17(5):1632-6. Epub 2013/03/05. doi:

10.1007/s10461-013-0426-z. PubMed PMID: 23456577. 

12. Holtgrave DR, Briddell K, Little E, Bendixen AV, Hooper M, Kidder DP, et al. Cost and threshold 

analysis of housing as an HIV prevention intervention. AIDS Behav. 2007;11(6 Suppl):162-6. Epub 

2007/07/10. doi: 10.1007/s10461-007-9274-z. PubMed PMID: 17616800. 

13. Kim JJ, Maulsby C, Zulliger R, Jain K, Charles V, Riordan M, et al. Cost and threshold analysis of 

positive charge, a multi-site linkage to HIV care program in the United States. AIDS Behav. 

2015;19(10):1735-41. Epub 2015/07/04. doi: 10.1007/s10461-015-1124-9 

10.1007/s10461-015-1124-9 [pii]. PubMed PMID: 26139421. 

14. Kim JJ, Maulsby C, Zulliger R, Jain K, Positive Charge Intervention T, Charles V, et al. Cost and 

threshold analysis of positive charge, a multi-site linkage to HIV care program in the United States. AIDS 

Behav. 2015;19(10):1735-41. doi: 10.1007/s10461-015-1124-9. PubMed PMID: 26139421. 

15. Aidala AA, Wilson MG, Shubert V, Gogolishvili D, Globerman J, Rueda S, et al. Housing Status, 

Medical Care, and Health Outcomes Among People Living With HIV/AIDS: A Systematic Review. Am J

Public Health. 2016;106(1):e1-e23. doi: 10.2105/AJPH.2015.302905. PubMed PMID: 26562123; PubMed 

Central PMCID: PMCPMC4695926. 

16. Espino SLR, Precht, A., Gonzalez, M., Garcia, I., Eastwood, E.A., Henderson, T., Blank, A.E., & 

Karasz, A. Implementing peer-based HIV interventions in linkage and retention programs: Successes and 

challenges. Journal of HIV/AIDS and Social Services. 2015;14(4):417-31. 

17. Garcia IA, Blank AE, Eastwood EA, Karasz A. Barriers and facilitators to the implementation of

SPNS interventions designed to engage and retain HIV positive women of color in medical care. AIDS 

Behav. 2015;19(4):655-65. doi: 10.1007/s10461-014-0837-5. PubMed PMID: 25107362. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 62 
 

 

 

 

 

Appendix 1: Local Cascades
	

Chicago:

http://www.cityofchicago.org/content/dam/city/depts/cdph/HIV_STI/2014HIVSTISurveillanceReport.pd 

f 

Retained in care in care- Total weighted percent of HIV-infected adults who had at least one 

documented HIV medical care visit in 2011 and had a documented HIV viral load of undetectable or 

<=200 copies/mL at most recent viral load test. 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 63 
 

 

 

Alabama:

http://adph.org/aids/assets/TreatmentCascade_2014_Preliminary.pdf

Alabama-Public Health Area 8, includes Montgomery County among others. 

Alabama-Public Health Area 4, includes Jefferson County among others.
	

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 64 
 

 

 

 

Washington, DC:

http://doh.dc.gov/sites/default/files/dc/sites/doh/page_content/attachments/HAHSTA%20HIV%20Care 

%20Dynamics%20supplement%20FINAL.pdf 

2014 Care dynamics among Ryan White Clients 

Indianapolis:

https://matecindiana.com/2015/04/13/april-13-2015-hiv-care-updates-for-the-indianapolis-carmelmetropolitan-statistical-area-hiv-continuum-of-care-measures-2014-vs-2013/ 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 65 
 

 

 

St. Louis:

http://stlplanningcouncil.com/files/treatmentcascade2014.pdf

Engagement in HIV Care, St. Louis HIV Care Region, 2014 (Provisional) 

New York City:

https://www.health.ny.gov/diseases/aids/ending_the_epidemic/meetings/docs/ete_nycdohmh.pdf 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 66 
 

Philadelphia: 

http://www.phila.gov/health/pdfs/2014%20Surveillance%20Report%20Final.pdf 

San Diego:

http://www.sandiegocounty.gov/hhsa/programs/phs/documents/STDMonthlyReport_Vol_6_Issue_1.pd 

f 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 67 
 

 

 

Boston:

http://www.mass.gov/eohhs/docs/dph/aids/2016-profiles/hiv-care-continuum-factsheet.pdf 

New Orleans:

http://new.dhh.louisiana.gov/assets/oph/HIVSTD/hivaids/2015/2013_STD_HIV_Surveillance_Report.pdf 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 68 
 

Los Angeles: 

http://publichealth.lacounty.gov/dhsp/Reports/HIV-STDsurveillanceReport2014.pdf 

AIDS United Social Innovation Fund Final Evaluation Report  May 2016 69