Evaluation Report Brief

John A. Hartford Foundation: IMPACT

Office of Research and Evaluation, Corporation for National and Community Service August 2018

nationalservice.gov/research 1 

What is the community challenge?

Low-income adults in medically underserved rural 

communities lack access to effective depression treatment 

services. Most of these areas lack the necessary mental health 

workforce so depression treatment, when it happens at all, 

occurs in primary care. However, rural primary care typically 

lacks the resources to offer treatments other than medication 

or to frequently and proactively follow-up with patients. 

What is the promising solution?

Collaborative care uses a team-based approach to depression 

treatment integrated into primary care. A behavioral care 

manager and a psychiatric consultant support the primary 

care provider so patients can be treated in the primary care 

clinic using the full range of evidence-based treatments and 

established principles of chronic illness care. This approach 

makes effective depression treatment more accessible while 

supporting patient engagement and empowerment. 

Program At-a-Glance

CNCS Program: Social Innovation Fund

Intervention: IMPACT (depression treatment model)

Grantee: John A. Hartford Foundation

Focus Area: Healthy Futures

Focus Population: Low-income adults living in 

medically underserved rural areas

Communities Served: Rural communities in 

Wyoming, Washington, Alaska, Montana, and Idaho

What was the purpose of evaluation?

Evaluation of collaborative care (also called IMPACT) in medically underserved rural communities began in 

2013 and finished reporting in 2017. The purpose of the evaluation conducted by the University of Washington 

AIMS Center was to replicate a proven program model in high-need rural areas that are often overlooked or 

underserved and, therefore, where patients may face challenges accessing effective depression care. The 

implementation evaluation looked at how well clinics implemented collaborative care, variation across sites, 

and factors accounting for observed variations in implementation. The impact evaluation also examined 

outcomes for 5,187 enrolled patients, including decreased depression and improved social and occupational 

functioning. 

What did the evaluation find?

As a grantee of SIF, the John A. Hartford Foundation engaged an independent evaluator to evaluate the 

implementation of IMPACT. 

•

•

Rural primary care clinics serving low-income patients can achieve improvement in depression 

symptoms comparable to those published in research trials and large-scale implementations. 

Patients with more severe depression at the start of treatment were less likely to experience complete 

absence of all symptoms by the end of treatment, which is not surprising with a low-income population 

experiencing the compound stressors associated with economic disadvantage. 
Office of Research and Evaluation, Corporation for National and Community Service August 2018

nationalservice.gov/research 2 

•

•

Clinic-level processes of care and patient-level clinical outcomes vary widely from clinic to clinic, 

which is consistent with other Collaborative Care implementations. Changing healthcare delivery is 

complex, especially when it requires new roles and changes to clinic workflows. 

Primary care providers, in short supply in rural clinics, were positive about their experiences with 

Collaborative Care which is important because provider burnout is associated with poorer health 

outcomes for patients and lower retention of providers. 

Notes on the evaluation

The Hartford Foundation chose to evaluate an existing program with a proven track record of strong evidence, 

encompassing multiple RCTs with positive results, by focusing on the experience of replicating the program in 

a rural context. The targeted and attained levels of evidence for the current study are preliminary. 

How is the John A. Hartford Foundation using the evaluation findings 

to improve?

The John A. Hartford Foundation is taking lessons learned from the 

implementation evaluation to refine future implementation. This 

includes taking into account the challenges of engaging and retaining 

rural patients in depression treatment (e.g., due to increased travel 

distances, lack of reliable/available telephone service, higher prevalence 

of stigma associated with mental health care, and more prevalent cultural 

expectations of independence and self-reliance). Results of this 

evaluation indicate it is valuable to include regular reviews of process 

and outcome data during implementation to adjust program 

implementation and better mirror treat-to-target principles clinics are 

already using at the individual patient level. 

Evaluation At-a-Glance

Evaluation Design: This was primarily 

an implementation evaluation, with an 

impact component looking at singlegroup pre-post differences across 8 

clinical sites. 

Study Population: Low-income adults 

living in medically underserved rural 

areas

Independent Evaluator: University of 

Washington AIMS Center

This Evaluation’s Level of Evidence*:

Preliminary

*SIF and AmeriCorps currently use different definitions of 

levels of evidence.

The content of this brief was drawn from the full evaluation report submitted to CNCS by the grantee/subgrantee. The section of the brief that discusses 

evaluation use includes contribution of the grantee/subgrantee. All original content from the report is attributable to its authors.

To access the full evaluation report and learn more about CNCS, please visit http://www.nationalservice.gov/research. 

The Social Innovation Fund (SIF), a program of the Corporation for National and Community Service (CNCS), combines public and private resources to grow the impact of innovative, 

community-based solutions that have compelling evidence of improving the lives of people in low-income communities throughout the U.S. The SIF invests in three priority areas: 

economic opportunity, healthy futures, and youth development.