Measuring Systems Change: A Quick Guide

Why is this important?

Many programs funded by the Administration for Community Living (ACL) focus 

on improving the systems that serve older adults and individuals with disabilities. This quick guide is intended as an initial resource for ACL staff who oversee 

such programs. It provides a brief introduction to systems and systems change. 

For more detailed guidance, see Measuring Systems Change: A Brief Guide.
What Do We Mean by System?

A system is a group of subsystems or entities that are interconnected or interrelated and function for a common purpose to form a complex 

and unified whole. In the context of ACL, systems consist of multiple actors, programs, policies, agencies, or institutions at the national, state, 

and local levels with the common goal of achieving better outcomes for older adults and persons with disabilities.

For evaluation purposes, 

systems may be 

operationalized as: 

■ Pathways which may be examined with regard to their capacity (i.e., scale, quality, and 

comprehensiveness) and their connectivity (i.e., linkages, alignment, and cross-system 

coordination).

■ Institutional structures that represent the context in which pathways exist and may be 

examined in terms of the existing incentives, constraints and opportunities that impact 

the pathways.
What Do We Mean by Systems Change and How Can We Measure It?

Systems change occurs when there are fundamental and sustained transformations in policies, processes, relationships, and power structures, 

as well as deeply held values and norms. In the context of ACL, positive systems change may be thought of in terms of improvements in the 

programs and services ACL funds and improvements in the coordination with other systems.

Using the two concepts, 

pathways and institutional 

structures, systems change 

may be measured along 

the following dimensions:

Capacity of Pathways

■ Scale (e.g., has the number of program slots increased?)

■ Quality (e.g., are more providers within the system engaging in continuous quality improvement efforts?)

■ Responsiveness (e.g., have service gaps been identified?)

Connectivity of Pathways

■ Linkages (e.g., has coordination among providers improved?)

■ Alignment (e.g., are outcomes between programs/providers aligned to ensure they contribute to the same 

ultimate goals?)

■ Coordination (e.g., has duplication of effort been reduced?)

Institutional Structures

■ Institutional incentives, constraints, opportunities (e.g., are there less restrictive policies or more supportive 

policies to encourage programs/providers to engage in joint planning or cross-training?)
What Evaluation Method May be Used for Assessing Systems Change?

To measure change along the dimensions listed above, evaluators should apply evaluation approaches or frameworks that can account for 

dynamic and unpredictable environments and allow for flexibility. Developmental evaluation is one approach that can be effective for such 

complex dynamic environments as it is iterative and facilitates assessment of the current state of a system and also how elements of the system 

change over time. 

In developmental evaluation, 

the evaluator:

■ becomes part of the team that is working to bring about systems change.

■ should bring evaluative thinking into the process of development and 

intentional change.

■ works with strategic decision makers to interpret data and identify implications.

■ can use an array of data collection methods, such as interviews, focus groups, 

surveys, and observation.

For questions, contact ACL’s Office of Performance and Evaluation.