National Community of Practice for Supporting Families Evaluation

April 30, 2019

CoP Annual Meeting: Presentation of Preliminary Reporting Tool Findings
1

Timeline of Evaluation Activities

Evaluation

Tool

Dates for Data 

Collection

Feb 

2019

Mar 

2019

Apr 

2019

May 

2019

Jun 

2019

Jul

2019

Aug 

2019

Reporting 

Tool

2/4/2019-

2/22/2019 ü

Interviews 3/25/2019-

4/26/2019 ü

Site Visits1

5/13/2019-

8/9/2019 «

Materials 

Review2

5/13/2019-

8/9/2019

1 Using state feedback, site visit approach revised to also include states from Groups 2 and 3 

2 Materials review will occur on an ad hoc basis
2

Evaluation Questions

Evaluation Questions Project Outcome

• To what extent did the national and state CoPs achieve consensus 

on a national framework for supporting families? 

• How are state CoPs using the CtLC framework to enhance supports 

to families of individuals with I/DD?

State and national consensus on a 

national framework and agenda for 

improving support for families with 

children with I/DD

• To what extent did components of the National CoP for Supporting 

Families contribute to enhanced state policies, practices, and 

systems? 

• What components of state CoPs contribute to enhanced state 

policies, practices, and systems? 

• How have the enhanced state policies, practices, and systems 

improved the supports available to families? 

• How have the lives of families of children with I/DD 

improved/changed as a result of the work of the CoP?

Enhanced State policies, practices, 

and sustainable systems that result 

in improved supports to families

• How are exemplary practices defined, identified, and verified?

• To what extent do state CoPs build capacity to sustain their work 

after formal affiliation with the National CoP for Supporting Families 

concludes? 

• Which practices defined as exemplary were most likely to be 

replicated and sustained, and why? 

• What secondary outcomes did state CoPs experience beyond the 

stated project outcomes (if any)?

Enhanced capacity of states to 

replicate and sustain exemplary 

practices to support families and 

systems
3

Evaluation Framework: Value Cycles

Source: Wenger, E., Trayner, B., and de Laat, M. (2011). Promoting and Assessing Value Creation in Communities and 

Networks: A Conceptual Framework. Rapport 18, Ruud de Moor Centrum, Open University of the Netherlands. 
4

Questions to Run On

• Do any of the major themes and findings surprise you or 

not accurately reflect the experiences in your state ? 

• As you think about the values generated by the CoP, are 

there missing themes not presented in the findings?
5

Cycle 1: Immediate Value

• Includes problem solving, inspiration, information sharing, 

collaboration, or mutual understanding 

• Examples: 

– Share information 

– Peer - to - peer support 

– Understand family perspectives 

– Establish state CoP structure
6

Cycle 1: Activities Generating Immediate Value
7

Roles of People with I/DD and Families in CoP
8

Activities Performed by CoP Leadership
9

Cycle 2: Potential Value

• Includes products, personal knowledge or new ideas, 

useful skills, relationships, reputation of the CoP, access 

to resources, and increased awareness about a particular 

domain 

• Examples: 

– Deliver training 

– Develop and disseminate products 

– Practice and apply CtLC tools 

– Identify focus areas
10

Cycle 2: Activities Generating Potential Value
11

Examples of Activities Generating Potential Value
12

Cycle 3: Applied Value

• Examines the ways in which knowledge leads to 

innovation, new approaches, or changes in practice 

• Examples: 

– Change policy 

– Change programs or services 

– Change forms and processes 

– Change staffing patterns 

– Launch new initiatives 

– Improve service monitoring
13

Cycle 3: Activities Generating Applied Value
14

Changes to State Policy

The District of Columbia enacted the Disability Services Reform 

Amendment Act of 2018 to improve the quality of life for people with I/DD by 

improving the complaint process, assuring no new people are committed 

and enhancing supported decision making.
15

Changes to Programs and Services

South Dakota approved requests from providers to implement 

various types of technology to promote independence and 

reduce reliance on the physical presence of paid supporters.
16

Changes to Initiatives

Delaware recently engaged in an initiative to evaluate supports and 

services for people with I/DD who are also deaf.

Ohio incorporated the CtLC framework into their statewide Technology 

First Initiative.
17

Cycle 4: Realized Value

• Includes improvements in performance that are 

meaningful to stakeholders 

• Examples: 

– Enhance family and self - advocate skills 

– Streamline access 

– Reduce waiting lists 

– Improve service experience 

– Increase progress toward better life
18

Cycle 4: Activities Generating Realized Value
19

Improvements Influenced by the CtLC framework
20

Applied and Realized Value
21

Cycle 5: Reframing Value

• Learning leads to redefining success, such as new goals, 

strategies, and performance metrics 

• Examples: 

– Measure quality in new and different ways 

– Change goals and strategies
22

Cycle 5: Activities Generating Reframed Value
23

Actions Taken to Support Sustainability (Group 1)

Oregon embedded CtLC in the scope of work and evaluation methods 

required of Family/Peer Support Contractors (OCFN) in order to better 

articulate the intent of those contracts and expected outcomes.
24

Next Steps

By mid - August 2019: 

• Conduct site visits to select states 

• Conduct materials review on an ad hoc basis 

By late September 2019: 

• Develop and share data summaries with each state 

• Develop and share final report of all findings from the past 

evaluation year
25

Conclusion
26

Questions

• Please feel free to contact the ACL program specialist or a 

member of the evaluation team: 

– Larissa Crossen , Larissa.Crossen@acl.hhs.gov 

– Cindy Gruman, Cindy.Gruman@Lewin.com 

– Diana Caldwell, Diana.Caldwell@Lewin.com 

– Natalie Boonchaisri, Natalie.Boonchaisri@Lewin.com
27

Thank you for your participation!