Administration on Aging

Chronic Disease Self-Management Program (CDSMP) 

Process Evaluation

HHSP233201100492G

Final Report Appendices

April 30, 2013

Authors: 

Cynthia Woodcock, IMPAQ International

Holly Korda, Altarum Institute

Erkan Erdem, IMPAQ International

Sarah Pedersen, IMPAQ International

Michelle Kloc, Altarum Institute

Erik Tollefson, Altarum Institute

Submitted to:

Project Officer: Susan Jenkins

Office of Performance and Evaluation

Administration for Community Living

 Department of Health and Human Services 

330 Independence Ave SW # 4760

Washington, DC 20201

Submitted by:

Project Director: Tina Barretto

IMPAQ International, LLC

10420 Little Patuxent Parkway 

Suite 300

Columbia, MD 21044

Telephone: (443) 367-0088
IMPAQ International, LLC Page i CDSMP Process Evaluation Final Report

Appendices

TABLE OF CONTENTS

Appendix A. CDSMP Technical Advisory Group Members

Appendix B. Site Visit Guide

Appendix C. Site Visit Summary

Appendix D. Key Informant Interview Questions

Appendix E. CDSMP Participants by AoA Region

Appendix F. CDSMP Participant Reach by Grantee

Appendix G. Class Size Fidelity by State Grantee

Appendix H. Statistical Tests for Chapter 6 Exhibits

Appendix I. CDSMP Workshops, Participants, and Completers BY QUARTER STATE GRANTEE

Appendix J. Logit Regression Analysis

Appendix K. CDSMP Evaluation Efforts by State Grantee
IMPAQ International, LLC Appendix A CDSMP Process Evaluation Final Report

Appendices

APPENDIX A. CDSMP TECHNICAL ADVISORY GROUP MEMBERS
IMPAQ International, LLC Page A-1 CDSMP Process Evaluation Final Report 

Appendices

APPENDIX A. CDSMP TECHNICAL ADVISORY GROUP MEMBERS

Name Affiliation Bio

Teresa Brady, PhD Senior Behavioral Scientist 

at the CDC - Atlanta

Dr. Brady has extensive chronic disease program 

knowledge and expertise in AoA/CDC partnership 

building.

Joshua Chodosh, MD UCLA Associate Professor in 

Residence, Geriatrics

Dr. Chodosh has extensive program knowledge, with a 

focus on outcomes for older adults.

Candace Goehring, MN RN Unit Manager,

Aging and Disability 

Services, Washington State 

Department of Social and 

Health Services

Ms. Goehring has extensive chronic disease 

program knowledge. She has experience working 

with the Aging Network and has served as an 

implementer for AoA EBDDP grantees.

Gerry Mackenzie, MSS Director of Community 

Resources, Education and 

Wellness in the Division of 

Aging and Community 

Services, NJ Dept. of Health 

and Senior Services.

Ms. Mackenzie has over 20 years experience in the 

management and development of state and 

community-based programs for older adults. She 

currently manages the older adult health and 

wellness program, senior health insurance program, 

Aging and Disability Resource Connection and the 

statewide information and assistance service. Ms. 

Mackenzie is the project manager for NJ’s EvidenceBased Disease Prevention for Older Adults program.

Ruth Palombo, PhD Assistant Secretary, MA 

Executive Office of Elder 

Affairs

Dr. Palombo has experience in the MA network to 

provide evidence-based program and ad research 

interests in the development of strategies to promote 

successful aging among older adults with chronic 

conditions and disabilities.

Peter Reed, PhD Senior Director of Programs 

for the National Office of 

the Alzheimer’s Association

Dr. Reed has extensive chronic disease program 

knowledge and has experience as a 

Researcher/Policy Developer and ARRA Reviewer in 

the field of Public Health.

Beth Richards Arthritis Program Director, 

Missouri 

Ms. Richards the State CDSMP Lead for the State of 

Missouri and has set a best practice in imitating the 

relationship between the Missouri Arthritis and 

Osteoporosis Program and State Health and Aging 

Departments to implement evidence-based 

interventions.

Carlene Russell Iowa Healthy Links Project 

Director, Department of 

Elder Affairs

Ms. Russell has extensive chronic disease program 

knowledge. She has experience working with the 

Aging Network and has served as an Implementer 

for AoA EBDDP grantees.

David Sobel, MD Regional Director of Patient 

Education for the Northern 

California Kaiser 

Permanente Medical Care 

Program

Dr. Sobel’s research and teaching interests include 

medical self-care, patient education, preventive 

medicine, behavioral medicine and psychosocial 

factors in health. He has co-authored seven books 

written for the general public including Living a 

Healthy Life with Chronic Conditions, The Healing 

Brain, and Healthy Pleasures. He has been Involved 

in CDSMP RCT design.
IMPAQ International, LLC Page A-2 CDSMP Process Evaluation Final Report 

Appendices

Name Affiliation Bio

Nancy Wilson Assistant Director for 

Program Development,

Baylor University,

Huffington Center on Aging

Ms. Wilson has extensive chronic disease program 

knowledge and has performed research concerning the 

Aging Network.
IMPAQ International, LLC Appendix B CDSMP Process Evaluation Final Report

Appendices

APPENDIX B. SITE VISIT GUIDE
IMPAQ International, LLC Page B-1 CDSMP Process Evaluation Final Report 

Appendices

APPENDIX B. SITE VISIT GUIDE

Chronic Disease Self-Management Program (CDSMP) Site Visit Guide

“As you know, we are employees of IMPAQ International and Altarum Institute, and we are talking 

with you today as part of our work with the Administration on Aging. The purpose of this discussion 

is to collect information about how your [state/site] is implementing the Chronic Disease SelfManagement Program (CDSMP) to inform a national evaluation that is currently being designed. 

Although during the course of our discussion we will ask specific questions about various aspects of 

your program, this interview is not intended to monitor or to evaluate your program. Many of your 

responses will inform what questions to include in a national web-based survey that is part of this 

project. Your participation in this interview is voluntary and you may choose to discontinue the 

interview at any time, for any reason. What we learn about your efforts from this interview will be 

combined with input from other respondents, within your state and across five other states, and 

incorporated into an evaluation design report to the Administration on Aging (AoA). Your name and 

your role in this effort may be disclosed to our colleagues at AoA, but no specific statement or 

quotes will be attributed to you in the report. 

Do you have any questions before we begin?”

Domain 1: CDSMP Populations, Marketing/Recruitment

Topic Site Visit Questions

1.1 Populations Served 1.1 Populations Served 

CDSMP Participants 1. How have the participants changed over time? 

2. How do actual participants map to targeted populations?

3. What challenges have sites experienced in reaching, enrolling and keeping 

participants (i.e., retention)?

1.2 Marketing/Outreach 1.2 Marketing/Outreach

Target Populations 1. What is your catchment area? Is it urban/rural?

2. What provider groups do you target in your marketing efforts (e.g., health plans, 

hospitals, FQHCs, primary care physicians, health homes, pharmacies, DME 

suppliers)? 

3. What is the message that you seek to communicate?

4. How do you segment the consumer market in your marketing and outreach 

efforts (e.g., by age, ethnicity, gender, chronic condition, type of health insurance 

(private, Medicare, Medicaid))? 

5. How would you describe your target market?

6. What consumer groups do you market to directly (e.g., churches, YMCA, local 

AARP chapter)? What is the message that you seek to communicate?

7. What challenges have you faced in working with provider groups? Are there some 

groups that you would like to work with that you have not been able to?

8. What approaches have been best for attracting provider groups?

9. How do you measure whether you are reaching your target population?
IMPAQ International, LLC Page B-2 CDSMP Process Evaluation Final Report 

Appendices

Distribution Channels 1. What distribution channels do you use to disseminate program information?

a) Targeted mailings: to whom, how many, response rates?

b) Website: host website, target market, URL, number of hits?

c) Flyers/brochures: distributed how and where?

d) Telephonic marketing: to whom, how many calls/year, who conducts the 

calls, response rates?

e) Radio or TV: which channels, type and length of ads, number of 

broadcasts/year?

2. For each distribution channel, which population(s) are you targeting?

3. How did you develop your dissemination approach? Was it new for this effort? 

Was it something that you used before? Was it preplanned or did it develop more 

organically?

4. Which approaches have worked best and why?

Referring Organizations 1. Have you established a formal referral network? If so, what organizations, 

agencies, programs, and providers are in the network?

2. Is there an informal referral network? If so, describe.

3. Do you notice a difference between consumers who self refer as compared to 

those that are referred by your various referral sources? (e.g., in consumer 

demographics, in completion rates, in participation/interest in the classes)

Marketing Organization 1. Who is responsible for developing a marketing plan for CDSMP? What is the 

marketing message(s), who are the target populations, and what are the 

distribution channels? How often is the marketing plan revised?

2. Who is responsible for implementing the marketing plan (or, in the absence of a 

marketing plan, carrying out marketing activities): the state grantee, host site(s), 

and/or implementation site(s)? 

3. Who is responsible for monitoring marketing/outreach efforts? 

4. Did you develop a new marketing campaign for your CDSMP program or is the 

marketing for CDSMP part of an existing effort?

Marketing Messages 1. What messages do you aim to communicate to providers and consumers?

2. Have you “branded” CDSMP in your state? How?

Marketing Budget 1. What was your marketing budget for CDSMP for the most recent fiscal year? 

2. How were marketing funds apportioned to the state level, host site level, and 

implementation site level?

Marketing Research 1. Have you collected any data on how providers and/or consumers find out about 

CDSMP (e.g., via an enrollment form or in a follow-up survey)? If so, what are the 

findings? 

2. Have you conducted any other marketing research? If so, what were the research 

questions? What did you find?

Lessons Learned 1. How do you measure whether your marketing efforts are paying off?

2. What approaches have you found to be most successful in marketing CDSMP?

3. What barriers have you encountered in marketing to specific provider and/or 

consumer populations? How might these barriers be overcome?

Doman 2: Site Implementation

Topic Site Visit Questions

2.1 Budget and Financing 2.2 Budget and Financing - Also see DOMAIN 5: Statewide Distribution and Delivery

Cost Structure 1. Do you know the cost per participant?

2. How did you calculate this cost?

Program Budget 1. How does the cost CDSMP compare to other programs you administer?

2. Over the past three years, has the budget increased or decreased? Has the 

program had a deficit or surplus?
IMPAQ International, LLC Page B-3 CDSMP Process Evaluation Final Report 

Appendices

Grant Funding 1. What are the types of funding you have received and types of funding that you 

will seek to sustain the program?

Medicare Reimbursement

Medicaid Reimbursement

1. Has the state been accredited by the American Diabetes Education Association to 

provide CDSMP for Medicare reimbursement?

2. Is CDSMP provided under Medicaid managed care in your state (i.e., through MCO 

capitation payments)?

Health Homes 1. Is CDSMP provided through medical/health homes in your state (e.g., Medicaid 

health home, ACO, FQHC)?

Fee Structure 1. What is the state’s fee structure for a CDSMP workshop for the different payers, if 

any, that reimburse for this service?

Financial Sustainability 1. Does the state have a strategy for funding CDSMP over the long term? If so, what 

is it? [Note: One of the challenges for programs like CDSMP is moving from a 

“grant culture” to a funding base structured around payment or reimbursement 

for services rendered. Probe whether the state recognizes this and is addressing it 

in their funding strategy.]

2. Does your site have formal business plan for sustainability?

2.2 Fidelity to Stanford 

Model

2.2 Fidelity to Stanford Model

Trainer/Leader Fidelity and 

Retention

1. What supports do sites offer to leaders or trainers?

2. Are master trainers tracked or followed?

3. Who paid for the facilitator training?

4. Do your facilitators work at multiple host sites?

Fidelity Monitoring 1. “At the agency level, fidelity refers to the how closely staff members follow the 

program that the developers provide. This includes consistency of delivery as 

intended and the time and cost of the program.” If not, how would you define 

“fidelity” to the Stanford model? To what extent do you think your program 

operates with fidelity to the Stanford model?

2. Do you have a Quality Assurance Plan? If so, obtain a copy. 

3. If the state’s completion rates are less than 70%, probe fidelity to see if there may 

be an issue.

4. Have you incorporated any changes to the Stanford model that you believe 

enables the program to better meets the needs of the 60+ population or increases 

completion rates and long-term compliance? Do you have any evidence of this?

5. Do you a process for regularly monitoring fidelity? If so, describe (e.g., protocol, 

frequency, metrics, responsible parties, feedback).

6. Do you have a process for continuous quality improvement? If so, describe.
IMPAQ International, LLC Page B-4 CDSMP Process Evaluation Final Report 

Appendices

Domain 3: CDSMP Eligibility, Enrollment, and Completion

Topic Site Visit Questions

3.1 Registration and 

Enrollment

3.1 Registration and Enrollment

Process 1. Do you conduct any type of eligibility screening or administer any kind of 

assessment prior to enrolling a consumer (e.g., age, chronic condition, functional 

limitations, financial status, readiness to participate)? If so, what are the criteria for 

determining “readiness” or eligibility/non-eligibility? What is the basis of turning 

people down? Is this a documented/written process used throughout the 

program? If yes, request a copy.

2. How do consumers enroll in CDSMP? Online, in-person, mail, or by telephone? 

3. Is enrollment centralized at the state level or host site level? Or do consumers 

enroll at the implementation site level?

4. Do people tend to enroll as individuals or as part of a couple or group (e.g., a group 

of friends join together)? How does this effect class participation and completion?

Workshop Structure 1. Are workshops targeted at specific population groups (e.g., age, gender, and 

ethnicity) or chronic conditions, or are workshops mixed groups?

2. Are workshops structured to take advantage of existing social networks? If so, how 

(e.g., location, group enrollments)?

3. Are the classes accessible to the target population in terms of how often classes 

are offered, locations of classes, etc?

4. Do your workshop sessions follow the curriculum recommended by Stanford (i.e., 

order of classes, number of hours, learning objectives and material covered in 

each class)? If not, how do your workshops differ and why do you do them 

differently? 

5. Do you keep the ordering of the topics covered in the six CDSMP sessions fixed? 

That is, first day you cover Topic A, second day you cover Topic B, etc. and this 

ordering is constant across all host sites and implementation sites. What is the 

ordering of the classes? [This is supposed to be the case and gives us a sense of 

variability across programs or adaptations?

6. Continuity: What is the longest period of time in the last 12 months that you have 

gone without a workshop?

Workshop Scheduling 1. What is the class schedule for the current year?

2. Is class scheduling centralized at the state level or host site level? Or do 

implementation sites do their own scheduling?

3. Does the state maintain a master schedule of classes? If so, where can consumers 

and providers access it?

Workshop Enrollment 1. What was actual enrollment for each class offered in the past year versus available 

class slots?

2. In general, does demand exceed supply of class slots or can consumers generally 

find a class slot when they want it?

3. Do you maintain a waiting list? If so, how many people are on it and at what 

locations? Approximately what percent of applicants are placed on the waiting 

list? What is the typical waiting time to start a workshop?

4. If you have a waiting list, what efforts have you made to eliminate it?

5. If you have a waiting list or cannot accommodate someone for a particular 

workshop, do you ever refer consumers to other (non-CDSMP) self-management 

programs? If so, which programs? Is a slot usually available?

Orientation 1. Do you have orientation sessions or Class Zero? If yes, describe the curriculum, 

session length, and level of participation.
IMPAQ International, LLC Page B-5 CDSMP Process Evaluation Final Report 

Appendices

Registration Fees 1. Do you charge a fee to workshop participants? If so, how much? 

2. If you charge a fee, do you offer a sliding fee schedule, scholarships, and/or free 

enrollment for individuals meeting certain criteria? What are those criteria?

3. If you charge a fee, are fees ever paid by a third party (e.g., health plan, 

health/medical home, FQHC, Medicaid, Medicare, health insurance)?

4. What infrastructure is in place for billing and processing receipt of fees (e.g., 

manual or electronic system, accept checks, cash, credit cards, part of a larger 

organization’s billing system)?

3.2 Workshop Participation 3.2 Workshop Participation

Enrolled Population 1. Do you set goals for enrollment by site? If so, what are they and were they 

achieved (i.e., is your enrolled population representative of the population you are 

trying to reach)?

2. What is the prevalence of workshop participants with cognitive deficits? Have you 

implemented strategies aimed at better serving this population? What are those 

strategies? Any lessons learned?

3. Do you track re-enrollment by non-completers? If so, what do you know about 

these individuals?

4. Do you track re-enrollment by completers who wish to take the program again? If 

so, what do you know about these individuals?

Workshop Participation 1. Do you regularly examine attendance logs for problems?

2. Have you examined attendance records by enrollee characteristics (e.g., age, 

gender, ethnicity, chronic condition, disabilities, health insurance)?

3. Do you track milestones or achievements by participants? Which ones and why? 

4. Do you offer incentives to participants to achieve specific goals or complete the 

program (e.g., cash, gift cards, services)? If yes, what is the incentive structure and 

what is the source of funding for the incentives?

5. Are there particular topics that participants seem to be most interested in?

Workshop Completion 1. Do you set goals for completion by site? If so, what are they and were they 

achieved?

2. Do you conduct “exit interviews” with participants (completers and noncompleters) to learn about satisfaction with the program and reasons for noncompletion?

3. What is your perception of non-completers versus completers (e.g., 

characteristics, availability of supports, barriers)? Any data to support these 

perceptions?

4. Have you implemented strategies to increase completion rates? If so, what?

Workshop Follow-up 1. Do you follow up with workshop participants in any way after the workshop is 

concluded (i.e., effectiveness evaluations, such as questionnaires 4 to 6 months 

following the program)?

2. Do you offer any kind of “refresher” course for individuals who have completed 

CDSMP?

3. Do you call participants who drop out of workshops?
IMPAQ International, LLC Page B-6 CDSMP Process Evaluation Final Report 

Appendices

Domain 4: Data Collection

Topic Site Visit Questions

4.1 Data Collection and 

Program Monitoring

4.1 Data Collection and Program Monitoring

Data Collection Capability 1. Reporting to NCOA: Do you report the required data to NCOA on a quarterly 

basis? How are the data collected, aggregated, and submitted? Who is responsible 

for data collection, aggregation, and submission?

2. What other program data do you collect on a regular basis (e.g., purpose, users, 

data elements, frequency, format)? How are the data collected and who is 

responsible for data collection and reporting? Please share sample reports.

3. To what extent are your data collection systems electronic and centralized? What 

kind of software do you use?

4. What staff resources (e.g., positions, skills/expertise) are available for data 

collection and reporting? 

5. Do you partner with a university or private research organization on data 

collection and reporting?

Program Monitoring 1. How and what data is used for performance monitoring? What metrics do you 

use? Reporting to whom?

2. How do you use data for continuous program improvement?

Assessment and Evaluation 1. Have you conducted any evaluations of CDSMP to measure the effects of CDSMP 

on health outcomes and costs? If so, what datasets did you use (e.g., MMIS, MDS, 

program data)? What was the study design? What were the findings? Are there 

reports available?

2. Have you followed any workshop participants beyond their workshop 

participation to see if there were sustained benefits of CDSMP? If so, how long? 

Findings?

3. How are evaluation findings being used to promote the program and ensure its 

sustainability?

4.2 Evaluability 4.2 Evaluability

Randomization 1. If asked to participate in the RCT, what process would you suggest for randomizing 

participants into CDSMP given the unique features and processes of your 

program? Randomization during a centralized registration process? 

Randomization using provider settings?

2. What are the chances that control group participants would enroll in another 

similar program?

3. How long would control group participants be willing to wait to join a CDSMP 

workshop? 6 months?

Leadership and Resources 1. Probe whether the state has the resources and expertise to participate in an 

electronic Participant Tracking System.

2. Probe whether the state has leadership and staff who recognize the importance of 

a RCT and have the requisite expertise and commitment to manage a RCT.

Domain 5: Statewide Distribution and Delivery System

Topic Site Visit Questions

5.1 Program 

Structure/Delivery System

5.1 Program Structure/Delivery System

State Grantee 1. Who is the leadership? What is their authority, roles, and responsibilities? How 

would you rate their level of commitment to CDSMP?

2. Is there a CDSMP state-level advisory board? If so, how often does it meet? Who is 

on it? How are members appointed? What authority does it have? 
IMPAQ International, LLC Page B-7 CDSMP Process Evaluation Final Report 

Appendices

Responsibilities? Funding? 

3. What state-level agencies partner with the state grantee on CDSMP (e.g., aging 

unit, department of health, Medicaid, human services department)? What is the 

role and responsibilities of each partner? Is there a written partnership 

agreement?

4. How many employees at the state level have CDSMP responsibilities? What are 

their positions, who do they report to, and what percent of their time do they 

spend on CDSMP, doing what? Is there a program coordinator?

5. Explain the organizational structure of CDSMP and the relationships/functions of 

the state grantee vs. host sites vs. implementation sites.

6. Is there an organizational chart for CDSMP? (If not, sketch one with the assistance 

of the state grantee.)

7. What specifically is the state grantee’s role in program planning, administration, 

and delivery?

8. What is the role of AAAs and ADRCs in program planning, administration, and 

delivery?

9. Who holds the Stanford license for CDSMP? 

10. To what extent is the CDSMP delivery system “institutionalized” with dedicated 

staff and resources at the state, host site, and implementation site levels?

11. To what extent is CDSMP delivered as a “solo” program versus a program that is 

part of a constellation of other programs (e.g., other evidence-based programs, 

other health promotion programs)? If the latter, how do you think this influences 

consumer participation in CDSMP as well as other programs?

12. Have you conducted an adoption evaluation to look at the settings and/or 

organizations that are offering a program and how successful each is?

Host Sites 1. Is there a council representing host sites? If so, how often does it meet? Who is 

the chairperson/ leadership? What authority does it have? Responsibilities? 

Funding? How are members appointed?

2. How many employees of each host site have CDSMP responsibilities? What are 

their positions, who do they report to, and what percent of their time do they 

spend on CDSMP, doing what? Is there a project coordinator and a dedicated 

recruiter at each host site?

3. What specifically is the role of host sites in program planning, administration, and 

delivery?

Implementation Sites—

Facilities

1. What is the relationship of these organizations with CDSMP? Are there employees 

at these organizations with responsibility for managing the CDSMP workshops? If 

so, what percent of their time do they spend on CDSMP? Or does the organization 

merely provide space for the workshops?

2. Describe the facilities and equipment available to CDSMP at implementation sites. 

Is it adequate or is additional equipment or space needed?

3. What fees, if any, does CDSMP pay to use the various implementation sites?

Master Trainers and Leaders 1. How many Stanford-certified master trainers and T-Trainers are available in your 

state? How many were available to train leaders during the past year? Do any of 

them lead workshops as well?

2. Are the master trainers employees of an agency or organization in your state, or 

do they work as independent contractors?

3. Does your state or your host sites provide any additional training to master 

trainers beyond what is recommended by Stanford? If so, on what topics and 

why?

4. How many leaders did the master trainers in your state train each year over the 

past three years? 

5. Do you ever conduct a Master Training with less than 2 Certified T-Trainers? If so, 
IMPAQ International, LLC Page B-8 CDSMP Process Evaluation Final Report 

Appendices


 

what difficulties are you experiencing with regard to meeting recommendations?

6. Do your master trainers facilitate one 4-day Leader training within one year of 

completing Master Training? How often do you have difficulty meeting this goal? 

What do you feel are the difficulties? Do you proactively plan to hold Leader

Trainings within one year of Master Trainers completing training?

7. Do you ensure that, if training multiple Master Trainers, each facilitates a Leader?

8. Do each of your master trainers lead a full 4-day Leader training at least once 

every year, to remain certified? If you hire Master Trainers, do you confirm they 

meet these training goals (do you use the Stanford Patient Education Center’s 

website to determine ‘active’ trainers?)

9. What process and curriculum do master trainers follow in training leaders? Are 

there documented learning objectives and competencies that must be met?

10. How consistently are you able to adhere to the recommended schedule for Leader 

Trainings (total of 4 days – most recommended 2 days per week for 2 weeks)?

11. How do you recruit master trainers and leaders? Do you have a recruiting strategy, 

goals, work plan? Who is responsible for recruiting?

12. If you hire Master Trainers, do you use the Stanford Patient Education Center’s 

website to determine ‘active’ trainers? Do you use Master Trainers who have not 

conducted a class within the past year?

13. Do you start leader trainings with less than 12 potential Leaders or with more than 

25? 

14. Do you have a protocol for directly observing Leaders during sessions? 

15. Do you have workshops scheduled by the Leaders complete training, so they can

meet the goal facilitating within 6 months of training? If Leaders are unable to

facilitate within 6 months, do you require a refresher prior to facilitating?

16. Do new Leaders co-facilitate their first workshop with an experienced, active

Leader?

17. Who are the master trainers and leaders (e.g., age, gender, ethnicity, chronic

conditions, disabilities, years of experience)? Are you achieving a 30% male trainer 

rate? Are they paid or unpaid? If paid, by whom?

18. Does your state have a central registry of master trainers and leaders? If so, who 

maintains it?

19. How do you assign leaders to workshops (e.g., randomly vs. try to match leaders 

with certain kinds of consumers)?

20. What is the attrition rate for master trainers and leaders? If you’ve been losing a

number of them, why do you think that is and what could be done about it?

21. Have you ever experienced shortages in master trainers or leaders? Has this

caused problems with staffing workshops or being able to train new leaders? 

22. Have you ever had to run a workshop with only one leader (not two as required)? 

Do you train 3 Leaders for new areas, such that there is a back-up?

23. What do you do if a leader is sick?

24. Do you have a process for periodically evaluating master trainers and leaders? Do 

you track completion rates by master trainer/leader? If so, what have you found 

and how do you use the evaluations for program improvement?

25. Do you ask workshop participants to evaluate their leaders? If so, what has been

their feedback? How do you use these evaluations for program improvement?

26. In your opinion, what are the attributes of the “best” master trainers and leaders?
IMPAQ International, LLC Page B-9 CDSMP Process Evaluation Final Report 

Appendices

Domain 6: Consumer and System-Level

Topic Site Visit Questions

6.1 State/System 

Environment

6.1 State/System Environment

State Budget 1. Has CDSMP by affected by state budget deficits? If so, how (i.e., funding levels, 

staffing, number of sites)?

Political Environment 1. To what extent is the current administration supportive of CDSMP? (Democratic or

Republican administration?)

Aging Network 1. What are some of the major programs of the state unit on aging? Would you 

consider CDSMP to be a priority program?

Medicaid Reform 1. Is your state considering any major Medicaid reform? If so, what (e.g., health 

homes, managed care, 1115 waiver)?

2. Are there ways in which CDSMP could benefit from Medicaid reform?

Competing Programs 1. Are there other CDSMP-type programs that could be considered competing 

programs? What programs? What is their catchment area? Target populations? 

Comprehensiveness of 

services

2. Are there other CDSMP-type programs that could be considered competing 

programs? What programs? What is their catchment area? Target populations? 

CDSMP’s role in other 

programs

3. Are participants linked to or from other prevention/health programs (e.g., falls 

prevention, PEARLS)? Is CDSMP part of a larger constellation of services? If so, 

what else is included in the constellation? Do they think this linkage makes a 

difference in terms of the types of consumers they get, enrollment rates, and/or 

completion rates?

Complicating Factors 1. Is the effect program influenced by disease progression or other age-related 

declines?

2. Have problems been reported with respondent memory issues?

3. What is the respondent and site burden?

4. Are there particular outcomes of interest that are not possible to capture with 

current data or techniques?

5. When using proxy respondents for those for diminished cognitive ability, what are 

the challenges?

What cultural factors or adjustments are made? How do these adjustments affect the 

outcomes?

6.2 Consumer Environment 6.2 Consumer Environment

Measures of interest to AoA 1. In addition to CDSMP-specific measures, AoA is interested in the following 

measures which may not directly by related CDSMP. Is your site exploring any of 

the following 1) Health maintenance, 2) Independence in the community, and 3) 

What is the effect of diminished cognitive ability on course adherence, completion 

and outcomes? In particular, does this program support health maintenance? 

Does it support helping participants remain independent in their communities?

Outcomes that CDSMP does 

not address

1. Are there issues that participants bring that CDSMP does not address? Does the 

program meeting their expectations in terms of what is advertised?

6.3 Participant Questions 6.3 Participant Questions
IMPAQ International, LLC Page B-10 CDSMP Process Evaluation Final Report 

Appendices


 

The goal of speaking to participants is to get a sense of whether individuals are willing 

to be randomized, or willing to wait 6 months to take the workshop (if it would mean 

important research could be conducted that benefits older adults, etc).

1) Participant introductions, when did you take the class, how many times have you 

taken the class

2) How did you hear about the program?

3) Did you come to the class with friend(s) or family?

4) Did you have to pay for the course?

5) Was there a class zero? How do you feel about having a class zero?

6) Did you have to wait to get into the class?

7) Was the class easy/convenient to get to?

8) Was the facility and accommodation adequate?

9) Were you able to come to all the classes? If not, why?

10) What were the other participants like?

11) What did you think of your leaders?

12) Was the course content (book, charts, information) useful to you?

Do you have recommendations or ideas for change?

13) Are you doing anything differently now as a result of the class? If so what?

14) Have you shared anything you learned or the course book (informally or formally) 

with family or friends?

15) Have/would you recommend the course to others?

16) Do your leaders follow up with you after the class? What do you think about the 

idea of a follow up after the class? Have you kept in touch with your leaders?

17) Are there other programs, like the CDSMP, that you have taken or are aware of? If 

so, what are those?

18) Do you have any thoughts about the marketing – is it adequate or do you have 

recommendations for improvements?

19) Is there anything else you would like to share?
IMPAQ International, LLC Appendix C CDSMP Process Evaluation Final Report

Appendices

APPENDIX C. SITE VISIT SUMMARY
IMPAQ International, LLC Page C-1 CDSMP Process Evaluation Final Report 

Appendices

APPENDIX C. SITE VISIT SUMMARY

Chronic Disease Self Management Program (CDSMP) 

Process Evaluation and Detailed Outcome 

Evaluation Design

HHSP233201100492G

Draft CDSMP Site Visit Summary

February 28, 2012

 Submitted To: 

 Susan Jenkins, COTR

Administration on Aging, DHHS

CC: 

Margo Katzper, CO

Program Support Center, DHHS

Submitted By: 

Daver C. Kahvecioglu, Project Director

IMPAQ International

Site Visit Team:

Michelle Kloc, Altarum Institute

Holly Korda, Altarum Institute

Sarah Ruiz, IMPAQ International

Cynthia Woodcock, IMPAQ International
IMPAQ International, LLC Page C-2 CDSMP Process Evaluation Final Report 

Appendices

SYNTHESIS OF SITE VISIT FINDINGS

During January and February 2012, researchers from IMPAQ International and Altarum Institute 

conducted site visits in six states to gain an understanding of the experience of states, host1

, and 

implementation2 sites in implementing the Stanford Chronic Disease Self Management Program 

(CDSMP) as funded by the U.S. Administration on Aging (AoA) under contract HHSP233201100492G:

CDSMP Process Evaluation and Detailed Outcome Evaluation Design. Information obtained through 

the site visits will inform development of a Web-based survey of states, host, and implementation 

sites, and process evaluation findings, and will provide input to guide the design of a rigorous 

outcome evaluation under the contract. 

We conducted two or three day site visits in Arkansas, California, Kansas, New York, Tennessee, and 

Vermont with teams of two or three individuals. Following review of National Council on Aging 

(NCOA) data3 tracking implementation status and analyses of State Progress Reports4

, the team 

developed an interview guide, including input from AoA representatives, to address key study 

domains to explore onsite. Using data from the NCOA Sustainable Infrastructure and Delivery System 

Self-Assessment, the project team selected thirteen state grantees of interest for AoA’s 

consideration. To obtain an understanding of the diversity of contexts in which CDSMP has been 

implemented, six of the thirteen states were selected to represent regional and organizational 

diversity, and included visits with both developmental and mature sites. Over the course of the site 

visits, the team spoke with eleven host sites, eight implementation sites, six groups of master 

trainers, ten groups of lay leaders, and seven groups of participants. This brief report highlights key 

site visit findings, followed by state summaries and one state comparison table5

. 

Overview

CDSMP Populations, Marketing and Recruitment, Enrollment, Eligibility 

Demand for CDSMP is generally weak across states. For most sites, recruiting participants is the 

biggest challenge, with only a few host sites reporting no difficulty. Recruitment and enrollment 

approaches are localized and vary considerably across sites. Sites that offer other evidence-based 

programs often use these programs as a source for recruitment into CDSMP or a platform for 

 1 Host organizations are the organization or agency that sponsors CDSMPs offered in each state. The Host 

Organization is often responsible for training CDSMP Master Trainers and Lay Leaders, and for planning and 

monitoring the implementation of workshops. Often (but not always) the Host Organization holds the license to 

train and offer CDSMP. Host Organization may also serve as an Implementation Site.

2 Implementation Sites are the physical location where CDSMP workshops are offered in the community. An 

Implementation Site may be identical to the Host Organization, or it may be a location (such as a community center, 

health care facility, church, etc.) that the Host Organization arranges to use. 3 To facilitate the collection and reporting of the required quantitative participant data, standardized pen and 

pencil forms and a web-based data entry system are available to State Grantees through NCOA. Grantees must be 

trained and license to access the system.

4 States grantees are required to submit standard Grantee Progress Report on a quarterly basis. Grantee Progress 

Reports include both quantitative and qualitative sections on program success.

5 See Table 1 at the end of this report.
IMPAQ International, LLC Page C-3 CDSMP Process Evaluation Final Report 

Appendices

offering a constellation of evidence-based programs. We heard some participants, leaders, and 

trainers say that they appreciate the value of taking the course more than once and also taking other 

self-management courses. One trainer stated that it would be best to take the generic CDSMP first, 

and then take the disease-specific courses. Some sites use informal methods to screen potential 

participants for enrollment or accommodate individuals with a disability (i.e. cognitive or visual 

impairment). Two sites reported efforts to screen participants for mental health and cognitive ability 

to participate. Sites use a variety of approaches to recruit participants, from targeted panel 

management through electronic health records (EHRs), websites, presentations at senior centers, 

personal contacts with prospective participants, advertisements in local newspapers and 

newsletters, and on local radio and television. Panel management involves identifying patients in a 

health care practice for outreach from medical records or claims data. Use of Class Zero6 is 

infrequent, given by either the host or implementation site staff. However, one state reported 

conducting Class Zero before about half of the workshops offered. Rarely does state-level staff 

deliver these presentations. Some states and some localities have given brand names to CDSMP, but 

the names vary.

Site Implementation and Fidelity

Workshops are provided in diverse settings including a bowling alley, restaurants, congregate meal 

locations, senior centers, senior housing, health centers, hospitals, and churches. Participants 

generally enjoy the social bonding with others in their workshops, and many continue to meet after 

the workshop has ended. A few enroll in CDSMP a second time. Lay leaders may receive stipends 

ranging up to $350 (plus mileage reimbursement) for 6 sessions. One state paid $1,500 to a host site 

for registering 10 participants and an additional $200 for each completer. A number of host sites 

have trained staff members as master trainers and lay leaders as a way to “embed” CDSMP in their 

organization before American Recovery and Reinvestment Act (ARRA) funding ends.

Fidelity7 is interpreted and implemented inconsistently across state grantees, host and 

implementation sites, master trainers8

, and lay leaders. Whether and the extent to which sites 

monitor fidelity varies from grantees stating that the program is implemented with fidelity to more 

extensive monitoring using state-specific check lists and frequent observations of workshops. States 

typically determine which aspects of fidelity are important and feasible to monitor. States may have 

“official” and “unofficial” policies, for example, to determine the number of participants required for 

 6 Class Zero, also referred to as a “pre-session” or “zero class,” is a recruitment session designed to increase 

enrollment for CDSMP workshops. Class zero sometimes serves as mechanism to screen clients who are not 

serious about participating in the workshop. 

7 Stanford’s Fidelity Manual can be found at http://patienteducation.stanford.edu/. Fidelity requirements include 

areas such as licensing, training, and how the program is monitored. The organization that holds the CDSMP 

license is responsible for monitoring fidelity and reporting information related to fidelity to Stanford. Every 

organization offering a Stanford program must have a license for the program and each license lasts for 3 years. 

Each licensed organization must submit a yearly report which includes the number of workshops, number of 

participants, number of leader trainings conducted, and the names of leaders trained with contact information. 8 Trained leaders (T-trainers, master trainers, lay leaders) also have an obligation to monitor fidelity. Master 

trainers provide some oversight of CDSMP workshops, train workshop leaders, work in pairs, and serve as 

workshop leaders themselves. Lay Leaders are individuals who facilitate the CDSMP workshops.
IMPAQ International, LLC Page C-4 CDSMP Process Evaluation Final Report 

Appendices

 

workshops. Sites are implementing fidelity adaptations ranging from class sizes below Stanford 

requirements, lay leader workshop presentation requirements (some providing only one workshop 

annually), providing one-on-one follow-up visits, and use of the same buddy for all six classes rather 

than changing buddies every session. One master trainer described the fidelity requirements as 

“fuzzy,” leaving some gray areas for interpretation. 

We also discovered some innovative ideas about ways to incorporate CDSMP into the health care 

continuum such as group visits for the CDSMP participants in a given workshop. Accommodations 

have been made for special groups, for example using audio for blind persons. Some sites reported 

that it would be nice if they were made aware of any changes in the training materials before they 

go out and use them. One leader reported that she was surprised to see two different versions of the 

CDSMP book at a workshop since she was not aware of any changes Stanford made. Another person 

reported that the most recent fidelity manual changed some important requirements (such as the 

requirements to maintain trainer certification) that led to differences in interpretation of when the 

new manual goes into effect. Some states noted they have trained too many (or too few) master 

trainers. States with too many are often unable to provide sufficient workshops for the trainers to 

ensure their continued certification, though these states are not aggressively monitoring trainers or 

leaders for ‘active’ certification. In rural areas, meeting Stanford’s requirements for leaders to 

maintain their certification can be a challenge. The smaller populations in rural areas means there is 

not a need for so many workshops in a given year. However, the reluctance of many leaders to travel 

long distances means that states still need to train a large cadre of leaders. 

Statewide Distribution and Delivery

We noted great variation in state infrastructure and delivery networks as well as the degree to which 

the lead agency has decentralized responsibility for program implementation, management, and 

oversight. One program was rooted in a statewide hospital network involved with broader state 

health reform activities; others were based in state public health departments and their local 

networks, aging services networks including AAAs, or were comprised of loosely affiliated

independent community-based organizations without formal networks. The site visit team

discovered that states with longer histories of offering CDSMP tended to have more established, 

sustainable infrastructures9 for CDSMP than the states with more recently established programs, 

many of which have found initial implementation and sustainability difficult to achieve in the current 

economic environment. 

Sustainability after the end of ARRA funding is not guaranteed. Some developmental states say they 

are not sure how they will continue offering CDSMP. Some used ARRA funding to train staff from 

host organizations and implementation sites as leaders and see this as a way to continue offering 

workshops. One mature state reported they will probably downsize but are trying to find funding to 

sustain CDSMP. One state reported that ARRA funding will not impact their commitment or ability to 

provide CDSMP.

9 Institutions are not as susceptible to fluctuations in grant and temporary funding and include organizations such 

as hospitals.
IMPAQ International, LLC Page C-5 CDSMP Process Evaluation Final Report 

Appendices

Data Collection and Feasibility of Outcome Evaluation

Sites need to modify enrollment procedures significantly to participate in a RCT. The sites also have 

to develop data collection capacity and participant tracking capability. Adequate enrollment will be a 

challenge in many states and localities. Enrollment numbers are likely to fall after ARRA funding ends 

and states limit their marketing activities. Achieving adequate enrollment for a national RCT will 

most likely require a major marketing campaign in evaluation sites. Sites were generally favorable 

towards a rigorous evaluation as they thought it was very important for sustainability (to get 

potential funders’ buy-in). Some sites even expressed that they would prefer an evaluation of this 

type sooner rather later. However, sites will generally be very reluctant to deny service (required for 

RCT). The evaluation should not reduce the total number of people served. The agency who funds 

the evaluation may have to provide additional funding to encourage more participants to take the 

workshops than otherwise would have taken the workshops in the absence of the evaluation. 

Reduced social isolation/increased social networks for the older adults may be an important benefit 

of CDSMP and a potential program outcome to evaluate to evaluate in addition to concrete 

measures such as healthcare utilization and expenditures.

Goals of the Evaluation:

 One interviewee noted that if the ultimate goal is to achieve sustainable systems to provide 

CDSMP, then it is critical for the evaluation to consider agendas of potential funders. For 

example, the evaluators should learn the specific outcomes/measures potential funders 

(e.g., insurance companies) are interested in and should incorporate those into the design.

 The end product of this project is to design a rigorous national outcome study of the CDSMP 

in AoA-funded settings. Sites are not implementing “the” CDSMP as defined by the more 

than 100 standards10 laid out in the Standard fidelity manual. If AoA evaluates CDSMP as it is 

implemented on the ground, it will be difficult to attribute the impacts to a well-defined 

program due to potentially important divergences from some of the CDSMP requirements. 

The fidelity dimension appears to need closer scrutiny. What are the few core standards out 

of the more than 100 standards currently listed in the manual? It is also important to 

consider the possibility that some of the adaptations may be effective even though they are 

inconsistent with the CDSMP must-do’s.

 10 These standards are called “must-do’s” in the manual. There are also a number of “nice-to-do’s” as well.
IMPAQ International, LLC Page C-6 CDSMP Process Evaluation Final Report 

Appendices

ARKANSAS SITE VISIT SUMMARY

The research team visited the Arkansas CDSMP program, Be Well, Live Well, on Tuesday and 

Wednesday, January 17 and 18, 2012. The lead agency is the Arkansas Department of Health 

(ADOH). The first meeting on Day 1 was with the state coordinator and data analyst from ADOH 

along with the representative from the Division on Aging and Adult Services (DAAS) with

responsibility for CDSMP within that agency. This team of three accompanied us on all interviews 

during our 2-day visit. 

Our second meeting on Day 1 was convened at a senior center in Rogers, AR, in the northwest region 

of the state. Participants included a master trainer who was an Area Agency on Aging (AAA) 

employee, two lay leaders, and five participants who had recently completed a workshop together. 

The lay leaders were a husband-wife team—a retired educator and a retired social worker. The 

Rogers site was still developing, but was the most advanced of the three we visited. The senior 

center is owned by the city, serves an affluent retiree population (Rogers is the home of Sam Walton 

and Wal-Mart), and has about 1,100 people come through the door each day.

On Day 2, our first meeting was with two lay leaders, one lay leader, and six participants at a senior 

center in Jacksonville, AR, not far from Little Rock. One of the lay leaders was the activities director 

for the senior center; the other was employed by the AAA. The lay leader was an employee of the 

senior center. The participants had recently completed the same workshop together. We also met 

with two lay leaders and a lay leader at a senior center in Hot Springs, AR. Area Agencies on Aging 

(AAA) managed the CDSMP programs in Rogers and Jacksonville; AAI managed the CDSMP program 

in Hot Springs. The below table provides an overview of the interviews from the Arkansas Site Visit. 

Arkansas: 

Overview of 

Visit

Arkansas: Overview of Visit - 

Interviews

Arkansas: Overview 

of Visit - Facility

Arkansas: 

Overview of Visit

- Location

Day 1, Session 1 State Officials-Aging and 

Health

Department of Aging 

(State Grantee) Little Rock, AR

Day 1, Session 2 Met with AAA staff person 

who is also a master trainer

Wellness Center (Host 

Site) Rogers, AR

Day 1, Session 2 Focus Groups with lay leaders 

and participants

Wellness Center (Host 

Site) Rogers, AR

Day 2, Session 1

Met with AAA and Wellness 

and Activity Center staff, one 

of who was a lay leader

Wellness and Activity 

Center (Host Site) Jacksonville, AR

Day 2, Session 1 Focus group with participants Wellness and Activity 

Center (Host Site) Jacksonville, AR

Day 2, Session 2

Met with staff from Medical 

Sciences staff and Center on 

Aging Staff

Medical Sciences 

Center (Host Site) Hot Springs, AR
IMPAQ International, LLC Page C-7 CDSMP Process Evaluation Final Report 

Appendices

1. CDSMP Populations, Marketing and Recruitment, Eligibility Screening

The state targets all adults aged 60 and older, but accepts younger people into Be Well, Live Well. 

Most participants have participated through senior centers and have been Caucasian; African 

Americans have been less likely to join. There are some Southeast Asian and Latino populations in 

the state, but few have participated in the program.

Most marketing is performed locally by host and implementation sites through newspapers, 

newsletters, and flyers at senior centers. There is no statewide Website that advertises Be Well, Live 

Well. The ADRC, Choices in Living, has a Website and an 800 telephone number, and staff members 

have access to workshop calendars for referrals. However, the state noted that they have not done 

as good a job as they could with this marketing channel. 

Host sites promote the program to faith communities. The Hot Springs site has been especially 

successful with the local churches, which seek out opportunities for their members. The state 

promotes the program at Medicare Improvements for Patients and Providers Act (MIPPA) health 

fairs promoting Medicare plans and with the State Health Insurance Assistance Program (SHIP), a 

federally funded program that provides counseling to Medicare beneficiaries on health insurance. To 

date, there has been no systematic outreach to health plans or providers at either the state or local 

level.

2. Site Implementation

Arkansas received a Centers for Disease Control and Prevention (CDC) Empowerment grant 5 to 6 

years ago for evidence-based program development. Startup under the AoA ARRA grant was 

delayed, with the state hiring two directors of short duration before a 0.5 FTE program coordinator 

was hired a year after the grant award and a 0.5 FTE data analyst 3 months later. This, in turn, 

delayed host site startup. Implementation and enrollment have been more challenging in rural areas 

due to low literacy and limited understanding of the benefits of the program, as well as a lack of 

transportation and sparse population density.

In addition to an eight-region network of AAAs, the state has a six-region network of Centers on 

Aging established by the University of Arkansas Medical Sciences (UAMS) Arkansas Aging Initiative 

(AAI) with tobacco settlement funds. Both the AAA and AAI regions cover the entire state and 

overlap with each other. Both the AAAs and AAI operate CDSMP, sometimes collaboratively. The 

role of the AAI Centers on Aging is education (not services); many at the state and in the AAAs feel 

that diverting tobacco settlement funds to AAI was not a wise use of public resources and makes for 

a duplicative delivery system.

Be Well, Live Well is offered free of charge in Arkansas. However, with ARRA funding about to end, 

local sites are considering charging for books. Some of the sites have lending libraries and only give 

books to participants if they complete at least four sessions. Using ARRA funding, ADOH pays each 

host site $300 for each completer, up to a maximum of $15,000 (50 completers). ADOH also pays 
IMPAQ International, LLC Page C-8 CDSMP Process Evaluation Final Report 

Appendices

each AAA region $2,400 for lay leader trainings. The Rogers site reported stipends of $100 per 6-

week workshop for lay leaders. The state requires a minimum of 8 people to start a class and a 

maximum of 15 per class. 

The state takes an informal approach to monitoring fidelity, observing some classes and monitoring 

attendance forms. Maintaining the certification of master trainers and leaders is a challenge for this 

mostly rural state, given the low population density and travel distances for workshops and trainings. 

Modifications to the CDSMP protocol include allowing participants to keep the same buddy 

throughout the 6-week workshop if the participant is more comfortable with that arrangement 

(Jacksonville site) and one instance of a workshop being compressed into 3 weeks. The Hot Springs 

site pleaded for the ability to use PowerPoint instead of some of the flipcharts to save time and 

paper.

The state does not offer Class Zero or an orientation, nor does it track participants after the 

conclusion of a workshop. Strategies to sustain the program included embedding lay leaders and 

master trainers as paid staff of ADOH, AAAs, and AAI, and a requirement in DAAS’s 4-year plan that 

each AAA must offer one EBP.

3. CDSMP Eligibility, Enrollment, Completion

Implementation sites and workshop leaders handle registration locally, mostly by telephone. The 

local sites do not perform any systematic eligibility screening. Class size and completion rates vary by 

class and implementation site. The state will meet its AoA enrollment and completion goals, but is 

finding it challenging to increase awareness of and demand for the program. The state reports 467 

completers and a goal of 500. AAI’s goal is 300 completers; the AAAs’ goal is 200 completers. The 

AAA sites have higher completion rates than the AAI sites.

4. Data Collection

Data on Be Well, Live Well participants, enrollment, and completion are collected by workshop 

leaders in hard copy and submitted to the ADOH Older Adults and Communities section directly, for 

sites supported by ADOH. For sites supported by the Arkansas Department of Human Services DAAS, 

hard copy data are reviewed by the Division and forwarded to the ADOH. All hard copy data are 

checked and entered into an Excel spreadsheet by a part-time staffer who submits information to 

the National Council on Aging (NCOA). Only information required by Stanford is collected.

5. Statewide Distribution and Delivery

Be Well, Live Well workshops have been offered on a very limited basis by DAAS in two Arkansas 

counties since 2006 through AoA Empowerment grants, and by ADOH with limited funding from the 

CDC. ARRA funding enabled expansion of the workshops through partnership of ADOH and DAAS 

and their respective stakeholder networks. A statewide stakeholder coalition has formed to help 
IMPAQ International, LLC Page C-9 CDSMP Process Evaluation Final Report 

Appendices

establish and support statewide implementation of CDSMP, including Medicare and state Medicaid 

program representatives. The state (ADOH and DAAS) has also reached out to faith-based 

communities, SHIP, and others.

Be Well, Live Well workshops are provided through the ADOH and DAAS using AAAs and senior 

centers as host sites. These sites, in turn, may reach out into the community for implementation 

partners, e.g., churches or retirement communities. A master trainer at one host site reported their 

senior center had conducted considerable outreach to local physicians, who have not been too 

receptive, and has attempted to recruit participants by offering other classes, such as healthy 

cooking sessions, at local churches in African American communities.

Overall, the state distribution and delivery infrastructure is developmental and faces continuing 

challenges in sustaining Be Well, Live Well workshops. The DAAS has been able to include, as a state 

policy requirement, that all AAAs offer at least one evidence-based program. This is seen as a 

positive step forward in sustaining CDSMP and other programs. The ADOH, prime grantee under the 

AoA ARRA grant, cannot sustain the program beyond ARRA funding. The current program director 

will be reassigned within ADOH. However, two lay leaders have been trained at ADOH and will be 

available as embedded staff to provide the program.

6. Feasibility of Outcome Evaluation

State ADOH and DAAS staff, workshop leaders, and participants expressed support for conducting a 

rigorous outcome evaluation, but funding challenges are likely to pose problems for workshop 

availability and presentation. Data collection will also pose challenges as the part-time state position 

to support data collection will be eliminated when ARRA funds expire.

Participants at all sites appeared willing to delay workshop participation to enable and evaluation. 

However, they emphasized the importance of informing participants at study start that they would 

be part of a research study. A workshop leader at one site stated that participation in research 

studies posed no problem, noting that the senior center has already participated successfully in 

several studies. This leader further stated that she would be very interested in participating in a pilot 

to assess the relative contribution of various fidelity elements in presenting the workshop if such a 

study were to be conducted.
IMPAQ International, LLC Page C-10 CDSMP Process Evaluation Final Report 

Appendices

CALIFORNIA SITE VISIT SUMMARY

The research team conducted site visits with California representatives involved with CDSMP 

workshops called Healthier Living. The interviews took place on Monday and Tuesday, February 13 

and 14, 2012. The lead agency is the California Department of Aging (CDA). The first Day 1 meeting 

was with two CDA officials, three Public Health Department officials, and the technical assistance 

contractor for the state. The second Day 1 meeting was a site visit of a rural and developing CDSMP 

host site. The team met with an AAA director, another AAA staff member, and the local program 

coordinator who is also a lay leader. At this site visit, the team also met with four lay leaders and 

three program graduates. 

The first Day 2 meeting took place with local city and county health collaborators who serve as a host 

site. A representative from a county school district, who adopted the program for adult education, 

also attended this meeting. Under this partnership, teachers who are part of the adult education 

program receive training as lay leaders and hold CDSMP workshops. The second Day 2 meeting was 

with a city healthcare district11 that received a grant from Kaiser Permanente to implement CDSMP 

in their area. This meeting took place with the director of the healthcare district center and the local 

program coordinator who also serves as a lay leader. The third Day 2 meeting was a visit to a mature 

urban senior center that served as a host and implementation site. The team met with the evidencebased program coordinator who also served as a lay leader, and nine program graduates. The below 

table provides an overview of the interviews from the California Site Visit.

California: 

Overview of 

Visit

California: Overview of Visit -

Interviews

California: Overview 

of Visit - Facility

California: 

Overview of Visit

- Location

Day 1, Session 1

State officials-Aging and 

Public Health, Technical 

Assistance Contractor

Department of Aging 

(State Grantee) Sacramento, CA

Day 1, Session 2

Site director and staff, 

program coordinator, lay 

leaders, program participants

Area Agency on Aging 

(Host Site) Vallejo, CA

Day 2, Session 1

Los Angeles Department on 

Aging director and contract 

staff, representative from 

school district, technical 

assistance contractor 

program staff and CEO

Los Angeles 

Department of Aging 

(Host Site)

Los Angeles, CA

 11 Healthcare districts in California were developed in the aftermath of World War II, when American 

soldiers return from war with extensive medical treatment needs. At the time, California was dealing 

with acute hospital bed shortages and legislated was based to enact the Local Hospital District Act, later 

known as the Health Care District Act. It authorizes communities to form Special Districts to construct 

and operate hospitals and other health care facilities to meet local needs, funded through property tax 

assessments with acute hospital bed shortages and enacted legislation for the Local Hospital District Act, 

later known as the Health Care District Act. 
IMPAQ International, LLC Page C-11 CDSMP Process Evaluation Final Report 

Appendices

California: 

Overview of 

Visit

California: Overview of Visit -

Interviews

California: Overview 

of Visit - Facility

California: 

Overview of Visit

- Location

Day 2, Session 2

Healthcare district director 

and CDSMP program 

coordinator

Health Care District 

(Host and 

Implementation Site)

Ventura County, 

CA

Day 2, Session 3

Evidence-based program 

director (lay leader), program 

participants

Senior Center 

(Implementation Site)

Downtown Los 

Angeles

1. CDSMP Populations, Marketing and Recruitment, Eligibility Screening

California serves an extremely diverse population, but with variation by region. AoA-funded CDSMP 

workshops target older adults over age 60, but encourage individuals of all ages to attend. Providing 

CDSMP in remote rural areas of California is much more difficult. At this point, California does not 

have statewide coverage, with many rural communities and regions outside of workshop coverage 

areas. For culturally and linguistically diverse populations, CDSMP is offered in other languages 

(Spanish, Chinese, Korean) when there are available lay leaders for the class. It is often difficult to

recruit and train bilingual lay leaders. 

Most upcoming workshops in California are advertised on the CDA Website. While the NCOA and the 

state technical assistance contractor, Partners in Care Foundation (PICF), provide templates for 

marketing and recruitment materials, most recruitment for the class occurs at the local level. Host 

sites reach out to their partners or constituents to participate in the class. At the AAA host site the 

team visited, the program coordinator reached out to individuals visiting the AAA and asked the 

information and referral specialist at the site to refer clients to the program. The program 

coordinator also reached out to AAA partners in the community (e.g. senior centers, housing) to 

recruit. The second host site the team visited, a city healthcare district, reached out to their plan 

members and individuals in the community. The final site visited, an urban senior center, recruited 

only from members at the center. The senior center plans to recruit from in the community in the

near future, but has experienced problems recruiting individuals through their partners for other 

evidence-based programs currently implemented at the site.

2. Site Implementation

The site team visited three host sites during the California visit including an AAA, a Health Care 

District site, and a senior center. The senior center also served as an implementation site for the 

program, and the AAA and Health Care District held workshops at their partner sites. All sites offered 

CDSMP free of charge to participants. The demand was highest in the senior center and Health Care 

District site. The senior center sometimes had waitlists for the class, but this was largely a result of 

individuals who had scheduling conflicts and wanted to take the workshop later. The Health Care 

District recently offered a “wellness club” or incentive program for individuals who participate in 

more than one class. The district experienced an increase in demand as a result. The AAA host site 

was new to offering CDSMP and experienced challenges in starting the program due to delays 
IMPAQ International, LLC Page C-12 CDSMP Process Evaluation Final Report 

Appendices

resulting from the California budget approval process, and difficulty training lay leaders. With no 

master trainers associated with the site, the AAA had to invest in lay leader training by bringing in 

two master trainers from southern California. Despite training 21 lay leaders, the AAA still 

experiences difficulty scheduling and offering workshops, and experienced a greater than expected 

drop off of lay leaders.

Funding sources mentioned throughout the state included Community Block Grants, CDC, AoA, and 

local host site and community resources. Sites plan to continue to offer the workshop after ARRA 

funding ends, but one host site reported that the pool of lay leaders has decreased significantly and 

there will not be funds available to conduct another lay leader training. Sustainability following ARRA 

funding is not assured. One host site reported that the internal funding experts would seek outside 

funding or grants to sustain the program. California has been able to leverage community block 

grants for CDSMP to provide a graduated payment scheme to sites. Sites receive decreasing levels of 

funding over a three-year period, with the expectation that they sustain the program when the 

funding ends.

3. CDSMP Eligibility, Enrollment, Completion

None of the sites visited reported any eligibility screening. Sites were encouraged to over enroll 

because drop out is viewed as inevitable. Class Zeros were offered inconsistently throughout 

California. PICF developed a Class Zero curriculum for sites to use that offers an effective “teaser” for 

individuals interested in enrollment. One lay leader reported that completion rates were higher for 

classes that resulted from a Class Zero recruitment session, as individuals had a better sense of 

workshop requirements and commitment. Registration usually occurs with the coordinator at the 

local site. Advertisements for the program generally request that local individuals be contacted.

4. Data Collection

Data packets from the workshop are all returned to a staff designee at PICF. Information required for 

ARRA funding is entered directly into NCOA database and the additional data elements are filed 

away, with the expectation that the additional data elements will be used in the future, pending new 

funding. Prior to receiving ARRA funding, PICF maintained a separate data system for the 2006 

CDSMP grantees. Starting with 2010 funding, PICF started to enter data into the NCOA database. 

5. Statewide Distribution and Delivery

The CDA is the prime grantee responsible for oversight of the CDSMP program. CDA addresses policy 

relating to the Healthier Living program and contracts program implementation and technical 

assistance to PICF. To distribute the ARRA funding, the state released an RFP to provide CDSMP, to 

which 22 applicants responded. PICF ranked the applications based on criteria that included 

partnering agreements and a plan for roll-out, but CDA selected the six sites that received funding. 

PICF serves as the statewide technical assistance contractor, collects program data, and provides 

site-level technical assistance for CDSMP and other evidence-based programs. At the outset of the 
IMPAQ International, LLC Page C-13 CDSMP Process Evaluation Final Report 

Appendices

grant, sites were able to select among three levels of support from PICF, from low levels of 

involvement to higher levels of support that might include a broader range of activities, including 

site-specific program consultation and support.

After ARRA funding ends, the program will be downsized. The statewide steering committee plans to 

meet less regularly. The CDA asked committee members which activities they would find most 

beneficial and prioritized those for future funding. The intent of this approach was to help ensure 

program sustainability after ARRA funding ends. The CDA believes that most sites will continue to 

offer CDSMP.
IMPAQ International, LLC Page C-14 CDSMP Process Evaluation Final Report 

Appendices

KANSAS SITE VISIT SUMMARY

The research team site visited the Kansas CDSMP program, Kansans Optimizing Health Program 

(KOHP), on Monday through Wednesday, January 9 through 11, 2012. The lead agency is the Kansas 

Department of Health (KDoH). Historically, the Kansas Department of Health and Environment 

(KDHE) led development of chronic disease management programs in the state and provides a 

strong leadership role for KOHP.

The first meeting on Day 1 was in Topeka with lead staff from KDoH and KDHE. That afternoon, the 

team traveled to Wichita and first met with a lay leader and a workshop participant who also 

coordinates delivery of KOHP at a senior center, followed by a meeting with seven lay leaders 

employed by a hospital, the AAA, local clinics, and a senior housing complex.

On Day 2, the team met with a master trainer and lay leader in Newton, followed by a meeting with 

three participants. That afternoon, the team went to Hutchinson for meetings with seven lay leaders 

and seven participants.

On Day 3, the team was back in Wichita to meet with the AAA director, a master trainer, three lay 

leaders, and two representatives from Wichita State University, all of whom participate in what is 

referred to as the KOHP community leaders group, which is taking the lead in implementing KOHP in 

Wichita and surrounding counties. The below table provides an overview of the interviews from the 

Kansas Site Visit.

Kansas: 

Overview of 

Visit

Kansas: Overview of Visit - 

Interviews

Kansas: Overview of 

Visit - Facility

Kansas: Overview 

of Visit - Location

Day 1, Session 1 State officials-Aging and 

Health

Department of Aging 

(State Grantee) Topeka, KS

Day 1, Session 2

Met with two participants 

who now serve as a lay leader 

and implementation site 

director 

Area Agency on Aging 

(Host Site) Witchita, KS

Day 1, Session 3 Focus Group with lay leaders Area Agency on Aging 

(Host Site) Witchita, KS

Day 2, Session 1 Met with master trainer, lay 

leaders, and participants

County Recreation 

Center (Host Site)

Harvey County, 

KS

Day 2, Session 2 Met with lay leaders and 

program partners

Department of Aging 

(State Grantee) Reno County, KS

Day 2, Session 2 Met with program 

participants

Department of Aging 

(State Grantee) Reno County, KS

Day 2, Session 1

Met with AAA director, 

master trainer, lay leaders, 

and local program 

coordinators

Area Agency on Aging 

(Host Site)

Sedgwick County, 

KS
IMPAQ International, LLC Page C-15 CDSMP Process Evaluation Final Report 

Appendices

1. CDSMP Populations, Marketing and Recruitment, Eligibility Screening

The state targets all adults aged 60 and older, but serves younger people as well. One of the host 

sites is the Veterans Administration, which serves just veterans. There are areas in Wichita, Kansas 

City, and southwest Kansas with large Latino populations; the state recently trained 2 master trainers 

and 12 leaders to teach Tomando. Tomando participants are typically females in the “middle 

generation” caring for children and parents.

Most marketing is performed locally by host and implementation sites through newspapers, 

newsletters, and flyers distributed through senior centers, county departments on aging, and the 

Kansas State Research & Extension. The state hosts a KOPH Website listing upcoming workshops; 

prospective participants must call the local sites to register. The Kansas State Research & Extension 

Service’s Website and the service’s individual county Websites have information on KOHP, but 

accessibility and content varies. Lay leaders take calls from prospective participants and manage the 

registration process, carrying out targeted one-on-one marketing in the process. 

Some community health centers are experimenting with flagging patients with chronic conditions in 

the patient’s chart so that they can be referred to KOHP. However, there are questions about the 

feasibility of this approach as the clinics move to electronic health records.

2. Site Implementation

Because of a series of staff reductions and retirements in KDoA, startup under the ARRA grant was 

delayed. Additional staff reductions and reorganizations are anticipated for all state agencies as the 

budget crisis continues and the state moves to a Medicaid-managed care program and enacts health 

reform. 

Historically, KDHE led development of chronic disease management programs (initially with a CDC 

grant for an arthritis program). KDHE continues to be the de facto lead because KDHE’s staff has the 

most experience with these programs. AAA participation in KOHP has progressed slowly. The Kansas 

State Research & Extension Service trains its own lay leaders and plays a strong role in KOHP delivery 

through its network of county offices.

In Wichita, the KOHP community leaders group—an informal coalition that includes representation 

from the AAA, a local medical center, clinics, a senior housing complex, and Wichita State University 

(WSU)—is beginning to build momentum. In a unique arrangement, WSU provides administrative 

and marketing support using graduate student interns in aging studies who receive credit for 

assisting the program 10 hours per week, looking toward capitalizing on future research and 

evaluation opportunities. WSU targets provider and consumer groups to market the program and 

build a referral network.

Local sites may choose to charge a fee for the workshops, but most do not. The fee in Wichita count 

is $25. The state is considering requiring a $25 fee to cover the cost of books, but local sites do not 
IMPAQ International, LLC Page C-16 CDSMP Process Evaluation Final Report 

Appendices

favor this, believing even a minimal fee would be a barrier to many participants in need of the 

program.

One Class Zero was offered in a housing complex in Wichita. The sites do not conduct any follow-up 

with participants who complete the program. There is no formal process for fidelity monitoring.

3. CDSMP Eligibility, Enrollment, Completion

Registration is handled locally (mostly by telephone) by lay leaders whose phone numbers are listed 

on Websites and marketing materials. There is no electronic registration system. The local sites do 

not perform any systematic eligibility screening.

The state has had difficulty generating demand for KOHP and filling classes, especially in rural areas. 

There are no waiting lists.

4. Data Collection

Data for NCOA on participants, enrollment, and completion rates are completed on paper and faxed 

or emailed to KDHE for data entry. In addition, the state requests that KOHP participants complete 

an evaluation form at the end of the workshop requesting feedback on how the participant heard 

about the workshop, satisfaction, and usefulness of the techniques taught. A data analyst funded 

through the ARRA grant periodically analyzes the data and provides reports to the state and local 

sites.

5. Statewide Distribution and Delivery

Responsibility for KOHP is largely decentralized, with AAAs and the Kansas State Research & 

Extension Service’s network of county offices responsible for program development, marketing, and 

implementation in their respective regions. The Veterans Administration also serves as a host site.

The state has sponsored two summits to bring together community partners to brainstorm about 

how to build and sustain a CDSMP program. The first summit was convened in January 2011 in the 

Wichita area (Sedgwick, Butler, and Harvey counties). The Wichita community leaders group was an 

outgrowth of this summit. The second summit was in September 2011 in Johnson County. 

The state has a “leader maintenance plan” that includes a listserv, quarterly Webinars, and annual 

meetings. The local sites reported that this assistance is very helpful. At an October 2011, annual 

meeting for leaders at Kansas State University, GMAMMA, an improvisational actor, used humor to 

communicate the importance of fidelity. 

The state has requested a 6-month, no-cost extension to their ARRA grant (they have unspent funds 

because of their delayed startup). To help sustain the program, the state has applied for a 

competitive CDC grant. The state is also actively pursuing a system where master trainers and 

leaders are embedded (i.e., employed) in a local system that offers KOHP. Local sites are also 
IMPAQ International, LLC Page C-17 CDSMP Process Evaluation Final Report 

Appendices

encouraged to raise their own funding. For example, the Department of Aging in Reno County has 

received three $1,000 grants from a local private senior center. The funds have been used for 

supplies and snacks.
IMPAQ International, LLC Page C-18 CDSMP Process Evaluation Final Report 

Appendices

NEW YORK SITE VISIT SUMMARY

The research team conducted site visits with New York representatives involved in AoA-funded 

CDSMP workshops. The interviews took place on Monday and Tuesday, January 23 and 24, 2012. 

The first meeting of Day 1 was with Quality and Technical Assistance Center (QTAC) staff to discuss 

state infrastructure. QTAC provides all of the technical assistance for the State of New York, including 

program coordination, training, and data management. The second meeting of Day 1 was a focus 

group with master trainers throughout the state. The final session was a site visit to a senior housing 

facility to conduct focus groups with lay leaders and program participants.

Day 2 started with a focus group with host and implementation site partners, including the 

Department for the Aging, the University of Albany’s Center for Excellence in Aging and Community 

Wellness, a community nursing program, a retirement community, and a naturally occurring 

retirement community (NORC). The final Day 2 session consisted of a discussion with all officials from 

the public health and aging departments that have been involved with CDSMP in New York. The AoA 

state grantee is the New York Department of Aging, which provides high-level oversight to the 

program and grant management. The below table provides an overview of the interviews from the 

New York Site Visit.

New York: 

Overview of 

Visit

New York: Overview of 

Visit - Interviews

New York: Overview of 

Visit - Facility

New York: 

Overview of 

Visit - Location

Day 1, Session 1

Quality and Technical 

Assistance Center (QTAC) 

staff on state infrastructure

University of Albany (TA 

contractor) Albany, NY

Day 1, Session 2 Focus group with master 

trainers 

Associated with: Center 

for Excellence in Aging 

and Community 

Wellness and the P2 

collaborative.

All over New 

York state

Day 1, Session 3 Focus group with lay 

leaders

Senior Housing Facility 

(Implementation Site) Albany, NY

Day 1, Session 3 Focus group with 

participants

Senior Housing Facility 

(Implementation Site) Albany, NY

Day 1, Session 4

Quality and Technical 

Assistance Center (QTAC) 

staff on data 

collection/reporting

University of Albany Albany, NY

Day 2, Session 1

Focus group with host and 

implementation site 

partners

Department for the 

Aging (Host Site)

New York City, 

NY
IMPAQ International, LLC Page C-19 CDSMP Process Evaluation Final Report 

Appendices

New York: 

Overview of 

Visit

New York: Overview of 

Visit - Interviews

New York: Overview of 

Visit - Facility

New York: 

Overview of 

Visit - Location

Day 2, Session 1

Focus group with host and 

implementation site 

partners

Center for Excellence in 

Aging and Community 

Wellness 

(Implementation Site)

Implementation 

sites all over 

New York

Day 2, Session 1

Focus group with host and 

implementation site 

partners

Community Nursing 

Program 

(Implementation Site)

Albany, NY

Day 2, Session 1

Focus group with host and 

implementation site 

partners

Neighborhood NORC 

Program 

(Implementation Site)

Albany, NY

Day 2, Session 1

Focus group with host and 

implementation site 

partners

Retirement Community 

(Implementation Site) Albany, NY

Day 2, Session 2 Focus group with state 

aging and public health staff

State Aging and State 

Public Health (State 

Grantee)

Albany, NY

1. CDSMP Populations, Marketing and Recruitment, Eligibility Screening

New York does not target any group, but serves a diverse group of participants, including 21.8 

percent African Americans and 16.4 percent Latinos. New York has been able to expand from three 

regions covered as part of their 2006 grantee status, to six regions. Roughly 50 percent of New York 

counties are currently covered in these six regions that include more than half the state’s population. 

New York provides CDSMP in a range of rural and urban settings. Master trainers conduct Class Zeros 

at new implementation sites. Implementation sites and lay leaders are primarily responsible for 

recruitment. 

2. Site Implementation

Host sites serve as regional coordinators. Implementation sites in New York often perform many of 

the functions traditionally assumed by host sites, such as marketing and recruitment, enrollment, 

and sometimes monitoring workshop fidelity. New York uses six AAAs as host sites, and has 

hundreds of implementation sites in the state. More so than other large states, New York tends to 

hold only one workshop per implementation site and moves throughout the community to reach as 

many target populations as possible. Despite a centralized technical assistance provider, 

implementation at the local level is decentralized and varies tremendously.
IMPAQ International, LLC Page C-20 CDSMP Process Evaluation Final Report 

Appendices

3. CDSMP Eligibility, Enrollment, Completion

According to a 2011 significant findings and evaluation report, 3,777 participants have completed 

data forms for CDSMP workshops. Of those participants, 82 percent were course completers, higher 

than the national average. The age of participants ranges from 17 to 100, with a mean age of 70, and 

much younger persons resulting from health plans. Almost 70 percent of participants currently live 

alone. There is no eligibility screening conducted.

4. Data Collection

Data are collected by the lay leaders and are typically mailed back to the QTAC for review and data 

entry/management. Upon receipt, data are reviewed to ensure all fields are complete and no issues 

have been identified. If there is a questionable form, QTAC leadership reviews and handles it, if 

needed. The QTAC staff is responsible for all data collection efforts. When a workshop is scheduled, 

QTAC staff assembles the data packets, including copies for all participants, and send the packets to 

the local program coordinator. Outstanding data packets are a source of quality assurance; center 

staff follow-up to ensure all data packets have been returned in a timely manner.

In addition to the data collected for the NCOA database, participants also receive a participant 

satisfaction survey. QTAC is engaged in additional data collection efforts, including an outcome 

study, using instruments similar to those used by Centers for Medicare and Medicaid Services (CMS) 

1,000-person pre-post study; a study with HMO organizations on healthcare expenditures compared 

to administrative matched sample; and a study through the Department of Public Health, the P2

collaborative12, where an addendum survey is collected.

5. Statewide Distribution and Delivery

QTAC supports many of the functions of a statewide distribution system by funding all training 

workshops for master trainers and lay leaders; providing all course materials; data collection, entry, 

and analysis; and providing technical assistance to sites. 

Covering 50 percent of New York counties and more than half of the state population, New York is 

performing on par with other states in geographic coverage. QTAC ensures they have adequate 

master trainers and lay leaders in the six regions, though this may sometimes mean that a leader or 

trainer works in two regions, or across multiple counties. QTAC designed and proposed the six 

regions to be covered under AoA funds, based on previous relationships, known networks, 

population size, etc. 

 12 The P2 Collaborative of Western New York is the planning arm of the Western New York Community Health 

Partners Institute. They are a nonprofit initiative dedicated to improving the health of their community with eight 

counties of New York (Allegany, Cattaraugus, Chautauqua, Erie, Genesee, Niagara, Orleans, and Wyoming County).
IMPAQ International, LLC Page C-21 CDSMP Process Evaluation Final Report 

Appendices

TENNESSEE SITE VISIT SUMMARY

The research team conducted site visits with Tennessee representatives involved in the AoA-funded 

CDSMP workshops. The interviews took place on Thursday and Friday, January 19 and 20, 2012. The 

first meeting of Day 1 consisted of three State Commission on Aging and Disability officials. The 

second meeting of Day 1 was a site visit of a CDSMP host site and meeting with an AAA director, the 

local program coordinator, who is also master trainer. The third meeting of Day 1 was a small group 

session with lay leaders and a master trainer from the AAA office.

The Day 2 meeting took place with a second AAA, meeting with the AAA director, the CDSMP 

coordinator, and two lay leaders. The AAAs did not invite participants to our meetings, as they felt 

uncomfortable doing so. 

The Tennessee Commission on Aging and Disability (TCAD) oversees the grant and program 

management of five AAAs. The AAAs have attempted to partner with the University of Tennessee 

Extension, which boasts a considerable number of health educators, many trained in CDSMP. 

However, this partnership has not worked as initially hoped due to an unexpected shortage of 

educators and communication and coordination problems. The below table provides an overview of 

the interviews from the Tennessee Site Visit.

Tennessee: 

Overview of 

Visit

Tennessee: Overview of Visit - 

Interviews

Tennessee: 

Overview of Visit - 

Facility

Tennessee: 

Overview of 

Visit - Location

Day 1, Session 1

Met with assistant director of 

commission, program 

coordinator, and grants 

manager

Aging and Disability 

Commission (State 

Grantee)

Nashville, TN

Day 1, Session 2

Met with AAA director and 

assistant director, local program 

coordinator and master trainer

Area Agency on Aging 

(Host Site) Chattanooga, TN

Day 1, Session 3

Met with lay leaders, master 

trainer, and program 

coordinator

Area Agency on Aging 

(Host Site) Chattanooga, TN

Day 2, Session 1 Met with AAA director, program 

coordinator, and lay leaders

Area Agency on Aging 

(Host Site) Knoxville, TN

1. CDSMP Populations, Marketing & Recruitment, Eligibility Screening

Living Well with Chronic Conditions is the brand name for CDSMP in Tennessee. The state serves a 

largely rural and low income population, with some racial and ethnic diversity. The program is 

reportedly open to all ages, though advertising materials say “55+”. AAA leaders indicate generally

enrollment is 60+ and largely female. There is some increasing focus on low income and minority 

populations.
IMPAQ International, LLC Page C-22 CDSMP Process Evaluation Final Report 

Appendices

Marketing and recruitment is generally performed by the AAA CDSMP coordinator or the leader. For 

some sites hosting CDSMP, the AAA CDSMP coordinator offers to provide course and marketing 

materials.

The University of Tennessee Extension partnership aimed to enable classes to be held in every 

county in the state, but this has not performed as well as they hoped, and to date, Tennessee only 

has 27 implementation sites. A great deal of decentralization exists, with much variation at the AAAregional level in implementation, recruitment, marketing, etc. Currently, little targeting of providers 

exists, due to difficulty accessing them. Focus is on direct participant recruitment rather than 

referrals.

2. Site Implementation

The five regional AAA CDSMP coordinators generally serve as the host sites. Implementation sites 

may be identified by these coordinators or, in some regions, by CDSMP advocates. We noted great 

flexibility and variation in marketing, site selection, and participant recruitment in the regions. To 

date, Tennessee has had 27 implementation sites. Site selection/identification is a challenge. Some 

AAAs reuse the same site, but indicate that they experience problems with recruitment/enrollment 

with subsequent sessions.

Obtaining sufficient numbers of participants is often a challenge. One leader stated that she had 

advertised a CDSMP at her church and had only two members show up for the class, the rest came 

from the community. Some leaders indicate that hospitals and rehabilitation facilities do not attract 

participants as well as other locations. Some AAAs offer Class Zeros more frequently than others. 

Southeast Tennessee Area Agency on Aging and Disability (SETAAAD), for example, will conduct a 

Class Zero when they have an implementation location identified--one per location. If a class date 

has not been set, the master trainer will take names of those interested and contact them via phone 

with class details. 

There are no participation fees. Some AAAs do offer incentives for attendance and/or completion, 

such as door prizes and other small gifts. The AAAs spend $20 to $30 on snacks per meeting, 

emphasizing the importance of healthy meals.

We noted only the most basic monitoring of fidelity at any level. Classes may begin with less than 10 

people, although they try to have at least 5. Classes do not force participants to do the buddy phone 

calls. For classes with small numbers of participants, the class will often be finished before the 2.5 

hours.

AAAs indicate concern about scaling back once funding ends; the state and AAA CDSMP coordinators 

will likely be moved to other programs, but remaining part-time on CDSMP. Some of the regional 

AAAs are hopeful of attracting corporate sponsors and there has been some talk of TennCare 

reimbursements, Affordable Care Act grants, or health insurance companies’ participation.
IMPAQ International, LLC Page C-23 CDSMP Process Evaluation Final Report 

Appendices

3. CDSMP Eligibility, Enrollment, Completion

Recruitment is often a challenge. Some leaders say that getting people to commit to the 6 weeks and 

2.5 hour classes is hard. Recruiting men is a challenge; it does seem that having a male leader may 

help. Use of senior centers is good for recruitment, but there is the problem of participants 

wandering in and out of class and not being consistent with attendance. Consistent attendance can 

be a challenge for some sites. Weather may impact attendance or ability of a site to offer a class. 

Some leaders provide transportation if it is requested, on an as-needed basis.

There is no screening for participation. The state has provided money for incentives for completion 

and “nice” snacks, though some AAAs are not aware of this. 

Participants contact the leader for the most part. Sometimes, the AAA staff helps with the 

enrollment process, and sometimes the implementation site will register participants. Phone calls 

are the primary means of registration/enrollment.

4. Data Collection

Tennessee only collects data required for submission to NCOA. Usually the AAA’s CDSMP 

coordinators gather the data for their region and ensure it is completed and sent to the state lead 

who enters it into the database. The coordinators have the Social Assistance Management System 

(SAMS) database as well as the NCOA, but no other systems. There is one state-level part-time data 

entry person. Most of the AAAs have a full time coordinator who will transition to support other 

programs once AoA funding ends.

5. Statewide Distribution and Delivery

There are five regional coordinators within the AAAs for CDSMP, and one state-level program 

coordinator. They have five master trainers in the state, but leverage some trainers and leaders from 

the University of Tennessee Extension. All recruitment is done at the local (AAA) level. Generally, the 

AAA coordinator recruits, markets, and holds Class Zeros. There may be some marketing support 

provided by the implementation site or leaders. Some AAAs rely heavily on a local advocate for site 

identification, participant recruitment, Class Zeros etc. 

The state CDSMP coordinator collects the data as received from the AAAs and ensures it is entered 

into the NCOA database. Tennessee recently experienced a change in the state-level program 

coordinator. The previous coordinator had begun to establish networks and regular communications 

with the AAAs. The new coordinator is still “learning the ropes” and regular communications have 

dropped, though she is available via phone or email to answer questions. Networking has also 

suffered during the transition.

The AAAs are concerned about their ability to continue to offer the same number of programs after 

funding ends and are each seeking other options. Each AAA seems to have its CDSMP champion 
IMPAQ International, LLC Page C-24 CDSMP Process Evaluation Final Report 

Appendices

taking the initiative to try to find other funding. While not institutionalized, the presence of AAA 

champions and interest in seeing the program continue is promising. 
IMPAQ International, LLC Page C-25 CDSMP Process Evaluation Final Report 

Appendices

VERMONT SITE VISIT SUMMARY

The research team conducted site visits with Vermont representatives involved in the AoA-funded 

CDSMP workshops. The interviews took place on Monday and Tuesday, January 30 and 31, 2012. 

During Day 1 of the site visit, the team met with officials from the Vermont Department of Health 

(VDH) and Department of Disability, Aging and Independent Living (DAIL), including several master 

trainers and one T-trainer on staff. The team also met with officials from the state health plan, 

Blueprint. During Day 1, the team completed a site visit at a private practice that serves as a host and 

implementation site. The team met with a master trainer and several lay leaders. The team also 

briefly observed a workshop and met with two participants. 

On Day 2, the team completed a site visit to a local community hospital that serves as a host site. The 

team met with local coordinators of the site, who are also lay leaders and workshop participants. A 

second site visit took place at a local community wellness center that serves as a host and 

implementation site. The team also met with several master trainers and lay leaders at this site. The 

below table provides an overview of the interviews from the Vermont site visit.

Vermont: 

Overview of 

Visit

Vermont: Overview of Visit - 

Interviews

Vermont: Overview 

of Visit - Facility

Vermont: 

Overview of Visit - 

Location

Day 1, Session 1

Blueprint and CDSMP state 

infrastructure staff (led by 

Public Health Department with 

an Aging Department 

representative)

Public Health 

Department Staff 

(State Grantee)

Williston, VT

Day 1, Session 2

Focus Group with St Albans site 

staff (including hospital, AAA, 

Veterans Department of 

Health)

Medical Center - 

Family Medicine 

Practice (Host and 

Implementation Site)

St Albans, VT

Day 1, Session 2 Focus group with lay leaders 

and program participants

Medical Center - 

Family Medicine 

Practice (Host and 

Implementation Site)

St Albans, VT

Day 2, Session 1 Focus Group with St. Johnsbury 

staff (host site)

Regional Hospital 

(Host Site) St. Johnsbury, VT

Day 2, Session 1 Focus group with lay leaders Regional Hospital 

(Host Site)

St. Johnsbury, VT

Day 2, Session 1 Focus group with program 

participants

Regional Hospital 

(Host Site)

St. Johnsbury, VT

Day 2, Session 2 Focus group with host site staff 

and master trainers

Wellness Center (Host 

and Implementation 

Site)

Copley, VT
IMPAQ International, LLC Page C-26 CDSMP Process Evaluation Final Report 

Appendices

1. CDSMP Populations, Marketing & Recruitment, Eligibility Screening

CDSMP, branded Healthier Living, has been offered in Vermont since 2003, with workshop data 

available since 2004. In a recently released report from the Vermont Department of Health, 2,073 

participants have completed the workshop and answered baseline questionnaires. The program 

coordinator believes that a Healthier Living workshop has been offered in nearly every county in the 

state. While completion rates have dropped in the last year (no one has been able to pinpoint why), 

the gender of participants in Vermont appear similar to the rest of the nation, with 77 percent 

female and an average age of 61.

All three host sites used panel management to target CDSMP participants of interest. Panel 

management is a technique that identifies individuals with particular characteristics (i.e., a certain 

chronic condition, low income, etc.) for referral to a CDSMP workshop. This technique has proven 

particularly effective in identifying individuals for targeted recruitment, although Vermont is in a 

unique position to do so because of the Blueprint for Health Care Reform, which provides a global 

statewide data infrastructure. One of the three host sites visited had documented and implemented 

a well thought out communication and marketing plan. A regional hospital host site, for example, 

had created a chart of stakeholder groups (i.e. public, employees, medical staff, corporate/trustees) 

to understand their points of view and the messages or tactics that would be most effective in 

reaching that stakeholder group. The same host site also developed a marketing plan through the 

marketing division of the hospital that includes advertising through a variety of mediums and 

presentations to community groups.

2. Site Implementation

The Vermont Blueprint for Health Care Reform funds the cost of the workshop, the trainings, the 

materials, and even a lay leader stipend, and estimates the per participant cost at $350. Vermont 

uses a global 1115 Medicaid waiver to fund evidence-based programs and provides reimbursement 

to sites for enrolling 10 participants and for each completer. Sites are not permitted to schedule a 

workshop with less than 10 participants enrolled. Host sites in Vermont are based around the 

patient-centered medical home model, with each community accessing similar hospitals, AAAs, and 

wellness centers.

Vermont has been able to implement CDSMP in rural settings with considerable success. When 

asked why this was possible, officials stated that Vermont’s rural culture and community health 

systems are heavily relationship-based, with a history of collaboration and cooperation.

3. CDSMP Eligibility, Enrollment, Completion

There are no eligibility requirements for CDSMP, although one lay leader reported that individuals 

who are uncomfortable with social settings and individuals with several mental health impairments, 

such as schizophrenia, are not good candidates for this workshop. The lay leaders and program 

coordinators believe that nearly everyone could benefit from the program, no matter their chronic 
IMPAQ International, LLC Page C-27 CDSMP Process Evaluation Final Report 

Appendices

condition or personality. Fidelity to the Stanford Model is a high priority. Master trainer observations 

of lay leaders are frequent.

4. Data Collection

The Vermont Blueprint for Health Care Reform is data-driven and captures data through electronic 

medical record templates, interfaces, and health information exchange; Blueprint registries; chart 

review; and other means. Evaluation, reporting, and analytics that address process and outcome 

measures, models, and knowledge are part of the Blueprint’s data activities. Data analysis for the 

Blueprint also uses multi-payer claims, NCQA scoring, and patient experience measurement.

Data on Healthier Living workshops are submitted to the state’s Blueprint evaluator, who conducts 

various population-based analyses by community health region. These data can be combined with 

data from other sources to provide a full picture of the workshops and their impact. The potential for

in-depth analyses of CDSMP and its impact is considerable as a result. Blueprint interviewees 

envision scenarios where workshop participants can enter their Action Plans through a patient portal 

to share with their primary care practitioners. Workshop leaders at some sites actively encourage 

participants to share their Action Plans with their providers.

5. Statewide Distribution and Delivery

Healthier Living workshops are embedded as an integral part of the Vermont Blueprint for Health 

Care Reform, described as a statewide “Learning Health System.” The Blueprint anchors health care 

delivery (including CDSMP and other wellness programs) around the state’s hospitals, all of which 

are non-profit and together comprise a statewide network. Community Health Teams serve patient 

panels of approximately 20,000. Teams in each region include a care coordinator (17 statewide) for 

intensive care management, social workers, nutrition specialists, community health workers, 

Medicaid care coordinators, and a public health specialist. Healthier Living workshops are included as 

part of the Community Health Teams available to the patients in the regional panels. Each region has 

a local community health coalition consisting of volunteer members and a half time paid project 

manager who oversees the Community Health Team and a budget of $350,000, provided by 

insurance companies in the state. The community health coalitions decide how to allocate funds. 

The state’s goal is to have every Vermont resident in a patient-centered medical home. Vermont also 

has a global Medicaid 1115 waiver, which has full flexibility across the Department of Health and 

Human Services and fits within the framework of the Blueprint. This robust, hospital-based 

infrastructure provides a solid, sustainable, statewide network for distribution and delivery of 

CDSMP.
IMPAQ International, LLC Page C-28 CDSMP Process Evaluation Final Report

Appendices

Table 1: Site Visit Summary Information for All States

Site Visit Overview Site Visit Overview

Arkansas January 17-18, 2012. Day 1: Met with lead agency staff from Arkansas Department of Health (ADH) and representative from 

Division on Aging and Adult Services (DAAS) in the Department of Human Services; 1 master trainer, 2 lay leaders, and 5 

participants at Rogers Adult Wellness Center, Rogers, AR. Day 2: Met with 2 lay leaders, 1 lay leader, and 6 participants at 

Jacksonville Senior Wellness & Activity Center, Jacksonville, AR; met with 2 lay leaders and 1 lay leader at Oaklawn Center on 

Aging, Hot Springs, AR.

California February 13-14, 2012. Day 1: Met with State Aging Department officials and Public Health Department officials. Meeting also 

included technical assistance contractor for state. Visited developing rural site and met with AAA director, local program 

coordinator who is also lay leader. Met with 4 lay leaders and 3 participants. Day 2: Met with local city and county health 

collaborators for program who serve as host site. Met with school district that adopted training program for adult education.

Visited mature urban host site (senior center) that also serves as implementation site. Met with city healthcare district. Met with 

local program coordinator of senior who also serves as lay leader. Met with 9 participants. Had conversations with TA contractor 

representatives (2 trained lay leaders) on fidelity and data collection/reporting systems.

Kansas January 9-11, 2012. Day 1: In Topeka, met with staff from Kansas Department of Aging (KDoA), lead agency for AoA/ARRA grant, 

and Kansas Department of Health and the Environment (KDHE), lead agency for CDC CDSMP grant and historically the lead for 

CDSMP. In Wichita, met with a lay leader and a workshop participant who also coordinates delivery of CDSMP at a senior center; 

met with 7 lay leaders employed by a hospital, the AAA, local clinics, and a senior housing complex. Day 2: In Newton, met with 

a master trainer and lay leader; met with 3 participants. In Hutchinson, met with 7 lay leaders; met with 7 participants. Day 3: In 

Wichita, met with AAA director, 1 master trainer, 3 lay leaders, and 2 representatives from Wichita State University.

New York Day 1: Met with statewide technical assistance contractors. Met with 5 master trainers who serve in 4 regions throughout state. 

Site visit at senior housing implementation site. Met with 6 lay leaders and 3 participants. Day 2: Met with lay leaders from 4 

rural implementation sites (church, senior housing, parish nurse association/Catholic church), and spoke with 2 officials from 

urban host site (on-phone, representing 80 senior centers). Met with 3 state public health representatives and 3 aging state 

representatives. Debrief with technical assistance contractors.

Tennessee Day 1: Met with Tennessee Commission on Aging and Disability (TCAD) CDSMP leadership in Nashville, including the program 

coordinator and grants manager. In the afternoon, met with Southeast Tennessee Area Agency on Aging and Disability (ETAAAD) 

in Chattanooga. Staff included the CDSMP coordinator, lay leaders, master trainers, and AAA oversight personnel. Day 2: Met 

with East Tennessee Area Agency on Aging and Disability (ETAAAD), including CDSMP coordinator and lay leaders

Vermont Day 1: Met with officials from public health and aging departments, including master trainers and one T-trainer. Met with 

officials from state health plan. Site visit at private practice that serves as host and implementation site. Met with master trainer 

and several lay leaders. Briefly observed workshop and met with 2 participants. Day 2: Site visit to local community hospital that 

serves as host site. Met with local coordinators of site, who are also lay leaders. Met with 2 participants. Site visit to local 

community health center that serves as host and implementation site. Met with several master trainers and lay leaders.
IMPAQ International, LLC Page C-29 CDSMP Process Evaluation Final Report

Appendices

Domain 1: CDSMP Populations, 

Marketing & Recruitment

Domain 1: CDSMP Populations, Marketing & Recruitment

History History

Arkansas The brand name for Arkansas’ program is Be Well, Live Well. The lead agency is the health agency (ADOH). The state began 

developing evidence-based programs about 5 to 6 years ago in 2 counties when they received an AoA Empowerment grant. 

Start-up was delayed under the ARRA grant, although site development seemed to gain momentum in the latter half of 2011. In 

addition to an 8-region network of AAAs, the state has a 6-region network of Centers on Aging established by the University of 

Arkansas Medical Sciences (USMS) Arkansas Aging Initiative (AAI) and funded with tobacco settlement funds. Both the AAAs and 

AAI operate CDSMP, sometimes collaboratively. The role of the AAI Centers of Aging is education (not services); many at the 

state and in the AAAs feel that diverting tobacco settlement funds to set up the Centers on Aging (instead of using the funds for 

the AAAs) was not a wise use of public resources.

California California is one of the original CDSMP 2006 Grantees. As a result of Stanford and their development of the CDSMP workshop 

and Kaiser Permanente who offer CDSMP as a plan benefit, classes have been offered in state since the 1990s.

Kansas The brand name for Kansas’ program is Kansans Optimizing Health Program (KOHP). KDHE initially began building a delivery 

system in 2005 when the state was 1 of 2 states to receive CDC funding for an arthritis program. In 2009, the state received a 

small “opportunity grant” from the National Association of Chronic Disease Director to investigate Medicaid and/or private 

insurance funding for EBPs. KDoA became involved upon receiving the AoA ARRA grant for CDSMP in 2010. The AAAs are integral 

to the delivery system, as is Kansas State University’s Research & Extension Service, a long-standing partner that offers 

workshops through its county offices and trains its own leaders. Other partners include 2 hospital systems, the Veterans 

Administration, and Wichita State University. Because of a slow start-up with the ARRA grant, the state has requested a 6-month, 

no-cost extension to the grant from AoA (through September 2012).

New York New York was a part of original CDSMP 2006 Grantees. QTAC staff member received grant in 2003 to provide CDSMP workshop.

Tennessee This was a new effort for Tennessee, though the University of Tennessee Extension had been training leaders to conduct the 

program. Six different contracts were written to cover the 5 AAA’s and the University of Tennessee Extension.

Vermont Vermont has offered CDSMP since 2003. Received ARRA Grant to include collaborations with AAAs.

Population/Participant 

Characteristics (Race/ethnicity, SES, 

age, other demographics)

Population/Participant Characteristics (Race/ethnicity, SES, age, other demographics)

Arkansas The state targets all adults age 60 and above.

California California serves a diverse population but this varies by region.

Kansas The state targets individuals aged 60 and older, but serves younger people as well. 29.6% were under age as of January 4, 2012. 

The local sites reported serving some minorities and a number of low-income participants, especially in rural areas. One of the 

host sites is the Veterans Administration, which markets the program only to VA patients, with the advantage that if a VA 

provider tells a veteran to attend CDSMP, typically the veteran will do so.

New York Program reaches participants in a variety of urban and rural settings. In New York City, CDSMP is offered to diverse audiences, 

including African Americans, Hindu, Korean, and Latinos.

Tennessee The program is open to all ages and race/ethnicity.
IMPAQ International, LLC Page C-30 CDSMP Process Evaluation Final Report

Appendices

Vermont Vermont reaches participants in a variety of urban and rural settings.

Underserved populations? Why? Underserved populations? Why?

Arkansas Most CDSMP participants come from senior centers and have been Caucasians and well educated; African Americans have been 

less likely to join workshops. State representatives noted that those who need it most are the less educated. There are some 

Southeast Asians and Latino populations, but few if any have participated. There are no tribal groups in AR.

California More difficult to provide CDSMP in rural areas of California. At this time, CDSMP is not statewide and many rural areas are not 

covered. It is only possible to offer CDSMP in other languages (Tomando, Chinese, Korean speaking versions) when there are 

available lay leaders for the class. It is sometimes difficult to find bilingual instructors.

Kansas There are areas in Wichita, Kansas City, and southwest Kansas with large Latino populations; the state has recently trained 2 

master trainers and 12 leaders in Tomando. Tomando participants are typically females in the “middle” generation caring for 

children and parents.

New York No known underserved populations.

Tennessee There are 5 regional AAA CDSMP coordinators who generally serve as the host sites. Implementation sites may be identified by 

these coordinators or in some regions by CDSMP advocates. There seems to be great flexibility and variation in marketing, site 

selection, and participant recruitment in the regions. To date Tennessee has had 27 implementation sites. There is some 

increasing focus on low income and minority populations.

Vermont Vermont has near statewide coverage, even in rural communities.

How do participants learn of 

classes?

How do participantslearn of classes?

Arkansas Most marketing is done locally by host and implementation sites through newspapers, newsletters, and flyers at senior centers. 

The state has a trainers’ Website that includes a blog, flyers, and promotional materials that can be downloaded and used by 

host and implementation sites. DAAS may include information on its Website, but it is not prominent.

California Marketing is largely localized but TA contractor provides support depending on the "package" of TA services purchased by the 

host site.

Kansas Most participants learn about KOPH through flyers and brochures distributed through local host and implementation sites and 

newspaper/newsletter advertisements. There is no 800 number; prospective participants must call a local AAA, department on 

aging, or Kansas-State Research & Extension county office. Lay leaders receive these calls (they have responsibility for enrolling 

participants) and play an important role in convincing individuals to enroll.

New York Classes are marketing through a variety of mechanisms, depending on the target audience. QTAC provides template materials 

and works with states.

Tennessee Flyers, newsletters, Class Zero, Facebook. One of the AAAs has CDSMP information and a short video on their Website. Free 

radio/TV ads

Vermont Each host site has a marketing strategy. Some learn of the program through panel management where patients receive a referral

as a result of their chronic condition.

Competing programs? Competing programs?
IMPAQ International, LLC Page C-31 CDSMP Process Evaluation Final Report

Appendices

Arkansas Diabetes Self-Management Program (DSMP): The state received CDC funding for this program and has 2 master trainers. A 

Matter of Balance (AMOB): Has been offered for about 2 years and is popular. Active Living Every Day (ALED): Reduced from 20 

weeks to 12 weeks, but the state still had difficulty recruiting participants so the program was discontinued as of May 2011.

California Other programs mentioned were Arthritis Tai Chi, Falls Prevention, Tomando, and locally developed program.

Kansas Kansas Arthritis Program sponsored by the Arthritis Foundation; Diabetes Self-Management Program.

New York Diabetes Self-Management Program; A Matter of Balance; Active Generations; Active Choices; Active Living Every Day; Health 

Eating Every Day; Arthritis Foundation Exercise Program

Tennessee Varies by AAA: MOB, Arthritis Exercise, Dining with Diabetes, Tai Chi. The programs are being trained to deliver DSMP. One AAA 

is looking for funding through the Affordable Care Act (ACA) to deliver Care Transitions.

Vermont Diabetes Self-Management Program; A Matter of Balance; Chronic Pain Self-Management

Marketing/Recruitment: Do you 

target marketing (participants, 

clinicians, other)? Why and how?

Marketing/Recruitment: Do you target marketing (participants, clinicians, other)? Why and how?

Arkansas Two main delivery channels: AAI (6 regional Centers on Aging) and Arkansas AAAs (8 regions). Both networks have senior centers 

that they use as host sites. AAA/AAI service areas overlap.

Most marketing is performed at the local level by host and implementation sites and leaders. There is no statewide Website 

advertising CDSMP to consumers. The DAAS ADRC, Choices in Living, has a Website with an 800 number and has access to 

CDSMP calendars for referrals, but the state noted they have not done as good a job as they could have with marketing through 

this vehicle. The state just printed prescription pads advertising CDSMP to distribute to providers. 

The state (ADOH, DAAS) promotes CDSMP to faith communities; at Medicare Improvements for Patients and Providers Act 

(MIPPA) health fairs promoting Medicare plans; and with the State Health Insurance Assistance Program (SHIP), a federally 

funded program that provides counseling to Medicare beneficiaries on health insurance. To date there has been no outreach to 

private health plans.

California PICF provides varying levels of TA, which sometimes include assistance in marketing and recruitment efforts. Market and 

recruitment varies tremendously and is very localized. One targeted recruitment effort was in a hospital emergency room setting 

where repeated emergency room clients were targeted. In this small study, CDSMP was found effective in reducing repeat 

emergency room clients.

Kansas Most marketing is performed locally by host and implementation sites through newspapers, newsletters, and flyers distributed 

through senior centers, county departments on aging, and Kansas-State Research & Extension. The state hosts a KOPH Website 

(http://www.kdheks.gov/arthritis/kohp.htm) that lists upcoming workshops. Prospective participants must call the local sites to 

register. The Kansas State Research & Extension Service’s Website and the service’s individual county Websites have information 

on KOPH, but accessibility and content varies.

The Quality of Care clinics (community health centers) are experimenting with flagging patients with chronic conditions in the 

patient’s chart so that they can be referred to KOPH. 

New York Marketing through local host or implementation sites and through health plans in some locations. Sending targeted letters to 

health plan participants has proven an effective method for recruitment.
IMPAQ International, LLC Page C-32 CDSMP Process Evaluation Final Report

Appendices

Tennessee The University of Tennessee Extension partnership aimed to enable classes to be held in every county, but this has not 

performed as well as they hoped and to date have only had 27 implementation sites. There is a great deal of decentralization, 

with much variation at the AAA-regional level in implementation, recruitment, marketing etc. Little targeting of primary care 

physicians, due to difficulty accessing them. Focus is direct participant recruitment.

Vermont Marketing through local host or implementation sites and through panel management. Sending targeted letters to health plan 

participants has proven an effective method for recruitment.

Identify successful approaches or 

barriers for specific populations

Identify successful approaches or barriers for specific populations

Arkansas None reported.

California Difficult to find bilingual lay leaders in other languages. Budget crisis in California and delay in approving budget delayed program 

funds by 1 year. Budget is normally approved in July and was not approved until October. It was necessary to chose sites that 

could get started before funds were available.

Kansas Face-to-face contact with participants and referral sources improves recruitment, as does persistence. This may make marketing 

and recruitment less successful for lay leaders who may have limited time and effort to devote to this.

New York QTAC feels there is a lack of strong and effective marketing materials for the program.

Tennessee Use of CDSMP ‘advocates’ is helpful in recruitment efforts. These are not necessarily the leaders, but may be an organization 

director, or other stakeholder interested in offering the program to his/her target population.

Vermont For individuals who are not good candidates for CDSMP workshops, lay leaders or master trainers may go to their home for a one 

and one session.

Domain 2: Site Implementation Domain 2: Site Implementation

Implementation challenges & 

opportunities

Implementation challenges & opportunities

Arkansas The ARRA CDSMP grant program is a partnership involving ADOH (grantee) and DAAS. The program at first floundered 

under ADOH leadership, with two directors of short duration before a 0.5 FTE coordinator was hired after 1 year and a 

director was hired 3 months later. This delayed host site startups. Implementation and enrollment have been more 

challenging in rural areas due to low literacy and limited understanding of CDSMP, as well as transportation and sparse 

population density.

California CDSMP is very difficult to deliver in rural areas of California. Monitoring fidelity is through class observations is inconsistent. 

Incoming data is monitored for fidelity but PICF does not have influence on sites reporting data outside of ARRA funding.

Kansas Because of a number of staff reductions and retirements in KDoA and KDHE, startup under the ARRA grant was delayed. 

More staff changes are anticipated, especially as the state budget crisis continues and the state moves to a Medicaidmanaged care program and enacts health reform. 

AAA participation in CDSMP has progressed slowly. Multiple-agency involvement in the program can be challenging to the 

program manager at the state level, with different approval processes for different agencies.
IMPAQ International, LLC Page C-33 CDSMP Process Evaluation Final Report

Appendices

New York Despite high expectations for some clinical or religious settings, sites have not been successful in recruitment participants. In 

one religious setting, a leader advocated for the workshop and only one person signed up. There is a reluctance to sign up 

for new programs.

Tennessee Site selection/identification a problem. Some AAAs re-use the same site, but indicate that they experience problems with 

recruitment/enrollment.

Vermont Even among sites resistant to participate, use of panel management have permitted sites to be successful in recruitment 

and enrollment.

Fees, scholarships? Fees, scholarships?

Arkansas There is no fee charged to participants for CDSMP. The ARRA grant pays for books and some sites have lending libraries. 

Sites said they may consider charging a fee to cover the cost of books when grant funds are no longer available.

California No fees. Incentives offered at one location through wellness program, including spa certificates.

Kansas Local sites may choose to charge a fee for the workshop, but most do not. The state is considering requiring a fee of $25 to 

cover the cost of the books; local sites are generally not in favor of this because they believe a fee would keep people from

enrolling.

New York No fees charged at this point. QTAC is examining whether charging for the course textbook will influence participation.

Tennessee No fees. Some AAAs offer incentives for attendance and/or completion, such as door prizes and other small gifts. They 

spend $20 to 30$ on snacks per meeting, emphasizing the importance of healthy meals. 

Vermont No cost to participants

Fidelity challenges, opportunities, 

changes to protocol?

Fidelity challenges, opportunities, changes to protocol?

Arkansas ADOH and DAAS reported that they request a minimum of 8 people and a maximum of 15 people in a class to ensure fidelity 

to the Stanford model. ADOH and DAAS perform some site visits to observe classes and they monitor attendance forms. 

Both the state and leaders said that the program is so well scripted; it is easy to follow and maintain fidelity. A challenge is 

maintaining the certification of master trainers and leaders, given low population density and travel distances for workshops

and trainings. Changes to CDSMP protocol: One site allowed participants to keep the same buddy throughout the 6-week 

workshop and there was one instance of a workshop being compressed into 3 weeks. One site would like to be able to use 

PowerPointinstead of flip charts to save time and paper.

California Lay leaders associated with health plans wanted to advertise health plans during workshop breaks (fidelity violation). New 

leaders paired with experience leaders to guard against problems with fidelity. Inconsistent spot checks and class 

observations held by TA contractor. One host site monitored fidelity by managing list of workshop leaders.

Kansas There is no formal process for monitoring fidelity, but KDHE plans to implement a process for monitoring leaders in classes 

in the near future.

New York QTAC developed a fidelity checklist that is considered a best practice. Master trainers frequently participate in observations 

of class to monitor fidelity. Fidelity is also monitored through the participant satisfaction survey given to all participants. If 

participants report information that indicates a problem with fidelity, all problems are investigated by QTAC.
IMPAQ International, LLC Page C-34 CDSMP Process Evaluation Final Report

Appendices

Tennessee There seems to be only the most basic monitoring of fidelity at any level. Classes may begin with less than 10 people, 

although they try to have at least 5. They don't force participants to do the buddy phone calls. For classes with small 

numbers of participants, the class will often be finished before the 2.5 hours. 

Vermont Vermont Department of Health developed their own fidelity monitoring tool and master trainers observe workshops for 

fidelity adherence.

Special accommodations? Special accommodations?

Arkansas Participants with dementia are handled on a case-by-case basis. One such participant didn’tseem to be a good fit, but the 

leaders managed to accommodate her.

California Accommodations for hearing or visual impairment.

Kansas A lay leader read materials to a visually impaired participant.

New York Accommodations are made for individuals with a cognitive or developmental disability. The leader may provide additional 

one-on-one guidance or use a “buddy system”. At one workshop for younger adults with developmental disabilities, each 

person in the workshop had a caregiver to aid the completion of workshop action planning. Accommodations are made for 

visual or auditory impairments.

Tennessee No, none specifically for older adults.

Vermont Same accommodations as New York.

Adaptations for older adults? Adaptations for older adults?

Arkansas Small incentives (e.g., coffee mugs) have been found to help motivate participants to complete CDSMP.

California No adaptations for older adults reported.

Kansas No adaptations for older adults reported.

New York Small incentives (cupcakes and graduation certificates) for older adult participants have been found to receive positive 

feedback.

Tennessee No adaptations for older adults reported.

Vermont No adaptations for older adults reported.

Demand: # classes, sufficient capacity to 

meet demand?

Demand: # classes, sufficient capacity to meet demand?

Arkansas State is seeking to increase enrollees; would like to increase demand as enrollment is limited in some areas and with some 

populations.

California Yes. There are waitlists but this is usually a result of scheduling and individuals who need to delay taking the class.

Kansas As of January 2012, 40 CDSMP classes were scheduled for 2012 across the state.

New York Capacity exceeds demand. There have been wait lists on very rare occasions.

Tennessee Obtaining sufficient numbers of participants is often a challenge. One leader stated that she had advertised a CDSMP at her 

church and had only 2 members show up for the class, the rest came from the community. Some leaders indicate that 

hospitals and rehabilitation facilities do not seem to attract participants as well as other locations.

Vermont Capacity exceeds demand. There have been wait lists on very rare occasions.
IMPAQ International, LLC Page C-35 CDSMP Process Evaluation Final Report

Appendices

Orientation: Sessions? Who conducts 

orientation? Workshops? Type of lay 

leader? Class Zero?

Orientation: Sessions? Who conducts orientation? Workshops? Type of lay leader? Class Zero?

Arkansas No Class Zero or formal orientation.

California Class Zero in some areas. Special Class Zero with "teaser" where a demonstration from the workshop is held and many 

individuals become engaged and given a taste of the workshop dynamic.

Kansas A Class Zero was offered in a high-rise apartment building for seniors on the recommendation of the activities director. No 

others have been offered.

New York Master trainers and lay leaders reported offering Class Zeros and related recruitment sessions as frequently as possible. It 

was estimated that roughly 50% of individuals who attend one of these workshops enrolls in the program.

Tennessee Some AAAs offer Class Zeros more frequently than others. Southeast Tennessee AAA, for example, will conduct a Class Zero 

when they have an implementation location identified, one per location. If a class date has not been set, the master trainer 

will take names of those interested and contact via phone with class details. 

Vermont Orientation or recruitment sessions offered infrequently.

Post-program follow-up? Post-program follow-up?

Arkansas No post follow-up, although some participants continue to get together themselves.

California At senior centers, program graduates continue to socialize and attend other classes. Unclear whether this occurs in other 

sessions.

Kansas No program follow-up.

New York For the outcome evaluation study, participants receive follow-up self-report surveys.

Tennessee Informal follow-up. Often in more rural areas, leaders will ”run into” participants. Participants often say they miss the class, 

would like to have follow-up or attend another class.

Vermont Some program graduatesshare their action plan with their health care providers and continue to work on it after workshop 

has ended.

Financial Sustainability: Challenges, 

Opportunities

Financial Sustainability: Challenges, Opportunities

Arkansas When the ARRA grant ends in March 2012, ADOH and DAAS hope to continue CDSMP. However, the coordinator (0.5 FTE) 

and data analyst (0.5 FTE) employed by ADOH will no longer be funded and will be moving on to other responsibilities or 

jobs.

California Sites plan to continue to offer workshops but one host site reporting the pool of lay leaders has diminished and there will 

not be funds available to conduct another training. Rural areas do not have master trainers available to conduct trainings, 

which is a barrier to sustainability of the program.

Kansas When the ARRA and CDC grants end, the state hopes to continue KOHP, but it is not clear how this will be done given the 

state’s fiscal climate. Local sites are encouraged to raise local funding. For example, the Department of Aging in Reno County 

has received 3-$1,000 grants from a local private senior center that has been used for supplies and snacks.
IMPAQ International, LLC Page C-36 CDSMP Process Evaluation Final Report

Appendices

New York QTAC will continue to provide TA after ARRA funding ends but it will be a streamlined version. For one site with a robust 

partner group of 180 senior centers, they will lose their CDSMP program coordinator after funding ends. There was a belief 

that workshops would continue but with less technical assistance.

Tennessee Indicated concern about scaling back once funding ends. There is concern about scaling back once funding ends. The state 

coordinator will likely be transferred to another program. Some of the regional AAAs are hopeful of attracting corporate 

sponsors and there has been some talk of TennCare reimbursements as a possibility.

Vermont Program is already sustainable.

Cost to provide Cost to provide

Arkansas $300/person based on the AoA cost calculator.

California This information was not available.

Kansas This information was not available.

New York Difficult to calculate, as many staff members perform many functions.

Tennessee This information was not available.

Vermont Cost of program thought to be $350 per person but communities reported it was more. Sites given $1,000 for enrolling 10 

participants and $200 for each completer.

Current funding, financing Current funding, financing

Arkansas Using ARRA funds, ADH pays AAI and AAA sites $300 for each completer, up to a maximum of $15,000 (50 completers). ADH 

also pays each AAA region $2,400 for lay leader trainings.

California CDC, AoA, Community Block Grants, local host site, community resources, and volunteer leader time

Kansas Current CDC grant ends June 30, 2012; the state is applying for a competitive CDC grant as a follow-on. The state hopes to 

receive a 6-month, no-cost extension to their ARRA grant.

New York Sites are very rarely given funds, but QTAC provides all program coordination, training, and workshop materials.

Tennessee AoA is only current funding source for CDSMP.

Vermont Funded through Blueprint/1115 Medicaid Waiver

Strategies to sustain funding, ensure 

financial viability

Strategies to sustain funding, ensure financial viability

Arkansas Host sites training paid staff as master trainers and lay leaders as a way of embedding CDSMP in their operations. DAAS’s 4-

year plan requires one ongoing evidence-based program (EBP) in each AAA region, thus promoting continuation of CDSMP 

and/or other EBPs.

California Seek outside funding or grants. California has leveraged community block grants to provide graduated payment scheme 

over several years to sites, with the end goal being sustainability.
IMPAQ International, LLC Page C-37 CDSMP Process Evaluation Final Report

Appendices

Kansas The state is actively pursuing a system where master trainers and leaders are embedded (i.e., employed) in a local system 

that supports CDSMP. 

The state has sponsored 2 summits to bring together community partners to brainstorm about how to build and sustain a 

CDSMP program. The first summit was convened in January 2011 in the Wichita area (Sedgwick, Butler, Harvey counties). 

The second summit was in September 2011 in Johnson County. The message the state tries to communicate to prospective 

partners is that self-management classes help people better manage their conditions and can have a positive impact on the 

community health system.

New York State officials have presented CDSMP for new Medicaid Incentives Program. Unfortunately, in the second round of 

consideration, CDSMP was removed from list of programs to receive Medicaid reimbursement.

Tennessee The AAAs are looking at various options, including other grants (through ACA), TennCare reimbursement/subsidization, 

insurance companies etc.

Vermont Program is already sustainable under Blueprint.

Domain 3: CDSMP Eligibility, Enrollment, 

Completion

Domain 3: CDSMP Eligibility, Enrollment, Completion

Eligibility screening? Why, How? Eligibility screening? Why, How?

Arkansas No eligibility screening reported.

California No eligibility screening reported.

Kansas There is no formal screening (i.e., for dementia), but the state works with leaders to help them understand that a certain 

level of functioning is required for participants to benefit from CDSMP.

New York None to report. Lay leaders and master trainers felt everyone could benefit from the course.

Tennessee No screening, no charge for participation, the state has provided money for incentives for completion and “nice” snacks, 

although some AAAs are not aware of this.

Vermont None to report. Lay leaders and master trainers felt everyone could benefit from the course.

Enrollment: Challenges, Opportunities Enrollment: Challenges, Opportunities

Arkansas Enrollment has been limited among African Americans, although some host sites are reaching out via faith organizations. 

Enrollment has been limited in the SE Delta areas and other rural areas and among low-income, low-literacy groups (books 

are intimidating to these groups). Enrollment has been easiest in central Arkansas, where more residents are white and 

middle class, and northwest Arkansas, where there is a strong senior center and affluent retiree population.

California Sites trained to over enroll because drop out is inevitable. Fewer drop out in the case when orientation session or Class 

Zero held.

Kansas The state has had difficulty generating demand for CDSMP and filling classes, especially in rural areas. There are no waiting

lists.

New York In 2 sites, still experiencing difficulty enrolling enough participants for one workshop. In the case of one site, they had been 

advertising for more than 6 months, but did not have enough interested participants. If there is too much lag time between 

when participants express interest, they may decide against enrolling in the workshop.
IMPAQ International, LLC Page C-38 CDSMP Process Evaluation Final Report

Appendices

Tennessee Recruitment is often a challenge. Some leaders say that getting people to commit to the 6 weeks and 2.5 hour classes is 

hard. Recruiting men is a challenge; it does seem that having a male leader may help. Use of senior centers is good for 

recruitment, but there is the problem of participants wandering in and out of class and not attendingconsistently.

Vermont Panel management was a successful tool in recruitment and enrollment. Targeted letters were successful in recruiting 

participants. Completion rates have dropped recently but health officials are unsure why.

Older adults only, mixed ages? Older adults only, mixed ages?

Arkansas The state enrolls all ages in CDSMP. Couples sometimes attend together, or one might be the caregiver for the other.

California Mixed ages.

Kansas Mixed ages, but mostly older adults.

New York Age range is 17 to 100.

Tennessee Largely over 60, some younger.

Vermont Mixed ages. Everyone covered under Blueprint.

Registration: How? Where? Process? 

Supporting data system?

Registration: How? Where? Process? Supporting data system?

Arkansas Registration is handled locally, either by the host site (e.g. a senior center) or by workshop leaders.

California Register with coordinator at local site.

Kansas Registration is typically handled at the local level by lay leaders based at senior centers, county departments on aging, and

the Kansas State Extension county offices. There is no electronic registration system.

New York Usually through local program coordinator or lay leader.

Tennessee Participants contact the leader for the most part, sometimes the AAA staff will help with the enrollment process, and 

sometimes the implementation site will register participants. Phone calls are the primary means of registration/enrollment.

Vermont Usually through local program coordinator or lay leader.

Completion: Successes & Challenges Completion: Successes & Challenges

Arkansas Class size and completion rates vary by class and implementation site. One class at a housing development in the Hot 

Springs area started with 11 participants and ended with 2 because many were ill and had to miss class for doctor 

appointments (many were couples). In contrast, classes at a senior center in northwest Arkansas are well attended with 

high completion rates. Transportation issues in rural areas can also be a barrier to completion.

California Local organizations felt it was easier to market the program and obtain buy-in because there was a "deliverable" (number 

of completers).

Kansas One of the host sites is a Veterans Affairs facility which refers its outpatients to the course – as they are ordered to attend, 

completion rates are very high. There is speculation that a physician’s order or recommendation may have similar effects. 

Residents of senior centers, for programs hosted at their center, have inconsistent attendance during a class and over the 

course of the program, impacting completion and ‘dosage’ effects. Participants who have taken other programs may be 

more likely to take CDSMP.

New York Higher completion rates for individuals who participated in a recruitment or Class Zero session.

Tennessee Consistent attendance can be a challenge for some sites; also getting participant buy-in to the 6 sessions. Weather may 

impact attendance or ability of a site to offer a class.
IMPAQ International, LLC Page C-39 CDSMP Process Evaluation Final Report

Appendices

Vermont Higher completion rates for individuals who participated in a recruitment or Class Zero session.

Challenges providing workshop? 

Accommodations to meet these? 

Challenges providing workshop? Accommodations to meet these?

Arkansas No challenges reported.

California Unlike states with harsh seasonal weather, California able to offer workshops year round. Demand is still low in some 

areas.

Kansas Most common difficulty is site location and recruitment.

New York Seasonal challenges to providing workshops. In winter, older adults do not want to leave their home and it can be 

dangerous for them to do so. Individuals reluctant to commit to activities over winter months.

Tennessee Transportation - some leaders will provide this

Vermont Seasonal challenges (although this appears less of an issue than seen in New York) and challenges to offering workshops in 

rural settings. One lay leader reported travelling 90 minutes to offer the workshop each week. Participants are often 

unwilling to travel this far.

Domain 4: Data Collection Domain 4: Data Collection

Types of Data: What is collected, by 

whom, how maintained, what format?

Types of Data: What is collected, by whom, how maintained, what format?

Arkansas Paper and pencil forms are completed by sites and sent to the data analyst at ADOH for review and entry into the NCOA 

database. ADOH does some analysis of the data, e.g., compliance with the 30/30 concept, whichmeans that everyone in 

the state can access CDSMP within 30 miles and within 30 days.

California All NCOA data is collected. Additional data elements include: information on insurance status (Medicaid, Medicare, 

private); Veteran status; self-management activities; and program usefulness.

Kansas In addition to the NCOA data, the state requests that CDSMP participants complete a “KOHP Participant Feedback Form” at 

the conclusion of the workshop. This is a course evaluation form that requests feedback on where the participant heard 

about the workshop, satisfaction with the workshop, and usefulness of the techniques taught.

New York QTAC collects and maintains all data.

Tennessee Only NCOA data is collected. Usually the AAAs gather the data for their region, ensure it is completed, and send to the state 

lead who enters it into the database.

Vermont Data is reported to analysis and collection arm of Vermont Department of Health. 

Enrollment Data Enrollment Data

Arkansas Standard enrollment data is collected.

California Standard enrollment data is collected.

Kansas Standard enrollment data is collected.

New York Standard enrollment data is collected.

Tennessee Standard NCOA enrollment figures.

Vermont Vermont will not allow the scheduling of workshops if 10 individuals are not enrolled.

Tracking participants Tracking participants
IMPAQ International, LLC Page C-40 CDSMP Process Evaluation Final Report

Appendices

Arkansas The state does not track participants after workshop completion.

California The state does not track participants after workshop completion.

Kansas The state does not track participants after workshop completion.

New York Participants are not routinely tracked, with the exception of those participating in the outcome evaluation study.

Tennessee The state does not track participants after workshop completion.

Vermont Sometimes participants are tracked through health plans.

Collect data other than Stanford 

requirements?

Collect data other than Stanford requirements?

Arkansas The state does not collect any data other than that required by NCOA.

California Additional data elements include: health insurance status, veteran status, and self-management/program usefulness 

scales.

Kansas Only standard data collected.

New York Yes, all participants asked to fill out participant satisfaction survey. Other data collection activities include an addendum 

survey through the public health departments in 8 rural counties, outcome evaluation study, and potential HMO study with 

matched administrative sample.

Tennessee No other data is collected.

Vermont Ability to link CDSMP data to health records. Currently doing double data entry.

Follow up on completers? Follow up on completers?

Arkansas Arkansas does not follow-up with completers.

California California does not follow-up with completers.

Kansas No follow-up other than the “KOHP Participant Feedback Form.”

New York For outcome evaluation study, they are following up with completers.

Tennessee Tennessee does not follow-up with completers.

Vermont Inconsistent follow-up with completers through health plans. If participant elects, they will discuss their action plans for the 

primary care physician.

Evaluation capacity, participation: Staff 

and systems available

Evaluation capacity, participation: Staff and systems available

Arkansas ADOH depends on NCOA for most analyses. A 0.5FTE data analyst at ADOH does some data analysis and manages a 

password-protected Website for leaders that currently provides workshop schedules, marketing materials, and a blog.

California Given history of involvement with program, evaluation capacity should be high if participant tracking component added to 

their system. At the moment, monitoring is inconsistent and data collection is not ideal with outcome evaluation study.

Kansas NCOA forms and the “KOHP Participant Feedback Form” are completed on paper and faxed or emailed to KDHE for data 

entry. A data analyst at KDHE funded through the ARRA grant periodically analyzes the data and provides reports to the 

state and local sites via the listserv and Webinars.

New York Staff reported they were interested but lay leaders and master trainers felt there were serious ethical implications to asking 

participants to wait.
IMPAQ International, LLC Page C-41 CDSMP Process Evaluation Final Report

Appendices

Tennessee They have the SAMS database as well as the NCOA. They have no other systems. There is one state-level part-time data 

entry person. Most of the AAAs have a full-time coordinator who will transition to support other programs once AoA 

funding ends.

Vermont Very strong evaluation capacity.

Domain 5: State Distribution and Delivery Domain 5: State Distribution and Delivery

Describe State Agency Role Describe State Agency Role

Arkansas ADOH is grantee; partners with DAAS. Leads of both agencies co-chair quarterly meetings, perform site checks of fidelity, 

and assist host sites with administration.

California Largely providing oversight. Made final decision after receiving TA contractor recommendation on 6 ARRA funded sites.

Kansas KDoA is the AoA grantee. KDHE has historically been the lead with CDSMP and has assumed a strong role under the ARRA 

grant.

New York State plays an advisory role. Very strong relationship with public health department.

Tennessee There are 5 regional coordinators within the AAAs for CDSMP. One state level program coordinator. They have 5 master 

trainers in the state, but leverage some trainers and leaders from the University of Tennessee Extension. The Extension 

claims to have educators available to train in every county in the state. All recruitment is done at the local (AAA) level. 

Generally the AAA coordinator recruits, markets, and holds Class Zeros. There may be some support provided by the 

implementation site or leaders. Some AAAs rely heavily on a local advocate for site identification, participant recruitment, 

Class Zeros, etc. Collects the data and enters it into the NCOA database; has some regular calls with the AAAs and master 

trainers. 

Vermont State involved in all program operations, providing TA to sites and monitoring fidelity.

Statewide CDSMP network? Centralized? 

Decentralized?

Statewide CDSMP network? Centralized? Decentralized?

Arkansas Responsibility for CDSMP implementation is largely decentralized, with AAAs and AAI Centers on Aging responsible for 

program development, marketing, and implementation in their regions.

California Statewide network but not statewide coverage. Some elements centralized through TA contractor.

Kansas Responsibility for CDSMP implementation is largely decentralized, with AAAs and the Kansas State Research & Extension 

Service’s network of county offices responsible for program development, marketing, and implementation in their 

respective regions. 

New York Only 50% of counties covered, but more than 50% of state residents served. TA is centralized but everything else is 

decentralized.

Tennessee Decentralized, most done at the AAA level (equivalent to host sites). 

Vermont Near statewide coverage. TA is centralized but local sites have flexibility in how they implement the program, if maintaining 

fidelity standards.

Technical Assistance or support? Technical Assistance or support?

Arkansas The password-protected Website for leaders managed by ADOH provides marketing materials, workshop schedules, and a 

blog.
IMPAQ International, LLC Page C-42 CDSMP Process Evaluation Final Report

Appendices

California Statewide TA contractor PICF

Kansas The state has a “leader maintenance plan” that includes a listserv, quarterly Webinars, and annual meetings. The local sites 

reported that this assistance is very helpful. At an October 2011, annual meeting for leaders at Kansas State University, 

GMAMMA, an improvisational actor, used humor to communicate the importance of fidelity to leaders. The state also 

shares NCOA data with the sites.

New York QTAC provides statewide TA, with program coordination, course materials, and data collection and analysis.

Tennessee The state AoA lead ensures the data is entered.

Vermont Vermont Department of Health provides TA and sites also receive support from host sites, normally patient centered 

medical homes.

Interface with State: Successes and 

Challenges

Interface with State: Successes and Challenges

Arkansas DAAS has been able to include in their 4-year state plan that all AAAs offer at least one EBP. The Governor’s office and 

Medicaid are represented on the Stakeholders Partnership group. Overlapping regions and responsibilities for CDSMP on 

the part of the AAAs and AAI has probably not been the best use of resources. To date, there has been no involvement of 

the Arkansas Center for Health Improvement (a state agency headed by the state’s surgeon general), which seems to be a 

missed opportunity.

California CDA advertises all workshops on the Website and provides oversight

Kansas The Kansas State Research and Extension Service provides a strong, established infrastructure at the county level, which 

has been helpful developing CDSMP delivery system.

New York Strong relationship between TA contractor, Aging (DAIL, Department of Disabilities, Aging, and Independent Living), and 

Vermont Department of Health

Tennessee Tennessee recently experienced a change in the state-level program coordinator. The previous coordinator had begun to 

establish networks and regular communications with the AAAs. The new coordinator is still “learning the ropes” and 

regular communications have dropped, though she is available via phone or email to answer questions. Networking has 

also suffered during the transition.

Vermont Strong relationship between medical homes and health department. As a result of ARRA funding, growing relationship 

between aging and public health departments (mainly AAAs).

How can state agency best support your 

site or others?

How can state agency best support your site or others?

Arkansas Insufficient information available.

California CDA was largely providing oversight. Main source of support was TA contractor.
IMPAQ International, LLC Page C-43 CDSMP Process Evaluation Final Report

Appendices

Kansas It is unclear how the program will be sustained at the state level after the ARRA grant ends and if continuing CDC funding is 

not secured. Sustainability at the state level is also at risk given the state fiscal crisis and health reform initiatives. The local 

sites are trying to “embed” leaders within their agencies to continue offering CDSMP.

In Wichita there is a developing partnership involving the AAA, a local medical center, senior housing developments, 2 

community health centers, and Wichita State University (WSU). Using unpaid student interns, WSU has volunteered to 

manage program administration and marketing to providers and consumers, with the prospect of future 

evaluation/research opportunities. Interns are personally promoting the program to provider groups as well as potential 

implementation sites.

New York State aging department is largely providing oversight. Main source of support was QTAC.

Tennessee Funding support would be the main thing. Perhaps refresher training or other educational and skills support.

Vermont Blueprint (state health plan) funds all sites.

Sustainability of Distribution/Delivery 

System

Sustainability of Distribution/Delivery System

Arkansas ADOH has trained 2 lay leaders who are now embedded in the agency and will deliver CDSMP after grant funding ends. It is 

unclear who will manage the program at the state level after salary support ends for the program coordinator and data 

analyst at ADOH.

California Extremely dependent. Delays in approving budget a large influence on program.

Kansas KDoA, KDHE, Kansas State Research & Extension Service, Veterans Administration, 2 hospital systems, AAAs and local 

senior centers, and county departments on aging. In Wichita, the Wichita State University.

New York Goal has always been sustainability, but some sites will be more successful than others.

Tennessee The AAAs are concerned about their ability to continue to offer the same number of programs after funding ends and are 

each seeking other options. Each AAA seems to have its CDSMP champion taking the initiative to try and find other funding. 

Vermont Program already sustainable under current system.

How vulnerable is distribution system and 

CDSMP program in state/county fiscal 

climate?

How vulnerable is distribution system and CDSMP program in state/county fiscal climate?

Arkansas Very vulnerable at state and local levels.

California Most sites will continuing offering CDSMP after ARRA funding ends, if pool of lay leaders still exists.

Kansas Very vulnerable, with only slightly more stability in Wichita.

New York To be sustainable, additional funding will be needed.

Tennessee Likely comparable to some other states.

Vermont System is not vulnerable as it is already sustainable.

Institutionalized or dependent on a single 

leader, agency, temporary funding?

Institutionalized or dependent on a single leader, agency, temporary funding?

Arkansas In the past year or so, CDSMP has finally gained some momentum in some areas of the state (e.g., Northwest, Hot Springs), 

but it is far from institutionalized.
IMPAQ International, LLC Page C-44 CDSMP Process Evaluation Final Report

Appendices

California CDSMP is institutionalized in some settings but not all. Institutionalization sometimes dependent on consistency of funding 

stream (i.e. Kaiser Permanente)

Kansas Very vulnerable at state and local levels.

New York Many components institutionalized but still largely dependent on actions of very few individuals.

Tennessee While it is not institutionalized, the presence of AAA champions and interest in seeing the program continue is promising. 

However, lack of funding will likely diminish this.

Vermont The program is institutionalized.

Domain 6: Feasibility of Outcome 

Evaluation

Domain 6: Feasibility of Outcome Evaluation

What data elements collected? What data elements collected?

Arkansas Insufficient information available.

California NCOA data plus several additional elements. Additional elements unknown.

Kansas Local sites said that they would welcome the opportunity to participate in an outcome evaluation so that they could 

demonstrate the effectiveness of KOHP for Kansans. Kansas was a test site for “Strong People Stay Young—Health Hearts,” 

a research project conducted at two sites in Kansas and two sites in Arkansas by Dr. Miriam Nelson at Tufts. Twenty 

Kansans were enrolled and stayed with the program throughout the research period. Researchers from the aging 

department at WSU said they would be well suited to collaborate on an evaluation.

New York All NCOA data and additional data elements described. Ranks in top 2 states visited for evaluation capabilities.

Tennessee Only NCOA data is collected at this time.

Vermont Very strong evaluation capabilities with records already linked to health plan expenditures.

Is data reliable? Why or why not? QA Is data reliable? Why or why not? QA

Arkansas Insufficient information available.

California Data reported for California also includes Kaiser Permanente-funded workshops and any other workshop willing to submit 

data packets to Partners in Care Foundation. They reach out to all sites who offer CDSMP, but do not have authority over 

non-AoA funded workshops. TA contractor noted that she sees larger fidelity issues outside of AoA-funded workshops.

Kansas There may be issues with incomplete data or missing data dependent on the lay leader responsible for collecting, some are 

more adherent to the policy than others.

New York Insufficient information available.

Tennessee Quality assurance performed at AAA level. It is unknown how consistent the monitoring process is between AAAs.

Vermont Health plan data is very reliable. 

What reports are generated and how are 

they used?

What reports are generated and how are they used?

Arkansas Insufficient information available.

California PICF generate reports to generate interest and advertise best or new practices.

Kansas Participation and completion are monitored, but no formal reporting is done.

New York Quarterly reports shared with aging and public health department officials who use them to influence state policy.
IMPAQ International, LLC Page C-45 CDSMP Process Evaluation Final Report

Appendices

Tennessee No reports generated.

Vermont Annual reports made public. Used to influence policy makers to continue to support the program.

Is it possible to share data not reported to 

NCOA?

Is it possible to share data not reported to NCOA?

Arkansas No additional data collected.

California The team did not receive any data from California

Kansas No additional data collected.

New York The team received copies of additional data collection instruments but not the data.

Tennessee No additional data collected.

Vermont The team did not receive any data from Vermont.

For evaluation purposes, would 

states/participants/sites be willing to 

share data?

For evaluation purposes, would states/participants/sites be willing to share data?

Arkansas Unknown.

California At this point some sites will not share all their NCOA data that is required to protect privacy of consumers. TA contractor 

cannot force any site to provide data.

Kansas Unknown.

New York QTAC is open to discussing this further, when data requirements are known.

Tennessee Yes

Vermont Vermont would have very little incentive to participate. The program is already sustained.

Is it possible to turn away and track 

participants?

Is it possible to turn away and track participants?

Arkansas Unknown.

California In certain settings, it may be possible.

Kansas Most leaders had concerns about doing this, mostly centering around the ethics of it.

New York Unknown: QTAC staff said it may be possible under certain conditions. Lay leaders and master trainers raised ethical issues.

Tennessee There was concern about the ability and ethics of doing this, but possibly less than other states.

Vermont Vermont is not interested in an evaluation that turns away participants.
APPENDIX D. KEY INFORMANT INTERVIEW QUESTIONS

IMPAQ International, LLC Appendix D CDSMP Process Evaluation Final Report

Appendices
APPENDIX D. KEY INFORMANT INTERVIEW QUESTIONS

Key Informant Interviews Questions Interview Probes

How are local sites implementing the CDSMP? For 

example, their organizational structure, financial 

resources and allocation and their fidelity to the 

Stanford CDSMP model? Do you expect this to change 

and how?

• What is the organizational structure of host sites and state? 

• Describe how fidelity is monitored.

• What are common adaptations implemented by the sites?

• What percentage of AAAs are part of CDSMP delivery system

• Describe your interaction with ADRCs in the state (e.g. serve 

as referral source, host or implementation site, etc)

What do you perceive as the barriers and supports 

that affect recruitment and completion rates? Do you 

expect the barriers and supports to change and how?

• Which of the barriers has the greatest influence on 

recruitment rates?

• Which of the barriers has the greatest influence on 

completion rates?

Other than the data reported to NCOA as part of the 

grant, what data, if any, does your state collect What is 

the state of your records systems? Do you expect the 

data collection efforts to change and how? 

• What elements are included in the data reporting system?

• Will you continue reporting to NCOA after ARRA funding has 

ended?

What activities do the state and local sites have to 

sustain the program after the end of the ARRA 

funding?

• How does the state and local sites use program data, 

including evaluations?

How would you describe your statewide distribution 

and delivery system? What other evidence-based self 

management programs are provided in your state? 

What aspects of your program will be sustained after 

ARRA funding ends? Do you expect this to change and 

how?

• Besides the AAAs/ADRCs, who are your top 1-5 major 

partners who have embedded CDSMP into their ongoing 

activities (i.e. offer workshops at least twice a year in 

multiple locations) or who have played other significant roles 

in helping you expand CDSMP statewide?

• What are your measures of sustainability?

• What program activities will continue or end (e.g. staffing, 

leadership, key partners, delivery infrastructure)?

• In addition to the Recovery Act funding, what other funding 

sources are supporting the health promotion and disease 

prevention programs offered by your state in FY 2012?

• How will staffing change? Administrators? Program 

coordinators? Lay leaders?

What populations does your state target and serve? 

Do you expect this to change and how?

• Do you foresee changes in ability to target diverse 

populations (i.e. rural, minority)?

IMPAQ International, LLC Page D-1 CDSMP Process Evaluation Final Report 

Appendices
APPENDIX E. CDSMP PARTICIPANTS BY AOA REGION

IMPAQ International, LLC Appendix E CDSMP Process Evaluation Final Report

Appendices
APPENDIX E. CDSMP PARTICIPANTS BY AOA REGION

Exhibit E.1 shows the extent to which AoA regions served the AoA target population (i.e., adults aged 

60 and older). Regions III, IV, V, and IX served the greatest numbers of participants aged 60 and 

older. As demonstrated by Exhibit E.2, the percentage of participants 60 and older served by state 

grantees largely reflected the percentage of Medicare beneficiaries living in each jurisdiction. While 

all regions served more participants who were aged 60 and older than those under 60, there is great 

variation across AoA regions. For instance, Region III served the greatest proportion of participants 

aged 60 and older (85 percent), while Region VI served the smallest proportion (57 percent). 

Exhibit E.1. CDSMP Participants by Age and AoA Region

AoA Region Participants

Under Age 60

Percent of 

Participants in 

Region Under 

Age 60

Participants

Age 60+

Percent of 

Participants in 

Region Age 60+

Total Number of 

Participants in 

Region

Region I 1,273 25% 3,746 75% 5,019

Region II 1,917 27% 5,184 73% 7,101

Region III 1,223 15% 6,903 85% 8,126

Region IV 3,517 23% 11,717 77% 15,234

Region V 4,264 27% 11,626 73% 15,890

Region VI 2,753 43% 3,707 57% 6,460

Region VII 721 26% 2,045 74% 2,766

Region VIII 936 34% 1,839 66% 2,775

Region IX 2,801 28% 7,226 72% 10,027

Region X 2,638 40% 3,877 60% 6,515

Total 22,043 28% 57,870 72% 79,913

Note: Excludes participants with missing date of birth.

Source: NCOA data

Exhibit E.2. CDSMP Participants Age 60+ and Medicare Beneficiaries by Region

AoA Region Participants

Age 60+

Percent of 

Participants

Age 60+

Beneficiaries Medicare 

Population (Age 65+)

Percent of 

Beneficiaries

Medicare Population 

(Age 65+)

Region I 3,746 6% 2,316,774 5%

Region II 5,184 9% 4,799,739 11%

Region III 6,903 12% 4,636,685 11%

Region IV 11,717 20% 9,540,724 22%

Region V 11,626 20% 7,797,970 18%

Region VI 3,707 6% 4,851,767 11%

Region VII 2,045 4% 1,638,149 4%

Region VIII 1,839 3% 838,493 2%

Region IX 7,226 13% 5,916,279 13%

Region X 3,877 7% 1,762,908 4%

Total 57,870 100% 44,099,488 100%

IMPAQ International, LLC Page E-1 CDSMP Process Evaluation Final Report 

Appendices
IMPAQ International, LLC Page E-2 CDSMP Process Evaluation Final Report 

Appendices

Source: NCOA data, Medicare population statistics obtained from the AoA CDSMP Program Announcement for Cooperative 

Agreements.

As shown in Exhibit E.3, completion rates for participants aged 60 and older were slightly higher than 

for the younger population (77 percent compared to 75 percent). The differences (between under 

60 and 60+) are statistically significant both in the aggregate and in all regions except in Region VIII. 

Exhibit E.3 also presents data on CDSMP workshops and completion rates by AoA region. 

Completion rates for older adults vary across region, from 72 percent in Region X to 81 percent in 

Region VII.

Exhibit E. 3. CDSMP Workshops and Completion Rates by Participant Age and AoA Region 

AoA Region Number of 

Workshops

Number of

Participants

Under Age 

60

Number of

Completers

Under Age 

60

Completion 

Rate (%)

Under Age 

60

Number of 

Participants

Age 60+

Number of

Completers

Age 60+

Completion 

Rate (%)

Age 60+

Region I** 562 1,273 945 74.2% 3,746 2,889 77.1%

Region II** 687 1,917 1,485 77.5% 5,184 4,172 80.5%

Region III** 645 1,223 923 75.5% 6,903 5,476 79.3%

Region IV*** 1,335 3,517 2,695 76.6% 11,717 9,309 79.4%

Region V*** 1,604 4,264 3,145 73.7% 11,626 8,871 76.3%

Region VI*** 586 2,753 2,357 85.6% 3,707 2,828 76.2%

Region VII*** 406 721 541 75.0% 2,045 1,650 80.6%

Region VIII 285 936 692 73.9% 1,839 1,378 74.9%

Region IX*** 1,012 2,801 1,936 69.1% 7,226 5,275 73.0%

Region X*** 627 2,638 1,711 64.9% 3,877 2,800 72.2%

Total*** 7,749 22,043 16,430 74.5% 57,870 44,648 77.2%

Note: Excludes participants with missing date of birth.

(*), (**), and (***) denote the 5%, 1%, and 0.1% level of significance, respectively, for the difference in completion rates 

between “under age 60” and “60 and older” age groups. Lower levels of significance indicate stronger test results.

Source: NCOA data
APPENDIX F. CDSMP PARTICIPANT REACH BY GRANTEE

IMPAQ International, LLC Appendix F CDSMP Process Evaluation Final Report

Appendices
APPENDIX F. CDSMP PARTICIPANT REACH BY GRANTEE

AoA expressed interest in the “reach” of CDSMP—that is, what proportion of the residents in a state 

participated in CDSMP during the ARRA grant period, and how participation varied across different 

population groups and across different states. Exhibit F.1 shows variations in CDSMP reach by sex, 

age, and race/ethnicity for each of the 45 state grantees. Values are presented by 100,000 residents. 

For example, Texas reached 2.7 out of every 100,000 males residing in the state, whereas Oklahoma 

reached 69.6 out of every 100,000 males. 

IMPAQ International, LLC Page F-1 CDSMP Process Evaluation Final Report 

Appendices
IMPAQ International, LLC Page F-2 CDSMP Process Evaluation Final Report 

Appendices

Exhibit F.1. CDSMP Participant Reach by Grantee

State Sex: 

Male

Sex: 

Female

Age: 

<65

Age: 

65+

Ethnicity: 

Hispanic 

or Latino

Ethnicity: 

Not 

Hispanic

or Latino

Race: 

White

Race: 

African 

American

Race: 

Asian

Race: 

American 

Indian

Race:

Native 

Hawaiian/

Pacific 

Islander

Race: 

Other

Alabama 15.5 57.0 11.3 194.9 14.5 37.2 35.3 41.8 14.9 4,100.2 32.7 26.8

Alaska 32.2 89.8 34.3 327.6 40.8 42.0 55.7 81.7 10.5 251.7 40.5 42.9

Arizona 13.5 39.5 11.5 113.9 14.2 30.1 28.0 29.0 15.3 440.8 47.4 10.6

Arkansas 11.3 46.5 6.9 144.3 7.5 28.5 26.9 34.0 2.8 2,710.4 0.0 17.9

California 10.4 29.3 9.5 100.7 15.3 5.9 16.0 27.6 15.8 944.3 22.2 4.2

Colorado 6.8 26.6 10.0 69.0 18.5 15.0 14.7 7.4 44.6 1,073.0 0.0 22.5

Connecticut 11.2 39.3 5.3 129.9 20.0 24.4 25.5 30.1 4.4 6,281.1 350.1 23.7

Florida 13.2 37.5 12.4 83.4 38.3 20.4 18.3 35.7 9.5 3,603.5 40.7 32.5

Georgia 8.0 35.8 7.6 141.4 4.5 21.6 12.3 42.7 22.3 2,220.8 73.5 9.6

Hawaii 16.0 69.7 14.8 182.9 19.9 42.0 38.6 28.0 50.1 3,122.0 44.3 21.6

Idaho 33.4 125.0 37.1 344.7 47.2 68.4 75.9 20.4 26.2 4,943.8 172.6 60.8

Illinois 11.6 48.1 11.8 150.6 27.4 28.9 23.9 51.2 43.8 4,990.6 123.5 19.4

Indiana 6.2 30.0 9.3 76.7 6.9 17.6 17.2 27.6 2.9 5,097.0 42.6 12.9

Kansas 12.8 58.3 15.7 149.7 22.7 32.0 31.3 41.7 8.9 2,660.7 0.0 19.9

Kentucky 10.4 42.8 12.2 113.6 18.8 23.8 19.9 81.5 28.6 7,480.2 0.0 22.2

Louisiana 4.1 23.2 3.8 83.4 4.7 13.0 12.8 16.2 4.3 1,190.4 0.0 10.6

Maine 16.8 80.8 28.3 146.4 23.6 47.4 45.6 89.1 14.7 6,734.4 292.4 43.6

Maryland 11.1 50.5 9.6 182.6 8.7 32.1 30.1 39.5 10.3 4,951.0 63.4 12.7

Massachusetts 11.7 49.2 8.8 137.6 78.2 22.8 21.3 27.2 35.7 5,941.6 90.0 66.0

Michigan 21.1 70.8 27.2 188.0 42.6 47.9 45.0 63.8 37.8 5,663.9 76.8 41.9

Minnesota 9.4 38.7 5.7 144.9 4.0 16.1 24.7 9.1 8.4 1,837.0 92.8 9.6

Mississippi 10.2 44.1 15.3 108.3 11.0 12.1 8.9 55.4 7.8 1,037.9 421.2 9.7

Missouri 9.2 41.3 10.1 99.1 29.2 18.8 22.0 15.0 5.1 3,985.2 47.9 31.7

Nebraska 7.8 40.8 8.7 132.6 19.7 23.6 24.9 7.2 12.4 2,127.3 0.0 16.9

Nevada 8.4 27.5 6.7 95.6 8.0 20.8 19.5 29.3 9.2 1,085.4 59.3 5.1
State Sex: 

Male

Sex: 

Female

Age: 

<65

Age: 

65+

Ethnicity: 

Hispanic 

or Latino

Ethnicity: 

Not 

Hispanic

or Latino

Race: 

White

Race: 

African 

American

Race: 

Asian

Race: 

American 

Indian

Race: 

Native 

Hawaiian/

Pacific 

Islander

Race: 

Other

New 

Hampshire 17.2 61.3 16.8 143.6 16.3 31.5 33.2 20.0 0.0 13,015.9 0.0 32.9

New Jersey 23.3 61.5 21.6 155.3 32.3 39.0 26.0 84.8 56.6 5,408.9 295.8 31.0

New Mexico 22.8 77.9 33.2 126.7 70.1 29.5 58.3 11.8 17.7 424.9 0.0 8.6

New York 6.9 29.0 5.3 82.1 13.6 15.2 16.3 15.0 5.5 1,945.6 11.4 14.7

North Carolina 10.2 43.5 9.2 136.9 3.6 27.0 23.8 41.1 6.2 1,273.4 30.3 8.7

Ohio 10.9 36.6 9.8 102.7 20.0 20.4 18.0 56.4 9.9 6,772.9 73.8 24.5

Oklahoma 69.6 53.9 54.6 103.2 39.5 47.3 43.3 233.8 12.3 364.6 0.0 56.4

Oregon 36.8 111.5 49.7 223.6 101.1 57.1 71.5 43.3 67.3 4,306.1 37.3 67.3

Pennsylvania 10.0 46.3 7.9 144.4 50.0 27.4 17.6 104.7 51.8 6,836.0 27.4 23.0

Puerto Rico 5.8 26.4 3.4 89.1 16.4 2.7 10.1 10.0 0.0 1,441.6 0.0 46.8

Rhode Island 23.4 69.3 22.6 186.3 63.5 40.2 39.7 63.1 6.6 5,612.4 0.0 78.2

South Carolina 11.4 49.1 9.8 135.5 11.5 27.4 14.5 61.7 16.9 2,269.0 147.8 13.4

Tennessee 6.7 30.0 7.4 82.7 4.1 17.9 17.7 22.6 13.2 4,361.3 137.3 11.9

Texas 2.7 11.7 3.2 43.1 11.0 4.6 7.5 5.7 0.5 776.2 9.2 8.1

Utah 42.1 101.9 35.2 426.9 81.5 63.8 58.6 68.3 30.7 4,236.6 1,861.2 1.7

Vermont 68.5 231.5 90.2 469.9 43.4 144.2 138.4 63.7 0.0 37,381.1 625.0 124.4

Virginia 9.7 47.0 8.5 162.9 15.5 28.6 25.6 41.1 14.1 4,800.7 33.4 23.4

Washington 16.2 47.1 20.0 113.3 79.7 24.1 21.5 19.2 22.5 1,074.4 177.9 18.0

West Virginia 14.8 39.6 17.5 69.6 18.0 26.4 22.8 131.5 8.1 10,456.8 233.6 42.1

Wisconsin 16.8 69.5 17.9 175.2 72.0 38.1 37.5 43.2 4.6 3,369.0 54.7 12.5

Source: NCOA and U.S. Census data

IMPAQ International, LLC Page F-3 CDSMP Process Evaluation Final Report 

Appendices
APPENDIX G. CLASS SIZE FIDELITY BY STATE GRANTEE

IMPAQ International, LLC Appendix G CDSMP Process Evaluation Final Report

Appendices
APPENDIX G. CLASS SIZE FIDELITY BY STATE GRANTEE

State

Violation of First

Session Must-Do: At 

least 10 Participants

Violation of Any 

Session Must-Do: 10-

16 Participants a 

Violation of Either 

Must-Do

Alaska 77.1% 91.7% 91.7%

Alabama 19.8% 51.6% 52.4%

Arkansas 36.4% 68.2% 71.2%

Arizona 39.9% 73.4% 74.8%

California 47.4% 74.6% 75.9%

Colorado 53.6% 82.1% 83.3%

Connecticut 49.4% 83.1% 83.1%

Florida 13.3% 48.9% 51.6%

Georgia 43.1% 67.8% 69.0%

Hawaii 58.3% 83.3% 83.3%

Idaho 47.2% 82.4% 82.4%

Illinois 43.7% 71.6% 73.9%

Indiana 51.8% 78.9% 79.8%

Kansas 71.7% 84.1% 84.1%

Kentucky 47.2% 73.6% 73.6%

Louisiana 15.6% 51.1% 51.1%

Massachusetts 53.1% 78.4% 80.3%

Maryland 56.0% 77.7% 77.7%

Maine 64.4% 83.6% 84.9%

Michigan 52.9% 80.8% 81.0%

Minnesota 67.1% 88.6% 89.2%

Missouri 72.5% 85.8% 87.0%

Mississippi 50.7% 67.1% 68.5%

North Carolina 55.9% 76.7% 78.0%

Nebraska 58.7% 73.9% 76.1%

New Hampshire 52.7% 89.1% 89.1%

New Jersey 45.9% 71.3% 73.1%

New Mexico 58.4% 76.2% 79.2%

Nevada 43.2% 59.5% 59.5%

New York 44.4% 71.6% 72.2%

Ohio 46.9% 67.6% 68.4%

Oklahoma 33.7% 54.1% 54.1%

Oregon 61.6% 83.0% 84.4%

Pennsylvania 23.2% 51.2% 53.1%

Puerto Rico 5.6% 36.1% 36.1%

Rhode Island 36.6% 63.4% 65.9%

South Carolina 52.8% 72.9% 75.0%

IMPAQ International, LLC Page G-1 CDSMP Process Evaluation Final Report 

Appendices
State

Violation of First

Session Must-Do: At

least 10 Participants

Violation of Any 

Session Must-Do: 10-

16 Participants a 

Violation of Either 

Must-Do

Tennessee 49.5% 70.7% 72.7%

Texas 53.4% 74.7% 77.0%

Utah 51.2% 76.1% 77.1%

Virginia 10.7% 50.9% 51.5%

Vermont 56.0% 85.7% 85.7%

Washington 54.9% 74.9% 77.4%

Wisconsin 58.5% 81.0% 81.5%

West Virginia 57.4% 70.2% 74.5%

Total 47.2% 72.5% 73.8% a

We define compliance as having at least four workshops with 10-16 participants.

IMPAQ International, LLC Page G-2 CDSMP Process Evaluation Final Report 

Appendices
APPENDIX H. STATISTICAL TESTS FOR CHAPTER 6 EXHIBITS

IMPAQ International, LLC Appendix H CDSMP Process Evaluation Final Report

Appendices
APPENDIX H. STATISTICAL TESTS FOR CHAPTER 6 EXHIBITS

Table A: Difference in Percentage Points by Type of Program Oversight

Type of Program 

Oversight Centralized Decentralized

Under 60, 

Decentralized -0.6% - 

Under 60, Shared -3.0%** -2.4%**

60+, 

Decentralized -3.3%*** - 

60+, Shared -0.9% 2.4%***

(1) Participants with missing date of birth are excluded.

(2) Values show the percentage difference in completion rates 

between the type of program oversight in the row and the type 

of program oversight in the column headings. 

(3) (*), (**), and (***) denote the 5%, 1%, and 0.1% level of 

significance, respectively, for the difference in completion rates 

between two types of program oversight. Lower levels of 

significance indicate stronger test results.

Source: NCOA data.

Table B: Difference in Percentage Points by Type of Delivery System Structure

Type of Delivery System 

Structure Centralized Decentralized

Under 60, Decentralized 0.9% - 

Under 60, Mixed 8.5%** 7.6%*

60+, Decentralized -2.4%*** - 

60+, Mixed -2.1% 0.3%

(1) Participants with missing date of birth are excluded.

(2) Values show the percentage difference in completion rates between 

the type of delivery system structure in the row and the type of delivery 

system structure in the column headings. 

(3) (*), (**), and (***) denote the 5%, 1%, and 0.1% level of significance, 

respectively, for the difference in completion rates between two types of 

delivery system structure. Lower levels of significance indicate stronger 

test results. 

Source: NCOA data. 

IMPAQ International, LLC Page H-1 CDSMP Process Evaluation Final Report 

Appendices
IMPAQ International, LLC Page H-2 CDSMP Process Evaluation Final Report 

Appendices

Table C: Difference in Percentage Points by Ethnicity

Ethnicity Hispanic or Latino 

Origin

Not Hispanic or 

Latino

Under 60, Not Hispanic or 

Latino -3.0%*** - 

Under 60, Unknown -8.0%*** -5.0%***

60 – 64, Not Hispanic or 

Latino 0.2% - 

60 – 64, Unknown -6.9%*** -7.1%***

65 – 74, Not Hispanic or 

Latino -1.2% - 

65 – 74, Unknown -4.0%*** -2.8%**

75 – 84, Not Hispanic or 

Latino 1.9% - 

75 – 84, Unknown -3.4%* -5.3%***

85+, Not Hispanic or Latino -0.9% - 

85+, Unknown -2.9% -2.1% 

(1) Participants with missing date of birth are excluded.

(2) Values show the percentage difference in completion rates between the ethnicity in 

the row and the ethnicity in the column headings. 

(3) (*), (**), and (***) denote the 5%, 1%, and 0.1% level of significance, respectively, 

for the difference in completion rates between two ethnic groups. Lower levels of 

significance indicate stronger test results.

Source: NCOA data
IMPAQ International, LLC Page H-3 CDSMP Process Evaluation Final Report 

Appendices

Table D: Difference in Percentage Points by Race

Age and Race

American 

Indian or 

Alaskan Native

Asian or 

Asian 

American

Black or 

African 

American

Hawaiian 

Native or 

Pacific 

Islander

Other / 

Multiracial

Unknown 

Race

Under 60, Asian or Asian American 3.75% - - - - - 

Under 60, Black or African American 2.72% -1.03% - - - - 

Under 60, Hawaiian Native or Pacific 

Islander 8.76% 5.01% 6.04% - - - 

Under 60, Other / Multiracial 5.72% 1.97% 3.00% -3.04% - - 

Under 60, Unknown Race -2.91% -6.86%* -5.83%*** -11.87%* -8.83%*** - 

Under 60, White or Caucasian 0.21% -3.54% -2.51%* -8.55% -5.51%*** 3.32%**

60 – 64, Asian or Asian American 1.96% - - - - - 

60 – 64, Black or African American 5.02% 3.06% - - - - 

60 – 64, Hawaiian Native or Pacific 

Islander 22.57%*** 20.60%*** 17.54%*** - - - 

60 – 64, Other / Multiracial 2.60% 0.64% -2.42% -19.96%*** - - 

60 – 64, Unknown Race -0.88% -2.85% -5.91%* -23.45%*** -3.49% - 

60 – 64, White or Caucasian 2.53% 0.57% -2.49% -20.03%*** -0.07% 3.42%

65 – 74, Asian or Asian American 7.55% - - - - - 

65 – 74, Black or African American 8.96%** 1.41% - - - - 

65 – 74, Hawaiian Native or Pacific 

Islander 23.17%*** 15.62%*** 14.02%*** - - - 

65 – 74, Other / Multiracial 9.15%* 1.59% 0.19% -14.02%*** - - 

65 – 74, Unknown Race 4.46% -3.08% -4.49%** -18.70%*** -4.68% - 

65 – 74, White or Caucasian 7.99%* 0.44% -0.97% -15.18%*** -1.16% 3.52%*

75 – 84, Asian or Asian American 5.36% - - - - - 

75 – 84, Black or African American 7.38% 2.02% - - - - 

75 – 84, Hawaiian Native or Pacific 

Islander 11.73% 6.37% 4.35% - - - 

75 – 84, Other / Multiracial 0.44% -4.92% -6.94%** -11.29% - - 

75 – 84, Unknown Race -4.06% -9.42%*** -11.44%*** -15.79%* -4.50% - 
IMPAQ International, LLC Page H-4 CDSMP Process Evaluation Final Report 

Appendices

Age and Race

American 

Indian or 

Alaskan Native

Asian or 

Asian 

American

Black or 

African 

American

Hawaiian 

Native or 

Pacific 

Islander

Other / 

Multiracial

Unknown 

Race

75 – 84, White or Caucasian 4.22% -1.13% -3.15%** -7.50% 3.79% 8.29%***

85+, Asian or Asian American -0.76% - - - - - 

85+, Black or African American 0.19% 0.95% - - - - 

85+, Hawaiian Native or Pacific 

Islander 16.00% 16.76% 15.81% - - - 

85+, Other / Multiracial -5.41% -4.65% -5.60% -21.41% - - 

85+, Unknown Race -6.37% -5.60% -6.56% -22.37% -0.95% - 

85+, White or Caucasian -5.35% -4.89% -5.54** -21.35% 0.06% 1.02%

(1) Participants with missing date of birth are excluded.

(2) Values show the percentage difference in completion rates between the race in the row and the race in the column headings. 

(3) (*), (**), and (***) denote the 5%, 1%, and 0.1% level of significance, respectively, for the difference in completion rates between two race groups. Lower levels of 

significance indicate stronger test results.

Source: NCOA data
IMPAQ International, LLC Page H-5 CDSMP Process Evaluation Final Report 

Appendices

Table E: Difference in Percentage Points by Type of Implementation Site

Type of Implementation Site

Area Agency on 

Aging

Faith-Based 

Organization

Health Care 

Organization

Residential 

Facility

Senior 

Center

Under 60, Faith-based 

organization 2.2% - - - -

Under 60, Health care 

organization -7.2%*** -9.3%*** - - - 

Under 60, Residential facility -5.3* -7.5%*** 1.8% - - 

Under 60, Senior center 0.1% -2.2 7.1%*** 5.3%*** - 

Under 60, Other 1.7% -0.5 8.9%*** 7.0%*** 1.7%

60+, Faith-based 

organization 1.2% - - - - 

60+, Health care 

organization -5.8*** -7.0%*** - - - 

60+, Residential facility -8.8%*** -10.0%*** -3.0%*** - - 

60+, Senior center -2.2% -3.4%*** 3.6%*** 6.7%*** -

60+, Other -2.6* -3.8%*** 3.2%*** 6.2%*** -0.4%

(1) Participants with missing date of birth are excluded.

(2) Values show the percentage difference in completion rates between the type of implementation site in the row and the type of implementation site in 

the column headings. 

(3) (*), (**), and (***) denote the 5%, 1%, and 0.1% level of significance, respectively, for the difference in completion rates between two implementation

site types. Lower levels of significance indicate stronger test results.

Source: NCOA data.
IMPAQ International, LLC Page H-6 CDSMP Process Evaluation Final Report 

Appendices

Table F: Difference in Percentage Points by Number of Reported Chronic Conditions

Number of Chronic 

Conditions 0 1 2 or 3 4 or 5 

1 11.0*** - - - 

2 or 3 11.9*** 0.8 - - 

4 or 5 11.5*** 0.4 -0.4 - 

6 or more 9.8*** -1.3 -2.1* -1.7

(1) Values show the percentage difference in completion rates between the number of reported 

chronic conditions in the row 

and the number of reported chronic conditions in the column headings. 

(2) (*), (**), and (***) denote the 5%, 1%, and 0.1% level of significance, respectively, for the 

difference in completion rates 

between two chronic conditions categories. Lower levels of significance indicate stronger test results.

Source: NCOA data.
APPENDIX I. CDSMP WORKSHOPS, PARTICIPANTS, AND COMPLETERS BY QUARTER STATE 

GRANTEE

IMPAQ International, LLC Appendix I CDSMP Process Evaluation Final Report 

Appendices
IMPAQ International, LLC Page I-1 CDSMP Process Evaluation Final Report 

 Appendices

 

 

 

# of Participants

# of Completers

# of Workshops
IMPAQ International, LLC Page I-2 CDSMP Process Evaluation Final Report 

 Appendices

 

# of Participants

# of Completers

# of Workshops
IMPAQ International, LLC Page I-3 CDSMP Process Evaluation Final Report 

 Appendices

 

 

# of Participants

# of Completers

# of Workshops
IMPAQ International, LLC Page I-4 CDSMP Process Evaluation Final Report 

 Appendices

 # of Participants

# of Completers

# of Workshops
IMPAQ International, LLC Page I-5 CDSMP Process Evaluation Final Report 

 Appendices

 

 

# of Participants

# of Completers

# of Workshops
IMPAQ International, LLC Page I-6 CDSMP Process Evaluation Final Report 

 Appendices

 

 

# of Participants

# of Completers

# of Workshops
IMPAQ International, LLC Page I-7 CDSMP Process Evaluation Final Report 

 Appendices

 
IMPAQ International, LLC Page I-8 CDSMP Process Evaluation Final Report 

 Appendices

 

# of Participants

# of Completers

# of Workshops
IMPAQ International, LLC Page I-9 CDSMP Process Evaluation Final Report 

 Appendices

 

 

# of Participants

# of Completers

# of Workshops
IMPAQ International, LLC Page I

-10 CDSMP Process Evaluation Final Report 

 Appendices

 
IMPAQ International, LLC Page I-11 CDSMP Process Evaluation Final Report 

 Appendices

 

 

# of Participants

# of Completers

# of Workshops
IMPAQ International, LLC Page I-12 CDSMP Process Evaluation Final Report 

 Appendices

 

 

# of Participants

# of Completers

# of Workshops
IMPAQ International, LLC Page I-13 CDSMP Process Evaluation Final Report 

 Appendices

 

 

# of Participants

# of Completers

# of Workshops
IMPAQ International, LLC Page I-14 CDSMP Process Evaluation Final Report 

 Appendices

 

# of Participants

# of Completers

# of Workshops
IMPAQ International, LLC Appendix J CDSMP Process Evaluation Final Report

Appendices

APPENDIX J. LOGIT REGRESSION ANALYSIS
IMPAQ International, LLC Page J-1 CDSMP Process Evaluation Final Report 

Appendices

APPENDIX J. LOGIT REGRESSION ANALYSIS

Logistic regression is a commonly used approach in cases where the outcome (or the dependent variable) has 

two possible values (e.g., success vs. failure). These outcomes are coded as "0" (for failure) and "1" (for 

success) so that the model investigates the changes in the probability of success by changes in the observable 

factors (or independent variables). In this report, we investigate the probability of completion for CDSMP 

participants. So, if we denote the outcome variable (whether or not the participant completed the CDSMP 

workshop) by Yi, and the probability that Yi equals "1" for participant i by Pi, then the regression model is given 

by:

The model includes the following control variables derived from participant characteristics as well as the 

attempt/interview data:

Xi contains characteristics for participant i, such as age and gender

Sj contains characteristics for state j,such as type of leadership or source of funding

Wk contains characteristics for workshop k,such as number of participants or location 

Єi is a zero mean disturbance term.

Because equation (1) does not describe a linear relationship between the independent variables and the 

outcome variable, exponentiation of both sides of the equation and solving for Pi converts the estimates for 

the coefficients (β) into odds ratios. An estimated odds ratio of greater (less) than one indicates an increase 

(decrease) in the probability that the outcome variable takes a value of one. For example, an odds ratio of 

1.25 indicates that the probability that the outcome variable takes a value of one (e.g., success) is 1.25 times 

(or 25 percent) higher for a one-unit change in the predictor variable. 
IMPAQ International, LLC Appendix K CDSMP Process Evaluation Final Report

Appendices

APPENDIX K. CDSMP EVALUATION EFFORTS BY STATE GRANTEE
IMPAQ International, LLC Page K-1 CDSMP Process Evaluation Final Report 

 Appendices

APPENDIX K. CDSMP EVALUATION EFFORTS BY STATE GRANTEE

State Overview Studies Measures & Methods Key Findings

Alabama Database includes intake/client 

enrollment form completed before each 

workshop. Data are collected beyond 

the scope of the grant. All localities send 

data to the Alabama Department of 

Social Services, which sends report to 

National Council on Aging (NCOA). 

No studies reported. No specific measures and methods reported. No findings reported 

Alaska State Program Director collects, 

maintains CDSMP data in an ACCESS 

database. Data include NCOA 

requirements and additional 

information, e.g., pre-post feedback 

forms, age, gender, number of sick days, 

number of well days, attendance forms, 

trainer IDs, certification/re-certification 

dates, etc.

State Program Director is conducting 

evaluations including: 1) clinical 

outcomes study of people with 

diabetes (diabetes self management 

program under a separate grant); 2) 

Medicaid health center diabetes study 

using case-controls; and 3) separate 

diabetes group matching on A1C levels. 

For CDSMP: Participants are pre-and post 

surveyed; leaders are all post-surveyed. 

Diabetes Outcome Study: Clinical measures 

collected: a) BMI; b) LDL; c) BP; d) A1C. Clinical 

studies focus on differences between 3-6 

month and 12-month follow- up data. 

Currently analyzing first group of 

diabetes self-management participants:

1) anecdotal evidence suggests 

workshop participants cost to Medicaid 

is 1/3 to others who did not take 

program; 2) seems to be cost savings in 

pre-post group. These findings are not 

final and not public. 

Arizona Arizona Living Well Institute (grantee)

created data system for CDSMP that is 

updated on regular basis.

No studies reported. No specific measures and methods reported. No findings reported 

Arkansas State Department of Health grantee 

collects NCOA-required data from 

Department of Aging in Excel file, carries 

out basic evaluations. The state 

conducts some GIS mapping.

Department of Health does some 

analysis, e.g., compliance with the 

30/30 concept (everyone in the state 

can access CDSMP within 30 miles and 

within 30 days).

Assessment reports based on: 1) fidelity; 2) 

leader satisfaction for CDSMP and AMOB; 3) 

evaluation of dropouts by non-attendees; 4) 

participant self-report of improved outcomes; 

and 5) cross-match of participants and GIS 

mapping of site locations/country 

participation.

No findings reported. 

California Partners in Care Foundation (PICF) 

provides technical assistance and data 

collection statewide under contract

with the State of California. NCOA data 

are collected, as well as insurance 

status (Medicaid, Medicare, private); 

Veteran status; self-management 

activities; program usefulness. State 

does not track participants after 

completion of workshops. Data include 

Kaiser Permanente-funded workshops 

and any other workshop willing to 

submit data packets to PICF.

PICF monitors data from sites for 

fidelity but does not have influence on 

sites reporting data outside ARRA 

funding. 

California has a limited evaluation 

program at the Los Angeles Site. 

Pre/post-test study using Session One data 

as the baseline, and following up at Session 

Six and a six-month follow-up. 

No findings reported. 
IMPAQ International, LLC Page K-2 CDSMP Process Evaluation Final Report 

 Appendices

State Overview Studies Measures & Methods Key Findings

Colorado All sites provide data through a data 

portal established by the Consortium 

for Older Adult Wellness. Pre/post 

evaluation data collected by local 

leads, in addition to NCOA data. Some 

counties collect excess data for funding 

purposes. 

Some pre and post studies are 

conducted. 

No specific measures and methods reported. No findings reported 

Connecticut State collects additional data (other 

than NCOA-required data), e.g., prepost program, level of fatigue, health, 

pain, mood, social activities, and 

diabetes measures. 

Evaluation efforts have focused on 

evaluation of the Tai Chi program. 

Pre-post workshop evaluations. The Tai Chi 

evaluation collected outcomes related to: 1) 

“Up and Go”; 2) 50- foot walk tests; 3) rise 

from a chair. Other data measures collected 

not related to the Tai Chi evaluation include: 

1) the eight question new patient 

questionnaire for CDC funding purposes. 

No findings reported. 

Delaware Department of Public Health (DPH) 

collects NCOA-required data; DPH 

recently acquired a new grant that will 

cover outcome research. 

Pre-post data available. No specific measures and methods reported. No findings reported. 

Florida NCOA database exists for all site 

contracts; Data exist out of the system 

not entered into by sites outside the 

contract. 

No studies reported. No specific measures and method reported. No findings reported.

Georgia State is conducting a survey using 

questions from BRFSS and a nutrition 

class, delivered pre-post and at 6 

months. Currently piloting use of 

evaluation data with a refining tool and 

a fidelity tool. 

A pre-post survey is being conducted. Questions used by BRFSS and a nutrition 

class are used in a survey, delivered pre-post 

and at 6 months. 

No findings reported. 
IMPAQ International, LLC Page K-3 CDSMP Process Evaluation Final Report 

 Appendices

State Overview Studies Measures & Methods Key Findings

Hawaii Department of Health has funded 

evaluation efforts to date. An outcome

evaluation is being conducted with the 

University of Hawaii. For this project, 

additional data are collected for 3- and 

6-month follow up. Data are collected 

on health behaviors, efficacy, health 

satisfaction at 3 and 6 months (since 

2007). Recently, data have been 

collected for measures of Diabetes Self 

Management Program (BMI, 

cholesterol, glucose, A1C).

Evaluation efforts have focused on 

pre-post assessment of different 

populations in the state (e.g. 

Caucasian, Asian and Pacific Islanders). 

The outcomes measured in the study 

included social and role activity limitations; 

communication with physicians; self-rated 

health; time spent engaging in stretching 

and strengthening exercises; self-reported 

physician visits; increases in time spent in 

aerobic exercise; ability to cope with 

symptoms; and self-efficacy.

The evaluation found that CDSMP can 

be modified to achieve behavior 

changes in different ethnic 

populations. All subgroupsstudied 

realized significant decreases in social 

and role activity limitations and 

significant increases in communication 

with physicians.

Articles:

a)Adapting Stanford’s Chronic Disease 

Self-Management Program to Hawaii’s 

Multicultural Population Gerontologist; 

b)Cost–Benefit Estimates of an Elderly 

Exercise Program on Kaua‘- I Hawaii 

Medical Journal

Idaho Before the ARRA grant, each 

participant received a pre-post 

questionnaire. After ARRA this 

changed to a pre-survey only (due to 

cost constraints). Workshop and 

leaders evaluations are collected for 

every workshop. Idaho contracts with 

Boise State University for evaluation, 

including collection and analysis of 

data collected on workshops. 

Pre and post survey data existed 

before AARA- after AARA, this 

changed to a pre-survey only along 

with demographics and behavior. 

Boise State is the evaluation 

contractor for collecting and analyzing 

the data outcomes. 

The LWI intervention was delivered to 

participants in the LWI programwho

completed a pre-intervention and a 6-month 

post-intervention survey. Findings from 

matched pre and post surveys were 

analyzed to determine changes in selfefficacy, health status, healthcare utilization 

and health behaviors. The study population 

consisted of primarily white females over the 

age of 65, residing in rural or frontier areas, 

and suffering from one or more chronic 

condition. Of the 298 participants, 61% 

reported income of less than $24,999 and 

65% had a high school education or less.

Findings demonstrated significant 

changes in health status. Those 

reporting lower incomes experienced 

significant changes in health status; 

days effected by physical or mental 

health; and improved communication 

with physicians. Participants with lower 

incomes also reported higher 

utilization of physician services and the 

ER. Those in lower income/lower 

education categories also reported 

lower post-intervention selfmanagement skills.

Illinois State monitors fidelity and CDSMP 

implementation. 

No studies reported. No specific measures and methods reported. No findings reported.

Indiana State has centralized database where 

NCOA data are maintained. Data are 

submitted through a secure site. The 

state requires AAA sites providing 

CDSMP to complete a monthly Excel 

sheetthat includes questions about 

the numbers of master trainers and lay 

leaders, completers, enrollees, master 

trainer and lay leader names, 

scheduled time of master trainer 

trainings, workshops, and outreach. 

No studies reported. No specific measures and methods reported. No findings reported. 
IMPAQ International, LLC Page K-4 CDSMP Process Evaluation Final Report 

 Appendices

State Overview Studies Measures & Methods Key Findings

Kansas Kansas Department on Aging manages 

submission of data to NCOA. 

No studies reported. No specific measures and methods reported. No findings reported 

Kentucky Data are collected for submission to 

NCOA. Data are also reported to 

Stanford annually to maintain the 

license. 

No studies reported. No specific measures and methods reported. No findings reported.

Louisiana State has an evaluation report; the 

state is also looking at data elements 

other than those required for report to 

NCOA. States matches NCOA data 

against survey data. Some follow up is 

conducted by University of LouisianaMonroe. 

Study used repeated measures 

design. 

The report focuses on results on engaging 

low-income and disparate populations; 

reports on changes in client’s health 

behavior, health status, and reduction in 

client’s health care utilization and health

care costs; qualitative evaluation around 

program effectiveness in a communitybased, low-income, rural, and/or AfricanAmerican population, and quality 

improvement of the program. (Evaluation in 

process)

No findings reported. 

Maine Maine’s Department on Aging 

contracts with University of Southern 

Maine for data entry and reporting, 

and relies on community partners to 

enter data. Pre-post surveys are 

conducted. Maine has developed its 

own data collection/ analysis tools for 

outcomes and evaluation, emphasizing 

management on the local level. 

The evaluation of CDSMP focused on 

a pre- and post-test (6 months) of 

participants in the program. Four 

different evaluation activities have 

been undertaken: 

1) The MaineHealth Center for Quality 

and Safety conducted an evaluation of 

CDSMP with pre test and 6 month 

post and looked at outcomes. 

2)Jay Yoe from DHHS has conducted 

two QI Snapshots for state leadership 

- one on EnhanceWellness, which will 

be updated in January with the final 

results, and one for CDSMP that was 

completed this fall - both have 

outcomes data. 

3) The work with the state employee 

and retiree LWBH workshop series 

project involves comparing participant 

health benefit utilization data. 

4) For MOB, several papers have been 

submitted for publication with the 

outcomes of the national 

dissemination, and one regarding the 

TUG project.

One study looked at health benefit data;the 

outcomes for the other studies were not 

provided. 

No findings reported- the studies 

mentioned are underway or in the 

analysis phase. 
IMPAQ International, LLC Page K-5 CDSMP Process Evaluation Final Report 

 Appendices

State Overview Studies Measures & Methods Key Findings

Maryland Sites submit data required for NCOA to 

Maryland Department on Aging 

(MDoA) portal. Evaluation efforts in 

Maryland focus on CDSMP coordinator 

input and participant demographic 

characteristics. State has partnered 

with Towson State University for some 

evaluations. 

Maryland has adopted a mixedmethod study design to understand 

the reach and efficacy of CDSMP 

programs. A CDSMP evaluation was 

performed by Towson University in 

Maryland. The evaluation includes 

a phone interview with CDSMP 

coordinators across the state and 

an analysis of the data entered into 

the NCOA system plus several other 

questions related to participant 

income and language that is not in 

the NCOA data base but are on the 

participant surveys. They are 

looking for issues or gaps that may 

impede program delivery and those 

issues that may have made 

program delivery more successful. 

MD is also looking for any issues 

that may improve program 

outcomes and help sustain 

partnerships that are beneficial to 

the program.

Specific outcome measures are not 

reported; however, there is interest in 

findings on participation rates among 

different demographic groups and structural 

impediments to the group. Also, there is 

interest in collecting and reporting outcome 

data.

No findings reported. 

Massachusetts Evaluation efforts in Massachusetts 

focus on two different areas: 

1) Healthy eating evaluation; there 

were a total 42 workshops reaching 

432 participants – pre and post 

outcome data (weight, BMI, blood 

pressure)

2)CDSMP- project process evaluation

The state reports that data collection is 

a real challenge for community 

programs. It is time consuming, often 

confusing, and very labor intensive, so 

the state is looking to identify essential 

elements to collect on an ongoing 

basis. Tomando is especially difficult 

for data collection as staff must collect 

data one-on-one with participants.

CDSMP- Three different evaluation 

exercises: 

1) In-depth facilitator survey to 

understand challenges, barriers, 

continuing education needs, 

motivators/incentives and 

infrastructure needed for ARRA 

CDSMP; 

2)Assessment of current fidelity 

practices and development of a 

fidelity and QI plan;

3) Outcome pilot evaluation (pre-post 

survey). 

The University of Massachusetts 

Donahue Institute conducted the 

main CDSMP evaluation.

The Donahue Institute disseminated CDSMP 

Leader and Master Trainer Survey final 

report on workforce diversity, motivations 

and challenges, fidelity, past and future 

capacity, and continuing education 

preferences. Outcome Evaluation Pilot in 

progress - 400 pre-workshop evaluation 

surveys received, beginning to receive 6-

month follow-up surveys.

Preliminary findings of the outcomes 

evaluation pilot point to improvements 

in participant self-assessment of 

general health, levels of fatigue, pain, 

stress, sleep, physical activity, and 

communication with their physicians. 

In addition, small decreases in health 

care utilization were noted. In the area 

of health care utilization, completers 

between the ages of 65 - 74 had 

slightly greater decreases in health care 

utilization. The preliminary findings 

from the fidelity pilot show excellent 

participant satisfaction; health most 

common reason for drop out.
IMPAQ International, LLC Page K-6 CDSMP Process Evaluation Final Report 

 Appendices

State Overview Studies Measures & Methods Key Findings

Michigan Michigan has maintained a centralized 

data collection process (ACCESS 

database) since 2007. This uses a 

scannable form system and 

registration process that enables 

collection of data from both ARRAfunded and non-ARRA funded sites (all 

sites statewide). Data on health 

insurance are collected. The program 

receives many legislative requests for 

data. An in-house evaluator uses the 

database and other sources for 

comparison data. Also, the state 

partners with Michigan State 

University Geriatric Education Center 

for some evaluation.

The state focuses program evaluation 

on fidelity issues. Evaluation efforts 

focus on two different aspects: 1) 

online survey and 2) phone and 

face to face interviews. Michigan 

State University Geriatric Education 

Center carried out the evaluation. 

Fidelity issues. The evaluators have been trained as 

master trainers in CDSMP or CDSMP 

leaders, so they all have led one or 

more workshops, lending credibility to 

the evaluation process. 

Minnesota NCOA-required data only are collected. No studies reported. No specific measures and method reported.. No findings reported.

Mississippi NCOA-required data only are collected. No studies reported. No specific measures and methods reported. No findings reported.

Missouri In 2008, Missouri established its own 

database including five evidence-based 

programs. Under the ARRA grant, the 

state carried out dual data entry for 

CDSMP, entering data for the state and 

also for NCOA. Missouri uses its 

database for GIS mapping to show 

availability of leaders and has cross 

tabulated data with BRFSS to show 

availability of trainers and location of 

persons with arthritis. 

No studies reported. No specific measures and methods reported. No findings reported.

Nebraska The state uses aGIS specialist to map 

the data and is using NCOA’s 

Salesforce data internally and to share 

with partners.

Nebraska was using Stanford’s 

questionnaire to conduct pre-post 

evaluations, but these were 

postponed due to lack of funds. 

No specific measures and methods reported. No findings reported.

Nevada NCOA-required data only are collected. No studies reported. No specific measures and methods reported. No findings reported.

New 

Hampshire

State has centralized database. NH 

developed a pre-post outcomes survey 

with assistance from The Dartmouth

Prevention Research Center. The 

survey is closely aligned with that 

developed by Maine and Vermont. A 

summary report should be available in 

Spring 2012.

Evaluation efforts in New Hampshire 

have focused on pre-and postoutcomes for programs. Dartmouth 

University helped develop the 

outcome survey, and would likely be 

included in the evaluation. 

At the time of interview, there were 60 prerecords and 40 post-records in the database.

Information included on the surveys include: 

demographic and chronic disease 

information; type of insurance; medical care; 

ADLs, and Patient Activation Measures 

(PAMs).

A preliminary analysis showed higher 

levels of patient activation after 

completing the workshop. The state 

plansto collect data through 

December 2012.
IMPAQ International, LLC Page K-7 CDSMP Process Evaluation Final Report 

 Appendices

State Overview Studies Measures & Methods Key Findings

New Jersey New Jersey has a centralized database 

and undertakes evaluations of 

programs. ARRA funds were used to 

contract with InfoMatics at University 

of Medicine and Dentistry of New 

Jersey (UMDNJ) to develop a 

supplemental ACCESS database for 

CDSMP, to maintain information on 

where workshops are offered and 

records on peer leaders and master 

trainers. State receives requests for 

data from partners and the 

Commissioner of DHHS. A staff person 

spends 80% of time collecting and 

reviewing data. Office of Minority and 

Cultural Health maintains a separate 

database with demographics and nonEnglish-speaking populations,

Evaluation efforts in New Jersey are 

currently performed by three 

providers: 1) two evaluations at 

UMDNJ (different divisions); and 2) 

one evaluation at NJ Department of 

Health and Human Services.

The state is moving forward with 

three studies (aimed at different 

populations): 1) UMDNJ “SAVE 

Women and Men” related to New 

Jersey Cancer Education and Early 

Detection; enroll 90 CDSMP subjects 

and measure outcomes; 2) UMDNJ 

measuring outcomes in prisonsCDSMP has been conducted for two 

years; 3) Initiative at FQHC – Southern 

Jersey Center integration of 

operations with data collections.

Samples of participants range from those at 

risk for heart disease and stroke to a prisonbased population. 

Clinical outcomes are the focus of the 

evaluation, although specific results are 

not mentioned. 

New Mexico State collects and submits data related 

to NCOA measures, and also collects its 

own data for planning, and to obtain 

participant feedback, program reach, 

completion and more. The state is 

applying for federal funds to hire an 

evaluatorto collect and analyze followup participants on data. 

No studies reported. No specific measures and methods reported. No findings reported.

New York QAT runs the database for workshop 

data. QTAC is finalizing evaluation of 

outcomes for CDSMP participants. 

No studies reported. No specific measures and methods reported. No findings reported. 

North Carolina In addition to building a state 

database, NC has also added an 

evaluation component to the NC Living 

Health Chronic Disease SelfManagement Program. 

The evaluation study focuses on 

participants who attended the NC 

Living Healthy CDSMP workshop from 

Nov. 2009-2010. A pre- post-test 

survey was conducted, spaced out a 

year after participation in the seminar. 

Four domains were explored in the 

questions: 1) general health; 2) physician 

communication; 3) symptom management; 

and 4) daily activities. The outcomes were 

divided into four different domains: 1) 

general health; 2) physician 

communication; 3) symptom 

management; and 4) daily activities. 

The NC and NY evaluations include patient 

activation measure and 6 domains with prepost outcomes. The original sample size 

included 186; 113 participated in the survey.

Statistical trends, using ANOVA testing, 

were found in the four self-reported 

health domains. Specifically, increases 

in physician communication, daily

activities, and symptom management

were found as well as improvements in 

general health. 
IMPAQ International, LLC Page K-8 CDSMP Process Evaluation Final Report 

 Appendices

State Overview Studies Measures & Methods Key Findings

Ohio ODA collects and submits data to 

NCOA; ODA also collects data in 

addition to NCOA requirements. 

No studies reported. No methods and measures reported. No findings reported. 

Oklahoma In addition to collecting CDSMP data, 

the state also conducted a small survey 

to ask questions about CDSMP. 

Survey addressed the following 

questions: 1) how participants heard 

about the workshops; 2) whether 

participants feel they are better able 

to deal with their conditions after 

taking the workshops; and 3) how 

participants feel they have benefited 

regarding enhanced abilities. 

No methods and measures reported. No findings reported. 

Oregon Oregon has produced an in-depth 

report evaluating CDSMP in the stateLiving Well (CDSMP) Impact Report

The program evaluation takes a macro 

approach looking at estimates for 

QALYS gained and health care costs 

and utilization reduced because of 

CDSMP.

Oregon State University carried out the 

evaluation. The sample size included in the 

report is roughly 3,900 individuals. There are 

a number of outcome measures included in 

the report but they focus on QALYS and 

costs saved on medical care. 

The report found: Savings in health 

care utilization would outstrip costs by 

$1,445 per participant in the program:

107 QALYS gained as a result. 

Pennsylvania The Department of Aging controls the 

inputting and reporting of data. 

The University of Pittsburgh 

conducted a pre-post evaluation of 

participants, with 6 month and 12 

month follow up to assess behavior 

changes such as levels of physical 

activitiy, healthy eating, and more. 

No methods and measures reported. No findings reported. 

Puerto Rico DOH is able to evaluate the leaders as 

well as specific workshops. DOH has 

an evaluator oversees data-related 

activities. 

The evaluator will evaluate fidelity 

using Stanford tools; is working on an 

evaluation of participants that 

completed workshops six months ago; 

and conducting follow up interviews. 

Collected data will be analyzed and 

published as part of the project 

evaluation report

Some of the additional elements collected 

include information from a pre-post 

questionnaire that asks about physical 

activity, how sure participants are in 

managing their conditions, levels of pain, 

flexibility and more. 

No findings reported 

Rhode Island As a health department, HEALTH 

describes itself as a data driven 

organization that collects data and 

uses data for decision making. In 

addition to data collected for NCOA 

the state is interested in behavior 

change, ROI, and ER visits. 

Data are collected pre and post 

workshops and at 6 month follow up. 

Data are used for program 

improvement.

No methods and measures reported. No findings reported. 
IMPAQ International, LLC Page K-9 CDSMP Process Evaluation Final Report 

 Appendices

State Overview Studies Measures & Methods Key Findings

South Carolina South Carolina collects the NCOArequired data and also collects 

participant satisfaction data. 

At one point, South Carolina was 

collecting outcomes data but they 

have since stopped those efforts citing 

too high a volume of paperwork. They 

also have data on how participants 

learned of the program, but it is not 

currently analyzed.

No methods and measures reported. No findings reported. 

Tennessee Implementation sites are required to 

telephone completers at 6 months and 

again at 12 months to administer a 

short survey about whether the 

participant feels he/she is managing 

chronic conditions betters.

No Data No methods and measures reported. No findings reported. 

Texas Texas only collects NCOA data, though 

they do occasionally survey the EBP 

community regarding barriers to 

delivery and needs. 

They have recently completed a pilot 

study at a women's prison and are 

hoping that they will have a dedicated 

site for program delivery at this 

facility. There was also a summit to 

discuss DSMP accreditation as a 

requirement for Medicare 

reimbursement. They are in process 

of completing the pilot required to 

show ability to deliver and have 

already identified Medicare partners 

(FQHCs, a nursing school and a home 

health organization).

No methods and measures reported. No findings reported. 

Utah Utah has BRFSS data, arthritis program 

data and hospitalization data, also 

arthritis specific through June 2012. 

Two weeks after the classes, the data 

manager follows up with leaders about 

the data. 

They currently use data collected for 

sustainability efforts and making a 

'case' for Utah. They report the 

number of conditions they serve and a 

description of the population served. 

They are attempting to reach more 

organizations, but the data is helpful 

with internal partners as well.

No methods and measures reported. No findings reported. 

Vermont As part of a larger state data collection 

effort, the state collects data on 

CDSMP. The state is discussing ways 

that action plans might be posted and 

shared online. Vermont collects 

comprehensive data as part of the 

Blueprint, and these data provide rich 

opportunities for research and 

evaluation. 

The state conducts some analyses 

currently and has the ability to link 

CDSMP participation to outcomes in 

many cases.

No methods and measures reported. No findings reported. 
IMPAQ International, LLC Page K-10 CDSMP Process Evaluation Final Report 

 Appendices

State Overview Studies Measures & Methods Key Findings

Virginia Virginia collects demographic data, and 

is currently collecting pre/post 

assessment (general health, medical 

care, coping and symptoms) data 

through the end of the grant. 

The state has about 4 years of 

pre/post data and 2 years of 

demographic data. They have worked 

to make the pre/post assessment 

more user-friendly to reduce the 

burden on data collection. While they 

have many years of pre/post data, it 

has not been analyzed since 2009.

Partnerships with health systems have led to 

access to clinical and utilization data. They 

also collect comments and feedback.

No findings reported. 

Washington Washington collects NCOA required 

data. 

No specific studies mentioned. No methods and measures reported. No findings reported. 

West Virginia WV, through West Virginia University, 

has a number of data collection efforts 

and studies under way. The evaluator 

at West Virginia University is working 

with Marshall University to design a 

leader and participant questionnaire 

that will be implemented during the 

final quarter of the grant. The 

information gathered from the 

interviews will be summarized to 

highlight program outcomes, both 

problems and successes which will be 

useful to help maintain fidelity

 

There is an outcomes study in 

progress based on the Orreymodel, 

which incorporated some questions 

from the BPH, as well as self-reported 

health, self efficacy, quality of life, 

communication, activities, prediabetes screen, pre-hypertension 

screen, and cholesterol levels (all selfreported). They are training leaders to 

distribute the outcomes survey preclass (so far 89 completions), a 6-week 

satisfaction survey, and outcomes 

again at 3 and 6 month post program. 

The state is also planning a leader 

survey to determine activity, barriers, 

needs, experiences and demographics 

(at this point, the survey has been 

drafted). Also, the state is conducting 

telephone interviews of participants 

and non-participants by region, with 

proportional representation (CDSMP, 

DSMP, non-completers and 

completers). They aim to survey at 

least one participant per leader. At 

this point, the survey instrument is 

complete.

Evaluation efforts will look at a number of 

outcomes including: 1) participant 

satisfaction of participants; 2) background, 

experience, and satisfaction of leaders; 3 ) 

outcomes for participants including selfefficacy, self-rated health, pain, fatigue, and 

disability. 

No findings reported. 
IMPAQ International, LLC Page K-11 CDSMP Process Evaluation Final Report 

 Appendices

State Overview Studies Measures & Methods Key Findings

Wisconsin The state boasts robust evaluation 

efforts. 

State efforts focused on program 

evaluation 

Outcomes have largely focused on cost 

savings associated with the program. In 

addition, evaluation efforts have measured 

fidelity adherence, leader motivation, and 

retention. 

Program evaluation conducted in 

November 2010 suggested a nearly 

$1.5 million savings in health care 

expenditures based on the same 

utilization indicators of emergency 

room and hospital days used by 

Stanford in their original evaluation of 

the program.