Paralysis Resource Center Evaluation: Final Report

Paralysis Resource Center Evaluation:

A Comparison of State and National 

Quality of Life Grant Programs

Final Report

Prepared for: Administration for Community Living 

Office for Performance and Evaluation

Submitted by: New Editions

The Lewin Group 

August 30, 2021
Paralysis Resource Center Evaluation: Final Report

ii

Contents

PARALYSIS RESOURCE CENTER EVALUATION: A COMPARISON OF STATE 

AND NATIONAL QUALITY OF LIFE GRANT PROGRAMS FINAL REPORT .......I

Executive Summary .......................................................................................................... 1

Introduction...................................................................................................................... 2

Background................................................................................................................ 2

Purpose ..................................................................................................................... 2

Evaluation Design............................................................................................................. 3

Data Collection and Analysis............................................................................................. 3

Applications and Reports............................................................................................ 3

Interviews.................................................................................................................. 4

Surveys...................................................................................................................... 4

Findings ........................................................................................................................... 5

Application Focus, Types of Projects, and Populations Served ...................................... 5

Outreach.................................................................................................................... 6

Administration............................................................................................................ 7

Application Process....................................................................................................................................7

Technical Assistance Needs.......................................................................................................................8

Data and Reporting.................................................................................................... 9

Challenges and Barriers........................................................................................... 11

Accomplishments and Outcomes................................................................................ 11

Effects of COVID-19................................................................................................. 14

State Grantees...........................................................................................................................................14

National Paralysis Resource Center Grantees.........................................................................................15

Discussion and Recommendations.................................................................................... 16

Appendix A: Evaluation Questions................................................................................... 18

Appendix B. Interview Questions..................................................................................... 19

Year One.................................................................................................................. 19

Year Two ................................................................................................................. 19

Appendix C: State Survey Questions................................................................................ 20

State Pilot Program Survey Questions....................................................................... 20

Designation...............................................................................................................................................20

State Grantee Information........................................................................................................................20

State Grantee Self Evaluation..................................................................................................................20

ACL Application Process Evaluation......................................................................................................23

Outcomes Evaluation...............................................................................................................................24

Impact of COVID-19 on Grant Activities...............................................................................................24

State Subawardee Survey Questions........................................................................... 24

Designation...............................................................................................................................................24

Community-Based Organization Information.........................................................................................24
Paralysis Resource Center Evaluation: Final Report

iii

Community-Based Organization Self Evaluation...................................................................................25

Community-Based Organization Application Process Evaluation.........................................................28

Outcomes Evaluation...............................................................................................................................29

Impact of COVID-19 on Grant Activities...............................................................................................29

Appendix D: Vanderbilt University NPRC Survey Questions............................................. 30

Appendix E: Survey Likert Scale Conversions.................................................................. 33

Appendix F: State Survey Responses Years 1 and 2........................................................... 34

Populations Served Across States.............................................................................. 34

State Application Process Survey Results................................................................... 34

State Self-Evaluation Survey Results.......................................................................... 34

State Outcomes Evaluation Survey Results................................................................. 34

COVID-19 Impacts on States..................................................................................... 34

Appendix G: Reporting Period for Applications and Reports Used..................................... 36

Appendix H: PRC Evaluation – Outcome Toolkit Review Memo....................................... 37

Background.............................................................................................................. 37

Process and Outcomes Evaluation Toolkits................................................................ 37

The Urban Institute: Key Steps in Outcome Management......................................................................37

Administration on Aging (AoA) Performance Measurement Toolkit....................................................38

Colorado Nonprofit Association Principles and Practices Evaluation Toolkit.......................................38

Recommendations..................................................................................................... 39

Appendix I: New Reeve Foundation National Paralysis Resource Center Grant Program to 

Address Social Isolation During the COVID-19 Pandemic.......................................... 40

Program Rationale................................................................................................... 40

Grants Program....................................................................................................... 40

Eligibility................................................................................................................. 41

Application Process.................................................................................................. 41

Program Timeline..................................................................................................... 41
Paralysis Resource Center Evaluation: Final Report

1

Executive Summary

The Administration for Community Living (ACL) supports people living with spinal cord injury, 

paralysis, and mobility-related disabilities and their families through the National Paralysis 

Resource Center (NPRC), which uses a competitive process to solicit, objectively evaluate, select, 

and make subawards to direct service providers (referred to as subawardees in this report). ACL 

launched the Paralysis Resource Center (PRC) State Pilot Program Grants in 2018 and funded four 

pilot programs in Kentucky, Pennsylvania, Ohio, and Texas to make similar subawards to 

community-based organizations in their states.

ACL contracted with The Lewin Group, as a subcontractor to New Editions Consulting, to 

research and evaluate the effectiveness of administering grants at the state level compared to the 

national level. The two-year evaluation included formative/process and outcomes/impact

components to determine if there is a difference in the effectiveness of the grants between 

subawards overseen at the state and national level. Data were collected from subawardee 

applications, progress reports at the state and national level, semi-structured interviews conducted 

via conference call with the state pilots and the NPRC, a subawardee focus group report shared by 

the NPRC, and surveys of the subawardees to capture self-evaluation, application process, and selfreported outcomes.

Findings were aggregated and presented on the following topics: 1) Application Focus, Types of 

Projects, and Populations Served, 2) Outreach, 3) Administration, 4) Data and Reporting, 5) 

Challenges and Barriers, 6) Accomplishments and Outcomes, and 7) Effects of COVID-19.

Overall, the evaluation found that PRC programs overseen at both the state and national level 

improve the lives of people living with paralysis, along with their families and communities. The 

state pilots and NPRC are motivated to provide quality technical assistance and support to their 

subawardees. The state pilot subawardees and NPRC grantees demonstrated an impressive ability 

to pivot due to the COVID-19 pandemic through ingenuity and support from their funders. While 

acknowledging that the state pilots were part of a new program that was developing as they were 

operating, the state pilots did not show substantial enough benefits over the NPRC in either 

implementation or outcomes to warrant a wholesale change in how ACL administers the PRC 

grants program. The state pilots’ experience did, however, provide valuable insight into potential 

revisions of the Quality of Life grant program approach. Based on the overall findings, the 

following recommendations are made to ACL: 

· Improve outcome collection and reporting through technical assistance to 

subawardees and increased tracking of reported metrics. Technical assistance to 

subawardees on program and participant-level outcomes could be built into future funding 

opportunities and include tools to guide development of collection processes. Inconsistency

of reporting could be improved by increased tracking of outcomes submitted in semiannual reports and follow-up when metrics are missing. 

· Continue to ensure close contact with subawardees to maintain engagement and 

satisfaction. Both the state pilots and NPRC subawardees reported high satisfaction in with 

their engagement and individualized support. 

· Encourage an applicant selection process for future funding cycles like that used by 

the NPRC. The robust process supports an effective application selection process worth 

encouraging for use in future funding cycles.

· Include stakeholder involvement as a criterion in the rubric used by the entity 

granting subawards. The state pilot experience showed how meaningful stakeholder 

involvement is key to ensure that the paralysis community has a say in the types of 

programs funded.

· Consider regional or other geographically based collaboration through a network 

lead entity model to maximize benefits of a state-like model. Add a regional network 

lead entity grant-type that would elevate community priorities beyond individual local-level 

organizations and promote collaboration across a region—a benefit of the state pilot model.

· Consider adding capacity-building grants targeted at organizations that have not yet 

had success in securing grant funding. Learnings from the state pilots and NPRC indicate 

that grant administration is an important technical assistance need. ACL could invest in the 

broader network by offering small capacity-building grants with a strong technical 

assistance component and the added incentive of priority consideration in upcoming 

funding cycles upon completion of the capacity-building grant goals.
Paralysis Resource Center Evaluation: Final Report

2

Introduction

Background

The Administration for Community Living (ACL) supports people living with spinal cord injury, 

paralysis, and mobility-related disabilities and their families through the National Paralysis 

Resource Center (NPRC) and Paralysis Resource Center State Pilot Program Grants. ACL has 

funded theChristopher & Dana Reeve Foundation (CDRF) to manage the NPRC since 2014.

1 The 

NPRC mission is to improve the quality of life for people living with paralysis through an array of

complimentary and synergistic services, including the Peer & Family Support Program, the 

Military & Veterans Program, Advocacy & Policy programs, and The Quality of Life grant 

program. The Quality of Life grant program awards grants to community-based organizations to 

implement programs designed to improve the health, wellness, and independence of people living 

with paralysis and mobility impairments. External 

evaluators from Vanderbilt University evaluate all 

of the NPRC’s Quality of Life grant programs, 

services, and materials.

In 2018, ACL launched the Paralysis Resource 

Center State Pilot Program Grants. Similar to the 

national Quality of Life grant program, state pilot 

programs use funding from ACL to make 

subawards to community-based organizations 

(referred to as subawardees in this report) 

experienced in providing services and supports to 

individuals with paralysis and their families and 

support networks. These subawards are intended 

to help people with paralysis live more 

independently and integrate more fully into their 

communities. ACL awarded grants to state pilot 

programs, that, in turn, use a competitive process 

to solicit, objectively evaluate, select, and make 

subawards to direct service providers throughout 

their state. ACL solicited applications for up to 

five state pilots in 2018 and awarded two grants: 

University of Kentucky Human Development 

Institute and the Texas State Independent Living 

Council. ACL awarded two additional grants in 

2019 to the State of Pennsylvania Paralysis 

Resource Center and the Ohio Statewide 

Independent Living Council. 

The funding opportunity announcements (FOAs) for both the Paralysis Center State Pilot Program 

Grants grant cycles included three objectives:

1. Use 65% of the funding for subawards to community-based disability organizationsin an 

effort to increase the capacity of states to serve people with paralysis, their families, and 

their support networks. 

2. Measure performance of the state and organizations’ proposed activities.

3. Collaborate and partner with other organizations, agencies, and programs that have similar 

goals, policy initiatives, and social practice.

Purpose

ACL contracted with The Lewin Group (Lewin), as a subcontractor to New Editions Consulting, to 

research and evaluate the effectiveness of administering grants at the state level compared to the 

national level. The evaluation ran from November 2019 to September 2021. This report 

summarizes the findings.

The evaluation sought to answer the following questions:

1 Prior to 2014, the NPRC was housed at the Centers for Disease Control and Prevention.

State Pilot Application Focus

Kentucky: Recreational programs – build 

capacity of local communities to enhance 

and facilitate access to recreational 

programs, so they might effectively serve 

individuals with paralysis and their support 

networks.

Pennsylvania: Social determinants of health

– improve the quality of life of 

Pennsylvanians living with paralysis and 

their caregivers by improving or increasing 

the services and supports offered by 

community-based disability organizations.

Ohio: Emergency preparedness and training

– help Ohioans with paralysis increase their 

skills and knowledge of emergency/disaster 

preparedness, increase the capacity of 

community-based organizations to serve 

individuals with paralysis, and equip family 

and caregivers to support the individual with 

paralysis in an emergency/disaster situation.

Texas: Virtual independent living services –

leverage a telehealth platform to provide 

Independent Living Services to Texans 

living with paralysis in rural areas. 
Paralysis Resource Center Evaluation: Final Report

3

§ Is there a difference in the impact for people living with paralysis between subawards 

overseen at the state or national level (impact and efficiency)? 

§ How do state and national oversight affect the number of people reached, quality of service 

provided, and matching the service to need?

The evaluation includes two components: (1) formative and process, and (2) outcomes and impact.

Appendix A lists evaluation questions by component.

As discussed in the findings of this report, the evaluation found that PRC programs overseen at 

both the state and national level improve the lives of people living with paralysis, along with their 

families and communities. The state pilots and NPRC also provided quality oversight to their 

subawardees. While there are meaningful and consequential lessons learned from the state pilot 

programs, overall, they did not show substantial enough benefits over the NPRC in either 

implementation or outcomes to warrant a wholesale change in how ACL administers the PRC 

grants program. 

Evaluation Design

This evaluation was designed to allow for comparison with a separate evaluation of the NPRC

conducted by Vanderbilt University. In the fall of 2019, Lewin performed a systematic review of 

the state pilot and subawardee applications and progress reports and the summary reports 

Vanderbilt University shared on the NPRC grantees. Based on this review, Lewin designed a twoyear study to address ACL’s evaluation questions. The evaluation consists of three data collection 

activities: (1) interviews, (2) surveys, and (3) application and report reviews. Data collection 

occurred during both Year One and Year Two of the evaluation and is denoted as such throughout

the findings.

The data collection survey instruments aligned with those developed by Vanderbilt University for 

the evaluation of the national Quality of Life Direct Effect grants. The NRPC has multiple types of 

Quality of Life grants in addition to Direct Effect. These include High Impact Priority, Expanded 

Impact, High Impact Innovative Assistive Technology, and a new grants program added in 

response to the COVID-19 pandemic called COVID-19: Addressing Social Isolation. This 

evaluation primarily uses data on the Direct Effect grants as these are the most similar to the state 

pilot subawardees. However, some metrics reviewed (e.g., populations served) were provided on 

all Quality of Life grant programs. This is noted where appropriate.

Data Collection and Analysis

Applications and Reports

Applications provided information about the services proposed at both the state pilot level and 

the subawardee level. Progress reports were examined to measure NPRC and state grantees’ 

progress across six outcome metrics, discussed in the Data and Reporting section. High level 

themes were also identified from each document to allow for comparison between the state pilot 

grantees and the NPRC. The table below shows the number of applications and reports reviewed 

for each state and for the NPRC. 

Table 1. Reviewed Applications and Progress Reports

PRC Subawardee 

Applications Progress Reports

Kentucky 16 5

Ohio 8 3

Pennsylvania 12 3

Texas 6 4

National N/A 7

In addition to a project work plan, proposed budget information and justification, and other 

supporting documentation, each state PRC’s application contained a project narrative that 

described how the PRC would meet the three objectives ACL called for in their initial FOA: 

1. Providing subawards to community-based disability organizations

2. Measuring performance

3. Collaborating and partnering with other organizations, agencies, and programs
Paralysis Resource Center Evaluation: Final Report

4

Each narrative detailed the relevance and need for ACL funding, the proposed intervention and 

technical assistance offered to subawardees, organizational capability, plan for identifying and 

reviewing subawardees, partnerships, outreach and marketing strategy, evaluation/performance 

metrics, and project management structure. The NPRC application followed a similar narrative 

structure. The subawardee applications varied in their content, depending on the state PRC’s 

application criteria. However, most subawardee applications described the proposed intervention, 

potential partners, outreach strategy, budget, and organizational qualifications. 

Using the information detailed in the state pilot and subawardee applications, the following 

topics were identified for further analysis: application focus, outreach, administration, data and 

reporting, challenges and barriers, accomplishments and outcomes, and effects of COVID-19. 

For each progress report, state awardees describe their accomplishments during the reporting 

period, including status on particular project objectives and tasks, challenges and barriers faced, 

products produced, and the Quality of Life subawards. The most recent state pilot progress

reports available were from the last half of calendar year 2020.

Interviews

Lewin conducted semi-structured interviews via conference call with each of the state pilot 

programs during Year One of the evaluation in May 2020. Two members of the Lewin evaluation 

team and the ACL Project Officer attended each interview. In March 2020, Vanderbilt University 

staff conducted focus groups with national Quality of Life program grantees, notes on which were 

shared with Lewin. The purpose of the focus groups was to gain insight into the experience of 

grantees and identify best practices and potential areas for growth. With ACL, Lewin developed 

interview questions (Appendix B – Year One) that aligned with focus group questions used with 

the national Quality of Life program grantees to allow for comparative analysis. State pilot 

interview notes were analyzed and aggregated first by theme (e.g., outreach, recruitment) across 

the four states and then cross walked with the Vanderbilt University summary for comparison.

During Year Two of the evaluation, Lewin conducted one joint interview with all four state pilot 

programs and representatives of the NRPC. Lewin worked with ACL to develop questions 

(Appendix B – Year Two) that would both follow up on Year One findings and address new 

learnings as the state pilots were further into their award cycles. The joint interview occurred in 

May 2021. 

Surveys

Lewin conducted surveys during Year One and Year Two with the state pilot programs and their 

funded subawardees to capture self-evaluation, application process, outcomes, and COVID-19 

impact information. The first survey ran from April 27, 2020 to May 15, 2020, and the second 

from March 13, 2021 to May 4, 2021. Vanderbilt University fielded similar surveys to the NPRC

subawardees in 2018 and 2019. The state survey questions aligned with the nationalsurvey, with 

the exception of the outcomes and COVID-19 sections. The state pilot and national survey 

questions can be found in Appendix C and Appendix D respectively.

In Year One, all four state pilot grantees and 26 of their 29 active subawardees submitted survey 

responses. In Year Two, the four state pilot grantees and 24 of their 37 active subawardees 

participated in the survey. All 34 of the active NPRC subawardees submitted responses to the 

national Quality of Life program grantee survey in 2018, and 118 of the 127 active subawardees 

submitted in 2019.2 The Likert scale answer choices were converted to numbers (as shown in 

Appendix E). Higher numbers denote a more positive response. Qualitative analysis was 

performed on the open text survey responses.

Tables with aggregated survey responses from the state pilot and NPRC subawardees are 

included throughout the Findings section below. Tables displaying responses from the four state 

pilots are included in Appendix F as there is not a directly comparable survey response from the 

NPRC. 

2 The survey results shared by the NPRC covered two cycles of funding in 2019, thus the higher number of respondents 

when compared to 2018.
Paralysis Resource Center Evaluation: Final Report

5

Findings

Application Focus, Types of Projects, and Populations Served

Comparing interviews, applications, and progress reports across the four state pilots showed that 

the focus of the applications ranged widely. The university-based pilot programs focused on 

enhancing and promoting existing recreational and social determinants of health programs—the 

latter being an area of research at one of the universities. Based on state plans developed from 

community needs assessments showing areas for new growth, the Statewide Independent Living 

Council (SILC)-based pilot programs focused on emergency preparedness and virtual independent 

living.

From the state pilot and NPRC surveys, a majority of both the state pilot and NPRC Direct Effect 

subawardees reported focusing on accessible physical activities and outdoor recreation. For 

additional detail on the types of subawardee projects, please refer to Table 2.

Table 2. Survey Response: Type of Projects by State and National Subawardees

Type of Project3

State 

Subawardees 

Year 1 (n=26)

State 

Subawardees 

Year 2 (n=24)4

NPRC

Subawardees 

Year 1

(n=34)

NPRC

Subawardees 

Year 2

(n=118)

Physical activities, fitness, 

outdoor recreation 13 (50%) 8 (33%) 10 (29%) 35 (30%)

Other: Continuity of care, 

camp, virtual workshops and 

discussion forums, consumer 

education, advocacy

4 (15%) 7 (29%) 4 (12%) 14 (12%)

Emergency preparedness 3 (12%) 3 (13%) 0 (0%) 0 (0%)

Art, creative 2 (8%) 3 (13%) 3 (9%) 4 (3%)

Facility accessibility 

modifications, transportation 1 (4%) 0 (0%) 4 (12%) 37 (31%)

Therapy: 

Physical/occupational, service 

animal, horseback riding

0 (0%) 0 (0%) 6 (18%) 14 (12%)

Healthcare: Transition from 

institution to home, durable 

medical equipment, caregiving

0 (0%) 1 (4%) 4 (12%) 7 (6%)

Assistive technology initiative 0 (0%) 2 (8%) 3 (9%) 7 (6%)

Gathered through the state pilot survey and the NPRC semi-annual progress reports, information 

on populations served by state pilot and NPRC subawardees is shown in Table 3. (Please note, 

the change in “n” from Table 2 to Table 3 for the NPRC subawardees is due to the source. 

Table 2 data come from the survey and Table 3 data from the semi-annual progress reports.) A 

greater percentage of state pilots subawardees indicated serving rural populations, and a slightly 

higher percentage reported serving racial/ethnic minority groups and medically underserved

populations. Across the board, there are not dramatic differences between the state pilot and 

NPRC subawardees. 

Table 3. Populations Served by State and NPRC5 Subawardees

Underserved 

Populations Reached

State 

Subawardees

Year 1 (n=26)

State 

Subawardees 

Year 2 (n=24)

NPRC 

Subawardees 

Year 16(n=133)

NPRC 

Subawardees 

Year 27(n=80)

3 The state subawardee columns werepopulated using the subawardees’free responses on Community Impact.Three 

respondents did not indicate the type of project in their response. The national subawardee column was populated using 

the multiple choice responses to the national survey’s designated Type of Project question. 

4 There were 11 subawardees that were funded in Year 1 and Year 2. There were 13 new subawardees that responded 

to the survey in Year 2. 

5 NPRC QoL grantees serve additional underserved populations, including newly injured or diagnosed persons with 

paralysis, military services members and/or Veterans, and limited English proficiency populations, among others. The 

populations in Table 3 are included for direct comparison across the state and national subawardees.
Paralysis Resource Center Evaluation: Final Report

6

Underserved 

Populations Reached

State 

Subawardees

Year 1 (n=26)

State 

Subawardees 

Year 2 (n=24)

NPRC 

Subawardees 

Year 16(n=133)

NPRC 

Subawardees 

Year 27(n=80)

Rural population 21 (81%) 18 (75%) 55 (41%) 32 (40%)

At or below 150% of 

poverty 10 (38%) 18 (75%) 94 (71%) 47 (59%)

Racial/Ethnic 

minority groups 16 (62%) 16 (66%) 66 (50%) 29 (36%)

Medically 

underserved8 Not collected 15 (62%) 48 (36%) Not collected

LGBTQ+ populations 6 (23%) 9 (37%) 31 (23%) 20 (25%)

Outreach

Each state pilot PRC application and most subawardee applications mentioned several outreach 

strategies, including how they would use existing partnerships to recruit applicants or 

participants. To conduct widespread outreach, state pilots noted that they were marketing their 

program through email listservs, social media campaigns, and by posting information on their 

websites. The state pilots all noted that they would leverage relationships with community-based 

and statewide organizations, including Centers for Independent Living, Area Agencies on Aging, 

state and national disability organizations, and other local organizations relevant to the 

intervention (e.g., adaptive sport organizations in states focused on improving access to 

recreational activities). Looking back during the Year Two group interview, the state pilots 

tended to agree that social media and listserv announcements were less effective at enticing 

applicants than word-of-mouth networking. 

At the subawardee level, organizations noted their relationships with local health organizations, 

hospitals, colleges and universities, schools, and other community-based organizations as an 

avenue to promote their proposed activities. In some cases, the subawardees’ facilities were colocated with a rehabilitation center or fitness center, thereby increasing community awareness 

and accessibility of their respective programs.

University-based state pilot programs shared that they had success in attracting a range of 

applications for the subawards. They used word-of-mouth with existing contacts, including 

advisory boards, to promote the funding opportunity. Their existing structures appeared to support 

the grant outreach. One university-based pilot mentioned conducting conference calls to describe 

the funding opportunity, while two state pilots noted employing direct calls to contacts in the 

disability and aging community, media, and community-based organizationsfor targeted 

outreach. One university-based pilot also asked each key personnel, Advisory Board member, 

and supporting organization to recruit at least five potential applicants per year to ensure a large 

pool of potential subawardees. This same pilot asked subawardees to note how they heard about 

the funding opportunity in their application, allowing them to determine which elements of their 

recruitment strategy were most successful. 

The SILC-based state pilot programs reported during the Year One interviews some difficulty in 

attracting an adequate number of applicants. Both SILCs reported plans to increase the number of 

applicants in future funding announcements through additional means of outreach, including social 

media and cross-dissemination with agencies outside of their network. Both also expected greater 

interest from future applicants due to the impact of the pandemic on their areas of focus—

emergency preparedness and virtual independent living. The Texas SILC shared during the May 

2021 group interview that the pandemic increased understanding of and interest in their virtual 

independent living services. Ohio shared that while the pandemic highlighted the importance of 

their emergency and disaster preparedness efforts, it also necessitated that organizations shift their 

focus to addressing immediate impacts of COVID-19 rather than planning for future emergencies. 

6 Retrieved from NPRC’s Semi-Annual Performance Report dated July 2020 and includes all QoL Grantees, not only 

Direct Effect. 

7 Retrieved from NPRC’s Semi-Annual Performance Report dated January 2021 and includes all QoL Grantees, not 

only Direct Effect.

8 Medically Underserved as a population was not included in the Year One state pilot subawardee survey and the 

NPRC’s Semi-Annual Performance Report dated January 2021. 
Paralysis Resource Center Evaluation: Final Report

7

Subawardees’ noted their use of social media platforms and listservs to generate interest in their 

programs, as well as public service announcements, presentations or promotional events, and 

distributing flyers in their communities. Many also mentioned direct calls or emails to key 

community members and organizations, such as local schools and churches, previous clients, or 

providers that can refer applicable individuals to their program. The subawardee outreach 

strategies emphasized a grassroots approach that leveraged local community awareness and 

relationships to attract participants.

The NPRC mentioned use of similar outreach platforms to recruit applicants, including press 

releases and social media ads (e.g. through Facebook or blog posts). Of note, the NPRC sends 

funding announcements to all past applicants, regardless of award history, which is an expansive 

network after decades of operation. The NPRC also reports reaching out to major sports leagues 

(e.g., National Hockey League, National Football League) and other employment/vocational 

groups to advertise their programs in general. In their progress reports, the NPRC includes data 

for each outreach campaign, which demonstrates their wide reach. For example, the four press 

releases they used to announce the grant cycles were shared by 421 media outlets, resulting in 

23,557 views, while a Facebook ad posted between August 26, 2019 and October 11, 2019,

resulted in 30,259 unique views. During the following grant cycle, the NPRC received 644

applications, of which it funded 133. 

Administration

Application Process

State pilot grantees indicated that their subawardees required varying levels of technical assistance 

during the application process (covered in greater detail in the following section). The NPRC focus 

group summary noted that grantees reported confusion about eligibility requirements (e.g., 

limitations to spinal cord injuries only) and when they would be allowed to apply for additional 

grants. Participant quotes, however, do suggest general satisfaction with the process. For example, 

one NPRC grantee said, “the whole process of applying for the grant was a good one” and

mentioned attending a helpful webinar on the grant process. 

The survey results indicated generally high levels of satisfaction with the application process, 

clarity and ease of the application, and feasibility of the timeline across state and national 

subawardees(see Table 4). This was true across both Year One and Year Two surveys, with a 

slight increase in satisfaction seen in Year Two for both state and NPRC subawardees. 

Table 4. State and National Application ProcessSurvey Results

Application Process 

Metric

State 

Subawardees 

Year 1 (n=26)

State 

Subawardees 

Year 2 (n=24)

NPRC 

Subawardees

Year 1 (n=34)

NPRC 

Subawardees 

Year 2 (n=118)

How satisfied were you 

with:

5 Point Likert Scale Average

Application process 4.3 4.5 4.2 4.6

Clarity of the application 4.1 4.5 4.4 4.5

Ease of the application 4.2 4.5 4.2 4.3

1 Point Average (No=0, Yes=1)

Did you find that the 

timeline was feasible? 0.9 0.8 0.9 0.9

This was a new program for the state pilot programs. They built and refined their process 

throughout the pilot. The NPRC has been operating for many years and receives hundreds of 

applications each reporting cycle. The NPRC relies on a cadre of external reviewers, including 

people with paralysis, doctors and mental health professionals, lawyers, advocates, experts in 

assistive technology, and caregivers. The external reviewers sign a conflict-of-interest agreement 

each cycle to ensure impartiality. The reviewers identify strengths and weaknesses in the 

applications and provide feedback to NPRC through a scoring rubric. Applicants scoring above a 

set threshold on that rubric move to the next round. An internal review panel made up of board 

members and staff not associated with Quality of Life grants rank the applications and make 

recommendations on which to award. These recommendations are then accepted by the Quality of 

Life grant team.
Paralysis Resource Center Evaluation: Final Report

8

Technical Assistance Needs

In their applications, each state pilot described their offered technical assistance and products for 

subawardees. The two university-based PRCs define a structured and individualized technical 

assistance program, which includes monthly calls to discuss subawardees’ progress and to 

problem-solve challenges, and the development of targeted resources based on identified 

challenges and needs. The university-based pilots’ technical assistance also emphasizes a focus 

on sustainability to ensure subawardees can continue their programs beyond the grant period. 

SILC-based pilots noted that they opted to provide larger grants to fewer subawardees, with the 

idea that the higher funding amounts would lead to less need for

individualized technical assistance from the state level and allow 

for more flexibility at the subawardee level to develop and 

implement their approach. 

The SILC-based pilots primarily focused on resource development 

for their subawardees, as outlined in their state plans, with the 

hope that the resource guides would serve as an effective tool for 

future programs after the pilot ends. This included the creation and 

maintenance of a virtual services platform, the creation of toolkits, 

provision of information and dissemination of culturally 

competent materials, and the compilation of a resource guide that 

organizations can use to replicate the program’s success. One 

SILC-based pilot contracted with an external vendor for trainings 

for subawardees. In their semi-annual progress reports, state pilots 

mentioned ongoing technical assistance activities such as creating 

toolkits, developing and maintaining a centralized virtual site to 

serve as a repository for project resources during and beyond the 

project life span, and partnering with subject matter experts to 

provide guidance to subawardees.

The NPRC provides structured and robust technical assistance to 

their grantees. NPRC staff created and regularly update a 

comprehensive Application and Program Guidelines for grantees

that outlines grant-related policies and procedures. They also have 

a Process Manual for staff, which outlines processes, procedures, 

and policies related to carrying out the best practices in grants 

management. Additionally, the NPRC offers technical assistance 

webinars to help with both application and programmatic issues, 

and archives the webinars on their website for easy access. The 

NPRC creates a corresponding FAQ document using audiencegenerated questions for each webinar. To further understand 

grantee challenges and provide individualized technical assistance, 

the NPRC conducts site visits, which also helps them assess the 

impact of their technical assistance support on awardee programs

in order to refine their processes.

During the Year One interviews, all four state pilot programs 

reported subawardees’ technical assistance needs related to grant 

management. This included grant writing, budget narratives, 

completing invoices, workflow development, and processing 

contracts and reimbursement. Two state pilot programs reported 

that additional funding is the most frequent request from their 

subawardees. One pilot increased award amounts as a result, but 

continued to receive requests from subawardees for additional 

funds. Other technical assistance needs reported during the 

interviews were support with ensuring Americans with Disabilities 

Act (ADA) compliance within facilities and in online programming 

and ensuring privacy when using web-based platforms for virtual 

programming. All state pilot programs reported success in meeting 

technical assistance needs to subawardees’ satisfaction, with the 

exception of being able to provide additional funding. Subawardees 

noted in the Year Two survey that they were satisfied with the 

availability, easy accessibility, and regular communication 

ACL TECHNICAL 

ASSISTANCE

“We felt really supported. 

Especially with the 

turnaround in setting up the 

contracts ...that was helpful 

especially with the first year. 

And always knowing that we 

could call [the ACL Project 

Officer] and say, ‘here’s 

what’s going on’ and having 

that open line of 

communication. We felt like 

we were given the support to 

be successful.” ~Kentucky

State Grantee

“[The ACL Project Officer]

was always great as project 

officer, asking ‘What do you 

need from me? How can I 

help through any issues you 

have?’ Because of the 

openness from the very 

beginning...we never had 

any hesitation. If there was 

something that came up, we 

immediately thought, ‘Let’s 

email [the ACL Project 

Officer], let’s talk to her.’ 

There was never a hesitation 

on our part that we couldn’t 

come to her for any type of 

support.” ~Ohio State 

Grantee

“Even after all these years of 

our work in this field, I still 

go to [the ACL Project 

Officer] because it’s that kind 

of partnership and knowledge 

sharing that is so 

important...The kind of 

connections made with ACL 

and CILs, and state AT 

programs also come out of 

not only working with [the 

ACL Project Officer] as a 

program officer and but also 

as a colleague. Those 

connections have been really 

helpful for us.” ~ NPRC 

Grantee
Paralysis Resource Center Evaluation: Final Report

9

opportunities (e.g., monthly calls) with the state pilot programs. The Year Two interview surfaced 

similar themes, including an emphasis on the importance of flexibility and open communication to 

the subawardees. Two state pilots shared that they were grateful for flexibility from ACL allowing 

them to be flexible with their subawardees as they received requests to innovate or pivot their 

programs. Additionally, one state pilot shared that their subawardees requested support creating 

and disseminating marketing materials for their programs, and the NPRC shared that evaluation 

and outcome collection support were frequent requests from their subawardees during Year Two. 

The national Quality of Life program grantees focus groups report noted that grantees were “very 

pleased with the level of support, professionalism and the grants process itself” and that grantees

wished other funders operated similarly. They reported that grantees struggled to identify areas of 

support lacking by the NPRC, which implies a high level of satisfaction with the technical 

assistance received. However, the report also noted that levels of support needed by grantees vary

by organization, with smaller organizations or volunteer-based organizations needing more

assistance than larger organizations, which have support staff. Similar to state pilot subawardees, 

the NPRC grantees did note a desire for larger funding amounts and longer grant periods beyond 

the current 12-month parameter. In the open text comments in the Year Two survey, most NPRC

subawardeesindicated satisfaction with the grant process and implementation. Shared data on 

people with paralysis, proposal and budget training and assistance, renewability and flexibility with 

funding, more personalized communication, and simplifying the application were areas noted for 

improvement.

Data and Reporting

State pilots and the NPRC collect outcomes data through various means to measure the progress 

and impact of their funded projects. The NPRC and state pilot subawardees included their own 

measures in their applications. With regard to the state pilots, they proposed collecting data on 

event attendance, number of virtual visits to their online platforms, and tracking participant goals

and progress through pre and post-event surveys. The subawardees with projects focused on 

physical activity included additional measures such as pre/post-event physical health screenings, 

stress reduction surveys, speed and stamina tests, and frequency of equipment use. Similarly, the 

NPRC requires each subawardee to report on outcome measures, including indicators used to 

measure project success.

As part of the ACL grant awards, the NPRC and state pilots are required to collect information 

from their subawardees on six metrics during their semi-annual and annual reporting: 1) award 

rate, 2) percentage of awards completed on time and on budget, 3) cost of administering the 

subawards, 4) number of people proposed to be served compared to those actually served, 5)

satisfaction of the subawardees with the application and award process, and 6) satisfaction of the 

beneficiaries with the outcome of the subaward. The timing of reporting for the state grantees 

depends on when their grant cycles began. The reporting periods for the data in this report are

found in Appendix G. Below is a summary of findings from analysis of the six metrics as 

extracted from the state pilot and NPRC semi-annual and annual progress reports. 

1. Award rate is the number of awards as a percentage of the number of applications 

received. Table 6 displays the awards rates for the state pilot programs and the NPRC

across reporting periods analyzed. With the exception of Pennsylvania, the state pilots 

reported a much higher award rate than the NPRC. This aligns with findings from the 

interviews where state pilots shared challenges in attracting qualified applicants. 

Table 6. Award Rate for State Pilot Programs and the NPRC

State Pilot Programs NPRC Direct Effect 

Grants KY OH PA TX

16/23

(70%)

15/19

(79%)

12/55

(22%)

10/12

(83%)

105/471

(22%)

2. Percentage of awards completed on time and on budget is the number of 

subawardees completing on time and on budget as a percentage of the total number of 

awardees. Inclusion of this metric in the progress reports varied, with the frequency of

reporting increasing in the second year. Looking across report submissions, Kentucky
Paralysis Resource Center Evaluation: Final Report

10

reported 100%, Ohio reported 50% in their most recent report, Pennsylvania reported 

100% for their first subawardee cycle in their first two reports and 92% for their second 

cycle subawardees in their latest report, and Texas reported 100% and 50% in their last 

two reports. The NPRC reported on the number of grants closed and the number 

receiving no-cost extensions. Both the NPRC and state pilots shared in the interviews

that many subawardees would like longer project periods, feeling that one year is not 

sufficient time to both ramp up and implement a new project. 

3. Cost of administering the subawards is intended to measure the cost burden that the 

state pilots and the NPRC incur to administer the subawards. Reporting of this metric 

was inconsistent, with some state pilot progress reports providing dollar amounts and 

others sharing percentages of total budgets. The NPRC reports included the total award 

funds. Due to these inconsistencies, a comparison across programs is not helpful. 

However, from the interviews and survey comments, there was clear consensus from 

the state pilots that the administrative burden is higher than the 35% allowed.

4. Number of people proposed to be served throughout the life of the subaward 

compared to how many people are served is intended to measure the success of the 

programs in reaching their targeted number of people. Once again, reporting of this 

metric was inconsistent. The majority of progress reports either did not report any 

counts of people served or reported counts of individual events held by one or two 

subawardees—not counts across all. 

5. Level of satisfaction of the subawardees with the application and award process is 

intended to be measured by having respondentsrate the question “I am satisfied with 

the application process” on a scale from one to four with one being strongly disagree 

and four being strongly agree. In the progress reports, there was a wide range of 

variability in how this was reported. Kentucky shared a metric of 3.64 out of four in 

most progress reports, Ohio and Texas shared in their narratives that satisfactions levels 

are generally high. NPRC shared survey results on the metric conducted by Vanderbilt 

University, which indicated that over 90% of subawards are moderately to highly 

satisfied. The survey of state pilots subawardees conducted as part of this evaluation 

also measured satisfaction with the application and award process and indicated 

generally high levels of satisfaction—averaging over four points on a five-point scale 

(see Tables 4a and 4b for full results). 

6. Level of satisfaction of the beneficiaries with the outcome of the subaward is 

intended to be measured by having respondents rate the question “I am satisfied with 

this event” on a scale from one to four with one being strongly disagree and four being 

strongly agree. Similar to satisfaction with the application and award process above, 

this metric had inconsistent reporting. For example, Kentucky shared high satisfaction 

ratings from attendees of various events hosted by subawardees (e.g., 99% rated a three 

or four out of four; 16 out of 17 participants "strongly agreed" that they were satisfied 

with an event) and Texas shared that participants were generally satisfied. The survey 

of state pilots subawardees indirectly measures satisfaction of the beneficiaries by 

asking the subawardees to rate their programs impacts on community- and participantlevel outcomes. 

Two themes emerged from the findings on the six outcomes metrics. The first is that based on 

ACL staffing levels, the outcome metrics of this evaluation (e.g., cost of administering the 

subawards, number of people proposed to be served compared to those served, satisfaction 

measures) over the life of the grants were not closely tracked nor was there follow up with grantees 

that have not consistently been able to report one or more outcomes. While technical assistance to 

the state pilots and NPRC has clearly not suffered as a result, it may be beneficial to ACL to begin 

more rigorous tracking of outcomes in order to provide real-time feedback and assistance to the 

grantees in outcomes collection and reporting. 

Secondly, the challenges of collecting outcomes data is apparent – both from the content of the 

progress reports and discussions had with the state pilots, NPRC, and ACL. We heard that the 

number of people served and the level of satisfaction of beneficiaries are especially difficult to 

collect for programs that focus on community-level activities, such as accessible parks or increased 

access to other community resources. Grantees find it easier to track attendance and gather 

participant satisfaction data for programs that provide more time- and participant-defined activities 

(e.g., recreational classes). However, they still find it challenging to show through these attendance 

and satisfaction data that the activity or program had the intended impact on participants and the 

community at large. This is not limited to the paralysis community – it is a common challenge 
Paralysis Resource Center Evaluation: Final Report

11

across health and social service programs. There was also inconsistent understanding and 

collection of the outcomes included in the survey, which all state pilots expect to improve as they 

move into future funding cycles. For instance, one pilot program reported that they did not 

consider measuring “changes to people’s perceptions of persons with paralysis,” which was 

included in the fielded survey.

At ACL’s suggestion, we performed a brief search of publicly available outcome collection tools 

that could help the PRC grantees in measuring the impact of their programs. The findings from this 

search were submitted to ACL in a memo on May 10, 2021, which is included as Appendix H. 

Challenges and Barriers

Challenges noted in the progress reports submitted by state grantees included delays in program 

start-up, a smaller than expected number of subawardee applicants, lack of experience at the 

subawardee level, including problems with invoicing and budget management, difficulties 

related to technology, hesitance on the part of organizations to apply for a short-term grant, and 

uncertainty around sustainability. Two mitigation strategies employed by a state pilot grantee to 

reduce these barriers were informational webinars to simplify and explicitly define subawardee 

expectations and strategic consultation to subawardees about sustainability. Additional 

challenges noted in relation to the pandemic included pauses in programming, which led to 

overall delays in meeting grant objectives and requests for extensions to the grant timeframes. 

One state pilot said that the pandemic exacerbated challenges related to accessing technology in 

rural areas as more programming moved online. Another pilot shared that the proliferation of 

COVID-19 related funding streams, such as the Coronavirus Aid, Relief, and Economic Security

(CARES) Act, available to local organizations reduced the appeal of the Quality of Life grant 

funds. 

In their report, the NPRC noted one grantee’s failure to comply with their award’s terms and 

conditions, which resulted in targeted conversations, a site visit for corrective action, and

eventual termination of the grant. Other challenges at the NPRC subawardee level included

staffing issues (e.g. staff turnover), lack of funding, and delays in recruitment and set-up. The 

barriers the NPRC subawardees reported were more nuanced and specific to their program 

infrastructure or community compared to the broader issues of experience and sustainability 

noted in the state pilot progress reports. However, awardees in both the state and NPRCs 

expressed similar concerns around short-term funding and program delays.

During the interviews, a state pilotshared that the pilot program requirement for in-state 

subawardees restricted their ability to respond to regional interests. They received inquiries from 

organizations in neighboring states, but were unable to include their programs. They suggested 

considering a regional approach, rather than adhering to strict state boundaries. Another pilot 

program also reported mild disappointment with the state boundary restriction, as there were 

interested applicants that were declined due to being in a neighboring state.

Three of the state pilot grantees acknowledged during the Year One interviews that budget and 

time limitations restrict subawardees’ ability to show outcomes (e.g., successful implementation 

may take five years or more to show). One state pilot also expressed concern that the pandemic 

would hamper subawardees’ ability to collect outcome data, as they use an in-person method. 

However, all four state pilots reported that they and their subawardees engaged in creative 

brainstorming to pivot due to the pandemic, both in providing programming and measuring 

outcomes. Subawardees have had to cancel or shift their program offerings due to the pandemic, 

which greatly limits their ability to serve individuals with paralysis and carry out their 

interventions.

During the Year Two group interview, there was broad consensus across the state pilots and 

NPRC that the one-year timeframe for the grants is insufficient for subawardees to stand-up a 

successful program. One grantee noted that even experienced contractors need more start-up 

time for planning. 

Accomplishments and Outcomes

Overall, reports from state pilots noted their ability to follow through with their objectives. As 

part of the initial funding process, they formed advisory panels to review their subawardees’

applications and make funding decisions. Support for subawardees included holding regular 

check-in calls, meetings, trainings, and connecting the subawardees with useful resources. State 
Paralysis Resource Center Evaluation: Final Report

12

pilots also reported creating databases and instruments for collecting outcomes data, though the 

results of this data collection were not clearly seen.

State subawardees noted their 

ability to expand their community 

reach, purchase quality adaptive 

equipment, and expand awareness 

of their programs through media 

coverage and advertising. They 

also saw measurable physical and 

mental health improvements 

among those that they served 

through their programs, including 

weight loss, gained strength, and 

positive changes in participants’ 

outlook on life. Community members 

also noted there were increased 

opportunities to socialize with other attendees at events.

Similar to the state pilots, the NPRC and their subawardees reported accomplishing their 

objectives. The NPRC reported successful technical assistance activities, including hosting 

webinars and distributing a wide array of resources, and high levels of engagement and 

satisfaction from subawardees and program participants. NPRC subawardeessuccessfully 

established their programs within their communities by hosting various in-person events, funding 

scholarships, and buying adaptive sports equipment. They reported an increase in participant 

engagement and community awareness of accessible resources. Additionally, several NPRC 

subawardees noted increased collaboration with other organizations in their community that 

improved existing programs and raised additional funds.

Generally, the self-evaluation survey metrics were similar across state and NPRC subawardees 

for both Years One and Two of the evaluation. Except for Increased Decision-Making Skills on 

the Year Two state subawardee survey, all metrics averaged a 4.5 out of seven or above on the 

Likert Scale. This indicates that most subawardees did see at least some impact of their programs 

across the measured metrics.Comparing Year One survey responses, the NPRC subawardees 

reported higher ratings than the state subawardees for eight of the 13 self-evaluation metrics. The 

state subawardees had higher ratings for Community Integration, Increased Knowledge of 

Resources, Increased Access to Community Resources and Increased Self-Determination 

metrics. Comparing Year Two survey responses, the NPRC subawardees reported higher ratings 

than the state subawardees for seven of the 13 metrics. Please refer to Table 8 for more 

information.

Table 8. State Pilot and National Subawardees Self-Evaluation Section Results

Word cloud from May 2021 interview with state pilots and NPRC

Subawardee Self-Evaluation 

Metric

State 

Subawardee 

Responses 

Year 1 (n=26)

State 

Subawardee 

Responses 

Year 2 

(n=24)

NPRC 

Responses 

Year 1 

(n=34)

NPRC 

Responses 

Year 2

(n=118)

Self-Evaluation: The grant/project 

has had significant impact on: 7 Point Likert Scale Average

Changes to People's Perception of 

Persons with Paralysis 4.9 4.9 5.2 4.9

Community Integration 5.7 5.3 5.6 5.7

Existing Collaborations 5.8 6.2 6.0 5.6

Functional Independence 5.2 5.0 6.0 5.4

Improvement in Abilities or Skills 5.1 4.6 5.9 5.4

Increased Access to Community 

Resources 6.0 5.6 5.8 5.1

Increased Community Interaction 5.9 5.3 6.1 5.5

Increased Decision-Making Skills 5.3 4.3 5.1 4.5

Increased Health Satisfaction/Status 4.9 5.4 6.0 4.8

Increased Knowledge of Resources 6.0 5.6 5.9 5.6

Increased Self-Determination 5.6 5.3 6.3 5.5
Paralysis Resource Center Evaluation: Final Report

13

Subawardee Self-Evaluation 

Metric

State 

Subawardee 

Responses 

Year 1 (n=26)

State 

Subawardee 

Responses 

Year 2 

(n=24)

NPRC 

Responses 

Year 1 

(n=34)

NPRC 

Responses 

Year 2

(n=118)

New Collaborations 5.5 5.8 5.5 5.2

Quality of Life 5.6 5.7 6.5 5.9

During the Year One interviews, all but one state pilot shared that they expect that the number of 

people reached through subawardees’ programs to meet or exceed the expected counts included in 

the subawardee applications. The one university-based pilot that does not expect to meet the 

numbers in the subawardees applications is perhaps the most negatively impacted by the pandemic 

as the programming relies primarily on in-person events. Achievements in other areas, however, 

are noted, including less tangible aspects, such as encouraging organizations to consider 

inclusiveness and supporting self-advocates to become leaders in these initiatives. 

The survey results also indicate that both state and national subawardees saw increases in

educational opportunities and awareness, more accessible and inclusive services, and building 

relationships in the community as consequential community impacts of their projects. 

Additionally, 44% of the NPRC subawardees noted receiving positive feedback on their 

activities in the Year One Survey and 36% of Year Two respondents reported empowering 

people to advocate for themselves. Full results are available in Table 9.

Table 9. Community Impact Question Results for State and National Subawardees

Community Impact Themes 

Surfaced from Narrative Survey 

Response

State 

Subawardees 

Year 1 (n=26)

State 

Subawardees 

Year 2 (n=24)

NPRC

Subawardees 

Year 1

(n=34)

NPRC

Subawardees 

Year 2

(n=118)

Increased educational 

opportunities and awareness 16 (62%) 10 (42%) 15 (44%) 42 (36%)

More accessible and inclusive 

services 9 (35%) 6 (25%) 15 (44%) 49 (42%)

Building and sustaining 

relationships among community 

members and professionals

9 (35%) 8 (33%) 13 (38%) 29 (25%)

Empowering people to advocate 

for themselves, become selfsufficient and build confidence7 (27%) 2 (8%) 9 (26%) 42 (36%)

Positive feedback to activities 2 (8%) 0 (0%) 15 (44%) 9 (8%)

Remove barriers like stigma, 

financial, transportation 2 (8%) 1 (4%) 0 (0%) 7 (6%)

Reduced isolation, increased 

virtual interaction 0 (0%) 4 (17%) 0 (0%) 8 (7%)

Resource provision to COVID 

pandemic affected individuals

with IDD

0 (0%) 1 (4%) 0 (0%) 0 (0%)

Improved participant health and 

sense of well-being 0 (0%) 5 (21%) 0 (0%) 11 (9%)

Participants felt proud and valued 0 (0%) 0 (0%) 0 (0%) 6 (5%)

More family and caregiver 

support 0 (0%) 0 (0%) 0 (0%) 12 (10%)

Three of the state pilot programs indicated that the sustainability of their subawardees is integral to 

their success. One pilot shared that the sustainability of their subawardees’ programs (e.g., 

equipment purchases, physical adaptation of recreational areas, connections made between 

organizations throughout the state) is a positive outcome that will benefit people living with 

paralysis for many years to come and that sustainability was an important topic of discussion 

during the application process. Two of the other pilot programs reported that the sustainability of 

their subawardees’ programs varies to an extent, but all have the goal to continue their 

programming after the grant funds are exhausted. 
Paralysis Resource Center Evaluation: Final Report

14

The focus group report from the national Quality of Life program grantees indicated that some 

participating grantees “struggled with being able to answer the impact their project has made on 

quality of life,” likely because their programs are still underway. They noted, however, that 

grantees believed that their programs align with and will ultimately support the priorities of the 

NPRC. For example, “We were able to provide a great experience for [a participant living in a 

nursing home] and enabled him to network and meet other people with spinal cord injury and other 

similar types of disabilities with mobility impairments. I think that was a great opportunity for him 

and for our program as well. We want to provide access to all the activities that we provide for 

those that we're working with.” Additionally, the NPRC captures the impact on beneficiaries 

through various campaigns such as “See Us” and the NPRC “Share Your Story.”

In the interviews, two state pilots shared that the close, personal contact they provide to their 

subawardees is a noted benefit of their approach. Their point of view is that this level of contact 

between funder and recipient would not be possible on a national scale. Interestingly, one 

acknowledged that they do not have the vast network of resources available at the national level 

and suggested a hybrid approach where the first round of funding would be at the state level, and 

then subsequent funding to sustain programs at the national level. Such a hybrid approach might 

also reduce the confusion at the local level about funding sources and timelines when state and 

national applications overlap. 

The SILC pilot programs both noted that the state approach allows for a targeted statewide 

initiative that is unique to their state’s needs and involves community and individual stakeholders 

across their states. They thought that this would not be achievable under the national approach, 

which funds local organizations independently. One SILC pilot shared that they thought that the 

focus of their initiative would wane and the impact dulled if community-based organizations were 

to apply for national grants individually. Understandably, the Vanderbilt University focus group 

participants were not asked about the benefits of the state approach compared to the national. The 

report summary and quotes from participants do indicate a high level of satisfaction with the 

national model. Grantees noted access to resources, responsiveness of the CDRF staff, and 

transparency in the grant process as benefits of the national program.

Effects of COVID-19

State Grantees

The COVID-19 pandemic significantly impacted the state pilot grantees and their subawardees

early on in 2020. As the situation has progressed, they have had to rethink ways to continue 

serving individuals with paralysis through inclusive and accessible activities. In the Year One 

survey, a majority of state subawardees recognized the following as consequences of the 

COVID-19 pandemic: delays in implementation of grant activities, changes in how the grant is 

implemented, and recipients unable to participate due to COVID-19 illness, caregiving, or stayat-home guidelines. The Year Two survey indicated similar results for delays in implementation 

of grant activities and changes in how the grant is implemented, but the percentage of 

subawardees reporting that recipients were unable to participate decreased from 50% to 37%. 

The survey included a space for subawardees to identify how they would like ACL to support 

them, given the far-reaching impacts of the pandemic. Of those that responded, five requested an 

extension on the timeline of their projects, two asked for more flexibility on the use of their 

funds, and two asked for increased funding in general. Table 10 provides additional detail on the 

impact of COVID-19 on the subawardees.

Table 10. COVID-19 Impacts on SubawardeeProjects

Please indicate how the COVID-19 response is affecting 

your grant:

State 

Subawardee 

Responses Year 

1 (n=26)

State 

Subawardee 

Responses Year 

2 (n=24)

Delaying implementation of grant activities 16 (62%) 15 (62%)

Changing how the grant is implemented (e.g., in-person 

to virtual) 13 (50%) 15 (62%)

Recipients of grant activities unable to participate due to 

COVID-19 illness, caregiving, or “stay-at-home” 

guidelines

13 (50%) 9 (37%)
Paralysis Resource Center Evaluation: Final Report

15

Please indicate how the COVID-19 response is affecting 

your grant:

State 

Subawardee 

Responses Year 

1 (n=26)

State 

Subawardee 

Responses Year 

2 (n=24)

Staff unable to complete grant activities due to sickness, 

caregiving, or “stay-at-home” guidelines 7 (27%) 4 (16%)

Staff time diverted away from grant activities to other 

priorities 5 (19%) 3 (12%)

Other:

Funding diverted away from/insufficient for grant 

activities 3 (12%) --

Planning team are first responders to the pandemic and 

have had to step back from conference planning. 1 (4%) --

Gathering limitations -- 2 (8%)

Unable to attend site -- 1 (4%)

Fear of COVID-19 during planned activities has 

impacted the participation -- 1 (4%)

Closed facilities where activities are held -- 1 (4%)

Change in services provided through grant -- 1 (4%)

Team turnover -- 1 (4%)

No effect -- 1 (4%)

Kentucky’s Wellness Edge noted that their grant has been affected mainly by delays in 

implementation of activities and changes from in-person to virtual events. In an effort to engage 

with individuals with paralysis and their caregivers, Wellness Edge held two virtual events in April 

and June 2020 called “Staying Apart Together.” The events included activities such as yoga, 

Zumba, virtual arts, virtual peer meetings, tutorials to use Google Hangouts, CrossFit, instructions 

for social distancing in parks, and health check-ins. Over the two events, a total of 771 individuals 

registered and 171 provided feedback during the interspersed polling sessions. Overall, the 

participants were highly satisfied with the events and enjoyed interacting and learning with one 

another. 

With their original in-person events canceled, Kentucky subawardees sought technical assistance 

from Wellness Edge to adapt their work. One subawardee, Gathering Strength, provides and 

improves access to physical activity and recreation opportunities for people with paralysis. They 

had planned on holding a symposium, flying in speakers who were experts on spinal cord injury,

and various physical recreational activities. Since they did not have the means to move the 

symposium online, Wellness Edge helped transition the event to a virtual platform. The speakers 

were online, and they included videos of yoga and other demonstrations of physical activities to 

engage the participants. They noted that the attendance was so high that they redirected the extra 

Zoom participants onto the YouTube livestream. 

Kentucky’s IDEAS xLab’s Universal Art program holds art classes for individuals with paralysis. 

They canceled their in-person event and uploaded video tutorials online for people to follow at 

their own pace. They also allowed people to sign up for at-home art kits with adaptive materials. 

Their initial presentation on the event and videos have received a lot of positive feedback, and they 

are planning a virtual art show for the participants. The in-person event had transportation and 

building-access barriers; so, moving the classes online ameliorated these issues altogether. 

Other state pilots and their subawardees have also changed course during this time. In 

Pennsylvania, subawardees like Fighting Back Scholarship and the Center for Independent Living 

Services PA have been conducting virtual health and training sessions. One of Ohio’s emergency 

preparedness subawardees has moved their classes online, and have found the content of their 

webinar materials very relevant to the pandemic. Since Texas focused their grant on virtual 

services, they have been able to continue their activities online and have noted an increase in 

participation overall, which they attribute to the pandemic.

National Paralysis Resource Center Grantees

The NPRC grantees have been similarly impacted by the pandemic. Of the 54 grantees receiving 

2

nd Cycle 2019 funding (grant period January to December 2020), 49 experienced closures of 

parks, pools, beaches, community centers, and other facilities, which resulted in a delay or 

suspension of programming. Since most projects offer in-person and/or on-site activities and 
Paralysis Resource Center Evaluation: Final Report

16

services, the number of individuals served and outcomes data will be affected and may be 

compounded by a continued period of inactivity. However, nine programs noted in their interim 

reports that they were able to transition to virtual programming and six programs were exploring 

other delivery options. The NPRC worked with two grantees to provide no-cost extensions related 

to COVID-19 and five grantees to sanction scope and/or budget changes related to COVID-19 

(e.g., purchase of iPads to enable participation in virtual programming).

Of the 73 grantees receiving 1st Cycle 2019 funding (grant period June 2019 to May 2020), 29 

experienced closures of parks, playgrounds, and other facilities that impacted programming. The 

closures affected the numbers of individuals served and outcomes data collected. For example, an 

adaptive winter sports program had to close three weeks earlier than planned and had to cancel 

over 250 lessons. Many of the organizations pivoted during the last few months of their grant to 

offer virtual programming. NPRC supported 10 grantees in no-cost extensions, of which five also 

had budget revisions due to COVID-19 (i.e., moving to virtual programming).

In early August 2020, the NPRC was scheduled to launch a new Opioid Prevention Grants 

Program to improve the well-being and outcomes for people living with paralysis affected by 

opioids and Opioid Use Disorder with the goal of prevention of opioid misuse, overdose, and 

death. In response to the COVID-19 pandemic, ACL asked that they pivot as well and develop a 

COVID-19-related grants program.

The NPRC spoke with their grantees, CDRF Information Specialist staff, members, peers and 

mentors from the Peer and Family Support Program, and stakeholders in the field about COVID19 impacts. They researched articles and journals, and reviewed surveys and data collected from 

the CDRF and stakeholders in the field of paralysis research. All data indicated that social isolation 

has had the number one negative impact on the lives of people living with disabilities relative to 

the COVID-19 pandemic. Based on this data, the NPRC developed and launched a $300,000 

grants program to address social isolation during the COVID-19 pandemic with the goal of 

enhancing connectedness of people living with paralysis and their caregivers to their communities 

and preventing adverse health outcomes (see Appendix I).

Discussion and Recommendations

Overall, we found that the PRC programs improve the lives of people living with paralysis, along 

with their families and communities. The state pilots and NPRC are motivated to provide quality 

technical assistance and support to their subawardees. The state pilot subawardees and NPRC 

grantees demonstrated an impressive ability to pivot due to the COVID-19 pandemic through 

ingenuity and support from their funders.While acknowledging that the state pilots were part of a 

new program that was developing as they were operating, the state pilots did not show substantial 

benefits over the NPRC in either implementation or outcomes to warrant a wholesale change in 

how ACL administers the PRC grants program. The state pilots’ experience did, however, provide 

valuable insight into potential revisions of the Quality of Life grant program approach. 

Based on the findings, it is recommended that ACL:

9

· Improve outcome collection and reporting through technical assistance to 

subawardees and increased tracking of reported metrics. At the conclusion of the first 

year of the evaluation, the decision was made to focus the second year on understanding 

program outcomes, especially regarding the six metrics required by ACL in semi-annual 

reporting. During Year Two, we learned that outcomes are not consistently collected and 

reported. Reporting could be improved by increased tracking of outcomes submitted in 

semi-annual reports and follow-up when metrics are missing. However, the larger issue is 

the challenges involved in outcome collection by the subawardees and PRC program 

managers (i.e., the state pilots and NPRC). This is especially true of participant-level 

outcomes, such as attendance, participation, and satisfaction. The outcome collection tools 

describe in Appendix G may be helpful to PRC programs. A focus group and/or survey of 

past and current subawardees to surface ideas of how to improve outcome collection and 

9 ACL is in the process of considering how and to what extent to implement these recommendations. As of July 2021, 

ACL has made the following decisions as a response to the recommendations: 

· ExtendedtheNPRC Quality of Life grant period from 12 to 24 months

· Engaged development of a guide for program staff to use when planning for outcome data collection
Paralysis Resource Center Evaluation: Final Report

17

ways that funders can support this is another method. We also suggest that technical 

assistance on collecting outcomes be built intentionally into future funding opportunities, 

with meaningful ACL support to the PRC program managers (e.g., make resources and 

subject matter experts available to both the program managers and the subawardees 

through webinars, office hours, and/or a resource center). 

· Continue to ensure close contact with subawardees to maintain engagement and 

satisfaction. State pilots said a main benefit of their model is the “personal touch” and 

close contact they keep with subawardees. However, the NPRC focus group report and 

survey results indicate high satisfaction from the grantees in the responsiveness of NPRC 

staff – even mentioning an example of how they appreciate being able to get to know them 

by name. These findings suggest that, regardless of funder, close engagement and 

individualized attention are important.

· Encourage an applicant selection process for future funding cycles like that used by 

the NPRC. As discussed in the Application Process section of this report, the NPRC has a 

robust process for reviewing the hundreds of applications received each reporting cycle. 

Their process includes a cadre of external reviewers with lived experience and expertise as 

well as an internal review panel. They have protocols to ensure impartiality, such as 

Conflict of Interest agreements and selection of internal reviewers outside of the Quality of 

Life program. The diversity of reviewers and neutrality protocols support an effective 

application selection process are worth encouraging for use in future funding cycles. 

· Include stakeholder involvement as a criterion in the rubric used by the entity 

granting subawards.The state pilots indicated that the state PRC approach provides a 

means to elevate the impact of particular priorities across an entire state. For example, the 

Ohio SILC’s grant focus, based upon their board-approved state plan, is to create 

opportunities for Ohioans with paralysis to increase the education and development of 

skills needed to appropriately respond in the event of an emergency/disaster, including 

preparation, response during, and recovery in the aftermath. This SILC funded local 

organizations to further those goals, thereby creating an expanded, statewide approach that 

maximizes the impact on people with paralysis. The individual nature of the NPRC grants 

does not support the same level of statewide or regional prioritization and implementation.

The benefits of a statewide approach to people with paralysis relies on the state-level entity 

including all relevant stakeholders in decisions about the grant focus, or in the case of the 

SILCs, decisions about the state plan, which directly influenced the grant focus.

Meaningful stakeholder involvement would need to be a key criterion for future funding 

opportunities to ensure that the paralysis community has a say in the types of programs that 

are funded. 

· Consider regional or other geographically based collaboration through a network 

lead entity model to maximize benefits of a state-like model. To reap the benefits of the 

state model stated above (i.e., elevate community priorities beyond individual local-level 

organizations and promote collaboration across a region), we suggest the addition of a 

grant-type targeted to organizations that would serve as the regional network lead entity—a 

role that would involve granting subawards based on priorities identified by a diverse set of 

stakeholders and supporting collaboration among the regional subawardees. 

· Consider adding capacity-building grants targeted at organizations that have not yet 

had success in securing grant funding. The state pilots shared the types of technical 

assistance requests made of them by their subawardees throughout the life of the grant. By 

far, the most common type of request early on in funding cycles relate to grant 

administration. Basic information on budget justifications, invoicing, and other managerial 

functions of grant administration are not well understood by many community-based 

organizations, especially those without a history of federal grant funding. One of the 

reasons why the NPRC has been so successful in its grants is its thorough vetting process 

for subawardees—they fund organizations that are well positioned in the community with a 

history of successful programs. One criterion examined during the NPRC award process is 

the percentage of funds that would be attributed to the Quality of Life grant, if awarded. 

The idea being that organizations with diverse funding streams have higher chances of 

success. While this has been a successful approach (and we do not suggest changing it), it 

does remove organizations from consideration that may be successful, but need additional 

organizational capacity to get there. ACL could invest in the broader network by offering 

small capacity-building grants with a strong technical assistance component and the added 

incentive of priority consideration in upcoming funding cycles upon completion of the 

capacity-building grant goals. 
Paralysis Resource Center Evaluation: Final Report

18

Appendix A: Evaluation Questions

Component Evaluation Questions

Formative 

and Process

1. How and to what extent was ACL able to attract a significant pool of applicants?

2. How and to what extent were state grantees able to attract applicants and 

select organizations with a history of providing services and supports to 

individuals with paralysis and their families and support networks in order to 

help them live more independently, to more fully integrate into their 

communities?

3. What kinds of outreach did grantees conduct to ensure that a wide range of 

potential sub grantees were aware of the opportunity?

4. What types of support and technical assistance did states need from ACL? To 

what extent has ACL been able to meet these needs?

5. What types of support and technical assistance have subawardees needed from 

states? To what extent have states been able to meet these needs?

6. To what extent have state and subawardees been able to define, measure and 

report their outcomes?

7. To what extent have state grantees been able to report performance measures 

as required by the grant award?

8. How and to what extent have states and subawardees leveraged their local and 

regional networks and connections to facilitate program activities and advance 

program goals?

9. What kinds of challenges or barriers have states and subawardeesencountered? 

How do these relate to or compare with challenges or barriers encountered by 

the NPRC and its subawardees?

Outcomes 

and Impact

1. To what extent have the state and subawardee programs been able to carry out 

their intended work, complete planned activities, and achieve milestones?

2. To what extent have state and subawardeeproject goals been achieved?

3. To what extent did subawardees engage the target populations (e.g., volume, 

demographics)

4. To what extent were the services provided by subawardees useful/helpful to the 

individuals/organizations served? And, what types of programs, offerings or 

activities did participants find most useful, beneficial or impactful as reported 

through final and interim reports and focus group interviews?

5. How do state and subawardee program activities compare in terms of scale, 

reach, impact, administrative effort and cost to the NPRC activities? Categories 

of NPRC grants include:

i. Direct effect (Tier 1: adaptive sports, accessible playground/ball field, 

accessible community spaces, assistive technology, advocacy, arts, 

and so on), 

ii. High impact priority (Tier 2: transportation, respite caregiving, 

disaster preparedness; Tier 3: nursing home transition; Tier 4: 

employment), 

iii. Expanded effect (Tier 5: expansion of previously awarded Quality of 

Life grantee programs), and 

iv. High Impact Innovative Assistive Technology (HIIAT).
Paralysis Resource Center Evaluation: Final Report

19

Appendix B. Interview Questions

Year One

Formative and Process Questions

§ How did you decide what to focus on in your application for ACL? Did you talk to 

community partners or consumer advocates about it?

§ What types of outreach did you use to recruit and attract applicants for subawards? Are 

there things that you would do differently if you were to do it again?

§ What types of support and technical assistance have subawardees asked from you? To 

what extent have you been able to meet these needs? Are there ways ACL could 

support you with this?

Outcomes and Impact Questions

§ Do you think the subawardees have been (or will be) successful at engaging their target 

populations (e.g., volume, demographics)? Why or why not?

§ To what extent do you think the subawardees will be able to measure outcomes and 

impact for participants? Why or why not?

§ Has anything surprised you so far in working on this? Have you seen any unintended 

benefits or run into any unanticipated problems?

§ If your subawardees ultimately show positive outcomes for their participants, to what 

do you think this could be attributed? In other words, what do they have in place now 

or what are they doing well now that think will lead to their success (e.g., strong 

partnerships in community, effective outreach, responsiveness to individual needs)? To 

what extent will the projects funded through this grant be sustainable?

§ Think for a moment about this grant, which is administered at the state level, as 

compared to a grant that might be made directly from a national-level organization. 

What benefits and challenges do you see to each approach?

Year Two

Formative and Process Questions

§ Knowing what you know now, what would help attracting qualified applicants?

§ What types of outreach ended up being successful and what types didn’t work as well? 

§ What worked well or didn’t work well in terms of implementing these grants?

§ How did the COVID pandemic affect your outreach, recruitment, or implementation 

efforts?

§ Looking back over the last year or so, what types of support and technical assistance 

were most frequently requested from your sub-awardees? With the experience you 

now have, are there ways you would have modified your support to sub-awardees?

Were there ways ACL could support you with this?

Outcomes and Impact Questions

§ Thinking about all of your awardees – those that were most successful, less successful, 

had a slow start, but became successful – what do you think are the biggest predictors 

of which awardees will most successfully engage their target population and meet their 

project goals? 

§ We know that measuring outcomes and impacts on individuals living with paralysis 

can be difficult, especially for projects that do not have set attendees or timeframes 

(e.g., accessing a recreational area). In general, what successes or challenges have 

awardees faced in measuring outcomes and impact on individuals living with 

paralysis?

§ What would be needed (resources, tools, follow-up procedures, etc.) to:

o better measure outcomes of Quality of Life grantee programs, and 

o collect demographic data on participants?

§ To what do you attribute positive participant outcomes among successful awardees? In 

other words, what do they have in place now or what are they doing well now that has 

led to their success? 

§ To what extent will the projects funded through this grant be sustainable? 
Paralysis Resource Center Evaluation: Final Report

20

Appendix C: State Survey Questions 

The state pilot and subawardees questions were fielded through the same survey tool with skip 

logic for Year One and Year Two.The text in red indicates edits that were incorporated into the 

Year Two survey.

The following description was displayed to both state pilots and their subawardees:

The Administration for Community Living (ACL) provides support to people living with paralysis 

through Paralysis Resource Centers (PRCs) – one at the national level through the Christopher and 

Dana Reeve Foundation and four pilot programs at the state level. The four ongoing state pilot 

program grants are: 

1. State of Pennsylvania Paralysis Resource Center, 

2. Ohio Statewide Independent Living Council, 

3. University of Kentucky Human Development Institute, and 

4. Texas State Independent Living Council. 

In turn, the state PRCs use funding from ACL to make subawards to community based 

organizations (CBOs) experienced in providing services and supports to individuals with paralysis 

and their families and support networks. 

Your response to the survey is integral to ACL’s evaluation of the work done by the state and CBO 

PRCs to help determine the most effective way to administer this program. If you have any 

questions about the survey, please email Srinithi.Suresh@Lewin.com. 

Thank you for your participation!

State Pilot Program Survey Questions

Designation

1. Please designate your organization level.

o State Grantee

o Community-Based Organization receiving a grant from a State

State Grantee Information

2. Name of person completing this survey:

3. Email:

4. State:

o Kentucky

o Ohio

o Pennsylvania

o Texas

State Grantee Self Evaluation

In the following section, please choose the answer that most closely reflects the extent to which 

you agree or disagree with each statement.

5. Effect on Quality of Life: To date, funded projects have had a significant effect on quality 

of life for individuals with paralysis and/or their families.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

6. Effect on Functional Independence: To date, funded projects have had a significant effect 

on functional independence for individuals with paralysis and/or their families.

o Strongly Agree
Paralysis Resource Center Evaluation: Final Report

21

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

7. Community Integration: To date funded projects have had a significant effect on inclusion 

or community integration for individuals with paralysis.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

8. Increased Knowledge of Resources: To date, funded projects have led to increased 

knowledge or awareness of available resources for individuals with paralysis and/or their 

families.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

9. Access to Community Resources: To date, funded projects have led to increased access to 

community resources (e.g., financial, education, social) for individuals with paralysis 

and/or their families.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

10. Increased Decision-Making Skills: To date, funded projects have led to increased decisionmaking skills for individuals with paralysis and/or their families.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

11. Increased Self-Determination: To date, funded projects have led to increased selfdetermination for individuals with paralysis.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree
Paralysis Resource Center Evaluation: Final Report

22

o Strongly Disagree

o Unable to Determine

o N/A

12. Improvement in Abilities or Skills: To date, funded projects have led to improvement in 

abilities or skills for individuals with paralysis. (For example, improvement in employment 

and job skills/abilities, but it could also be used to capture improvement in physical abilities 

and skills such as core strength due to therapeutic horseback riding or even improved 

ability to ride a horse or sit up.)

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

13. Increased Community Interaction: To date, funded projects have increased the number of 

interactions of persons with paralysis and community members.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

14. Changes to People's Perception of Persons with Paralysis: To date, funded projects have 

affected the way people think about persons with paralysis.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

15. Increased Health Satisfaction: To date, funded projects have led to increased health 

satisfaction of individuals with paralysis.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

16. New Collaborations: To date, funded projects have led to NEW collaborations with your 

organization and other disability-related agencies, organizations, nonprofits.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A
Paralysis Resource Center Evaluation: Final Report

23

17. Existing Collaborations: To date, funded projects have led to furthering EXISTING 

collaborations with your organization and other disability related agencies and/or nonprofit 

organizations.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

18. Community Impact: In what ways have funded projects affected the community of persons 

with and without paralysis? Please provide a narrative response.

19. Underserved Populations: To date, funded projects have served traditionally underserved 

populations within the individuals with paralysis and family/caregiver community. Check 

all that apply to indicate the underserved populations served through funded projects.

o Rural

o At or below 150% of poverty

o Racial/Ethnic minority groups

o LGBTQ+ populations

o Medically underserved

20. How was the information for the above questions collected? Check all that apply.

o State grantee collected information directly from project beneficiaries (e.g., through 

a surveyor interview)

o Sub grantee collected information directly from project beneficiaries (e.g., through 

a surveyor interview)

o Administrative data (e.g., attendance log sheets)

ACL Application Process Evaluation

Please provide feedback so that we might improve the process in future years.

21. Application process: How satisfied were you with the application process?

o Very Satisfied

o Highly Satisfied

o Moderately Satisfied

o Slightly Satisfied

o Not at All Satisfied

o Unable to Judge

22. Clarity of the application: How satisfied were you with the clarity of the application?

o Very Satisfied

o Highly Satisfied

o Moderately Satisfied

o Slightly Satisfied

o Not at All Satisfied

o Unable to Judge

23. Ease of the application: How satisfied were you with the ease of filling out the application?

o Very Satisfied

o Highly Satisfied

o Moderately Satisfied

o Slightly Satisfied

o Not at All Satisfied

o Unable to Judge

24. Timeline/feasibility: Was the timeline of implementation feasible for the scope of funded 

projects?

o Yes

o No

25. Application support provided: How and in what way(s) does ACL support your 

organization through the grant process?

26. Application support provided (continued): To what extent do you find this support helpful?

27. Process improvement: How, if at all, can the grantmaking process be improved?
Paralysis Resource Center Evaluation: Final Report

24

Outcomes Evaluation

In the following section, please choose the answer that most closely reflects the extent to which 

you met the goals included in your application.

28. Served the intended types of people/groups (e.g., demographics, needs, geography)

o Significantly more than expected

o More than expected

o Fewer than expected

o Significantly fewer than expected

29. Served the intended number of people/groups.

o Significantly more than expected

o More than expected

o Fewer than expected

o Significantly fewer than expected

30. Met timeliness of project implementation

o Significantly more than expected

o More than expected

o Fewer than expected

o Significantly fewer than expected

31. Do you have anything else you would like to add?

Impact of COVID-19 on Grant Activities

Please answer the following questions thinking only about your activities at the state-level. Do not 

include effects of COVID-19 on your subawardees—only consider your staff activities within your 

state agency. 

32. Please indicate how the COVID-19 response is affecting your grant (select all that apply):

o Delaying implementation of grant activities

o Changing how the grant is implemented (e.g., in-person to virtual)

o Staff time diverted away from grant activities to other priorities

o Staff unable to complete grant activities due to sickness, caregiving, or “stay-athome” guidelines

o Funding diverted away from grant activities

o Funding insufficient to react to changing environment for grant activities due to 

COVID-19

o Other: _____________________

33. Please provide any other comments on the impact COVID-19 is having on your grant at the 

state-level:

34. What technical assistance do you expect that your subawardees will require as they support 

their participants’ reintegration into the community as stay-at-home orders and other 

restrictions are lifted?

35. Consider the changes and/or new practices that you and/or your subawardees have adopted 

in response to the COVID-19 pandemic. Are there any that you expect to keep in place 

following the pandemic? If so, please provide a brief summary below. 

State Subawardee Survey Questions

Designation

1. Please designate your organization level.

o State Grantee

o Community-Based Organization receiving a grant from a State

Community-Based Organization Information

2. Name of person completing this survey:
Paralysis Resource Center Evaluation: Final Report

25

3. Email:

4. State:

a. Kentucky

i. Bluegrass Tennis Association

ii. Center for Accessible Living

iii. Community Foundation Louisville

iv. Community Foundation of Louisville

v. Easter Seals Cardinal Hill

vi. Easter Seals – Cardinal Hill Adaptive Recreation

vii. Fayette County Public Schools

viii. IdeasX Lab

ix. Independence Place

x. KY PVA

xi. KY-IN PVA Chapter

xii. LFUCG – Division of Parks and Recreation

xiii. Spinal Bifida Association

xiv. The Ethan Foundation

xv. Warren Co. Parks

b. Ohio

i. Center for Disability Empowerment

ii. Linking Employment, Abilities, and Potential (LEAP)

iii. University of Cincinnati Center for Excellence in Developmental 

Disabilities (UCCEDD) at Cincinnati Children’s Hospital Medical Center 

(CCHMC)

c. Pennsylvania

i. Center for Independent Living

ii. Fighting Back Scholarship Program

iii. HOPE Network

iv. IM Able Foundation

v. Inglis Foundation 

vi. PA Center for Adapted Sports

d. Texas

i. Heart of Central Texas Independent Living (HOCTIL)

ii. Life Inc.

iii. Mounting Horizons

iv. NMD

v. National SCI Association Houston

vi. Valley Association for Independent Living (VAIL)

5. Organization Name:10

Community-Based Organization Self Evaluation

In the following section, please choose the answer that most closely reflects the extent to which 

you agree or disagree with each statement.

6. Effect on Quality of Life: To date, the project has had a significant effect on quality of life 

for individuals with paralysis and/or their families.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

7. Effect on Functional Independence: To date, the project has had a significant effect on 

functional independence for individuals with paralysis and/or their families.

10 During Year One, Lewin had the names of all subawardees to include as drop down options in the survey. During 

Year Two, all active subawardees were not known, so an open field for organization name was included in place of the 

drop down options with subawardee names. 
Paralysis Resource Center Evaluation: Final Report

26

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

8. Community Integration: To date the project has had a significant effect on inclusion or 

community integration for individuals with paralysis.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

9. Increased Knowledge of Resources: To date the project has led to increased knowledge or 

awareness of available resources for individuals with paralysis and/or their families.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

10. Access to Community Resources: To date, the project has led to increased access to 

community resources (e.g., financial, education, social) for individuals with paralysis 

and/or their families.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

11. Increased Decision-Making Skills: To date, the project has led to increased decisionmaking skills for individuals with paralysis and/or their families.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

12. Increased Self-Determination: To date, the project has led to increased self-determination 

for individuals with paralysis.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree
Paralysis Resource Center Evaluation: Final Report

27

o Strongly Disagree

o Unable to Determine

o N/A

13. Improvement in Abilities or Skills: To date, the project has led to improvement in abilities 

or skills for individuals with paralysis. (For example, improvement in employment and job 

skills/abilities, but it could also be used to capture improvement in physical abilities and 

skills such as core strength due to therapeutic horseback riding or even improved ability to 

ride a horse or sit up.)

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

14. Increased Community Interaction: To date, the project has increased the number of 

interactions of persons with paralysis and community members.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

15. Changes to People's Perception of Persons with Paralysis: To date, the project has affected 

the way people think about persons with paralysis.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

16. Increased Health Satisfaction: To date, the project has led to increased health satisfaction of 

individuals with paralysis.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

17. New Collaborations: To date, the project has led to NEW collaborations with your 

organization and other disability-related agencies, organizations, nonprofits.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A
Paralysis Resource Center Evaluation: Final Report

28

18. Existing Collaborations: To date, the project has led to furthering EXISTING 

collaborations with your organization and other disability related agencies and/or nonprofit 

organizations.

o Strongly Agree

o Agree

o Somewhat Agree

o Neither Agree nor Disagree

o Somewhat Disagree

o Disagree

o Strongly Disagree

o Unable to Determine

o N/A

19. Community Impact: In what ways has your project affected the community of persons with 

and without paralysis? Please provide a narrative response.

20. Underserved Populations: To date, funded projects have served traditionally underserved 

populations within the individuals with paralysis and family/caregiver community. Check 

all that apply to indicate the underserved populations served through this project.

o Rural

o At or below 150% of poverty

o Racial/Ethnic minority groups

o LGBTQ+ populations

o Medically underserved

21. How was the information for the above questions collected? Check all that apply.

o State grantee collected information directly from project beneficiaries (e.g., through 

a surveyor interview)

o Sub grantee collected information directly from project beneficiaries (e.g., through 

a surveyor interview)

o Administrative data (e.g., attendance log sheets)

Community-Based Organization Application Process Evaluation

Please provide feedback so that we might improve the process in future years.

22. Application process: How satisfied were you with the application process?

e. Very Satisfied

f. Highly Satisfied

g. Moderately Satisfied

h. Slightly Satisfied

i. Not at All Satisfied

j. Unable to Judge

23. Clarity of the application: How satisfied were you with the clarity of the application?

k. Very Satisfied

l. Highly Satisfied

m. Moderately Satisfied

n. Slightly Satisfied

o. Not at All Satisfied

p. Unable to Judge

24. Ease of the application: How satisfied were you with the ease of filling out the application?

q. Very Satisfied

r. Highly Satisfied

s. Moderately Satisfied

t. Slightly Satisfied

u. Not at All Satisfied

v. Unable to Judge

25. Timeline/feasibility: Was the timeline of implementation feasible for your project scope?

w. Yes

x. No

26. Application support provided: How and in what way(s) does the state support your 

organization through the subgrant process?

27. Application support provided (continued): To what extent do you find this support helpful?

28. Process improvement: How, if at all, can the grantmaking process be improved?
Paralysis Resource Center Evaluation: Final Report

29

Outcomes Evaluation

In the following section, please choose the answer that most closely reflects the extent to which 

you met the goals included in your application.

29. Served the intended types of people/groups (e.g., demographics, needs, geography)

o Significantly more than expected

o More than expected

o Fewer than expected

o Significantly fewer than expected

30. Served the intended number of people/groups.

o Significantly more than expected

o More than expected

o Fewer than expected

o Significantly fewer than expected

31. Met timeliness of project implementation

o Significantly more than expected

o More than expected

o Fewer than expected

o Significantly fewer than expected

32. Do you have anything else you would like to add?

Impact of COVID-19 on Grant Activities

33. Is COVID-19 affecting how you are operating your grant?

o Yes (if selected, proceed to second and third COVID-19 questions)

o No (if selected, go to the end of the survey)

34. Please indicate how the COVID-19 response is affecting your grant (select all that apply):

o Delaying implementation of grant activities

o Changing how the grant is implemented (e.g., in-person to virtual)

o Staff time diverted away from grant activitiesto other priorities

o Staff unable to complete grant activities due to sickness, caregiving, or “stay-athome” guidelines

o Funding diverted away from grant activities

o Funding insufficient to react to changing environment for grant activities due to 

COVID-19

o Recipients of grant activities unable to participate due to COVID-19 illness, 

caregiving, or “stay-at-home” guidelines

o Other: _____________________

35. Please provide any other comments on the impact COVID-19 is having on your grant:

36. Given the far-reaching impacts of COVID-19, what support or response would you like to 

see from ACL? (Not included in Year Two)

37. How do you expect to work with participants to support them in reintegration into the 

community as stay-at-home orders and other restrictions are lifted?

38. Consider the changes and/or new practices that your organization has adopted in response 

to the COVID-19 pandemic. Are there any that you are going to keep in place following the 

pandemic? If so, please provide a brief summary below. 
Paralysis Resource Center Evaluation: Final Report

30

Appendix D: Vanderbilt University NPRC Survey Questions

Impact Evaluation Questions: Grantee Self-evaluation on the impact of their projects

Volunteer opportunities* (* indicates required question in online system)

Did your project create any volunteer opportunities in the community?

Yes/No

Military/Veteran Programs*

Did your project serve any military or veteran populations of persons with paralysis?

Yes/No

******************************************************************************

In the following section, please choose the answer that most closely reflects the extent to which 

you agree or disagree with each statement. Choices are:

· Strongly disagree 

· Somewhat disagree 

· Disagree 

· Neither Agree or Disagree 

· Slightly Agree 

· Somewhat Agree 

· Strongly Agree 

· N/A Unable to Determine

Effect on Quality of Life*

To date, the project has had a significant effect on quality of life for individuals with paralysis 

and/or their families.

Effect on Functional Independence*

To date, the project has had a significant effect on functional independence for individuals with 

paralysis and/or their families.

Community Integration*

To date, the project has had a significant effect on inclusion or community integration for 

individuals with paralysis.

Increased Knowledge of Resources*

To date, the project has led to increased knowledge or awareness of available resources for 

individuals with paralysis and/or their families.

Access to Community Resources*

To date, the project has led to increased access to community resources(e.g., financial, 

education, social) for individuals with paralysis and/or their families.

Increased Decision-Making Skills*

To date, the project has led to increased decision-making skills for individuals with paralysis 

and/or their families.

Increased Self-Determination*

To date, the project has led to increased self-determination for individuals with paralysis.

Improvement in Abilities or Skills*

To date, the project has led to improvement in abilities or skills for individuals with paralysis. 

(For example, improvement in employment and job skills/abilities, but it could also be used to 

capture improvement in physical abilities and skills such as core strength due to therapeutic 

horseback riding or even improved ability to ride a horse or sit up.)
Paralysis Resource Center Evaluation: Final Report

31

Increased Community Interaction*

To date, the project has increased the number of interactions of persons with paralysis and 

community members.

Changes to People's Perception of Persons with Paralysis*

To date, the project has affected the way people think about persons with paralysis.

Increased Health Status*

To date, the project has led to increased health status of individuals with paralysis.

New Collaborations*

To date, the project has led to NEW collaborations with your organization and other disabilityrelated agencies, organizations, nonprofits.

Existing Collaborations*

To date, the project has led to furthering EXISTING collaborations with your organization 

and other disability related agencies and/or nonprofit organizations.

Underserved Populations*

To date, the project has served traditionally underserved populations within the individuals with 

paralysis and family/caregiver community.

Impact on Fundraising*

The partnership with the Christopher & Dana Reeve Foundation has let to my organization's 

ability to acquire more funds from other foundations/donors.

Community Impact*

In what ways did your project affect the community of persons with and without paralysis?

Please provide a narrative response.

*****************************************************************************

The last section of this report, which will enable the Reeve Foundation to improve upon its 

processes, please respond to the question “How satisfied were you with the following?” using 

these responses:

· Not At All Satisfied

· Slightly Satisfied

· Moderately Satisfied

· Highly Satisfied

· Very Satisfied

· N/A or Unable to Judge

Application process*

How satisfied were you with the application process?

Clarity of the application*

How satisfied were you with the clarity of the application?

Ease of the application*

How satisfied were you with the ease of filling out the application?

Templates*

How satisfied were you with the application templates (budget and budget narrative)?
Paralysis Resource Center Evaluation: Final Report

32

Webinar – Submission information*

How satisfied were you with the technical assistance webinar in preparation for what 

information you needed to submit the application?

Webinar – Program goals*

How satisfied were you with the technical assistance webinar in helping you better understand 

the goals of the program?

Timeline/feasibility*

Was the 12-month timeline of implementation feasible for your project scope? 

o Yes

o No

Process improvement*

How can the Reeve Foundation improve upon its grantmaking process?

Narrative Response
Paralysis Resource Center Evaluation: Final Report

33

Appendix E: Survey Likert Scale Conversions

Scale Legend

State Subawardees SelfEvaluation Score National Subawardees SelfEvaluation Score

Strongly Agree 7 Strongly Agree 7

Agree 6 Somewhat Agree 6

Somewhat Agree 5 Slightly Agree 5

Neither Agree Nor Disagree 4 Neither Agree Nor Disagree 4

Somewhat Disagree 3 Slightly Disagree 3

Disagree 2 Somewhat Disagree 2

Strongly Disagree 1 Strongly Disagree 1

N/A or Unable to Judge 0 N/A or Unable to Judge 0

State Subawardees Outcomes 

Evaluation Both: Application Process

Significantly more than expected 4 Very Satisfied 5

More than expected 3 Highly Satisfied 4

Fewer than expected 2 Moderately Satisfied 3

Significantly fewer than expected 1 Slightly Satisfied 2

Not At All Satisfied 1

N/A or Unable to Judge 0
Paralysis Resource Center Evaluation: Final Report

34

Appendix F: State Survey Responses Years 1 and 2

Populations Served Across States 

Underserved Populations Reached State Responses 

Year 1 (n=4)

State Responses 

Year 2 (n=4)

Rural population 4 (100%) 4 (100%)

Racial/Ethnic minority groups 2 (50%) 4 (100%)

Medically underserved -- 4 (100%)

At or below 150% of poverty 2 (50%) 3 (75%)

LGBTQ+ populations -- 2 (50%)

State Application Process Survey Results

Application Process Metric State Responses 

Year 1 (n=4)

State Responses 

Year 2 (n=4)

How satisfied were you with: 5 Point Likert Scale Average

Application process 4.3 4.5

Ease of the application 4.2 4.5

Clarity of the application 4.3 4.3

1 Point Average (No=0, Yes=1)

Did you find that the timeline was feasible? 0.9 0.5

State Self-Evaluation Survey Results

Self-Evaluation Metric State Responses 

Year 1 (n=4)

State Responses 

Year 2 (n=4)

Self-Evaluation: The grant/project has had significant 

impact on: 7 Point Likert Scale Average

Community Integration 6.0 7.0

Increased Community Interaction 5.9 7.0

New Collaborations 5.6 7.0

Existing Collaborations 5.9 7.0

Functional Independence 5.1 6.8

Improvement in Abilities or Skills 5.5 6.8

Quality of Life 5.8 6.5

Increased Knowledge of Resources 6.1 6.5

Increased Access to Community Resources 6.1 6.5

Increased Self-Determination 5.6 6.3

Increased Decision-Making Skills 5.1 5.8

Increased Health Satisfaction/Status 4.8 4.5

Changes to People's Perception of Persons with Paralysis 5.0 4.3

State Outcomes Evaluation Survey Results

Outcomes Evaluation Metric State Responses Year 

1 (n=4)

State Responses 

Year 2 (n=4)

Outcomes Evaluation: What was the extent to 

which you: 4 Point Scale Average

Served the intended types of people/groups 

(e.g., demographics, needs, geography) 2.8 3.3

Served the intended number of people/groups. 2.8 3.0

Met timeliness of project implementation 2.8 2.3

COVID-19 Impacts on States

Please indicate how the COVID-19 response is 

affecting your grant:

State Responses 

Year 1 (n=4)

State Responses 

Year 2 (n=4)

Delaying implementation of grant activities 2 (50%) 4 (100%)

Changing how the grant is implemented (e.g., inperson to virtual) 2 (50%) 3 (75%)
Paralysis Resource Center Evaluation: Final Report

35

Please indicate how the COVID-19 response is 

affecting your grant:

State Responses 

Year 1 (n=4)

State Responses 

Year 2 (n=4)

Staff time diverted away from grant activities to other 

priorities -- 1 (25%)

Staff unable to complete grant activities due to 

sickness, caregiving, or “stay-at-home” guidelines 1 (25%) 1 (25%)
Paralysis Resource Center Evaluation: Final Report

36

Appendix G: Reporting Period for Applications and Reports Used

State Reporting Period

KY

Project Period: July 1, 2018 – June 30, 2021 

Semi-annual reports used:

· July 1, 2018 – December 31, 2018

· January 1, 2019 – June 30, 2019

· July 1, 2019 – December 31, 2019

· January 1, 2020 – June 30, 2020

· July 1, 2020 – December 31, 2020

TX

July 1, 2018 – July 30, 2021 

Semi-annual reports used:

· July 1, 2018 – December 31, 2018

· January 1, 2019 – June 30, 2019

· January 1, 2020 – June 30, 2020

· July 1, 2020 – December 31, 2020

OH

July 1, 2019 – June 30, 2021 

Semi-annual reports used:

· July 1, 2019 – December 31, 2019

· January 1, 2020 – June 30, 2020

· July 1, 2020 – December 31, 2020

PA

July 1, 2019 – June 30, 2021

Semi-annual reports used:

· June 1, 2019 – December 31, 2019

· January 1, 2020 – June 30, 2020

· July 1, 2020 – December 31, 2020
Paralysis Resource Center Evaluation: Final Report

37

Appendix H: PRC Evaluation – Outcome Toolkit Review Memo

Memo

To: Elizabeth Leef and Susan Jenkins, Administration for Community Living

From: Kristen Vangeloff and Nithi Suresh, Lewin Group

Date: May 10, 2021

Subject: PRC Evaluation – Outcome Toolkit Review

Background

As part of the Year One Paralysis Resource Center (PRC) Evaluation, we found that the six 

metrics required as part of the ACL awards were not consistently reported in the semi-annual and 

annual reports. For that reason, we opted to remove the partially reported metrics from the Year 

One Report and noted that the full metrics would be included in the Final Report. The six metrics 

are:

1. Award rate (number of awards/number of applications received)

2. Percentage of awards completed on time and on budget

3. Cost of administering the subawards

4. Number of people proposed to be served throughout the life of the subaward compared 

to how many people are served

5. Level of satisfaction of the subawardees with the application and award process: 

respondents were asked to rate the question “I am satisfied with the application 

process” on a scale from one to four with one being strongly disagree and four being 

strongly agree.

6. Level of satisfaction of the beneficiaries with the outcome of the subaward: respondents 

were asked to rate the question “I am satisfied with this event” on a scale from one to 

four with one being strongly disagree and four being strongly agree.

In discussion with ACL, the state PRCs, and the National PRC (NPRC), we heard that the 

fourth and sixth metrics can be difficult to measure. The number of people served and the level of 

satisfaction of beneficiaries are especially difficult to collect for programs that focus on 

community-level activities, such as accessible parks or increased access to other community 

resources. Grantees find it easier to track attendance and gather participant satisfaction data for 

programs that provide more time- and participant-defined activities (e.g., recreational classes). 

However, they still find it challenging to show through these attendance and satisfaction data that 

the activity or program had the intended impact on participants and the community at large. This is 

not limited to the paralysis community – it is a common challenge across health and social service 

programs. At ACL’s suggestion, we performed a brief search of publicly available outcome 

collection tools that could help the PRC grantees with their difficulties in measuring the impact of 

their programs. This memo summarizes our findings and provides high-level suggestions for next 

steps that ACL may choose to take if pursuing this topic further. 

Process and Outcomes Evaluation Toolkits

The Urban Institute: Key Steps in Outcome Management11

The Urban Institute recognizes the challenges faced by the nonprofit sector in providing 

evidence of the impact of its interventions on beneficiaries. They created a guide that provides 

discrete stages in measuring outcomes for the nonprofit sector. They elaborate on the steps below:

11 The Urban Institute: Key Steps in Outcome Management
Paralysis Resource Center Evaluation: Final Report

38

· Set up

o Selecting programs to include

o Determine the team involved in the process

o Establish a schedule

· Deciding what and how to measure

o Mission, objectives and clients

o Identify outcomes

o Select specific indicators to measure outcomes

o Select data sources and data collection procedures for each indicator

o Identify key client and service characteristics to link to outcome information

o Pilot test

· Analyzing outcome information

o Examine outcome data

o Report findings

o Seek explanations for unusual findings

· Using the results

o Use outcome information to improve services

It contains further information on implementation, and sample schedules, agendas and 

reports to aid in the process. Additional guides provide detail on surveying both current and past 

beneficiaries, analysis and interpretation of outcome data, and developing common sets of 

indicators for funders. 

Administration on Aging (AoA) Performance Measurement Toolkit12

AoA developed this toolkit through the Performance Outcome Measurement Project 

(POMP) to measure the impact of initiatives funded under the Older Americans Act (OAA). The 

toolkit is geared toward community-based providers of OAA services. It outlines the step-by-step 

process to using the survey instruments developed by POMP to measure performance and use the 

results to improve services provided. 

The first step outlined is to select the survey instrument. The toolkit contains nine 

instruments focused on topics ranging from specific services provided (e.g., case management and 

transportation) to service recipient characteristics (e.g., demographics and social and emotional 

well-being). Next, there is guidance on determining the best method for data collection. This can 

be done in person, by telephone, by mail or through internet surveys. The toolkit explains the 

benefits and limitations of each method. Once a data collection approach is selected, a work plan 

and budget are developed. The Toolkit provides templates survey timeline planning and cost 

estimates for each step.

The toolkit also contains a sample size calculator and guidelines for each method of survey 

dissemination. The POMP website provides database utilities for the sample surveys included in 

the Toolkit. The toolkit further describes data analysis methods and recommendations for 

disseminating results through reports, newsletters, handouts, fact sheets, and service provider inhouse memos, among other documents. The Toolkit includes examples of each of these products. 

Finally, the toolkit provides information on how organizations can compare elements of their 

results and specific measures against national data. 

Colorado Nonprofit Association Principles and Practices Evaluation Toolkit13

The Colorado Nonprofit Association developed this toolkit to help the state’s nonprofit 

organizations with evaluating compliance, strategic planning and operational evaluation. Similar to 

the Urban Institute, the Association recognizes the difficulties in evaluating organizational 

effectiveness and impact. It is also in an effort to improve quality, transparency and accountability 

of these organizations. 

They explain the importance of and different approaches to evaluation. The Association 

recommends the use of both process- and outcomes-based evaluations to measure implementation 

12 Administration on Aging Performance Measurement Toolkit

13 Colorado Nonprofit Association Principles and Practices Evaluation Toolkit
Paralysis Resource Center Evaluation: Final Report

39

and impact, respectively. The toolkit then provides information on the evaluation process. Much 

like the POMP toolkit, it includes information on how to plan outcome measurement (e.g., logic 

models, theory of change), gather data, and analyze results. It also contains an organizational 

assessment and planning tool that supports organizations in tracking their effectiveness at 

measuring outcomes. 

Recommendations

The measurement and evaluation steps provided by each of these guides follow a similar process. 

In general, the following are high-level steps to follow: 

· Determine objectives and outcome goals (e.g., logic models, Theory of Change 

frameworks)

· Choose a tool and method for collecting information

· Create a work plan and budget for the process

· Collect data

· Analyze data

· Disseminate findings (e.g., reports, newsletters, factsheets)

· Develop recommendations for improving services and outcomes.

To support PRC grant recipients, ACL could develop a Toolkit tailored for the Quality of Life 

grants (e.g., sample surveys, list of outcomes of interest, best practices in data collection, sample 

reports). This would require drawing on experts within the paralysis community to develop the 

materials and review by stakeholders (i.e., current and former grantees, funders, and evaluators). 

Alternatively, ACL could include one or more of these tools as recommended reading in future 

funding announcements. Organizations applying for funding may benefit from reviewing these 

materials from the earliest stages of their program development. 
Paralysis Resource Center Evaluation: Final Report

40

Appendix I: New Reeve Foundation National Paralysis Resource Center 

Grant Program to Address Social Isolation During the COVID-19 

Pandemic

Program Rationale

Emerging research on COVID-19 shows that social isolation and loneliness due to the coronavirus 

pandemic have been linked to many physical and mental health problems including heart disease, 

diabetes, heart failure, stroke, dementia, anxiety, depression and suicide. The health damage caused 

by isolation and loneliness is estimated to increase the risk of early death by 26 percent, has been 

equated to smoking 15 cigarettes a day, and is estimated to cumulatively cost Medicare an 

additional $6.7 billion each year14. Studies have also documented, social isolation contributes to 

elevated blood pressure, morning spikes in cortisol levels, and disrupted sleep15. 

Research shows that adults with chronic conditions and physical or cognitive disabilities are more 

than two times as likely to report feeling socially isolated (37%) than adults who do not have these 

health issues (15%)16. Research on the psychological impact of mass trauma (e.g., natural disasters, 

flu outbreaks) suggests that the pandemic might particularly harm the mental health of 

marginalized populations who have less access to socioeconomic resources and supportive social 

networks17.

In addition, earlier quarantines provide some indications of the potential fallout from extended 

periods of isolation. After being quarantined during the epidemic of sudden acute respiratory 

syndrome (SARS), individuals who were quarantined or had relatives who contracted SARS were 

2 to 3 times more likely to report high levels of post-traumatic stress disorder (PTSD) symptoms 

than the rest of the population.18

Lastly, surveys and data collected from the Reeve Foundation and from stakeholders in the field of 

paralysis research have indicated that social isolation has had the number one negative impact on 

the lives of people living with disabilities relative to the COVID-19 pandemic.

Grants Program

The aim of the Reeve Foundation’s new grants program is to address social isolation during the 

COVID-19 pandemic with the goal of enhancing connectedness of people living with paralysis 

and their caregivers to their communities and preventing adverse health outcomes.

Though we are looking to the field for innovative ways to address the program’s aim and goals, 

funds may support programs and projects to:

· Connect isolated individuals and remotely foster engagement and connection through

o Virtual support groups 

o Online activities to help support self-care and stress management

o Virtual and online adaptive physical activity and health promotion programs

o Art sessions

o Gaming 

· Adapt services or create new services to continue to support and engage family caregivers

· Facilitate connection through the use of technology, including

o Increasing access to internet, broadband service, and tablets (via loan closets)

· Provide programs with online opportunities for volunteering (Volunteering has been shown 

to have many positive health benefits)

· Create strategies to strengthen and/or build social networks

· Provide access to tools and resources to help people remain connected and engaged

The budget for this program is $300,000. It is expected that grants will be awarded within the 

range of $25,000-$50,000.

14 U.S. Department of Health and Human Services data

15 Cacioppo JT, Hawkley LC, Norman GJ et al. Social isolation. Ann NY Acad Sci 2011;1231(1):17. 

doi:10.1111/j.1749-6632.2011.06028.x

16 The Consequences of Social Isolation and Loneliness - https://www.commonwealthfund.org/blog/2020/how-covid19-pandemic-could-increase-social-isolation-and-how-providers-and-policymakers

17 Goldmann & Galea, 2014

18 Psychiatry Advisor: April 29, 2020.The Costs of Social Isolation: Loneliness and COVID-19 Y.Douglas, PhD.
Paralysis Resource Center Evaluation: Final Report

41

Eligibility

Nonprofit organizations with IRS 501(c)(3) status, municipal and state governments, school 

districts, recognized tribal entities, and other institutions such as community or veterans hospitals

may apply. An organization must have its own 501(c)(3) tax status (or be a part of or chapter of a 

national organization that is a 501(c)(3) nonprofit organization). Fiscal Sponsors are not allowed to 

apply on behalf of non-501(c)(3) nonprofit organizations.

Application Process

Nonprofit organizations are asked to prepare a one or two-page Letter of Intent that includes brief 

descriptions of:

1) The organization (you must include your organization’s EIN/Tax ID#)

2) The proposed project (indicating a proposed outreach plan to people living with paralysis or 

their caregivers), and 

3) Amount requested. 

Submit Letters of Intent to QOL@ChristopherReeve.org with the subject line “LOI: COVID-19 

Grants.”

Selected organizations will be invited to submit a full application.

Program Timeline

Letters of Intent (3-week open period begins September 22nd) due October 13, 2020

Invitations to submit full applications will be made by October 30, 2020

Applications due by November 20, 2020 

Awards announced by December 31, 2020

Projects begin January 1, 2021

Projects end December 31, 2021