**State Partnership Systems Change Initiative (SPI)**

**Project Office**

**Conclusions Drawn from the State Partnership Initiative**

## 

## October 2005

**Revised December 2005**

**Final Revision May 2006**

![](media/image1.png){width="1.3770833333333334in"
height="2.0902777777777777in"}

**In Partial Fulfillment of Deliverables 9.1 and 9.2 of**

**Contract #0600-03-60161**

**State Partnership Initiative Evaluation and Information Office**

**Virginia Commonwealth University**

**Rehabilitation Research and Training Center**

**John Kregel, Principal Investigator**

**P.O. Box 842011**

**Richmond, Virginia 23284-2011**

**804-828-1851**

**Conclusions Drawn from the Findings of the State Partnership
Initiative (SPI)**

## 

## Table of Contents

I. Summary of the SPI Initiative 3

II\. Summary of the SPI Interventions 6

III\. Summary of the SPI Systems Change Activities 17

IV\. What Effect did SPI have on Participant Employment? 19

V. What Effect did SPI have on Reduction of SSA Benefits Received? 27

VI\. Summary of Final Evaluation of the SSI Work Incentives
Demonstration Project 29

VII\. Implications for Future SSA Demonstrations 31

References 37

**I. Summary of the SPI Initiative**

**Overview of SPI**

The State Partnership Systems Change Initiative (SPI) was comprised of
18 State projects designed to identify, implement and evaluate
innovative strategies to promote employment opportunities for SSI/SSDI
beneficiaries, as well as recipients of other types of public supports.
Twelve of the State Projects were funded primarily by SSA and the other
six Projects were funded primarily by the Rehabilitation Services
Administration (RSA) of the U.S. Department of Education. The RSA-funded
state projects focused mainly on changing service systems, and less on
providing direct services. Most state projects targeted beneficiaries
with severe mental illness, although many also targeted people with
other disabilities. The state projects were funded in fall 1998, and the
first state began enrollment in January 1999. Most SSA-funded state
projects provided services through September 2004. This Final Report of
the SPI Project includes not only the measurable findings from the
SSA-funded projects, but also the systems change accomplishments from
both the SSA and RSA funded state projects.

SPI was a major component of the Federal governments overall efforts to
improve employment opportunities for individuals with disabilities,
especially those who receive Supplemental Security Income (SSI), Social
Security Disability Insurance (SSDI) or other types of public
assistance. The State Partnership/Systems Change Initiative represented
a unique collaboration among SSA, RSA, the Department of Labor (DOL),
and the Substance Abuse Mental Health Services Administration (SAMHSA),
all working in conjunction with the Presidential Task Force on the
Employment of Adults with Disabilities beginning in the Fall of 1998 and
ending in the Fall of 2004. The SPI initiative was part of the
government's overall response to the frustrating and vexing problem of
chronic unemployment among citizens with disabilities. The frustration
among consumers, the Administration and the Congress stems from the fact
that although tremendous advances occurred over the previous decade,
little had been done to impact the 65%-70% unemployment rate and
increase workforce participation of individuals with disabilities.

The demonstration and systems change activities initiated by the
projects varied widely from State to State. However, major project
activities fell into a number of identifiable activities, including
Medicaid Buy-in Activities, Social Security Waivers, Benefits Planning
and Assistance, One-Stop Career Center Collaboration, Interagency
Collaborative Activities, Consumer Mentoring Activities, and Direct
Employment Services.

During Fall 2000 and Winter 2001, the SSA SPI Projects were required to
define the interventions which would be the focus of their service
provision for the next three years, identify the systems change
activities which they would pursue, and develop an internal evaluation
plan to test the effect of the interventions being offered. Each Project
was required to submit at the end of December 2003, an internal
evaluation report, called the Year 5 Evaluation Report, which was a
combination of the preliminary findings of their process evaluation and
the preliminary results of the outcomes resulting from the
implementation of their respective interventions. One of the final
requirements of each Project was the submission of its final evaluation
report within ninety days of the close of their project (or by December
31, 2004). These final evaluation reports, which were all submitted by
December 31, 2005, provided an update on system changes accomplished by
the projects as well as presented outcome findings of the impacts of the
interventions.

**Influence of Other Employment Policy Initiatives During SPI
Implementation**

SSA's efforts to promote beneficiary employment and self-support began
even before passage of the 1980 amendments to the Social Security Act,
which added several work incentives to the SSI program. More recently,
the Ticket to Work and Work Incentive Improvement Act of 1999 (Public
Law 106-170) created several important new initiatives that affect
people who receive disability benefits. In addition, several important
recent executive initiatives (the New Freedom Initiative and the
President's Task Force on Employment of Adults with Disabilities) have
sought to identify and eliminate barriers to employment for people with
disabilities.

The implementation of these other demonstrations and initiatives
substantially affected the SPI demonstration and its evaluation. The
influx of additional resources enabled some state projects to offer
their SPI participants enhanced services, or to offer more beneficiaries
services similar to those provided in their state projects. In addition,
the new demonstrations and initiatives affected the environments against
which the state projects are compared. To the extent that these
initiatives promoted the viability of work for all beneficiaries, the
effect of services that the state projects provided are harder to detect
and interpret.

Since the start of SPI, nine other major initiatives have begun to
provide services or to change policies designed to promote employment
among people with disabilities, including people who are receiving
benefits from SSA. The following list provides an overview of these
policy initiatives:

-   ***Benefits Planning, Assistance, and Outreach (BPAO)*.** This SSA
    program funds benefits planning for beneficiaries with disabilities
    who are trying to return to work. Benefit planners provide direct
    advice and assistance to SSI and SSDI beneficiaries by (1)
    explaining SSA work incentives and the effects of work on benefits,
    and (2) providing information on state vocational rehabilitation
    (VR) systems and other available supports. BPAO providers provide
    services to the entire United States.

-   ***Medicaid Buy-In*.** Recently enacted legislation enables states
    to modify their Medicaid programs to provide workers who have
    disabilities with better access to health insurance. The buy-in
    programs expand coverage by expanding Medicaid income and resource
    eligibility standards, and by creating sliding-scale premium
    arrangements to encourage people with disabilities to maintain
    employment. Nine of the SPI states started buy-in programs.[^1]
    Currently, about 30 states have Medicaid buy-in programs (White et
    al. 2005). However, many of those programs began operations after
    enrollment for SPI had ended.

-   ***Medicaid Infrastructure Grant*.** This Centers for Medicare and
    Medicaid Services (CMS) grant program provides funding to states
    that want to modify their Medicaid programs to implement a buy-in
    program, or to provide other employment incentives for people with
    disabilities.

-   ***Demonstration to Maintain Independence and Employment*.** This
    CMS-funded program was not offered in any of the SPI states. It
    originally supported efforts in three states (Mississippi, Rhode
    Island, and Texas) and the District of Columbia to enable people
    with chronic, disabling conditions to obtain medical benefits
    without having to first qualify for disability benefits (which
    typically requires that people quit their jobs). It has since
    expanded to additional states. The demonstration allows states to
    provide health care services and supports to working people who need
    to manage the progression of their diseases.

-   ***Work Incentive Grants*.** The Work Incentive Grant Program is
    funded by DOL to enhance employment opportunities for people with
    disabilities. The grants encourage One-Stop Career Centers to
    develop innovative ways to ensure that this population can obtain
    comprehensive, accessible employment services that will address
    their barriers to employment.

-   ***Employment Assistance Grants Through DOL's Office of Disability
    Employment Policy*.** This grant program targets planning and
    implementation activities to enhance the availability and provision
    of employment services for people with disabilities within the
    One-Stop delivery system. To improve employment outcomes for people
    with disabilities, technical assistance grants also are offered to
    One-Stop Career Centers, State and Local Workforce Investment
    Boards, Youth Councils, and Workforce Investment Act Grant
    recipients who serve adults and youths.

-   ***Ticket to Work*.** This SSA program introduced a new
    performance-based method of paying for services to help disabled
    beneficiaries to obtain and hold jobs, while exercising more
    consumer choice. SSA issues eligible beneficiaries a ticket that
    they can take to the service provider of their choice. Providers
    have the option of deciding whether to accept the Ticket. If they do
    accept it and try to help the beneficiary to obtain employment,
    their payments are based on achievement of specific milestones,
    particularly whether the beneficiary successfully moved from the
    disability rolls to self-supporting employment. The Ticket program
    was introduced in 13 states during 2002 and was operating in every
    state by September 2004.

-   ***Olmstead Grants*.** This CMS grant program helps states to place
    into an integrated setting qualified people with disabilities who
    are in institutions or who are being assessed for
    institutionalization. The initiative includes three categories of
    systems grants to states: (1) Nursing Facility Transition
    Grants, (2) Community-Integrated Personal Service and Support
    Grants, and (3) "Real Choice" System Change Grants.

-   ***Indexing of the Substantial Gainful Activity (SGA) Amount*.**
    Since 1999, SSA has adjusted the average monthly earnings amount
    used to determine whether work performed by beneficiaries with
    disabilities is considered SGA. The annual adjustments are intended
    to correct for inflation. Before 1999, the Social Security
    Commissioner instituted regulations specifying the appropriate level
    to be used to set the SGA.

The initiatives described above created a dynamic environment that
complicated the SPI evaluation. Although employment and training
evaluations have long faced the challenge of accounting for local
variation in service environments, introduction of these new initiatives
took place during the SPI effort, and, in many cases, the new
initiatives offered service interventions that the state projects also
offered. They therefore affected the mix of services available to
participants and potential comparison group members at the same time
that the SPI state projects tried to deliver new services to
participants. If the new initiatives successfully expanded the
availability of employment-support services, the net extent to which the
state projects could have expanded services to participants is reduced.
This outcome, in turn, would reduce the potential impacts produced by
the state projects relative to what was expected at the time that the
projects had been designed.

**II. Summary of the SPI Interventions**

All 12 SSA-funded SPI Projects developed Intervention Plans using an
outline provided by the Virginia Commonwealth University (VCU) Project
Office during the first 2 years of their project's operation. These
plans were reviewed and approved by SSA in the Spring of 2001. The
interventions implemented by the 12 SPI State Projects targeted the
service and policy gaps that were identified at the time that the
projects began in 1998. As shown in Table 1, all State Projects provided
benefits planning and assistance services; all Projects except North
Carolina, Ohio and Oklahoma had Medicaid Waivers and Buy-ins; most
Projects provided interventions involving One-Stop Centers, case
management and placement assistance services; and relatively few State
Projects provided other types of services such as mental health and
developmental disabilities support, and peer mentoring.

SPI interventions, which address the major areas of concern, fall into 3
corresponding categories: benefits planning and assistance services;
direct employment supports; and increased health care.

**TABLE 1\
\
SERVICES PROVIDED BY THE SSA STATE PROJECTS TO REMOVE EMPLOYMENT
BARRIERS**

<table>
<colgroup>
<col style="width: 11%" />
<col style="width: 11%" />
<col style="width: 1%" />
<col style="width: 5%" />
<col style="width: 1%" />
<col style="width: 4%" />
<col style="width: 4%" />
<col style="width: 4%" />
<col style="width: 8%" />
<col style="width: 4%" />
<col style="width: 4%" />
<col style="width: 4%" />
<col style="width: 5%" />
<col style="width: 4%" />
<col style="width: 1%" />
<col style="width: 8%" />
<col style="width: 7%" />
<col style="width: 8%" />
</colgroup>
<thead>
<tr class="header">
<th></th>
<th colspan="13"><blockquote>
<p>Direct Interventions</p>
</blockquote></th>
<th></th>
<th colspan="3"><blockquote>
<p>System Change</p>
</blockquote></th>
</tr>
<tr class="odd">
<th></th>
<th><blockquote>
<p>Increase<br />
Understanding<br />
of How Work<br />
Affects<br />
Benefits</p>
</blockquote></th>
<th></th>
<th><blockquote>
<p>Make<br />
Work<br />
Pay</p>
</blockquote></th>
<th></th>
<th colspan="9"><blockquote>
<p>Improve Human Capital</p>
</blockquote></th>
<th></th>
<th><blockquote>
<p>Increase<br />
Access to<br />
Health<br />
Insurance</p>
</blockquote></th>
<th><blockquote>
<p>Build<br />
More<br />
Person-<br />
Centered<br />
Service<br />
Systems</p>
</blockquote></th>
<th><blockquote>
<p>Reduce<br />
Employer<br />
Barriers</p>
</blockquote></th>
</tr>
<tr class="header">
<th></th>
<th><blockquote>
<p>Benefits/Work Incentive Counseling</p>
</blockquote></th>
<th></th>
<th><blockquote>
<p>SSA Work Incentives</p>
</blockquote></th>
<th></th>
<th><blockquote>
<p>Case Management</p>
</blockquote></th>
<th><blockquote>
<p>Placement Assistance</p>
</blockquote></th>
<th><blockquote>
<p>Job Training</p>
</blockquote></th>
<th><blockquote>
<p>Services/Supports from Local MH/DD Service Provider</p>
</blockquote></th>
<th><blockquote>
<p>Job Accommodations</p>
</blockquote></th>
<th><blockquote>
<p>Job Service Voucher</p>
</blockquote></th>
<th><blockquote>
<p>Psychosocial Rehabilitation</p>
</blockquote></th>
<th><blockquote>
<p>Peer<br />
Mentoring</p>
</blockquote></th>
<th><blockquote>
<p>Situational Assessment</p>
</blockquote></th>
<th></th>
<th><blockquote>
<p>Medicaid Waiver/Buy-In</p>
</blockquote></th>
<th><blockquote>
<p>One-Stop Center</p>
</blockquote></th>
<th><blockquote>
<p>Outreach to<br />
Employers</p>
</blockquote></th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td>California</td>
<td>X</td>
<td></td>
<td>X</td>
<td></td>
<td>X</td>
<td>X</td>
<td></td>
<td>X</td>
<td></td>
<td></td>
<td>X</td>
<td></td>
<td></td>
<td></td>
<td>X</td>
<td>X</td>
<td></td>
</tr>
<tr class="even">
<td>Iowa (SSA)</td>
<td>X</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>X</td>
<td>X</td>
<td>X</td>
</tr>
<tr class="odd">
<td>Minnesota</td>
<td>X</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>X</td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>New Hampshire</td>
<td>X</td>
<td></td>
<td></td>
<td></td>
<td>X</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>X</td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>New Mexico</td>
<td>X</td>
<td></td>
<td></td>
<td></td>
<td>X</td>
<td>X</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>X</td>
<td>X</td>
<td></td>
</tr>
<tr class="even">
<td>New York</td>
<td>X</td>
<td></td>
<td>X</td>
<td></td>
<td>X</td>
<td>X</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>X</td>
<td>X</td>
<td>X</td>
<td></td>
<td>X</td>
<td>X</td>
<td>X</td>
</tr>
<tr class="odd">
<td>Ohio</td>
<td>X</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>X</td>
<td>X</td>
</tr>
<tr class="even">
<td>Oklahoma</td>
<td>X</td>
<td></td>
<td></td>
<td></td>
<td>X</td>
<td>X</td>
<td>X</td>
<td></td>
<td></td>
<td>X</td>
<td>X</td>
<td></td>
<td>X</td>
<td></td>
<td></td>
<td>X</td>
<td></td>
</tr>
<tr class="odd">
<td>Vermont</td>
<td>X</td>
<td></td>
<td>X</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>X</td>
<td>X</td>
<td></td>
</tr>
<tr class="even">
<td>Wisconsin</td>
<td>X</td>
<td></td>
<td>X</td>
<td></td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td></td>
<td>X</td>
<td>X</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
</tbody>
</table>

Source: Based on the services that the state projects reported as
project funded in service data through March 31, 2002. We also relied on
information collected by VCU in site visits to document the use of
One-Stop Centers and employer outreach. Information on the use of the
Medicaid Buy-In is taken from the service data, State Project reports,
and (Deke and Peikes 2003).

Note: This table excludes Supported Employment, Transitional Employment
Program, Transportation Assistance, and School to Work Transition
Services. It also excludes services that projects reported delivering to
fewer than 10 participants.

Benefits Planning and Assistance Services

Each of the 12 SSA funded projects provided benefits planning and
assistance to project participants. In general, the benefits planning
and assistance services were delivered in a very uniform manner. All
states provided information and referral services, problem solving,
benefits analysis and advisement, intensive benefit support, and
advocacy on behalf of individual beneficiaries. Each of these categories
is defined below (Kregel & Head, 2004).

**Information and Referral** involves providing basic written and verbal
information in response to inquiries about all Federal and State benefit
programs, and/or referral to government agencies and other community
resources. This level of service may involve one to several contacts
over a relatively short period of time.

Problem Solving generally occurs over a period of several weeks to
several months, involves providing time-limited, intensive assistance to
recipients in solving specific federal and state benefit and work
incentive problems, and may involve advocating on behalf of the
recipient with other agencies and programs. Examples of the types of
specific problems and situations that fall under the area of Problem
Solving include:

>  

-   Changes in life status (living arrangement, marriage status, etc.)
    that may cause changes to disability benefits status, such as an SSI
    beneficiary who requests information on the effect of marriage on
    her future benefits.

-   Changes in disability benefit status that affect other federal
    benefits, such as a beneficiary who receives notification of
    Medicaid cessation and expresses concern over why the Medicaid
    eligibility was discontinued and requests information on how
    eligibility might be reinstated.

-   Problems related to post entitlement earnings reporting, such as an
    an SSI individual who continues to receive full benefits even though
    he or she has been working and reporting earnings to SSA for six
    months.

**Benefits Analysis and Advisement** requires the specialist to assess
the real or potential effects that employment or other such changes will
have on the recipients overall financial well-being, and to inform the
recipient of the various options available and the projected outcomes
for each.

**Benefits Support Planning** involves time-limited services aimed at
directly assisting recipients in constructing plans to promote effective
monitoring and management of their benefit programs and work incentives.

**Benefits Management**, which generally occurs on a scheduled basis
over an extended period of time, involves providing ongoing,
comprehensive, benefits monitoring and management assistance to
recipients who are likely to experience employment, benefits, or other
changes that will dramatically affect their benefits status, health
care, or overall financial well being.

**Individual Advocacy** involves supporting and advocating for a
specific beneficiary by writing letters, making telephone calls,
accompanying the beneficiary to FOs, attending meetings and other
similar activities that assist the beneficiary in issues related to
program eligibility, use of work incentives, communication with FOs, and
other related concerns.

**Systems Advocacy** involves working to achieve systemic changes that
will benefits groups of beneficiaries by serving on committees or task
forces, developing procedures with FOs or AWICs, advocating with Federal
or State government agencies, and other related activities.

**Staffing** Many states involved other staff members in the delivery of
benefits planning and assistance services. These staff positions were
termed employment counselors, employer liaisons, consumer outreach
specialists, service coordinators, or resource consultants. These other
staff positions would generally perform some, but not all, of the
functions described above. The specific functions performed by the
benefits specialists and other staff members are identified in Table 2.

Highly skilled Project staff members were able to provide training and
support to benefits specialists. Many of the state projects (Iowa,
Minnesota, New Hampshire, New York, Vermont, and Wisconsin) employed
project staff members who had developed national reputations for their
ability to train and support benefits specialists. Most SPI projects
shared information, training materials, and even training staff members
across projects in order to promote the consistent delivery of high
quality services. This not only improved the overall quality of the
services provided by the benefits specialists, but also dramatically
increased the consistency of the intervention within individual sites,
across various sites in a specific state, and across multiple states.
The consistency in the delivery of benefits planning and assistance
services will significantly aid the interpretation of outcome data from
the state internal evaluations, as well as the national net outcome
evaluation.

**Table 2**

**Benefits Planning and Assistance Services Provided by**

Benefits Specialists and Other SPI Project Staff Members

<table>
<colgroup>
<col style="width: 7%" />
<col style="width: 7%" />
<col style="width: 20%" />
<col style="width: 9%" />
<col style="width: 8%" />
<col style="width: 8%" />
<col style="width: 8%" />
<col style="width: 11%" />
<col style="width: 8%" />
<col style="width: 8%" />
</colgroup>
<thead>
<tr class="header">
<th><blockquote>
<p><strong>State</strong></p>
</blockquote></th>
<th><blockquote>
<p><strong>Number of</strong></p>
<p><strong>Staff</strong></p>
</blockquote></th>
<th><blockquote>
<p><strong>Position Title</strong></p>
</blockquote></th>
<th><blockquote>
<p><strong>Information</strong></p>
<p><strong>and Referral</strong></p>
</blockquote></th>
<th><blockquote>
<p><strong>Problem</strong></p>
<p><strong>Solving</strong></p>
</blockquote></th>
<th><blockquote>
<p><strong>Benefits Analysis and Advisement</strong></p>
</blockquote></th>
<th><blockquote>
<p><strong>Benefits Support</strong></p>
<p><strong>Planning</strong></p>
</blockquote></th>
<th><blockquote>
<p><strong>Benefits Management</strong></p>
</blockquote></th>
<th><blockquote>
<p><strong>Individual Advocacy</strong></p>
</blockquote></th>
<th><blockquote>
<p><strong>Systems Advocacy</strong></p>
</blockquote></th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td rowspan="2">CA</td>
<td>3</td>
<td>Benefits Coordinator</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
</tr>
<tr class="even">
<td>3</td>
<td>Service Coordinator</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>X</td>
<td>X</td>
</tr>
<tr class="odd">
<td>IL</td>
<td>3</td>
<td>Benefits Specialist</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
</tr>
<tr class="even">
<td>IA</td>
<td>2</td>
<td>Benefits Planner</td>
<td>X</td>
<td><h1 id="x">X</h1></td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
</tr>
<tr class="odd">
<td>MN</td>
<td>6</td>
<td>Benefits Specialist</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
</tr>
<tr class="even">
<td rowspan="2">NH</td>
<td>1</td>
<td>Benefits Specialist</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td></td>
</tr>
<tr class="odd">
<td>2.5</td>
<td>Resource Consultant</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td><h1 id="x-1">X</h1></td>
<td>X</td>
<td>X</td>
<td></td>
</tr>
<tr class="even">
<td>NM</td>
<td>3</td>
<td>Benefits Advisor</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
</tr>
<tr class="odd">
<td rowspan="2">NY</td>
<td>6</td>
<td>Benefits Advisor</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td></td>
</tr>
<tr class="even">
<td>6</td>
<td>Employment Coordinator</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>X</td>
<td>X</td>
</tr>
<tr class="odd">
<td>NC</td>
<td>1.5</td>
<td>Benefits Advisor</td>
<td>X</td>
<td><h1 id="x-2">X</h1></td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
</tr>
<tr class="even">
<td rowspan="2">OH</td>
<td>4</td>
<td>Benefits Counselor</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td></td>
</tr>
<tr class="odd">
<td>4</td>
<td>Employer Liaison</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>X</td>
<td>X</td>
</tr>
<tr class="even">
<td>OK</td>
<td>4</td>
<td>Consumer Outreach Specialist</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td><h1 id="x-3">X</h1></td>
</tr>
<tr class="odd">
<td>VT</td>
<td>6</td>
<td>Benefits Counselor</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
</tr>
<tr class="even">
<td>WI</td>
<td>30</td>
<td><p>Other Staff</p>
<p>(all non-SPI dollars)</p></td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
</tr>
</tbody>
</table>

# Direct Employment Supports

The SPI projects began the implementation of their interventions in the
fall of 1998, only weeks after the signing of the Workforce Investment
Act (WIA) of 1998 (P.L. 105-220). WIA represented a major reorganization
of the nation's employment programs that was intended to consolidate
preparation and employment services into a unified system of support.
Key to this reorganization was the creation of a national network of
One-Stop Career Centers throughout the United States and its territories
to be a single place where job seekers and potential job seekers can
receive the services that they need to become employed or re-employed
(Morris & Farah, 2002). Although previous Department of Labor employment
programs such as the Job Training Partnership Act had been largely
ineffective in serving individuals with significant disabilities, WIA
presented a new opportunity for beneficiaries to successfully access and
benefit from the same employment services as those available to the
general public (Bader, 2003).

The SPI projects were among the first national efforts to attempt to
include SSA beneficiaries in the WIA services and programs offered
through One-Stop Career Centers. A majority of the SSA funded SPI
projects made the One-Stop Centers a key component of their employment
intervention. As indicated in Table 3, California, Iowa, New Mexico, New
York, North Carolina, Ohio, Oklahoma, and Vermont either located project
staff directly inside One-Stop Centers, recruited participants directly
from One-Stop Centers, or developed memoranda of understanding which
committed the One-Stop Centers to providing employment services to
project participants. SPI projects had considerable prior experience in
benefits planning and assistance, but they had little previous
familiarity with Department of Labor (DOL) funded employment programs.
SPI project staff members forged working relationships with State
Workforce/Economic Development agencies and local One-Stop Centers, and
five states physically located project staff within One-Stop Centers.

The SPI projects used a variety of approaches to increase the ability of
beneficiaries to access and benefit from services provided by One-Stop
Centers. The projects developed several different types of positions to
work directly with beneficiaries and One-Stop Centers, such as
Employment Coordinators, Consumer Navigators, Employer Liaisons,
Consumer Outreach Specialists, and Service Coordinators. Some of these
positions were located in One-Stop Centers, while others were located in
rehabilitation facilities or vocational rehabilitation agencies. In some
states these staff members maintained specialized caseloads, whereas in
other states the positions focused on supporting other staff members in
One-Stop Centers and increasing the ability of the Centers to respond to
the unique needs of beneficiaries.

**Table 3**

**Direct Employment Supports Provided by the SSA SPI Projects**

<table style="width:100%;">
<colgroup>
<col style="width: 10%" />
<col style="width: 7%" />
<col style="width: 9%" />
<col style="width: 9%" />
<col style="width: 7%" />
<col style="width: 5%" />
<col style="width: 7%" />
<col style="width: 8%" />
<col style="width: 8%" />
<col style="width: 8%" />
<col style="width: 9%" />
<col style="width: 7%" />
</colgroup>
<thead>
<tr class="header">
<th>State</th>
<th>One-Stop Center</th>
<th>Supported Employ-ment</th>
<th><p>Navigator</p>
<p>Function</p></th>
<th>Case Manage-ment</th>
<th>Job Train-ing</th>
<th>Place-ment Assist-ance</th>
<th>Job Service Voucher</th>
<th>Peer Mentor-ing</th>
<th><p>Services/</p>
<p>Supports from Local MH/DD Service Provider</p></th>
<th>Transi-tional Employ-ment Program</th>
<th>Psycho-social Rehab.</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td>California</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td></td>
<td>X</td>
<td></td>
<td></td>
<td>X</td>
<td>X</td>
<td>X</td>
</tr>
<tr class="even">
<td>Illinois</td>
<td></td>
<td>X</td>
<td></td>
<td>X</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Iowa</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Minnesota*</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>New Hampshire</td>
<td></td>
<td></td>
<td>X</td>
<td>X</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>New Mexico</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td></td>
<td>X</td>
<td></td>
<td></td>
<td></td>
<td>X</td>
<td></td>
</tr>
<tr class="odd">
<td>New York</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td></td>
<td>X</td>
<td></td>
<td>X</td>
<td></td>
<td>X</td>
<td>X</td>
</tr>
<tr class="even">
<td>North Carolina</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>X</td>
<td></td>
<td></td>
</tr>
<tr class="odd">
<td>Ohio</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td>X</td>
<td></td>
</tr>
<tr class="even">
<td>Oklahoma</td>
<td>X</td>
<td>X</td>
<td></td>
<td>X</td>
<td>X</td>
<td>X</td>
<td>X</td>
<td></td>
<td></td>
<td>X</td>
<td>X</td>
</tr>
<tr class="odd">
<td>Vermont</td>
<td>X</td>
<td>X</td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td></td>
</tr>
<tr class="even">
<td>Wisconsin</td>
<td></td>
<td>X</td>
<td></td>
<td>X</td>
<td>X</td>
<td>X</td>
<td></td>
<td>X</td>
<td>X</td>
<td></td>
<td>X</td>
</tr>
</tbody>
</table>

> \* Minnesota designed its intervention based solely on the delivery of
> benefits planning and assistance services. As a part of its benefits
> planning program, staff members frequently provided information to
> beneficiaries regarding the availability of employment services
> agencies in local communities, but did not directly provide those
> services as a part of the SPI intervention.

Increased Access to Health Care

The Ticket to Work and Work Incentives Improvement Act (Public Law
106-170) included several provisions designed to expand authority
originally granted to states under the 1997 Balanced Budget Act (BBA) to
provide Medicaid coverage to working people with disabilities. These
programs are generally referred to as Medicaid Buy-in programs, since
both the BBA and the Ticket Act authorized states to require
beneficiaries to pay small premiums for this coverage. The various
programs are targeted at individuals who otherwise would not qualify for
Medicaid coverage because their earnings or personal assets exceed
established thresholds. In addition to the Medicaid Buy-in program, the
Ticket Act also established the Medicaid Infrastructure Grants (MIGs)
through the Center for Medicaid and Medicare Services (CMS) to enable
states to develop Buy-in program infrastructure and conduct a variety of
activities designed to promote the employment of people with
disabilities.

As of June 2004, 30 states had implemented Buy-in programs and three
additional states were anticipated to initiate Buy-In programs in the
immediate future. Of the 30 states, nine were SSA funded SPI states who
launched Buy-in programs authorized by the BBA or the Ticket Act either
shortly before or during the implementation of the SPI project. North
Carolina, Ohio, and Oklahoma have not yet established a Medicaid Buy-in.
In addition, all nine SPI states with Buy-in programs also operated a
MIG grant designed to enhance each state's ability to develop an
infrastructure that would support the implementation of the Buy-in.

Table 2.4 identifies the year of implementation, type of Buy-in (BBA or
Ticket Act), and program enrollment as of March 2004. In five of the
nine states, the Medicaid Buy-In was implemented after the award of the
SPI project, but prior to the start of the first Medicaid Infrastructure
Grants on January 1, 2001. Five of the states developed Buy-in under the
guidelines authorized by the BBA. Minnesota was the first state in the
nation to establish a Medicaid Buy-in program under the Ticket Act
authority of 1999.

The SPI states generally have some of the highest Buy-in program
enrollment rates in the nation. As seen in Table 4, the nine SPI states
had enrolled over 24,000 individuals into their Medicaid Buy-in programs
as of March 2004. The SPI states accounted for 38% of all Buy-in
enrollees in the country at that time (51% of all enrollees excluding
the 16,508 individuals enrolled in the state of Missouri). Five of the
ten states with the highest Buy-in enrollments in the country were
states with SPI projects (Iowa, Minnesota, Wisconsin, New Hampshire and
New York).

+--------------+------------------+---------------+--------------------+
| Table 4      |                  |               |                    |
|              |                  |               |                    |
| Medicaid     |                  |               |                    |
| Buy-In       |                  |               |                    |
| Enrollment   |                  |               |                    |
| by State     |                  |               |                    |
+==============+==================+===============+====================+
| **State**    | **Implementation | **BBA or      | **Enrollment March |
|              | Year**           | Ticket Act**  | 2004**             |
+--------------+------------------+---------------+--------------------+
| Iowa         | 2000             | BBA           | 6,520              |
+--------------+------------------+---------------+--------------------+
| Minnesota    | 1999             | Ticket Act    | 6,221              |
+--------------+------------------+---------------+--------------------+
| Wisconsin    | 2000             | BBA           | 6,096              |
+--------------+------------------+---------------+--------------------+
| New          | 2002             | Ticket Act    | 1,294              |
| Hampshire    |                  |               |                    |
+--------------+------------------+---------------+--------------------+
| New York     | 2003             | Ticket Act    | 1,146              |
+--------------+------------------+---------------+--------------------+
| New Mexico   | 2001             | BBA           | 977                |
+--------------+------------------+---------------+--------------------+
| California   | 2000             | BBA           | 960                |
+--------------+------------------+---------------+--------------------+
| Illinois     | 2002             | Ticket Act    | 556                |
+--------------+------------------+---------------+--------------------+
| Vermont      | 2000             | BBA           | 497                |
+--------------+------------------+---------------+--------------------+
| Total        |                  |               | 24,267             |
+--------------+------------------+---------------+--------------------+

Source: Goodman and Livermore, 2004

The SPI states' Buy-in program varied somewhat in terms of the gross
income and earnings limits for Buy-in participants. As described in
Table 5, most SPI states set an annual gross income limit of \$47,570,
reflecting the fact that five of the SPI states established their
Buy-ins under the more restrictive BBA legislation. Minnesota places no
limits on income for Buy-in participants and New Hampshire had the
highest established limit of \$84,810. Illinois had the lowest earnings
limits among the SPI states.

+----------+-------+-------+-------+-------+-------+--------+--------+
| **Table  |       |       |       |       |       |        |        |
| 5**      |       |       |       |       |       |        |        |
|          |       |       |       |       |       |        |        |
| **Buy-in |       |       |       |       |       |        |        |
| Program  |       |       |       |       |       |        |        |
| Gross    |       |       |       |       |       |        |        |
| Income   |       |       |       |       |       |        |        |
| and      |       |       |       |       |       |        |        |
| Earnings |       |       |       |       |       |        |        |
| Limits   |       |       |       |       |       |        |        |
| for**    |       |       |       |       |       |        |        |
|          |       |       |       |       |       |        |        |
| *        |       |       |       |       |       |        |        |
| *Persons |       |       |       |       |       |        |        |
| with     |       |       |       |       |       |        |        |
| Selected |       |       |       |       |       |        |        |
| Income   |       |       |       |       |       |        |        |
| -Related |       |       |       |       |       |        |        |
| Characte |       |       |       |       |       |        |        |
| ristics, |       |       |       |       |       |        |        |
| by       |       |       |       |       |       |        |        |
| State**  |       |       |       |       |       |        |        |
+==========+=======+=======+=======+=======+=======+========+========+
|          | No    | With  |       | With  |       | M      |        |
|          | Une   | \$60  |       | \$120 |       | arried |        |
|          | arned | 0/mo. |       | 0/mo. |       | with   |        |
|          | I     | In    |       | In    |       | \$6    |        |
|          | ncome | Une   |       | Une   |       | 00/mo. |        |
|          |       | arned |       | arned |       | In     |        |
|          |       | I     |       | I     |       | Un     |        |
|          |       | ncome |       | ncome |       | earned |        |
|          |       |       |       |       |       | Income |        |
+----------+-------+-------+-------+-------+-------+--------+--------+
| State    | A     | A     | A     | A     | A     | Annual | Annual |
|          | nnual | nnual | nnual | nnual | nnual | Ea     | Ea     |
|          | Gross | Ear   | Gross | Ear   | Gross | rnings | rnings |
|          | I     | nings | I     | nings | I     | Limit: | Limit: |
|          | ncome | Limit | ncome | Limit | ncome | Spouse | Spouse |
|          | Limit |       | Limit |       | Limit | with   | with   |
|          |       |       |       |       |       | \$10   | \$15   |
|          |       |       |       |       |       | 00/mo. | 00/mo. |
|          |       |       |       |       |       | Cou    | Cou    |
|          |       |       |       |       |       | ntable | ntable |
|          |       |       |       |       |       | Income | Income |
+----------+-------+-------+-------+-------+-------+--------+--------+
| Ca       | \$4   | \$4   | \$5   | \$4   | \$6   | \$     | \$     |
| lifornia | 7,570 | 7,570 | 4,770 | 7,570 | 1,970 | 25,070 | 13,070 |
+----------+-------+-------+-------+-------+-------+--------+--------+
| Illinois | \$3   | \$2   | \$3   | \$    | \$2   | \$     | \$580  |
|          | 8,260 | 3,860 | 1,060 | 9,460 | 3,860 | 12,580 |        |
+----------+-------+-------+-------+-------+-------+--------+--------+
| Iowa     | \$4   | \$3   | \$4   | \$1   | \$3   | \$     | \$     |
|          | 7,570 | 3,170 | 0,370 | 8,770 | 3,170 | 25,070 | 13,070 |
+----------+-------+-------+-------+-------+-------+--------+--------+
| M        | unli  | unli  | unli  | unli  | unli  | unl    | unl    |
| innesota | mited | mited | mited | mited | mited | imited | imited |
+----------+-------+-------+-------+-------+-------+--------+--------+
| New      | \$8   | \$7   | \$7   | \$6   | \$7   | \$     | \$     |
| H        | 4,810 | 0,410 | 7,610 | 0,810 | 5,210 | 75,030 | 63,030 |
| ampshire |       |       |       |       |       |        |        |
+----------+-------+-------+-------+-------+-------+--------+--------+
| New      | \$4   | \$3   | \$4   | \$2   | \$3   | \$     | \$     |
| Mexico   | 7,570 | 3,170 | 0,370 | 3,570 | 7,970 | 33,170 | 33,170 |
+----------+-------+-------+-------+-------+-------+--------+--------+
| New York | \$4   | \$3   | \$4   | \$2   | \$3   | \$     | \$     |
|          | 7,570 | 3,170 | 0,370 | 3,570 | 7,970 | 33,170 | 33,170 |
+----------+-------+-------+-------+-------+-------+--------+--------+
| Vermont  | \$4   | \$3   | \$4   | not   | not   | \$     | \$     |
|          | 7,570 | 3,170 | 0,370 | eli   | eli   | 25,070 | 13,070 |
|          |       |       |       | gible | gible |        |        |
+----------+-------+-------+-------+-------+-------+--------+--------+
| W        | \$4   | \$3   | \$4   | \$2   | \$3   | \$     | \$     |
| isconsin | 7,625 | 3,170 | 0,370 | 3,570 | 7,970 | 25,070 | 13,070 |
+----------+-------+-------+-------+-------+-------+--------+--------+

Source: Goodman and Livermore, 2004.

Many of the SPI projects were extremely involved in the development and
implementation in their states. In a majority of states SPI projects
staff members played key roles in the design and development of Medicaid
Buy-in programs. In a number of states, planning for Medicaid Buy-in
programs had been well underway within the State Medicaid agencies and
State Legislatures at the time of the SPI project initiation. In some
states, such as Wisconsin ad Staff members who had been working on the
planning and implementation of the Buy-in subsequently moved into SPI
project staff positions. In these states, the SPI project was not
responsible of the development of the Buy-in.

In other states, such as Iowa and New Mexico, the activities of SPI
project staff were directly related to the development of Buy-in
programs by State Medicaid agencies and State Legislatures. In these
states, the SPI projects were heavily involved in facilitating planning
groups and planning activities, conducting background studies and
participating in the formulation of specific policies. The SPI projects
used their interagency governmental structures to promote collaboration
in the design of the program. The SPI project resources provided the
capacity to conduct background analyses and develop broad support for
the program at a time when State Medicaid agencies were beginning to see
the effects of reductions in state budgets

SPI projects have worked closely with Medicaid Infrastructure Grants
(MIGs) to promote other health care initiatives such as personal
assistance services (PAS) at the work site. All nine SPI states with
Buy-in programs also operated MIG grants that were designed to develop
the infrastructure components necessary to effectively implement the
Medicaid Buy-in program, as well as to promote the expansion of personal
assistance services in the participating states. SPI project staff
members were involved in the development and actual operation of MIG
grants in at least five SPI states (Iowa, New Hampshire, New Mexico,
North Carolina, and Wisconsin). In virtually every state the SPI project
attempted to coordinate training and systems change activities with the
MIG project. Specific examples of coordinated activities that the SPI
projects conducted with the MIG projects included:

-   In New Hampshire, Project Dollars and Sense staff provided a fiscal
    impact statement, conducted Buy-In research and data development,
    staffed the Buy-In Workgroup, developed drafts of legislation,
    provided data for decision-making, drafted rules and regulations,
    assisted with the development of a Buy-In outreach plan and
    materials, and assisted with the development of an evaluation plan.

-   In Iowa, the Bridge to Employment project engaged in a number of
    collaborative activities with the state MIG grant, including: 1) the
    creation and implementation of Iowa's Medicaid Buy-In program,
    Medicaid for Employment People with Disabilities (MEPD); and 2) the
    promotion of MEPD to SSA beneficiaries and employers.

-   The Minnesota Work Incentives Connection worked collaboratively with
    the state MIG grant to perform statewide outreach for MA-EPD, the
    state's Medicaid Buy-in program. Through its Hotline and Benefits
    Analysis services, project staff also assisted consumers in
    accessing MA-EPD, helped them constructively resolve disputes with
    their financial workers, and ensured that MA-EPD rules were applied
    properly in individual situations.

-   In North Carolina, the SPI project spearheaded an effort to change
    North Carolina's Medicaid State Plan to allow Medicaid coverage of
    personal care or assistance services for individuals with
    disabilities to be provided outside the home to support competitive
    employment.

-   In Wisconsin, the WPTI (Wisconsin Pathways to Independence) worked
    with the state MIG project to create and promote the implementation
    of the Medicaid Purchase Plan (MAPP) to preserve and extended health
    care services for certain, especially SSDI, participants in the
    program. MAPP has become a permanent component of the state's
    Medicaid plan and has served a growing population.

The Medicaid Buy-in programs in the SPI states have been generally
successful, but SPI project staff members have expressed concern that
the program has not led to significant increases in employment outcomes
for participants. The SPI State Projects are generally operated medium
to large Medicaid Buy-in programs. From an employment intervention
perspective, the Buy-in eligibility criteria for virtually all SPI
states is higher than the threshold for the 1619 (b) (Working while
disabled) provision, indicating that the Buy-in has successfully
increased work incentives for SSI beneficiaries. For Title II
beneficiaries, SPI project staff members indicate that the program
provided a strong incentive for beneficiaries who can access Medicaid
through spend down or poverty programs.

**III. Summary of the SPI Systems Change Activities**

The amount and type of systems change activities conducted by the SPI
projects varied considerably between the SSA funded demonstrations and
the RSA systems change projects. The SSA funded SPI projects were first
and foremost research demonstrations that developed intensive,
specialized interventions targeted toward a small group of
beneficiaries, implemented under the limitations of a formal evaluation
design. Systems change activities were allowable tasks under the SSA
funded SPI cooperative agreements, but most programs used the
overwhelming amount of their resources on research interventions and
allocated relatively few resources to systems change efforts. In
contrast, the RSA funded systems change projects had much greater
freedom to direct project resources towards systems change efforts, both
at the local and state level. The information presented below focuses on
the 12 SSA funded projects.

There was considerable variation in the manner in which the various SSA
funded SPI projects engaged in systems change activities. Several states
(e.g. Iowa, New Mexico, and Vermont) devoted considerable resources to
promoting state level policy and legislative changes in areas related to
the expansion of health care options, improved access to the workforce
development system, and improvements to existing funding streams. Other
states (e.g. New Hampshire and New York) focused their attention on the
development of multi-agency governance structures that served as a
vehicle for various state level initiatives. Still other states
(Minnesota, Oklahoma, and Wisconsin) focused on the creation of
statewide service delivery structures that coordinated the use of SPI
resources with other funds to expand benefits planning and employment
services to the largest number of individuals as possible.

Several SPI projects (California, Illinois, North Carolina, and Ohio)
focused virtually all their resources on implementing research
interventions with a targeted population of beneficiaries and spent
relatively little time in systems change activities. While staff members
in these states frequently represented the SPI project on state and
local work groups or other collaborative efforts, systems change
activities comprised only a small part of the projects' overall work
scope.

The SPI Projects in certain states had fairly specific focuses which are
worth mentioning. Illinois had a Psychiatric Specialist Training
Initiative and attempted to get a Medicaid Buy-In. Iowa concentrated on
developing partnerships with businesses. Minnesota concentrated on being
the premier BPAO provider. New Hampshire concentrated on developing
capacity and infrastructure for providing benefits counseling and work
incentives information. Additionally they provided Individual Career
Accounts and Credit Union Access for a subset of their participants. New
Mexico developed a strong emphasis on Peer Associates. Vermont's major
system change initiative was to implement the Individual Placement
Support (IPS) model in the Vermont Community Mental Health System. IPS
is a "person-centered team-based" approach, which is a consumer directed
process which identifies barriers to employment and includes in-depth
benefits analysis and counseling.

**Summary of Systems Change Accomplishments**

The SSA funded SPI projects achieved considerable success in terms of
forming new collaborations and partnerships across multiple state
agencies, encouraging State VR agencies to focus attention on the unique
needs of SSA beneficiaries, and promoting the expansion and coordination
of benefits planning and assistance services in their states. The
projects were less successful in terms of encouraging greater
participation on the part of state mental health agencies, business
organizations, and employers in efforts to promote improved employment
outcomes among SSA beneficiaries.

Sustainability, defined as being able to use and apply what was
developed in, or learned through, the projects to promote improved
employment outcomes for people with disabilities requires special
attention. For example, the Wisconsin project was able to generate
capacity that could be used by external parties to replicate the WPTI
approach, but at the state level, sustainability may be unlikely because
in an era of state funding shortfalls there is little incentive to
bundle separate fee-for-service authorizations into a funding package
that can support an integrated service approach as required in a SPI
Project like Wisconsin's. The WPTI developed and/or funded materials
that documented service approaches and how to train and support the
staff who would use these materials to work with consumers. These
materials are now available for use elsewhere. A substantial portion of
the capacity developed through the WPTI to facilitate the employment
goals of beneficiaries with serious disabilities remains after the
cessation of the SPI Project in Wisconsin. This includes the increased
availability of high quality benefits counseling through the growth in
the number of trained and available benefits counselors and through the
establishment of the Wisconsin Disability Benefits Network which
resulted from the SPI Project.

Benefits counseling existed before the SPI Project, but it was not
delivered as uniformly or as intensely as it was by the SPI Projects
that implemented it as an intervention. It was the intensity level of
the benefits counseling in SPI Projects that created a model that would
serve well for adoption by other entities within States that are trying
to address the issues faced by SSA beneficiaries seeking employment.
Statewide benefits counseling networks and organizations emerged during
the time that SPI existed, such as in Wisconsin and Ohio, that continue
to promote the delivery of consistent, accurate, and timely information
on SSA and other public benefits.

The SPI population benefits from ongoing benefits counseling, during all
phases of the working experience. The New Mexico project noted that
services need to be viewed as stable and consistent over time in order
for participants to fully trust the programs. California reported that
benefits counselors are effective before employment and, for those
clients who went back to work before getting benefits counseling, during
employment. Finally, Illinois reported that benefits counselors were
needed on an ongoing basis in order to deal with errors or
inconsistencies in SSA benefits. In Wisconsin, participants required
significant support, such as job coaching and transportation services to
retain employment. Several projects reported a need to identify
resources for long-term supports from county social service agencies or
case management organizations. Overwhelmingly, the process analysis of
the state projects showed the importance of benefits counseling, concern
about the SGA, consumer satisfaction with increased choice and control.

IV. **What Effect Did SPI Have on Participant Employment?**

**Summary of the Findings from the Impact Evaluation**

The most reliable estimates of the effect of SPI on employment outcomes
come from the analyses in the three states that used random assignment:
New York, New Hampshire, and Oklahoma. Prior analyses (Peikes, Orzol,
Moreno, & Paxton, 2005) indicate that the nonexperimental methods used
by most of the states' internal evaluations and a simple analysis of
participants' outcomes based on the SPI core data set may not provide
reliable estimates of program impacts. Even a sophisticated
nonexperimental method (propensity score matching) that took advantage
of hundreds of variables did not correctly estimate program impacts. A
simple comparison of changes over time among participants without
considering a control group can lead to erroneous conclusions. For
example, in New York, the employment rate of the treatment group offered
benefits counseling and waivers fell over time, but because the control
group's employment outcomes fell even more, it can be concluded that the
project actually increased employment.

To summarize outcome findings in the three states, while benefits
counseling and employment services can sometimes increase the proportion
of beneficiaries who work, none of the interventions increased earnings,
and participants in one project actually reduced earnings substantially.
These findings are based on FICA-covered earnings reported in the
Summary Earnings Record data. The presence of any earnings was used to
identify whether the beneficiary had been employed.

It is important to note that these findings are based on very short
follow-up periods of six to twelve months. It is possible that increased
work exposure will lead to greater earnings in the long term. For
example, the average length of stay in state vocational rehabilitation
agencies is just over two years, and vocational rehabilitation clients
often receive much more intensive interventions than those available to
participants in the SPI project. In most states, the SPI project
intervention represented a relatively modest commitment of resources
that was focused on benefits counseling. While the evidence is somewhat
mixed, the services seem to have increased employment rates but have not
affected or reduced earnings in the short-term. Long-term effects are
uncertain and will depend on the use of other services and the effects
of entry-level employment on long-term career growth and advancement.
While benefits counseling alone seemed to not produce the desired effect
of increasing the rate at which people earned their way off the rolls,
it may, nevertheless, be an important component of a more comprehensive
and long-term strategy.

The three state projects with randomized designs served a population
that was disproportionately SSI and nearly all had a primary or
secondary diagnosis of mental illness. Seventy-four percent were
SSI-only beneficiaries, and 24.5 percent were concurrent beneficiaries;
94 percent had a mental illness. However, the three state projects
designed and delivered interventions that were "typical of those
provided by other projects, emphasizing benefits counseling and targeted
case management that attempted to connect participants with available
employment services.

The three state projects that used randomized designs offered four
different intervention packages**.** New York offered two separate
intervention packages to SSI beneficiaries with psychiatric disabilities
in New York City or Buffalo. The first package provided benefits
counseling and tested changes to SSI regulations that allowed SSI
beneficiaries who worked to retain and save more money. The waivers to
SSI regulations make work more lucrative by letting people who work
retain 75 percent of their earnings, after a disregard, instead of the
usual 50 percent. The waivers increase the wage rate beneficiaries
received from working. Theoretically, this may increase or decrease work
effort and earnings, depending on the beneficiaries\' preferences about
work and non-work activities. On the one hand, increased wages can
increase earnings as the higher wages encourage people to work more and
reduce non-work activities (called the \"substitution effect\").
Increased wages can also decrease earnings as people can earn the same
amount from working less (called the \"income effect\"). The second
package New York tested provided the same intervention as the first and
added employment services to help people to find, apply for, and
maintain work. Unfortunately, New York did not collect data documenting
the hours of project services that it provided.

Oklahoma targeted SSI beneficiaries with psychiatric disabilities in
northern Oklahoma City, Muskogee, Payne, or Tulsa who were not employed
at randomization. Oklahoma provided participants with benefits
counseling and a voucher they could use to obtain services of their own
choosing. Through March 2002, participants who used the vouchers
received an average of 44 hours of services, or four hours per person
per enrolled month. All participants received benefits counseling
(averaging ten hours per month) and job services through the vouchers
(averaging five hours). More than three-quarters of participants
received case management (averaging seven hours). Fewer than one-quarter
of all participants each of the following services: supported
employment, placement assistance, situational assessment, job training,
psychosocial rehabilitation, job accommodations, or transportation
assistance.

New Hampshire targeted SSI and SSDI beneficiaries in Derry, Keen,
Manchester, or Portsmouth. The intervention used a service resource
consultant facilitate beneficiary choice and control over their
vocational services. Participants who completed the resource planning
components of the intervention became eligible to use funds that might
otherwise have been available to them through social service agencies,
such as the state's vocational rehabilitation (VR) agency. The funds
were placed in an Individual Career Account and could be accessed
through a fiscal intermediary. The account allowed the participant,
rather than the agencies, to direct vocational spending. The treatment
group received an average of 15 hours of benefits counseling and eight
hours of case management.

The control groups in New York and Oklahoma had access to the usual
package of services and supports available in their communities. In
addition to the usual services available in the community, the New
Hampshire control group also received an average of 2.5 hours of
benefits counseling from the State Partnership Initiative (SPI) project
(Peikes & Sarin 2005).

**Summary of Impact Effects** - Three of the four intervention packages
fielded by the three states with randomized designs increased the
proportion of participants who worked during the year after the
randomization year compared to the year before randomization by 9 to 18
percentage points relative to a control group. However, in one state
project with a small sample (New Hampshire), the proportion employed
decreased by 30 percentage points relative to the control group.

Despite the promising effects on employment for three of the
intervention packages, the interventions either had no effect on
earnings or a *negative* and statistically significant effect on the
annual earnings of participants ranging between \$1,080 and \$1,633.

Placing these findings in the context of other SSA demonstrations, the
positive employment effects are comparable to those found in the
evaluation of the Transitional Employment Training Demonstration (TETD),
which found that the intervention increased the probability of being
employed by nine percentage points during the sixth year after
randomization. Project NetWork had no effect on the proportion employed.
The negative or no effect on earnings in the SPI interventions, however,
are disappointing because both TETD and Project NetWork increased
earnings by an average of about \$714 per year over the six years after
randomization.

These findings are not influenced by low participation rates in SPI and
are unlikely to be influenced by missing data. The impact estimates
presented above were calculated among participants only. Because not
everyone randomized in Oklahoma and New York actually participated, the
analysis first calculated overall impacts comparing the whole treatment
and control groups. Since it was assumed that nonparticipating people in
the treatment group would not have changed their employment behaviors,
the analysis calculated per participant impacts by dividing the overall
impacts by the participation rates. For example, if the overall impacts
were \$100 but only 25 percent participated, participants' earnings
increased by \$400 (0.25\*400 + 0.75\*0=100). In terms of missing data,
the estimates rely on federal income tax earnings data contained in the
Summary Earnings Record, so the only issue of missing data is that the
files do not cover "off the books" employment; federal, state, local,
and railroad employees; domestic workers earning less than \$1,500 per
employer, farm workers earnings less than \$2,500, and self-employed
people earning less than \$400. As a result, we would expect the
earnings we estimated for both the treatment and control groups to
underestimate their true earnings. For these data exclusions to alter
the impact estimates, the treatment group would need to be more likely
to enter the excluded types of employment as a result of the
demonstration than the control groups. This is possible, but seems
unlikely. Using state unemployment insurance data that does cover state
and local employees, Peikes (2004) estimated the impacts in NY, and the
results were similar to the results based on SER data.

There are several possible explanations for these findings of negative
or no effects on earnings. First, it is possible that benefits
counseling (and the SSI waivers) may enable people to work less to
retain the same income level. This would occur if their benefit amounts
increase sufficiently to offset the decline in earnings, leaving them
with the same total income. Second, it is possible that the findings
reflect the specific population served, which was disproportionately
comprised of SSI beneficiaries with psychiatric disabilities. Randomized
demonstrations with different populations may produce different results.
Third, the results reported above are short-term findings, and state
project staff reported that changing participants' attitudes about work
would take time. Thornton et al. (2005) found that beneficiaries
generally receive VR services for an average of 26 months (780 days)
before a "successful closure" occurs (indicating that the person has
been employed for 90 days). We therefore recommend observing a longer
follow-up period to determine whether annual earning measures increase
to reflect any short-term human capital investments in their human
capital through education or training programs or changes in attitudes
towards work.

A final possible explanation for the lack of positive earnings impacts
is that the interventions employed by the SPI projects were simply not
of sufficient intensity to achieve the intended outcomes. A major
finding of the SPI initiative is that benefit counseling by itself is
not a sufficient intervention that can be expected to significantly
impact the employment of beneficiaries. Without access to intense,
ongoing direct employment services, it appears unlikely that
beneficiaries will be able to obtain and maintain employment at levels
that would lead to economic self-sufficiency and reduced dependence on
SSA benefits. While the three projects that employed randomization
designs all delivered some type of direct or indirect employment
supports (as did the majority of all SPI projects), it appears that
these supports were not of sufficient intensity to overcome the
obstacles to employment faced by beneficiaries.

In summary, the SPI projects had a very weak effect on the employment of
participants. The lack of a strong, positive effect appears to be caused
by the inability of the SPI projects to deliver the amount and type of
employment supports necessary to overcome barriers to employment faced
by participants. It is highly unlikely that the lack of strong positive
impacts is an artifact of low SPI participation rates or missing data.

The major implication of these findings is that SSA should not assume
that the delivery of benefits counseling in isolation will not have any
type of effect on the employment participation of SSA beneficiaries.
When designing future demonstrations and program reforms, SSA should
ensure that beneficiaries have access to employment services of
sufficient intensity and duration to enable them to obtain employment in
their chosen field, successfully adapt to the work setting, and retain
employment even during periods of changing health status and service
interruptions.

**Summary of Findings from the Individual State Project Evaluations**

All Projects attempted to develop a comparison group design and a
pre-post analyses. Projects differed in the extent to which comparison
groups were chosen randomly, had members who were matched to the
participants on relevant characteristics at baseline, or was essentially
a convenience sample. Seven Projects (California, Iowa, Minnesota, North
Carolina, Ohio, Vermont and Wisconsin) had evaluations that included
matched comparison group designs. New Mexico used a non-equivalent
comparison group design.

Four Projects (New York, New Hampshire, Illinois, and Oklahoma)
developed experimental designs with some level of randomization to
participant or comparison group. Only Oklahoma had a true control group,
with SSI beneficiaries randomly assigned to the treatment or control
group prior to any contact by the project. New York, New Hampshire, and
Illinois randomly assigned individuals who expressed interest in the
project to either the treatment or the comparison group.

**Nature of Comparison Groups**

Matched comparison groups were designed to minimize differences between
treatment and comparison groups on key factors such benefit status,
prior educational status, demographic characteristics, and program
status. Most projects wanted to minimize differences between the
participant and comparison groups with respect to SSA benefit status at
baseline (e.g.; SSI, SSDI, or concurrent). Second, they matched on
demographics that have a high correlation with earnings (labor market
proxies) at intake, including prior education, primary disability, and
gender. Third, some projects, notably Vermont and New Mexico, matched on
many characteristics describing the participant relative to the
Vocational Rehabilitation (VR) system (e.g., maturation in the VR
system). California, New Hampshire, Oklahoma and Wisconsin had more than
one comparison group, in an attempt to examine geographical and timing
issues associated with the evaluations.

**The Comparison Being Made: Outcome Measures**

Generally, the evaluation designs and data collection procedures were
designed to detect differences (in both the pre-post analyses and
between participants and comparison group members) on the dependent
variables of employment, earnings and SSI/SSDI benefits (both presence
or absence and amounts). Some Projects also compared trends in benefit
levels between these groups for other federal aid programs, such as food
stamps and subsidized housing, or for state aid programs, especially
state supplements to Medicare and Medicaid. Use of work incentives by
participants was also tracked by five projects.

Most Projects measured employment related outcomes by considering
differences in employment rates, hours worked and earnings, either on a
quarterly or monthly basis relative to intake. Some Projects modeled
differences in these measures between participant and comparison groups
over time. All projects controlled for differences in the groups at
baseline (e.g.; the "difference in difference" approach, etc.).

**Sources of Data**

All Projects collected a core set of data elements (the SPI Core data
set), allowed for the aggregation of data across SPI Projects. Most
Projects also took advantage of administrative data collected from state
level sources. With the exception of Ohio, which used no administrative
data, the Projects accessed these data by means of memoranda of
understanding (MOUs) among the different state agencies. The two main
sources included individual unemployment insurance (UI) data from the
state's Department of Labor (DOL) and data from the state's Vocational
Rehabilitation (VR) system.

**Longitudinal Considerations in the Evaluations**

As described in Piekes, et.al., (2005), obtaining sufficient
longitudinal data to complete planned analyses was a problem for many
projects. Longitudinal considerations enter the evaluation design in
three primary ways. Most Project evaluation designs chose to conduct the
comparison group evaluations on participants and corresponding
comparison group members with at least six or 12 months of follow-up.

Projects differed with regard to the statistical procedures used to
accommodate the repeated measures nature of a longitudinal analysis.
Almost all Projects implemented pre-post Chi-square tests of the
categorical outcome variables of employment, SSI participation, and SSDI
participation. Major analysis methods used by the Projects for
continuous variables such as earnings and SSA benefit amounts include
the difference in differences methodology applied to logistic regression
(for employment or program participation) and ordinary least squares
(for wages and mean benefits). Projects using this approach include New
York, Minnesota, and North Carolina. Repeated measures analysis of
variance (ANOVA or MANOVA) was a primary analysis method in New
Hampshire, New Mexico, and Ohio. Generalized linear models (e.g., mixed
effects and generalized estimating equations) were used in Iowa, Vermont
and Wisconsin. California concentrated on time trend methodology, and
New Hampshire used hierarchical linear modeling.

**Overall Strengths of the State Evaluation Designs**

The evaluation designs employed by the SPI Projects possessed a number
of strengths. The projects served an average of over 600 individuals.
All projects developed comparison groups and four implemented random
assignment designs. Extensive baseline data was collected in all
projects, which allow the use of pre-post analyses. Eight of the
projects created memoranda of understanding (MOUs) to allow confidential
data sharing. The Projects also had access to SSA data for both
participants and comparison group members, which greatly assisted in the
completion of net outcomes analyses.

The rigorous planning, data collection, and outcomes requirements from
SSA helped to expand the human service research base of many of the
Projects and their states. The participant data collected through the
SPI core data set used standard definitions and data collection
procedures. The SPI core data set was a managed database. The Projects
sent their data quarterly to the SPI Project office at VCU, where it was
assessed for quality and completeness, and discrepancy reports were
generated for the Projects. The Projects then corrected the
discrepancies and obtained any missing data. This allowed the Project
Office to aggregate data across Projects for more powerful analyses.
Since the Projects worked directly with the participants, they had a
great deal of success obtaining baseline and follow-up data. This
alleviated many of the issues related to missing or inappropriate data
that often plague administrative data sets.

The richness among the SPI States in their interventions, evaluation
designs and analysis methods are invaluable in understanding what worked
best for improving the employment of participants with disabilities.
Each project documented the service and policy gaps that it sought to
address in order to improve employment and financial independence among
people with disabilities. The evaluation designs attempted to uniformly
address the comparison being made in the evaluation, the methods for
selecting comparison groups, the data used in the evaluation and the
analysis methods used by each state. These designs generated rich data
sets that enable SSA to understand how the characteristics and
participation patterns of participants differed among the states, and
how participants differed from eligible beneficiaries who did not
participate.

**Overall Weaknesses of the State Evaluation Designs**

The SPI projects faced many evaluation challenges that had a serious
negative impact on their ability to analyze the effect of their
intervention participant outcomes. These challenges included
difficulties in identifying appropriate comparison groups, problems in
implementing random assignment designs, contamination of the comparison
group, and lack of control over administrative data, and lack of
sufficient follow-up data collection.

**Challenges in Identifying Appropriate Comparison Groups** -- Obtaining
complete information on comparison group members often made it difficult
to determine the extent to which comparison group members where similar
to participants. It was often not possible to match comparison group
members to participants on key variables such as prior earnings or
education. In some instances, comparison groups were nothing more than a
convenience sample of data on individuals served in one of the partner
agencies participating in an individual project.

**Problems in Implementing Random Assignment Designs** - Random
assignment to the treatment and comparison groups is the best way to
eliminate bias. However, random assignment is very difficult to execute
in human services research. Four Projects (New York, New Hampshire,
Illinois, and Oklahoma) had experimental designs with some level of
randomization to participant or comparison group. Only Oklahoma had a
true control group, as SSA recipients were randomly assigned to the
treatment or control group prior to any contact by the project. New
York, New Hampshire, and Illinois randomly assigned individuals who
expressed interest in the project to either the treatment or the
comparison group.

**Contamination of Comparison Groups** - Contamination of the evaluation
design by comparison group members becoming participants was problematic
for some of the Projects. For example, in Ohio, some comparison group
members were actually wait listed for the project services. Perhaps most
significantly, in all eight projects employing comparison group designs,
the fact that some comparison group members got services that were much
like those provided to the participant. The fact that the BPAO
initiative began implementation during the SPI Project research cycle
made it a challenge for many Projects to administer the comparison group
analyses. Comparison group members got the standard vocational
rehabilitation services, including some employment supports. The service
mix changed over time so that some of the comparison group members were
getting BPAO assistance, which was very similar to the SPI Project
demonstration. Although interventions were as controlled as possible,
there is likely some level of contamination of the comparison groups in
some of the states. The effect of this contamination is to make it more
difficult to prove that the SPI Project had an impact (i.e., to find
significant differences between the participant group and the comparison
group), because members of the comparison group may have received BPAO
services.

**Lack of Control Over Administrative Data** - The lack of control over
the administrative data became a significant problem for the projects.
For example, undercoverage of employment and wages in the UI data is
problematic for all SPI Project designs using UI data. The Wisconsin
project carefully considered this issue. For 279 participants, the
Wisconsin project compared self-reported employment rates to
administrative employment rates at three points in time -- the quarter
of project entry, four quarters after project entry and eight quarters
after project entry. At all three time points, the self-reported
employment rate is higher than the administrative employment rate,
suggesting that the UI system is not capturing some forms of employment
or employers for the WPTI participants. Trend data from the two sources
diverge somewhat over time as well.

**Lack of Sufficient Follow-Up Data** - As was noted earlier, the
minimum follow-up timeframe used by most Projects is much too short to
assess the effectiveness of the intervention on the initial primary goal
of the SPI Initiative; to increase participants' self sufficiency enough
so that they feel comfortable being removed from the SSA rolls. It was,
however, long enough to see significant improvement in that direction.
All Projects recorded significant improvement in earnings of
participants, and several noted reductions in SSA benefits. However, in
order to achieve sufficient sample sizes, a few projects were forced to
change their outreach and recruitment efforts over the course of
implementation. In at least one instance (Wisconsin), this led to cohort
effects, in which individuals enrolled in the latter stages of
implementation differed significantly from individuals enrolled earlier
in the process.

V.  **What Effect did SPI Have on Reduction of SSA Benefits Received?**

**Summary of the Findings from the Analyses Using the SPI Core
Database**

Final analyses using the SPI core database (VCU, 2005) reveal that
participants experienced statistically significant reductions in SSA
benefit payments at each of the 6, 12, 18 and 24 month follow-up
periods. Although some participants experienced a large decrease in SSA
benefits over time, overall the reduction was fairly small. Therefore,
this could not be considered a strong effect. These findings may be
explained by the number of Projects used in the analyses and the type of
analyses performed.

The majority of participants received Medicaid or Medicare, with lesser
percentages of participants receiving Housing Assistance (25%), Food
Stamps (25%) and other State or Local Support (10%). Very few
participants received the other public benefits on which information was
collected. There was considerable movement of participants both on and
off several of the public benefits that were tracked in the database,
primarily occurring in the first six months post intake. In many
instances, this increase followed immediately after the delivery of
benefits counseling services, when the Participant would have become
aware of the potential benefit of the program.

**Limitations of the Findings**

All SPI aggregate analyses performed by the VCU Project Office utilized
SPI Core data, which is the core set of data elements that all Projects
were required to collect.

Although changes in self-reported benefits payments were tracked, it was
not possible to accurately determine the effect of SPI participation on
the number of beneficiaries actually leaving the SSA rolls. Because of
the complexities of SSA enrollment (various levels, including
eligibility with non-payment and overpayments), participant self-reports
of benefits status over time were highly unreliable. Although SSA
administrative data through August of 2003 was made available to the
Projects, and some Projects utilized these data for their individual
Project analyses, many projects were still actively recruiting and
serving participants at that time.

The aggregate employment outcome analyses that were performed by the VCU
Project Office span intake through September 30, 2004. Since an
individual is allowed a period of employment (which was extended for
those participants who utilized SSA waivers) before being removed from
the SSA rolls, it was not possible to perform valid outcome analyses
with SSA administrative data from August 2003. Therefore, the VCU
Project Office did not use SSA administrative files to document this
potential effect of SPI. However, it may be possible to perform these
analyses in the future with SSA administrative data. Benefit periods
vary greatly from participant to participant depending on individual
circumstances, in addition to the types of benefits received (e.g.; SSI,
SSDI, Blind, or Concurrent). Benefits end the month after the first
month that a beneficiary engages in substantial gainful employment
following the extended period of employment. To fully assess the effect
of SPI on participants leaving the SSA rolls, a minimum of three
additional years follow-up (post services) is recommended. By extending
follow-up three years, a larger cohort would have reached the
termination point in their benefit process, and a more valid analysis of
the impact of SPI participation on long-term benefits status could have
been conducted.

VI. **Summary of Final Evaluation of the SSI Work Incentives
    Demonstration Project**

The Social Security Administration (SSA) authorized implementation of
the SSI Work Incentives Demonstration Project, conducted under the
authority of section 1110(b) of the Social Security Act, on January 26,
2001. Also known as the SSI Waiver Demonstration Project, the SSI Work
Incentives Demonstration was implemented by the State Partnership
Initiative (SPI) Projects in California, New York, Vermont and
Wisconsin. Although the Vermont Project enrolled a small number of
participants in March and April, implementation across all four of the
Projects began in May of 2001. The SSI Waiver Demonstration Project
implementation ceased as of September 30^th^, 2004. The Final Evaluation
Report of the SSI Work Incentives Demonstration Project, completed by
the SPI VCU Project Office in 2005, and submitted with revisions to SSA
in May, 2006 (VCU, 2006), constitutes the research analyses and final
report for this endeavor.

The Waiver outcomes were distinguished from the full SPI study outcomes
by performing three independent outcomes analyses. First, to identify
overall change in the Waiver demonstration study sample, statistical
outcome analyses were performed comparing the employment outcomes of
participants who used the Waivers to their employment situations at
intake. Next, to determine whether there are differences in employment
outcomes between the Waiver demonstration study sample and SPI
participants who received the same SPI services, statistical outcome
analyses were performed comparing the employment outcomes of
participants who used the waivers to participants within the same
projects who were eligible for the Waivers, but did not enroll. Finally,
outcomes of participants (who would have been eligible for the Waivers)
served by Projects who did not offer the Waivers were compared to the
outcomes of those participants who used the Waivers.

Two comparison groups were therefore chosen: 1) the participants within
the three state Projects that did not enroll all participants in the
Waivers; and 2) the eligible participants in the other SPI Projects that
were not included in the Waiver demonstration. For analyses to highlight
the Waivers specifically, the Waiver data was aggregated across the four
states. This aggregation is valid, even though there are slight
differences in Waiver tracking methodologies across the four states. In
an attempt to separate the effect of the Waivers from the effect of the
SPI interventions, the primary SPI Project intervention, benefits
counseling, was used as a covariate in the statistical analyses.

The Waiver Demonstration was conducted in addition to other services
provided as components of the State Partnership Initiative. Viewed as a
complete package, the analyses provide an indication of the effects of
the Waiver Demonstration above and beyond the SPI Project interventions,
but not independent of the SPI implementation. The analyses reviewed the
data from three different vantage points to attempt to differentiate the
Waiver component from the rest of SPI. However, wide variation in the
implementation of the benefits counseling component of SPI precluded the
analyses from being as independent as originally conceptualized.
Therefore, to provide a valid comparison, any replication of this Waiver
Demonstration would need to include at least the benefits counseling
component as piloted in SPI.

For some Waiver participants, especially those who were unemployed at
intake, participation in the Waiver Project had a sizable positive
effect on their employment outcomes. Although over a third of
participants did not attain employment, and participants who were
employed at intake experienced far less positive outcomes, overall there
was significant improvement.

Even when the comparison group analyses accounted for the variance in
gross wage change related to these demographic and prior experience
differences in the sample, the Waiver participants had a significantly
greater mean improvement in wages over those who were employed, but were
either served by the same Projects but did not received the Waivers, or
were served by Projects that did not offer the Waivers. It is very
interesting that the two comparison group analyses turned out so much
alike. Although different independent variables were chosen (because
they were based on the comparisons of the differences in demographic
variables between the groups), the same variables ended up being
significantly different in the two analyses.

For both of the comparison group analyses, the most significant variable
was employment at intake, with those who were not employed at intake
having a much greater increase in income. This is logical, as those who
were employed at intake also have the potential to have a decrease in
earnings, and in fact many participants did. Further review of the data
in the form of case studies could reveal which of the wage reductions
was related to Project services, and which were totally unrelated (e.g.;
participant became ill or moved out of the area). Although there was a
great deal of variance in change in wages for those Waiver participants
employed at intake, the average change was positive.

For both of the comparison group analyses, in addition to receipt of the
Waivers and employment at intake, primary disability, race and prior
education were found to have a significant relationship with changes in
gross earnings. Prior education was the most highly significant of these
demographic variables, with those with college experience having greater
improvement (VCU, 2005, pp. 41 and 43). Race was just barely
significant, and is possibly due to regional differences. Primary
disability was probably significant because the New York Project
targeted individuals with mental illness, and did not contribute
participants to either comparison group. Regardless, the statistical
analysis used partitioned the variance in gross wage attributable to
these three variables, and concluded that Waiver participation had a
significant relationship to earnings even when demographic difference in
the samples were accounted for.

Although these analyses show that the Waivers did indeed have a positive
effect, the Waiver (and SPI in general) did not help every participant.
A very sobering fact is that over one-third of the Waiver demonstration
sample never became employed during the course of this study. A full 644
waiver participants (38.4% of the 1676 Waiver participants for whom data
were available) remained unemployed throughout the study. Additional
case reviews could possibly reveal potential reasons for why these
participants did not obtain employment, both related and unrelated to
the Waiver demonstration and State Partnership Initiative.

**VII. Implications for Future SSA Demonstrations**

The findings of the State Partnership Initiative are mixed. If one was
to look at the impact estimates and outcome analyses, the SPI projects
were not able to demonstrate a significant impact on the employment
outcomes and benefits status of SSI and SSDI beneficiaries. In general,
Participants did not experience increased earnings, and there was only a
minimal increase in the rate of employment (Peikes &Sarin, 2005; Peikes,
Orzol, Marino, & Paxton, 2005; VCU, 2005). SSI Waiver Demonstration
participants, especially those who were not employed at intake, faired
better than the general SPI participant group, but in absolute terms
their employment and benefit outcomes were not large enough to have a
significant impact on their economic self-sufficiency (VCU, 2005).

Yet, the SPI projects did have a major positive impact both on many
Participants' lives and on the State systems that provide employment
services and supports to SSA beneficiaries. This finding is confirmed
throughout the process evaluations conducted by the State Projects and
reported in their individual Internal Evaluation Reports. Not only did
SPI have an impact on system change within States, it also had a
national impact as evidenced by the following (VCU, 2005):

-   SPI Projects developed interagency governance structures that
    resulted in numerous state agencies working together to address
    barriers to employment for SSA beneficiaries for the first time.

-   The SPI Projects led the way in the establishment of a nationwide
    system of Benefits Planning Assistance and Outreach (BPAO) Projects,
    with many staff involved in the on-going training provided to these
    projects.

-   Several SPI projects were instrumental in facilitating the
    development and/or implementation of Medicaid Buy-Ins in State
    Projects, at first through the Balanced Budget Amendment and later
    through the Ticket to Work and Work Incentives Improvement Act
    (TWWIIA).

-   The model for the Disability Navigators initiative within the
    One-Stop Career Center system that is currently under the Employment
    and Training Administration of the U.S. Department of Labor was
    initially developed through the Colorado RSA-funded SPI Project.

-   The SPI Projects had considerable success in encouraging State VR
    agencies to place a new emphasis on services and supports leading to
    employment of SSA beneficiaries.

-   In a number of SPI Projects, the use of benefits planning and
    assistance services by the State Vocational Rehabilitation agency
    became a "routine" component of service delivery for SSA
    beneficiaries.

-   Multiple SPI projects demonstrated effective strategies for
    coordinating the efforts of employment service projects with local
    SSA Field Office staff.

While the State Partnership Initiative failed to validate a set of
evidence-based practices that have a strong, positive impact on the
employment status, earnings, and benefits status of beneficiaries, the
Initiative generated a great deal of information that can assist SSA in
the design and implementation of future demonstrations. Major
implications of the initiative are discussed below. Where appropriate,
recommendations for the design of future SSA demonstrations are also
provided.

**Data collection systems need to be developed prior to enrollment.**
The cooperative agreements with the State Projects and the contract
creating the SPI Project Office came into existence almost concurrently
at the beginning of FY 1999. Although most Projects did not begin
enrollment of participants right away, the California Project began to
serve clients in January 1999. At that point, the data collection system
was still under development and still needed testing. VCU did provide
California, and the other Projects that began enrollment, with draft
data forms, but these forms changed significantly with additional data
elements being added to meet the expectations of SSA, as well as
clarification as to what were the required data elements for the RSA
Projects. Much of the confusion and additional work that was required in
going back and collecting the additional information and then putting it
into the format of the final data entry system could have been avoided
if the data collection system had been ready for testing prior to
enrollment.

Once a data collection system has been created, tested, and approved, it
is imperative that there be clear definitions of the data elements in
writing, and that all data entry personnel from all Projects receive
training in the use of the data collection system. The VCU SPI Project
Office developed a very extensive, complete dictionary for the data
elements, but the Project Office did training and interpretation of
definitions as questions arose, rather than prior to data collection.

> **Recommendation: When designing future demonstrations, SSA should
> ensure that data collection systems are developed prior to
> participation enrollment. In addition, SSA should ensure that training
> in the use of the data system be available in multiple formats and
> available throughout the course of the demonstration.**

**For a research and evaluation initiative such as SPI that involves
multiple states and multiple sites within states, SSA should require the
use of a single data entry program.** For multiple reasons, including
the accommodation of Projects that had a previously existing data
system, the VCU Project Office allowed three different methods of data
collection for the SPI Project. One was the direct collect of
information using the VCU forms and submitting the hard copies that the
VCU SPI Project Office entered into the data system. The second method
was through the use of a computerized data program that was identical to
VCU data forms. The third method involved using a data collection
system/database that the State Project had access to, and putting in and
pulling from it the data elements required in the SPI database. This
third method of data collection usually involved a combination of direct
collect data and administrative data.

At the beginning of the SPI Initiative, State Projects were given the
expectation that all data would be collected directly from participants
no matter the method of data collection, but this quickly became
unrealistic for a number of the Projects. Therefore, a combination of
direct collect and administrative data was accepted, but the expectation
remained that follow-up contacts would result in direct data collection
on those applicable data elements. The Projects using both direct and
administrative data, and those not using the SPI data collection forms,
resulted in a need for translation of the Project's database into the
SPI database. This had to be done individually for each of these
Projects because methods of data collection were different. This was
extremely cumbersome and took up much more time than expected. Also, it
required many iterations of verification.

> **Recommendation: When designing future demonstrations, SSA should
> determine the data elements that must be collected directly from
> participants and the data elements that can be obtained through
> administrative data sets, and then require each individual
> demonstration site to follow the same data collection approach.**

**The SPI projects had an extremely difficult time collecting detailed
service data from collaborating agencies and organizations.** The SPI
projects found that collecting detailed data on the services provided by
project staff to individual participants was valuable for both their
process and outcome evaluations. This was particularly useful for
projects that were developing and piloting new staff roles such as
benefits planners, consumer navigators, or employment support
counselors. At the same time, the projects had an extremely difficult
time obtaining information from partner organizations such as One-Stop
Career Centers and community mental health centers on the amount and
type of services provided to specific project participants.

When conducting rigorous demonstrations, it is imperative that SSA
understands the extent to which beneficiaries are able to access and
benefit from services provided by partner agencies such as vocational
rehabilitation, workforce development, developmental disabilities, and
mental health. Self-reports from beneficiaries through surveys or other
direct data collection efforts have proven unreliable. Administrative
and program management databases maintain by partner agencies appear to
be the most valid source of this information, but the SPI initiative
illustrates the difficulties inherent in obtaining this data.

> **Recommendation: When designing future demonstrations, SSA should
> require that each individual site develop a plan (including the use of
> financial incentives) for collecting accurate and reliable information
> on the type and amount of services provided to participants by
> specified partner agencies.**

**The SPI projects were able to access other state administrative data
sets, but had to overcome administrative and logistical obstacles to
obtain these data**. The majority of the SPI projects were able to
access administrative data from other agencies in their state. These
data were essential for verifying information obtained through direct
data collection from participants, obtaining follow-up data on project
participants, and acquiring data on comparison or control group members.
However, the process for obtaining these data was very challenging in
many states. One state had to involve the state attorney general's
office to obtain permission for data sharing. The HIPPA requirements
made it necessary for many states to review existing data sharing
agreements and temporarily suspend access to administrative data. Other
states had a very difficult time actually merging the data.

> **Recommendation: When designing future recommendations, SSA should
> require that individual demonstration sites have appropriate
> agreements in place with multiple state agencies to ensure access to
> any administrative data sets deemed necessary for evaluation purposes.
> These agreements should be in place prior to the initiation of
> demonstration activities.**

**The SPI projects should have used more state administrative data in
their process evaluation activities.** After the SPI projects revised
their evaluation designs to implement primarily quasi-experimental
impact designs, they primarily focused on the impact analyses as the
primarily use of administrative. However, many SPI projects could have
benefited from administrative data that would have addressed process
evaluation questions. For example, the California and Iowa projects
could have addressed process evaluation questions in part by access to
their state VR 911 data system. The Colorado project attempted to access
data from the state workforce development system's Job Link
administrative set.

> **Recommendation: When designing future demonstration activities, SSA
> should require individual sites to consider the use of administrative
> data sets in the development and implementation of internal process
> evaluation activities.**

**The SPI projects found that tracking participants over time was much
more difficult and time consuming than they had initially anticipated**.
The projects that relied extensively or exclusively on data directly
collected from participants found that the process became increasing
difficult as the follow-up period lengthened. Several projects that
originally proposed to collect data from participants over the entire
course of the project subsequently requested permission from their
project officer to shorten the follow-up period to 12 or 24 months. The
problems experienced by the SPI projects in tracking participants over
time were exacerbated by the projects general lack of data collection
resources. Several projects added additional staff in the later years of
the project to focus on locating participants and obtaining follow-up
information. Others attempted to elicit support from partner agencies in
locating individuals.

> **Recommendation: When designing future demonstration initiatives, SSA
> should require that each individual site allocate sufficient resources
> to enable them to track participant experiences for a minimum of 24
> months after the receipt of services. This will assist SSA in (1)
> determining the long term impact of project interventions and (2)
> interpreting the findings of national evaluations relying on
> administrative data.**

**Several SPI projects did not have sufficient technical expertise to
implement rigorous internal evaluation designs**. A few of the State
Projects had external evaluators built into their Project's budgets, and
so did not require more than perfunctory interaction with the VCU
Project office for the implementation of the evaluation design of their
Project. Other Projects had an evaluation design written into their
Project's proposal but did not have any staff associated with their
Project with an evaluation background. Therefore, the VCU SPI Project
Office developed the flexibility of providing maximum evaluation support
(evaluation design, implementation, and analysis) with the Project's
that needed a particular level of assistance. By being flexible in the
amount and way technical assistance was provided, the State Projects
have been able to use the resources provided through the VCU SPI Project
Office to complete an interim evaluation of the process they have
undergone and the outcomes of their interventions.

Many SPI projects initially hired evaluation specialists who were basic
database programmers with prior experience conducting process evaluation
and participation analyses. When the evaluation requirements were
tightened for the state projects in the second year SPI these evaluation
staff members were not equipped to perform the tasks necessary to
implement the more rigorous evaluation designs. Specific problems
included: the methodological skills required to identify comparison
groups; the database manipulation skills that allowed the merging of
multiple administrative data sets; and the statistical knowledge and
data manipulation skills necessary to prepare the resulting large data
sets for analysis and perform the appropriate analyses. A few projects
even had difficulty finding qualified SAS programmers, performing
analyses on large data sets using Microsoft Access or other database
programs.

> **Recommendation: When designing future demonstration initiatives, SSA
> should require that individual sites obtain sufficient technical
> expertise and allocate sufficient resources to enable each site to
> conduct a rigorous internal evaluation and support the efforts of
> SSA's national or aggregate evaluation efforts.**

**A research design for evaluation purposes needs to be developed prior
to implementation of Social Security Administration Waiver
Demonstrations.** The SSI Waiver Demonstration implementation began in
the Spring of 2001, almost 2 ½ years after the initiation of the SPI
Project. The four State Projects that are involved in the SSI Waiver
Demonstration agreed to evaluate the impact of the Waivers through
participant outcomes, but developed their evaluation plans for the
impact of the Waivers after implementation began. Three of the four
State Projects (Wisconsin, New York, California) implemented all four
components of the Waiver Demonstration, while one Project (Vermont) only
implemented three of the components. The absence of a research design
prior to implementation resulted in little consistency in the Waiver
specific data elements that were collected among the 4 Demonstration
State Projects.

The VCU SPI Project Office was asked to provide SSA with preliminary
results of the SSI Waivers in 2004. This was very difficult, not only
because of the above inconsistencies in the data being collected, but
also because there was no over-all research design developed prior to
the implementation of the Demonstration.

> ***Recommendation: When incorporating waivers of program rules into
> future demonstration activities, SSA should require that an over-all
> research design be in place prior to implementation and that core data
> elements be specified and collected by each individual demonstration
> site. The research design should allow SSA to assess the effect of the
> waivers independent of all other demonstration interventions.***

**References**

Bader, B.A. (2003). [Identification of best practices in One Stop Career
Centers that facilitate use by people with disabilities seeking
employment]{.underline} (Doctoral dissertation, Virginia Commonwealth
University, 2003). Dissertation Abstracts International, AAT 3091823.

Goodman, N., & Livermore, G.A. (2004). [The effectiveness of Medicaid
Buy-In programs in promoting the employment of people with
disabilities]{.underline}. Washington, DC: Cornell University Institute
for Policy Research.

Kregel, J. and Head, C. (2004). [The experiences of the first 100,000
individuals participating in the national BPAO initiative]{.underline}.
Richmond, VA: Virginia Commonwealth University, Benefits Assistance
Resource Center.

Morris, M., & Farah, L. (2002). [Building relationships at a community
level: Lessons learned from Work Incentives Grantee (WIGs).]{.underline}
Iowa City, IA: University of Iowa College of Law, Law, Health Policy &
Disability Center.

Peikes, D. (2004). [New York Works: Summary of impact
estimates]{.underline}. Princeton, NJ: Mathematica Policy Research, Inc.

Peikes, D., Orzol, S., Moreno, L., & Paxton, N. (2005).
[State]{.underline}

[Partnership Initiative: Selection of comparison groups for the
evaluation]{.underline}

[and selected empact estimates.]{.underline} Princeton, NJ: Mathematica
Policy Research,

Inc.

Peikes, D., & Sarin, A. (2005). [State Partnership Initiative: Synthesis
of impact estimates generated by the state projects'
evaluation.]{.underline} Princeton, NJ: Mathematica Policy Research,
Inc.

Thornton, C., Fraker, T., Livermore, G., Stapleton, D., O'Day, B.,
Silva, T., Martin, E.S., Kregel, J., & Wright, D. (2005). [Evaluation of
the Ticket to Work Program: Second evaluation report]{.underline}.
Washington, DC: Mathematica Policy Research, Inc.

Virginia Commonwealth University (VCU). (2005). [2005 evaluation
report]{.underline}. Richmond, VA: Virginia Commonwealth University
Rehabilitation Research and Training Center on Workplace Supports.

Virginia Commonwealth University (VCU). (2006). [Final evaluation report
of the SSI Work Incentives Demonstration Project]{.underline}. Richmond,
VA: Virginia Commonwealth University Rehabilitation Research and
Training Center on Workplace Supports.

[^1]:  These states are California, Illinois, Iowa, Minnesota, New
    Hampshire, New Mexico, New York, Vermont, and Wisconsin.
    Implementation in New York began in 2003, after enrollment had
    stopped.