TICKET TO WORK

AND WORK INCENTIVES

ADVISORY PANEL

Advice Report to the Commissioner

of the Social Security

Administration:

Design Issues Relating to

the Adequacy of Incentives Study

June 18, 2002

TABLE OF CONTENTS

Executive Summary 1

Message from the Panel 7

Summary of A Social Security Administration Report:

Working Paper 003—Design of the Study of the Adequacy of Incentives

for the Ticket to Work Program 9

Summary of Experts Roundtable, July 27, 2001 11

Summary of Issues 15

Questions and Recommendations 19

Summary 33

Appendices 35

1. Ticket To Work And Work Incentives Advisory Panel

2. Design And Evaluation Committee

3. List Of Experts, Experts Roundtable, July 27, 2001

4. Public Commenters, July 27, 2001

5. Individuals Providing Public Comment In Writing

Following July 27, 2001

6. Staff, July 27, 2001

7. Presenters, July 27, 2001

8. References and Bibliography

1

EXECUTIVE SUMMARY

The purpose of the Panel’s Advice Report is to provide analysis and recommendations

regarding design of the Adequacy of Incentives Study authorized by the Ticket to Work

and Work Incentives Improvement Act of 1999. The Adequacy of Incentives Study was

mandated by Congress to evaluate how the Ticket to Work and Self-Sufficiency Program

can be used to increase employment among those with significant disabilities. The

Commissioner is required to report to Congress on recommendations for a method or

methods to adjust payment rates to Employment Networks to ensure the participation of

individuals with disabilities in four specific groups. The Commissioner must implement

the necessary adjusted payment rates prior to full implementation of the Ticket to Work

and Self-Sufficiency Program. The statute requires that the Panel be consulted during the

development and evaluation of the study.

This Advice Report is based on relevant documents, including reports from an Experts

Roundtable and public testimony coordinated and convened by the Design and

Evaluation Committee of the Work Incentives Advisory Panel in Washington, DC, on

July 27, 2001. In addition, this Advice Report is based on related research on

employment of people with significant disabilities and targets the four groups in the

disability population noted in the Adequacy of Incentives provision of the legislation who

are likely to be “involuntary nonparticipants.” These four groups are individuals with a

need for ongoing support and services, individuals who need high-cost accommodations,

individuals who earn a subminimum wage, and individuals who work and receive partial

cash benefits.

Summarized in this Advice Report are a report by the Social Security Administration

(Working Paper 003, Design of the Study of the Adequacy of Incentives for the Ticket to

Work Program, Huynh and O’Leary, July 14, 2001); key points from the July 27, 2001,

Experts Roundtable; and Panel and Experts Roundtable recommendations that will

provide a framework for additional research related to the Adequacy of Incentives. The

points offered in this Advice Report are intended to support and enhance, not supplant,

the overall proposal described by the Social Security Administration’s Office of Policy in

Working Paper 003 and discussed at the Experts Roundtable.

PANEL RECOMMENDATIONS

The 13 recommendations offered here include recommendations from the Panel related to

the ongoing process for the Adequacy of the Incentives Study. They are followed by

recommendations from the Experts Roundtable on nine specific issues raised by the

Social Security Administration (SSA) and/or the Panel.

Design Process of the Study

Panel Recommendation 1: An ongoing structure must be established for review of

emerging information, analysis of trends, and creation of recommendations regarding

modification of incentives and making use of multiple evaluation methods.
2

Panel Recommendation 2: The Social Security Administration should convene an

Adequacy of Incentives Advisory Team, similar to the team of consultants approach

being used under the $1 for $2 research, for ongoing technical support for the Adequacy

of Incentives Study, specifically related to the four targeted groups. This group should

consist of diverse individuals who have research background, knowledge, and/or

expertise in the successful employment of individuals in the four groups. This group

could include, but not be limited to, the experts identified by the Panel for the Adequacy

of Incentives Expert Roundtable. These experts would be charged to further develop

recommendations from the Experts Roundtable for serious consideration by the Social

Security Administration. At least two members of the Adequacy of Incentives Advisory

Team should become members of the Technical Evaluation Support Group, the technical

group advising Social Security Administration on overall Ticket evaluation, to ensure

coordination of the two research efforts.

Panel Recommendation 3: Given the impact of this and other studies the Office of

Policy conducts on the lives of people with disabilities and their families, the Office of

Policy should develop a policy and practice for incorporating the input and experiences

of consumers, their families, and/or their representatives as it develops new research

designs, including the Adequacy of Incentives Study.

Panel Recommendation 4: As the Ticket to Work and Self-Sufficiency Program is

implemented, the Social Security Administration should provide to the Panel, Congress,

and the President ongoing interim reporting on the participation of members of the four

targeted groups in the Ticket to Work and Self-Sufficiency Program.

Panel Recommendation 5: The Panel recommends that Congress approve a technical

amendment to require the Social Security Administration to provide to the Panel,

Congress, and the President ongoing interim reporting on the participation of members of

the four targeted groups in the Ticket to Work and Self-Sufficiency Program.

Panel Recommendation 6: Ongoing data collected and outcomes determined for the

Adequacy of Incentives Study should be included in the Social Security Administration’s

overall Ticket evaluation research.

Panel Recommendation 7: Due to delays in issuing the tickets, the deadline for the

Adequacy of Incentives Report to Congress should be extended by the amount of time the

ticket rollout has been delayed (13 months). The Panel recommends that Congress

approve an amendment to adjust the timeline for the Adequacy of Incentives Report so

that it is due 49 months, rather than 36 months, after the enactment date. Implementation

of the necessary adjusted payment rates should occur as soon as practical following the

release of the Adequacy of Incentives Report but no later than 1 year after its release.

Further, the Panel requests preliminary progress reports on the Adequacy of Incentives

Study at 36 and 42 months after the date of enactment of the law.
3

Identifying the Four Groups

Panel Recommendation 8: While there is a need to take advantage of what is known

about the diagnostic categories with high proportions of those most likely to fall into one

of the four groups relative to work capability, the Social Security Administration, in

addition to diagnostic data, should use other information, such as survey information,

information on earnings, onset date, and medical improvement assessments. SSA should

also go to outside data sources, such as those identified in Expert Roundtable

Recommendation 1 (below), to identify members of the four target groups.

Survey Methodology

Panel Recommendation 9: The Social Security Administration has proposed a survey to

supplement its administrative data, improving its ability to target the four groups. If the

survey uses a telephone protocol, the protocol should include two other elements. First,

there should be face-to-face home interviews if the phone survey fails for a given

respondent, except for cases of incorrect phone numbers, empty residences, and so on.

Second, if the potential respondent’s disability prevents effective participation in the

survey, the interviewer should use as a proxy another individual who knows the

respondent well and, preferably, who has communicated on the respondent’s behalf in the

past, such as a parent, spouse, sibling, or supervisor in a group home. Members of the

Adequacy of Incentives Advisory Team (see Panel Recommendation 2) should review

the interview protocol used by the contractor conducting the survey.

Supplementary Research

Panel Recommendation 10: Recognizing the need for research to supplement what can

be studied using Social Security administrative data, SSA has proposed a program of

supplementary research. It has solicited the suggestions of the Panel for possible research

topics to supplement the research using SSA data. The Panel recommends the following:

 Review existing research on the four groups or related groups, including relevant

behavioral demonstrations and field experiments.

 Conduct or commission new research on these groups using State data or other

Federal data.

 Conduct or commission new research related to Employment Networks and other

providers.

 Review and consider best practices of successful programs relating to employment of

those with significant disabilities.

 Review best practice systems of results-based funding and/or individual milestones

payments.

 Conduct or commission a review of supported employment literature, research, and

best practices.

 Conduct or commission ongoing demonstration projects, modeling, and evaluation of

the effectiveness of alternative incentives identified.
4

Employment Networks

Panel Recommendation 11: SSA should identify the data elements that Employment

Networks (ENs) should collect on whom they serve and do not serve from the four

functional groups identified in the statute. The data should include descriptions of why a

ticket is accepted or is not accepted on the basis of the EN’s perception of cost, degree of

difficulty, or lack of expertise on the part of the EN to provide the necessary services.

SSA should use a sampling of ENs rather than requiring all ENs to collect the data. SSA

should pay the ENs to offset the administrative burden of management information

system design.

Panel Recommendation 12: SSA should conduct or commission qualitative research,

including interviews and focus groups as appropriate, to assess the adequacy of current

incentives to large and small providers, including community rehabilitation programs.

Information and recommendations should be collected regarding the amount and timing

of payments and other factors that affect potential participation in the Ticket Program and

integrated community employment at minimum wage or higher levels for individuals in

the four groups.

Various and Diverse Populations

Panel Recommendation 13: SSA should ensure that the data collected include attention

to national demographics such as rural and urban settings, minority representation, and

socioeconomic conditions across the country.

RECOMMENDATIONS FROM THE EXPERTS ROUNDTABLE

The Panel convened an Experts Roundtable on July 27, 2001, as part of its effort to bring

expertise to bear in advising SSA on the design of the Adequacy of Incentives Study.

Following are the recommendations that emerged from the Experts Roundtable in

response to the nine questions posed by the Social Security Administration and the Panel.

The Panel strongly urges the Social Security Administration to seriously consider and

further develop these recommendations in designing research related to the Adequacy of

Incentives Study and its context within the broader evaluation of the Ticket to Work and

Self-Sufficiency Program.

Identifying the Four Groups

Roundtable Recommendation 1: In addition to the administrative data sets SSA has

proposed using, use appropriate Federal databases (e.g., Office of Special Education and

Rehabilitative Services, Rehabilitation Services Administration, and Department of

Labor); State databases (e.g., State developmental disability agencies and State mental

health agencies); data from providers of day treatment, day activity programs, sheltered

workshops, and supported employment programs (e.g., data from the Employment

Intervention Demonstration Program funded by the Center for Mental Health Services);

and existing research studies in order to identify and gather information from and about
5

members of the four groups. In addition, the Helen Keller National Center has data on

7,000 persons who are deaf and blind, some of whom are seeking employment.

Roundtable Recommendation 2: Oversample potential members of these four groups

who can be located in the administrative data from Social Security, Office of Special

Education and Rehabilitative Services, Rehabilitation Services Administration, Medicaid,

the Department of Labor, and/or other Federal sources.

Outcomes of Interest

Roundtable Recommendation 3: Track basic employment outcomes (jobs, wages,

hours, choice, etc.) for members of the four groups once identified.

Roundtable Recommendation 4: Gather information about access to and use of the

Ticket, with particular attention to whether nonparticipation is voluntary or involuntary.

Roundtable Recommendation 5: Conduct or commission qualitative research, including

interviews and focus groups, to gather more extensive information about participation,

economic and other benefits, and ways to attain improved incentives.

Roundtable Recommendation 6: Analyze the data collected related to patterns of

outcomes, cost/benefit, participation, and modifications of incentives for members of

these four groups compared with others making use of the ticket.

Maximizing Outcomes

Roundtable Recommendation 7: The Social Security Administration should ensure that

studies of these four groups include attention to the issue of the “mix” of funding,

government programs, and other incentives and disincentives that affect the employment

of people with disabilities. These studies should include analysis of how various funding

streams can be used together to enhance incentives to Employment Networks and

providers and result in employment for people in the four groups.

Successful Efforts to Employ People With Significant Disabilities in Community

Employment at Minimum Wage Levels or Higher

Roundtable Recommendation 8: The Social Security Administration should study the

successes of supported employment programs, Rehabilitation Services Administration

Choice projects, and other demonstration projects of the past decade for members of

these four groups.

Other Methods to Identify Members of the Four Groups

Roundtable Recommendation 9: The Social Security Administration should consult

other databases to identify members of the four groups. Such databases should include

other existing national samples as well as State databases for persons with developmental
6

disabilities, persons with mental illness, persons with deafness and blindness, persons

with brain injuries, and persons with other disabilities.

Individualized Milestone Payments

Roundtable Recommendation 10: The Social Security Administration should study the

current and recent experiences of States that have begun using results-based funding or

some form of payment for individualized milestones toward employment.

Roundtable Recommendation 11: The Social Security Administration should conduct

or commission cost/benefit analyses and create demonstration projects on modifying the

incentives to Employment Networks and providers of services, showing how members of

the four groups can benefit from the Ticket and from employment that partially reduces

benefits.

Various and Diverse Populations

Roundtable Recommendation 12: The Social Security Administration should ensure

that the data collected include attention to national demographics such as rural and urban

settings, minority representation, and socioeconomic conditions across the country.

Including Consumers, Family Members, and/or Their Representatives in the Study

Design and Implementation

Roundtable Recommendation 13: Given the impact of this and other studies the Office

of Policy conducts on the lives of people with disabilities and their families, the Office of

Policy should develop a policy and practice for incorporating the input and experiences

of consumers, their families, and/or their representatives as it develops new research

designs, including the Adequacy of Incentives Study.

Including Employment Networks in the Study Design and Implementation

Roundtable Recommendation 14: The Social Security Administration should gather

data related to the costs to Employment Networks and providers of services and the

timeframes for providing access to the Ticket and employment for members of these four

groups.

Roundtable Recommendation 15: The Social Security Administration should study the

relationship between perceived disincentives and actual incentives for Employment

Networks and providers of services, including incentives for small providers of services.
7

MESSAGE FROM THE PANEL

In enacting the Ticket to Work and Work Incentives Improvement Act of 1999, Congress

recognized that the Ticket to Work and Self-Sufficiency Program would not be equally

available to all individuals with disabilities. In particular, some people with significant

disabilities might be involuntary nonparticipants. That is, they might be willing to work

but, because of their need for ongoing support and services, high-cost accommodations,

limited wages, and/or partial cash benefits, as well as outside factors, they might not find

an Employment Network willing or able to take their ticket under the existing financial

incentives of the Ticket to Work and Self-Sufficiency Program.

To address this equity issue, Congress included a requirement in the statute that the

Social Security Administration identify and implement a payment system that would

allow this population to participate in the Ticket Program. The Commissioner is required

to report to Congress on recommendations for a method or methods of adjusting payment

rates to Employment Networks to ensure equitable participation for individuals in four

specific groups. The Commissioner must implement the necessary adjusted payment rates

prior to full implementation of the Ticket to Work and Self-Sufficiency Program. The

statute requires that SSA consult with the Panel during the development and evaluation of

the study—commonly referred to as the Adequacy of Incentives (AOI) Study. The

statutory language follows:

Title I, Sec.101 (h)(C) REPORT ON THE ADEQUACY OF INCENTIVES—The

Commissioner shall submit to the Congress not later than 36 months after the

date of the enactment of the Ticket to Work and Work Incentives Improvement Act

of 1999 a report with recommendations for a method or methods to adjust

payment rates under subparagraphs (A) and (B), that would ensure adequate

incentives for the provision of services by employment networks of—

(i) individuals with a need for ongoing support and services;

(ii) individuals with a need for high-cost accommodations;

(iii) individuals who earn a subminimum wage; and

(iv) individuals who work and receive partial cash benefits.

The Commissioner shall consult with the Ticket to Work and Work Incentives

Advisory Panel established under section 101(f) of the Ticket to Work and Work

Incentives Improvement Act of 1999 during the development and evaluation of the

study. The Commissioner shall implement the necessary adjusted payment rates

prior to full implementation of the Ticket to Work and Self-Sufficiency Program.

The Panel has carefully considered issues related to equitable participation in the Ticket

Program. This has included consultation with the Social Security Administration, other

Federal agencies, and various experts; opportunities for public comment; and Panel

deliberations. The Panel’s Design and Evaluation Committee convened an Experts
8

Roundtable of nationally recognized experts, Social Security Administration officials,

and Panel members on July 27, 2001. The Roundtable addressed the many complex

issues involving the employment of individuals with significant disabilities. The impact

of the overall Ticket evaluation on individuals in the four groups and the design of the

AOI Study were carefully considered. Subsequently, a well-known expert in the field, Dr.

David Mank, assisted the committee and Panel in drafting this report.

This Advice Report represents the Panel’s effort to bring expertise to bear in advising the

Social Security Administration on these complex issues and the design of the AOI Study.

It became evident during this process that ongoing consideration and advice on this

important topic will be necessary. The Panel looks forward to continuing to address these

issues and provide advice as the Ticket is implemented and research goes forward.
9

SUMMARY OF A SOCIAL SECURITY ADMINISTRATION REPORT:

Working Paper 003—Design of the Study of the Adequacy of Incentives for the Ticket to

Work Program

An overall design has been crafted by the Social Security Administration for the

Adequacy of Incentives Study that is based in part on Federal databases and includes a

process evaluation and an impact evaluation (Huynh and O’Leary, 2001). Such research

is required by the Ticket to Work and Self-Sufficiency Program. Specifically, the Social

Security Administration is to assess the Adequacy of Incentives structure in serving

members of groups considered unlikely to participate and to develop and test alternative

incentives as necessary. The Act targets four groups that are considered likely to be

“involuntary nonparticipants.” These four groups are individuals with a need for ongoing

support and services, individuals who need high-cost accommodations, individuals who

earn a subminimum wage, and individuals who work and receive partial cash benefits.

It is important that the Adequacy of Incentives design include specific attention to these

four groups from the beginning of the Ticket to Work program evaluation process. As

Huynh and O’Leary say, “ ... it enables us to ask: What data do we want to have when

we get to the point of implementing the evaluation, rather than what do we have at that

point” (Huynh and O’Leary, 2001, p. 2).

Working Paper 003 asks two important questions relative to this Advice Report: Are

incentives adequate for the target groups? And, if not, how can incentives be modified to

better serve these groups? The paper goes on to describe the proposed process for the

study of the Adequacy of Incentives.

Data for addressing the Adequacy of Incentives will be drawn from administrative data in

the Social Security Administration and from Internal Revenue Service tax files and

possibly Center for Medicaid and Medicare Services (formerly known as the Health Care

Finance Administration) data and Rehabilitation Services Administration data. The

intention here is to “compare the characteristics of those served in the Ticket to Work

program with the characteristics of the SSDI [Social Security Disability Insurance] and

SSI [Supplemental Security Income] disabled populations.” (p. 4) The authors also note

that while the administrative data will be useful, they are not likely to include important

information about ongoing publicly funded supports, accommodations, and hourly wages

earned. Because of the lack of some needed data in the administrative data sets, the paper

also proposed a survey that would ask beneficiaries about work, wages, access to the

Ticket, access to vocational rehabilitation services, and other questions. A phone survey

is planned to attempt to collect these data, supplemented by home interviews when

information cannot be collected over the phone. Issues with this approach are also

discussed, including sampling issues and survey subjectivity. For the purpose of

determining how to modify the incentives of the Ticket to Work, the authors propose a

theoretical modeling approach because, they argue, “testing alternative incentives is ...

not currently feasible given the limited information ... on alternative incentives to test.”

(p. 8) The authors expect the Social Security Administration administrative data to be
10

useful, but they note that its usefulness may be somewhat limited. In addition, the survey

noted earlier will be designed to help address the issue of determining how to modify

Ticket to Work and Self-Sufficiency Program incentives. Issues of subjectivity and

defining eligibility are also noted and discussed related to possible modification of these

incentives.

Working Paper 003 also indicates that additional input is needed on three specific

questions: What are objective means of identifying the groups? What are the outcomes of

interest? What array of services should be considered to maximize outcomes to members

of these four groups?
11

SUMMARY OF EXPERTS ROUNDTABLE, JULY 27, 2001

An Experts Roundtable was convened by the Design and Evaluation Committee of the

Work Incentives Advisory Panel on July 27, 2001, in Washington, DC. The Panel

assembled experts in several fields, with the main focus being efforts to employ those

with significant disabilities. Lists of experts in attendance and of those providing public

comment are provided in the appendixes to this Report. The meeting provided the

opportunity for review and discussion of the design of the study on the adequacy of

incentives for individuals in the four groups noted. After presentation of the study design

by the Office of Policy and its consultants, the experts offered comments in response to a

series of questions posed by both the Social Security Administration and the Work

Incentives Advisory Panel. Members of the public also had an opportunity to share their

thoughts and impressions on this topic through testimony.

While the overall design may represent important methods to begin to assess the use of

the Ticket to Work and Self-Sufficiency Program and the Adequacy of Incentives, the

following points summarize the concerns and input of the Experts Roundtable, public

testimony, and testimony received in writing after July 27, 2001:

EXPERT COMMENTS

 The design should be formative in nature, immediately seeking ways to modify and

test incentives rather than waiting for a period of time to conduct a summative

evaluation to determine whether the design of incentives was adequate. The design

should emphasize the question, What will it take for the incentives to be adequate for

Employment Networks to serve members of the four groups?

 There is a need for qualitative studies (in-person interviews and focus groups) in

addition to quantitative studies.

 There is a need for specific attention to collecting data regarding the issue of

incentives to, and perceptions of, providers of services.

 There is concern that the phone survey protocol, even supplemented by home

interviews, will be inadequate because of communication barriers for some members

of the four groups and lack of understanding of the nature of the Ticket to Work and

Self-Sufficiency Program.

 There is concern that the survey should be sensitive to minority and cultural issues as

well as to geographical differences.

 There is a need to use existing databases from supported employment data, State data

from developmental disability agencies, and State data from mental health agencies.

 There is ample evidence, based on the supported employment initiative,

Rehabilitation Services Administration choice projects, and other projects, that

individuals in these groups can work productively, although often part time.

 Demonstration projects with members of the four groups are essential to show how

the incentives could be modified to work with members of the four groups.
12

 Evaluation of the Adequacy of Incentives should make use of the experience in States

using results-based funding or milestone payments in the vocational rehabilitation

system.

 The evaluation design of the Adequacy of Incentives should be considered in the

context of the other benefits and services received by members of the four groups.

PUBLIC COMMENTS

 The AOI Study is the only hope that persons with high support needs have for

utilization of their tickets.

 SSA must look beyond its own data banks to employment researchers who have spent

years studying supported employment and the successes and the impact work has had

on the lives of persons with high support needs.

 Everyone needs to have the opportunity to work, regardless of disability. No one can

be left behind.

 The Ticket to Work Program is an extremely exciting opportunity for people with

disabilities to gain employment while being in control of funding. It actualizes their

role as customers, changing the perceptions and focus of the provider community and

the general public. People with more severe disabilities may not be able to benefit

from the Ticket to Work Program as it is currently designed. SSA and the Panel must

analyze the types of supports needed by these individuals, the associated expenses,

and the impact on the person’s life style.

 Because of the multitude of issues that surround individuals with significant

disabilities, it is incumbent upon this process to take a broad view of both cost and

benefit. Taking a narrow, econometric approach to the study may not reveal an

effective payment strategy for these individuals—reducing potential outcomes that

might be experienced otherwise.

 A broad-based study will decrease reliance on Social Security and other benefits.

 Individuals with disabilities can become employed and be successful with proper

assistance. Without these intensive, individualized supports, though, these individuals

are destined to remain on Social Security benefits and other public assistance

programs.

 In order to prevent failure of this program, input about the actual cost of supports

needs to come from agencies that are proficient in serving individuals with

disabilities, and not determined in a vacuum or through sterile research

methodologies.

 Expertise of people with disabilities, their families, and agency practitioners must be

used in the research design.

 SSA and the Panel must do nothing less than draw on the best information in the field

to access the answers SSA needs to design payment strategies that will work for all

persons who have a disability—especially those who have the most severe

disabilities. This cannot be an academic debate about competing research

methodologies; it should be about determining where the answers lie and designing a

study that will lead to those answers.
13

 The AOI Study potentially opens another avenue that is desperately needed to open

employment for people with significant disabilities.

 In Ohio, only 8 percent of people with severe disabilities are employed. In the

proposed Ticket regulations, there is little incentive for providers to accept tickets

from individuals who need intense and/or expensive supports. The AOI Study is

needed to determine alternative payment strategies so these people can use their

tickets and become employed with the needed supports.

15

SUMMARY OF ISSUES

The following represents a synthesis of the points derived from the Experts Roundtable,

public testimony, literature review, and Design and Evaluation Committee discussions

and provides the conceptual underpinning for the Panel and Expert Roundtable

recommendations made in this Report.

1) Individuals in these four groups are less likely to participate in the Ticket Program

because they are less likely to stop receiving Social Security benefits altogether.

2) Individuals in these four groups will be extremely difficult to identify in Federal

databases because data on relevant dimensions defining these four groups are not

directly collected in Federal databases. Some Federal data may allow inferences for

some of the members of some of these groups (e.g., there may be relevant data from

the Department of Labor regarding individuals earning subminimum wages; there

may be relevant data on subsets of the four groups in Medicaid data and in data from

the vocational rehabilitation system).

3) It is likely that individuals in these four groups will require a greater degree of effort

and long-term support for successful employment and may not be able to work

enough hours or earn enough money to eliminate Social Security benefits (e.g., Mank,

1994; Wehman, 2001).

4) Phone survey methodologies will be of limited utility for members of these four

groups because of intellectual or other developmental disability, or communication

difficulties based on disability, in addition to the issues of English as a second

language and minority populations (including Native Americans), which will also

affect the broader evaluation design (HSRI, 1999).

5) There is a need for a “formative” evaluation approach and other research and

evaluation designs for these four groups (which asks the question “Under what

circumstances can and do members of these four groups participate?”), in addition to

a summative evaluation about the extent to which members of these groups

participate (Campbell and Stanley, 1963; Weiss, 1972).

6) There will be major ongoing costs to Social Security as a result of widespread

nonparticipation of people in these groups. Individuals in these groups are likely to

receive a lifetime of benefits if they do not participate in employment (e.g., Braddock,

2001).

7) There is widespread evidence that individuals who are part of these four groups can

work successfully (although often part time) with individualized support structures

(supported employment research and evaluation studies, Rehabilitation Services

Administration “Choice” projects, etc.). These data are extremely relevant in

considering the employment rates and potential of people in the four groups and in

determining what Employment Networks need to assist individuals with significant
16

disabilities to be employed (e.g., Mank, Cioffi, and Yovanoff, 1999; Mank, O’Neill,

and Jensen, 1998; Revell, Inge, Mank, and Wehman, 1999; Sowers and Powers,

1991; Wehman, 2001).

8) The research proposed by the Social Security Administration on the Adequacy of

Incentives should be supplemented in part because Social Security administrative data

do not include all the information needed to identify the four groups and in part

because of communication issues associated with efforts to survey these groups.

Promising sources of information include existing State and local databases as well as

efforts to more directly collect new data relative to the individuals in these groups in

addition to the adequacy of the incentives to the Employment Networks and providers

of employment services (Campbell and Stanley, 1963; Weiss, 1972).

9) The methodologies of the evaluation design must be considered in the complex

context of a range of factors that affect the extent to which people with disabilities

work and the extent to which the incentives in the Ticket Program are adequate.

Additional factors that have an impact on the employment of people with disabilities

include benefits management, provider availability and capacity in local areas, and

vocational rehabilitation initiatives (Novak, Mank, Revell, and O’Brien, 1999;

Wehman, 2001). The nature and funding, including milestone payments, of

vocational rehabilitation services, long-term day services, and employment support

services by State developmental disabilities agencies or State mental health agencies

will also have an impact on employment. The nature of States’ Medicaid Waiver

programs and Medicaid Buy-In will influence employment. In addition, the extent to

which long-term employment supports and services are funded or not funded in a

given State will have an impact (e.g., Braddock, 2001; Butterworth, Gilmore,

Kiernan, and Schalock, 1999).

10) “Purposeful” sampling will be useful for qualitative research and focus groups (Guba,

1978; McMillan and Schumacher, 1984). This strategy should include individuals in

the four groups as well as Employment Networks and providers. The overall design

proposed provides the broad structure for the large quantitative study of the Adequacy

of Incentives. The four groups noted present specific challenges for the Adequacy of

Incentives and for the study design. State samples provide a method for more closely

identifying members of the four groups and for collecting more detailed information.

In addition, they provide a context for creating studies specific to Employment

Networks and providers in the specific context of the samples of individuals in the

four groups.

11) There is a need to ensure that all methods used are designed with attention to

geography, demographics, and economic conditions, as well as the representation and

participation of Native Americans and other minority groups.

12) While the limits of the available administrative data have been noted, it must also be

noted that available State databases and databases from national samples have limits

related to the completeness and representativeness of the data. Thus, a combination of
17

research, evaluation, and database methodologies is needed to create the most

complete (albeit inherently imperfect) picture possible of the participation of

members of these four groups and the modification of incentives.

13) While each of the research and evaluation strategies discussed in this Report has

limits, it must also be recognized that the combined strategies discussed represent the

best possible approach, given the limits, and should be implemented, because the cost

of “doing nothing” is failing to provide any useful information about the participation

of members of these groups and the modification of incentives.

14) These issues will not be fully resolved in the short run because of the weaknesses in

the administrative databases, the need to create alternative evaluation designs for the

evaluation of the Adequacy of Incentives, and the complex context of services and

funding streams in which services are provided to members of these four groups. This

complexity underscores the need for ongoing evaluation and reporting requirements

in addition to the Adequacy of Incentives Study, report, and implementation

requirements in the statute.

Note that the appendices include references as well as a more complete bibliography

regarding employment initiatives and research related to likely members of the four

groups.

19

QUESTIONS AND RECOMMENDATIONS

The analysis and input are considered here in the context of the three primary questions

posed by the Social Security Administration (questions 1–3 below) and six additional

questions from the Panel (questions 4–9):

1) What are objective means of identifying, through the Social Security Administration

administrative data, the target populations as defined in the Ticket to Work and Work

Incentives Improvement Act?

2) What are the outcomes of interest?

3) What array of services should be considered to maximize outcomes to members of

these four groups?

4) What experts and information resources should be considered regarding successful

efforts to employ people with severe disabilities in community employment?

5) What other methods should be used to identify members of the four groups?

6) How can the Social Security Administration demonstrate how individualized

milestone payments could be used?

7) How can the Social Security Administration ensure that various and diverse

populations are represented in the study?

8) How can consumers, family members, and/or their representatives be included in the

study design and implementation?

9) How should Employment Networks be consulted in the Adequacy of Incentives

Study?

In addition, the Ticket to Work and Work Incentives Advisory Panel believes that the

ongoing process for the design of the Adequacy of Incentives Study is critical to the

success of the Ticket to Work and Self-Sufficiency Program for members of these four

groups. Thus, recommendations about the ongoing process for study and reporting are

defined.

21

PANEL RECOMMENDATIONS

Design Process of the Study

There are important issues concerning the plan for the evaluation of the Adequacy of

Incentives specifically related to the four groups in question. Because it is agreed that the

available administrative data and the alternative strategies have limits in providing

information about members of the four groups, it will be necessary to define a process for

design and implementation of the study that is sensitive to the need for current and

ongoing information about participation and about the adequacy of incentives to the

Employment Networks and the providers of services. Because the Act mandates attention

to the participation of these four groups and to the modification of incentives in the

interests of these four groups, the following overarching study design principle

recommendation is made:

Panel Recommendation 1: An ongoing structure must be established for review of

emerging information, analysis of trends, and creation of recommendations regarding

modification of incentives and using multiple evaluation methods.

The following recommendations are intended to support the ongoing evaluation design.

Panel Recommendation 2: The Social Security Administration should convene an

Adequacy of Incentives Advisory Team, similar to the team of consultants approach

being used under the $1 for $2 research, for ongoing technical support for the Adequacy

of Incentives Study, specifically related to the four targeted groups. This group should

consist of diverse individuals who have research background, knowledge, and/or

expertise in the successful employment of individuals in the four groups. This group

could include, but not be limited to, the experts identified by the Panel for the Adequacy

of Incentives Expert Roundtable. These experts would be charged to further develop

recommendations from the Experts Roundtable for serious consideration by the Social

Security Administration. At least two members of the Adequacy of Incentives Advisory

Team should become members of the Technical Evaluation Support Group, the technical

group advising Social Security Administration on overall Ticket evaluation, to ensure

coordination of the two research efforts.

Panel Recommendation 3: Given the impact of this and other studies the Office of

Policy conducts on the lives of people with disabilities and their families, the Office of

Policy should develop a policy and practice for incorporating the input and experiences

of consumers, their families, and/or their representatives as it develops new research

designs, including the Adequacy of Incentives Study.

Panel Recommendation 4: As the Ticket to Work and Self-Sufficiency Program is

implemented, the Social Security Administration should provide to the Panel, Congress,

and the President ongoing interim reporting on the participation of members of the four

targeted groups in the Ticket to Work and Self-Sufficiency Program.
22

Panel Recommendation 5: The Panel recommends that Congress approve a technical

amendment to require the Social Security Administration to provide to the Panel,

Congress, and the President ongoing interim reporting on the participation of members of

the four targeted groups in the Ticket to Work and Self-Sufficiency Program.

Panel Recommendation 6: Ongoing data collected and outcomes determined for the

Adequacy of Incentives Study should be included in the Social Security Administration’s

overall Ticket evaluation research.

Panel Recommendation 7: Due to delays in issuing the tickets, the deadline for the

Adequacy of Incentives Report to Congress should be extended by the amount of time the

ticket rollout has been delayed (13 months). The Panel recommends that Congress

approve an amendment to adjust the timeline for the Adequacy of Incentives Report so

that it is due 49 months, rather than 36 months, after the enactment date. Implementation

of the necessary adjusted payment rates should occur as soon as practical following the

release of the Adequacy of Incentives Report but no later than 1 year after its release.

Further, the Panel requests preliminary progress reports on the Adequacy of Incentives

Study at 36 and 42 months after the date of enactment of the law.

Identifying the Four Groups

The Social Security Administration should not rely on diagnostic data alone in attempting

to identify the four groups of interest to Congress. Such efforts will yield less than perfect

outcomes. Using standard diagnostic categories that would include but not be limited to

developmental disability, mental retardation, psychiatric disability, and multiple

disabilities (such as deaf and blind or blind and cerebral palsy) as a major identification

tool for the four groups fails to recognize the wide heterogeneity within these categories.

The categories that would “most likely” include a significant portion of the population in

the four target groups would also include a number of individuals who do not fall within

the target groups. Likewise, individuals in the target groups may be found in other

diagnostic categories that would not be expected to include them. Moreover,

supplementing existing data through surveys is problematic because a high proportion of

the population of interest will have disabilities that impede survey participation.

Panel Recommendation 8: While there is a need to take advantage of what is known

about the diagnostic categories with high proportions of those most likely to fall into one

of the four groups relative to work capability, the Social Security Administration, in

addition to diagnostic data, should use other information, such as survey information,

information on earnings, onset date, and medical improvement assessments. SSA should

also go to outside data sources, such as those identified in Expert Roundtable

Recommendation 1, to identify members of the four target groups.

Survey Methodology

Panel Recommendation 9: The Social Security Administration has proposed a survey to

supplement its administrative data, improving its ability to target the four groups. If the
23

survey uses a telephone protocol, the protocol should include two other elements. First,

there should be face-to-face home interviews if the phone survey fails for a given

respondent, except for cases of incorrect phone numbers, empty residences, and so on.

Second, if the potential respondent’s disability prevents effective participation in the

survey, the interviewer should use as a proxy another individual who knows the

respondent well and, preferably, who has communicated on the respondent’s behalf in the

past, such as a parent, spouse, sibling, or supervisor in a group home. Members of the

Adequacy of Incentives Advisory Team (see Panel Recommendation 2) should review

the interview protocol used by the contractor conducting the survey.

Supplementary Research

Panel Recommendation 10: Recognizing the need for research to supplement what can

be studied using Social Security administrative data, the SSA has proposed a program of

supplementary research. It has solicited the suggestions of the Panel for possible research

topics to supplement the research using SSA data. The Panel recommends the following:

 Review existing research on the four groups or related groups, including relevant

behavioral demonstrations and field experiments.

 Conduct or commission new research on these groups using State data or other

Federal data.

 Conduct or commission new research related to Employment Networks and other

providers.

 Review and consider best practices of successful programs relating to employment of

those with significant disabilities.

 Review best practice systems of results-based funding and/or individual milestones

payments.

 Conduct or commission a review of supported employment literature, research, and

best practices.

 Conduct or commission ongoing demonstration projects, modeling, and evaluation of

the effectiveness of alternative incentives identified.

Employment Networks

Panel Recommendation 11: SSA should identify the data elements that Employment

Networks (ENs) should collect on whom they serve and do not serve from the four

functional groups identified in the statute. The data should include descriptions of why a

ticket is accepted or is not accepted on the basis of the EN’s perception of cost, degree of

difficulty, or lack of expertise on the part of the EN to provide the necessary services.

SSA should use a sampling of ENs rather than requiring all ENs to collect the data. SSA

should pay the ENs to offset the administrative burden of management information

system design.

Panel Recommendation 12: SSA should conduct or commission qualitative research,

including interviews and focus groups as appropriate, to assess the adequacy of current

incentives to large and small providers, including community rehabilitation programs.
24

Information and recommendations should be collected regarding the amount and timing

of payments and other factors that affect potential participation in the Ticket Program and

integrated community employment at minimum wage or higher levels for individuals in

the four groups.

Various and Diverse Populations

Panel Recommendation 13: SSA should ensure that the data collected include attention

to national demographics such as rural and urban settings, minority representation, and

socioeconomic conditions across the country.

RECOMMENDATIONS FROM THE EXPERTS ROUNDTABLE

Following are the recommendations that emerged from the Experts Roundtable in

response to the nine questions posed by the Social Security Administration and the Panel.

The Panel strongly urges the Social Security Administration to seriously consider and

further develop these recommendations in designing research related to the Adequacy of

Incentives and its context within the broader evaluation of the Ticket to Work and SelfSufficiency Program.

IDENTIFYING THE FOUR GROUPS

Summary of Comment

Public comment regarding identifying members of the four subgroups noted the

importance of including people with Down syndrome, people with significant disabilities,

and people who are deaf and blind. Information was shared about databases on those who

are deaf and blind. In addition, comment included the importance of ensuring that the

Ticket is available in the transition years from school to adult life. Public comment also

emphasized the importance of ensuring that the Ticket incentives are modified in the

interest of people whose Social Security benefits may not be eliminated.

Discussion

Information from the original design of the study, the Experts Roundtable, and the public

testimony show that there will be difficulty in identifying individuals in these four groups

in the Federal administrative data available. As a result, it will be necessary to add

methods to locate these individuals in order to collect data about use of the Ticket. It is

clear that many of the individuals in these four groups are now eligible for, or receiving,

services via State developmental disability programs or mental health agencies. Those

receiving services and likely to be eligible for the Ticket are likely to be found in day

activity programs, day treatment programs, sheltered workshops, supported employment,

and other appropriate databases. Because there are important questions about the

availability of Federal data, State sources of information will be needed in order to find

and understand the participation or nonparticipation of people from the four groups.
25

Sources that can provide information to locate members of the four groups will include

State mental retardation and developmental disability agencies; State mental health

agencies; providers of sheltered work, day activity, and day treatment programs;

providers of supported employment services; and some State vocational rehabilitation

agencies. Existing supported employment databases in some States could be very useful.

Other databases may be of use, such as available data about people with deafness and

blindness and data related to community rehabilitation facilities that serve people in these

four groups. Through such sources, it will be possible to identify relatively large samples

of people who fit the descriptions of each of the four groups.

Identifying samples will depend in part on the availability of State-level data regarding

people in the four groups. Some States will have databases that make such analysis

possible; other States may have a limited ability to provide such data. These data will be

useful for several kinds of studies. First, identifying a relatively large sample in each

State will allow specific tracking to note the participation or nonparticipation of people in

the four groups. In addition, identification of people in the four groups will allow data to

be collected from the individuals and their families. In light of the communication issues,

it can be expected that it will be necessary, at least in part, to conduct face-to-face

interviews and to talk to some family members or significant others regarding

participation. The information collected should include the needed disability and

demographic information as well as information about knowledge of the Ticket,

availability of the Ticket, and access to Employment Networks and providers of service.

In addition, it may be possible to gather information about interest in and the feasibility

of full-time or near full-time work in order to possibly work enough to eliminate Social

Security financial benefits. While such samples will not be random samples from a

national perspective, they will provide additional information needed for the Adequacy of

Incentives Study.

It must be noted that such State and other databases have serious limits in their

completeness and the extent to which they represent in totality the members of the four

groups. The four groups, as defined in the Act, have never before been defined in quite

this way. As a result, no existing databases can be expected to be complete or

representative of the four groups. Because of this, a combination of national

administrative data and State and other data should be used to construct the best possible

understanding of the groups and the best possible understanding of the incentives to

Employment Networks and providers in providing access to the Ticket to members of the

four groups. This approach is supported by the Experts Roundtable and by the public

input provided to the Panel.

Roundtable Recommendation 1: In addition to the administrative data sets SSA has

proposed using, use appropriate Federal databases (e.g., Office of Special Education and

Rehabilitative Services, Rehabilitation Services Administration, and Department of

Labor); State databases (e.g., State developmental disability agencies and State mental

health agencies); data from providers of day treatment, day activity programs, sheltered

workshops, and supported employment programs (e.g., data from the Employment
26

Intervention Demonstration Program funded by the Center for Mental Health Services);

and existing research studies in order to identify and gather information from and about

members of the four groups. In addition, the Helen Keller National Center has data on

7,000 persons who are deaf and blind, some of whom are seeking employment.

Roundtable Recommendation 2: Oversample potential members of these four groups

who can be located in the administrative data from Social Security, Office of Special

Education and Rehabilitative Services, Rehabilitation Services Administration, Medicaid,

the Department of Labor, and/or other Federal sources.

OUTCOMES OF INTEREST

Summary of Comment

Public comment related to outcomes supported the emphasis on basic employment

outcomes, including wages and integration in the workforce. However, commenters also

indicated that it is important to emphasize that the Ticket should increase choices and

control by people with disabilities and their families. Commenters also expressed concern

that a phone survey will not be adequate to gather outcome and participation data from

the people in the four targeted groups. Commenters also agreed that qualitative methods

for collecting data would be needed in addition to quantitative methods. In addition, one

commenter emphasized the use of existing databases that include people with significant

disabilities. Experts and commenters agreed that an essential outcome is the opportunity

for individuals in the four groups to use a ticket to obtain successful employment. There

was agreement that successful employment should be defined as real work in integrated

settings in the community at minimum or higher wage, based on individual choice and

control.

Discussion

There are two levels of outcomes of interest: the individual outcomes of interest and the

broad program outcomes of interest. The individual outcomes of interest include type of

job, wages earned, hours worked, job retention, and the process (and timing) of gaining

access to the Ticket through the Employment Networks and providers of services. When

members of the four groups do not use or have access to the Ticket, the Social Security

Administration will need to know the reasons why individuals are not participating: Is it

voluntary or involuntary nonparticipation? It will be important to know if the Ticket

Program increases choices of employment services for individuals. It will also be

important to know if partial reduction of benefits is possible and helpful to individuals.

At a broad program level, it will be important to understand the patterns of participation,

outcomes, costs, and incentives that may be adequate to produce desired outcomes.

Roundtable Recommendation 3: Track basic employment outcomes (jobs, wages,

hours, choice, etc.) for members of the four groups once identified.
27

Roundtable Recommendation 4: Gather information about access to and use of the

Ticket, with particular attention to whether nonparticipation is voluntary or involuntary.

Roundtable Recommendation 5: Conduct or commission qualitative research, including

interviews and focus groups, to gather more extensive information about participation,

economic and other benefits, and ways to attain improved incentives.

Roundtable Recommendation 6: Analyze the data collected related to patterns of

outcomes, cost/benefit, participation, and modifications of incentives for members of

these four groups compared with others making use of the ticket.

MAXIMIZING OUTCOMES

Summary of Comment

Public comment on the array of services emphasized the need for flexibility in how the

Ticket Program is used to ensure that members of the four groups have access to the

Ticket. Commenters noted that there are a number of best practice programs around the

country that can provide information about successful employment of people with

significant disabilities. Commenters also noted that the Ticket Program should

supplement and not supplant other services received by members of the four groups. In

addition, comment included attention to the importance of ensuring that the Ticket

Program incentives were adequate for the providers of services. Finally, comments

focused on the need to understand results-based funding and milestone payment

programs for employment outcomes.

Discussion

A third important question concerns the array of services that should be provided to

members of these four groups. In considering essential services and supports, it is

important to recognize that a large number of people who typically would fall into one of

the four groups have become successfully employed through an array of services and

supports. It is critical that the study include an evaluation of the array of services and

supports that led to successful employment. The experiences of the supported

employment initiative, the Rehabilitation Services Administration Choice projects, and

the milestone payment systems in vocational rehabilitation will be especially relevant in

this regard. In addition, the combination of services (vocational rehabilitation,

developmental disabilities programs, mental health programs, Medicaid, etc.) received by

some members of these four groups will be important to understand. For people in these

four groups, there are other macro and micro factors that relate to the opportunity to

work. Such factors include benefits planning, assistance and outreach projects,

availability (and payment methodology, for example, fee-for-service vs. results-based

funding) of vocational rehabilitation funds, availability and structure of long-term

employment support funds (via mental retardation/developmental disability and mental

health agencies), use of Medicaid funds, Plan for Achieving Self-Support and

Impairment-Related Work Expense use, housing allowances and funding, and the
28

perception of disincentives to employment. Factors such as these already complicate

work decisions for people in these four groups and have an impact on their

unemployment or underemployment. In addition, issues of continued health care

coverage and States’ use of the Medicaid Buy-In process will have an impact on

employment. Analysis of the Adequacy of Incentives will depend in part on analysis of

this complex context and of possible interaction effects. There are also related questions

about combining funding sources in the interest of employing these individuals. Another

area for analysis is the extent to which those eligible for tickets are sufficiently motivated

to work.

Roundtable Recommendation 7: The Social Security Administration should ensure that

studies of these four groups include attention to the issue of the “mix” of funding,

government programs, and other incentives and disincentives that affect the employment

of people with disabilities. Studies should include analysis of how various funding

streams can be used together to enhance incentives to Employment Networks and

providers and result in employment for people in the four groups.

SUCCESSFUL EFFORTS TO EMPLOY PEOPLE WITH SIGNIFICANT

DISABILITIES IN COMMUNITY EMPLOYMENT AT MINIMUM WAGE

LEVELS OR HIGHER

Summary of Comment

The public comment about experts and information resources to consider focused on

initiatives in the past 10 to 15 years that have emphasized employment for people with

severe disabilities. These initiatives include supported employment, the Rehabilitation

Services Administration Choice projects, and other demonstration projects, including

those for persons with mental illness as well as persons who are deaf and blind.

Discussion

Public comment and the discussion of the Experts Roundtable emphasized that members

of the four groups have benefited from employment initiatives in the recent past.

Evidence of these successful strategies is described in some detail in the supported

employment literature, in reports regarding projects to create greater choice and control

in the vocational rehabilitation process, and in a number of demonstration projects that

have been created in the past 15 years. Evidence of such projects is summarized in a

number of books and published research reports noted in the bibliography. While these

initiatives have not solved the widespread unemployment or underemployment of people

in the four groups, there is ample evidence that these people can work if they are

provided with creative and individualized supports. Study of these successes can help

inform the process of discovering how to modify incentives related to the Ticket.

Roundtable Recommendation 8: The Social Security Administration should study the

successes of supported employment programs, Rehabilitation Services Administration
29

Choice projects, and other demonstration projects of the past decade for members of

these four groups.

OTHER METHODS TO IDENTIFY MEMBERS OF THE FOUR GROUPS

Summary of Comment

Public comment supported the concern about administrative data and the planned phone

surveys as a way to gather information from members of the target groups. Comment

included an emphasis on the need to consult other databases that may be from States,

regions, or national samples of persons likely to be members of the four groups.

Discussion

There is concern that administrative data will not be adequate to locate members of the

four groups. As a result, commenters and the Experts Roundtable emphasized the need to

use other databases to find people in the four groups in order to gather information about

their participation and about ways to modify the incentives of the Ticket. Owing to the

heterogeneity of the population with disabilities, large samples and information

specifically needed to analyze key subgroups would be especially useful.

Roundtable Recommendation 9: The Social Security Administration should consult

other databases to identify members of the four groups. Such databases should include

other existing national samples as well as State databases for persons with developmental

disabilities, persons with mental illness, persons with deafness and blindness, persons

with brain injuries, and persons with other disabilities.

INDIVIDUALIZED MILESTONE PAYMENTS

Summary of Comment

Public comment and the Experts Roundtable cited the recent experiences in

Massachusetts, Oklahoma, and other States with creating milestone payments in the

interest of employment for people with disabilities and including members of the four

groups.

Discussion

Results-based funding and milestone payment systems are emerging in at least a dozen

States across the country (e.g., Massachusetts and Oklahoma) in the interest of increasing

employment for people with significant disabilities, some of whom are members of the

four groups. These experiences can provide information about effective means of funding

providers of services to create employment outcomes. They can also provide a

framework for creating demonstration projects that make use of emerging payment

mechanisms in combination with best practices for employment programs.
30

Roundtable Recommendation 10: The Social Security Administration should study the

current and recent experiences of States that have begun using results-based funding or

some form of payment for individualized milestones toward employment.

Roundtable Recommendation 11: The Social Security Administration should conduct

or commission cost/benefit analyses and create demonstration projects on modifying the

incentives to Employment Networks and providers of services, showing how members of

the four groups can benefit from the Ticket and from employment that partially reduces

benefits.

VARIOUS AND DIVERSE POPULATIONS

Summary of Comment

Input related to diversity issues included the need for attention to the unique differences

of minorities related to gaining access to employment. Concern was also expressed about

regional differences nationwide and about Native American people.

Discussion

Because of the diversity in the country, it will be important to ensure that the study

design is sensitive to a range of diversity issues. National demographics should be

considered in the design and implementation of the study, with particular attention to

rural and urban settings, minority representation, and emerging economic issues.

Roundtable Recommendation 12: The Social Security Administration should ensure

that the data collected include attention to national demographics such as rural and urban

settings, minority representation, and socioeconomic conditions across the country.

INCLUDING CONSUMERS, FAMILY MEMBERS, AND/OR THEIR

REPRESENTATIVES IN THE STUDY DESIGN AND IMPLEMENTATION

Summary of Comment

Input related to participation in the study design process noted the need for people with

disabilities and families to be included. Others commented that research methodology

used should be accessible and not limit participation.

Discussion

Because participation or nonparticipation in the Ticket Program will profoundly affect

people with disabilities and their families, as well as the overall success of the program, it

is important that they clearly be a part of the process for design, implementation, and

ongoing tracking of the Adequacy of Incentives evaluation.

Roundtable Recommendation 13: Given the impact of this and other studies the Office

of Policy conducts on the lives of people with disabilities and their families, the Office of
31

Policy should develop a policy and practice for incorporating the input and experiences

of consumers, their families, and/or their representatives as it develops new research

designs, including the Adequacy of Incentives Study.

INCLUDING EMPLOYMENT NETWORKS IN THE STUDY DESIGN AND

IMPLEMENTATION

Summary of Comment

Input regarding the role of Employment Networks and providers emphasized the need for

the incentives to be clearly adequate from a financial standpoint for both large and small

providers. Commenters noted that the Ticket payment schedule might be too far in the

future for providers to believe the incentive will be adequate. There was also some

concern about the possibility that Employment Networks would experience an excessive

reporting burden if they were not told in advance—before being contracted—about the

extent of information needed. It was further suggested that Social Security

Administration consider random sampling of Employment Networks to obtain this

information and consider providing a financial incentive for participation in gathering

this information.

Discussion

As tickets are released and Employment Networks begin service to beneficiaries, it is

critical to ensure that a stream of information and data is steadily collected and analyzed

relative to how Employment Networks serve beneficiaries in these four categories. To do

this effectively, the Social Security Administration should identify up front what specific

data need to be collected, to enable ENs to incorporate the data collection into their

database design. To further facilitate accurate and timely collection, SSA should consider

asking for the data from a representative sample of ENs rather than from all ENs and

should pay a reasonable fee to participating ENs to cover the cost of the administrative

burden. In addition to collecting data on those served by an EN, data should be collected

on those whose tickets the EN did not accept. It is expected that some ENs will reject

tickets on the basis of the perception, real or imagined, that people in the four targeted

groups will ultimately cost more to serve than is available in the outcome/milestone

payment program. Surveys and focus groups could be used to garner recommendations

from ENs as to what they believe sufficient incentives to be. Incentives could be

measured in terms of time, money, and difficulty (e.g., availability of certain services in

rural areas, coordination with the One-Stop Career Centers for job skills training funds,

cooperation with the State vocational rehabilitation agency). Social Security should

compare data among ENs to determine whether or not there is consistency among and

between ENs in terms of whether they accept tickets, costs they claim to incur for various

services, and perceived difficulty in serving consumers in the four targeted groups.

Roundtable Recommendation 14: The Social Security Administration should gather

data related to the costs to Employment Networks and providers of services and the
32

timeframes for providing access to the Ticket and employment for members of these four

groups.

Roundtable Recommendation 15: The Social Security Administration should study the

relationship between perceived disincentives and actual incentives for Employment

Networks and providers of services, including incentives for small providers of services.
33

SUMMARY

This Advice Report provides a framework for recommendations related to the design of

the Adequacy of Incentives Study. This guidance is intended to enhance and supplement,

not supplant, the overall design proposed by the Social Security Administration in order

to understand the impact of the Ticket Program for members of the four noted groups and

to determine how incentives can be modified to promote access to employment services

and supports for all beneficiaries of Social Security Administration disability benefit

programs.

35

APPENDIX 1:

TICKET TO WORK AND WORK INCENTIVES ADVISORY PANEL

Sarah Wiggins Mitchell, R.N., M.S.W., J.D., Chair

President and Executive Director

New Jersey Protection and Advocacy, Inc.

Richard V. Burkhauser, Ph.D.

Professor of Policy Analysis

Chair, Department of Policy Analysis and Management

Cornell University

Ithaca, NY

Kristen E. Flaten, M. Div.

Employment Consultant

Lifetrack Resources

St. Paul, MN

Thomas P. Golden, M.S., C.R.C.

Extension Faculty

Program on Employment and Disability

School of Industrial and Labor Relations

Cornell University

Ithaca, NY

Frances Gracechild

Executive Director

Resources for Independent Living, Inc.

Sacramento, CA

Christine M. Griffin, J.D.

Executive Director

Disability Law Center

Boston, MA

Larry D. Henderson

Executive Director

Independent Resources, Inc.

Wilmington, DE

Jerome Kleckley, M.S.W., C.S.W.

Director

Hospital Services

Eastern Paralyzed Veterans Association

Jackson Heights, NY
36

Stephanie Smith Lee

Governmental Affairs

National Down Syndrome Society

Oakton, VA

Bryon R. MacDonald

Public Policy Advocate

World Institute on Disability

Oakland, CA

Stephen L. Start

President and CEO

S.L. Start Associates

Spokane, WA

Susan Webb

President

Webb Transitions, Inc.

Phoenix, AZ
37

APPENDIX 2:

DESIGN AND EVALUATION COMMITTEE

Thomas P. Golden, M.S., C.R.C., Chair

Extension Faculty

Program on Employment and Disability

School of Industrial and Labor Relations

Cornell University

Ithaca, NY

Richard V. Burkhauser, Ph.D.

Professor of Policy Analysis

Chair, Department of Policy Analysis and Management

Cornell University

Ithaca, NY

Kristen E. Flaten, M. Div.

Employment Consultant

Lifetrack Resources

St. Paul, MN

Stephanie Smith Lee

Governmental Affairs

National Down Syndrome Society

Oakton, VA

Bryon R. MacDonald

Public Policy Advocate

World Institute on Disability

Oakland, CA

Susan Webb

President

Webb Transitions, Inc.

Phoenix, AZ

39

APPENDIX 3:

LIST OF EXPERTS

EXPERTS ROUNDTABLE, JULY 27, 2001

Peter Blanck

Professor of Law, Director

Law, Health Policy and Disability Center

University of Iowa

Iowa City, IA

Crystal R. Blyler, Ph.D.

Social Science Analyst

Center for Mental Health Services

Rockville, MD

Jane Burke, M.S.

Research Specialist and Project Manager

University of Illinois at Chicago

Mental Health Services Research Program

Chicago, IL

Claire T. GhiLoni

Director

Statewide Employment Services

Massachusetts Rehabilitation Commission

Boston, MA

Richard L. Horne, Ed.D.

Senior Policy Advisor

Presidential Task Force on Employment of Adults With Disabilities

Washington, DC

William E. Kiernan, Ph.D.

Director

Institute for Community Inclusion

Boston, MA

John Kregel

Associate Director – VCU RRTC

Virginia Commonwealth University

Richmond, VA

David Mank

Director

Indiana Institute on Disability

Bloomington, IN
40

Constance Pledger, Ph.D.

Project Officer

NIDRR

U.S. Department of Education

Washington, DC

Ronald E. Rucker

President and CEO

VIA of the Lehigh Valley, Inc.

Allentown, PA

Priscilla Sanderson

Director

American Indian Research and Training Center

Arizona University Affiliated Program

Institute for Human Development

Northern Arizona University

Flagstaff, AZ

Robert Silverstein

Director

Center for the Study and Advancement of Disability Policy

Washington, DC

Sue Swenson

Commissioner

Administration on Developmental Disabilities

U.S. Department of Health and Human Services

Washington, DC
41

APPENDIX 4:

PUBLIC COMMENTERS, JULY 27, 2001

Dr. William Halleran

U.S. Department of Education

Washington, DC

Jim Larson

President and CEO

Morningside

Olympia, WA

Susan Goodman

Director of Governmental Relations

National Down Syndrome Congress

Oakton, VA

Jan Hoyme

Employment Link

Boulder, CO

Jamie Ruppman

Director of Governmental Affairs

TASH

Washington, DC

Joseph McNulty

Director

Helen Keller National Center

New York, NY

43

APPENDIX 5:

INDIVIDUALS PROVIDING PUBLIC COMMENT IN WRITING

FOLLOWING JULY 27, 2001

Barbara Carroll

The Arc of Tuscaloosa County

Brenda D. Carson

United Cerebral Palsy of Greater Birmingham, Inc.

Sue Gunn

Region VII Community Rehabilitation Regional Continuing Education Program

David L. Horne

Oklahoma Association for Persons in Supported Employment

Diane Kaatz

Wildwood Programs

Ed Kaler

Mid-Coast Mental Health Center

Robert A. Lawhead

Boulder, CO

Debra Martin Luecking

University of Tennessee

Brenda McComb

Alabama Department of Mental Health and Mental Retardation

Celane McWhorter

Association for Persons in Supported Employment

Paul Mendez

National Association of Workforce Development Professionals

Leslie K. Paull

Ohio Developmental Disabilities Planning Council

Susan Rinne

Options for Better Living, Inc.

Carol Rydell

Advocate
44

Steve Savage

Indiana Association for Persons in Supported Employment

Yvonne Swanson

Human Technologies Corporation

Peggy Todd

University of Missouri at Columbia
45

APPENDIX 6:

STAFF, JULY 27, 2001

Social Security Administration Staff

Steve Fear

Minh Huynh

Howard Iams

Deborah Morrison

Paul O’Leary

Advisory Panel Staff

Marie Strahan, Executive Director

Gordon Richmond

Theda Zawaiza

Ilene Zeitzer

47

APPENDIX 7:

PRESENTERS, JULY 27, 2001

Jane Burke, M.S.

Research Specialist and Project Manager

University of Illinois at Chicago Mental Health Services Research Program

Chicago, IL

Gina Livermore

Research Associate

Cornell Center for Policy Research

Washington, DC

Paul O’Leary

Project Director

Office of Policy

Social Security Administration

Washington, DC

Priscilla Sanderson

Director

American Indian Rehabilitation Research and Training Center

Arizona University Affiliated Program

Institute for Human Development

Northern Arizona University

Flagstaff, AZ

49

APPENDIX 8:

References and Bibliography

Albin, J. M., Rhodes, L., and Mank, D. (1994). Changeover to community employment:

The problem of realigning organizational culture, resources, and community roles.

Journal of the Association for Persons with Severe Handicaps, 19(2), 105–115.

Bellamy, G. T., Horner, R. H., and Inman, D. P. (1979). Vocational habilitation of

severely retarded adults: A direct service technology. Austin, TX: PRO-ED.

Bellamy, G. T., Rhodes, L. E., Mank, D. M., and Albin, J. M. (1988). Supported

employment: A community implementation guide. Baltimore: Paul H. Brookes

Publishing Co.

Berkell, D. E. (1987). Vocational assessment of students with severe handicaps: A review

of the literature. Career Development for Exceptional Individuals, 10(2), 61–75.

Beyer, S. (1999). Relationships between agency organization, cost, and worker

outcomes—the S.E.P.O.D. project. Paper presented at the European Union of

Supported Employment, Rome.

Braddock, D. (2001). Public financial support for disability programs at the close of the

20th century. A working paper. Boulder, CO: Coleman Institute for Cognitive

Disabilities, University of Colorado.

Braddock, D., Hemp, R., Parish, S., and Rizzolo, M. C. (2000). The state of the states in

developmental disabilities: 2000 study summary. Chicago: University of Illinois at

Chicago, Department of Disability and Human Development.

Buckley, J. T., and Bellamy, G. T. (1985). National survey of day and vocational

programs for adults with severe disabilities: A 1984 profile. In P. Ferguson (Ed.),

Issues in transition research: Economic and social outcomes (pp. 1–12). Eugene, OR:

Specialized Training Program, University of Oregon.

Butterworth, J., and Gilmore, D. (2000). Are we there yet? Trends in employment

opportunities and supports. TASH Newsletter, 26(6/7), 5–7.

Butterworth, J., Gilmore, D., and Kiernan, W. E. (2000). Trends and issues in sheltered

employment services. Boston: Children’s Hospital, Institute for Community Inclusion.

Butterworth, J., Gilmore, D., Kiernan, W. E., and Schalock, R. (1999). State trends in

employment services for people with developmental disabilities: Multiyear

comparisons based on state MR/DD agency and vocational rehabilitation (RSA) data.

Boston: Children’s Hospital, Institute for Community Inclusion.
50

Callahan, M. (2000, December). Choice and self-determination in employment. Common

Sense, 8, 1–4.

Callahan, M. J., and Garner, J. B. (1997). Keys to the workplace: Skills and supports for

people with disabilities. Baltimore: Paul H. Brookes Publishing Co.

Callahan, M., and Mank, D. (1997). Choice and control of employment for people with

disabilities. Paper prepared for The Robert Wood Johnson Foundation SelfDetermination Initiative, Princeton, NJ.

Campbell, D. T., and Stanley, J. C. (1963). Experimental and quasi-experimental designs

for research. Boston: Houghton Mifflin Company.

Carr, E. G., Horner, R. H., Turnbull, A. P., Marquis, J. G., McLaughlin, D. M., McAtee,

M. L., Smith, C. E., Ryan, K. A., Ruef, M. B., and Doolabh, A. (1999). In D.

Braddock (Ed.), Positive behavior support for people with developmental disabilities:

A research synthesis. Washington, DC: American Association on Mental Retardation.

Church, G., and Glennen, S. (1992). The handbook of assistive technology. San Diego,

CA: Singular Publishing Group, Inc.

Everson, J. M. (1995). Supporting young adults who are deaf-blind in their communities:

A transition planning guide for service providers, families, and friends. Baltimore:

Paul H. Brookes Publishing Co.

Gaylord-Ross, R. (1986). The role of assessment in transitional, supported employment.

Career Development for Exceptional Individuals, 9(2), 129–134.

Gilmore, D., Schuster, J., Timmons, J., and Butterworth, J. (2000). An analysis of trends

for people with mental retardation, cerebral palsy, and epilepsy receiving services

from state vocational rehabilitation agencies: Ten years of progress. Rehabilitation

Counseling Bulletin, 4(1), 30–38.

Gold, M. (1973). Research on the vocational rehabilitation of the retarded: The present,

the future. In N. Ellis (Ed.), International review of research in mental retardation

(Vol. 6, pp. 97–147). New York: Academic Press.

Golden, T. P., O’Mara, S., Ferrell, C., and Sheldon, J. (2001). Benefits planning,

assistance and outreach: Supporting the career development and employment of

people with disabilities. Publication Number 63-003. Washington, DC: U.S.

Government Printing Office, Social Security Administration.

Golden, T. P., O’Mara, S., and Czechowicz, J. (1998). Plans for achieving self-support:

An improved tool for self-directed vocational rehabilitation. Journal of Applied

Rehabilitation Counseling, 29, 4.
51

Griffin, C. (1998). Rural facts: Rural supported employment. Missoula, MT: Research

and Training Center on Rural Rehabilitation Services, Montana University Affiliated

Rural Institute on Disabilities, University of Montana. Retrieved January 16, 2001,

from the World Wide Web:

http://ruralinstitute.umt.edu/rtcrural/Pubs/allRTCPubs.htm.

Guba, E. G. (1978). Toward a methodology of naturalistic inquiry in educational

research. CSE Monograph Series in Evaluation. 8. Los Angeles: Center for the Study

of Evaluation, UCLA Graduate School of Education.

Human Services Research Institute (HSRI). (1999). Consumer Survey. Cambridge, MA:

HSRI.

Huynh, M., and O’Leary, P. K. (2001). Design of the Study of the Adequacy of

Incentives for the Ticket to Work Program. Working Paper 003. Washington, DC:

Office of Policy, Social Security Administration.

Hughes, C., and Carter, E. W. (2000). The transition handbook. Strategies high school

teachers use that work! Baltimore: Paul H. Brookes Publishing Co.

Izzo, M. (2000). National longitudinal transition study. Presentation at World Congress

on Disability, Atlanta, GA.

Izzo, M. V., Cartledge, G., Miller, L., Growick, B., and Rutkowski, S. (2000). Increasing

employment earnings: Extended transition services that make a difference. Career

Development for Exceptional Individuals, 23(2), 139–156.

Jenaro, C., Mank, D., Bottomley, J., Doose, S., and Tuckerman, P. (in press). Supported

employment in the international context: An analysis of processes and outcomes.

Journal of Vocational Rehabilitation.

Kiernan, W. E., and Schalock, R. L. (Eds.). (1997). Integrated employment: Current

status and future directions. Washington, DC: American Association on Mental

Retardation.

Koegel, L. K., Koegel, R. L., and Dunlap, G. (1996). Positive behavioral support:

Including people with difficult behavior in the community. Baltimore: Paul H.

Brookes Publishing Co.

Koehler, F., and Ellis, J. (Eds.). (1990). Project WIN, Work Incentives Network, training

manual. Richmond, VA: Association for Persons in Supported Employment.

Kregel, J. (2001). Potential strategies for “Operationalizing” the target populations

defined in Section 101 (h) (5) (c) of P. L. 106-70, The Ticket to Work and Work

Incentives Improvement Act. Richmond, VA: Virginia Commonwealth University.
52

Mank, D. M. (1994). The underachievement of supported employment: A call for

reinvestment. Journal of Disability Policy Studies, 5(2), 1–24.

Mank, D. M. (1996). Evolving roles for employers and support personnel in the

employment of people with severe disabilities. Journal of Vocational Rehabilitation,

6(1), 83–88.

Mank, D. (1997). Systems change strategies for integrated employment: A blueprint for

the future. In W. E. Kiernan and R. Schalock (Eds.), Integrated employment: Current

status and future directions (pp. 107–120). Washington, DC: American Association

on Mental Retardation.

Mank, D., Buckley, J., Cioffi, A., and Dean, J. (1996). Do social systems really change?

Retrospective interviews with state supported employment systems change project

directors. Journal of Autism and Developmental Disabilities, 11(4), 243–250.

Mank, D., Cioffi, A., and Yovanoff, P. (1997). Patterns of support for employees with

severe disabilities. Mental Retardation, 35(6), 433–447.

Mank, D., Cioffi, A., and Yovanoff, P. (1998). Employment outcomes for people with

severe disabilities: Opportunities for improvement. Mental Retardation, 36(3), 205–

216.

Mank, D., Cioffi, A., and Yovanoff, P. (1999). Impact of coworker involvement with

supported employees on wage and integration outcomes. Mental Retardation, 37(5),

383–394.

Mank, D., O’Neill, C. T., and Jensen, R. (1998). Quality in supported employment: A

new demonstration of the capabilities of people with severe disabilities. Journal of

Vocational Rehabilitation, 11(1), 83–95.

McDowell, J., and Ferrell, C. (Eds.). (2000). Project DOOR’s guide to using social

security work incentives. Bloomington, IN: Indiana University, Indiana Institute on

Disability and Community.

McGaughey, M., and Mank, D. (2001). Empirical evidence of systems change in

supported employment. Journal of Disability Policy Studies, 11(4), 210–222.

McGaughey, M., and Mank, D. (2001). State policy innovation and systems change: The

context for the supported employment initiative. Journal of Disability Policy Studies,

11(4), 202–209.

McMillan, J. H., and Schumacher, S. (1984). Research in education: A conceptual

introduction. Boston: Little, Brown and Company.

Menchetti, B. M., and Rusch, F. R. (1988). Vocational evaluation and eligibility for

rehabilitation services. In P. Wehman and M. S. Moon (Eds.), Vocational
53

rehabilitation and supported employment (pp. 79–90). Baltimore: Paul H. Brookes

Publishing Co.

Moon, M. S., Inge, K. J., Wehman, P., Brooke, V., and Barcus, J. M. (1990). Helping

persons with severe mental retardation get and keep employment: Supported

employment issues and strategies. Baltimore: Paul H. Brookes Publishing Co.

Murphy, S. T., and Rogan, P. M. (1995). Closing the shop: Conversion from sheltered to

integrated work. Baltimore: Paul H. Brookes Publishing Co.

Nisbet, J. (1992). Natural supports in school, at work, and in the community for people

with severe disabilities. Baltimore: Paul H. Brookes Publishing Co.

Novak, J. (2000). Changing supported employment systems: Findings from a national

survey of state vocational rehabilitation agencies. Preliminary report. Bloomington,

IN: Indiana Institute on Disability and Community, Indiana University.

Novak, J., Mank, D., Revell, G., and O’Brien, D. (1999). Paying for success: Resultsbased approaches to funding supported employment. In G. Revell, K. Inge, D. Mank,

and P. Wehman (Eds.), The impact of supported employment for people with

significant disabilities: Preliminary findings from the National Supported

Employment Consortium (pp. 25–42). Richmond, VA: Virginia Commonwealth

University.

Olson, D., Cioffi, A., Mank, D., and Yovanoff, P. (2001). Employers’ perceptions of

employees with mental retardation. Journal of Vocational Rehabilitation, 16(2),

125–133.

Presidential Task Force on Employment of Adults with Disabilities. (2000, December).

Recharting the course: Turning points. (Third report.) Washington, DC: Author.

Revell, G., Inge, K. J., Mank, D., and Wehman, P. (Eds.). (1999). The impact of

supported employment for people with significant disabilities: Preliminary findings

from the National Supported Employment Consortium. Richmond, VA: Virginia

Commonwealth University.

Rogan, P., Banks, B., and Howard, M. (2000). Workplace supports in practice: As little

as possible, as much as necessary. Focus on Autism and Other Developmental

Disabilities, 15(1), 2–11.

Sands, D. J., and Wehmeyer, M. L. (1996). Self-determination across the life span:

Independence and choice for people with disabilities. Baltimore: Paul H. Brookes

Publishing Co.
54

Schalock, R. L., and Braddock, D. L. (Eds.). (1999). Adaptive behavior and its

measurement: Implications for the field of mental retardation. Washington, DC:

American Association on Mental Retardation.

Sheldon, J., and Trach, J. S. (1998). SSDI and SSI work incentives with

recommendations for policy change. Journal of Applied Rehabilitation Counseling,

29, 4.

Sowers, J., and Powers, L. (1991). Vocational preparation and employment of students

with physical and multiple disabilities. Baltimore: Paul H. Brookes Publishing Co.

Stiles, K. (1999). Beyond borders: Global supported employment and people with

disabilities. St. Augustine, FL: Training Resource Network, Inc.

Taylor, H. (2000). Harris Poll #59, Conflicting trends in employment of people with

disabilities—1986–2000. Retrieved January 16, 2001, from the World Wide Web:

http://www.harrisinteractive.com/harris_poll/index.asp?PID=121.

U.S. Department of Health and Human Services. (2001). Olmstead update no. 5 (SMDL

#01-007). Baltimore, MD: Author.

Unger, D. D. (1999). Workplace supports: A view from employers who have hired

supported employees. Focus on Autism and Other Developmental Disabilities, 14(3),

167–179.

Wehman, P. (2001). Life beyond the classroom. Transition strategies for young people

with disabilities. 3rd ed. Baltimore: Paul H. Brookes Publishing Co.

Wehman, P. (Ed.). (2001). Supported employment in business: Expanding the capacity of

workers with disabilities. St. Augustine, FL: Training Resource Network, Inc.

Wehman, P., and Bricout, J. (1999). Supported employment: Critical issues and new

directions. In G. Revell, K. Inge, D. Mank, and P. Wehman (Eds.), The impact of

supported employment for people with significant disabilities: Preliminary findings

from the National Supported Employment Consortium (pp. 1–24). Richmond, VA:

Virginia Commonwealth University.

Wehman, P., and Moon, M. S. (1988). Vocational rehabilitation and supported

employment. Baltimore: Paul H. Brookes Publishing Co.

Weiss, C. H. (1972). Evaluation Research: Methods of assessing program effectiveness.

Englewood Cliffs, NJ: Prentice Hall, Inc.