United States Government Accountability Office 

GAO	 Report to the Committee on Finance, U.S. 

Senate, and the Committee on Ways and 

Means, House of Represetatives 

November 2004 SOCIAL SECURITY 

DISABILITY 

Improved Processes 

for Planning and 

Conducting 

Demonstrations May 

Help SSA More 

Effectively Use Its 

Demonstration 

Authority 

On November 9, 2004, this report was reposted to 

the Web because the Highlights page was missing 

from the electronic copy. 

GAO-05-19


a
Why GAO Did This Study 

Highlights 

Accountability Integrity Reliability 

www.gao.gov/cgi-bin/getrpt?GAO-05-19. 

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact Robert E. 

Robertson at (202) 512-7215 or 

RobertsonR@gao.gov. 

Highlights of GAO-05-19, a report to 

Committee on Finance, U.S. Senate, and 

the Committee on Ways and Means, 

House of Representatives 

Since 1980, the Congress has 

required the Social Security 

Administration (SSA) to conduct 

demonstration projects to test the 

effectiveness of possible program 

changes that could encourage 

individuals to return to work and 

decrease their dependence on 

Disability Insurance (DI) benefits. 

To conduct these demonstrations, 

the Congress authorized SSA, on a 

temporary basis, to waive certain 

DI and Medicare program rules and 

to use Social Security Trust Funds. 

The Congress required GAO to 

review SSA’s use of its DI 

demonstration authority and to 

make a recommendation as to 

whether this authority should be 

made permanent. 

What GAO Recommends 

GAO recommends that SSA 

develop a formal agenda for its 

demonstrations, establish an expert 

panel to guide the design and 

implementation of demonstrations, 

and establish formal processes to 

ensure full consideration of 

demonstration results. GAO also 

identifies several matters for the 

Congress to consider, including 

continuation of DI demonstration 

authority on a temporary basis, 

establishment of additional 

reporting requirements for 

demonstrations, and clearer 

specification of the methodological 

and evaluation requirements for 

demonstrations. 

In its comments on a draft of this 

report, SSA agreed with GAO’s 

recommendations. 

November 2004 

SOCIAL SECURITY DISABILITY 

Improved Processes for Planning and 

Conducting Demonstrations May Help 

SSA More Effectively Use Its 

Demonstration Authority 

What GAO Found 

SSA has not used its demonstration authority to extensively evaluate a wide 

range of DI policy areas dealing with return to work. Despite being given the 

authority to assess a broad range of policy alternatives, SSA has, until very 

recently, focused its demonstration efforts mostly on a relatively narrow set 

of policy issues—those dealing with the provision of vocational 

rehabilitation and employment services. SSA’s recently proposed or initiated 

demonstrations have begun to address a broader range of policy issues, such 

as provisions to reduce, rather than terminate, benefits based on earnings 

above a certain level. However, the agency has no systematic processes or 

mechanisms for ensuring that it is adequately identifying and prioritizing 

those issues that could best be addressed through use of its demonstration 

authority. For example, the agency has not developed a formal 

demonstration research agenda explicitly identifying its broad vision for 

using its DI demonstration authority and explaining how ongoing or 

proposed demonstration projects support achievement of the agency’s goals 

and objectives. 

SSA’s demonstration projects have had little impact on the agency’s and the 

Congress’ consideration of DI policy issues. This is due, in part, to 

methodological limitations that have prevented SSA from producing project 

results that are useful for reliably assessing DI policy alternatives. In 

addition, SSA has not established a formal process for ensuring that its 

demonstration results are fully considered for potential policy implications. 

For example, SSA does not maintain a comprehensive record of its 

demonstration results that could be used to build a body of knowledge for 

informing policy decisions and planning future research. Furthermore, SSA’s 

reporting of demonstration project results has been insufficient in ensuring 

that the Congress is fully apprised of these results and their policy 

implications. 

United States Government Accountability Office 
Contents 


Letter 

Results in Brief 


Background


SSA Has Not Used Its Demonstration Authority to Evaluate a Wide 


Range of DI Return-to-Work Policy Issues 

SSA’s Demonstration Projects Have Had Little Influence on 

Consideration of DI Policy Changes 

Conclusions 

Recommendations 

Matters for Congressional Consideration 

Agency Comments and Our Evaluation 

1 

3 

4 

10 

18 

23 

24 

25 

27 

Appendix I Scope and Methodology 

Appendix II Comments from the Social Security Administration 

Appendix III GAO Contacts and Staff Acknowledgments 34 

GAO Contacts 34 

Staff Acknowledgments 34 

Tables 

Table 1: Completed or Nearly Completed Projects Conducted 

under SSA’s DI Demonstration Authority 12 

Table 2: Recently Proposed or Initiated Projects Conducted under 

SSA’s DI Demonstration Authority 15 

Page i GAO-05-19 Social Security Disability 

28 

30 
Abbreviations 

DI Disability Insurance 


HHS Department of Health and Human Services 


OASDI Old-Age and Survivors Insurance and Disability Insurance 


OPDR Office of Program Development and Research 


RDP Research Demonstration Program 


RFP request for proposal 


RSVP Referral System for Vocational Rehabilitation Providers 


SGA substantial gainful activity 


SPI State Partnership Initiative 


SSI Supplemental Security Income 


SSA Social Security Administration 


SSAB Social Security Advisory Board 


This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 

Page ii GAO-05-19 Social Security Disability 
United States Government Accountability Office 

Washington, DC 20548 

November 4, 2004 


The Honorable Charles E. Grassley 


Chairman 


The Honorable Max Baucus 


Ranking Minority Member 


Committee on Finance 


U. S. Senate 


The Honorable William M. Thomas 


Chairman 


The Honorable Charles B. Rangel 


Ranking Minority Member 


Committee on Ways and Means 


House of Representatives 


From 1982 to 2002, the number of disabled workers receiving benefits 


under the Disability Insurance (DI) program doubled from 2.6 million to 


5.5 million, while payments quadrupled from about $14.8 billion to 


$60 billion. Although technological and medical advances, along with 


social changes, have increased the potential for some people with 


disabilities to participate in the labor force, less than 1 percent of DI 


beneficiaries leave the rolls each year because they have reentered the 


workforce. The enactment of various DI work incentives that are intended


to encourage beneficiaries to work—and, potentially, to leave the rolls—


has also had little discernible impact on beneficiaries’ success in returning 


to the workforce. This low rate of return to work, coupled with escalating 


growth in DI program enrollment and benefit costs over the past few 


decades, threatens to exhaust the DI Trust Fund by 2029. Yet relatively 


little is known about the factors that help beneficiaries overcome 


employment challenges and disincentives and that inhibit them from


achieving an earnings level that leads to self-sufficiency. GAO has 


designated federal disability programs as a high-risk area, in part because 


Page 1 GAO-05-19 Social Security Disability 
of the significant growth in the DI program and the long-standing 

challenges in devising effective return-to-work policies.1 

In response to these challenges, the Congress has, since 1980, required the 

Social Security Administration (SSA) to conduct demonstration projects to 

test the effectiveness of possible program changes that could encourage 

individuals to return to work and decrease their dependence on DI 

benefits. In fostering return to work, these demonstrations and the 

program changes they test are intended to produce savings in the Trust 

Funds or improve DI program administration. To achieve these objectives, 

SSA’s DI demonstration authority contains several key features that 

provide SSA with a potentially valuable tool for assessing the effectiveness 

of policy alternatives. One of these features is SSA’s authority to waive 

certain DI and Medicare program rules. For example, when conducting 

demonstrations, SSA is permitted to exempt certain beneficiaries from 

requirements that workers with disabilities earn below a certain amount to 

remain eligible for DI benefits or that they wait 24 months to become 

eligible for Medicare benefits. Another key aspect of SSA’s demonstration 

authority is the requirement that DI demonstration projects be of sufficient 

scope and conducted on a wide enough scale to ensure a thorough 

evaluation and results that are applicable to the DI program as a whole. In 

addition, the legislation authorized SSA to use DI Trust Fund and Old-Age 

and Survivors Insurance Trust Fund monies to pay for the demonstrations 

and required SSA to periodically report to the Congress on its 

demonstration activities, providing, when appropriate, recommendations 

for legislative or administrative changes. The Congress initially granted 

SSA this demonstration authority for a 5-year period but subsequently 

renewed the authority several times, most recently extending it through 

2005. 

The Ticket to Work and Work Incentives Improvement Act (P.L. 106-170) 

mandated that GAO review SSA’s use of its DI demonstration authority 

and provide a recommendation as to whether SSA should be given this 

authority permanently. To develop a basis on which to make this 

1

GAO identifies federal programs and operations that are high-risk because of their greater 

vulnerability to fraud, waste, abuse, and mismanagement. GAO has also increasingly used 

its high-risk designation to draw attention to the challenges faced by government programs 

and operations in need of broad-based transformation. See GAO, High-Risk Series: An 

Update, GAO-03-119 (Washington, D.C.: January 2003) and GAO, Major Management 

Challenges and Program Risks: Social Security Administration, GAO-03-117 

(Washington, D.C.: January 2003). 

Page 2 GAO-05-19 Social Security Disability 
Results in Brief 

recommendation, we examined the extent to which (1) SSA has used the 

demonstration authority to test a range of program changes and (2) SSA’s 

demonstration authority has been used to inform policy decisions. In 

conducting our examination, we reviewed relevant federal statutes and 

regulations governing DI demonstration activities and interviewed SSA 

officials with current and prior responsibility for demonstration projects. 

We also reviewed documents related to SSA’s planning, implementation, 

and evaluation of demonstration projects, including agency reports to the 

Congress, public notices, and reports analyzing project results. In addition, 

we examined other reviews of SSA’s demonstrations (including prior GAO 

and Inspector General reports) and interviewed officials from disability 

research, advisory, and advocacy organizations. Furthermore, we obtained 

information on research and demonstrations conducted by other federal 

agencies. We performed our work in accordance with generally accepted 

government auditing standards between October 2003 and August 2004. 

For more details about our scope and methodology, see appendix I. 

SSA has not used its demonstration authority to extensively evaluate a 

wide range of DI policy areas dealing with return to work. Despite being 

given the authority to assess a broad range of policy alternatives, SSA has, 

until very recently, focused its demonstration efforts mostly on a relatively 

narrow set of policy issues—those dealing with the provision of vocational 

rehabilitation and employment services. But even in that policy area, SSA’s 

use of DI demonstration authority has not been comprehensive and, 

therefore, did not extensively address key policy issues that the agency is 

currently grappling with in its efforts to implement a new approach for 

providing return-to-work assistance. SSA’s recently proposed or initiated 

demonstrations have begun to address a broader range of policy issues, 

such as provisions to reduce, rather than terminate, benefits based on 

earnings above a certain level. However, the agency has no systematic 

processes or mechanisms for ensuring that it is adequately identifying and 

prioritizing those issues that could best be addressed through use of its 

demonstration authority. For example, the agency has not developed a 

formal demonstration research agenda explicitly identifying its broad 

vision for using its DI demonstration authority and explaining how 

ongoing or proposed demonstration projects support achievement of the 

agency’s goals and objectives. 

SSA’s demonstration projects have had little impact on the agency’s and 

the Congress’ consideration of DI policy issues. This is due, in part, to 

methodological limitations that have prevented SSA from producing 

project results that are useful for reliably assessing DI policy alternatives. 

Page 3 GAO-05-19 Social Security Disability 
Background 

In addition, SSA has not established a formal process for ensuring that its 

demonstration results are fully considered for potential policy 

implications. For example, SSA does not maintain a comprehensive record 

of its demonstration results that could be used to build a body of 

knowledge for informing policy decisions and planning future research. 

Furthermore, SSA’s reporting of demonstration project results has been 

insufficient in ensuring that the Congress is fully apprised of these results 

and their policy implications. 

This report contains recommendations to help ensure the effectiveness of 

SSA’s DI demonstration efforts. It also identifies several matters that the 

Congress may wish to consider, including continuation of SSA’s 

demonstration authority on a temporary basis. In its comments on a draft 

of this report, SSA agreed with our recommendations. SSA also agreed 

that it has not, in the past, used its demonstration authority to extensively 

evaluate DI policy, but it noted that its recently initiated or proposed 

demonstrations will play a vital role in testing program and policy 

changes. SSA’s comments appear in appendix II. 

Established in 1956, DI is an insurance program that provides monthly 

cash benefits to workers who are unable to work because of severe longterm disability. Workers who have worked long enough and recently 

enough are insured for coverage under the DI program. To meet the 

definition of disability under the DI program, an individual must have a 

medically determinable physical or mental impairment that (1) has lasted 

or is expected to last at least 1 year or to result in death and (2) prevents 

the individual from engaging in substantial gainful activity (SGA). 

Individuals are considered to be engaged in SGA if they have countable 

earnings above a certain dollar level.

2

 Once a person is on the disability 

rolls, benefits continue until (1) the beneficiary dies, (2) the beneficiary 

becomes eligible for Social Security retirement benefits at full retirement 

age,3

 (3) SSA determines that the beneficiary is no longer eligible for 

benefits because his or her earned income exceeds the SGA level, or 

(4) SSA decides that the beneficiary’s medical condition has improved to 

2

For 2004, SSA considers countable earnings above $810 a month to be substantial gainful 

activity for persons who are not blind and $1,350 a month for persons who are blind. 

3

The full retirement age is the earliest age at which an unreduced retirement benefit is 

payable. The age for full retirement benefits is scheduled to rise gradually from age 65 to 

67. 

Page 4 GAO-05-19 Social Security Disability 
the point that he or she is no longer considered disabled. In 2002, SSA paid 

about $60 billion in DI cash benefits to 5.5 million disabled workers,4

 with 

average monthly benefits amounting to $834 per person.5

 In addition to 

receiving cash assistance, beneficiaries automatically qualify for Medicare 

after 24 months of DI entitlement. 

During the 1970s, as the number of disability awards increased 

significantly and resulted in substantial cost increases for the DI program, 

the Congress became concerned about the growth of the DI program and 

program rules that provided disincentives to returning to work. To 

encourage DI beneficiaries to return to work—and, potentially, to leave 

the benefit rolls—the Congress has, over the years, enacted legislation 

providing various work incentives. Such incentives include a trial work 

period during which beneficiaries may earn any amount for 9 months 

within a 60-month period and still receive full cash and medical benefits 

and continued Medicare coverage that allows beneficiaries to maintain 

eligibility for Medicare for at least 39 months following a trial work period 

as long as medical disability continues. 

In an effort to further address these issues, the Congress, in 1980, required 

SSA to conduct demonstration projects6

 to evaluate the effectiveness of 

policy alternatives that could encourage DI beneficiaries to reenter the 

4

In 2002, the DI program also paid about $6 billion in cash benefits to about 1.7 million 

spouses and children of disabled workers. DI cash benefits are paid from the Federal 

Disability Insurance Trust Fund. This trust fund is funded through payroll deductions paid 

by employers and workers. 

5

For disabled and blind individuals who have low income and limited resources, SSA 

provides cash benefits under the Supplemental Security Income (SSI) program. Although 

both the DI and the SSI programs use the same definition of disability, SSI has no prior 

work requirement. In 2002, SSA paid about $26 billion in SSI benefits to about 5.5 million 

people with disabilities, of whom 3.9 million were working age adults aged 18 to 64. The 

average monthly federal SSI cash benefit amounted to about $398 per person. In 2002, 

about 1.2 million DI beneficiaries were dually eligible for SSI benefits because of the low 

level of their income and resources. 

6

This DI demonstration authority was provided under Section 505(a) of the Social Security 

Disability Amendments of 1980 (Pub. L. No. 96-265). Section 505(b) of this act amended 

Section 1110 of the Social Security Act to provide SSA with similar demonstration authority 

for the SSI program, including authority to waive SSI program rules. However, unlike the 

DI demonstrations, SSI demonstration projects were to be funded from congressional 

appropriations. Although SSA’s DI and SSI demonstration authorities are separate and 

somewhat distinct, the agency’s disability demonstration projects often involve both DI and 

SSI beneficiaries and applicants. When a demonstration is conducted under both the DI 

and SSI demonstration authorities, funding for the project is also split between trust fund 

(i.e., DI) and appropriated (i.e., SSI) sources. 

Page 5 GAO-05-19 Social Security Disability 
workforce.7

 A key aspect of this demonstration authority is SSA’s ability to 

waive DI and Medicare program rules to the extent needed in conducting 

these projects. The legislation granting DI demonstration authority also 

identified a variety of policy alternatives for SSA to consider testing, 

including (1) alternative ways of treating DI beneficiaries’ work-related 

activity such as methods allowing for a reduction in benefits based on 

earnings and (2) modifications in other rules, such as the trial work period 

and Medicare eligibility waiting period, that may serve as obstacles to DI 

beneficiaries returning to work. In addition, this legislation identified 

several requirements pertaining to the design and evaluation of DI 

demonstration projects. In particular, these projects were required to be of 

sufficient scope and carried out on a wide enough scale to permit a 

thorough evaluation of the policy alternatives studied such that the results 

would be generally applicable to the operation of the DI program. The law 

additionally required SSA to submit reports to the Congress announcing 

the initiation of DI demonstration projects as well as periodic reports 

describing the status of these projects and a final report on all projects 

carried out under the demonstration authority. SSA was directed to make 

recommendations, when appropriate, for legislative or administrative 

changes in its reports to the Congress. 

Another important aspect of SSA’s DI demonstration authority is that 

unlike other SSA research activities, which are funded through 

congressional appropriations, these projects can be paid for with DI Trust 

Fund and Old-Age and Survivors Insurance Trust Fund monies. Therefore, 

SSA is not required to obtain congressional approval for DI demonstration 

expenditures, although it is required to receive approval from the Office of 

Management and Budget for an annual apportionment of Trust Funds for 

these demonstrations. 

SSA’s DI demonstration authority has always been granted on a temporary 

basis and therefore has been subject to periodic review and renewal by the 

Congress.8

 After initially granting this authority for a 5-year period, the 

7

In a report discussing federal evaluation activities, GAO defined demonstration projects as 

“those that aim to provide evidence of the feasibility or effectiveness of a new approach or 

practice.” See GAO, Program Evaluation: Improving the Flow of Information to 

Congress, GAO-PEMD-95-1 (Washington, D.C.: January 30, 1995). This definition is 

consistent with the purpose of SSA’s DI demonstration projects, as spelled out in 

legislation. 

8

Although SSA has temporary authority for DI demonstrations, it has permanent authority 

to conduct SSI demonstrations. 

Page 6 GAO-05-19 Social Security Disability 
Congress subsequently renewed it several times, in 1986, 1989, 1994, 1999, 

and 2004. The renewal of SSA’s authority has sometimes been delayed so 

that SSA has, on several occasions, gone without DI demonstration 

authority. For example, after its demonstration authority expired in June 

1996, SSA was not again granted DI demonstration authority until 

December 1999. Most recently, the Congress extended this demonstration 

authority through December 2005.9 

In addition to granting this general DI demonstration authority, the 

Congress may enact legislative mandates for SSA to conduct specific DI 

demonstration projects. For example, the Ticket to Work and Work 

Incentives Improvement Act of 1999 required SSA to conduct a 

demonstration to assess the effectiveness of a benefit offset program 

under which DI benefits are reduced by $1 for every $2 in earnings (above 

a certain level) by a beneficiary. SSA’s authority to conduct this 

demonstration is similar in some respects to the authority it has under its 

general DI demonstration statute. For instance, the statute allows waiver 

of DI and Medicare program provisions to carry out this benefit offset 

demonstration. However, some differences exist between the two 

authorities. In particular, the benefit offset demonstration authority 

provides a more detailed and comprehensive list of demonstration 

objectives for SSA to fulfill than does SSA’s general authority. For 

example, the benefit offset demonstration authority lists six “matters to be 

determined,” which include assessments of project costs; savings to the 

Trust Funds; and project effects on employment outcomes such as wages, 

occupations, benefits, and hours worked. 

Regardless of the authority under which they are carried out, 

demonstration projects examining the impact of social programs are 

inherently complex and difficult to conduct. Measuring outcomes, 

ensuring the consistency and quality of data collected at various sites, 

establishing a causal connection between outcomes and program 

activities, and separating out the influence of extraneous factors raise 

formidable technical and logistical problems. Thus, these projects 

generally require a planned study and considerable time and expense. 

Adding to these complexities are other administrative or statutory 

requirements affecting SSA’s DI demonstrations. For example, SSA’s 

9

In addition to extending the authority to initiate DI demonstration projects through 2005, 

Pub. L. No. 108-203 would also allow DI demonstrations that SSA has initiated on or before 

December 17, 2005, to be completed thereafter. 

Page 7 GAO-05-19 Social Security Disability 
policy is that its demonstration projects must not make those who 

participate in the project worse off, which could limit the specific types of 

policy alternatives the agency can study or the methods used to study such 

alternatives. 

Although the legislation granting DI demonstration authority does not 

prescribe the use of particular methodological approaches, SSA has 

repeatedly recognized that the law’s general requirements for 

demonstration evaluations require SSA to conduct these projects in a 

rigorous manner that provides the agency with a reliable basis for making 

policy recommendations.10 Rigorous methods are required to estimate the 

net impact of a tested disability policy option because many other factors, 

such as the economy, can influence whether a beneficiary returns to work. 

In an August 2002 report to the SSA Commissioner, an SSA advisory panel 

stated that it is widely agreed that experimental designs,11 “when feasible 

from operational and budgetary perspectives and when they can be 

conducted without serious threats to their validity, are the best 

methodology for determining the effects of changes in government 

programs.”12 In addition, SSA officials and other researchers have noted 

the advantages of experimental designs in providing policymakers with 

more clear-cut results that are less subject to debate than results derived 

from other methods.13 However, when experimental designs are not 

10The results of a rigorous demonstration may lead SSA to recommend adoption of a 

particular policy option or, alternatively, to recommend that such an option not be 

pursued. Also, to the extent that the results of a demonstration are not definitive enough to 

make such policy recommendations, SSA may instead propose that additional research be 

conducted to further assess policy alternatives. 

11Experimental designs involve random assignment of study participants to either a 

treatment group or a control group. The treatment group is subjected to the new program 

or policy, and the control group is not. The strength of the experimental design is in its 

assurance that those who experience the treatment are like, in all important ways, those 

who do not experience the treatment except for the difference in receiving the treatment 

itself. 

12Ticket to Work and Work Incentives Advisory Panel, Advice Report to the Commissioner 

of Social Security: Statutory Requirements and Design Issues Related to SSDI $1 for $2 

Benefit Offset Research, p. 12 (Washington, D.C.: August 2002). 

13For example, in a 1997 publication examining welfare reform evaluations, Besharov and 

others state that with experimental designs “Policymakers can then focus on the 

implications of findings, rather than ‘become entangled in a protracted and often 

inconclusive scientific debate about whether the findings of a particular study are 

statistically valid.’” See Douglas J. Besharov, Peter Germanis, and Peter H. Rossi, 

Evaluating Welfare Reform, A Guide for Scholars and Practitioners, the University of 

Maryland, School of Public Affairs, 1997, p. 42. 

Page 8 GAO-05-19 Social Security Disability 
feasible or desirable, the use of quasi-experimental designs14 offers a 

reasonably rigorous evaluation alternative that may, under certain 

circumstances, offer advantages over experimental designs.15 

Other factors may also limit the rigor of DI demonstrations, including 

insufficient sample sizes, inconsistency in demonstration design or 

implementation across multiple project sites, and deficiencies in data 

collection. Such design, implementation, and evaluation weaknesses may 

hamper SSA’s use of project results as a basis for making policy 

recommendations because they limit the agency’s ability to (1) control for 

factors external to the demonstration, (2) generalize demonstration results 

to a wider population of DI beneficiaries, and (3) isolate the effects of 

specific policy interventions from the overall effects produced by a 

demonstration. 

The Office of Program Development and Research (OPDR) is the entity 

within SSA that develops and implements demonstration projects for the 

DI and Supplemental Security Income (SSI) Programs.16 OPDR program 

and research staff—sometimes with the assistance of outside research 

organizations—identifies the broad outlines and requirements of disability 

program demonstration projects, including the basic objectives, scope, 

and methodological standards for these projects. SSA then issues formal 

notices requesting public or private sector organizations to submit offers 

to conduct the demonstration projects, which may include development of 

a detailed design plan, provision of technical support, collection of project 

14Alternative methods involving quasi-experimental designs do not involve the use of 

randomized control groups. Instead, such designs rely on the nonrandom selection of a 

comparison group of nonparticipants with characteristics similar to those of study 

participants. 

15Although experimental methods are generally preferred, they may not always be feasible 

or may not necessarily provide the optimal approach for conducting a demonstration 

because of various implementation or ethical considerations. Quasi-experimental methods 

may instead be preferable or may need to be used in conjunction with experimental 

methods to produce the most rigorous results. For instance, quasi-experimental methods 

may be preferred when there is a chance that a randomized approach will not remain 

intact, such as when participants in the control group are likely to receive elements of the 

intervention. 

16OPDR is one of several organizational components under SSA’s Office of Disability and 

Income Security Programs (ODISP), which has responsibility for carrying out DI and SSI 

program functions. Prior to OPDR’s creation in November 2002, responsibility for disability 

demonstration projects was shared by components within ODISP and SSA’s Office of 

Policy. 

Page 9 GAO-05-19 Social Security Disability 
data, or evaluation of project results. On the basis of SSA’s review of 

submitted proposals and bids, the agency may enter into grants, 

cooperative agreements, or contractual arrangements with one or more 

organizations to carry out demonstration projects.17 For example, a single 

demonstration may involve cooperative agreements with states to design 

and implement projects as well as contracts with one or more research 

institutions to provide technical assistance to the states and evaluate 

demonstration results. Project managers in OPDR have the primary 

responsibility for overseeing demonstration projects to ensure that they 

meet SSA’s technical and programmatic requirements. OPDR collaborates 

with SSA’s Office of Acquisition and Grants in issuing formal project 

notices and solicitations and, subsequently, in overseeing grant or contract 

performance. 

SSA Has Not Used Its 

Demonstration 

Authority to Evaluate 

a Wide Range of DI 

Return-to-Work Policy 

Issues 

SSA has not used its demonstration authority to extensively evaluate a 

wide range of DI policy areas dealing with return to work. Until very 

recently, SSA has focused its demonstration efforts primarily on a 

relatively narrow set of policy issues dealing with the provision of 

vocational rehabilitation and employment services, despite being given the 

authority to assess a much broader range of policy alternatives. Even in 

the area of vocational rehabilitation and employment issues, SSA’s use of 

DI demonstration authority has not been comprehensive and, therefore, 

did not extensively address key policy issues that the agency is currently 

grappling with under its Ticket to Work program. SSA’s recently initiated 

or proposed demonstrations have begun to address a broader range of 

policy issues. However, the agency has no systematic processes or 

mechanisms for ensuring that it is adequately identifying and prioritizing 

those issues that could best be addressed through use of its demonstration 

authority. 

SSA Has Not Used Its 

Demonstration Authority 

Extensively 

The DI demonstration projects that SSA has conducted since 1980 have 

not extensively addressed a wide range of return-to-work policy issues. 

Since first being granted DI demonstration authority 24 years ago, SSA has 

used this authority to complete four projects, with another project nearing 

17While grants and cooperative agreements are closely related to each other, cooperative 

agreements generally allow for greater federal agency involvement in the funded activities. 

Contracts generally provide federal agencies with more control over funded activities than 

either grants or cooperative agreements. 

Page 10 GAO-05-19 Social Security Disability 
completion.18 Total costs for these projects amount to at least $107 million, 

of which about $42 million was paid for from the Old-Age and Survivors 

Insurance and Disability Insurance (OASDI) Trust Funds.19 

The legislation granting DI demonstration authority to SSA provided the 

agency with an opportunity to examine a broad set of return-to-work 

policy alternatives and even identified some specific alternatives for SSA 

to consider studying, including (1) reducing, rather than terminating, 

benefits based on earnings; (2) lengthening the trial work period; 

(3) decreasing the 24-month waiting period for Medicare benefits; 

(4) altering program administration; (5) earlier referral of beneficiaries for 

rehabilitation; and (6) using employers and others to stimulate new forms 

of vocational rehabilitation. The projects SSA has conducted thus far have 

focused predominantly on the latter category of issues involving 

vocational rehabilitation and have focused to a lesser extent—or not at 

all—on other key policy issues affecting return to work (see table 1). More 

specifically, examination of policy alternatives dealing with the provision 

of vocational rehabilitation and employment services has been the primary 

objective of four of the five completed or nearly completed 

demonstrations. Although two of these projects also examined other DI 

return-to-work policy issues—such as the possible effects of changes in 

program work incentives and alterations in the provision of medical 

benefits—they did so to only a limited extent. None of the projects looked 

at other potentially significant DI policy issues, such as the possibility of 

changing SSA’s benefit structure to allow for a reduction in benefits, 

rather than a complete cutoff of benefits, based on earnings. 

18SSA also initiated three other projects under its DI demonstration authority, which were 

subsequently terminated. In addition, SSA has sometimes conducted demonstration 

projects involving DI beneficiaries under other research authorities (rather than under its 

DI demonstration authority). Although our discussion in this report focuses only on those 

projects that were conducted under SSA’s DI demonstration authority, our review of these 

other demonstration projects suggests that our findings would apply similarly to them. 

19We were not able to obtain complete cost figures for these projects because of the limited 

information maintained by SSA. Because most of these projects involved both DI and SSI 

beneficiaries, they were funded through the use of OASDI Trust Funds (under SSA’s DI 

demonstration authority) and general appropriations (as provided under SSA’s SSI 

demonstration authority). 

Page 11 GAO-05-19 Social Security Disability 
Table 1: Completed or Nearly Completed Projects Conducted under SSA’s DI Demonstration Authority 

Project Policy issues studied Year initiated Year completed Cost 

Research Demonstration Mostly focused on provision of 1987 

Program (RDP) 	 vocational rehabilitation and 

employment services but also 

touched on other policy issues 

such as changes in SSA work 

incentives 

Pain Assessment Focused on the testing of pain 1990


Instruments assessment questionnaires for 


Development Project determining the impact of pain 


Most projects under the $32 million, about half of 

RDP appear to have which was funded through 

been completed by the the Trust Funds, with the 

early 1990s. other half coming from 

appropriations 

1994 	 A $3.8 million contract was 

awarded by SSA on this 

project, but it is unclear if 

there were any additional 

project costs. All of the costs 

of this project appear to 

have been funded through 

the Trust Funds. 

on an applicant’s or 

beneficiary’s ability to return to 

work 

Project Network Focused on provision of 1991 1999 $25.4 million, about $13.7 

vocational rehabilitation and million of it funded through 

employment services the Trust Funds, with the 

remainder coming from 

appropriations 

Project Referral System Focused on provision of 1997 No identifiable No information available 

for Vocational vocational rehabilitation completion date; project 

Rehabilitation Providers was eventually 

(Project RSVP) superseded by the 

Ticket to Work program 

State Partnership Mostly focused on provision of 1998 Expected completion in Projected to cost $46.1 

Initiative (SPI) vocational rehabilitation and 2006 million, about $8.7 million of 

employment services but also it funded through the Trust 

touched on other policy issues Funds and $37.4 million 

such as increased availability of coming from appropriations 

health insurance 

Source: GAO analysis of SSA data. 

Note: Most of these projects involved both DI and SSI beneficiaries and therefore were jointly 

conducted and funded under SSA’s DI demonstration authority and its SSI demonstration authority. 

Furthermore, SSA has not used its DI demonstration authority to 

comprehensively examine issues involving vocational rehabilitation, 

including key policy issues with which the agency is currently grappling. 

For example, SSA did not extensively test key elements of what eventually 

became the Ticket to Work program.20 Although the ticket program was 

20Under this program, beneficiaries are issued a “ticket,” or voucher, which they can use to 

obtain vocational rehabilitation, employment, or other return-to-work services from an 

approved public or private provider of their choice. 

Page 12 GAO-05-19 Social Security Disability 
not formally proposed by SSA in a legislative package until 1997, as early 

as 1989, in an annual report to the Congress on SSA’s demonstration 

activities, SSA noted that among its ideas for improving SSA’s ability to 

assist beneficiaries in returning to work was a voucher program that could 

be used to pay for vocational rehabilitation services from private 

providers. SSA told the Congress that such a program, as well as other 

possible policy changes, would need to be thoroughly tested as a 

prerequisite to developing a new nationwide program. However, only one 

project completed under SSA’s DI demonstration authority—Project 

Referral System for Vocational Rehabilitation Providers (Project RSVP), 

initiated in 1997—addressed an issue directly relevant to the ticket 

program, namely, the use of a contractor to perform certain administrative 

functions for an expanded vocational rehabilitation referral and 

reimbursement program. But our review of project documentation and our 

discussions with SSA officials indicate that Project RSVP was more of an 

effort to make an operational change in the way SSA managed its 

vocational rehabilitation program than a study to evaluate the advantages 

and disadvantages of such a change. In fact, we could not identify any end 

product or final results for this project. 

SSA also made another attempt, ultimately unsuccessful, to directly 

address issues related to establishment of a ticket program. In the 

Omnibus Budget Reconciliation Act of 1990, the Congress mandated that 

SSA use its DI demonstration authority to assess the advantages and 

disadvantages of permitting DI beneficiaries to select from among both 

public and private vocational rehabilitation providers. But in January 

1993, SSA reported to the Congress that it would be unable to conduct this 

demonstration because of an insufficient number of providers willing to 

participate in the project. SSA explained that the performance-based 

reimbursement provisions of the proposed project appeared to be the 

reason why providers were reluctant to participate.21 Despite the Congress’ 

expressed interest in these issues, SSA did not attempt to identify 

alternative ways to carry out such a demonstration. In particular, given 

that SSA remained very interested in the expanded use of private 

rehabilitation providers for the DI program, the difficulties encountered in 

recruiting providers for the demonstration should have highlighted the 

need for SSA to further study the issue of provider reimbursement before 

21These provisions permitted reimbursement only for rehabilitation services which result in 

a beneficiary’s performance of substantial gainful activity for a continuous period of 9 

months. The provisions were identical to those in place for reimbursing state vocational 

rehabilitation agencies. 

Page 13 GAO-05-19 Social Security Disability 
proceeding with any policy initiatives in this area. SSA’s current Deputy 

Commissioner for Disability and Income Security Programs told us that if 

SSA had used its demonstration authority to study these types of issues in 

the 1990s, SSA might have been able to identify and possibly resolve these 

issues then rather than struggling to do so now. In addition, such 

information could have been helpful in the Congress’ consideration of the 

ticket legislation’s merits as it deliberated whether to enact this program. 

SSA’s Recently Proposed 

Demonstrations Address a 

Broader Range of Policy 

Issues 

In contrast to the completed and nearly completed demonstration 

projects, SSA’s more recent projects, which are generally in the early 

planning or proposal stages, represent a much more wide-ranging set of 

demonstrations (see table 2). For example, the projects, as currently 

described, will deal with a variety of issues such as early provision of cash 

and medical benefits and a change in the benefit payment structure to 

allow a benefit offset for beneficiaries earning above the SGA level. This 

more comprehensive approach to demonstrations is due in part to 

legislative changes. For example, the Ticket to Work Act mandated that 

SSA conduct a benefit offset demonstration and also permitted SSA, for 

the first time, to conduct demonstrations involving DI applicants, thereby 

allowing SSA to test ideas such as early provision of cash and medical 

benefits and vocational rehabilitation services to individuals who have not 

yet entered the disability rolls. In addition, SSA has recently placed a high 

priority on conducting disability demonstration projects that examine the 

key issues affecting beneficiaries’ return to work. This priority was 

reflected in the SSA Commissioner’s September 25, 2003, testimony before 

the House Committee on Ways and Means, Subcommittee on Social 

Security, in which she announced several new demonstrations as part of a 

broader strategy to improve the DI and SSI programs. SSA estimates that 

these recently proposed and initiated projects will cost about $357 million, 

$293 million of which will be paid for from the OASDI Trust Funds. 

Page 14 GAO-05-19 Social Security Disability 
Table 2: Recently Proposed or Initiated Projects Conducted under SSA’s DI Demonstration Authority 

Actual/planned Expected project 

Project Policy issues studied year of initiation duration Expected cost 

DI Benefit Offset 	 Focused on assessment of a 

benefit offset that would allow 

beneficiaries to retain their 

eligibility for benefits when 

earning above the SGA level 

while offsetting these benefits 

by $1 for every $2 in earnings; 

it would also assess various 

employment support 

interventions in conjunction 

with the offset 

2004 6 years 	 $106 million, all from the 

Trust Funds 

Early Intervention Focused on provision of cash 2005 Pilot is expected to $34.4 million, of which 

and medical benefits and last 4 years; duration $28.4 million is expected to 

employment supports to DI of full demonstration be funded through the 

applicants (if conducted) is Trust Funds 

unknown 

Interim Medical Benefits Focused on provision of Project in 6 years $59.8 million, of which 

medical benefits to DI preliminary stage; $54.9 million is expected to 

applicants with no medical too early to be funded through the 

insurance whose condition is determine start Trust Funds 

likely to improve with 

treatment 

date (These costs apply to both 

the Interim and Ongoing 

Medical Benefits 

demonstrations) 

Ongoing Medical Benefits 	 Focused on provision of Project in Project in preliminary See above 

medical benefits to DI preliminary stage; stage; too early to 

beneficiaries who want to too early to determine duration 

work but have no affordable determine start 

access to health insurance date, although 

SSA has proposed 

starting a pilot 

project in 2005 

Mental Health Treatment Focused on provision of Project in 6 years $66.9 million, of which 

Study outpatient mental health preliminary stage; $59.7 million is expected to 

treatment and vocational too early to be funded through the 

rehabilitation services to DI determine start Trust Funds 

beneficiaries for whom a date 

mental health disorder is the 

primary diagnosis 

Temporary Allowance Focused on provision of Project in 5 years $32 million, all from the 

immediate cash and medical preliminary stage; Trust Funds 

benefits to DI applicants likely too early to 

to benefit from aggressive determine start 

medical care date 

Page 15 GAO-05-19 Social Security Disability 
 Actual/planned Expected project 

Project Policy issues studied year of initiation duration Expected cost 

a

Predictive Modeling	 Focused on analyses to 

develop screening tools that 

will be used to identify 

candidates for several of 

SSA’s other demonstration 

projects 

2004 2 years 	 $9 million, of which $7.2 

million is expected to be 

funded through the Trust 

Funds 

Youth Transition Process Focused on states’ 2003 6 years $48.5 million, of which $4.9 b

Demonstration development of integrated 

service delivery systems to 

improve employment 

outcomes for youth 

transitioning to adulthood 

million is expected to be 

funded through the Trust 

Funds 

Source: GAO analysis of SSA data. 

Note: Some of these projects are expected to involve both DI and SSI beneficiaries and therefore will 

be jointly conducted and funded under SSA’s DI demonstration authority and its SSI demonstration 

authority. 

a

This is not a demonstration project designed to assess policy alternatives. Instead, this project is 

intended to develop a specific product for use in certain other SSA demonstrations. 

b

This project is focused on the SSI program. However, SSI beneficiaries who are dually eligible for DI 

benefits may also participate in this demonstration. 

SSA Lacks a Formal 

Process for Establishing a 

Demonstration Research 

Agenda 

Despite SSA’s recent broadening of the scope of its projects, the agency 

does not have in place any systematic processes for identifying and 

assessing potential issues that could be well suited for study under SSA’s 

demonstration project authority. Therefore, there is no assurance that the 

agency will, in future demonstration efforts, maintain its current focus on 

a broad array of return-to-work policy issues. Our discussions with SSA 

officials and review of a study examining earlier demonstration efforts 

indicate that the agency’s agenda for demonstration projects is subject to 

significant change over time resulting, in part, from changes in executive 

branch and SSA leadership and senior management. The effects of such 

changes may include termination of projects or significant delays and 

modifications in their planning and implementation. For example, in its 

1994 report examining SSA’s Research Demonstration Program, the 

agency’s Inspector General noted that changes in SSA leadership had 

disrupted the accomplishment of RDP objectives. The disability research 

and advisory officials we spoke with also indicated that SSA’s project 

priorities and decisions are significantly influenced by larger political and 

organizational changes, which may prevent SSA from focusing on longterm research objectives. One advisory official noted that these difficulties 

in long-term planning have occurred despite the fact that the Congress—in 

making SSA an independent agency and establishing a 6-year term for the 

Page 16 GAO-05-19 Social Security Disability 
SSA Commissioner—intended that SSA would be better able to engage in 

the type of long-range planning required to address its program needs. 

SSA’s approach for identifying and prioritizing demonstrations has varied 

through the years. Soon after being granted DI demonstration authority in 

1980, SSA developed a detailed demonstration research plan to directly 

address the policy issues identified in SSA’s authorizing legislation. 

However, our discussions with SSA officials and review of internal agency 

documents indicate that the plan was never acted upon because of 

competing organizational priorities and concerns over the potential cost of 

the demonstrations and possible technical limitations, such as the 

adequacy of systems support. Consequently, as its DI demonstration 

authority was due to expire in 1985, SSA had not used it to conduct any 

demonstrations. In the second half of the 1980s, after its demonstration 

authority was renewed, SSA changed course. Partly on the basis of 

solicitation of ideas from the public, SSA identified priority areas dealing 

mostly with vocational rehabilitation and employment services issues for 

which it would issue grants to public and private organizations to conduct 

demonstrations. The specific priority areas identified changed from year to 

year as SSA attempted to stimulate, test, and coordinate effective 

approaches toward employment assistance. 

In its required 1991 annual report to the Congress on its DI demonstration 

activities, SSA said that it was proceeding with broader testing of key 

elements of a comprehensive employment and rehabilitation system. But 

our review of agency documents and discussions with SSA officials 

indicate that SSA has not developed a formal, comprehensive, long-term 

agenda for conducting DI demonstration projects. Senior SSA officials told 

us that the agency’s current demonstration project decisions are, to some 

extent, based on discussions with outside research, advocacy, and other 

groups. But SSA has no formal mechanisms and requirements in place to 

ensure that the agency obtains such input and to decide how such input 

should be factored in with other considerations in determining the 

agency’s demonstration priorities. 

The need for explicit planning concerning SSA research, including 

demonstrations, has been identified in past reviews of SSA’s disability 

programs. For example, in 1998, the Social Security Advisory Board 

(SSAB) noted the need for SSA to develop a comprehensive, long-range 

research and program evaluation plan for DI and SSI that would guide the 

Page 17 GAO-05-19 Social Security Disability 
agency’s research and define priorities.22 SSAB also said that SSA’s 

research plan should reflect broad consultation with the Congress, other 

agencies, SSAB, and others and recommended the establishment of a 

permanent research advisory panel to advise in the development of a longrange plan. In a 1996 report on SSA’s disability programs, the National 

Academy of Social Insurance noted the “dearth of rigorous research on the 

disability benefit programs” since the 1980s and said that SSA needed a 

comprehensive, long-range research program to address this deficiency.23 

In addition, officials from disability research, advisory, and advocacy 

groups told us that they believe the establishment of a formal research 

agenda or an advisory panel with regard to demonstration projects would 

be helpful in ensuring that SSA adequately identifies its demonstration 

priorities and maintains its commitment to these priorities even in the face 

of political or administrative changes. 

SSA’s demonstration projects have had little influence on the agency’s and 

the Congress’ consideration of DI policy issues. This is due, in part, to 

methodological limitations that have prevented SSA from producing 

project results that are useful for reliably assessing DI policy alternatives. 

In addition, SSA lacks a formal process for fully considering the potential 

policy implications of its demonstration results. Furthermore, SSA’s 

reports on demonstration projects have not fully apprised the Congress of 

project results and their policy implications. 

SSA’s Demonstration 

Projects Have Had 

Little Influence on 

Consideration of DI 

Policy Changes 

Limitations in Project The demonstration projects SSA has conducted under its DI 


Design, Implementation, demonstration authority have generally not been designed, implemented, 


and Evaluation Lessen or evaluated in a rigorous enough manner to allow the agency to reliably 


Usefulness of assess the advantages and disadvantages of specific policy alternatives. 


While SSA’s major DI demonstrations have varied significantly in their Demonstration Results methodological rigor, all of them have experienced at least some 

22Social Security Advisory Board, Strengthening Social Security Research: The 

Responsibilities of the Social Security Administration, p. 6-7 (Washington, D.C.: January 

1998). 

23National Academy of Social Insurance, The Environment of Disability Income Policy: 

Programs, People, History, and Context, p. 108-109 (Washington, D.C.: 1996). 

Page 18 GAO-05-19 Social Security Disability 
significant methodological limitations.24 For example, SSA’s first major DI 

demonstration, the Research Demonstration Program, was characterized 

by a number of fundamental design and evaluation flaws such as the 

limited scope and small sample sizes of the RDP projects and the limited 

use of control groups. In its 1994 report on the RDP, the Department of 

Health and Human Services’ (HHS) Inspector General noted that because 

of such limitations, “grantees were unable to conduct research that SSA 

deemed necessary for definitive tests of alternatives to help beneficiaries 

obtain work.”25 In addition, SSA did not develop a plan for evaluating the 

overall RDP results as part of its initial project design. In a required 

1994 annual report to the Congress on its demonstration activities, SSA 

acknowledged that the lack of a rigorous project design and the omission 

of a strong evaluation component limited the ways in which the project 

results could be generalized. But SSA also described a number of 

“observations” that resulted from the RDP and noted that this project had 

helped to identify the agency’s future demonstration priorities. However, 

given the significant limitations of the RDP, it is unlikely that its results 

could have provided a reliable basis for effectively establishing such 

priorities. 

In its next major DI demonstration effort, Project Network, which was 

initiated as the RDP projects were being completed, SSA avoided many of 

the major shortcomings of the RDP. For example, Project Network was 

rigorously designed, using an experimental approach based on the random 

assignment of beneficiaries to treatment and control groups. As a result, 

this project produced some reasonably clear results, which SSA 

thoroughly evaluated in an effort to assess the overall impact of the tested 

policy alternatives. Despite its generally rigorous design, Project Network 

24Our discussion in this section is based on our review of SSA’s three major DI 

demonstrations conducted since 1980—the Research Demonstration Program, Project 

Network, and the State Partnership Initiative. For the two other projects SSA has 

conducted under its DI demonstration authority—the Pain Assessment Instruments 

Development Project and Project RSVP—SSA was unable to provide, and we were unable 

to otherwise identify, any documentation or information other than relatively brief project 

descriptions provided in SSA reports or notices. However, these project descriptions as 

well as information obtained in discussions with SSA officials who were knowledgeable of 

these projects indicate that these projects were not research efforts designed to assess 

particular return-to-work policy options but instead represented attempts by SSA to 

implement operational changes in the DI program. 

25Department of Health and Human Services, Office of Inspector General, Impact of SSA’s 

Disability Research Demonstration Program, OEI-04-91-01660, pp. 5 (Washington, D.C.: 

September 1994). 

Page 19 GAO-05-19 Social Security Disability 
also had some limitations that may have, to some extent, limited its 

usefulness for policy consideration. For example, in examining the effects 

of a case management approach for providing vocational rehabilitation 

services, Project Network used four different service delivery models.26 

Although the Project Network evaluation provided information on the 

overall effects of a case management approach, it did not provide a basis 

for reliably assessing and comparing the separate effects of the four 

models even though such an assessment may have provided useful 

information for policy considerations. In addition, Project Network did not 

produce results that could be generalized to the larger population of 

beneficiaries, which, in turn, limited SSA’s ability to assess whether the 

tested policy should be implemented on a nationwide basis.27 

As was the case with Project Network, SSA has made a significant effort 

under its State Partnership Initiative demonstration to avoid some of the 

problems encountered under the RDP. For example, SSA contracted with 

two research institutions to design an evaluation plan for the 

demonstration and to provide assistance with technical issues and data 

collection to the various states conducting this demonstration. Our 

discussions with SSA and contractor officials who have been involved in 

this demonstration as well as our own review of SPI project documents 

indicate that the efforts of the contractors appear to have introduced a 

certain degree of rigor in the design, implementation, and, potentially, 

evaluation of this demonstration.28 For example, SSA’s contractors have 

indicated that the SPI “core evaluation” will likely produce useful results 

regarding the effects on beneficiary employment of the overall package of 

policy alternatives tested under the demonstration. But despite these 

efforts, the SPI design also has a number of limitations that could 

substantially reduce the usefulness of its results for evaluating the effects 

26The researchers involved in Project Network described case management as “a tool for 

facilitating employment-oriented interventions customized for each individual, given that 

persons with disabilities face a range of barriers to work.” The four models for providing 

case management services used in Project Network included one that was operated by a 

private contractor and another operated by state vocational rehabilitation agencies, as well 

as two other models that were operated by SSA but differed in terms of the intensity of 

services offered. 

27Because participation in the demonstration was voluntary, the project’s results could not 

be generalized to the broader population of beneficiaries, most of whom did not volunteer. 

For example, the project’s volunteers were generally healthier and had fewer work 

limitations than those who did not volunteer. 

28The SPI demonstration is still in process and is expected to be completed in 2006. 

Page 20 GAO-05-19 Social Security Disability 
of the demonstration’s individual policy alternatives. For example, SSA 

gave each of the 12 participating states significant discretion in designing 

and conducting projects, which resulted in 12 distinct state projects. Each 

project tested different combinations of policy alternatives, applied 

different research methods to study these alternatives, and used varying 

approaches to select beneficiaries for participation in the project. SSA 

officials told us that such differences across projects make it unlikely that 

SPI will produce final results that allow for reliable evaluations of specific 

policy alternatives on a national level. SSA and one of its SPI contractors 

have also noted other potential limitations in the design and 

implementation of SPI, such as problems with the quality of states’ data 

collection, that may detract from SSA’s ability to evaluate specific policy 

alternatives.29 

SSA officials currently responsible for planning and conducting DI 

demonstrations acknowledged that the agency’s past demonstrations have 

generally not provided useful information for policy making largely 

because of the limited rigor with which these projects were conducted. 

However, they emphasized that the agency has, over the past couple of 

years, placed a new emphasis on ensuring that DI demonstrations are 

rigorously designed so that the results can be used to effectively evaluate 

specific policy options and develop recommendations. In particular, the 

officials noted the importance of using, whenever feasible, an 

experimental approach in its demonstration projects and of ensuring that 

demonstration results can be generalized to the larger population of DI 

beneficiaries.30 The officials also emphasized the need for SSA to hire 

additional staff with the expertise needed to carry out methodologically 

rigorous demonstration projects. 

29Because the SPI demonstration is not yet complete, it is possible that SSA and its 

contractors will take additional steps to address these limitations. However, information 

obtained through our discussions with SSA and SPI contractor officials provided little 

indication that such problems were likely to be addressed. 

30The officials noted that being able to generalize to the larger DI population does not 

necessarily mean that a policy tested through a demonstration has to be applicable to all or 

most DI beneficiaries, but could instead be applicable to a subgroup of beneficiaries. For 

example, an SSA demonstration might evaluate the costs and benefits of a policy option 

that is targeted toward beneficiaries with mental impairments. In this case, SSA would 

want to be assured that the project’s results are generalizable to the larger population of 

beneficiaries with mental illnesses, but it would not expect generalizability to other DI 

beneficiaries (i.e., those with other types of impairments). 

Page 21 GAO-05-19 Social Security Disability 
Aside from the SPI demonstration, all of SSA’s other current DI 

demonstrations are in the early design phase or have been proposed only 

recently. Therefore, we were not able to assess the methodological rigor of 

these projects. However, our review of SSA’s request for proposal (RFP) 

for its Benefit Offset demonstration indicates that SSA is making a serious 

effort to comprehensively and rigorously study this policy issue. For 

example, SSA has proposed using an experimental design with random 

assignment to treatment and control groups. Nevertheless, the scope and 

complexity of SSA’s proposal suggest that this will be a very challenging 

project for SSA to carry out successfully, and that the agency will need to 

ensure that its project design avoids some of the pitfalls that have limited 

the usefulness of past demonstrations, such as insufficient sample size and 

lack of uniformity in tested interventions across sites. 

SSA Does Not Ensure That 

Project Results Are 

Adequately Considered 

and Communicated 

SSA does not have procedures or processes in place to ensure that project 

results—regardless of any limitations that they may have—are fully 

considered by senior officials within the agency for their policy 

implications or their implications for future SSA research and 

demonstrations. Without such processes, projects that begin with the 

support of senior managers under one administration may not receive 

adequate attention from a new group of senior managers under a future 

administration. Our discussions with current and former SSA officials and 

with officials from disability research, advocacy, and advisory 

organizations indicate that such shifting priorities have been the norm for 

SSA’s DI demonstration projects. For example, several of these officials 

told us that when Project Network was completed in 1999, its results were 

not formally reviewed and considered by senior SSA managers, in part 

because of the changes in presidential administrations and in senior 

agency leadership that had occurred since the start of the project. Officials 

from one of the groups we spoke with told us that SSA’s consideration of 

project results could be improved by the establishment of a panel to 

review project results and explore their policy implications. 

An additional factor that could limit SSA’s consideration of demonstration 

results is the lack of an adequate historical record—reflecting the 

outcomes and the problems or issues encountered—of the various 

projects that the agency has conducted under its demonstration 

Page 22 GAO-05-19 Social Security Disability 
authority.31 SSA has not maintained a formal record of its disability 

demonstration project activities and results, so basic information on these 

projects—such as project notices, design documents, and evaluation 

documents—is in some cases no longer available. As a result, information 

on some projects can be obtained only by relying on the recollection of 

SSA employees who were around when the study was conducted. While 

formal document retention requirements may not dictate that SSA 

maintain such information, several SSA officials told us that the agency 

would benefit from an institutional record of demonstration activity. 

According to these officials, such a record would constitute a body of 

knowledge that the agency should be building to improve DI return-towork policies. This becomes even more important in light of the expected 

retirement of a large percentage of SSA staff during this decade.32 

In addition to having shortcomings in its consideration of demonstration 

results, SSA has not sufficiently communicated the status and results of its 

demonstration projects to the Congress. Although SSA had been required 

to issue various reports to the Congress regarding its DI demonstration 

projects, it has not always produced such reports. For example, although 

SSA was required to submit final reports on the use of its demonstration 

authority in 1985, 1990, 1993, and 1996, the only final report that SSA 

submitted was in 1996. In addition, SSA did not submit annual reports on 

its demonstration activities in 7 of the 16 years in which these reports 

were required. Furthermore, when these reports have been produced, they 

have not provided all of the information needed to fully inform the 

Congress of demonstration activities and results. For example, our review 

of these reports indicates that they have frequently lacked key information 

such as a discussion of a project’s potential policy implications, its 

limitations, and the costs of conducting the project. 

Conclusions 	 In allowing SSA to waive program provisions and use OASDI Trust Fund 

dollars, SSA’s DI demonstration authority provides the agency with a 

special, and potentially very valuable, means of studying policy 

31Although our focus in this report is on DI demonstrations, during the course of our work, 

we had also requested information on SSI demonstrations and were similarly unable to 

identify a historical record at SSA. 

32In its strategic plan for fiscal years 2003–2008, SSA said that during this decade, over 

28,000 of its federal employees will retire and another 10,000 will leave the agency for other 

reasons, which represents approximately 59 percent of the agency’s current workforce. 

Page 23 GAO-05-19 Social Security Disability 
alternatives to improve the agency’s return-to-work programs. SSA has 

spent tens of millions of dollars from the OASDI Trust Funds to conduct 

these projects—in addition to tens of millions of dollars from SSA’s 

general appropriations—and expects to spend hundreds of millions more 

within the next 10 years. While these amounts may be small as a 

percentage of the total Trust Funds, they nevertheless represent a 

substantial use of increasingly limited federal resources. After having this 

authority for more than two decades, SSA has yet to use it to propose or 

assess major policy options that could result in savings to the Trust Funds. 

Because SSA’s use of its DI demonstration authority has yet to achieve the 

Congress’ intended results—and because SSA is permitted to draw on 

increasingly limited Trust Funds to conduct these demonstrations—we 

believe it is important for the Congress to maintain close oversight of 

SSA’s use of this authority. We also believe that such oversight would be a 

greater challenge if the Congress were to grant this demonstration 

authority on a permanent basis. 

As the DI Trust Fund approaches exhaustion, the need for programmatic 

improvements becomes greater and greater. As part of a broader effort to 

address this need, SSA has recently initiated or proposed a number of DI 

demonstration projects that, according to SSA officials, are geared toward 

producing useful and methodologically sound results. Such results could 

provide an important basis for SSA to address some of the long-standing 

issues that have led GAO to identify federal disability programs as a highrisk area. However, the challenges SSA has historically faced in 

conducting demonstration projects and the potential for changing 

priorities to adversely affect long-range research plans suggests that, in the 

long run, SSA may be unable to fulfill these demonstration goals. This is 

especially likely if SSA continues its informal approach to prioritizing and 

planning demonstrations and assessing their results. Without more formal 

mechanisms for establishing its commitment to effective and thorough DI 

demonstrations—including the submission of regular reports to the 

Congress on the results and implications of its demonstration projects— 

SSA will be unable to ensure that the extensive amount of time, effort, and 

funding devoted to these demonstrations is well spent. 

Recommendations 	 To help ensure the effectiveness of SSA’s DI demonstration projects, we 

recommend that the Commissioner of Social Security take the following 

actions: 

• 	 Develop a formal agenda reflecting the agency’s long-term plans and 

priorities for conducting DI demonstration projects. In establishing this 

Page 24 GAO-05-19 Social Security Disability 
agenda, SSA should consult broadly with key internal and external 

stakeholders, including SSA advisory groups, disability researchers, 

and the Congress. 

• 	 Establish an expert panel to review and provide regular input on the 

design and implementation of demonstration projects from the early 

stages of a project through its final evaluation. Such a panel should 

include SSA’s key research personnel as well as outside disability 

experts and researchers. SSA should establish guidelines to ensure that 

its project plans and activities adequately address the issues or 

concerns raised by the panel or provide a clear rationale for not 

addressing such issues. 

• 	 Establish formal processes to ensure that, at the conclusion of each 

demonstration project, SSA fully considers and assesses the policy 

implications of its demonstration results and clearly communicates 

SSA’s assessment to the Congress. Such processes should ensure that 

SSA consults sufficiently with internal and external experts in its 

review of demonstration project results and that SSA issues a report to 

the Congress clearly identifying (1) major project outcomes, (2) major 

project limitations, (3) total project costs, (4) any policy options or 

recommendations, (5) expected costs and benefits of proposed options 

or recommendations, and (6) any further research or other actions 

needed to clarify or support the project’s results. Another key aspect of 

such formal processes should be a requirement that SSA maintain a 

comprehensive record of DI demonstration projects. This record would 

help SSA in establishing an empirically based body of knowledge 

regarding possible return-to-work strategies and in deriving the full 

value of its substantial investments in demonstration projects. 

To facilitate close congressional oversight and provide greater assurance 

that SSA will make effective use of its DI demonstration authority, the 

Congress should consider the following actions: 

• 	 Continue to provide DI demonstration authority to SSA on a temporary 

basis but allow SSA to complete all projects that have been initiated 

prior to expiration of this authority. This would provide SSA with 

greater certainty and stability in its efforts to plan and conduct 

demonstration projects while preserving the Congress’ ability to 

periodically reassess and reconsider SSA’s overall use of DI 

demonstration authority. 

• 	 Require that SSA periodically provide a comprehensive report to the 

Congress summarizing the results and policy implications of all of its 

Matters for 

Congressional 

Consideration 

Page 25 GAO-05-19 Social Security Disability 
DI demonstration projects. The due date for this report could either 

coincide with the expiration of SSA’s DI demonstration authority or, if 

this authority is made permanent or extended for a period greater than 

5 years, be set for every 5 years. Such reports could serve as a basis for 

the Congress’ assessment of SSA’s use of its demonstration authority 

and its consideration of whether this authority should be renewed. 

• 	 Establish reporting requirements that more clearly specify what SSA is 

expected to communicate to the Congress in its annual reports on DI 

demonstrations. Among such requirements could be a description of all 

SSA projects that the SSA Commissioner is considering conducting or 

is conducting some preliminary work on. For each demonstration 

project that the agency is planning or conducting, SSA should provide 

clear information on the projects’ specific objectives, potential costs, 

key milestone dates (e.g., actual or expected dates for RFP, award of 

contracts or grants, start of project operations, completion of 

operations, completion of analysis, and final report), potential 

obstacles to project completion, and the types of policy alternatives 

that SSA might consider pursuing depending on the results of the 

demonstration. This would provide the Congress with a more complete 

understanding of the direction and progress of SSA in its efforts to 

fulfill its DI demonstration requirements. 

• 	 More clearly specify the methodological and evaluation requirements 

for DI demonstrations to better ensure that such projects are designed 

in the most rigorous manner possible and that their results are useful 

for answering specific policy questions and for making, where 

appropriate, well-supported policy recommendations. Such 

requirements should not be entirely prescriptive given the need for SSA 

to have sufficient flexibility for choosing the right methodological 

approach based on the specific circumstances and objectives of a 

particular demonstration project. However, the requirements could call 

for SSA to choose, to the extent practical and feasible, the most 

rigorous methods possible in conducting these demonstrations. 

Whatever methods are ultimately selected, SSA should be sure that the 

methods used will allow for a reliable assessment of the potential 

effect on the DI program of the individual policy alternatives being 

studied. Finally, SSA’s legislative requirements could be revised to 

include a more explicit list of project objectives—such as assessments 

of specific employment outcomes, costs and benefits, and Trust Fund 

savings—similar to the language that was included under Sections 

302(b)(1) and (b)(2) of the Ticket to Work and Work Incentives 

Improvement Act. 

Page 26 GAO-05-19 Social Security Disability 
Agency Comments 


and Our Evaluation 


In commenting on a draft of this report, SSA agreed with our 

recommendations. SSA agreed that in the past it has not used its 

demonstration authority to extensively evaluate DI policy but noted that 

its recently initiated or proposed demonstrations will play a vital role in 

testing program and policy changes. SSA also agreed that the use of 

experts in developing demonstration projects is very useful and 

commented that it has used the expertise of particular individuals on an ad 

hoc basis and plans to continue to use the advice and recommendations of 

experts in the development of future demonstrations. Finally, SSA agreed 

that a central source of information regarding the results and policy 

implications of disability demonstrations needs to be established and 

stated that it planned to fully analyze the results of demonstration projects 

to inform DI policy decisions. SSA’s comments appear in appendix II. 

Copies of this report are being sent to the Commissioner of SSA, 

appropriate congressional committees, and other interested parties. The 

report is also available at no charge on GAO’s Web site at 

http://www.gao.gov. If you have any questions about this report, please 

contact me at (202) 512-7215. Other contacts and staff acknowledgments 

are listed in appendix III. 

Robert E. Robertson 

Director, Education, Workforce, 

and Income Security Issues 

Page 27 GAO-05-19 Social Security Disability 
Appendix I: Scope and Methodology 


To address the mandated objectives, we reviewed legislation authorizing 

the Social Security Administration (SSA) to conduct Disability Insurance 

(DI) demonstration projects, Congressional reports related to this 

legislation, and SSA regulations governing DI demonstration activities. We 

also examined internal SSA memorandums and planning documents 

discussing proposals to conduct demonstration projects and the nature, 

purpose, requirements, and distinguishing features of SSA’s demonstration 

authority. We interviewed a wide range of current and former SSA officials 

who have had involvement in or responsibility for conducting disability 

program demonstration projects, including officials from the Office of 

Disability and Income Security Programs (ODISP) and two offices 

operating under ODISP—the Office of Program Development and 

Research and the Office of Employment Support Programs—as well as 

officials from the Office of the Chief Actuary, the Office of Acquisition and 

Grants, the Office of Budget, the Office of Strategic Management, and the 

Office of Research, Evaluation, and Statistics. We also interviewed 

officials from disability research, advisory, and advocacy organizations. In 

addition, we examined other reviews of SSA’s disability demonstration 

and research programs, including prior GAO and Inspector General 

reports and reports from disability research and advisory groups. We also 

reviewed SSA budget documents identifying agency spending on disability 

program demonstrations and SSA testimony describing agency priorities 

related to the DI program in general and demonstration projects in 

particular. In addition, we examined SSA’s strategic plan, annual 

performance plans, and annual accountability reports. 

To obtain detailed information on SSA’s DI demonstration projects, we 

reviewed various documents related to SSA’s design, implementation, and 

evaluation of demonstration projects including agency reports to the 

Congress; public notifications of demonstration projects issued in the 

Federal Register; contract, grant, and cooperative agreement solicitation 

and award notices issued in the Federal Register or in the Commerce 

Business Daily; and project reports submitted to SSA by grantees or 

contractors, including project design and evaluation documents. We used 

information from these sources to identify key characteristics and 

outcomes of each project, including its broad goals, specific study 

objectives, types of program waivers applied, methodology, actual or 

expected costs, funding sources, major project milestones including actual 

or expected initiation and completion dates, project duration, involvement 

of outside contractors and grantees, key project strengths and limitations, 

and final project results, including any recommendations that may have 

been made. The type and extent of information we obtained for each 

demonstration project varied widely, in large part because SSA has not 

Page 28 GAO-05-19 Social Security Disability 
Appendix I: Scope and Methodology 

maintained comprehensive documentation on its prior demonstrations. In 

addition, documentation on SSA’s more recent demonstrations was very 

limited given that these projects are in the early planning and design 

stages. 

To provide a broader context for understanding SSA’s use of its 

demonstration authority, we reviewed other federal agencies’ legislative 

authorities for conducting demonstration and research activities. We also 

examined reports from GAO and other organizations that evaluated 

demonstration and research projects conducted by other federal agencies 

or that identified key evaluation and methodological issues related to such 

projects. 

We performed our work at SSA headquarters in Baltimore, Maryland, and 

at various locations in Washington, D.C. We conducted our work between 

October 2003 and August 2004 in accordance with generally accepted 

government auditing standards. 

Page 29 GAO-05-19 Social Security Disability 
Appendix II: Comments from the Social 

Security Administration 

Page 30 GAO-05-19 Social Security Disability 
Appendix II: Comments from the Social 

Security Administration 

Page 31 GAO-05-19 Social Security Disability 
Appendix II: Comments from the Social 

Security Administration 

Page 32 GAO-05-19 Social Security Disability 
Appendix II: Comments from the Social 

Security Administration 

Page 33 GAO-05-19 Social Security Disability 
Appendix III: GAO Contacts and Staff 

Acknowledgments 

GAO Contacts 

Staff 

Acknowledgments 

(130323) 

Shelia D. Drake, Assistant Director, (202) 512-7172 

Mark Trapani, Analyst-in-Charge, (202) 512-6513 

The following individuals also made important contributions to this report: 

Jacquelyn D. Stewart, Erin M. Godtland, Corinna A. Nicolaou, Daniel A. 

Schwimer, Ronald La Due Lake, Michele C. Fejfar. 

Page 34 GAO-05-19 Social Security Disability 
GAO’s Mission	 The Government Accountability Office, the audit, evaluation and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help 

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

Obtaining Copies of The fastest and easiest way to obtain copies of GAO documents at no cost 

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts GAO Reports and newly released reports, testimony, and correspondence on its Web site. To 

Testimony have GAO e-mail you a list of newly posted products every afternoon, go to 

www.gao.gov and select “Subscribe to Updates.” 

Order by Mail or Phone	 The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548 

To order by Phone:	 Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

To Report Fraud, Contact:


Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm


E-mail: fraudnet@gao.gov Federal Programs Automated answering system: (800) 424-5454 or (202) 512-7470 

Congressional	 Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 Relations Washington, D.C. 20548 

Public Affairs	 Susan Becker, Acting Manager, BeckerS@gao.gov (202) 512-4800 

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548