United States Government Accountability Office

GAO Report to Congressional Requesters

VOCATIONAL 

REHABILITATION 

Improved Information 

and Practices May 

Enhance State Agency 

Earnings Outcomes 

for SSA Beneficiaries 

May 2007 

 

GAO-07-521 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

May 2007

VOCATIONAL REHABILITATION

Improved Information and Practices May 

Enhance State Agency Earnings 

Outcomes for SSA Beneficiaries 

Highlights of GAO-07-521, a report to 

congressional requesters 

www.gao.gov/cgi-bin/getrpt?GAO-07-521.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact Denise Fantone 

at (202) 512-4997 or fantoned@gao.gov. 

State vocational rehabilitation (VR) 

agencies, under the Department of

Education (Education), play a 

crucial role in helping individuals 

with disabilities prepare for and 

obtain employment, including 

individuals receiving disability

benefits from the Social Security 

Administration (SSA). In a prior 

report (GAO-05-865), GAO found 

that state VR agencies varied in the

rates of employment achieved for 

SSA beneficiaries. To help

understand this variation, this 

report analyzed SSA and Education 

data and surveyed state agencies to

determine the extent to which (1) 

agencies varied in earnings

outcomes over time; (2) differences

in state economic conditions, client

demographic traits, and agency

strategies could account for agency

performance; and (3) Education’s 

data could be used to identify 

factors that account for differences 

in individual earnings outcomes. 

What GAO Recommends

GAO recommends that Education 

promote certain promising 

practices identified in our analysis,

reassess the data it collects on 

clients, and consider economic 

factors when measuring state

agency performance. Education 

generally agreed with our

recommendations, but disagreed 

that economic factors should be

incorporated into performance 

measures. It considers these 

factors during monitoring and 

believes its approach to be 

effective. We maintain that these 

factors are critical to measuring 

agencies’ relative performance. 

Our analysis of data on state agency outcomes for SSA beneficiaries 

completing VR found that state agencies varied widely across different 

outcome measures for the years of our review. For example, from 2001 to 

2003 average annual earnings levels among those SSA beneficiaries with 

earnings during the year after completing VR varied across state agencies 

from about $1,500 to nearly $17,000. 

Distribution of State Agency Average Annual Earnings for SSA Beneficiaries during the Year 

after VR 

Percentage of agencies

0

5

10

15

20

25

30

Source: GAO analysis of SSA data.

Average annual earnings from 2001-2003 for SSA beneficiaries with earnings during the year after VR

1,000

2,000

3,000

4,000

5,000

6,000

7,000

8,000

9,000

10,000

11,000

12,000

13,000

14,000

15,000

16,000

17,000

Note: Earnings are in 2004 dollars. 

After controlling for a range of factors, we found that much of the 

differences in state agency earnings outcomes could be explained by state 

economic conditions and the characteristics of the agencies’ clientele. 

Together state unemployment rates and per capita income levels accounted 

for roughly one-third of the differences between state agencies in the 

proportion of SSA beneficiaries that had earnings during the year after VR. 

The demographic profile of SSA clients being served at an agency—such as

the proportion of women beneficiaries—also accounted for some of the 

variation in earnings outcomes. 

We also found that after controlling for other factors, a few agency practices 

appeared to yield positive earnings results. For example, state agencies with 

a higher proportion of state-certified counselors had more SSA beneficiaries 

with earnings during the year after completing VR. 

However, we were unable to determine what factors might account for 

differences in earnings outcomes at the individual level. This was due in part 

to Education’s data, which lacked information on important factors that 

research has linked to work outcomes, such as detailed data on the severity 

of clients’ disabilities. Although Education collects extensive client-level 

data, some key data are self-reported and not always verified by state 

agencies. 
Contents 

Letter 1

Results in Brief 3

Background 6

State VR Agencies Consistently Showed Very Different Rates of 

Success for SSA Beneficiaries Who Completed VR Programs 11 

State Economic Conditions and SSA Beneficiary Characteristics 

Account for Much of the Difference in State VR Agency Success 

Rates 19

A Few Agency Practices Appeared to Yield Better Earnings 

Outcomes, while the Results of Other Practices Were 

Inconclusive 24

Limitations in Education’s Data May Have Hampered Analyses of 

Individual Earnings Outcomes 27 

Conclusions 29

Recommendations for Executive Action 30 

Agency Comments and Our Evaluation 30 

Appendix I Scope and Methodology 33 

Section 1: Data Used, Information Sources, and Data Reliability 33 

Section 2: Study Population and Descriptive Analyses 44 

Section 3: Econometric Analyses 46 

Section 4: Limitations of our Analyses 52 

Appendix II Comments from the Department of Education 55 

Appendix III Comments from the Social Security Administration 64 

Appendix IV GAO Contacts and Staff Acknowledgments 73 

Related GAO Products 74 

Tables 

Table 1: Explanatory Variables from the TRF Subfile 35 

Table 2: Explanatory Variables from Education’s RSA-2 Data 36 

Page i GAO-07-521 Vocational Rehabilitation
Table 3: State Economic and Demographic Explanatory Variables 

and Their Sources 38 

Table 4: Explanatory Variables from the VR Agency Survey Data 40 

Table 5: Dependent Variables Used in the Analyses 45 

Table 6: Coefficients for Multivariate Models Estimating the Effects 

of State and Agency Characteristics on Three VR

Outcomes, and the Proportion of Variance Explained (RSquared) by Each Model 50 

Figures 

Figure 1: Distribution of State VR Agencies by Percentage of SSA 

Beneficiaries with Earnings during the Year after VR 12 

Figure 2: Distribution of State VR Agency Average Annual Earnings 

for SSA Beneficiaries with Earnings during the Year after 

VR 13

Figure 3: Distribution of State VR Agencies by Percentage of SSA 

Beneficiaries Leaving the Rolls 14 

Figure 4: Range across State VR Agencies of the Percentage of SSA 

Beneficiaries with Earnings during the Year after VR by 

Year 15

Figure 5: Range of State VR Agency Average Earnings for SSA 

Beneficiaries by Year 16 

Figure 6: Range across State VR Agencies of the Percentage of SSA 

Beneficiaries with Earnings during the Year after VR by 

Agency Type 17 

Figure 7: Range of State VR Agency Average Earnings for SSA 

Beneficiaries by Agency Type 18 

Figure 8: Range of State VR Agency Average Rates of SSA 

Beneficiaries Leaving the Rolls by Agency Type 19 

Page ii GAO-07-521 Vocational Rehabilitation
Abbreviations 

CPI-U Consumer Price Index for All Urban Consumers 

CSPD Comprehensive System of Personnel Development 

DI Disability Insurance 

GSP gross state product 

IPE individual plan of employment 

MEF Master Earnings File 

OLS ordinary least squares 

SSA Social Security Administration 

SSI Supplemental Security Income 

TRF Ticket Research File 

VR vocational rehabilitation 

WIA Workforce Investment Act 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 

Page iii GAO-07-521 Vocational Rehabilitation
United States Government Accountability Office

Washington, DC 20548 

May 23, 2007

The Honorable Charles B. Rangel 

Chairman 

The Honorable Jim McCrery 

Ranking Minority Member 

Committee on Ways and Means 

House of Representatives 

The Honorable Charles B. Rangel 

Chairman 

The Honorable Jim McCrery 

Ranking Minority Member 

Committee on Ways and Means 

House of Representatives 

The Honorable Michael R. McNulty 

Chairman 

The Honorable Sam Johnson 

Ranking Minority Member 

Subcommittee on Social Security 

Committee on Ways and Means 

House of Representatives 

The Honorable Michael R. McNulty 

Chairman 

The Honorable Sam Johnson 

Ranking Minority Member 

Subcommittee on Social Security 

Committee on Ways and Means 

House of Representatives 

The Honorable Sander M. Levin 

House of Representatives 

State vocational rehabilitation (VR) agencies, under the auspices of the 

Department of Education (Education), play a crucial role in helping 

individuals with disabilities prepare for and obtain employment. In fiscal

year 2005, state VR agencies received $2.6 billion to provide people with 

disabilities a variety of supports such as job counseling and placement, 

diagnosis and treatment of impairments, vocational training, and 

postsecondary education. The VR program serves about 1.2 million people 

each year, and over a quarter of those who complete VR are beneficiaries 

of the Disability Insurance (DI) or Supplemental Security Income (SSI) 

programs administered by the Social Security Administration (SSA). This 

proportion has increased steadily since 2002. As our society ages, the 

number of SSA disability beneficiaries is expected to grow, along with the 

cost of providing SSA disability benefits, and it will be increasingly 

important to manage this growth by optimizing the ability of VR programs 

to help and encourage SSA beneficiaries to participate in the workforce. 

State vocational rehabilitation (VR) agencies, under the auspices of the 

Department of Education (Education), play a crucial role in helping 

individuals with disabilities prepare for and obtain employment. In fiscal

year 2005, state VR agencies received $2.6 billion to provide people with 

disabilities a variety of supports such as job counseling and placement, 

diagnosis and treatment of impairments, vocational training, and 

postsecondary education. The VR program serves about 1.2 million people 

each year, and over a quarter of those who complete VR are beneficiaries 

of the Disability Insurance (DI) or Supplemental Security Income (SSI) 

programs administered by the Social Security Administration (SSA). This 

proportion has increased steadily since 2002. As our society ages, the 

number of SSA disability beneficiaries is expected to grow, along with the 

cost of providing SSA disability benefits, and it will be increasingly 

important to manage this growth by optimizing the ability of VR programs 

to help and encourage SSA beneficiaries to participate in the workforce. 

Page 1 GAO-07-521 Vocational Rehabilitation
In 2005, GAO reported that state VR agencies varied substantially in terms 

of the employment rates they achieved for their clients,1

 particularly for 

SSA beneficiaries who, according to research, attain lower employment 

and earnings outcomes than other VR clients.2

 Depending on the state 

agency, as many as 68 percent and as few as 9 percent of SSA beneficiaries 

exited VR with employment. In addition, GAO found that Education’s 

management of the VR program was lacking in several respects and 

recommended that Education revise its performance measures to account

for economic differences between states, make better use of incentives for 

state VR agencies to meet performance goals, and create a means for 

disseminating best practices among state VR agencies. Education agreed

with these recommendations but has yet to implement them. 

As a follow-up to our 2005 report, you asked us to determine what may 

account for the wide variations in state VR agency outcomes with respect 

to SSA beneficiaries. Therefore, we examined the extent to which (1) 

differences in VR agency outcomes for SSA beneficiaries continued over 

several years and across different outcome measures, (2) differences in VR

agency outcomes were explained by state economies and demographic 

traits of the clientele served, (3) differences in VR agency outcomes were 

explained by specific policies and strategies of the VR agencies, and (4) 

Education’s data allowed for an analysis of factors that account for 

differences in individual-level (as opposed to agency-level) outcomes. 

To perform our work, we used several data sources: (1) a newly available 

longitudinal dataset that includes administrative data from Education and 

SSA on SSA beneficiaries who completed the VR program between 2001 

and 2003, 3 (2) original survey data collected by GAO from 78 of the 80

 1

GAO, Vocational Rehabilitation: Better Measures and Monitoring Could Improve the 

Performance of the VR Program, GAO-05-865 (Washington, D.C.: September 2005). 

2

David C. Stapleton and William A. Erickson, “Characteristics or Incentives: Why Do 

Employment Outcomes for the SSA Beneficiary Clients of VR Agencies Differ, on Average, 

from Those of Other Clients?” (Rehabilitation Research and Training Center for Economic

Research on Employment Policy for Persons with Disabilities, Cornell University, Ithaca, 

New York, Oct. 2004). 

3

The longitudinal dataset from SSA and Education contains information on beneficiaries

for a longer time horizon (i.e., 1998 through 2004). However, we focused on the cohorts 

completing VR between 2001 and 2003 because, at the time of our analysis, data were not 

available on earnings after 2004. Further, we excluded earlier cohort years due to

limitations associated with collecting survey data from VR agencies prior to 2000. See

appendix I for more information on our data. 

Page 2 GAO-07-521 Vocational Rehabilitation
state VR agencies, (3) data from Education on yearly spending information 

by service category for each VR agency, and (4) data from the Census 

Bureau, Bureau of Labor Statistics, and other data sources regarding state 

demographic and economic characteristics. We conducted reliability 

assessments of these data and found them to be sufficiently reliable for 

our analyses. 

We took several steps to analyze these data. To answer our questions, we 

analyzed outcomes by state agency using three different earnings 

outcomes: (1) the percentage of beneficiaries with earnings during the 

year after VR, (2) the average beneficiary’s annual earnings level during

the year after VR, and (3) the percentage of beneficiaries that left the 

disability rolls by the close of 2005.4

 For objective one, we conducted 

descriptive statistical analyses of the data. For objectives two, three, and

four, we conducted econometric analyses that controlled for a variety of 

explanatory factors.5

 We also identified and interviewed academic and 

agency experts in an effort to determine what variables to include in our 

models. As is the case with most statistical analyses, our work was limited 

by certain factors, such as the unavailability of certain information and the 

inability to control for unobservable characteristics and those that are not 

quantifiable. Our results only describe earnings outcomes of SSA 

beneficiaries included in our study and cannot be generalized beyond that 

population. We conducted our review from December 2005 through April 

2007 in accordance with generally accepted government auditing 

standards. See appendix I for a more detailed description of our scope and 

methods. 

When we analyzed state agency outcomes for SSA beneficiaries who 

completed VR between 2001 and 2003, we found that differences in agency 

outcomes continued over several years and across several outcome 

measures—i.e., rates of beneficiaries with earnings, earnings levels, and 

departures from the disability rolls. The proportion of beneficiaries with 

earnings during the year after their completion of the VR program ranged

from as little as 0 percent in one state agency to as high as 75 percent in 

another. Similarly, average annual earnings levels among those SSA 

beneficiaries with earnings varied across state agencies from $1,500 to

Results in Brief 

 4

For the purposes of our study, leaving the rolls is defined as the termination of cash

disability benefits due to work. 

5

We conducted our analyses using multivariate regression analysis. 

Page 3 GAO-07-521 Vocational Rehabilitation
nearly $17,000 in the year following VR. Additionally, the proportion of 

SSA beneficiaries who left the disability rolls varied greatly among 

agencies, with departure rates ranging anywhere from 0 to 20 percent. 

After controlling for certain economic, demographic, and agency factors, 

we found that state economic conditions and the characteristics of 

agencies’ clientele accounted for much of the differences in average

earnings outcomes across state agencies. Specifically, state unemployment 

rates and state per capita income levels accounted for a substantial 

portion—as much as one-third—of the differences between state agencies’

VR outcomes for SSA beneficiaries. For example, significantly fewer SSA 

beneficiaries had earnings during the year after VR in those states with 

higher unemployment rates and lower per capita incomes. Despite the 

significant effect that state economies have on state agency outcomes, 

Education currently does not consider such factors when analyzing state

agency outcomes and assessing their performance. Variations in the 

demographic profile of SSA client populations also accounted for some of 

the differences in earnings outcomes among agencies. For example, state 

VR agencies serving a higher percentage of women beneficiaries had 

significantly fewer SSA clients with earnings during the year after VR. 

We also found, after controlling for the same factors, that a few agency 

practices helped explain differences in state agency outcomes for SSA 

beneficiaries—and some were associated with positive outcomes. For 

example, agencies with a higher proportion of state-certified VR 

counselors—a certification now mandated by Education—had more SSA 

beneficiaries exiting the VR program with earnings. Further, agencies with 

closer ties to the business community also achieved higher average annual 

earnings for SSA beneficiaries and higher rates of departures from the 

disability rolls. Currently, Education promotes ties to the business 

community through an employer network. Our findings also show that 

agencies that received a greater degree of support and cooperation from 

other public programs or that spent a greater proportion of their service 

expenditures on training of VR clients had higher average annual earnings 

for SSA beneficiaries completing VR. 

We were unable to account for differences in individual beneficiary 

outcomes, which might further explain differences in state agency 

outcomes, in part because of limitations in Education’s data. Our 

statistical models were able to explain a greater percentage of the 

differences in earnings outcomes when we analyzed state agency earnings 

outcomes compared to individual earnings outcomes (i.e., as much as 77 

percent compared to 8 percent). With so little variation explained by our 

Page 4 GAO-07-521 Vocational Rehabilitation
analyses of individual-level outcomes, we decided not to report our 

individual-level analyses. Education’s data lack information that we 

believe is critical to assessing earnings outcomes, and this may have

hindered our ability to explain the variation in individual earnings 

outcomes. Specifically, although Education collects extensive client-level

data, it does not systematically collect data that research has linked to 

work outcomes, such as detailed information on the severity of the client’s

disability—data that some state agencies independently collect for 

program purposes. Knowing the severity of a disability can indicate 

whether a person is physically or mentally limited in his or her ability to 

perform work, a fact that may influence the person’s earnings outcomes. 

Further, other key data are self-reported and may not be verified by state 

agencies. 

We are recommending that Education consider the implications of the 

results of our analyses in its management of the VR program. Specifically,

Education should further promote certain agency practices that we found 

show an effect on state agency outcomes and reassess the client-level data

it collects through its state agencies. We also continue to believe that, as 

we recommended in our 2005 report, Education should consider economic 

factors, such as unemployment rates, when evaluating state agency

performance. 

We received written comments on a draft of this report from Education 

and SSA. While Education generally agreed with the substance of our 

recommendations, it disagreed on when economic conditions and state 

demographics should be considered in assessing performance. Instead of

using this information to help set performance measures, the department 

said that it takes these factors into account when it monitors agency 

performance results and believes that its approach is more effective. We 

continue to believe that incorporating this contextual information in 

assessing performance measures is essential to provide the state agencies 

with a more accurate picture of their relative performance. Although 

Education stated that it was open to our recommendation on improving 

data quality, it suggested that validating self-reported information would 

be a potential burden to state agencies and suggested other approaches, 

such as conducting periodic studies. Our recommendation that Education 

explore cost-effective ways to validate self-reported data was based on the 

experience of some VR agencies that have obtained data successfully from 

official sources and not relied solely on self-reported information. 

SSA stated that our report has methodological flaws that introduced 

aggregation bias and false correlations, and suggested that we should have 

Page 5 GAO-07-521 Vocational Rehabilitation
focused on individual-level analysis or reported the results of both 

individual and aggregate-level analysis. We used aggregated data—a 

widely used means of analysis—because our primary objective was to 

understand better the wide variation in outcomes for state VR agencies 

that serve SSA beneficiaries rather than the outcomes for individuals. We 

used appropriate statistical techniques to ensure against bias and false 

correlations. Both Education and SSA provided additional comments, 

which we have addressed or incorporated, as appropriate. Education’s and 

SSA’s comments are reprinted in appendixes II and III respectively, along 

with our detailed responses. 

In 2005, the Social Security Administration provided income support to 

more than 10 million working age people with disabilities. This income 

support is provided in the form of monthly cash benefits under two 

programs administered by the Social Security Administration—the 

Disability Insurance program and the Supplemental Security Income 

program. Some individuals, known as concurrent beneficiaries, qualify for

both programs. The federal government’s cost of providing these benefits 

was almost $101 billion in 2005. 

Background 

Challenges Facing the 

Social Security Disability 

Program 

Over the last decade, the number of disability beneficiaries has increased, 

as has the cost of both the SSI and DI programs. This growth, in part, 

prompted GAO in 2003 to designate modernizing federal disability 

programs as a high-risk area—one that requires attention and 

transformation to ensure that programs function in the most economical, 

efficient, and effective manner possible. GAO’s work found that federal 

disability programs were not well positioned to provide meaningful and 

timely support for Americans with disabilities. For example, despite 

advances in technology and the growing expectations that people with 

disabilities can and want to work, SSA’s disability programs remain 

grounded in an outmoded approach that equates disability with incapacity 

to work. In 1999, GAO testified that even relatively small improvements in 

return-to-work outcomes offer the potential for significant savings in 

program outlays. GAO estimated that if an additional 1 percent of working 

age SSA disability beneficiaries were to leave the disability rolls as a result 

of returning to work, lifetime cash benefits would be reduced by an 

estimated $3 billion. 

Page 6 GAO-07-521 Vocational Rehabilitation
SSA has had a long-standing relationship with Education's VR program, 

whereby SSA may refer beneficiaries to the VR program for assistance in 

achieving employment and economic independence.6 As part of this 

relationship, SSA reimburses VR state agencies for the cost of providing 

services to beneficiaries who meet SSA’s criteria for successful 

rehabilitation (i.e., earnings at the substantial gainful activity level for a 

continuous 9-month period). To further motivate beneficiaries to seek VR 

assistance and expand the network of VR providers, Congress enacted 

legislation in 1999 that created SSA's Ticket to Work (Ticket) Program.7

Under the Ticket program, beneficiaries receive a document, known as a 

ticket, which can be used to obtain VR and employment services from an 

approved provider such as a state VR agency. Thus far, only a small 

fraction of SSA beneficiaries have used the Ticket program to obtain VR 

services. Administered by SSA, this program was intended to (1) increase 

the number of beneficiaries participating in VR by removing disincentives 

to work, and (2) expand the availability of VR services to include private 

VR providers. To date private VR providers have not participated heavily

in the Ticket program, with over 90 percent of SSA beneficiaries 

participating in the Ticket program still receiving services from state VR 

agencies. 

Despite programs such as Ticket, SSA beneficiaries who wish to 

participate in the workforce still face multiple challenges. As we have 

previously reported, some SSA beneficiaries will not be able to return to

work because of the severity of their disability.8

 But those who do return 

to work may face other obstacles that potentially deter or prevent them 

from leaving the disability rolls, such as (1) the need for continued health 

care, (2) lack of access to assistive technologies that could enhance their 

work potential, and (3) transportation difficulties.

 6

Individuals may be referred from SSA to state VR agencies by state disability 

determination services (DDS), which are funded by SSA to render the initial decision on 

whether an individual qualifies for DI or SSI benefits, and thus are in a good position to 

consider whether the individual is an appropriate candidate for VR. 

7

Ticket to Work and Work Incentives Improvement Act of 1999, Pub. L. No. 106-170 (1999). 

The Ticket to Work Program was implemented in three phases, beginning in February 2002. 

Under the Ticket program, VR agencies and other providers can opt for one of two

different reimbursement methods, one based on a successful outcome, the other based on 

successfully reaching milestones. State VR agencies can also continue to be reimbursed 

under the traditional cost reimbursement program if the beneficiary does not utilize his or 

her ticket to obtain services. 

8

 GAO, Social Security: Disability Programs Lag in Promoting Return to Work, 

GAO/HEHS-97-46 (Washington, D.C.: March 1997). 

Page 7 GAO-07-521 Vocational Rehabilitation
The Vocational Rehabilitation Program is the primary federal government

program helping individuals with disabilities to prepare for and obtain 

employment. Authorized by Title I of the Rehabilitation Act of 1973, the VR

program is administered by the Rehabilitation Services Administration, a 

division of the Department of Education, in partnership with the states. 

The Rehabilitation Act contains the general provisions states should 

follow in providing VR services. Each state and territory designates a 

single VR agency to administer the VR program—except where state law

authorizes a separate agency to administer VR services for blind 

individuals. Twenty-four states have two separate agencies, one that 

exclusively serves blind and visually impaired individuals (known as blind 

agencies) and another that serves individuals who are not blind or visually

impaired (known as general agencies). Twenty-six states, the District of 

Columbia, and five territories have a single combined agency that serves 

both blind and visually impaired individuals and individuals with other 

types of impairments (known as combined agencies). In total, there are 80 

state VR agencies. 9

Description of Education’s 

Vocational Rehabilitation 

Program 

Although Education provides the majority of the funding for state VR 

agencies, state agencies have significant latitude in the administration of 

VR programs. Within the framework of legal requirements, state agencies 

have adopted different policies and approaches to achieve earnings 

outcomes for their clients. For example, although all state VR agencies are 

required to have their VR counselors meet Comprehensive System of 

Personnel Development (CSPD) standards, states have the ability to define

the CSPD certification standard for their VR counselors. Specifically, 

under the CSPD states can establish certification standards for VR 

counselors based on the degree standards of the highest licensing, 

certification, or registration requirement in the state, or based on the 

degree standards of the national certification. For example, if an agency

bases its certification standard on the national standard, VR counselors 

are required to have a master’s degree in vocational counseling or another

closely related field, hold a certificate indicating they meet the national

requirement, or take certain graduate-level courses. Regardless of the 

individual state’s definition of the certification standard, research has 

shown that VR agencies are concerned about meeting their needs for statecertified counselors because many experienced VR counselors may retire 

 9

In this report, when we refer to state VR agencies, we are including agencies in the states 

and territories.

Page 8 GAO-07-521 Vocational Rehabilitation
in the coming years, and a limited supply of qualified VR counselors are 

entering the labor market.10

VR agencies also vary in their locations within state government and their 

operations. Some are housed in state departments of labor or education,

while others are free-standing agencies or commissions. Similarly, while 

all VR agencies are partners in the state workforce investment system, as 

mandated in the Workforce Investment Act (WIA) of 1998, VRs vary in the 

degree to which they coordinate with other programs participating in this

system. 11 For example, some VRs have staff colocated at WIA one-stop 

career centers, while others do not. 

By law, each of the 80 VR agencies is required to submit specific 

information to Education regarding individuals that apply for, and are

eligible to receive, VR services. Some of the required information includes 

(1) the types and costs of services the individuals received; (2)

demographic factors, such as impairment type, gender, age, race, and 

ethnicity; and (3) income from work at the time of application to the VR 

program. Education also collects additional information such as (1) the 

weekly earnings and hours worked by employed individuals, (2) public 

support received,12 (3) whether individuals sustained employment for at 

least 90 days after receiving services,13 and (4) summary information on 

agency expenditures in a number of categories from each state VR agency.

Education also monitors the performance of state VR agencies, and since

2000, Education has used two standards for evaluating their performance. 

 10Tsze Chan, Recruiting and Retaining Professional Staff in State VR Agencies: Some 

Preliminary Findings from the RSA Evaluation Study, a special report prepared at the

request of the Department of Education, October 2003. 

11WIA requires states and localities to bring together a number of federally funded 

employment and training services into a single system—the one-stop system. Funded 

through different federal agencies, these programs are to provide services through a

statewide network of one-stop career centers to adults, dislocated workers, and youth. 

12Public support refers to cash payments made by federal, state, or local governments for 

any reason, including an individual’s disability, age, economic, retirement, and survivor

status. This excludes any noncash support payments such as Medicaid, Medicare, food 

stamps, and rental subsidies. 

13Education tracks individuals in terms of seven types of case closures, which can be

collapsed into four categories for individuals who (1) exited without employment, during 

the application phase; (2) exited without employment, with limited services; (3) exited 

without employment, after receiving services under an employment plan; and (4) exited 

with at least 90 days of employment, after receiving services under an employment plan. 

Page 9 GAO-07-521 Vocational Rehabilitation
One assesses the agencies’ performance in assisting individuals in 

obtaining, maintaining, or regaining high-quality employment. The second 

assesses the agencies’ performance in ensuring that individuals from 

minority backgrounds have equal access to VR services. Education also 

publishes performance indicators that establish what constitutes minimum 

compliance with these performance standards. Six performance indicators 

were published for the employment standard, and one was published for 

the minority service standard. To have passing performance, state VR 

agencies must meet or exceed performance targets in four of the six 

categories for the first standard, and meet or exceed the performance 

target for the second standard. 

In 2005, GAO reported that Education could improve performance of this

decentralized program through better performance measures and 

monitoring.14 Specifically, we recommended that Education account for 

additional factors such as the economies and demographics of the states’

populations in its performance measures, or its performance targets, for 

individual state VR agencies to address these issues. We also noted that 

whatever system of performance measures Education chooses to use, 

without consequences or incentives to meet performance standards, state 

VR agencies will have little reason to achieve the targets Education has set

for them. We recommended that Education consider developing new 

consequences for failure to meet required performance targets and 

incentives for encouraging good performance. While Education agreed

with our recommendations, it is currently considering them as part of the 

development of its VR strategic performance plan, and has not adopted 

them to date. 

Earlier this year, GAO reported on national-level earnings outcomes for 

SSA beneficiaries who completed VR from 2000 to 2003.15 Among other 

findings, this report estimated that as a result of work, some DI and 

concurrent beneficiaries saw a reduction in their DI benefits—for an 

overall annual average benefit reduction of $26.6 million in the year after 

completing VR compared to the year before VR. Further, we reported that

10 percent of SSA beneficiaries who exited VR in 2000 or 2001 were able to 

 14GAO-05-865, 39. 

15GAO, Vocational Rehabilitation: Earnings Increased for Many SSA Beneficiaries after 

Completing VR Services, but Few Earned Enough to Leave SSA’s Disability Rolls,

GAO-07-332 (Washington, D.C.: March 2007). 

Page 10 GAO-07-521 Vocational Rehabilitation
leave the disability rolls at some point. However, almost one quarter of 

those who left had returned by 2005 for at least 1 month. 

Before controlling for factors that might explain differences in outcomes 

among state VR agencies, our analysis of state agency outcomes over a 3-

year period showed very different rates of success for SSA beneficiaries. 

This was the case in terms of the proportion of beneficiaries with earnings, 

earnings levels, and departures from the disability rolls. The wide range in

average earnings outcomes among agencies was generally consistent from 

2001 through 2003 and within each of the three types of agencies—

referred to as blind, general, and combined agencies. 

Between 2001 and 2003, VR agencies varied widely in terms of outcomes 

for SSA beneficiaries who completed their VR programs. While the agency 

average for beneficiary earnings was 50 percent, the proportion of 

beneficiaries with earnings during the year following VR varied 

substantially among agencies: from 0 to 75 percent. (See fig. 1.) 

State VR Agencies 

Consistently Showed 

Very Different Rates 

of Success for SSA 

Beneficiaries Who 

Completed VR 

Programs 

Proportion with Earnings, 

Earnings Levels, and 

Departures from the 

Disability Rolls for SSA 

Beneficiaries Differed 

Substantially among State 

Agencies

Page 11 GAO-07-521 Vocational Rehabilitation
Figure 1: Distribution of State VR Agencies by Percentage of SSA Beneficiaries with

Earnings during the Year after VR 

0

5

10

15

20

25

Percentage of agencies

Source: GAO analysis of SSA data.

Percentage of SSA beneficiaries with earnings during the year after VR for 2001-2003 exit cohorts

5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100

Note: n = 234, average = 50 percent. The 234 observations result from 78 VR agencies providing 

data for 3 years (2001 through 2003). 

Similarly, while the agency average for annual earnings levels for SSA 

beneficiaries who had earnings was $8,140, such earnings ranged by 

agency from about $1,500 to nearly $17,000. (See fig. 2.) 

Page 12 GAO-07-521 Vocational Rehabilitation
Figure 2: Distribution of State VR Agency Average Annual Earnings for SSA 

Beneficiaries with Earnings during the Year after VR

0

5

10

15

20

25

30

Percentage of agencies

Source: GAO analysis of SSA data.

Average annual earnings from 2001-2003 for SSA beneficiaries with earnings during the year after VR

1,0002,0003,000

4,000

5,000

6,000

7,000

8,000

9,000

10,000

11,000

12,000

13,000

14,000

15,000

16,000

17,000

Note: n = 232, average = $8,140. The number in figure 2 differs from that in figure 1 because two 

agencies did not have any beneficiaries with reported earnings in fiscal year 2002. All earnings are in 

2004 dollars. 

Agencies also differed in the proportion of SSA beneficiaries who had left

the disability rolls by 2005, with departure rates ranging anywhere from 0 

to 20 percent. The average departure rate was 7 percent. (See fig. 3.) 

Page 13 GAO-07-521 Vocational Rehabilitation
Figure 3: Distribution of State VR Agencies by Percentage of SSA Beneficiaries 

Leaving the Rolls 

0

5

10

15

20

25

30

Percentage of agencies

Source: GAO analysis of SSA data.

Percentage of SSA Beneficiaries Leaving the Rolls by 2005

2 4 6 8 10 12 14 16 18 20

Note: n = 234, average = 7 percent. 

Trends Were Similar over 

Time and by Agency Type 

In general, the range of earnings outcomes across agencies was similar 

over the 3 years we examined. While the average percentage of SSA 

beneficiaries with earnings during the year after VR declined slightly over

this period from 53 percent in 2001 to 48 percent in 2003, the spread in the 

percentage of beneficiaries with earnings remained widely dispersed 

across agencies for all 3 years, as shown in figure 4. 

Page 14 GAO-07-521 Vocational Rehabilitation
Figure 4: Range across State VR Agencies of the Percentage of SSA Beneficiaries

with Earnings during the Year after VR by Year 

0

10

20

30

40

50

60

70

80

2001 2002 2003

Percentage of SSA beneficiaries with earnings during the year after VR

Source: GAO analysis of SSA data.

Year

75

53

24

69

48

0

65

48

10

(n = 78) (n = 78) (n = 78)

Likewise, the range of average earnings among agencies was similar for all 

3 years, as shown in figure 5.16

 16See appendix I for an explanation of why we did not compare agencies’ rates of SSA 

beneficiaries leaving the rolls over this period. 

Page 15 GAO-07-521 Vocational Rehabilitation
Figure 5: Range of State VR Agency Average Earnings for SSA Beneficiaries by

Year 

0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

16,000

18,000

2001 2002 2003

Average earnings for SSA beneficiaries with earnings during the year after VR

Source: GAO analysis of SSA data.

Year

$15,902

$8,236

$4,830

$13,246

$7,994

$1,528

$16,805

$8,186

$2,092

(n = 78) (n = 76) (n = 78)

Note: Two agencies did not have any beneficiaries with reported earnings in fiscal year 2002. All 

earnings are in 2004 dollars. 

There were also wide differences in performance within the three types of 

agencies that serve different types of clientele—known as blind, general, 

and combined agencies. Specifically, among blind agencies, the percentage 

of SSA beneficiaries with earnings during the year after VR ranged from 23 

to 67 percent, with an average of 46 percent. Among general agencies, the 

percentage of SSA beneficiaries with earnings after VR varied from 37 to 

74 percent, with an average of 55 percent, and for combined agencies the

percentage varied from 0 to 75 percent, with an average of 49 percent. (See 

fig. 6.) 

Page 16 GAO-07-521 Vocational Rehabilitation
Figure 6: Range across State VR Agencies of the Percentage of SSA Beneficiaries

with Earnings during the Year after VR by Agency Type

0

10

20

30

40

50

60

70

80

Blind General Combined

Percentage of SSA beneficiaries with earnings during the year after VR

Source: GAO analysis of SSA data.

Agencies serving different types of clientele

67

46

23

74

55

37

75

49

0

(n = 72) (n = 72) (n = 90)

Average annual SSA client earnings among blind agencies varied the 

most—from $4,582 to $16,805, with an average of $10,699 per year. SSA 

client earnings among the combined agencies varied anywhere from 

$1,528 to $10,889, with an average of $7,088 per year. General agencies 

showed the least variation in earnings among their SSA clients—from 

$4,654 to $9,424—but the lowest average ($6,867). (See fig. 7.) 

Page 17 GAO-07-521 Vocational Rehabilitation
Figure 7: Range of State VR Agency Average Earnings for SSA Beneficiaries by

Agency Type

0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

16,000

18,000

Blind General Combined

Average earnings for SSA beneficiaries with earnings during the year after VR

GAO analysis of SSA data.

Agencies serving different types of clientele

$16,805

$10,699

$4,582

$9,424

$6,867

$4,654

$10,889

$7,088

$1,528

(n = 72) (n = (n = 72) 88)

Note: Two combined agencies did not have any beneficiaries with reported earnings in fiscal year 

2002. All earnings are in 2004 dollars. 

Finally, for rates of departure from the SSA disability rolls by 2005, blind 

agencies ranged from 0 to 16 percent, with an average of 6.7 percent; 

general agencies varied from 4 to 15 percent, with an average of 7.5 

percent; and combined agencies varied from 0 to 20 percent, with an 

average of 7 percent. (See fig. 8.) 

Page 18 GAO-07-521 Vocational Rehabilitation
Figure 8: Range of State VR Agency Average Rates of SSA Beneficiaries Leaving

the Rolls by Agency Type

0

5

10

15

20

25

Blind General Combined

Percentage of SSA beneficiaries leaving the rolls by 2005

Source: GAO analysis of SSA data.

Agencies serving different types of clientele

16

6.7

0 0

15

7.5

4

20

7.0

(n = 72) (n = 72) (n = 90)

After controlling for a range of factors, we found that much of the 

differences in state VR agency success rates could be explained by state

economic climates and the characteristics of the SSA beneficiary 

populations at the VR agencies. Specifically, among a range of possible 

factors we considered, the economic conditions of the state appeared to

explain up to one-third of the differences between state agency outcomes 

for SSA beneficiaries.17 Additionally, differences in the characteristics of 

the clientele accounted for some of the variation in performance among 

VR agencies. 

State Economic

Conditions and SSA 

Beneficiary 

Characteristics 

Account for Much of 

the Difference in State 

VR Agency Success 

Rates 

 17All findings discussed in this section are statistically significant at the 0.05 level, unless 

otherwise noted. 

Page 19 GAO-07-521 Vocational Rehabilitation
When we controlled for a variety of factors using multivariate analysis, we 

found that state economic conditions accounted for a substantial portion 

of the differences in VR outcomes across state agencies. Not surprisingly, 

we found that fewer SSA beneficiaries had earnings during the year after

completing VR in states with high unemployment rates after controlling for 

other factors. Moreover, our analysis showed that for each 1 percent 

increase in the unemployment rate, the percentage of SSA beneficiaries 

who had earnings during the year after completing VR decreased by over 2 

percent.18 Across agencies, unemployment rates ranged from 2.3 to 12.3

percent between 2001 and 2003, with an average of 4.7 percent. 

Differences in Agency 

Outcomes Were Largely 

Due to a State’s Economic 

Conditions 

We also found that after controlling for other factors, VR agencies in states 

with lower per capita incomes saw fewer SSA beneficiaries who had 

earnings, lower earnings levels, and fewer departures from the disability 

rolls in the year after VR. Across states, per capita incomes ranged from 

approximately $4,400 to $46,000 dollars, with an average of approximately 

$28,000. Together, state unemployment rates and per capita incomes 

explained over one-third of the differences between states agencies in the 

proportion of SSA beneficiaries that had earnings during the year after VR

and the proportion that left the rolls. 19 

Agency officials commented that difficult economic environments result in 

lower earnings outcomes because a state’s economy has a direct impact 

on an agency’s ability to find employment for individuals. Our findings are 

also consistent with past research that has found labor market conditions 

to be among the most influential determinants of agency performance.20

Education, however, does not currently consider state economic 

 18Unless otherwise indicated, the effects being discussed in this and the next section are 

marginal effects (i.e., the effect of a 1 unit change in the explanatory variable on the

dependent variable, holding other factors constant). See appendix I, section 3, for more

details on our econometric analyses. 

19We also found that in states with larger populations, fewer SSA beneficiaries (1) had

earnings during the year after completing VR and (2) left the disability rolls. A study 

conducted by RTI International noted that states with small populations reported having

improved access to other agencies and better collaboration with state leaders due to closer 

work and personal relationships. 

20Michael D. Tashjian, et al., Study of Variables Related to State Vocational Rehabilitation

Agency Performance Revised Draft Final Report, a special report prepared at the request 

of the Department of Education, October 2004. 

Page 20 GAO-07-521 Vocational Rehabilitation
conditions when evaluating agency performance.21 Although Education 

agreed with our prior recommendation to consider economic and 

demographic characteristics when evaluating agency performance, 

Education is currently considering it as part of the development of its VR 

strategic performance plan and has not yet adopted this recommendation. 

Demographic 

Characteristics and the 

Types of Disabilities of 

Clientele Also Accounted 

for Some of the Disparities 

in State Agency 

Performance 

After controlling for a variety of factors, certain characteristics of the 

clientele served by state agencies accounted for some of the state agency 

differences in earnings outcomes for SSA beneficiaries. Among the factors 

we examined the influence of were: demographic characteristics, types of 

disabilities, and the proportion of SSA beneficiaries served by each state 

agency.

22

Several clientele characteristics influenced state agency earnings 

outcomes.23 In particular, after controlling for other factors, state agencies 

that served a higher proportion of women beneficiaries had fewer 

beneficiaries with earnings during the year after completing VR. According 

to our analysis, a 10 percent increase in the percentage of women served 

by a VR agency resulted in a 5 percent decrease in the percentage of SSA 

beneficiaries with earnings. Research shows that for the population of 

low-income adults with disabilities, women were found to have lower 

employment rates than men.24

Demographic Differences 

Further, we found that after controlling for other factors, state agencies 

serving a larger percentage of SSA beneficiaries between 46 and 55 years 

old when they applied for the VR program saw fewer SSA beneficiaries 

 21Although state economic and demographic conditions are not factored into performance

measures and targets, Education considers these factors through its monitoring of state 

agencies. In addition, the statutory funding formula for VR agencies allocates relatively 

more funds to poorer states based on per capita income to help offset a lack of resources. 

22See appendix I for a detailed list of the factors we controlled for. 

23These clientele characteristics appeared to influence one or more of the earnings 

outcomes measured, but not necessarily all three. 

24David Wittenburg and Melissa Favreault, “Safety Net or Tangled Web? An Overview of 

Programs and Services for Adults with Disabilities” (Occasional Paper No. 68, the Urban 

Institute, Washington, D.C.: 2003). 

Page 21 GAO-07-521 Vocational Rehabilitation
leave the disability rolls.25 For every 10 percent increase in the percentage

of beneficiaries in this age group, the percentage of SSA beneficiaries 

leaving the rolls decreased by approximately 1 percent. 

When we considered the influence of various types of medical 

impairments on earnings outcomes, we found that some state agency 

outcomes were related to the proportion of SSA beneficiaries who had 

mental or visual impairments. Average earnings and departures from the 

disability rolls for SSA beneficiaries were lower in agencies that served a 

larger percentage of individuals with mental impairments, after controlling 

for other factors. Specifically, our analysis indicated that a 10 percent 

increase in the proportion of the beneficiary population with a mental 

impairment resulted in a decrease of almost 1 percent in the proportion of 

SSA beneficiaries who left the rolls. Some SSA beneficiaries may not leave 

the disability rolls because, as research has shown, they fear a loss of their 

public benefits or health coverage.26 This is particularly true for individuals

with mental impairments. 

Differences in Types of 

Disabilities

Agencies with a higher proportion of blind or visually impaired 

beneficiaries had fewer departures from the disability rolls after 

controlling for other factors. We found that an increase of 10 percent in 

the proportion of individuals with a visual impairment resulted in a 

decrease of 0.5 percent of beneficiaries leaving the rolls. Some SSA 

beneficiaries with visual impairments are classified as legally blind. As 

such, they are subject to a higher earnings threshold, in comparison to 

those that are not legally blind, before their benefits are reduced or 

ceased. Our analysis also showed that holding other factors equal, blind 

agencies—those serving only clientele with visual impairments—had 

fewer SSA beneficiaries with earnings during the year after completing VR 

than agencies that served a lower proportion of beneficiaries with visual 

impairments.27

 25While other variables were significant at the 0.05 level, this variable was significant at the

0.10 level. See appendix I for more information. 

26Timothy Tremblay et al., “Effect of Benefits Counseling Services on Employment 

Outcomes for People with Psychiatric Disabilities,” Psychiatric Services, vol. 57, no. 6 

(2006). 

27Specifically, holding other factors constant, agencies known as combined or general 

agencies had more SSA beneficiaries with earnings during the year after VR than agencies 

known as blind agencies. 

Page 22 GAO-07-521 Vocational Rehabilitation
Differences in the proportion of SSA beneficiaries served by an agency 

also affected earnings outcomes for SSA beneficiaries. Specifically, 

agencies with a greater proportion of SSA beneficiaries had more 

beneficiaries with earnings during the year after VR, but saw lower 

earnings levels for their SSA beneficiaries, holding other factors constant. 

VR state agency officials and experts with whom we consulted were 

unable to provide an explanation for this result.28

Proportion of SSA Beneficiaries 

Served 

We also found that after controlling for other factors, agencies with a 

higher proportion of SSA beneficiaries who were DI beneficiaries had 

lower average annual earnings among SSA beneficiaries and a lower 

percentage of beneficiaries leaving the rolls. The earnings result might be 

explained by differences in the work incentive rules between the two 

programs. Specifically, the work incentive rules are more favorable for SSI 

beneficiaries who want to increase their earnings while not incurring a net 

income penalty.29 The lower rates of departures from the rolls among 

agencies with a greater proportion of DI beneficiaries might be due to the 

limited time frames of our study and the fact that DI beneficiaries are 

allowed to work for a longer period of time before their benefits are 

ceased.30

 28In its comments on our report, Education suggested that VR agencies with high 

proportions of SSA beneficiaries may also have high levels of collaboration with other

agencies because the long-term supports that may be required to live in the community 

necessitate cooperation with other public programs. The department noted that this may 

account for our findings because benefit eligibility may be necessary to receive certain 

supports from outside agencies. 

29See GAO-07-332 for a more detailed description of the differing DI and SSI benefit 

structures.

30See appendix I, section 4, for a detailed description of why, given the time frames of our 

study, the rates of departures from the rolls might be lower for DI beneficiaries. 

Page 23 GAO-07-521 Vocational Rehabilitation
When we analyzed outcomes at the agency level, a few agency practices 

appeared to yield some positive results, albeit in different ways. 

Specifically, after controlling for other factors, we found that state 

agencies with a higher proportion of state-certified VR counselors, or 

stronger relationships with businesses or other public agencies appeared

to have better earnings outcomes. Further, agencies that devoted a greater 

proportion of their service expenditures to training of VR clients had 

higher average annual earnings for SSA beneficiaries completing VR, 

holding other factors equal. On the other hand, our multivariate analyses 

suggest that agencies using in-house benefits counselors saw fewer 

beneficiaries with earnings following VR, but these results may not be 

conclusive because the benefits counseling program has changed 

considerably since the time period of our study. 

State VR agencies that reported employing a higher percentage of 

counselors meeting the state certification standards had higher rates of 

beneficiaries with earnings among those beneficiaries who completed VR 

between 2001 and 2003, holding other factors constant. On average, 62 

percent of counselors at an agency met the states’ certification 

requirements, but the range was from 0 to 100 percent. According to our 

analysis, for every 10 percent increase in the percentage of counselors 

meeting state requirements, the percentage of SSA beneficiaries with 

earnings during the year after VR increased by 0.5 percent. This appeared 

to be consistent with research indicating that more highly qualified VR 

counselors are more likely to achieve successful earnings outcomes.31

While the certification requirements vary by state, agency officials 

reported that counselors with master’s degrees in vocational rehabilitation 

are more likely to be versed in the history of the VR program and the 

disability rights movement and are likely to be more attuned to the needs 

of their clients than those without specialized degrees. 

A Few Agency 

Practices Appeared to 

Yield Better Earnings 

Outcomes, while the 

Results of Other 

Practices Were 

Inconclusive 

Agencies with StateCertified Counselors or 

Strong Relationships with 

Businesses or Other Public 

Agencies Appeared to 

Have Better Earnings 

Outcomes 

VR agencies that had stronger relationships with the business community 

had higher average earnings among SSA beneficiaries during the year after

completing VR and higher rates of departures from the disability rolls, 

holding other factors equal. These were agencies that reported interacting

with the business community more frequently by sponsoring job fairs, 

 31Edna Mora Szymanski, “Relationship of Level of Rehabilitation Counselor Education to 

Rehabilitation Client Outcome in the Wisconsin Division of Vocational Rehabilitation,” 

Rehabilitation Counseling Bulletin, vol. 35, no. 1 (1991). 

Page 24 GAO-07-521 Vocational Rehabilitation
hosting breakfasts, attending business network meetings, meeting with 

local businesses, meeting with local chambers of commerce, and 

interacting with civic clubs. To support these practices, Education has 

helped establish the Vocational Rehabilitation Employer Business and 

Development Network, which aims to connect the business community to 

qualified workers with disabilities through the efforts of staff located at 

each of the VR agencies who specialize in business networking.32 VR 

agency officials with whom we spoke said that through interaction with 

the business community, they could dispel myths about the employability 

of people with disabilities, and they could tailor services for their clients to

the specific needs of different businesses. 

In addition to business outreach, our multivariate analysis indicated that 

agencies that reported receiving a greater degree of support and 

cooperation from more than one public program—such as from state 

social services, mental health, and education departments—also showed 

higher average earnings among SSA beneficiaries. One VR agency official

commented that people with disabilities need multiple supports and 

services and therefore are more effectively served through partnerships 

between government programs.33 Another VR official said that 

coordination with other programs facilitated the provision of a complete 

package of employment-related services. For example, VR might provide 

employment training to an individual, while the department of labor might 

provide transportation services to get the person to work. Although many 

agencies said they were successful in coordinating with other programs, 

some reported difficulties. For example, they cited barriers to 

coordinating with WIA one-stops such as inability to share credit for 

successful earnings outcomes, staff that are not trained to serve people 

with disabilities, and inaccessible equipment, particularly for those with 

visual or hearing impairments. 

 32The Council of State Administrators of Vocational Rehabilitation is also developing a

national VR-business network whose aim is to coordinate VR outreach efforts to 

businesses. In addition to these national-level efforts, state VR agencies also participate in 

state-level business networks. In Utah, for example, the VR agency participates in the Utah

Business Employment Team, which serves as a business-to-business network for 

recognizing and promoting best practices in hiring, retaining, and marketing to people with 

disabilities. 

33Past GAO reports have highlighted the need for greater coordination among agencies 

delivering services to people with disabilities. See, for example, GAO, Federal Disability

Assistance: Wide Array of Programs Needs to Be Examined in Light of 21st Century

Challenges, GAO-05-626 (Washington, D.C.: June 2, 2005). 

Page 25 GAO-07-521 Vocational Rehabilitation
Additionally, agencies with a greater proportion of their service 

expenditures spent on training of VR clients—including postsecondary 

education, job readiness and augmentative skills, and vocational and 

occupational training—had higher average annual earnings for SSA 

beneficiaries completing VR, holding other factors equal.34 The average 

percentage of service expenditures devoted to training of VR clients was 

47 percent, but this ranged from 3 to 84 percent across agencies. Research

has shown that the receipt of certain types of training services, such as 

business and vocational training, leads to positive earnings outcomes.35

Our analysis suggests that after controlling for other factors, agencies with 

in-house benefits counselors—counselors who advise VR clients on the 

impact of employment on their benefits—had lower rates of SSA 

beneficiaries with earnings during the year after completing VR than 

agencies without them. Over the years we studied, only 14 percent of state 

agencies reported using in-house benefits counselors. However, this was a 

period of transition for the benefits counseling program. There was wide 

variation in how this service was provided, and clients in states that did 

not have on-site benefits counselors may have received benefits 

counseling from outside the agency. According to one researcher, the 

benefits counseling program has become more standardized since that 

period. In fact, other empirical research shows that benefits counselors 

have had a positive effect on earnings.36

Agency Expenditures on 

Training Yield Positive 

Outcomes 

Effect of Using In-house 

Benefits Counselors is 

Unclear 

 34The expenditures considered for this calculation do not include assessment, counseling,

guidance, and placement services provided directly by VR personnel since these services

are generally provided to all VR clients. The total expenditures in this calculation include 

those optional services that are provided to clients based on their specific needs. 

35Becky J. Hayward and Holly Schmidt Davis, Longitudinal Study of the Vocational 

Rehabilitation Services Program Final Report 2: VR Services and Outcomes, a special 

report prepared at the request of the Department of Education, 2003.

36Other research finds a positive effect of benefits counseling on earnings among 

beneficiaries with psychiatric disabilities and clients in the state of Vermont. See Timothy 

Tremblay, et al., “Effect of Benefits Counseling Services on Employment Outcomes for

People with Psychiatric Disabilities,” Psychiatric Services, vol. 57, no. 6 (2006). 

Page 26 GAO-07-521 Vocational Rehabilitation
Some agency officials credited certain other practices with yielding 

positive results, but we were not able to corroborate their ideas with our 

statistical approach. For example, VR agency officials cited the following 

practices as being beneficial: (1) collaborative initiatives between the state

VR agency and other state agencies aimed to help specific client 

populations, such as individuals with mental impairments or 

developmental disabilities; (2) unique applications of performance 

measures, such as measuring performance at the team level rather than 

the individual counselor level; and (3) improved use of computer 

information systems, such as real-time access to the status of individual 

employment targets. Although we were able to examine many state 

practices with our survey data, there were not enough agencies employing 

these practices for us to determine whether these practices led to 

improved earnings outcomes for SSA beneficiaries among state VR 

agencies. 

Although we were able to explain a large amount of the differences in 

earnings outcomes among state agencies, we could only explain a small 

amount of the differences in earnings outcomes among individual SSA 

beneficiaries. Specifically, while our models accounted for between 66 and 

77 percent of the variation in agency-level earnings outcomes, our models 

using the individual-level data had low explanatory power, accounting for

only 8 percent of variation in earnings levels across individuals and rarely

producing reliable predictions for achieving earnings or leaving the rolls. 

With so little variation explained in individual-level outcomes, we could 

not be confident that our individual-level analyses were sufficiently 

reliable to support conclusions. As a result, we chose not to report on 

these analyses. Other researchers told us they have experienced similar 

difficulties using Education’s client database to account for individual 

differences in earnings outcomes among VR clients. 

VR Officials in Some 

Agencies Credited Other 

Practices with Yielding 

Results 

Limitations in 

Education’s Data May 

Have Hampered 

Analyses of Individual 

Earnings Outcomes 

Education’s data lack information that we believe is critical to assessing 

earnings outcomes, and not having this information may have hindered 

our ability to explain differences in individual earnings outcomes.37

Specifically, Education does not collect certain information on VR clients 

that research has linked to work outcomes, such as detailed information 

 37We cannot say with certainty that our results were detrimentally affected by these 

limitations because we do not have data without these limitations with which to test our 

hypotheses. 

Page 27 GAO-07-521 Vocational Rehabilitation
on the severity of the disability and historical earnings data. Research 

indicates that both of these factors are, or could be, important to 

determining employment success for people with disabilities.38 With regard 

to obtaining information on the severity of the client’s disability, knowing 

the severity of the disability can indicate the extent to which a person is 

physically or mentally limited in the ability to perform work, a fact that 

may influence the person’s earnings outcomes. While Education’s client 

data include information indicating whether a disability is significant—

which is defined by the Rehabilitation Act—the data do not include more 

detailed information on the severity of the disability, such as the number 

and extent of functional limitations.39 Additionally, Education does not 

collect information on a client’s historical earnings, which may provide a 

broader understanding of the client’s work experience and likelihood to 

return to work. States may be able to obtain earnings data from other 

official sources, such as other state and federal agencies. 

Another limitation with Education’s data is the collection of self-reported 

information from the client that may not be validated by the VR agency. 

For example, one agency official said that clients are asked to report their 

earnings at the time of application—information that Education is legally

required to collect—and that these data may not be accurate. Reliable 

information on a client’s earnings at the time of application to VR is 

essential for evaluating the impact of the VR program on earnings. 

However, some clients may misreport their earnings. One researcher 

reported, for example, that VR clients sometimes report net as opposed to 

gross earnings. Instead of relying on self-reported information, agencies 

may be able to obtain or validate this information from official sources. 

Specifically, some state VR agencies have agreements with other state and 

federal agencies to obtain earnings data on their clients. For example, 

agency officials from one state told us that they match their data against 

earnings data from the Department of Labor, while another agency relies 

on data from their state’s Employment Development Department. 

However, in some cases state agencies are required to pay for these data. 

 38Mitchell P. LaPlante and H. Stephen Kaye, “The Employment and Health Status of 

Californians with Disabilities” (Institute of Health and Aging, University of California, San 

Francisco, June 2005). 

39In evaluating the significance of a disability, some state VR agencies already collect such 

information. 

Page 28 GAO-07-521 Vocational Rehabilitation
The federal-state vocational rehabilitation program is still the primary 

avenue for someone with a disability to prepare for and obtain 

employment. Given the growing size of the disability rolls and the 

potential savings associated with moving beneficiaries into the workforce, 

it is important to make the nation’s VR program as effective as possible to 

help people with disabilities participate in the workforce. 

Conclusions 

Our findings indicate that it will be difficult to maximize the effectiveness 

of the VR program with assessments of state agency performance that do

not account for important factors, such as the economic health of the 

state. Such comparisons will be misleading. Without credible indicators,

VR agencies do not have an accurate picture of their relative performance, 

and Education may continue its reluctance to use sanctions or incentives 

to encourage compliance. Our findings underscore the recommendation 

that we made in 2005 that Education consider economic factors in 

assessing the performance of state vocational rehabilitation agencies. 

Moreover, our study points to deficiencies in Education’s data that may 

hinder more conclusive analyses of individual-level earnings outcomes. 

Without data on the severity of a client’s disability or information on 

historical earnings, VR programs may not be able to conduct valuable 

analysis to explain differences in individual outcomes. With the growing 

emphasis on the role of VR in helping people with disabilities enter the 

workforce, the need for such analyses—and data that can be used to 

conduct them—is likely to increase.

Despite the deficiencies in Education’s data, our findings show that certain

agency practices may improve VR success across the country and give 

weight to current efforts by Education to promote such practices. The fact 

that agencies with stronger ties to the business community have achieved 

higher earnings among their SSA beneficiaries suggests the importance of 

such practices, such as Education’s initiative to promote business 

networks. Our findings also demonstrate the value of having VR 

counselors meet state certification standards and having agencies 

collaborate with more than one supportive public agency to help their 

clients. Our study also suggests that other practices, such as state agencies 

devoting more resources to targeted training services for VR clients, may 

have positive benefits. 

Page 29 GAO-07-521 Vocational Rehabilitation
To improve the effectiveness of Education’s program evaluation efforts 

and ultimately the management of vocational rehabilitation programs, we

recommend that the Secretary of Education: 

1. Further promote agency practices that show promise for helping more 

SSA disability beneficiaries participate in the workforce. Such a 

strategy should seek to increase: 

• the percentage of VR staff who meet state standards and 

certifications established under the CSPD, 

• partnership or involvement with area business communities, and 

• collaboration with other agencies that provide complementary 

services. 

2. Reassess Education’s collection of VR client data through consultation 

with outside experts in vocational rehabilitation and the state 

agencies. In particular, it should: 

• consider the importance of data elements that are self-reported by 

the client and explore cost-effective approaches for verifying these 

data, and 

• consider collecting additional data that may be related to work 

outcomes, such as more detailed data on the severity of the client’s 

disability and past earnings history, collaborating whenever 

possible with other state and federal agencies to collect this 

information. 

3. In a 2005 report, we recommended that Education revise its 

performance measures or adjust performance targets for individual 

state VR agencies to account for additional factors. These include the 

economic conditions of states, as well as the demographics of a state’s 

population. We continue to believe that Education should adopt this 

recommendation, especially in light of our findings on the impact of 

state unemployment rates, per capita incomes, and demographic 

factors on earnings outcomes. 

We received written comments on a draft of this report from Education, 

which oversees the VR program, and SSA, from which we received data

that were used to evaluate its Ticket to Work program. Education 

commended our use of multiple data sources and said that it opens up new 

analytical possibilities in evaluating how VR programs serve SSA 

beneficiaries, including identifying low-performing and high-performing 

Recommendations for 

Executive Action 

Agency Comments 

and Our Evaluation 

Page 30 GAO-07-521 Vocational Rehabilitation
VR programs. However, Education also questioned whether the statistical 

relationships we found can be applied to how it administers a stateoperated formula grant program. We continue to believe our findings have 

important implications for improving what data are collected and how VR

services are delivered. While Education generally agreed with the 

substance of our recommendations, it disagreed on when economic 

conditions and state demographics should be considered in assessing 

agency performance. Instead of using this information to help set 

performance measures, the department said that it takes these factors into

account when it monitors agency performance results and believes that its 

approach is effective. We believe that incorporating this contextual 

information into assessing performance is essential to provide the state 

agencies with a more accurate picture of their relative performance. 

Although Education stated that it was open to our recommendation on 

improving data quality, it suggested that validating self-reported 

information would be a potential burden to state agencies and suggested 

other approaches, such as conducting periodic studies. Our 

recommendation that Education explore cost-effective ways to validate 

self-reported data was based on the experience of some VR agencies that

have obtained data successfully from official sources and not relied solely 

on self-reported information. We made additional technical changes as 

appropriate based on Education’s comments. See appendix II for a full 

reprinting of Education’s comments and our detailed responses. 

SSA stated that our report has methodological flaws that introduced 

aggregation bias and false correlations, and suggested that we should have 

focused on individual-level analysis or reported the results of both 

individual and aggregate-level analyses. We used aggregated data—a 

widely used means of analysis—because our primary objective was to 

understand better the wide variation in outcomes for state VR agencies 

that serve SSA beneficiaries rather than the outcomes for individuals. 

Further, we used appropriate statistical techniques to ensure the lack of 

bias due to clustering of individual cases within agencies (see app. I for a 

more detailed discussion). Because we used aggregated data, we did not 

attempt to infer the effects of individual behavior or individual outcomes. 

Additionally, SSA had concerns about the implications of our analysis of 

state economic factors on agency-level outcomes. Our findings related to

the influence of state economic characteristics were highly statistically 

significant as well as corroborated by previous research, and we believe 

these results have important implications for VR agency performance 

measures. SSA provided additional comments, which we addressed or 

incorporated, as appropriate. See appendix III for a full reprinting of SSA’s 

comments as well as our detailed responses. 

Page 31 GAO-07-521 Vocational Rehabilitation
Copies of this report are being sent to the Secretary of Education, the 

Commissioner of SSA, appropriate congressional committees, and other 

interested parties. The report is also available at no charge on GAO’s Web 

site at http://www.gao.gov. If you have any questions about this report, 

please contact me at (202) 512-7215. Other major contributors to this 

report are listed in appendix IV. 

Denise M. Fantone 

Acting Director, Education, Workforce, 

 and Income Security Issues 

Page 32 GAO-07-521 Vocational Rehabilitation
Appendix I: Scope and Methodology 

Page 33 GAO-07-521

Appendix I: Scope and Methodology 

To understand the variation in state agency outcomes for Social Security 

Administration (SSA) disability beneficiaries completing the vocational 

rehabilitation (VR) program, we conducted two sets of analyses. First, we 

used descriptive analyses to compare agency performance with three 

measures of earnings outcomes from 2001 to 2003. Second, using agency 

and survey data, we conducted econometric analyses of the three 

measures of earnings outcomes to determine how state and agency 

characteristics related to state agency performance. 

We developed our analyses in consultation with GAO methodologists, an

expert consultant, and officials from SSA and the Department of 

Education (Education).1

 To choose the appropriate variables for our 

analyses, we reviewed pertinent literature and consulted with agency

officials and academic experts. 

This appendix is organized in four sections: Section 1 describes the data 

that were used in our analyses and our efforts to ensure data reliability. 

Section 2 describes the study population, how the dependent variables 

used in the analyses were constructed, and the descriptive analyses of 

those variables. Section 3 describes the econometric analyses. Section 4 

explains the limitations of our analyses. 

This section describes each of the datasets we analyzed, the variables from 

each dataset that were used in our analyses, and the steps that were taken 

to assess the reliability of each dataset. 

To conduct our analyses, we used several data sources: (1) a newly 

available longitudinal dataset that includes information from several SSA 

and Education administrative databases on all SSA disability beneficiaries 

who completed the VR program from 2001 through 2003; (2) data from 

Education on yearly spending information by service category for each 

state VR agency; (3) data from the Census Bureau, the Bureau of Labor 

Statistics, and other data sources regarding state demographic and 

economic characteristics; and (4) original survey data collected by GAO 

from state VR agencies. To perform our analyses, we used variables from 

Section 1: Data Used, 

Information Sources, 

and Data Reliability 

 1

We are especially grateful to Professor Herbert Smith—Professor of Sociology and 

Director, Population Studies Center at the University of Pennsylvania, and an expert in the

area of statistical analysis—who provided valuable advice on our statistical methods. 

Vocational Rehabilitation
Appendix I: Scope and Methodology 

each of the above datasets by merging, by agency and year, each of the

datasets into one large data file. 

Education and SSA 

Beneficiary Data 

We obtained a newly available longitudinal dataset—a subfile of SSA’s 

Ticket Research File (TRF)—which contains information from several 

SSA and Education administrative databases on all SSA disability 

beneficiaries who completed the federal-state VR program between 1998 

and 2004.2

 SSA merged this dataset with its Master Earnings File (MEF), 

which contains information on each beneficiary’s annual earnings from 

1990 through 2004. The combined data provide information about each 

beneficiary’s disability benefits, earnings, and VR participation.3 See 

section 2 of this appendix for a description of how these data were used to 

create our dependent variables on earnings outcomes. 

We were interested in how earnings outcomes were affected by 

differences across agencies, including differences in characteristics of the 

individuals served by the different agencies. Table 1 shows information 

from the TRF subfile on characteristics of our study population that we 

included among our explanatory variables.4

 2

In 2003, SSA contracted with Mathematica Policy Research to conduct a full evaluation of

the Ticket to Work Program. As part of this evaluation, Mathematica constructed the Ticket 

Research File, a compilation of longitudinal data from SSA. An extract of the TRF was 

merged with vocational rehabilitation data from the Department of Education’s RSA-911 

database by an SSA official. 

3

Education’s data on VR closures were available from 1998 to 2004. Data from SSA’s TRF 

database were available from 1994 to 2004, with MEF earnings data available from 1990 to 

2004. Social Security’s MEF data are annual earnings based on Internal Revenue Service W2 tax filings. At the time we obtained this dataset from SSA, earnings data for 2005 were not

available. 

4

For the purposes of this study, the term “explanatory variable” is used to describe a 

variable that is used to predict the value of another variable, and the term “dependent 

variable” is used to describe a variable whose values are predicted by the explanatory 

variable. 

Page 34 GAO-07-521 Vocational Rehabilitation
Appendix I: Scope and Methodology 

Table 1: Explanatory Variables from the TRF Subfile 

State agency demographic characteristics 

Percentage of beneficiaries between the ages of 18 and 25 

Percentage of beneficiaries between the ages of 26 and 35

Percentage of beneficiaries between the ages of 36 and 45

Percentage of beneficiaries between the ages of 46 and 55

Percentage of beneficiaries between the ages of 56 and 64

Percentage of female beneficiaries 

Percentage of white beneficiaries 

Percentage of African-American beneficiaries

Percentage of Native-American beneficiaries

Percentage of Asian and Pacific Islander beneficiaries 

Percentage of Hispanic beneficiaries 

Percentage of multiracial beneficiaries 

Stage agency medical characteristics 

Percentage of beneficiaries who are blind or have visual impairments 

Percentage of beneficiaries with sensory impairments 

Percentage of beneficiaries with physical impairments 

Percentage of beneficiaries with mental impairments 

State agency program participation

Percentage of beneficiaries receiving Supplemental Security Income 

Percentage of beneficiaries receiving Disability Insurance 

Percentage of concurrent beneficiaries (receiving both SSI and DI) 

Proportion of SSA beneficiaries served by an agencya

Source: SSA and Education data. 

a To construct this variable, additional information was obtained from Education on the total number of 

clients completing the VR program. 

To determine the reliability of the TRF subfile, we 

• reviewed SSA and Education documentation regarding the planning for 

and construction of the TRF subfile, 

• conducted our own electronic data testing to assess the accuracy and 

completeness of the data used in our analyses, and 

• reviewed prior GAO reports and consulted with GAO staff 

knowledgeable about these datasets.

Page 35 GAO-07-521 Vocational Rehabilitation
Appendix I: Scope and Methodology 

On the basis of these steps, we determined that despite the limitations 

outlined in section 4, the data that were critical to our analyses were 

sufficiently reliable for our use. 

VR Agency Administrative 

Data 

To determine whether differences in agency size and expenditure patterns 

affected earnings outcomes, we obtained information on state VR agency 

expenditures for the years 2000 through 2002 from the RSA-2 data, an 

administrative dataset compiled by Education. The RSA-2 data contain 

aggregated agency expenditures for each of the 80 state VR agencies as 

reported in various categories, such as administration and different types 

of services. Table 2 shows the variables that were derived from the RSA-2 

data. 

Table 2: Explanatory Variables from Education’s RSA-2 Data 

Agency structure 

Type of agency: (1) general, (2) blind, and (3) combined agencies 

Number of people receiving services (proxy for size) 

Total expenditures on services (proxy for size) 

Agency expenditures 

Percentage of all service expenditures spent on assessment 

Percentage of all service expenditures spent on diagnosis/treatment 

Percentage of all service expenditures spent on training services for VR clients 

Percentage of all service expenditures spent on maintenance 

Percentage of all service expenditures spent on transportation 

Percentage of all service expenditures spent on personal assistance services 

Percentage of all service expenditures spent on placement services 

Percentage of all service expenditures spent on post employment services 

Percentage of all service expenditures spent on other services 

Percentage of total service expenditures (not including assessment, counseling, 

guidance, and placement) spent on assessmenta

 

Percentage of total service expenditures (not including assessment, counseling, 

guidance, and placement) spent on diagnosis/treatmenta

 

Percentage of total service expenditures (not including assessment, counseling, 

guidance, and placement) spent on training services for VR clientsa

Percentage of total service expenditures (not including assessment, counseling, 

guidance, and placement) spent on maintenancea

 

Percentage of total service expenditures (not including assessment, counseling, 

guidance, and placement) spent on transportationa

 

Page 36 GAO-07-521 Vocational Rehabilitation
Appendix I: Scope and Methodology 

Percentage of total service expenditures (not including assessment, counseling, 

guidance, and placement) spent on personal assistance servicesa

 

Percentage of total service expenditures (not including assessment, counseling, 

guidance, and placement) spent on placementa

 

Percentage of total service expenditures (not including assessment, counseling, 

guidance, and placement) spent on post employment servicesa

 

Percentage of total service expenditures (not including assessment, counseling, 

guidance, and placement) spent on other servicesa

Percentage of total expenditures spent on administration 

Percentage of total expenditures spent on services provided directly by VR personnel 

Percentage of total expenditures spent on purchased services 

Percentage of total expenditures spent on services purchased from public vendors 

Percentage of total expenditures spent on services purchased from private vendors 

Percentage of total expenditures spent on services to individuals with disabilities 

Percentage of total expenditures spent on services to groups with disabilities 

Source: Education data. 

a

These total expenditures include those optional services that are provided to clients based on their 

specific needs. They do not include assessment, counseling, guidance, and placement services 

provided directly by VR personnel since these services are generally provided to all VR clients. 

To determine the reliability of the RSA-2 data, we 

• reviewed relevant agency documentation and interviewed agency 

officials who were knowledgeable about the data, and 

• conducted our own electronic data testing to assess the accuracy and 

completeness of the data used in our analyses. 

On the basis of these steps, we determined that the data that were critical

to our analyses were sufficiently reliable for our use. 

State Economic and 

Demographic Data 

We were interested in how differences in state characteristics affected 

earnings outcomes of SSA beneficiaries completing VR at different VR 

agencies. The state characteristics we considered included economic 

conditions (unemployment rates, per capita income, and gross state 

product growth rates), population characteristics (including size, density, 

and percentage living in rural areas and on Disability Insurance), and 

availability of the Medicaid Buy-in program. Data on state characteristics 

were downloaded from several sources, including federal agencies and 

research institutes. The research institutes from which we obtained data

included Cornell University Institute for Policy Research and Mathematica 

Policy Research, Inc., both authorities in social science research. Table 3 

Page 37 GAO-07-521 Vocational Rehabilitation
Appendix I: Scope and Methodology 

summarizes the state data that were collected and the sources for those 

data. 

Table 3: State Economic and Demographic Explanatory Variables and Their Sources

Variable Data source

Annual state unemployment rates Department of Labor, Bureau of Labor Statistics 

Gross state product (GSP) growth rate Department of Commerce, Bureau of Economic Analysis 

Annual per capita income Department of Commerce, Bureau of Economic Analysis 

Annual population Department of Commerce, Census Bureau 

Population density Department of Commerce, Census Bureau 

Percentage of rural population Department of Commerce, Census Bureau 

Medicaid Buy-In participation Cornell University Institute of Policy Research and Mathematica Policy 

Research, Inc. (primary sources) 

Ticket to Work program implementation Mathematica Policy Research, Inc. 

Source: Various data sources listed in table. 

For each of these data sources we reviewed documentation related to the 

agency’s or research organization’s efforts to ensure the accuracy and

integrity of their data. On the basis of these reviews, we concluded that the 

data were sufficiently reliable for the purposes of our review. 

VR Agency Survey Data We were also interested in how differences in the VR agencies themselves 

affected earnings outcomes. To obtain information about the policies, 

practices, and environment of each state VR agency, we conducted a 

detailed survey of all state agencies. The survey was intended to collect 

information that may be relevant to explaining earnings outcomes of SSA 

beneficiaries who exited the VR program between federal fiscal years 2001 

through 2003. Specifically, we collected information on the structure of 

the VR program, staffing and turnover rates, performance measures, 

service portfolios, and the extent of integration with outside partners such 

as other state and federal agencies and the business community.5

 In 

developing our survey, we identified relevant areas of inquiry by 

conducting a review of the literature on state VR agency performance and 

consulting with state agency officials and outside researchers. 

 5

Electronic copies of the survey are available upon request. 

Page 38 GAO-07-521 Vocational Rehabilitation
Appendix I: Scope and Methodology 

For the final survey, we sent e-mail notifications asking state agency

officials to complete either a Web-based version of the survey (which was 

accessible to those with visual impairments) or a Microsoft Word version 

of the survey by August 4, 2006. We closed the survey on August 22, 2006. 

We obtained survey responses from 78 of the 80 state VR agencies, for a 

response rate of 98 percent. 

Because this was not a sample survey, it has no sampling errors. However, 

the practical difficulties of conducting any survey may introduce errors, 

commonly referred to as nonsampling errors. For example, difficulties in 

interpreting a particular question or sources of information available to 

respondents can introduce unwanted variability into the survey results. 

We took steps in developing the questionnaire, collecting the data, and 

analyzing them to minimize such nonsampling error. For example, we 

pretested the content and format of our survey with officials from 17 state 

agencies to determine if it was understandable and the information was 

feasible to collect, and we refined our survey as appropriate. When the 

data were analyzed, an independent analyst checked all computer 

programs. Since the data were collected with a Web-based and Word 

format survey, respondents entered their answers directly into the 

electronic questionnaire, thereby eliminating the need to key data into a 

database, minimizing another potential source of error. 

The variables that we analyzed from the survey data are presented in table 

4. These included the structure of the agency (stand-alone agencies, 

umbrella agencies with and without autonomy over staff and finances, and 

others), agency staffing, agency management, indicators of the existence

of performance targets and incentives, specialized caseloads, case 

management systems and system components, and integration with 

outside partners and the business community. Since we had data on each

of the earnings outcomes and most of the state and agency characteristics 

for each of the 3 years, we included in our analysis an indicator for year.

Page 39 GAO-07-521 Vocational Rehabilitation
Appendix I: Scope and Methodology 

Table 4: Explanatory Variables from the VR Agency Survey Data 

Agency structure 

Agency structure 1—indicates whether agency is (1) part of an umbrella agency with autonomy over its own staff and finances, (2) 

part of an umbrella agency without autonomy over its own staff and finances, (3) a stand-alone agency, and (4) other type of agency 

Agency structure 2—indicates whether agency is part of an umbrella agency 

Agency structure 3—indicates whether agency is in an umbrella agency that was a part of (1) social services, (2) education, (3) labor 

(4) human services, (5) a stand-alone, or (6) other type of agency 

Agency staffing 

Percentage of service delivery sites staffed full-timea

Percentage of service delivery sites staffed part-timea 

Percentage of service delivery sites shared with social servicesa 

Percentage of service delivery sites shared with educationa

Percentage of service delivery sites shared with labora 

Percentage of service delivery sites shared with human servicesa 

Percentage of service delivery sites shared with other agenciesa 

Indicates whether the VR program experienced a hiring freeze in a given fiscal year 

Indicates whether the VR program experienced a large number of retirements in a given fiscal year 

Indicates whether the VR program experienced a large influx of new hires in a given fiscal year 

Indicates whether the VR program experienced downsizing through layoffs in a given fiscal year 

Indicates whether the VR program experienced unusual changes in staffing in a given fiscal year 

Indicates whether VR counselors were affiliated with a union in a given fiscal year 

Agency management 

Number of clients per VR counselora 

Number of counselors employed (proxy for agency size)

a

 

Indicates whether the director had authority over developmental disability services 

Indicates whether the director had authority over independent living services 

Indicates whether the director had authority over disability determination services 

Indicates whether the director had authority over other programs or services 

Percentage of counselors who left VR agency (turnover)a

Percentage of counselors meeting comprehensive system of personnel development (CSPD) standards

a

Percentage of senior managers who left VR agency (turnover)a 

Length of time director has held his/her position (director tenure)a 

Length of time director has been with the VR agency (director experience)a 

Length of time the director has held his/her position as a percent of their time at the agency

a

 

Indicates whether the agency operated under an order of selection 

Indicates whether the program had a wait list 

Length of wait list 

Indicates whether the program had a wait list and, if so, its length 

Page 40 GAO-07-521 Vocational Rehabilitation
Appendix I: Scope and Methodology 

Performance targets/incentives 

Scale indicating number of reported specific and numerical targets including SSA reimbursements, individual plans for employment 

(IPE) initiated, client referrals, contacts with businesses, client satisfaction, and other client employment outcomes by year 

Indicates whether counselors had performance expectations with numerical targets based on successful VR employment outcomes 

(status 26 closures) 

Nature of performance expectations 

Indicates whether counselors had numerical targets in their performance expectations 

Average number of status 26 case closures required for satisfactory performancea 

Indicates whether there were performance expectations that contained numerical targets for SSA reimbursements 

Indicates whether there were performance expectations that contained numerical targets for the number of IPEs initiated 

Indicates whether there were performance expectations that contained numerical targets for the number of client referrals 

Indicates whether there were performance expectations that contained numerical targets for the number of contacts made with 

businesses for job development 

Indicates whether there were performance expectations that contained numerical targets for client satisfaction rates 

Indicates whether there were performance expectations that contained numerical targets for any other outcomes 

Indicates whether there were monetary performance incentives to VR counselors 

Indicates how frequently a VR program reported on agencywide performance 

Specialized caseloads

Indicates whether there were in-house benefits counselors 

Number of benefits counselors

a

 

Indicates whether there were job development specialists 

Number of job development specialistsa 

Scale measuring the number of types of specialized caseloads covered, including transitioning high school students, mental health, 

developmental disabilities, traumatic brain/spinal cord injuries, hearing impairments, visual impairments (not counted for blind-serving 

agencies), or other groups 

Percentage of counselors with specialized caseloads serving transitioning high school studentsa 

Percentage of counselors with specialized caseloads serving clients with mental health issuesa 

Percentage of counselors with specialized caseloads serving clients with developmental disabilitiesa 

Percentage of counselors with specialized caseloads serving clients with traumatic brain/spinal cord injuriesa 

Percentage of counselors with specialized caseloads serving clients with hearing impairmentsa

Percentage of counselors with specialized caseloads serving clients with visual impairmentsa

Percentage of counselors with specialized caseloads serving any other group of clientsa 

Case management system 

Scale indicating the sophistication of the case management system including the ability of the case management system to collect 

Education data, collect fiscal data, generate IPEs, generate client letters, produce state-level management reports, and produce 

counselor-level management reports 

Indicates whether an agency used an automated case management system 

Indicates whether the automated case management system was new if an agency used one 

Indicates whether an agency used an automated case management system and if so, whether the system was new 

Indicates whether case management system could collect RSA 911 data 

Page 41 GAO-07-521 Vocational Rehabilitation
Appendix I: Scope and Methodology 

Indicates whether case management system could collect fiscal data 

Indicates whether case management system could generate IPEs 

Indicates whether case management system could generate client letters 

Indicates whether case management system could generate state level management reports 

Indicates whether case management system could generate reports at VR counselor level

Integration with outside partners 

Indicates whether any VR staff worked full-time or part-time at Workforce Investment Act (WIA) one-stops 

Total number of staff (both full- and part-time) that worked at a WIA site 

Indicates whether VR program purchased any services from public or private vendors 

Indicates how many purchased services had fee for service arrangements 

Indicates how many purchased services had contracts with outcome-based performance measures 

Indicates how many purchased services had vendor fees tied to meeting performance measures 

Indicates how many purchased services had renewal of their contracts tied to meeting performance measures 

Indicates how many purchased services were evaluated by VR to see whether performance measures were met at contract end 

Indicates how many purchased services were evaluated by VR by group or type of vendor 

Scale indicating the average support level received from different types of programs including WIA one-stops, social service 

departments, mental health departments, education systems, Medicaid program, Medicare program, substance abuse departments, 

and developmental disabilities programs 

Indicates the extent to which a VR program received support from the state WIA one-stop system 

Indicates the extent to which a VR program received support from state social services 

Indicates the extent to which a VR program received support from the state mental health department 

Indicates the extent to which a VR program received support from the state education system 

Indicates the extent to which a VR program received support from the state Medicaid program 

Indicates the extent to which a VR program received support from the state Medicare program 

Indicates the extent to which a VR program received support from the state substance abuse department 

Indicates the extent to which a VR program received support from the state development disabilities program 

Indicates the extent to which a VR program received support from another state program

Integration with business community 

Scale indicating agency’s level of integration with the business community, including the average frequency with which the agency 

sponsors job fairs, attends business network meetings, meets with local businesses, meets with chambers of commerce, interacts 

with civic clubs, and hosts employer breakfasts 

Frequency with which agency sponsored job fairs 

Frequency with which agency representatives attended job fairs 

Frequency with which agency representatives attended meetings of business networks 

Frequency with which agency met with local businesses 

Frequency with which agency met with local chambers of commerce 

Frequency with which agency representatives interacted with civic clubs 

Frequency with which agency hosted employer breakfasts 

Frequency with which agency representatives participated in other business outreach 

Source: GAO survey data. 

Page 42 GAO-07-521 Vocational Rehabilitation
Appendix I: Scope and Methodology 

a Indicates variables that were categorized. 

To determine whether the survey data were sufficiently reliable for our 

analysis, we collected and analyzed additional data. Specifically, we 

included questions in the survey that were designed to determine whether 

each state VR agency uses certain practices to monitor the quality of 

computer-processed data that were used to complete the survey.6

 From 

these questions, we developed a variable to indicate whether a particular

agency might have unreliable data. To determine whether there was a 

relationship between agencies with data reliability issues and the earnings

outcomes we were studying, we included this variable in our three models 

of earnings outcomes (described below). 

We found two issues associated with the survey data that are related to 

our findings. First, net of other effects, agencies that reported having a 

data reliability issue had significantly lower rates of SSA beneficiaries 

departing the disability rolls.7

 Although we suspect that data quality issues 

do not have a direct effect on the rates of SSA beneficiaries departing the 

rolls, poor data quality might be correlated with some other characteristic 

that we were not able to measure (e.g., agency efficiency), which may have 

an impact on the rate of departures from the rolls. Second, 11 agencies did 

not report the percentage of CSPD-certified counselors (a variable that we

found to be significantly related to the percentage of SSA beneficiaries 

with earnings during the year after completing VR) for at least 1 year. For 

these agencies, the percentage of counselors was imputed using the mean

derived from agencies that did report. Statistical tests were conducted to 

ensure that the observations for which data were imputed did not have 

significantly different rates of having earnings than those for which the 

data were not missing. 

 6

Specifically, we inquired about whether (1) there were written procedures that define data

elements or specify how the data for each data system were collected and if so, how well 

the procedures were followed; (2) anyone conducted routine internal reviews of the data to 

check for errors in completeness, accuracy, or reasonableness; (3) anyone independent of

the organization conducted periodic monitoring or audits of the data to check for errors in

completeness, accuracy, or reasonableness; and (4) there were any potential problems or 

limitations with the reliability of the data that were used to answer the survey questions. 

7

This variable was significant at the 0.10 level. 

Page 43 GAO-07-521 Vocational Rehabilitation
Appendix I: Scope and Methodology 

In consultation with SSA officials and contractors as well as Education 

officials, we selected as our study population working age individuals who 

(1) were either receiving Disability Insurance (DI) only, Supplemental 

Security Income (SSI) only, or both DI and SSI benefits concurrently; and 

(2) exited VR after having completed VR services.

8

 To use the most recent 

data available, we further refined this population to include those 

beneficiaries who 

Section 2: Study 

Population and 

Descriptive Analyses 

Study Population 

• Began receiving VR services no earlier than 1995 and who completed 

VR after having received services in fiscal years 2001 though 2003. 

• Had received a DI or SSI benefit payment at least once during the 3

months before application for VR services. Beneficiaries were defined 

as concurrent if they received both DI and SSI benefits for at least 1 

month in the 3 months before VR application. We selected a 3-month 

window to account for the fact that many beneficiaries, SSI 

beneficiaries in particular, fluctuate in their receipt of benefits for any 

given month.

We excluded from our study population those disability beneficiaries who 

• Completed VR after 2003, because we lacked at least 1 year of post-VR 

earnings data. 

• Applied for or started VR services, but did not complete VR. 

• Began receiving disability benefits after receiving VR services because 

these beneficiaries may have differed in certain important 

characteristics from those receiving benefits before VR participation. 

• Reached age 65 or died at any point in their VR participation or during 

the time frame of our study. We excluded the beneficiaries who died or 

reached age 65 because they would have left the disability rolls for 

reasons unrelated to employment. For example, beneficiaries who 

reach age 65 convert to SSA retirement benefits. 

 8

Our study population included disabled adult children and disabled widow(er)s, who may

receive DI benefits based on their parents’ or spouses’ Social Security earnings record. 

While their benefits are paid from the Old-Age and Survivors Insurance Trust Fund, these 

individuals are disabled and are eligible for VR services. 

Page 44 GAO-07-521 Vocational Rehabilitation
Appendix I: Scope and Methodology 

Using the Ticket Research File (TRF) subfile combined with data from 

SSA’s Master Earnings File (MEF), we computed three measures of 

earnings outcomes for the 2001 through 2003 exit cohorts for each state 

VR agency: (1) the percentage of beneficiaries who had earnings during 

the year after receiving VR services, (2) the average amount they earned,9

and (3) the percentage that left the disability rolls by 2005. The data 

sources for our three earnings outcomes or dependent variables are 

shown in table 5. 

Table 5: Dependent Variables Used in the Analyses 

Dependent 

 variable 

Dataset from which 

variable was derived

Percentage of beneficiaries with earnings during the year 

after VR 

MEF 

Average annual earnings for SSA beneficiaries among 

those with earnings during the year after exiting VR 

MEF 

Percentage of beneficiaries that left the rolls by 2005 TRF subfile 

Source: SSA data. 

To adjust for inflation, all of our earnings figures were computed in 2004 

dollars using the Consumer Price Index for All Urban Consumers (CPI-U). 

The CPI-U, maintained by the Bureau of Labor Statistics, represents 

changes in prices of all goods and services purchased for consumption by 

urban households. The CPI-U can be used to adjust for the effects of 

inflation, so that comparisons can be made from one year to the next using 

standardized dollars. We standardized the value of average annual 

earnings to 2004 dollars because this was the most recent year for which 

earnings data were available at the time of our analysis. 

To determine whether disability beneficiaries left the rolls before 2005, we 

used data from the TRF subfile that indicated the month in which a 

beneficiary left the rolls because of work. We included all beneficiaries 

who left the rolls after their VR application date. Concurrent beneficiaries 

were considered to have left the rolls only if they stopped receiving 

benefits from both programs. 

Computation of Dependent 

Variables

Departures from the 

Disability Rolls 

 9

To determine a beneficiary’s earnings in the year after VR, we calculated earnings in the 

calendar year after the year in which beneficiaries completed VR. For example, whether

the beneficiary completed VR in January or December 2000, earnings from January 2001 

through December 2001 would have been used to determine earnings in the year after VR. 

Page 45 GAO-07-521 Vocational Rehabilitation
Appendix I: Scope and Methodology 

To depict the variation of agency performance in earnings outcomes of 

SSA beneficiaries completing VR from 2001 to 2003, we performed two 

descriptive analyses. First, we developed distributions of each earnings 

outcome. Second, we computed the means and ranges of these outcomes 

by year and agency type. With data from 78 agencies over 3 years (from 

persons who exited the state VR programs from 2001 to 2003), we had 234 

cases in our data file.10 Both sets of analyses are presented in the findings 

section of the report. 

To identify key factors related to the earnings outcomes of SSA 

beneficiaries completing VR programs, we used econometric methods to 

analyze data from various sources related to VR agencies and the SSA 

beneficiaries who exited them from 2001 through 2003. Our econometric 

analyses focused on the differences across agencies for the three different 

dependent variables: (1) the percentage of beneficiaries who had earnings 

during the year after leaving VR; (2) among those with earnings, the 

average beneficiary earnings level during the year after leaving VR; and (3) 

the percentage of beneficiaries that left the disability rolls as a result of 

finding work by the end of 2005. 

Descriptive Analyses 

Section 3: 

Econometric Analyses 

We began our econometric analysis with ordinary least squares (OLS) and 

logistic regression models to analyze differences in outcomes based on 

individual characteristics. That is, we started with as many observations as 

there were individuals in our study population, each observation being 

assigned the characteristics of the agency as well as of the individual. 

Given that our data were multilevel (i.e., included information on both 

individuals and agency-level characteristics), we used statistical 

techniques to assess the feasibility of using ordinary least squares and 

logistic regression at the individual level rather than hierarchical modeling 

techniques.11 As a result of these analyses, we chose to use robust standard 

 10We have only 232 observations in our model of earnings because we considered average 

earnings among only those beneficiaries with earnings in the year following their exit from

VR. Two agencies did not have any beneficiaries with reported earnings from employment

in 2002. 

11We used Stata’s xtreg and rtlogit commands to calculate the intraclass correlation 

coefficient rho. These analyses revealed minimal clustering among individuals within 

agencies (rho of 0.02 and below); that is, individuals’ characteristics and employment 

outcomes appeared to vary as much within agencies as across agencies. This suggests that 

inferences derived from OLS and logistic regressions with robust standard errors are not

misleading as a result of failure to hierarchically account for clustering of individuals

within agencies. 

Page 46 GAO-07-521 Vocational Rehabilitation
Appendix I: Scope and Methodology 

errors to account for clustering in agencies rather than hierarchical 

modeling techniques. However, preliminary analyses using the individuallevel data to model binary outcomes and each individuals’ earnings 

revealed that regression and logistic models frequently failed statistical 

tests when compared to a null model with no explanatory variables, and 

only accounted for a small fraction of the variability outcomes of interest 

to us.12

Because our econometric models using individual-level data explained 

very little variation in earnings outcomes (i.e., low predictive power), we 

proceeded to model outcomes at the agency level. Specifically, we 

combined data on the aggregate characteristics of individuals within 

agencies (such as the percentage of female beneficiaries or Disability 

Insurance recipients within an agency) with agency-level data on 

structure, expenditures, and policies and practices. In other words, rather 

than assess whether individuals differed in the likelihood of getting a job 

or leaving the rolls or had different earnings, we analyzed whether the 

agencies’ earnings outcomes varied as a function of the characteristics of 

the agencies, the aggregate characteristics of beneficiaries within each

agency, and the characteristics of the states the agencies were located in.13

Our dependent variables thus contained, for each agency in a given fiscal

year, the average earnings level among those with jobs, the percentage at 

each agency who had earnings during the year after completing VR, and 

the percentage of those leaving the rolls due to work. 

As with our descriptive analysis, we had 234 cases in our data file, a 

number that was fairly small relative to the large number of agency

characteristics whose effects we wanted to estimate.14 We could not, as a 

 12For example, our multivariate models of earnings were only able to explain, at best, 

approximately 8 percent of the variation in individuals’ earnings. 

13Although the alternative of looking at individual outcomes with individual data might 

have allowed us to control for individual characteristics somewhat better before estimating 

the effects of the state and agency characteristics, we believe modeling the variability in 

outcomes using the aggregate data was more appropriate given the objective of assessing 

which agency-level characteristics are related to employment outcomes. However, because

aggregation reduces our degrees of freedom and may compound individual measurement 

error in variables such as earnings, we recognize that our estimated coefficients may not be

as precise as ones generated using individual characteristics. See section 4 of this appendix 

for more information on measurement issues. 

14We have only 232 observations in our model of earnings because two agencies did not 

have any beneficiaries with reported earnings from employment in 2002. 

Page 47 GAO-07-521 Vocational Rehabilitation
Appendix I: Scope and Methodology 

result, fit models that estimated the effects of all of the characteristics of 

interest simultaneously to determine which were statistically significant.

We therefore chose to proceed by first estimating, in a series of bivariate

regression models, which state and clientele characteristics (or 

characteristics of the types of SSA beneficiaries served in each agency) 

were significant. After obtaining preliminary estimates, we aggregated sets 

of significant state and clientele characteristics into single models for each 

of the three outcomes, and reassessed the significance of their net effects

when they were estimated simultaneously in a multivariate regression 

model.15 We next tested the stability and magnitude of statistically 

significant coefficients for the state and clientele characteristics under 

different model specifications, and proceeded to introduce the agency 

characteristics (e.g., structure, management, expenditures, etc.) one at a 

time into these base models with the significant state and case mix 

characteristics. After determining individually significant agency 

characteristics, we used an iterative procedure to reassess agency-level 

effects by testing model stability and which variables were and were not 

significant when others were included, and retesting the significance of 

selected state, case mix, and agency characteristics that were marginally 

significant in prior models.16 In all cases we used robust regression 

procedures to account for the clustering of cases within agencies (i.e., the 

lack of independence within agencies over time), and weighted the cases 

in our analyses according to either the total number of beneficiaries in 

each agency in each year (for models of having earnings or leaving the 

rolls) or the total number of beneficiaries with earnings due to work in 

each year (for models of earnings). 

Ultimately, we obtained the models shown in table 6. Each of the models

consisted of 7 to 9 characteristics that jointly accounted for between 66 

and 77 percent of the variability in each dependent variable. Although 

certain characteristics were significant in some specifications for each 

 15Statistical significance was measured at a p-value <0.05 and marginal significance was 

measured at p-value <0.10. 

16Although we considered this full range of variables in the series of models leading to our 

final specifications for each outcome, not all variables are significant for each outcome. We

used a variety of factors to decide which characteristics to include or exclude in the model

for each outcome. We considered statistical significance, magnitude of each effect, stability 

of included coefficients across model specifications with different regressors, changes in

the proportion of variance explained using F-tests for nested models (when appropriate),

and theoretical considerations based on past research and input from agencies we

surveyed and interviewed. 

Page 48 GAO-07-521 Vocational Rehabilitation
Appendix I: Scope and Methodology 

outcome, the limited degrees of freedom prevented us from including all 

but the most consistently significant variables with greatest stability 

across models. In the models that estimated factors affecting the 

percentage of SSA beneficiaries who had earnings and factors affecting 

average earnings, state characteristics accounted for a substantial portion

of the explained variance. Although state characteristics were also 

important in the model estimating the percentage getting off the rolls by 

2005, the year that beneficiaries exited the agency accounted for the 

greatest portion of the variance explained, a result reflecting that those 

who exited the rolls earlier had more time to do so. 

Page 49 GAO-07-521 Vocational Rehabilitation
Appendix I: Scope and Methodology 

Table 6: Coefficients for Multivariate Models Estimating the Effects of State and Agency Characteristics on Three VR 

Outcomes, and the Proportion of Variance Explained (R-Squared) by Each Model 

Significant explanatory variables for percentage of beneficiaries 

with earnings during the year after VR (R-squared = 0.66) 

Effect

 coefficient

Robust

 standard error P-value

Unemployment rate -2.22 .358 <.001

Per capita income (per $10,000) 3.90 1.42 .008

Population size (per 1 million) -.40 .047 <.001

Percentage of female beneficiaries -.508 .202 .014

Combined agency 8.08 1.95 <.001

General agency 10.35 2.22 <.001

Proportion of SSA beneficiaries served .397 .140 .006

Percentage of counselors meeting CSPD requirements 5.63 2.06 .008

In-house benefits counselor -.3.61 1.251 .005

Constant 60.19 9.89 <.001

Significant explanatory variables for average 

earnings among SSA beneficiaries (R-squared = 0.77) 

Effect

coefficient

Robust

 standard error P-value

Per capita income (per $10,000) 1684.54 185.25 <.001

Percentage of beneficiaries with mental impairments -82.16 3.61 <.001

Percentage of beneficiaries on Disability Insurance -64.52 8.20 <.001

Agency integration with business community 727.37 350.63 .041

Degree of support/cooperation with other agencies 858.15 249.87 .001

Percentage of expenditures on training 12.54 3.95 .002

Proportion of SSA beneficiaries served -27.55 11.77 .022

Constant 9059.65 824.75 <.001

Significant explanatory variables for percentage of beneficiaries 

leaving the disability rolls (R-Squared = 0.76) 

Effect

coefficient

Robust

standard error P-value

Exit year 2002 -1.89 .170 <.001

Exit year 2003 -4.07 .194 <.001

Per capita income (per $10,000) 1.98 .326 <.001

Population size (per 1 million) -.068 .014 <.001

Percentage of beneficiaries on Disability Insurance -.040 .016 .021

Percentage of beneficiaries 46 to 55 years of age -.078 .043 .073

Percentage of beneficiaries with mental impairments -.084 .016 <.001

Percent of beneficiaries with visual impairments -.045 .010 <.001

Agency integration with business community 1.62 .788 .044

Constant 11.64 1.58 <.001

Source: GAO analysis of SSA, Education, GAO survey, and data from various sources listed in table 3. 

Page 50 GAO-07-521 Vocational Rehabilitation
Appendix I: Scope and Methodology 

Note: While earnings are coded in units of dollars, per capita income is coded in units of $10,000 so 

that the coefficient represents the effect of a $10,000 change in per capita income. Population is 

coded in per million persons. Percentage and proportion variables are coded between 0 and 100; this

includes the percentage of beneficiaries with earnings, leaving the rolls, on Disability Insurance, and 

with mental and visual impairments, as well as the percentage of agency budget spent on training 

and the percentage of CSPD-certified counselors. Agency integration with the business community 

and support and cooperation with other agencies are scaled between 0 and 1. 

Taking each outcome one at a time, the coefficients in table 6 suggest the 

following: 

• The percentage of SSA beneficiaries who had earnings was 

significantly lower in more populous states and states with higher 

unemployment rates, and significantly higher in states with higher per 

capita incomes. The percentage of SSA beneficiaries who exited VR 

agencies and had earnings was also significantly lower in agencies in 

which greater percentages of the beneficiaries are female. Independent 

of these effects of the state in which the agencies are located, and the 

gender composition of the beneficiaries who exit the agencies, there 

were also significant net effects of certain agency characteristics. 

Agencies that served only blind beneficiaries had lower percentages of 

SSA beneficiaries who had earnings than combined agencies or general 

agencies that did not serve the blind. The percentage of SSA 

beneficiaries who had earnings was also higher in agencies that served 

a higher proportion of SSA beneficiaries and had a higher percentage of 

counselors who met CSPD requirements, but lower in agencies that 

had an in-house benefits counselor. 

• Among the SSA beneficiaries who had earnings, those who were in 

agencies in states that had higher per capita incomes had higher 

average incomes. Net of these effects, agencies that (1) were more 

integrated with the business community, (2) had a higher degree of 

support and cooperation from other agencies, and (3) spent more of 

their total budget on training had higher average annual incomes 

among SSA beneficiaries completing VR services. Agencies with a 

higher percentage of beneficiaries on Disability Insurance, with mental 

impairments, and higher proportions of SSA beneficiaries served had 

lower average annual incomes among SSA beneficiaries completing VR 

services. 

• With respect to leaving the disability rolls by 2005, our final model 

showed that beneficiaries who exited agencies more recently were less 

likely to leave the rolls by 2005. (See section 4 for an explanation of 

why this might be the case.) Net of this, agencies in states with larger 

populations had lower percentages of beneficiaries who left the rolls by 

2005, while agencies in states with higher per capita incomes had 

Page 51 GAO-07-521 Vocational Rehabilitation
Appendix I: Scope and Methodology 

higher percentages who left the rolls by 2005. The characteristics of 

clientele served by each agency had a significant effect on the

percentage of SSA beneficiaries who got off the rolls by that year. 

Agencies with higher percentages of beneficiaries who were blind or 

had mental impairments had lower percentages getting off the rolls by 

2005. Those agencies with a higher proportion of DI beneficiaries had 

lower percentages that got off the rolls by 2005. The only agency 

characteristic with consistent statistical significance was the 

integration with the local business community; a greater proportion of 

beneficiaries in agencies with better integration with businesses left 

the rolls.17 Agencies with a greater proportion of beneficiaries from 46 

through 55 years of age had fewer recipients leaving the rolls, but this 

effect is only significant at the 90 percent confidence level in our final 

model. 

Our results cannot be generalized to the larger population of all SSA 

disability beneficiaries or all VR participants because we looked only at VR

participants who were SSA beneficiaries. Because VR participation is 

voluntary, beneficiaries who participate in VR may have certain

characteristics that make them different from other SSA beneficiaries and,

therefore, more likely or less likely to succeed in the workforce. 

Section 4: Limitations 

of our Analyses

Because our primary goal was not to conduct an impact evaluation of the 

VR program, but rather to conduct a comparative analysis of earnings 

outcomes across state VR agencies to determine what might account for 

differences in state agency performance, we felt that our analysis did not 

require a control group of SSA beneficiaries that did not receive VR 

services. Nonetheless, as a secondary analysis goal, we attempted to 

identify such a group using the data that were available to us on SSA 

beneficiaries that had applied for but did not receive VR services. 

However, we were unable to identify a subset of this group that was 

sufficiently similar to the VR participants to feel confident that any 

differences in earnings outcomes that we found between them and those 

that completed the VR program would be attributable to the VR program 

and not to the differences in individual characteristics. Therefore, our 

findings do not allow us to report on the overall effectiveness of the VR 

program. 

 17Several other agency characteristics, notably the percentage of expenditures on 

purchased services, had marginally significant effects and were not included in the final

model. 

Page 52 GAO-07-521 Vocational Rehabilitation
Appendix I: Scope and Methodology 

Our earnings data had several limitations that may have resulted in an 

under- or overestimation of beneficiaries’ earnings. For example, although 

the beneficiary earnings data were provided to SSA by the Internal 

Revenue Service and are considered to be the most comprehensive and 

accurate measure of earnings available, they excluded several categories 

of workers who participated in alternative retirement systems and whose 

earnings may not have been reported to SSA.18 Such omissions could have 

resulted in an under- or overestimate of beneficiary earnings. On the other 

hand, some earnings reported to SSA may have included income derived 

from work activity in a previous year, such as commissions or bonuses. 

Further, the earnings data included some forms of nonwork income, such

as sick leave earnings and profit sharing. These additional sources of 

income could not be identified and, therefore, could result in an 

overestimation of beneficiaries’ earnings in a particular year. The data did

not allow us to estimate the magnitude of the effect of these factors on our 

analyses. 

Our findings that beneficiaries receiving DI and beneficiaries from later 

cohort years were less likely to leave the rolls are likely due to several 

factors related to program structure and the updating of data. First, under 

current program rules, DI beneficiaries are allowed a trial work period (9 

months) and an extended period of eligibility (36 months) before they are 

considered off the rolls.19 SSI beneficiaries who earn enough so that they 

do not receive a benefit for 12 months are taken off the rolls. Therefore, 

given that we measured whether beneficiaries left the rolls by 2005, 

beneficiaries from earlier cohort years would have had more time to leave 

the rolls. Further, by 2005 many DI beneficiaries may not yet have entered 

or completed their extended period of eligibility or reached the point 

where they would have been considered off the rolls. 

In addition, delays in the reporting of earnings may also have contributed 

to our finding that relatively more SSI beneficiaries and beneficiaries from 

earlier cohort years left the rolls. There can be a significant delay—up to 3 

years—between when beneficiaries begin work and when SSA is notified 

or learns of their earnings. This delay is more likely to occur with DI 

beneficiaries, whose earnings were reviewed on a yearly basis as 

compared to monthly earnings reviews for SSI beneficiaries during the 

 18Workers who may have been excluded include federal civilian employees hired before

1984 and certain state and local government employees. 

19The 9-month trial work period must occur within a 60-month period. 

Page 53 GAO-07-521 Vocational Rehabilitation
Appendix I: Scope and Methodology 

time frame of our study. Because of this reporting delay, the TRF subfile 

data that indicated whether a beneficiary had left the rolls may not have 

contained completely up-to-date data, especially for later cohorts. For 

these two reasons, we did not include comparisons of the rates of 

departures from the disability rolls by exit year because they would be 

misleading. 

With respect to earnings after VR, we included all earnings in the calendar

year after VR, irrespective of the time gap between VR completion and the 

first month of earnings. Therefore, the start month for calculating earnings 

in the year after VR could have ranged from 1 to 12 months after VR, 

depending on which month the beneficiary exited. In other words, 

beneficiaries who exited VR in January 2000 would have their 2001 annual

earnings calculated beginning in January 2001—12 months after their exit

from VR. In contrast, beneficiaries who completed VR in December 2000 

would have been out of VR for 1 month when their 2001 annual earnings 

calculation started in January 2001. We have no indication of clustering in

earnings relative to VR completion and therefore expect a fairly even 

distribution of earnings over time. We do not expect the time lag in the 

earnings calculation to vary systematically by year or cohort. 

Finally, our analysis of the impact of agency characteristics on earnings 

outcomes of particular exit cohorts was limited by the time frames of our 

agency data. Specifically, we used information on the agency that 

pertained to the year before each exit cohort completed VR to explain the 

earnings outcomes of that exit cohort (e.g., agency data from 2000 were 

used to explain 2001 beneficiary outcomes). We did this for two reasons. 

First, beneficiaries, on average, receive services from VR for 

approximately 2 years. Therefore, for any given exit cohort, data on the 

agency from the year prior to exit will cover the most beneficiaries in that 

exit cohort. Second, although data from previous years might also explain 

beneficiary outcomes for a given cohort, we did not want to impose an 

inordinate burden on our survey respondents by collecting data on many 

years, especially those prior to 2000. In conducting preliminary tests of our 

survey questions, we also learned that the quality of the data may have

been lower in earlier years because some agencies retain data for a limited 

time period. 

Page 54 GAO-07-521 Vocational Rehabilitation
Appendix II: Comments from the Department Appendix II: Comments from the Department of Education 

of Education 

Note: GAO’s comments 

supplementing those in 

the report text appear at 

the end of this appendix. 

See comment 1. 

Page 55 GAO-07-521 Vocational Rehabilitation
Appendix II: Comments from the Department 

of Education 

See comment 2. 

See comment 3. 

Page 56 GAO-07-521 Vocational Rehabilitation
Appendix II: Comments from the Department 

of Education 

See comment 4. 

See comment 5. 

See comment 6. 

Page 57 GAO-07-521 Vocational Rehabilitation
Appendix II: Comments from the Department 

of Education 

See comment 7. 

Page 58 GAO-07-521 Vocational Rehabilitation
Appendix II: Comments from the Department 

of Education 

See comment 8. 

See comment 9. 

Page 59 GAO-07-521 Vocational Rehabilitation
Appendix II: Comments from the Department 

of Education 

Page 60 GAO-07-521 Vocational Rehabilitation
Appendix II: Comments from the Department 

of Education 

The following are GAO’s comments on the Department of Education’s 

letter dated April 17, 2007. 

1. Education noted that more contextual information would help readers 

better understand the data and findings in the report. We added additional 

information to the report about VR reimbursement for successful SSA 

beneficiary rehabilitations, the role of disability determination services in 

VR referral, as well as a reference on where to find more information on 

the structure of federal disability programs. 

GAO Comments 

2. We disagree with Education that our measures of statistical significance

do not necessarily translate into issues of programmatic significance for 

the VR program because it is a formula grant program or that the agency

characteristics we identify as having a significant impact on agency-level 

performance with respect to SSA beneficiaries may be overstated. Given 

Education’s important leadership role in overseeing the VR agencies, we 

believe that our findings are relevant to the guidance and information that 

Education may choose to provide to state VR agencies. While we 

acknowledge that many SSA disability beneficiaries will not be able to 

return to work and leave the rolls for a variety of reasons, such as the 

severity of their disability, we analyzed numerous versions of our models 

and only reported on the relationships that were consistently significant 

across many versions of the model. As such, we believe these 

relationships are valid and deserve careful consideration. 

3. Education stated that there is a significant difference between helping 

more SSA beneficiaries participate in the workforce versus helping more 

leave the disability rolls. While we agree, we believe that participating in

the workforce is an important first step and improves SSA beneficiaries’ 

potential for leaving the rolls. 

4. We agree with Education that our description of the states’ CSPD 

certifications could be misconstrued. We clarified the CSPD language in 

the background, findings, and recommendation sections of the report. 

5. Education stated that overall collaboration between VR agencies and

other agencies providing complementary services may improve outcomes, 

but that some collaboration resulting in better services to individuals with 

disabilities may not always support more or better rehabilitations for SSA 

beneficiaries. When we tested the effect of receiving support from specific 

agencies on SSA beneficiary outcomes, we did not find support from 

individual agencies to be significant. (See table 4 in app. I under 

Page 61 GAO-07-521 Vocational Rehabilitation
Appendix II: Comments from the Department 

of Education 

“Integration with Outside Partners” for a list of variables we tested.) 

However, we found that when these relationships were aggregated, 

agencies that received a greater degree of support from more than one 

public agency had significantly higher levels of earnings among SSA 

beneficiaries. 

6. Education suggested that VR agencies with high proportions of SSA 

beneficiaries may also have high levels of collaboration with other 

agencies because SSA beneficiaries may require long-term supports to live 

in the community, which may in turn necessitate cooperation with other 

public programs. The department noted that this may account for our 

findings because benefit eligibility may be necessary to receive certain 

supports from outside agencies. We added a footnote with this potential 

explanation to the report.

7. Education noted that we questioned the validity of certain self-reported 

data in this report, but deemed similar data acceptable in our 2005 VR 

report.1

 In relation to our 2005 report, this report references different selfreported data for different purposes. Specifically, this report refers to 

clients’ earnings data at the time of application, whereas our 2005 report 

used clients’ earnings data after exiting VR (which was not used in this 

report). More importantly, this report used data as part of an econometric 

model whereas our 2005 report used self-reported data for descriptive 

purposes. While it is always preferable to verify self-reported data, our 

reliability tests are limited to our intended use of the data, and the data’s 

reliability for that purpose. Education said it was open to our 

recommendation, but sensitive to the reporting burden on the VR 

agencies. Our recommendation that Education explore cost-effective ways 

to validate self-reported data was based on the experience of some VR 

agencies that have obtained data successfully from official sources and not 

relied solely on self-reported information. 

8. Education disagreed with our recommendation on when economic 

conditions and state demographics should be considered in assessing 

agency performance. Instead of using this information to help set 

performance measures, the department said that it takes these factors into

account when it monitors agency performance results and believes that its 

approach is effective. However, on the basis of the statistical significance

of economic factors in our analysis, we believe that incorporating this 

 1

GAO-05-865. 

Page 62 GAO-07-521 Vocational Rehabilitation
Appendix II: Comments from the Department 

of Education 

contextual information in assessing performance measures is essential to 

provide state agencies with a more accurate picture of their relative

performance. Education also stated that the VR program’s statutory 

funding formula allocates relatively more funds to poorer states based on

per capita income to offset the lack of other resources in the state. 

However, if the additional funds allocated to VR agencies located in states 

with low per capita incomes actually offset the lack of other resources in 

the state, we would not have found a significant relationship between per

capita income and state VR performance. Finally, Education stated the 

overall state unemployment rate may not entirely correspond to the jobs 

available to SSA beneficiaries. In our analysis, however, this variable was 

highly significant in explaining the percentage of SSA beneficiaries with 

earnings for state VR agencies. 

9. We agree with Education that the report’s title should indicate that our 

sample was limited to SSA beneficiaries and we modified the title 

accordingly.

Page 63 GAO-07-521 Vocational Rehabilitation
Appendix III: Comments from the Social Appendix III: Comments from the Social Security Administration 

Security Administration 

Note: GAO’s comments 

supplementing those in 

the report text appear at 

the end of this appendix. 

See comments 1 and 2. 

See comment 5. 

Page 64 GAO-07-521 Vocational Rehabilitation
Appendix III: Comments from the Social 

Security Administration 

See comment 1. 

See comment 2. 

Page 65 GAO-07-521 Vocational Rehabilitation
Appendix III: Comments from the Social 

Security Administration 

See comment 3. 

Page 66 GAO-07-521 Vocational Rehabilitation
Appendix III: Comments from the Social 

Security Administration 

See comment 4. 

See comment 5. 

See comment 2. 

See comment 6. 

Page 67 GAO-07-521 Vocational Rehabilitation
Appendix III: Comments from the Social 

Security Administration 

See comment 2. 

See comment 7. 

See comment 8. 

See comment 2. 

Page 68 GAO-07-521 Vocational Rehabilitation
Appendix III: Comments from the Social 

Security Administration 

See comment 9. 

Page 69 GAO-07-521 Vocational Rehabilitation
Appendix III: Comments from the Social 

Security Administration 

The following are GAO’s comments on the Social Security 

Administration’s letter dated April 13, 2007. 

1. We disagree with SSA that many of the comments provided on our 

previous report (GAO-07-332) apply to this report because the methods 

and data we used differed significantly from our earlier report. Prior to 

submitting this report for agency comment, we carefully reviewed and 

incorporated any comments from the earlier report that were relevant. 

GAO Comments 

2. SSA stated that the report has methodological flaws that introduced 

aggregation bias and false correlations. It suggested that we should have 

focused on individual-level analysis or reported the results of both the 

individual- and aggregate-level analyses. We disagree, as the primary goal 

of our analysis was to analyze agency-level outcomes, not individual-level 

outcomes. Specifically, our objective was to understand what “may 

account for the wide variation in state VR agency outcomes with respect 

to SSA beneficiaries.” In doing so, we used aggregated data, which is a 

widely used and, at times, necessary means of analysis throughout all 

social sciences. Because we used aggregated data, we did not attempt to 

infer the effects of individual behavior or individual outcomes and noted 

such in our report. For example, we did not find that a lower percentage of 

women beneficiaries had earnings relative to male beneficiaries. Rather, 

we found that agencies serving a higher proportion of women 

beneficiaries had lower percentages of beneficiaries with earnings relative 

to other agencies. 

We did not report the results from the individual-level analyses, as 

recommended by SSA, because we did not find them sufficiently reliable

upon which to base findings. Specifically, we did not find the individuallevel results to be reliable, as we were not able to control for some factors 

at the individual level—for example, severity of disability—that were 

crucial to an individual-level analysis, but not crucial to analyses at the 

aggregated level. Although we chose not to report individual-level results, 

they were, in fact, consistent with the results of our aggregate analyses. 

We conducted statistical tests prior to our agency-level analyses to ensure 

that our aggregate analyses were not biased by a failure to account for 

certain types of correlations between individuals within agencies. Our 

tests did not reveal such correlations. In the absence of such correlations, 

Page 70 GAO-07-521 Vocational Rehabilitation
Appendix III: Comments from the Social 

Security Administration 

several respected authorities agree that aggregate-level analyses that 

incorporate aggregated individual-level characteristics will not result in 

biased estimates.1

 To further ensure our methods were appropriate and 

robust, our final report was reviewed and validated by an expert in 

statistical analysis.2

3. We agree with SSA that state agency rules about whether and how 

disability beneficiaries are referred to VR may have an affect on agency 

success rates, and controlled for it to the extent possible in our analysis. 

While we were not able to control for differences in the way states target 

beneficiaries for referral to VR as SSA suggested, we did include a variable 

reflecting the percentage of SSA beneficiaries served by a VR agency

(computed as the percentage of all clientele served at that agency). This 

variable was significant in two of our three models. 

4. SSA had concerns that the Ticket to Work program was implemented 

during the time frame of our study and should have been controlled for in 

our analysis. Although there was a very slight overlap between the time 

frame of our study and the timing of the Ticket to Work program, we 

nevertheless conducted tests to determine whether the rollout of the 

Ticket program had an effect on VR agency outcomes for SSA 

beneficiaries. The rollout of the Ticket program was not significant and,

therefore, we did not report its effect. 

5. SSA questioned the value of measuring state-level economic factors and 

the resultant implications for VR. Our findings on the influence of state 

economic characteristics were highly significant and are corroborated by 

previous research, and therefore we believe that implications and 

recommendations can be offered from our analysis of state economic 

factors on agency-level outcomes. However, nowhere in our report do we 

indicate that our findings suggest that during times of high unemployment, 

less funding should be allocated to VR agencies. To the contrary, we 

 1

See, for example, Leigh Burstein, “The Analysis of Multilevel Data in Educational Research 

and Evaluation,” Review of Research in Education, vol. 8, p. 158–233 (1980); Judith Singer, 

“Using SAS PROC MIXED to Fit Multilevel Models, Hierarchical Models, and Individual 

Growth Models,” Journal of Educational and Behavioral Statistics, vol. 24, no. 4 (1998); 

and Stephen W. Raudenbush and Anthony S. Bryk, Hierarchical Linear Models: 

Applications and Data Analysis Methods, second ed. (Thousand Oaks, California: Sage

Publications, 2002), 99-159. 

2

Herbert Smith, Professor of Sociology, Director of Population Studies, University of 

Pennsylvania. 

Page 71 GAO-07-521 Vocational Rehabilitation
Appendix III: Comments from the Social 

Security Administration 

suggest that economic factors should be controlled, or accounted for, 

when assessing agency performance. Moreover, while we agree that 

economic conditions are associated with tax revenues, we found that total 

state agency expenditures on services (and several other expenditure 

variables listed in table 2 of app. I) were not significant predictors of 

agency-level earnings outcomes for SSA beneficiaries. 

6. SSA had concerns about our findings on benefits counselors because 

they differed from those of other research. While prior research focused 

on the impact of benefits counseling in one state, our analysis focused on 

the impact of benefits counseling across all state agencies. Additionally, 

we noted that the time frame of our study was a period of transition for 

the benefits counseling program. Therefore, while we believe our findings 

are accurate, we also noted the contradictory findings in other research.

7. We agree with SSA that the higher earnings thresholds for the legally 

blind allow them to earn more than other categories of workers with 

disabilities while still keeping their disability benefit and have modified 

our explanation of the results on beneficiaries who are blind.

8. SSA stated that SSI beneficiaries generally lack a work history that 

qualifies them for DI benefits, and that this lack of work history is more 

likely to lead SSI beneficiaries into entry-level jobs, resulting in lower 

average annual earnings than DI beneficiaries. While we agree past work 

history can be a contributing factor, we found the opposite effect. We 

found that agencies with a higher proportion of SSA beneficiaries who 

were DI beneficiaries had lower average annual earnings among SSA 

beneficiaries and a lower percentage of beneficiaries leaving the rolls. We 

offer potential explanations for these results in the report. 

9. We incorporated SSA’s technical comments as appropriate. 

Page 72 GAO-07-521 Vocational Rehabilitation
Appendix IV: 

A

GAO Contacts and Staff 

cknowledgments 

Page 73 GAO-07-521

Appendix IV: GAO Contacts and Staff 

Acknowledgments 

Denise M. Fantone, Acting Director, (202) 512-7215, fantoned@gao.gov

In addition to the contact named above Robert E. Robertson, Director; 

Michele Grgich, Assistant Director; Amy Anderson; Melinda Cordero; Erin 

M. Godtland; Jay Grussing; Robert Marek; Brittni Milam; Nisha R. Unadkat; 

and Rick M. Wilson made significant contributions to all phases of this 

report. In addition, Susan Bernstein, Cindy Gilbert, Lisa Mirel, Thomas 

McCool, Anna Maria Ortiz, Daniel A. Schwimer, Doug Sloane, and Shana B. 

Wallace provided technical assistance. 

GAO Contact 

Acknowledgments 

Vocational Rehabilitation
Related GAO Products Related GAO Products 

Vocational Rehabilitation: Earnings Increased for Many SSA 

Beneficiaries after Completing VR Services, but Few Earned Enough to 

Leave SSA’s Disability Rolls. GAO-07-332. Washington, D.C.: March 2007.

Vocational Rehabilitation: Better Measures and Monitoring Could 

Improve the Performance of the VR Program. GAO-05-865. Washington,

D.C.: September 2005.

SSA Disability: SGA Levels Appear to Affect the Work Behavior of 

Relatively Few Beneficiaries, but More Data Needed. GAO-02-224. 

Washington, D.C.: January 2002.

SSA Disability: Other Programs May Provide Lessons for Improving 

Return-to-Work Efforts. GAO-01-153. Washington, D.C.: January 2001. 

Social Security Disability Insurance: Multiple Factors Affect

Beneficiaries’ Ability to Return to Work. GAO/HEHS-98-39. 

Washington, D.C.: January 1998.

Social Security: Disability Programs Lag in Promoting Return to Work. 

GAO/HEHS-97-46. Washington, D.C.: March 1997.

SSA Disability: Program Redesign Necessary to Encourage Return to 

Work. GAO/HEHS-96-62. Washington, D.C.: April 1996. 

Vocational Rehabilitation: Evidence for Federal Program’s Effectiveness 

is Mixed. GAO/PEMD-93-19. Washington, D.C.: August 1993. 

(130534) Page 74 GAO-07-521 Vocational Rehabilitation
GAO’s Mission The Government Accountability Office, the audit, evaluation and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To

have GAO e-mail you a list of newly posted products every afternoon, go 

to www.gao.gov and select “Subscribe to Updates.” 

The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, D.C. 20548 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Mail or Phone 

To Report Fraud,

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Public Affairs 

PRINTED ON RECYCLED PAPER