MODERNIZING SSA 

DISABILITY 

PROGRAMS

Progress Made, but 

Key Efforts Warrant 

More Management 

Focus

Report to the Chairman, Subcommittee 

on Social Security, Committee on Ways 

and Means, House of Representatives

June 2012

GAO-12-420

United States Government Accountability Office

GAO
United States Government Accountability Office

Highlights of GAO-12-420, a report to the 

Chairman, Subcommittee on Social Security, 

Committee on Ways and Means, House of 

Representatives 

June 2012 

MODERNIZING SSA DISABILITY PROGRAMS

Progress Made, but Key Efforts Warrant More 

Management Focus

Why GAO Did This Study

SSA administers two of the largest 

federal disability programs. GAO 

designated federal disability programs 

as a high-risk area, in part because 

eligibility criteria had not been updated 

to reflect medical and technological 

advances and labor market changes. 

Given the size and cost of its disability 

programs, SSA needs updated criteria 

to appropriately determine who 

qualifies for benefits. GAO has been 

asked to assess SSA’s efforts to 

update its medical criteria and develop 

a new occupational information 

system, and to identify other steps 

taken to modernize disability 

determination criteria. To do this, GAO 

reviewed relevant publications and 

federal laws and regulations; assessed 

agency plans, cost estimates, 

schedules, and other documentation 

against established project 

management criteria; and interviewed 

SSA officials, experts, and 

stakeholders.

What GAO Recommends

GAO recommends that SSA (1) 

explicitly identify resources needed to 

achieve its 5-year time frame for 

updating its medical listings; (2) follow 

best practices in its cost estimate, 

schedule, and risk assessment for the 

occupational information system; and 

(3) conduct limited, focused studies on 

how to more fully consider assistive 

devices and workplace 

accommodations in its disability 

determinations. SSA agreed with the

first two recommendations and 

disagreed with the third, stating that 

such studies would be inconsistent 

with Congress’ intentions. GAO 

continues to believe the 

recommendation has merit, as 

discussed more fully within the report.

What GAO Found

The Social Security Administration (SSA) has taken steps that hold promise for 

improving the process for updating its medical criteria, but continues to face 

challenges ensuring timely updates. SSA now uses a two-tiered system for 

ongoing revisions to its medical listings. First, it completes a comprehensive 

review of all medical conditions listed within each of 14 body systems, making 

needed revisions. For subsequent updates for a body system, the agency uses a 

targeted approach, selecting for review and revision only those medical 

conditions most in need of change. To date, SSA has completed comprehensive 

revisions for 8 of the 14 body systems and now is reviewing conditions under 

them to determine where targeted revisions are appropriate. However, some of 

these targeted revisions have experienced delays. Moreover, SSA has yet to 

complete comprehensive revisions for six body systems that have been ongoing 

for 19 to 33 years. SSA officials attributed delays to a lack of staff and expertise, 

along with the complexity and unpredictability of the regulatory process. 

SSA has embarked on an ambitious plan to design by 2016 an occupational 

information system for use in its disability decision-making process, but has fallen 

short of best practices for estimating costs, maintaining a schedule, and 

considering risks and alternatives. SSA currently relies on occupational 

information developed by the Department of Labor which has not had a major 

update since 1977. In 2008, SSA initiated a project to develop its own 

occupational information system (OIS), which SSA expects will provide up-todate information on the physical and mental demands of work to support its 

decision-making process. To guide the creation of its OIS, SSA established an 

advisory panel, collaborated with outside experts and other agencies, and in July 

2011 issued a research and development plan detailing relevant activities 

through 2016. SSA has made progress on some baseline activities in the plan. 

However, SSA’s cost estimate and schedule had key deficiencies, such as not 

including any estimate of the cost of producing, maintaining, and operating the 

system, which can inform design options. SSA also did not adequately consider 

inherent risks or potential alternatives, which could heighten the risk of additional 

costs or project failure.

Consistent with modern views of disability, SSA has taken some concrete steps 

toward greater consideration of an individual’s ability to function with a disability 

but faces constraints in fully modernizing. SSA has incorporated some criteria 

into its medical listings to determine whether a claimant’s impairments result in 

functional limitations that can prohibit the ability to work. SSA is also sponsoring 

research through the National Institutes of Health to evaluate how functional 

abilities can further be considered in determining disability. One project aims to 

develop a computerized tool to assist adjudicators in evaluating how various 

impairments affect an individual’s function and ability to work. However, SSA 

officials maintain that other modern concepts of disability cannot be fully 

incorporated into SSA’s disability decisions. Specifically, SSA faces constraints 

considering the extent to which assistive devices and workplace 

accommodations can mitigate work disability, because these are not universally 

available and SSA lacks the resources to conduct individualized assessments. View GAO-12-420. For more information, 

contact Daniel Bertoni at (202) 512-7215 or 

bertonid@gao.gov. 
Page i GAO-12-420 Modernizing SSA Disability Criteria 

Letter 1 

Background 2

SSA Has Improved Its Process for Updating Medical Listings, but 

Still Faces Challenges with Timely Updates 7

SSA Has Begun an Ambitious Project to Develop Updated 

Occupational Information, but Many Uncertainties Remain 14

SSA Has Taken Other Steps to Modernize Criteria but Faces 

Limitations 25

Conclusions 30

Recommendations for Executive Action 31

Agency Comments and Our Evaluation 31

Appendix I Objectives, Scope, and Methodology 35

Appendix II Progress Updating Six Body Systems since Last 

Comprehensive Revision 38

Appendix III Description of Cost Estimating Best Practices 39

Appendix IV Description of Scheduling Best Practices 40

Appendix V Comments from the Social Security Administration 41

Appendix VI GAO Contact and Staff Acknowledgments 44

Tables

Table 1: Key Dates for Comprehensive Revisions to Body System 

Listings 11

Table 2: Status of Select OIS Activities as of May 2012 17

Table 3: Assessment of OIS Cost Estimate 20

Table 4: Assessment of OIS Schedule 21

Contents
Page ii GAO-12-420 Modernizing SSA Disability Criteria 

Table 5: Six Sound Project Management Practices Used to Evaluate 

SSA’s Efforts 36

Figures

Figure 1: SSA’s Five-Step Sequential Evaluation Process for 

Determining Disability 4

Figure 2: Status of Comprehensive and Targeted Revisions for 

SSA’s Body System Listings 9

Abbreviations 

ADA Americans with Disabilities Act of 1990

DI Social Security Disability Insurance

DOT Dictionary of Occupational Titles

Labor Department of Labor

ICF International Classification of Functioning, Disability and 

 Health

NIH National Institutes of Health

O*NET Occupational Information Network

OIS Occupational Information System

SSA Social Security Administration

SSI Supplemental Security Income

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-12-420 Modernizing SSA Disability Criteria 

United States Government Accountability Office 

Washington, DC 20548 

June 19, 2012

The Honorable Sam Johnson

Chairman

Subcommittee on Social Security

Committee on Ways and Means

House of Representatives

Dear Mr. Chairman:

The Social Security Administration (SSA) manages two of the largest 

federal disability benefit programs—Social Security Disability Insurance 

and Supplemental Security Income—which together resulted in payments 

of more than $165 billion to about 13.5 million people with disabilities and 

their families in fiscal year 2010.1

You recently asked us to assess SSA’s plans and efforts for revising its 

disability criteria. Accordingly, we examined the status and management 

of SSA’s efforts to update its medical listings to reflect current medical 

knowledge and develop a new occupational information system to reflect 

labor market changes, and we identified other steps taken by SSA to 

incorporate a modern view of disability into its eligibility criteria.

Given the extensive size and cost of its 

disability programs, SSA must have current and appropriate criteria by 

which to assess whether an applicant’s medical conditions affect his or 

her ability to perform work in the national economy. However, we 

designated federal disability programs as high risk, in part because the 

medical criteria and occupational information that SSA relies on to make 

benefit decisions were found to be out of date. Moreover, we and others 

have found SSA’s disability programs—which historically have tended to 

equate severity of a medical condition with an inability to work—to be out 

of sync with a more modern concept of disability that considers the 

presence or lack of assistance that an individual with medical 

impairments might need to function more effectively in his or her 

environment.

 

1

These data are from the fiscal year 2012 President’s Budget.

 
 

Page 2 GAO-12-420 Modernizing SSA Disability Criteria 

To perform this work, we reviewed our prior reports and SSA Office of 

Inspector General reports; relevant federal laws and regulations; program 

documentation, including policies, procedures, strategic goals, and 

supporting project plans and cost estimates; relevant literature; and 

position papers and testimonies from disability groups and commissions. 

We interviewed SSA officials, key project contractors and stakeholders, 

disability experts, and representatives of other agencies that administer 

disability programs. We also evaluated SSA’s plans to update its criteria 

against sound project management and cost estimating practices. We 

narrowed our scope to criteria used for initial adult disability 

determinations. See appendix I for more on our scope and methodology.

We conducted this performance audit from April 2011 through June 2012 

in accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.

SSA administers two disability programs: the Social Security Disability 

Insurance (DI) program, enacted in 1956, and the Supplemental Security 

Income (SSI) program, enacted in 1972.2 In order to be eligible for DI or 

SSI benefits based on a disability, an individual must meet the definition 

of disability for these programs—that is, they must have a medically 

determinable physical or mental impairment that (1) prevents the 

individual from engaging in any substantial gainful activity and (2) has 

lasted or is expected to last at least 1 year or result in death.3

To determine eligibility for both programs, SSA uses a five-step 

sequential process that is intended, in part, to expedite disability 

 

2

The DI program provides monthly cash benefits to eligible applicants who have a 

sufficient work record. The SSI program provides monthly benefits to eligible applicants 

with limited income and resources who are disabled, blind, or age 65 or older.

3

42 U.S.C. §§ 423(d)(1)(A) and 1382c(a)(3)(A). Substantial gainful activity is generally 

work activity involving significant physical or mental activities that are done for pay or 

profit, whether or not a profit is realized. 20 C.F.R. §§ 404.1572 and 416.972. In 2012, the 

substantial gainful activity threshold was $1,690 per month for blind recipients and $1,010 

per month for individuals with other disabilities. 

Background
 

Page 3 GAO-12-420 Modernizing SSA Disability Criteria 

decisions when possible and limit administrative costs by conducting less 

intensive assessments at earlier steps (see fig. 1).4

• At steps 1 and 2 of the process, SSA determines whether an applicant 

is working and meets income thresholds, as well as the medical 

severity of impairments. If not working (or not meeting income 

thresholds) and with the determination of a severe impairment, the 

applicant moves to step 3 of the process.

• At step 3, SSA examiners assess the applicant’s medical impairments 

against the Listings of Impairments, also known as the medical 

listings, which are organized into 14 major body systems for adults 

and reflect medical conditions that have been determined by the 

agency to be severe enough to qualify an applicant for benefits.5

When using the listings to determine eligibility, SSA generally relies 

on information on the applicant’s diagnoses, including laboratory 

findings, diagnostic tests, and symptoms, as well as some limited 

consideration of the applicant’s functional limitations. If the individual’s 

impairment meets or is equal in severity to one or more of those in the 

listings, the individual is determined to have a disability at step 3.6

• If a disability determination is not made at step 3, SSA performs an 

assessment of the individual’s physical and mental residual functional 

capacity.7

 

4

See 20 C.F.R. §§ 404.1520 and 416.920.

Based on this assessment, SSA determines whether the 

5

According to SSA officials, there are almost 300 medical impairments organized in 119 

listings under the 14 major body systems for adults.

6

SSA has also implemented Compassionate Allowances to quickly identify diseases and 

other medical conditions that invariably qualify under the medical listings based on 

minimal, objective medical information. According to SSA officials, many of the 

Compassionate Allowances conditions are rare, and thus unfamiliar to examiners, such as 

certain cancers and early-onset Alzheimer’s disease. In April 2012, SSA announced that 

165 conditions would be on the list effective in August 2012, up from 113 conditions as of 

the time this report issued in June 2012. The Compassionate Allowances initiative is one 

of two parts of the agency’s fast-track system for certain disability claims. When combined 

with the Quick Disability Determination process, which uses a predictive model to identify 

cases with a high likelihood of being allowed, Social Security reported approving more 

than 100,000 fast-track cases in 2009, usually in less than 2 weeks.

7

SSA’s physical and mental residual functional capacity assessments establish the extent 

to which an individual’s medically determinable impairments, including any related 

symptoms, such as pain, may cause physical or mental limitations or restrictions that may 

affect his or her capacity to do work-related physical and mental activities. 
 

Page 4 GAO-12-420 Modernizing SSA Disability Criteria 

individual is able to perform past relevant work (step 4) or any work 

that is performed in the national economy (step 5). To inform 

determinations at steps 4 and 5, SSA uses a Department of Labor 

database—known as the Dictionary of Occupational Titles (DOT), 

which is an inventory of occupations performed in the national 

economy. At step 5, SSA also uses a set of rules and guidelines, 

referred to as the grid rules, to evaluate the combined effect of an 

individual’s physical residual functional capacity, age, education, and 

work experience.8

Figure 1: SSA’s Five-Step Sequential Evaluation Process for Determining Disability

 

8

SSA relies on the grid rules to represent the total number of unskilled jobs in the nation at 

three physical strength levels as classified in the DOT, and to provide for consistent 

“rulemaking” or application of case facts to help ensure uniform decisions. 
 

Page 5 GAO-12-420 Modernizing SSA Disability Criteria 

While originally created for expediency,9 over time the medical listings used 

at step 3 were relied on less to make program determinations as they 

became increasingly outdated. In the early years of the program, more than 

90 percent of cases were decided based on medical conditions specified in 

the listings; in 2010 only 47 percent of allowances were made at step 3. 

Experts attribute the decline in allowances based on the medical listings to 

changes in the program, workplace, and medical treatment that the medical 

listings had not kept pace with. Since the 1990s, we, along with SSA’s 

Office of Inspector General and the Social Security Advisory Board,10

In 2008, we reported that SSA had recently established a new process—

referred to by SSA as the “business process”—for revising the listings to 

better incorporate feedback into its continuous updates.

have 

expressed concerns that the medical listings being used no longer provide 

current and relevant criteria to evaluate disability applicants’ inability to 

work. In 2003, we deemed SSA’s and other federal disability programs as 

high-risk areas, in part, because their programs continue to emphasize 

medical conditions in assessing work capacity, without adequate 

consideration of work opportunities afforded by advances in medicine, 

technology, and changes in the labor market.

11 This process, 

which has been in effect since 2003, incorporates feedback from multiple 

parties, including medical experts and claims examiners, to update the 

medical criteria. Under this process, SSA gathers external feedback from 

comments associated with regulatory actions, such as the publication of 

advanced notices of proposed rulemaking (advanced notices) and notices 

of proposed rulemaking (notices) in the Federal Register.12

 

9

Historically, SSA created the listings to have a clear set of medical conditions that, if 

present, would preclude work. This in turn would limit the number of cases requiring a 

more involved and individualized assessment of function.

In addition, 

10In 1994, when the Congress passed legislation establishing SSA as an independent 

agency, it also created the seven-member bipartisan Social Security Advisory Board in 

part to advise the President, Congress, and the Commissioner of Social Security on Social 

Security and SSI policy. Pub. L. No. 103-296, § 103, 108 Stat. 1464, 1467.

11GAO, Federal Disability Programs: More Strategic Coordination Could Help Overcome 

Challenges to Needed Transformation, GAO-08-635 (Washington, D.C.: May 20, 2008).

12Any changes to the medical listings have to proceed according to an established 

process for rulemaking outlined in federal law. As such, when changes are made to the 

listings, a notice of proposed rulemaking must generally be published in the Federal 

Register with a public comment period before the final rule is issued. 5 U.S.C. § 553. Also, 

the Office of Management and Budget must review and approve any notice that is 

economically significant (i.e., has costs or savings greater than $100 million). 
 

Page 6 GAO-12-420 Modernizing SSA Disability Criteria 

this process includes conducting postimplementation reviews one year 

after a revision is made to assess the impacts of a revised listing, areas to 

improve, and whether expectations from the revisions have been 

achieved.13

With respect to information on jobs in the national economy that supports 

SSA’s occupational criteria, we and others have reported that the DOT is 

outdated, although SSA still relies on it to assess eligibility at steps 4 and 

5 of the process. The DOT has not had a major update since 197714 and 

the Department of Labor (Labor) replaced it with a new database in 1998 

called the Occupational Information Network (O*NET).15

Beyond dated medical criteria and vocational information, numerous 

disability experts have expressed concern that SSA’s disability 

programs—which statutorily require that an adult’s medical condition 

prevents that person from engaging in substantial gainful activity—have 

historically tended to equate the severity of medical conditions with 

inability to work and thus are out of sync with modern concepts of 

disability. Modern concepts focus on an individual’s functional abilities in 

the workplace environment, including consideration of the presence or 

lack of assistance, for example, per the requirements for reasonable 

accommodation by the Americans with Disabilities Act of 1990.

However, SSA 

determined that O*NET is not sufficiently detailed for evaluating DI and 

SSI disability claims, so SSA has begun developing its own occupational 

information system to better reflect the physical and mental demands of 

work in the national economy.

16

 

13Such reviews typically rely on reviews of decision data as well as surveys of field staff 

responsible for applying the revised listings.

These 

modern views are reflected in the International Classification of 

Functioning, Disability and Health (ICF), which is the World Health 

Organization’s framework for assessing health and disability. This 

framework takes into account the interaction of health conditions and 

14The DOT was updated in 1991, but for less than 20 percent of the occupations. 

15O*NET is a general purpose occupational information database used for workforce 

development, economic development, career exploration, and academic and policy 

research. Selected occupations within the O*NET system are updated each year. 

According to Labor officials, by July 2012, 900 occupations will have been 

comprehensively updated, and 413 will have more than one update.

16See, for example, 42 U.S.C. § 12112(b)(5).
 

Page 7 GAO-12-420 Modernizing SSA Disability Criteria 

contextual factors, such as products and technology, attitudes, and 

services, on an individual’s functional capacity, rather than viewing 

disability solely as a medical or biological issue. Along these lines, 

experts have recommended that SSA incorporate more consideration of 

individual function in its medical listings when doing so can improve their 

use as a screening tool for determining inability to work. Further, several 

groups, such as the Social Security Advisory Board17 and the Urban 

Institute,18 have reported that SSA’s disability programs should focus 

more on whether an individual can work given appropriate environmental 

or other supports, and that SSA—through its demonstration authority or 

other means—could play a role in determining how. These suggestions 

are consistent with SSA’s long-term targeted outcomes, which include not 

only regularly updating regulations and policies to incorporate the most 

recent medical advances, but also making it easier for individuals with 

disabilities to return to work.

Since our last review in 2008, SSA has made several changes that hold 

promise for improving its medical listings updates. First, the agency is 

using a two-tiered system for ongoing revisions to the listings. Under this 

system, SSA first completes a comprehensive listings update for a body 

system, which entails reviewing all the diseases and disorders listed 

within that system and making necessary revisions. Second, following a 

comprehensive revision, SSA will pursue a more targeted approach—that 

is, SSA will conduct ongoing reviews and updates of a smaller number of 

medical diseases or disorders within that body system. Agency officials 

 

17See Social Security Advisory Board, The Social Security Definition of Disability

(Washington, D.C.: October 2003).

18David Wittenburg and Pamela Loprest, A More Work Focused Disability Program?: 

Challenges and Options (Washington, D.C.: Nov. 1, 2003). 

SSA Has Improved Its 

Process for Updating 

Medical Listings, but 

Still Faces Challenges 

with Timely Updates

SSA Has Improved Its 

Strategy to Update Medical 

Listings
 

Page 8 GAO-12-420 Modernizing SSA Disability Criteria 

told us that targeted updates should be completed more quickly than 

comprehensive updates, allowing them to focus on the most critical 

changes needed.19

Another change, according to agency officials, is that in 2010, the SSA 

Commissioner set a 5-year cycle for updating listings for each body 

system.

As of early March 2012, SSA had begun the ongoing 

review process to identify opportunities for targeted revisions for 8 of the 

14 adult body systems that were recently comprehensively revised.

20

 

19Officials noted that these ongoing reviews could also result in major or even no 

changes, as appropriate.

Previously, SSA established cycles for periodically updating 

listings under each body system, ranging from 3 to 8 years, but frequently 

extended them. SSA officials believe that conducting targeted reviews will 

generally allow the agency to conclude any necessary revisions prior to 

the 5-year period. Additionally, they expect that using the “business 

process,” which requires early public notification of changes and obtaining 

necessary data and feedback from internal and external parties, should 

help keep continuous reviews on track. See figure 2 for the status and 

expiration date by which the listings should be reviewed and updated, if 

needed, for the 14 body systems undergoing review for either 

comprehensive or possible targeted revisions, as of early March 2012. 

20The 5-year period will be applied to listings under a body system upon completion of 

their current revision. SSA began applying the 5-year period in 2011 when it 

comprehensively updated the endocrine body system listings. For other body system 

listings updated prior to 2011, SSA generally assigned periods extending beyond 5 years. 
 

Page 9 GAO-12-420 Modernizing SSA Disability Criteria 

Figure 2: Status of Comprehensive and Targeted Revisions for SSA’s Body System 

Listings 

a

For the two-tiered revision process, the beginning point is the date of the last comprehensive or 

targeted revision.

b

SSA officials told us they have grouped the disorders in the special senses and speech listings into 

two broad sections—vision and hearing—and two standalone medical conditions. SSA has not yet 

comprehensively revised the two standalone medical conditions that cover speech and disturbances 

of labyrinthine-vestibular function. The two sections have undergone comprehensive revisions and 

are undergoing reviews that began in 2007 for the vision section and in 2010 for the hearing section. 

Special senses and speech listings are set to expire in 2015. 

c

According to SSA officials, SSA published limited revisions for the hematological disorders in 1988. It 

also published final rules for limited revisions for the mental disorders listings in 2000 and the 

musculoskeletal system listings in 2001. 

SSA has made another change by more extensively engaging the 

medical community to identify ways to improve the medical listings. For 

example, SSA contracted with the Institute of Medicine to study its 

medical criteria for determining disability and to make recommendations 

for improving the timeliness and accuracy of its disability decisions, 
 

Page 10 GAO-12-420 Modernizing SSA Disability Criteria 

resulting in a 2007 report with recommendations21 and a symposium of 

experts in 2010. SSA has addressed some of the Institute of Medicine’s 

recommendations, such as making better use of its administrative data to 

update criteria and creating a standing committee through the Institute of 

Medicine to provide recommendations for listings revisions. In addition, 

through the Institute of Medicine, SSA created consensus committees to 

conduct research and provide other assistance with updating SSA’s 

cardiovascular listings and its Human Immunodeficiency Virus (HIV) 

listing, which culminated in two reports with recommendations.22

SSA has made progress, but continues to face delays in completing both 

comprehensive and ongoing targeted updates. As of early March 2012, 

SSA officials told us they had completed comprehensive revisions of 

listings for eight body systems, some of which resulted in significant 

changes. For example, in 2011, SSA removed the endocrine body system 

listings for all the adult disorders, such as diabetes, because they found 

that they were now generally diagnosed early and treated very 

successfully.23 Nevertheless, according to SSA, the agency still needs to 

complete the comprehensive revisions for listings of the remaining six 

body systems—a process that has been ongoing for the last 19 to 33 

years, with numerous extensions beyond the original expiration periods 

(see table 1). For example, it has been at least 27 years since SSA 

finalized comprehensive revisions for two of the six body system 

listings—mental and neurological disorders, which are among those SSA 

uses most frequently in its eligibility determination process.24

 

21Institute of Medicine, Improving the Social Security Disability Decision Process 

(Washington, D.C.: 2007). 

SSA has 

made progress on four of the six body systems set to expire in 2012, but 

will likely miss targeted time frames. SSA issued a notice of proposed 

rulemaking to revise its listings of mental disorders in 2010 and has told 

22For the reports, see Institute of Medicine, Cardiovascular Disability: Updating the Social 

Security Listings (Washington, D.C.: 2010), and HIV and Disability: Updating the Social 

Security Listings (Washington, D.C.: 2010).

23SSA officials noted that while the most common endocrine disorder, diabetes, can be 

disabling, this is due to its effects on other organs such as the heart or kidneys; as such, 

they can be assessed under listings for these body systems or the effects on functionality 

can be assessed at steps 4 or 5.

24According to SSA officials, SSA made limited, but not comprehensive, revisions to the 

mental disorders listings in 2000. 

SSA Has Experienced 

Delays with Its Revision 

Process
 

Page 11 GAO-12-420 Modernizing SSA Disability Criteria 

us that it plans to finalize this comprehensive revision by the end of 2012, 

after its current expiration date in July 2012.25

Table 1: Key Dates for Comprehensive Revisions to Body System Listings

SSA is still developing 

notices for the neurological and two other body system listings, which are 

also set to expire in 2012. Because SSA has generally taken more than a 

year to move from publishing the notice to final revision and may also 

need additional time for any internal revision and review, as well as a 

review by the Office of Management and Budget, it is also unlikely that 

SSA will meet its planned 2012 time frames for updating these listings.

See appendix II for details on progress updating these listings.

Body system and section

Year of last 

comprehensive 

revision 

Years initially set 

to review and 

revise 

Years since last 

comprehensive 

revisiona Current 

expiration dateb

Special senses and speechc

sections: 

• Speech

• Disturbances of labyrinthinevestibular function

1979 8 33 2015

Neurological disorders 1985 8 27 2012

Mental disorders 1985 3 27 2012

Hematological disorders 1985 8 27 2012

Musculoskeletal system 1985 5 27 2013

Respiratory system 1993 6 19 2012

Source: GAO analysis based on information provided by SSA officials.

a

According to SSA officials, SSA published limited revisions for the hematological disorders in 1988. It 

also published final rules for limited revisions for the mental disorders listings in 2000 and the 

musculoskeletal system listings in 2001. 

b

SSA has continually extended the expiration dates of the listings. 

c

The special senses and speech listings have two sections—vision and hearing—which have been 

comprehensively revised. 

Updates using SSA’s targeted approach seem to be moving at a faster 

pace than the comprehensive revisions, but some delays have occurred 

here as well, and more are expected. For example, SSA has already 

extended the expiration date for its targeted review of the cardiovascular 

system. According to SSA officials, it may also need to extend expiration 

 

25See 75 Fed. Reg. 51,336 (Aug. 19, 2010).
 

Page 12 GAO-12-420 Modernizing SSA Disability Criteria 

dates for listings under two other body systems set to expire in 2012. 

Extensions may be needed because SSA has not yet published the 

notice of proposed rulemaking for those two body system listings and 

may not have time to publish these notifications, respond to comments, 

and complete the final updates by their current expiration date. 

At the same time that SSA has been experiencing delays completing 

timely revisions, agency officials reported challenges with other steps in 

the business process. Whereas SSA’s business process includes a 

postimplementation review of case data at the 1-year mark to determine 

whether expectations from a revision were met, several did not undergo 

this review, such as those for skin disorders, genitourinary impairments, 

and impairments affecting multiple body systems. SSA officials told us 

they only began conducting these reviews in 2010, and to date, SSA has 

completed just one in 2011 that involved a targeted sample of 175 cases.

SSA officials told us they are conducting or planning to conduct two more 

reviews at the 1-year mark in 2012. While disability experts we 

interviewed spoke highly of SSA’s business and targeted review 

processes to obtain feedback early on and update the listings more 

promptly, ongoing delays raise questions about the agency’s ability to 

fully follow its current business process while completing continuous and 

timely revisions for all 14 body systems.26

SSA officials offered two key reasons for the delays in updating the 

listings: (1) limitations in the number and expertise of staff and (2) the 

complexity and unpredictability of the regulatory process. According to 

SSA officials, revising the medical listings requires research, deliberation, 

testing, regulatory review, and consensus with many stakeholders, and 

consequently is difficult and time-consuming to achieve. According to an 

SSA official in the Office of Medical Listings Improvement, the office is 

short-staffed and there is a lack of expertise needed to perform this work. 

To address these constraints, SSA has contracted with the Institute of 

Medicine to review and develop recommendations for revising two of the 

body system listings. While not finalized as of March 2012, SSA officials 

reported that the agency plans to renew this contract after it expires in 

2012 and extend it to 2015. However, SSA has not yet determined how it 

 

 

26According to agency officials, in addition to updating adult listings the agency must also 

update listings for children, which adds to its workload and resource needs.
 

Page 13 GAO-12-420 Modernizing SSA Disability Criteria 

will use the Institute of Medicine to revise the listings or the extent to 

which the contract will address staffing shortfalls. 

Also contributing to delays is the time required for internal review and 

public comment under the regulatory process, which depends largely on 

the number and the substance of comments received, according to an 

SSA official. Obtaining public comment is one way SSA receives critical 

information for identifying areas for revision and obtaining stakeholder 

consensus. As such, SSA reviews and responds to each comment that 

would result in a significant change, and the time required for doing so 

varies depending on the number of comments and resources. For 

example, SSA officials told us that the advanced notice of proposed 

rulemaking to the listings of mental disorders in 2003 resulted in 500 

comments, which took SSA 5 years to incorporate into a draft notice of 

proposed changes for regulatory review. SSA officials reported they have 

considered options for automating and thereby speeding up the process 

of obtaining and reviewing public comment. For example, they told us that 

they eventually plan to use a web-based tool to obtain early public 

feedback on medical listing updates to help target their limited resources 

and more quickly make changes. According to an SSA official, the agency 

plans to retain its optional use of the advanced notice of proposed 

rulemaking because it can also help to identify appropriate areas to focus 

on to ultimately make timely updates.27

 

27Under the Administrative Procedure Act, a notice of proposed rulemaking is generally 

required for regulatory revision, but an advanced notice is not required. See 5 U.S.C. § 

553.
 

Page 14 GAO-12-420 Modernizing SSA Disability Criteria 

In 2008, SSA began a multiyear project to research and design a new 

source of occupational information that will replace the outdated 

information currently being used to determine if claimants are able to do 

their past work or any other work in the national economy. Since the 

1960s, SSA has been using the DOT, which contains a list of job titles 

found in the national economy and had a last major update in 1977.

28 The 

DOT provides SSA with descriptions of the physical demands of work—

such as climbing, balancing, and environmental requirements—for each 

of the more than 12,000 occupations listed.29

 

28The DOT provides a wide range of occupational information that could be used for job 

placement, occupational research, career guidance, labor-market information, curriculum 

development, and long-range job planning. 

 According to SSA, these 

descriptions have been essential to its evaluations of how much a 

claimant can do despite his or her impairment and whether this level of 

functioning enables the claimant to do his or her past work or any other 

work. After its last limited update, Labor decided to replace the DOT with 

O*NET, which has far fewer occupational titles compared with the DOT 

and has served Labor’s purposes more efficiently. According to an SSA 

report, after investigating potential alternatives, SSA decided that O*NET 

and other existing databases with occupational information were not 

sufficiently detailed and not able to withstand legal challenges for use in 

its decision-making process. SSA further decided to develop its own 

occupational information system (OIS), which would contain detailed 

information as in the DOT, but would also include additional information, 

29In 1966, SSA contracted with Labor to produce a companion volume to the DOT entitled 

the Selected Characteristics of Occupations that provides measures for additional physical 

demands of work for DOT occupations, such as climbing, balancing, reaching, handling, 

special senses requirements (visual acuity, hearing, etc.), and environmental 

requirements (noise levels, exposure to cold, etc.).

SSA Has Begun an 

Ambitious Project to 

Develop Updated 

Occupational 

Information, but Many 

Uncertainties Remain

SSA is Designing a New 

Occupational Information 

System, but Has Fallen 

behind with Some 

Activities
 

Page 15 GAO-12-420 Modernizing SSA Disability Criteria 

such as the mental demands of work. In addition, SSA has determined 

that the OIS will (1) meet its legal, program, and data requirements; (2) be 

flexible enough to incorporate changes in its policies and processes; and 

(3) be able to be updated to reflect the evolving workplace environment.

In 2008, SSA began taking steps to guide the development of its OIS. 

SSA created an internal office and working group, as well as an 

Occupational Information Development Advisory Panel. While the number 

of panel members has fluctuated over time, in April 2012, 14 external 

experts were serving on the panel and represented various affiliations, 

such as medicine, disability law, rehabilitation, and industrial 

organizational issues.30

To further inform its efforts, SSA has sought input from agencies or 

organizations that either collect occupational information or also use the 

DOT. For example, SSA officials held initial meetings with Labor and U.S. 

Census Bureau officials to gain information on sampling methods used for 

O*NET, the Occupational Employment Statistics program, and U.S. 

Census Bureau’s household surveys.

The advisory panel holds quarterly public 

meetings and has several subcommittees that review material and make 

recommendations to SSA on developing various components of the OIS. 

For example, in a 2009 report, the advisory panel supported the need for 

SSA to develop a new source of occupational information, rather than 

adapt O*NET, and recommended the type of data SSA should collect, as 

well as approaches for classifying occupations.

31

 

30The Occupational Information Development Advisory Panel was formed in December 

2008 under the Federal Advisory Committee Act, and broadly includes representatives of 

organizations that have used the DOT or other occupational information systems, and 

subject matter experts. The panel was originally comprised of 11 members, including an 

SSA official. Over time, some members resigned and additional members were added. 

Additionally, since February 2011, 

SSA and Labor have been in the process of approving a memorandum of 

understanding to formalize their collaboration efforts on the new OIS. 

According to an SSA official, as the OIS project progresses, SSA plans to 

convene ad hoc roundtables with experts and other agency officials to 

31The Occupational Employment Statistics program produces employment and wage 

estimates for approximately 800 occupations. The U.S. Census Bureau’s household 

surveys include (1) the American Community Survey, which is an ongoing survey that 

provides annual data on demographics such as age, education, and disabilities, and (2) 

the Current Population Survey, which is primarily a labor force survey, conducted every 

month by the U.S. Census Bureau for the Bureau of Labor Statistics and provides data 

such as the national employment rate.
 

Page 16 GAO-12-420 Modernizing SSA Disability Criteria 

explore specific subject areas, such as sampling issues. Besides working 

with Labor and U.S. Census Bureau officials, SSA officials and panel 

members have sought input from other experts and current users of the 

DOT, such as SSA disability adjudicators and external rehabilitation 

professionals. SSA officials conducted a user needs analysis in 2009 and 

have presented on the OIS project at events and conferences.

In July 2011, SSA published a detailed research and development plan 

outlining all activities related to researching, developing, and testing the 

key components of the OIS in order to implement it by 2016 at an 

estimated total cost of $108 million. For example, the plan includes 

several baseline activities to identify and study other occupational 

information systems and various approaches for analyzing occupations 

that may inform or could be leveraged in SSA’s OIS data collection. The 

plan also includes activities to identify the primary occupational, 

functional, and vocational characteristics of current beneficiaries. Other 

key components of the plan include developing descriptions of work 

requirements, such as the physical and mental demands for jobs; 

developing data collection and analysis strategies; and identifying the 

occupations, categories, and definitions that will constitute the structure of 

the new OIS. SSA also plans to develop a strategy for piloting how it 

would ultimately collect data for the OIS nationwide within this time frame.

As of May 2012, SSA had made progress on many of the baseline 

activities outlined in its research and development plan for the OIS.

32

 

32Some of the activities that were part of the research and development plan were 

completed in years prior to the plan’s issuance. 

For 

example, according to an SSA official, its investigation of existing 

occupational information systems, now complete, resulted in useful 

information about design issues other organizations have confronted and 

mitigated when creating their own system. Additionally, SSA’s preliminary 

analysis of its own administrative data identified the most frequently cited 

occupations and functional and vocational characteristics of disability 

applicants. SSA officials told us the agency will target the occupations 

identified in this analysis for its pilot studies of the OIS. Also in 2011, SSA 

completed a comprehensive framework for assessing an individual’s 

capacity to work—key to informing the OIS content, according to SSA 

officials—which was based on recommendations of outside experts as 

well as SSA’s policy and program requirements.
 

Page 17 GAO-12-420 Modernizing SSA Disability Criteria 

While SSA has made progress on several key activities, agency officials 

delayed 2011 completion dates for certain activities and anticipate making 

additional changes to its timeline as a result of not meeting its staffing 

goals for fiscal year 2011 (see table 2). Some activities that were delayed 

by several months included finalizing reports for the baseline studies and 

conducting a literature review that would inform how occupations might 

be analyzed. SSA officials told us that they did not have enough staffing 

to complete all of the 2012 planned activities within the estimated 

schedule. Further, SSA officials said they did not have the budget to hire 

new staff in September 2011. To address this challenge, SSA officials 

hired consultants to meet some of their needs. SSA officials also met with 

the Office of Personnel Management to explore the possibility of an 

interagency agreement that would allow SSA to use one or two of the 

office’s industrial organizational psychologists to help on a part-time 

basis.

Table 2: Status of Select OIS Activities as of May 2012

Activity and objective Fiscal year target Status 

Investigate existing OISs. To identify lessons learned about features, requirements, 

processes, and options through the examination of the development and operation 

of existing domestic and international OISs.

2010-2011 Completed in fiscal year 

2012 (late)

Occupational and medical-vocational study. To identify and record the primary 

occupational, functional, and vocational characteristics of adult disability applicants 

at steps 4 or 5 at the initial and hearings levels. 

2010-2011 In progress

(late)

Job analysis methodologies (performed by contractor). To identify existing job 

analysis methods, their features, and their potential usefulness for OIS 

development.

2011 Completed on time

Business processes for recruiting, training, and certifying job analysts (performed 

by contractor). To identify existing business processes used to recruit, train, and 

certify job analysts and potential usefulness of these processes for OIS 

development.

2011 Completed on time

Identify OIS usability standards. To identify standards and criteria for assessing the 

operational usability of the new OIS and to incorporate those standards into OIS 

research and development activities. 

2009-2012 Completed 3 of 4 

activities (one completed 

late) 

Identify OIS scientific standards. To identify relevant scientific standards, 

guidelines, and best practices that enable SSA to meet its responsibilities under 

applicable federal requirements, and to incorporate those standards into OIS 

research and development activities. 

2011 Completed

Identify OIS legal requirements. To identify standards for evaluating and ensuring 

that the new OIS accurately reflects the language and the intent of the vocational 

requirements of the Social Security Act. 

2011 Completed

Identify OIS elements. To establish basic design parameters and requirements to 

ensure that OIS meets SSA’s legal, scientific, and usability requirements for 

occupational information.

2012 In progress
 

Page 18 GAO-12-420 Modernizing SSA Disability Criteria 

Activity and objective Fiscal year target Status 

OIS work taxonomy. To identify a comprehensive set of constructs that may form 

the basis of OIS. These constructs must reflect the physical and mental-cognitive 

requirements of work and context of work that can be objectively measured or 

delineated to develop an occupational classification. 

2011-2012 In progress

Source: GAO based on information provided by SSA officials.

Our Cost Estimating and Assessment Guide33

We compared SSA’s cost estimate to three best practices for assuring a 

reliable cost estimate

identifies a number of best 

practices for effective cost estimating and scheduling that should result in 

reliable and valid cost estimates that management can use for making 

informed decisions. Per these criteria, the success of any program 

depends in part on having reliable cost estimates and a reliable schedule.

A reliable cost estimate provides the basis for informed investment 

decision making, realistic budget formulation and program resourcing, 

meaningful progress measurement, proactive course correction when 

warranted, and accountability for results. A reliable schedule defines, 

among other things, when work activities will occur, how long they will 

take, and how they are related to one another. As such, the schedule not 

only provides a road map for systematic execution of a program, but also 

provides a means by which to gauge progress, identify and address 

potential problems, and promote accountability.

34

 

33GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and 

Managing Capital Program Costs, 

and found SSA only minimally or partially met 

each of these (see table 3). First, we found SSA’s estimated cost of the 

OIS research and development phase—$108 million—was not welldocumented. For example, after reviewing all of SSA’s documentation 

supporting this cost estimate as of December 2011, we did not find step 

by step documentation showing how the estimate was derived so that 

someone unfamiliar with the project could use the documentation to 

recreate the estimate and get the same results. Well-documented cost 

estimates are considered a best practice and without good 

documentation, SSA is not in a position to defend the reliability of its 

estimate. Additionally, SSA only partially met the best practice of having 

GAO-09-3SP (Washington, D.C.: March 2009). 

34We did not assess SSA’s estimate on a fourth criterion—credibility—which evaluates 

any limitations of the analysis because of uncertainty or biases surrounding data or 

assumptions, as this criterion is most applicable to major capital acquisitions.

SSA’s Cost Estimate and 

Schedule Fell Short of Best 

Practices in Key Areas
 

Page 19 GAO-12-420 Modernizing SSA Disability Criteria 

an accurate cost estimate. While SSA provided documentation showing 

how some of their estimates compared to actual costs and the reasons 

for any variances, SSA did not provide adequate documentation for us to 

determine, for example, if the estimate took inflation into account or if 

there were any calculation errors. Generally speaking, in the absence of a 

detailed cost model, third party reviewers cannot be certain that cost 

estimate calculations are accurate and account for all costs.

We also found the cost estimate was not comprehensive as it does not 

include any costs beyond the research and development phase, such as 

the costs of producing, maintaining, and updating the final data system, 

which could be significant. SSA officials told us that they are still in the 

process of determining what information will be included in the data 

system, how it will be collected, and how many occupations will be 

covered, each of which will influence the cost of developing and 

maintaining the OIS. As such, they maintain that it is too early to estimate 

the costs of the final system or the costs for maintaining the system.

According to industry best practices, cost estimates should be 

comprehensive and include all costs necessary to achieve agency 

objectives and should be updated as the agency proceeds with the 

project and gains more information. At this point in the project, estimating 

total costs of options under consideration could enhance decision making 

by evaluating the potential tradeoffs of different designs. Without any 

estimate for the cost of producing the OIS, SSA risks designing a system 

that would not be a viable or affordable option to complete. Additionally, 

without maintenance cost estimates, SSA is at risk of designing a system 

that would be too costly to maintain on a regular basis resulting in 

outdated information. Other federal information systems provide some 

basis for estimating the cost of producing or maintaining an OIS. For 

example, Labor officials indicated that the cost of maintaining O*NET—an 

occupational information system that includes approximately 1,000 

occupations and uses a paper or web survey to collect data—is roughly 

$6 million a year.35

 

35O*NET’s annual maintenance cost may constitute a lower bound estimate for the OIS, 

since SSA has considered more intensive data collections than a paper- or web-based 

survey questionnaire and a larger number of occupations. Another occupational 

information system maintained by the U.S. Bureau of Labor Statistics—the Occupational 

Employment Statistics program—costs around $30 million annually to maintain. This 

program also covers about 800 occupations predominantly using a mail survey, but may 

involve additional surveys, telephone interviews, or site visits to reach nonrespondents.
 

Page 20 GAO-12-420 Modernizing SSA Disability Criteria 

Table 3: Assessment of OIS Cost Estimate 

Best practice Extent best practice met

Well-documented Minimally met

Accurate Partially met

Comprehensive Partially met

Source: GAO analysis of SSA’s cost estimate for the OIS.

Note: Not met—SSA provided no evidence that satisfies any of the criteria; minimally met—SSA 

provided evidence that satisfies a small portion of the criteria; partially met—SSA provided evidence 

that satisfies about half of the criteria; substantially met—SSA provided evidence that satisfies a large 

portion of the criteria; and met—SSA provided complete evidence that satisfies the entire criteria. See 

appendix III for more information on these best practices. 

We also reviewed SSA’s master schedule for the OIS against nine best 

practices for a reliable schedule, and found that SSA did not meet four 

and minimally met five of these practices (see table 4). For example, 

among those minimally met, we found a significant number of activities 

within SSA’s schedule were not logically sequenced in the order that they 

were going to be carried out (best practice 2) and activities that were 

dependent on completion of a prior activity were not identified (best 

practice 5, 6, and 7). For example, SSA officials and experts 

acknowledged that activities associated with the “OIS work taxonomy” 

part of the research and development plan—activities that together 

determine which information should be included in the OIS—are an 

essential building block that will inform other OIS activities, such as 

developing the “OIS work analysis instrument,” which will determine how 

OIS information will ultimately be collected.

Nevertheless, when we tested SSA’s schedule for sequencing and 

linkages associated with the OIS work taxonomy activity, we found a 

significant delay of almost 1,000 days that should have significantly 

delayed the project actually barely affected the final completion date in 

the OIS project. Such missing links between key activities in the schedule 

represent broken logic that reduces the reliability of the forecasted dates. 

The OIS schedule also did not meet the best practice of conducting a 

schedule risk analysis (best practice 8), which is an essential tool for 

project managers to understand the most important risks to the project 

and focus on mitigating them. Based on the schedule provided to us, we 

do not have any indication that SSA has considered the vulnerability 

within its schedule for meeting time frames for individual activities which 

would in turn impact the time frames for the entire project. Without 

explicitly identifying risks to its schedule, SSA either does not know or is 
 

Page 21 GAO-12-420 Modernizing SSA Disability Criteria 

not conveying the probability of completing research and development 

activities on time, and its relationship to overall costs for this phase. 

However, in order to conduct a schedule risk analysis the schedule has to 

be properly sequenced and networked with all logic links in place for the 

analysis to produce credible results.

Table 4: Assessment of OIS Schedule 

Best practice

Extent best practice 

met

1. Capturing all activities Minimally met

2. Sequencing all activities Minimally met

3. Assigning resources to all activities Minimally met

4. Establishing the duration of all activities Minimally met

5. Integrating schedule activities horizontally and vertically Not met

6. Establishing the critical path for all activities Not met

7. Identifying float between activities Minimally met

8. Conducting a schedule risk analysis Not met

9. Updating the schedule using logic and durations to 

determine dates

Not met

Source: GAO analysis of SSA’s schedule for the OIS.

Note: Not met—SSA provided no evidence that satisfies any of the criteria; minimally met—SSA 

provided evidence that satisfies a small portion of the criteria; partially met—SSA provided evidence 

that satisfies about half of the criteria; substantially met—SSA provided evidence that satisfies a large 

portion of the criteria; and met—SSA provided complete evidence that satisfies the entire criteria. See 

appendix IV for more information on these best practices. 

Beyond gaps in their cost estimate and schedule of specific activities, 

SSA faces broader challenges that could impede the success of the OIS 

design and implementation; however, SSA has not done a formal risk 

analysis of these challenges.36

 

36Risk assessment helps decision makers identify and evaluate potential risks to an 

agency’s mission, so that countermeasures can be designed and implemented to prevent 

or mitigate the risks. As part of a risk assessment, agencies could identify alternatives to 

alter the likelihood or outcome of a high risk situation. 

While SSA officials said they had recently 

begun discussing various risks and how they might address them, they 

said it was premature to provide us with this information. Examples of 

potential challenges for OIS that experts and stakeholders we spoke with 

cited include:

Other Challenges with OIS 

Development Could 

Undermine the Project’s 

Success
 

Page 22 GAO-12-420 Modernizing SSA Disability Criteria 

• SSA’s lack of expertise with designing an OIS. SSA does not have 

prior experience with designing a complex and, in some respects, 

unprecedented occupational information system. As such, SSA will 

need to depend on many outside experts and contractors to complete 

the system. While the Occupational Information Development 

Advisory Panel members can provide some expert counsel, most of 

its members lack the technical and scientific background essential to 

informing this complicated effort. To help design the OIS, SSA 

recently hired a research psychologist and plans to hire two more 

individuals in the near future, but may need additional resources. In 

March 2012, SSA put out a request for information and, according to 

an agency official, received ideas from knowledgeable experts about 

how to move forward with technical aspects of the project. Until the 

agency secures required and sufficient expertise, SSA may not be 

able to move ahead with key technical decisions, such as defining the 

number of occupations to include and the data collection 

methodology, which may cause further delays.

• Cost of maintaining an OIS. SSA has not yet made design decisions 

on the OIS that will ultimately define the overall cost of producing, 

implementing, and maintaining the OIS system. For example, SSA 

officials told us that they were not sure how many occupations they 

would include in the OIS, although they believe the total number will 

be somewhere between the number of occupations in the O*NET and 

the DOT, or between approximately 1,000 and 12,000 occupations. A 

2010 study conducted by the National Academy of Sciences 

reviewing Labor’s O*NET cited the three major cost drivers of an 

occupational information system as (1) size of sample, (2) number of 

occupations, and (3) frequency of updates. This study noted that as 

the agency increases any one of these cost drivers, it does so at the 

expense of the others or of the overall cost. For example, if SSA 

chooses to frequently update the entire OIS, the agency may need to 

make tradeoffs with the number of occupations or sample size it 

chooses if it wishes to contain costs.

• Managing large, multiyear projects. In the past, SSA has experienced 

difficulties managing complex, multiyear efforts. In prior work, we 

reported that SSA has cancelled numerous demonstration projects 

due to limitations or weaknesses in design or implementation and 

lacked sufficient controls to ensure effective management of its 
 

Page 23 GAO-12-420 Modernizing SSA Disability Criteria 

demonstration projects.37 We have also reported on challenges that 

SSA faced managing two separate redesigns of its disability decisionmaking process.38 While the OIS project alone represents a complex 

effort, stakeholders have emphasized the importance of this project 

progressing in close concert with other ongoing research managed by 

another part of SSA, placing greater emphasis on broad and effective 

project planning.39

Given the range of challenges and potential risks they pose to the 

success of the OIS project, it is important to identify and carefully explore 

feasible alternatives that may mitigate these risks. Examples of potential 

alternatives suggested by experts and other observers in the disability 

field include:

In light of past experiences and as SSA faces a 

potential change in leadership in 2013, the management 

shortcomings we identified increase the vulnerability of this complex 

project.

• Leveraging O*NET. In a report issued in 2010, SSA’s advisory panel 

found that it did not believe that O*NET would meet SSA’s needs, in 

part, because it is not detailed enough. However, the National 

Academy of Sciences also issued a report in 2010 and concluded that 

O*NET could be altered to better meet SSA’s needs for disability 

adjudication. SSA officials have told us that modifying O*NET would 

not result in savings for the agency, but they have not conducted any 

analysis to determine this. While most disability experts agree that 

O*NET in its current form would not be a suitable source of 

occupational information for SSA, the National Academy of Sciences 

study noted that there are potential linkages between O*NET and an 

 

37GAO, Social Security Disability: Management Controls Needed to Strengthen 

Demonstration Projects, GAO-08-1053 (Washington, D.C.: Sept. 26, 2008).

38SSA’s most recent redesign effort—the Disability Service Improvement initiative—was 

suspended in large part in 2007 by the new Commissioner, due to budgetary and other 

considerations.

39Specifically, while the OIS is identifying functional requirements in the workplace, 

another SSA office is working through an interagency agreement with the National 

Institutes of Health to identify methods to more accurately capture the functional capacity 

of individuals. Both SSA and its advisory panel recognize that SSA’s disability criteria 

must reflect both the capabilities of people and work demands, and that the OIS must be 

designed and developed to support SSA’s process for assessing individual function. As 

such, some stakeholders believe it is critical that these two efforts be fully aligned for the 

overall success of the disability determination process.
 

Page 24 GAO-12-420 Modernizing SSA Disability Criteria 

SSA OIS that are worthy of more careful exploration on SSA’s part 

and that leveraging one system to serve broader purposes could be 

most cost-effective overall.

• Adjusting the scope of the OIS. SSA could reduce costs by limiting the 

scope of the OIS. For example, the OIS could be designed to capture 

fewer occupations. SSA officials have told us that since they are early 

in their design, they do not know how many occupations will be 

included in their OIS. Since this decision is fundamental to informing 

the OIS data collection methods, feasibility, and cost, consideration of 

alternative scope should be given high priority attention.

• Limiting data collection methods. Another key factor that SSA must 

consider is the method they will use to collect data for their OIS as 

some methods are more costly than others. For example, among the 

methods SSA is considering, conducting on-site job analyses of 

occupations would be more expensive than surveys, which are also 

more resource and time intensive than collecting information through 

telephone interviews.

• Leveraging resources from other agencies and OIS users. Other 

federal agencies with experience in data collection and occupational 

information have resources that SSA could leverage. SSA officials 

told us that they have been considering how they could benefit from 

the infrastructure that other agencies such as Labor and the U.S. 

Census Bureau have for data collection, but that they have not 

identified specific ways to leverage those resources. Additionally, 

while SSA needs to assure the OIS serves its most basic needs, it 

may be appropriate for SSA to explore the possibility of cost sharing 

or applying users’ fees with organizations that currently rely on the 

outdated DOT and could ultimately benefit from using a more modern 

and comprehensive OIS database.40

 

40Current users of the DOT include organizations in the private and public sectors, such 

as companies providing life insurance and short and long term disability insurance, and 

organizations providing state and federal vocational rehabilitation services, among others. 
 

Page 25 GAO-12-420 Modernizing SSA Disability Criteria 

Although SSA’s adult disability programs were initially built upon the 

assumption that certain severe medical conditions equate to work 

incapacity, through its medical listings updates and ongoing research, SSA 

has taken steps to modify its eligibility criteria. SSA now is taking a more 

modern view of disability that looks beyond the claimant’s medical condition 

by giving greater consideration to his or her functional capacity, that reflects 

the International Classification of Functioning, Disability and Health (ICF) 

framework.41

Updates to the medical listings have been one vehicle through which SSA 

can include an assessment of an individual’s functional abilities to 

determine whether an impairment prevents work. For example, as part of 

SSA’s comprehensive revision to the listings for the immune system, the 

agency included several functional criteria, such as performing activities 

of daily living, maintaining social functioning, and completing tasks in a 

According to the 2007 Institute of Medicine report, a modern 

concept of disability should recognize that disability is not just inherent in 

the individual and his or her medical condition, but is the result of complex 

interactions between the person, the person’s medical impairments, 

assistive devices to which they have access, and features of his or her 

socioeconomic environment, such as the presence or lack of accessible 

transportation and workplace accommodations. Under this concept, as 

described in this report, two people with the same impairment might have 

different degrees of work disability for a variety of reasons.

 

41As noted earlier, the ICF focuses on ability to function despite a medical impairment, 

including taking into account the impact of environmental factors, such as products and 

technology, attitudes, and services. The World Health Organization developed the ICF as 

a universal classification of disability and health for use in health and health-related 

sectors, especially as a planning and policy tool for decision makers. All 191 World Health 

Organization member states endorsed the use of the ICF as the international standard to 

describe and measure health and disability. As a result, the international community now 

has classification schema that provides both a common language and underlying code for 

disability terminology. There are four basic ICF domains, with associated codes, in the ICF 

classification: Body Functions, Body Structures, Activities and Participation, and 

Environmental Factors.

SSA Has Taken Other 

Steps to Modernize 

Criteria but Faces 

Limitations

SSA Has Taken Steps to 

Incorporate Modern 

Concepts of Disability into 

Eligibility Criteria
 

Page 26 GAO-12-420 Modernizing SSA Disability Criteria 

timely manner despite deficiencies in concentration or persistence.

Generally, SSA officials, adjudicators, and disability experts we spoke 

with support incorporating appropriate functional criteria into the medical 

listings to facilitate a more reliable assessment of an individual’s ability to 

work. However, some have also noted that as SSA continues to more 

broadly incorporate functional criteria into listings updates, such efforts 

may result in a more subjective assessment by adjudicators that could 

increase the difficulty of making step 3 disability determinations.

Since 2008, SSA has had an ongoing interagency agreement with the 

National Institutes of Health (NIH) to conduct short- and long-term 

research that has informed SSA’s efforts to incorporate functional 

information into the disability criteria. For example, in its 2011 annual 

report to SSA, NIH presented its findings on the use of functional criteria, 

as defined by the ICF, in the adult listings. The objective of the project 

was to quantify the comprehensiveness and consistency of functional 

criteria among the adult listings and any influence this has in 

determination of outcomes. NIH found that while the use of functional 

terms were particularly apparent in five body systems, nearly one-half of 

the 14 body systems did not capture the influence of health conditions 

and impairment on human functioning. NIH is currently working on 

another project, which examines the presence and consistency of 

functional terms in the listings criteria. It is estimated the project will be 

completed by 2013.

Besides exploring how medical listings could further incorporate 

functional considerations, SSA is also sponsoring longer term research 

through the NIH to develop an automated method to more quickly and 

comprehensively evaluate a claimant’s functional abilities for its disability 

determinations. Specifically, project researchers are developing a 

computer-based tool to rapidly and reliably assess the functional abilities 

of individual claimants considering their medical conditions.42

 

42Boston University’s Health and Disability Research Institute is developing this tool under 

a subcontract with NIH. The tool is based on Item Response Theory using Computer 

Adaptive Testing. The tool is adaptive in the sense that each item that is administered is 

tailored to the unique level of functioning of an individual. The items are selected based on 

the individual’s prior responses assessing that individual’s level of function, while precision 

is gained regarding a person’s placement along a continuum of functioning by asking 

more questions at the level that is appropriate for that individual.

 As 

envisioned, the claimant, a medical provider, or both would respond to a 

series of questions through the computer-based tool, which would likely 
 

Page 27 GAO-12-420 Modernizing SSA Disability Criteria 

take an hour to complete the entire battery, perhaps less if a more limited 

assessment is warranted. The project’s intended scope encompasses six 

areas of function consistent with the ICF activity domain: (1) mobility, (2) 

interpersonal/social interaction and relationships, (3) self-care, (4) 

communication, (5) general tasks and demands (such as multitasking and 

carrying out daily routines), and (6) learning and applying knowledge. NIH 

researchers said they plan to consider the use of common personal 

assistive devices, such as wheelchairs, in developing the tool. As of April 

2012, NIH had completed preliminary testing of instruments in two of the 

six areas—mobility and interpersonal/social interaction and 

relationships—with a sample of claimants and providers and had begun 

analyzing the data. SSA and NIH officials anticipate several benefits from 

the functional assessment tool, such as providing information on the 

impact of impairments more consistently, comprehensively, and early in 

the disability determination process. This information would help 

adjudicators more accurately assess whether a person can perform 

certain kinds of work given his or her functional and occupational 

capabilities.

While this research is promising, there are several unknown variables.

SSA officials said they have not yet determined when or how the tool will 

be integrated into the disability determination process. NIH officials 

indicated they believe that the tool may be most useful early in the 

determination process. SSA officials said they expect to pilot the 

functional assessment tool after all relevant domains have been tested 

and validated, which will likely be by 2016. Additionally, NIH researchers 

told us that, ultimately, this tool, which collects information on the 

applicant, would need to be aligned with the OIS, which collects 

information on the demands of work. SSA and NIH officials reported they 

have held meetings to share their progress to date, but both projects are 

in the early stages and additional coordination will be needed going 

forward.

SSA is in the preliminary stages of taking additional steps to broaden its 

use of functional criteria throughout the disability determination process in 

response to other findings by NIH. Specifically, NIH evaluated a number 

of forms that SSA uses during the disability application and determination 

processes to determine how well these forms captured information on 

claimant functional activity as described by the ICF. The researchers 

found that current SSA assessment processes had major gaps in 

covering the ICF concepts. For example, SSA captured only limited 

information relative to sensory experiences, such as watching or listening, 

learning and applying knowledge, communication, and interpersonal 
 

Page 28 GAO-12-420 Modernizing SSA Disability Criteria 

interactions and relationships, all of which NIH considered particularly 

relevant to work. NIH concluded that the gaps need to be addressed in 

order to characterize individual functioning more comprehensively in 

relationship to the demands of the workplace. Another step the agency 

reports that it plans to take is the issuance of a Federal Notice of 

Solicitation of Collaboration from federal agencies in developing a 

standard for coding functional capacity in federal disability programs 

based on the ICF. The notice will explain that SSA believes using the ICF 

would help standardize how agencies describe and measure different 

aspects of disability, improve the clarity and comparability of research 

findings, and strengthen the base of scientific knowledge that guides 

public policies and health practices. In addition, SSA has asked the 

Institute of Medicine to plan an international symposium focused on how 

best to use and assess function in the disability determination process.

SSA officials also stated that the agency may ultimately revise the 

residual functional capacity forms based on criteria from the ICF.

Consistent with modern views of disability, a key consideration in 

assessing disability is that the environment can hinder or enhance an 

individual’s ability to function. While assistive devices and workplace 

accommodations can play a critical role in an individual’s ability to 

function in the work environment, SSA does not always consider them in 

its assessment of disability. Regarding the incorporation of assistive 

devices into the medical listings, SSA officials and experts we spoke with 

expressed concern about the extent to which specific technologies should 

be incorporated if they are not widely available. SSA officials told us that 

they currently incorporate assistive devices into the medical listings once 

these devices become standard in the medical community—a threshold 

that SSA officials described as generally involving some combination of 

availability, accessibility, and insurance coverage. After an assistive 

device, such as a prosthetic device for walking, is incorporated into a 

listing, adjudicators must evaluate the individual’s ability to walk with the 

device being used. For example, evaluations of people who have had 

amputations involving a lower extremity or extremities are to be done with 

the prosthetic device in place. When we asked if wheelchairs are 

considered standard in the medical community, and whether SSA 

considers how individuals with wheelchairs might function in today’s 

knowledge-based labor market, given their age and education, we heard 

conflicting information from SSA officials. Some explained that individuals 

in wheelchairs are generally allowed at step 3 on the basis of their 

underlying medical condition, without evaluating how their disability might 

be assessed at steps 4 or 5. Other officials maintained that according to 

SSA Faces Constraints to 

Incorporating Modern 

Concepts
 

Page 29 GAO-12-420 Modernizing SSA Disability Criteria 

SSA’s policy, an individual in a wheelchair would be evaluated for upper 

body strength or other medical issues. However, SSA officials we spoke 

with said they have not evaluated these types of allowances specifically.

Regarding workplace accommodations, SSA officials said their policy is to 

not consider them for several reasons. First, officials cited SSA’s inability 

to ensure that workplace accommodations are provided by employers—a 

concern shared by other disability experts we interviewed. SSA officials 

also indicated the agency would be unable to assess the effectiveness of 

workplace accommodations for claimants. Further, officials noted that 

SSA already faces resource constraints managing its disability claims 

workload and expanding the scope of individualized assessments would 

exacerbate those constraints. Finally, they noted that data on the 

availability and use of workplace accommodations are lacking.

SSA’s policy notwithstanding, some opportunities exist for SSA to learn 

more about the availability of workplace accommodations. For example, 

developing the OIS may provide an opportunity to obtain some, albeit 

limited, information on workplace accommodations for the disability 

determination process. Specifically, while SSA officials reported that they 

do not plan to collect specific information on workplace accommodations 

as they develop the system, they may collect more specific information on 

the physical requirements of different jobs, such as options for sitting or 

standing. Some experts we spoke with agreed that while information on 

workplace accommodations would be immensely useful to include in the 

OIS, given the current scope of that project, they agreed collecting this 

information via the OIS would likely be too great a task for SSA to 

accomplish with existing budget constraints and time frames. However, 

another potentially viable option for collecting more information about 

workplace accommodations might be the Disability Research Consortium, 

which SSA is creating pursuant to section 1110 of the Social Security Act, 

as amended.43

 

43See 42 U.S.C. § 1310.

SSA envisions the consortium as a 5-year cooperative 

agreement that will serve as a national resource for fostering high quality 

research, communication, and education on matters related to disability 

policy, such as identifying or eliminating barriers encountered by people 

with disabilities in returning to or maintaining work.
 

Page 30 GAO-12-420 Modernizing SSA Disability Criteria 

Researchers and disability experts have commented on the limitations of 

SSA’s current disability program eligibility criteria to fully consider whether 

an individual can function in the workplace due to limited consideration of 

assistive devices, including those in the workplace—limitations that we 

have also noted.44 Moreover, officials we spoke with from an organization 

of vocational examiners expressed frustration when they see young 

individuals who could work with minor accommodations being provided 

disability benefits likely throughout their working life, rather than receiving 

support to pursue work. Representatives of the organization added that 

minor accommodations can include a stool for sitting or devices to assist 

with vision impairments. Although giving broad consideration of assistive 

devices and workplace accommodations may be difficult to incorporate 

into the current disability criteria and process, considering how common 

and inexpensive workplace supports may affect work disability seems 

feasible and reasonable.

SSA has taken important and concrete steps toward modernizing its 

disability criteria, but varied challenges may prevent the agency from 

achieving its goals. Specifically, SSA has a better process in place for 

updating medical listings but delays in meeting its own goals will likely 

continue unless the agency explicitly identifies and assigns the resources 

needed to achieve them. SSA is also taking bold and needed steps to 

replace outdated occupational information with a new OIS. However, 

especially in an era of governmentwide fiscal constraints, success of its 

OIS may hinge on SSA prioritizing its analysis of risks and feasible 

alternatives to address potential funding shortfalls and other significant 

challenges. Regardless of the shape and scope of the OIS, absent a 

complete, reliable, and transparent cost estimate and schedule, SSA risks 

schedule and performance shortfalls. Finally, SSA is sponsoring 

promising research to increase consideration of functional ability in its 

medical listings and other aspects of eligibility criteria—research that 

experts believe must be aligned with the design of SSA’s new OIS system 

at the earliest opportunity to best serve the disability determination 

process. On the other hand, SSA is not considering the full range of 

assistive devices and workplace accommodations available today. While 

SSA and others raise valid concerns as to their universal availability and 

 

44GAO, SSA Disability Programs: Fully Updating Disability Criteria Has Implications for 

Program Design, GAO-02-919T (Washington, D.C.: July 11, 2002).

Conclusions
 

Page 31 GAO-12-420 Modernizing SSA Disability Criteria 

other considerations, in the absence of studies on how certain assistive 

devices and workplace accommodations are playing a role in helping 

individuals with impairments to stay at work or return to work, and their 

costs in comparison with potentially many years of disability benefit 

payments, SSA may be missing an opportunity to potentially assist 

individuals with disabilities to re-engage with the workforce and to target 

finite resources efficiently and effectively.

1. To achieve the goal of updating listings for each body system within 

SSA’s 5-year time frame, we recommend that the Commissioner of 

Social Security explicitly identify the resources needed to achieve this 

goal, such as staff, contractors, and technology aids, and its plans to 

overcome any resource limitations.

2. To ensure that its work to revise occupational information is feasible 

and cost effective, and to improve its chance for success, we 

recommend that the Commissioner of Social Security:

• formally assess risks to the success of the OIS—addressing such 

challenges as related to controlling cost, acquiring expertise, 

managing project complexity, and coordinating with ongoing and 

related SSA research—and develop appropriate mitigation 

strategies, and

• develop a comprehensive and reliable cost estimate and schedule 

for the life cycle of the project, in accordance with best practices.

3. To help ensure that SSA’s disability decisions are as equitable and 

consistent with modern views of disability as possible, we recommend 

the Commissioner of Social Security conduct limited and focused 

studies on the availability and effects of considering more fully 

assistive devices and workplace accommodations in its disability 

determinations.

We provided a draft copy of this report to SSA and the Departments of 

Commerce, Health and Human Services, and Labor for review and 

comment. The Department of Commerce did not provide comments. 

SSA, Labor, and the Department of Health and Human Services provided 

technical comments, which we incorporated as appropriate. SSA also 

provided written comments, which are reproduced in appendix V.

Recommendations for 

Executive Action

Agency Comments 

and Our Evaluation
 

Page 32 GAO-12-420 Modernizing SSA Disability Criteria 

SSA agreed with the first two recommendations and disagreed with the 

third recommendation that the agency should conduct limited and focused 

studies on the availability and effects of considering more fully assistive 

devices and workplace accommodations in its disability determinations. 

SSA stated that conducting such studies is inconsistent with Congress’ 

intentions. Specifically, SSA noted that the Americans with Disabilities 

Act of 1990 (ADA), as amended, addresses requirements for workplace 

accommodations. SSA referred to a 1999 Supreme Court case which 

acknowledged the complexity of SSA involvement in determining the 

availability of reasonable accommodations and also noted that ADA

determinations are separate from disability determinations under the 

Social Security Act. SSA asserts that because Congress has made no 

effort to change the balance between its disability programs and the ADA 

in the past 22 years, it would be inappropriate for SSA to spend its limited 

administrative resources on “an initiative that would undermine the 

balance Congress chose to strike.”

Notwithstanding SSA’s response, we continue to believe that SSA should 

conduct limited and focused studies on the availability and effects of 

considering more fully assistive devices and workplace accommodations 

in its disability determination process. Although SSA asserts that 

workplace accommodations are addressed by the ADA, this act serves a 

very different purpose than SSA’s disability programs. We do not think 

that the fact that workplace accommodations are addressed by the ADA 

would necessarily preclude SSA from potentially considering them in 

making disability determinations. Further, although we agree with SSA 

that Congress has not explicitly directed the agency to consider 

incorporating assistive devices and workplace accommodations into its 

disability determinations process, we also note that Congress has not 

explicitly prohibited SSA from making such considerations. Because the 

language of the Social Security Act is silent on this issue, and in the 

absence of clear Congressional intent, we think it would be reasonable for 

SSA to conduct limited studies on the feasibility of considering such 

factors.

SSA further cites the 1999 Supreme Court decision in Cleveland v. Policy 

Management Systems Corporation to point out that assessing reasonable 

accommodation may turn on highly disputed workplace-specific matters, 

and that an SSA misjudgment about that detailed matter could deprive a 

disabled person of the financial support the statute provides. While this is 

true, SSA fails to acknowledge that by not incorporating such factors, it 

may be providing benefits to individuals who are capable of working with 

accommodations, thereby potentially expending scarce government 
 

Page 33 GAO-12-420 Modernizing SSA Disability Criteria 

resources in a manner that may not be most appropriate. SSA’s response 

acknowledges that the agency’s resources are limited—a concern that we 

share. By conducting limited and focused studies on this issue, SSA 

would be in a better position to thoughtfully weigh the costs and benefits 

of these various policy options before deciding on an appropriate course 

of action. 

As we discussed in the report, SSA has avenues to research the 

availability of select devices or accommodations and the impact of their 

inclusion on disability determinations, such as through the SSA funded 

Disability Research Consortium, which is charged to conduct research to 

identify and eliminate barriers encountered by people with disabilities in 

returning to work or maintaining their ability to work. SSA could also 

incorporate discussion of assistive devices and workplace 

accommodations into its planned symposium with the Institute of 

Medicine that will assess the inclusion of functional measures in the 

disability determination process. Through these efforts, SSA may be able 

to identify certain common and widespread workplace accommodations 

that could be incorporated into the disability criteria, just as it incorporates 

certain medical treatments and assistive devices into the medical listings 

when they become a medical standard. Despite SSA’s efforts to better 

address an individual’s functioning in its disability criteria, without at least 

evaluating the costs and benefits of considering more assistive devices 

and workplace accommodations, SSA is likely to face ongoing barriers in 

its efforts to integrate a more modern concept of disability into its 

determination process.

As agreed with your office, unless you publicly announce the contents of 

this report earlier, we plan no further distribution until 30 days from the 

report date. At that time, we will send copies of this report to the 

Commissioner of the Social Security Administration and the Secretaries 

of Commerce, Health and Human Services, and Labor; relevant 

congressional committees; and other interested parties. In addition, this 

report will be available at no charge on GAO’s website at 

http://www.gao.gov. 
 

Page 34 GAO-12-420 Modernizing SSA Disability Criteria 

If you or your staff have any questions about this report, please contact 

me at (202) 512-7215 or bertonid@gao.gov. Contact points for our 

Offices of Congressional Relations and Public Affairs may be found on 

the last page of this report. GAO staff who made key contributions to this 

report are listed in appendix VI.

Sincerely yours,

Daniel Bertoni, Director

Education, Workforce, and Income Security Issues
Appendix I: Objectives, Scope, and 

Methodology

Page 35 GAO-12-420 Modernizing SSA Disability Criteria 

We were asked to assess the Social Security Administration’s (SSA) 

plans and efforts to revise its disability criteria. Specifically, we examined 

the status and management of efforts to update its medical listings to 

reflect current medical knowledge and develop a new occupational 

information system (OIS) to reflect labor market changes, and we 

identified other steps taken by SSA to incorporate a modern view of 

disability into its eligibility criteria. For this review, we narrowed our scope 

to criteria used for initial adult disability determinations that involve the 

medical listings at step 3 or a medical-vocational assessment at steps 4 

and 5 of SSA’s five-step sequential process.

To determine the status of activities to revise the medical listings and to 

develop the OIS, we reviewed our prior reports on the subject and SSA 

Office of Inspector General reports, relevant federal laws and regulations, 

and program documentation. This documentation included policies, such 

as those listed in SSA’s Program Operations Manual System for 

examiners; strategic goals for fiscal years 2011 and 2012 presented in 

the Agency Performance Plan for 2012; and other guidance. We also 

interviewed SSA officials from relevant offices with direct responsibility for 

revising disability criteria, as well as those offices that provide support for 

these efforts, and key project contractors and stakeholders. To obtain 

contextual information on modernization, we reviewed relevant literature, 

including studies, position papers, and testimonies from disability groups 

and commissions and interviewed disability experts. We assessed the 

reliability of data used in this report and found it to be sufficiently reliable 

for our purposes.

To determine the extent to which SSA’s efforts to revise its medical 

listings and develop a new OIS were anchored in sound project 

management practices, we first identified sound project management 

practices by reviewing A Guide to the Project Management Body of 

Knowledge, our guidance on internal controls, and the Government 

Performance and Results Act of 1993. We also identified our recent work 

(for fiscal years 2008, 2009, and 2010) that evaluated federal planning 

efforts. Through these efforts, we identified six practices for sound project 

management which we used as a framework for evaluating SSA’s efforts 

to revise the medical listings and develop the OIS. Although there is no 

established set of requirements for all plans, we determined that these 

practices help implementing parties and decision makers effectively 

shape policies, programs, priorities, and resource allocations so that they 

can achieve desired results while ensuring accountability. While these 

practices may be organized in a variety of ways or use different terms, for 

the purposes of this report, we grouped them into six categories, from 

Appendix I: Objectives, Scope, and 

Methodology 
Appendix I: Objectives, Scope, and 

Methodology

Page 36 GAO-12-420 Modernizing SSA Disability Criteria 

plan conception to implementation. Given the differences among SSA’s 

efforts to update the medical listings and develop the OIS, we selected 

broad, higher-level criteria that may apply to a wide variety of projects or 

plans (see table 5). 

Table 5: Six Sound Project Management Practices Used to Evaluate SSA’s Efforts 

Project management 

practice Description

Plan to identify problem, 

goals, and methodology

In establishing the problem, goals, and methodology, the agency identifies the problem to be addressed 

and the causes of the problem, as well as the strategy, including goal and objectives, and the methodology 

for achieving these goals and objectives.

Activities and timeline An agency should identify and document the specific activities that must be performed in order to complete 

the project. The agency should develop a schedule that defines, among other things, when work activities 

will occur, how long they will take, and how they are related to one another, as well as interim milestones 

and checkpoints to gauge the completion of the project.

Resources The agency should identify the sources and types of resources or investments needed (e.g., budgetary, 

human capital, information technology, research and development, contracts) to complete project activities. 

If resource availability cannot be assured, the agency will need to establish alternate plans. The agency 

should develop a reliable estimate of the costs of these resources.

Coordination The agency should identify stakeholders—individuals and organizations that are involved in or may be 

affected by project activities—and ensure that they are included in developing and executing the project 

plan, allowing them to contribute appropriately. In addition to internal communications, management 

should ensure there are adequate means of communicating with, and obtaining information from, external 

stakeholders that may have a significant impact on the agency achieving its goals.

Risk Risk assessment generally includes estimating the significance of risks from both external and internal 

sources, assessing the likelihood of its occurrence, and deciding how to manage the risk.

Performance evaluation The agency should describe how goals will be achieved and establish performance indicators to be used in 

measuring or assessing the relevant outputs, service levels, and outcomes of each program activity, and 

identify a process to monitor and report on progress.

Source: GAO analysis based on Project Management Institute information, the Government Performance and Results Act of 1993, and 

GAO reports.

Additionally, we compared the cost estimate and schedule for completing 

SSA’s OIS and related documents with best practices in our Cost 

Estimating and Assessment Guide. We compared these practices to the 

OIS project because it requires a significant commitment of resources 

and time by SSA to complete and will result in an end product. For SSA’s 

OIS cost estimate and schedule, we scored each best practice as follows:

• Not met—SSA provided no evidence that satisfies any of the criteria.

• Minimally met—SSA provided evidence that satisfies a small portion 

of the criteria.
Appendix I: Objectives, Scope, and 

Methodology

Page 37 GAO-12-420 Modernizing SSA Disability Criteria 

• Partially met—SSA provided evidence that satisfies about half of the 

criteria.

• Substantially met—SSA provided evidence that satisfies a large 

portion of the criteria.

• Met—SSA provided complete evidence that satisfies the entire 

criteria.

We provided the results of our schedule and cost analyses to SSA 

officials and met with them to confirm the results. Based on the interviews 

and additional documentation provided by SSA officials, we updated the 

results of our analyses, as needed.
Appendix II: Progress Updating Six Body 

Systems since Last Comprehensive Revision 

Page 38 GAO-12-420 Modernizing SSA Disability Criteria 

Note: SSA’s special senses and speech listings include impairments associated with speech and 

balance (i.e., disorders of the labyrinthine-vestibular function). 

Appendix II: Progress Updating Six Body 

Systems since Last Comprehensive Revision
Appendix III: Description of Cost Estimating 

Best Practices 

Page 39 GAO-12-420 Modernizing SSA Disability Criteria 

Characteristica Characteristic description

Comprehensive The cost estimate should include all costs necessary to achieve agency objectives including government and 

contractor labor costs as well as any necessary material or equipment costs. Comprehensive cost estimates 

should be structured in sufficient detail to ensure that cost elements are neither omitted nor double counted. 

Specifically, the cost estimate should be based on a standardized structure that allows a program to track cost 

and schedule consistently over time. Finally, where information is limited and judgments must be made, the cost 

estimate should document all cost-influencing ground rules and assumptions. 

Well documented The cost estimate should be supported by detailed documentation that describes the purpose of the estimate, 

the program background and system description, the scope of the estimate, the ground rules and assumptions, 

all data sources, estimating methodology and rationale, and the results of the risk analysis. Moreover, this 

information should be captured in such a way that the data used to derive the estimate can be traced back to, 

and verified against, the sources.

Accurate The cost estimate should be based on an assessment of most likely costs (adjusted for inflation), documented 

assumptions, historical cost estimates, and actual experiences on other comparable programs. Estimates should 

be cross-checked against an independent cost estimate for accuracy, double counting, and omissions. In 

addition, the estimate should be updated to reflect any changes.

Source: GAO analysis. 

a

We did not assess SSA’s estimate on a fourth characteristic—credibility—which evaluates any 

limitations of the analysis because of uncertainty or biases surrounding data or assumptions, as this 

criterion is most applicable to major capital acquisitions. 

Appendix III: Description of Cost Estimating 

Best Practices
Appendix IV: Description of Scheduling Best 

Practices 

Page 40 GAO-12-420 Modernizing SSA Disability Criteria 

Best practice Description

Capturing all activities A schedule should reflect all activities defined in the program’s work breakdown structure and include 

all activities to be performed by the government and contractor.

Sequencing all activities The schedule should be planned so that all activities are logically sequenced in the order they are to 

be carried out.

Assigning resources to all 

activities

The schedule should realistically reflect the resources (i.e., labor material and overhead) needed to 

do the work, whether all required resources will be available when needed, and whether any funding 

or time constraints exist.

Establishing the duration of all 

activities

The schedule should reflect how long each activity will take to execute.

Integrating schedule activities 

horizontally and vertically

The schedule should be horizontally and vertically integrated—that is, it should link already 

sequenced activities with outcomes while also delineating the relation of supporting tasks and 

subtasks to upper-level milestones. Such mapping among levels enables different groups to work to 

the same master schedule.

Establishing the critical path for 

all activities

The schedule should identify the critical path, or those activities that, if delayed, will negatively impact 

the overall project completion date. The critical path enables analysis of the effect delays may have 

on the overall schedule.

Identifying float between 

activities

The schedule should identify float—the amount of time an activity can slip in the schedule before it 

affects other activities—so that flexibility in the schedule can be determined. As a general rule, 

activities along the critical path typically have the least amount of float.

Conducting a schedule risk 

analysis

The schedule should include a schedule risk analysis that uses statistical techniques to predict the 

probability of meeting a completion date. A schedule risk analysis can help management identify and 

understand the most important risks and focus on mitigating them.

Updating the schedule using 

logic and durations to 

determine dates

The schedule should use realistic durations for activities and be monitored to determine when 

forecasted completion dates differ from the planned dates. This analysis can be used to assess 

whether schedule variances will affect future work.

Source: GAO analysis.

Appendix IV: Description of Scheduling Best 

Practices 
Appendix V: Comments from the Social 

Security Administration

Page 41 GAO-12-420 Modernizing SSA Disability Criteria 

Appendix V: Comments from the Social 

Security Administration
Appendix V: Comments from the Social 

Security Administration

Page 42 GAO-12-420 Modernizing SSA Disability Criteria 
Appendix V: Comments from the Social 

Security Administration

Page 43 GAO-12-420 Modernizing SSA Disability Criteria 
Appendix VI: GAO Contact and Staff 

Acknowledgments

Page 44 GAO-12-420 Modernizing SSA Disability Criteria 

Daniel Bertoni, (202) 512-7215 or bertonid@gao.gov

In addition to the contact named above, Michele Grgich, Assistant 

Director; James Bennett; Kate Blumenreich; Tisha Derricotte; Jennifer 

Echard; Julie DeVault; Alex Galuten; Sheila McCoy; Patricia M. Owens; 

Carol Petersen; Karen Richey; Anjali Tekchandani; Kathleen Van Gelder; 

and Walter Vance made key contributions to this report. Also contributing 

to this report were Jaime Allentuck, Susan Bernstein, and Amy Frazier.

Appendix VI: GAO Contact and Staff 

Acknowledgments 

GAO Contact

Staff 

Acknowledgments

(131069)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (www.gao.gov). Each weekday afternoon, 

GAO posts on its website newly released reports, testimony, and 

correspondence. To have GAO e-mail you a list of newly posted products, 

go to www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.