Report to the Chairman, Subcommittee 

on Social Security, Committee on Ways 

and Means, House of Representatives 

United States General Accounting Office 

GAO

July 2004 SOCIAL SECURITY 

ADMINISTRATION 

More Effort Needed to 

Assess Consistency of 

Disability Decisions 

GAO-04-656 
www.gao.gov/cgi-bin/getrpt?GAO-04-656.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact Robert 

Robertson, 202-512-7215, 

robertsonr@gao.gov. 

Highlights of GAO-04-656, a report to the 

Chairman, Subcommittee on Social 

Security, Committee on Ways and Means, 

House of Representatives 

July 2004

SOCIAL SECURITY ADMINISTRATION

More Effort Needed to Assess 

Consistency of Disability Decisions 

SSA has only partially implemented its process unification initiative. 

Although the agency initially made improvements in its policies and training 

intended to address inconsistency in decisions made at the two adjudication 

levels, it has not continued to actively pursue these efforts. Further, as part 

of this initiative, the agency implemented a review of hearings level 

decisions to identify ways to improve training and policies, but no new 

improvements were made as a result of the review. Finally, the agency 

began tests of two process changes intended to improve the consistency of 

decision making between the two adjudication levels. One test, which is 

ongoing, was not well designed and therefore will not provide conclusive 

results. The other test was abandoned because of implementation 

difficulties. 

SSA’s assessments have not provided a clear understanding of the extent and

causes of possible inconsistencies in decisions between adjudication levels. 

The two measures SSA uses to monitor inconsistency of decisions have 

weaknesses, such as not accounting for the many factors that can affect 

decision outcomes, and therefore do not provide a true picture of the 

changes in consistency. Furthermore, SSA has not sufficiently assessed the 

causes of possible inconsistency. For example, SSA conducted an analysis 

in 1994 that identified potential areas of inconsistency, but it did not employ 

more sophisticated techniques—such as multivariate analyses, followed by 

in-depth case studies—that would allow the agency to identify and address 

the key areas and leading causes of possible inconsistency. SSA has yet to 

repeat or expand upon this 10-year-old study. 

SSA’s new proposal incorporates changes intended to improve consistency 

in decisions between levels. However, challenges may hinder its 

implementation. Most stakeholder groups for adjudicators and claimant 

representatives told us that a number of aspects of the proposal hold 

promise for improving consistency. These included one change, being tested 

as part of the process unification initiative, that requires state adjudicators 

to more fully develop and document their decisions, as well as several new 

changes, such as providing both adjudication levels with equal access to 

medical expertise. However, stakeholder groups also told us that 

insufficient resources and other obstacles might hinder the implementation 

of some changes. Adding to uncertainties about the proposal’s overall 

success is its dependence on a new electronic folder system that would 

allow cases to be easily accessed by various adjudicators across the country. 

However, this technically complex project has not been fully tested. 

Each year, about 2.5 million people 

file claims with the Social Security 

Administration (SSA) for disability 

benefits. If the claim is denied at 

the initial level, the claimant may 

appeal to the hearings level. The 

hearings level has allowed more 

than half of all appealed claims, an 

allowance rate that has raised 

concerns about the consistency of 

decisions made at the two levels. 

To help ensure consistency, SSA 

began a “process unification” 

initiative in 1994 and recently 

announced a new proposal to 

strengthen its disability programs. 

This report examines (1) the status 

of SSA’s process unification 

initiative, (2) SSA’s assessments of 

possible inconsistencies in 

decisions between adjudication 

levels, and (3) whether SSA’s new 

proposal incorporates changes to 

improve consistency in decisions 

between adjudication levels. 

 To build an effective strategy to 

address possible inconsistencies in 

its decisions, we recommend that 

SSA quickly expand its assessment 

of inconsistency by implementing 

several specific enhancements. 

In its comments, SSA had some 

reservations concerning our 

findings, conclusions, and 

recommendations but agreed to 

pilot one recommendation and 

consider the others as it refines its 

new proposal. We continue to 

believe that SSA should implement 

our recommendations without 

delay to ensure the effectiveness of 

its new proposal. 
Page i GAO-04-656 SSA Disability Decisions 

Letter 1

Results in Brief 3

Background 5

SSA Has Partially Implemented Its Process Unification Initiative 14

SSA Lacks a Clear Understanding of the Extent and Causes of 

Inconsistency between Levels 20

SSA’s New Proposal Incorporates Efforts to Improve the 

Consistency of Decisions, but Challenges May Impede 

Successful Implementation 26

Conclusions 32

Recommendations 33

Agency Comments and Our Evaluation 34

Appendix I Excerpt of SSA’s Testimony Announcing Its New 

Proposal to Improve Its Disability Decision-Making 

Process 38 

Appendix II Comments from the Social Security Administration 44 

GAO Comments 50

Appendix III GAO Contacts and Staff Acknowledgments 53 

GAO Contacts 53

Staff Acknowledgments 53

Tables 

Table 1: SSA’s Process Unification Efforts 9

Table 2: Newly Proposed Changes to the Disability DecisionMaking Process 13

Figures 

Figure 1: SSA’s Disability Decision-Making Process and Outcomes 

for Fiscal Year 2003 7

Figure 2: SSA Allowances at Initial Level versus Hearings Level, by 

Proportion of Allowances 21

Contents 
Page ii GAO-04-656 SSA Disability Decisions 

Abbreviations 

AeDib Accelerated Electronic Disability System 

ALJ Administrative Law Judge 

DDS Disability Determination Services 

DI Disability Insurance 

OASDI Old-Age, Survivors and Disability Insurance 

OHA Office of Hearings and Appeals 

OQA Office of Quality Assurance and Performance Assessment 

RO Reviewing Official 

SSA Social Security Administration 

SSI Supplemental Security Income 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 
Page 1 GAO-04-656 SSA Disability Decisions 

July 2, 2004 

The Honorable E. Clay Shaw, Jr. 

Chairman, Subcommittee on Social Security 

Committee on Ways and Means 

House of Representatives 

Dear Mr. Chairman: 

The Social Security Administration (SSA) is the nation’s largest provider 

of income assistance to individuals with disabilities, paying $91 billion in 

federal benefits to 11.4 million beneficiaries with a disability and their 

families in 2003.1

 Each year, about 2.5 million people file claims with SSA 

for disability benefits. State agencies called Disability Determination 

Services (DDS) decide whether claimants meet SSA’s definition of 

disability by applying SSA’s decision-making criteria. If a DDS ultimately 

decides, after an initial determination and then a reconsideration of this 

decision, that a claimant does not meet SSA’s definition for disability, the 

claimant may appeal to the hearings level, where an SSA Administrative 

Law Judge (ALJ) reviews the claim to decide if the claimant should be 

allowed benefits. About one-third of disability claims denied at the state 

level were appealed to the hearings level; of these, SSA’s ALJs have 

allowed over one-half, with annual allowance rates fluctuating between 58 

percent and 72 percent since 1985. While it is appropriate that some 

appealed claims, such as those in which a claimant’s impairment has 

worsened and prohibits work, be allowed benefits, representatives from 

SSA, the Congress, and interest groups have long been concerned that the 

high rate of claims allowed at the hearings level may indicate that decision 

makers at the two levels are interpreting and applying SSA’s criteria 

differently. If this is the case, adjudicators at the two levels may be making 

inconsistent decisions that result in similar cases receiving dissimilar 

decisions. 

Concerned about the possibility that adjudicators are making inconsistent 

decisions, SSA embarked on a “process unification” initiative in 1994 with 

 1

The figures include federal payments for the Disability Insurance and the Supplemental 

Security Income programs to beneficiaries who have a disability or are blind and their 

families. 

United States General Accounting Office 

Washington, DC 20548 
Page 2 GAO-04-656 SSA Disability Decisions 

the goal of ensuring that adjudicators at both levels consistently apply 

SSA’s policy guidance and make similar decisions on similar cases. Partly 

on the basis of early studies of potential causes of inconsistent decisions, 

SSA included in its process unification initiative efforts to provide 

consistent guidance to all adjudicators, clarify policy, provide training, test 

potential process changes, and perform a new quality review of 

allowances decided by ALJs. However, SSA continues to face challenges in 

its efforts to provide consistent disability decisions. These challenges, and 

others associated with modernizing its disability programs, contributed to 

our decision to include federal disability programs on our list of high-risk 

government programs.2

 In September 2003, SSA’s Commissioner unveiled 

a new proposal that laid out the vision of a long-term strategy for 

improving the disability decision-making process and helping people with 

disabilities return to work. Several of the changes in the new proposal are 

intended to improve the accuracy, timeliness, and consistency of 

decisions, such as having the DDS decision makers more fully develop and 

document their decisions, providing for centralized quality review of all 

decisions, and providing both adjudication levels equal access to medical 

expertise. 

In response to your interest in the effectiveness of SSA’s past and future 

efforts to improve and assess the consistency of decisions between levels, 

we evaluated these agency efforts. Specifically, we examined (1) the status 

of SSA’s process unification initiative, (2) SSA’s assessments of possible 

inconsistencies in decisions between adjudication levels, and (3) whether 

SSA’s new proposal incorporates changes to improve consistency in 

decisions between adjudication levels. 

To assess the extent to which SSA has implemented its planned activities 

under the initiative, we evaluated agency documentation describing SSA’s 

process unification efforts. To evaluate SSA’s efforts to assess consistency 

in decisions between levels, we interviewed officials from SSA’s Office of 

Quality Assurance and Performance Assessment (OQA) and reviewed 

summary data and reports from SSA’s quality assurance and performance 

management systems, including findings from SSA’s Disability Hearings 

Quality Review, which is a quality review of DDS adjudicators’ and ALJs’ 

decisions and the associated case files. While we evaluated SSA’s methods 

 2

For additional information on the major problems facing federal disability programs and 

our decision to designate them as high-risk, see U.S. General Accounting Office, High-Risk 

Series: An Update, GAO-03-119, (Washington, D.C.: January 2003). 
Page 3 GAO-04-656 SSA Disability Decisions 

and approaches for assessing the consistency of decisions, we generally 

did not trace figures cited by SSA back to their original source 

documents.3

 To further evaluate SSA’s process unification efforts and its 

new proposal, we interviewed selected officials who have firsthand 

knowledge about these issues, including SSA officials, staff of the Social 

Security Advisory Board, and leaders of SSA stakeholder groups 

representing state and hearings office adjudicative staff and claimant 

attorneys.4

 We also reviewed recent testimony and other documents on the 

consistency of SSA’s decisions from the Social Security Advisory Board 

and other stakeholder groups. We conducted our work between February 

2003 and March 2004 in accordance with generally accepted government 

auditing standards. Our ability to evaluate SSA’s new proposal has been 

limited by a lack of detailed information, such as specific information on 

how changes will be implemented and their costs, because the agency is 

still in the process of developing and refining its proposal. 

SSA has partially implemented its process unification initiative. At the 

beginning of its process unification efforts, the agency took decisive steps 

by issuing clarifying guidance in a number of key policy areas to all 

adjudicators. However, SSA ultimately abandoned its plans to update older 

policy guidance and to provide a single policy guide for both initial- and 

hearings-level adjudicators, concluding that these efforts would not be 

cost-effective. Similarly, while SSA initially provided extensive process 

unification training to adjudicators at both levels, the scope of SSA’s 

training efforts on this issue has since diminished. Also as part of this 

initiative, SSA implemented a quality review of ALJ allowances, in part to 

identify the need for new policies and training to improve consistency, but 

no new improvements were made as a result of findings from the review. 

Finally, the agency began two tests to determine if process changes would 

improve the consistency of decisions. However, the test of having DDS 

 3

We took additional steps to assess the reliability of data included in our report. For the 

proportion of initial and hearings levels allowances, we verified SSA’s calculations of 

summary data from its workload reporting systems. For the information from the Disability 

Hearings Quality Review, we reviewed the weights and calculations used by SSA to 

determine national support rates for reviewed decisions and found them to be correctly 

calculated and reliable. 

4

These groups included the National Association of Disability Examiners, the National 

Council of Disability Determination Directors, the Association of Attorney Advisors, the 

Association of Administrative Law Judges, the National Treasury Employee Union, the 

Public Employees Federation, the Social Security Section of the Federal Bar Association, 

and the National Organization of Social Security Claimants’ Representatives. 

Results in Brief 
Page 4 GAO-04-656 SSA Disability Decisions 

adjudicators more fully document decisions—which is ongoing—will, 

because of design flaws, be unable to provide conclusive information on 

how this change might affect the consistency of decisions. SSA abandoned 

its other test of having the initial level reevaluate appealed cases for which 

new medical information was submitted prior to the hearing, because of 

several major difficulties encountered during testing, such as difficulty 

identifying cases to be sent back to the initial level. 

SSA’s assessments have not provided the agency with a clear 

understanding of the extent and causes of possible inconsistencies in 

decisions between adjudication levels. The two measures SSA uses to 

monitor changes in the extent of inconsistency of decisions have 

weaknesses, such as not accounting for the many factors that can affect 

decision outcomes, and therefore do not provide a true picture of changes 

in consistency. SSA has also not sufficiently assessed causes of possible 

inconsistency. For example, 10 years ago SSA analyzed cases in which 

reviewers representing the initial and hearings level disagreed over the 

final decision and identified two prevalent areas of disagreement: 

assessments of claimants’ mental impairments and assessments of 

claimants’ ability to work. Although SSA continues to collect information 

that would support this analysis, it has not repeated this initial effort, nor 

has it expanded on it by employing more sophisticated techniques—such 

as multivariate analyses, followed by in-depth case studies—that would 

allow the agency to identify and address the key areas and leading causes 

of possible inconsistency. 

SSA’s recent proposal to improve the disability decision-making process 

incorporates efforts intended to address inconsistencies in decisions 

between levels. However, challenges may hinder the implementation of 

the proposal. Most stakeholder groups for SSA adjudicators and claimant 

representatives told us that several aspects of the proposal hold promise 

for improving consistency between adjudication levels. For example, they 

thought that requiring state adjudicators to more fully develop and 

document their decisions—a process change that SSA is still testing under 

its process unification initiative—might improve consistency. In addition, 

they said that other new ideas proposed by SSA, such as centralizing 

medical expertise to give both adjudication levels equal access to it, may 

also improve consistency. However, stakeholders told us that insufficient 

resources and other obstacles might hinder the implementation of some 

changes. For example, they were concerned that, as has happened in the 

past, limited resources would hinder state adjudicators’ ability to more 

fully document their decisions. Stakeholders also questioned SSA’s ability 

to attract and retain sufficient medical expertise. Adding to uncertainties 
Page 5 GAO-04-656 SSA Disability Decisions 

about the proposal is its dependence upon the successful implementation 

of a new electronic folder system that would allow cases to be easily 

accessed by adjudicators across the country. However, this technically 

complex project has not been fully tested. 

GAO is making several recommendations in this report to the 

Commissioner of Social Security to improve SSA’s methods for assessing 

the inconsistency between DDS and ALJ decisions and thereby build a 

more effective strategy to address potential inconsistencies. In 

commenting on the draft of this report, SSA had several reservations 

concerning the report’s findings, conclusions, and recommendations. 

Although SSA agreed to pilot one of our recommendations, the agency 

believes our other recommendations need to be reevaluated as the design 

of its Commissioner’s new approach to disability decision making matures. 

We continue to believe that SSA should begin implementing our 

recommendations without delay so that it has the critical information 

needed to build a new approach to decision making that will improve the 

consistency of decisions between adjudication levels. 

SSA operates the Disability Insurance (DI) and Supplemental Security 

Income (SSI) programs—the two largest programs providing cash benefits 

to people with disabilities. The law defines disability for both programs as 

the inability to engage in any substantial gainful activity by reason of a 

severe physical or mental impairment that is medically determinable and is 

expected to last at least 12 months or result in death. The programs have 

grown substantially, from 10.7 million beneficiaries and $61 billion in 

benefits in 1995 to 11.4 million beneficiaries and $91 billion in federal 

benefits to individuals with disabilities in 2003.5

 While disability benefits 

account for only 15 percent of SSA’s total benefit payments for its Old-Age, 

Survivors and Disability Insurance (OASDI) programs, administering the 

disability benefits accounted for 45 percent of the agency’s annual 

administrative expenses.6

 The relatively high cost of administering the DI 

program reflects the complex and demanding nature of making disability 

decisions. SSA estimates that the cost of the disability programs will rise 

 5

The figures include federal payments for the Disability Insurance and the Supplemental 

Security Income programs to beneficiaries who have a disability or are blind and their 

families. The figures were calculated based on statistical information from SSA’s web site. 

6

These figures are based on information provided on SSA’s web site from the 2004 OASDI 

Trustees Report, Part III. Financial Operations of the Trusts Funds and Legislative 

Changes in the Last Year. 

Background 
Page 6 GAO-04-656 SSA Disability Decisions 

substantially in the near future as the baby boom generation reaches its 

disability-prone years. 

The disability determination process begins at a field office, where an SSA 

representative determines whether a claimant meets the programs’ nonmedical eligibility criteria. Claims meeting these criteria are forwarded to 

the state DDS to determine if a claimant meets the agency’s definition of 

disability. At the DDS, the disability examiner takes the lead, or works as a 

team with the medical or psychological consultants, to analyze a 

claimant’s documentation, gather additional evidence as appropriate, and 

approve or deny the claim. A denied claimant may ask the DDS to 

reconsider its finding, at which point a different DDS team reviews the 

claim. If the claim is denied again, the claimant may appeal the 

determination to SSA’s Office of Hearings and Appeals (OHA), where it 

will be reviewed by an ALJ. The ALJ usually conducts a hearing in which 

the claimant and others may testify and present new evidence.7

 In making 

the disability decision, the ALJ uses information from the hearing and 

from the state DDS, including the findings of the DDS medical consultant. 

A claimant whose appeal is denied may request a review by SSA’s Appeals 

Council and, if denied again, may file suit in federal court. 

Figure 1 provides an overview of SSA’s disability decision-making process 

and outcomes for 2003. 

 7

Others who may testify at ALJ hearings frequently include vocational and medical experts. 
Page 7 GAO-04-656 SSA Disability Decisions 

Figure 1: SSA’s Disability Decision-Making Process and Outcomes for Fiscal Year 2003 

Notes: 

The data provided by SSA did not include the number or rate of claims appealed to the next decision 

step. Without this information, we cannot determine the extent to which claimants appealed or 

abandoned their denied claims. 

Twenty-five percent of the initial DDS determinations are subject to an alternative process that does 

not include the reconsideration step. 

Under certain specified circumstances, ALJs and Appeals Council judges can dismiss a claim. For 

example, an ALJ can dismiss a claim if the claimant’s request for a hearing is not timely and lacks a 

good cause for the delay. 

The Appeals Council can remand a claim by returning it to an ALJ for further proceedings and a new 

hearing decision. 

Because of rounding, decisional outcomes may not equal 100 percent. 

37%

Allow

63%

Deny

Initial DDS

determinations

2,477,413

15%

Allow

85%

Deny

61%

Allow

26%

Deny

13%

Dismiss

25%

Remand

72%

Deny

2% 

Allow

2%

Dismiss

DDS reconsiderations

567,425

ALJ decisions

477,568

Appeals Council

decisions

91,316

Claims

Appeals

Appeals

Appeals

Source: SSA data.
Page 8 GAO-04-656 SSA Disability Decisions 

SSA uses a sequential evaluation process when determining disability. 

First, SSA field office representatives determine whether a claimant is 

performing substantial gainful work.8

 If not, DDS or ALJ adjudicators will 

assess the severity of a claimant’s medical condition(s) to determine 

whether it meets or equals the criteria in SSA’s regulations (commonly 

referred to as the medical listings). For a claimant whose conditions do 

not meet or equal the listings, adjudicators then focus on the functional 

consequences of the claimant’s medically determined impairments—that 

is, whether the claimant can perform work he or she has done in the past, 

and, if not, whether the claimant can perform other work in the national 

economy. 

Concerns about the rate of appeals for hearings, ALJs’ allowance rates, 

and the accuracy and consistency of ALJ decisions led the Congress to 

direct SSA to conduct a study in 1980 to determine the extent to which 

hearings decisions conformed to legal requirements and binding SSA 

policy.9

 Since the allowance rates at the hearings level could be influenced 

by many factors, such as the introduction of new evidence, the purpose of 

the 1980 study was to present the same evidence on cases to different 

reviewers representing different adjudication levels. In determining the 

extent to which decision makers agreed on whether to allow or deny 

benefits, the study concluded that different levels of decision makers had 

significantly different allowance rates. Specifically, the ALJs decided to 

allow 64 percent of the cases, whereas the SSA’s central office quality 

assurance reviewers, comprising medical consultants and disability 

examiners, decided that only 13 percent of cases should be allowed. The 

study identified several possible causes of the disparity, including 

inconsistency in the standards and procedures, interpretation of the 

standards, and weight given to the evidence. The study also found that 

disability decisions are complex and necessarily involve some degree of 

subjectivity by adjudicators. 

 8

Substantial gainful work is a level of work activity that involves doing significant physical 

or mental work, or a combination of both, that is productive. SSA has established earning 

criteria as a reasonable indication of whether claimants are able to engage in substantial 

gainful activity. In 2004, SSA generally considered claimants to be engaging in substantial 

gainful activity if their earnings averaged over $810 a month. 

9

See Implementation of Section 304 (g) Public Law 96-265, Social Security Disability 

Amendments of 1980 (the Bellmon Report), Secretary of Health and Human Services 

(Washington, D.C.: January 1982). 
Page 9 GAO-04-656 SSA Disability Decisions 

To help address concerns raised by this and other studies,10 SSA began its 

process unification efforts to ensure that both levels more consistently 

interpreted and applied SSA’s policy guidance. SSA’s plans for its process 

unification initiative were part of SSA’s larger effort to redesign its 

disability claims process and were modified over time. SSA’s process 

unification plans included six major efforts, as described in table 1. 

Table 1: SSA’s Process Unification Efforts 

Process unification effort Planned activities 

Develop a single presentation 

of policy 

Publish all new guidance in the same wording to all adjudicators. 

Revise older guidance. 

Provide one policy manual for adjudicators. 

Create additional policy 

guidance 

Publish nine process unification rulings to clarify policy areas contributing to inconsistent DDS and 

ALJ decisions that are binding on all SSA adjudicators. 

Publish regulations and instructions to clarify selected process unification rulings, including (1) the 

weight to be given to the treating physician’s opinion when evaluating a claim, (2) the weight to be 

given to the DDS medical consultant’s opinion, and (3) the evaluation of the residual functional 

capacity for claimants who are limited to performing less than a full range of sedentary work. 

Publish a regulation to clarify the agency’s process for acquiescing to court decisions and an action 

plan to implement the regulation. 

Provide training Provide ongoing training to all adjudicators to increase their understanding of the three most complex 

disability areas—assessing (1) the opinion evidence from physicians and others, (2) the claimant’s 

symptoms, and (3) the claimant’s remaining capacity to work (i.e., residual functional capacity). 

Fully document DDS decisions Test procedures for fully developing and documenting DDS decisions to determine their impact on 

DDS accuracy, allowance rates, and other aspects of the claims process. 

Remand cases awaiting a 

hearing to a DDS when new 

medical evidence is received 

Test the impact of returning cases to a DDS that are awaiting a hearing and have received new 

medical information to determine if a reevaluation of the cases by medical consultants residing at a 

DDS will help to improve the consistency of decisions between levels, with a 1-year goal of 

remanding 100,000 cases. 

Review selected ALJ 

allowance decisions 

Conduct a joint Appeals Council and OQA review of ALJ allowances that have not yet received a final 

decision (i.e., the claimant has not been awarded benefits) to identify policy areas leading to 

inconsistent decisions between levels. 

Source: SSA documents and prior GAO reports. 

 10Findings from SSA’s 1994 Disability Hearings Quality Review Process report provided the 

agency with additional information on potential causes of inconsistency. The report 

identified two assessment areas associated with inconsistent decisions. In addition, quality 

reviewers found that when applying standards used by the initial level to adjudicate claims, 

29 percent of the appealed DDS reconsideration decisions and 51 percent of ALJ decisions 

were not supported by the decision makers. These findings help to support SSA’s decision 

to include efforts to have DDSs more fully develop and document their decisions and to 

assess ALJ decisions as part of its process unification initiative. 
Page 10 GAO-04-656 SSA Disability Decisions 

In 1997, we reported on the possible reasons for the inconsistency of 

decisions between the initial and hearings levels. Our report found that 

differences in state DDSs’ and ALJs’ views on the claimants’ functional 

abilities was a key factor in explaining why ALJs allowed benefits on 

appealed cases.11 We also reported that poorly documented state DDS 

evaluations of the claims were of limited use to ALJs and SSA quality 

reviews did not focus on identifying inconsistency in decisions. To support 

SSA’s process unification efforts, the report recommended that SSA, using 

available systems and data collected so far, move quickly ahead to 

implement its quality assurance initiative to provide consistent feedback 

to DDS and ALJ adjudicators as soon as possible. In addition, we 

recommended that SSA expand its effort to return cases to a DDS for 

review when new evidence is introduced on appeal. Last, we 

recommended that SSA set goals for measuring the effectiveness of 

process unification in reducing inconsistent decisions. 

More recently, the Social Security Advisory Board issued a 2001 report 

that identified many factors that could potentially affect the overall 

consistency of disability decision making between adjudication levels.12

Some of the factors the board suggested as potentially affecting 

consistency included: 

• the fact that most claims are decided based on a paper review of case 

evidence without face-to-face contact with an adjudicator until a 

claimant has an ALJ hearing, 

• involvement of attorneys and other claimant representatives at the ALJ 

hearing, 

• the fact that claimants are allowed to introduce new evidence and 

allegations at each stage of the appeals process, 

• differences in quality assurance procedures applied to initial- and 

hearings-level decisions, 

• differences in the training given to ALJs and state examiners, and 

• lack of clear and unified policy guidance from SSA. 

 11U.S. General Accounting Office, Social Security Disability: SSA Must Hold Itself 

Accountable for Continued Improvement in Decision-Making, GAO/HEHS-97-102

(Washington, D.C.: Aug. 12, 1997). 

12The board also identified factors that could affect consistency within adjudication levels, 

as opposed to strictly between adjudication levels. For more information see the Social 

Security Advisory Board, Disability Decision Making: Data and Materials, January 2001, 

pp. 5-6. 
Page 11 GAO-04-656 SSA Disability Decisions 

Despite SSA’s process unification efforts and related studies to improve 

the consistency of decisions, recent ALJ allowance rates—which declined 

after process unification began, but started increasing in 1999 to reach 

61 percent in fiscal year 2003—still raise questions as to whether initialand hearings-level decision makers are consistently applying the agency’s 

guidance. In addition to inconsistent application of SSA’s policy guidance, 

there are several other reasons why a large number of ALJ allowances are 

made. For example, some ALJ allowances should be expected because, by 

law, cases can remain open throughout the hearings process, allowing new 

evidence to be submitted that may not have been available to the state 

adjudicators. Such new evidence could show that the claimant’s condition 

has worsened and prohibits work. Also, SSA’s decision-making criteria 

require that a great deal of professional judgment be applied. As a result, 

some allowances at the hearings level could simply reflect the differing 

judgments of two adjudicators reviewing a case. While a claimant’s 

deteriorating health, changes in the characteristics of a claim over time, 

and the complexity of disability decisions may help to explain some of the 

ALJ allowances, studies have not sufficiently explained why consistently 

over half the cases appealed to the hearings level are allowed. Instead, 

studies indicate that systemic differences in the assessment of claims at 

both adjudication levels are contributing to the ALJ allowance rate. For 

example, our 1997 report noted a difference in state DDSs’ and ALJs’ views 

on the claimant’s functional abilities was a key factor in explaining why 

ALJs allowed cases on appeal. 

Inconsistency in decisions may create several problems. High hearings 

allowance rates may create the perception that the hearings level is 

applying SSA’s criteria less strictly than the initial level and create an 

incentive for claimants to appeal to an ALJ for a more favorable decision.13

If deserving claimants must appeal to the hearings level for benefits, this 

situation increases the burden on claimants, who must wait, on average, 

almost a year for a hearing decision and frequently incur extra costs to pay 

 13An appeal adds significantly to costs associated with making a decision. According to 

SSA’s Performance and Accountability Report for fiscal year 2001, the average cost per 

claim for an initial DDS disability decision was about $583, while the average cost per claim 

of an ALJ decision was an estimated additional $2,157. 
Page 12 GAO-04-656 SSA Disability Decisions 

for legal representation.14 In addition, to the extent that the ALJ allowance 

rates include inappropriate allowances, SSA could be incurring 

unwarranted program costs. Although SSA has tried to address these 

problems, its inability to resolve them has contributed to our decision to 

include federal disability programs on our list of high-risk government 

programs.15

Renewing its effort to address long-standing and critical problems with the 

disability programs, SSA’s Commissioner recently announced a new 

proposal to improve these programs. (See app. I for an excerpt of the 

announcement that describes the newly proposed decision-making 

process.) In addition to proposing demonstration projects that provide 

work incentives and supports to help people with disabilities return to 

work, SSA has proposed significant changes to both the process of 

adjudicating disability claims and the structure and management of the 

agency’s quality management system to improve the timeliness, accuracy, 

and consistency of the disability decision-making process. The agency 

believes that several of these changes will help to improve consistency 

between DDS and ALJ decisions. For example, SSA plans to provide more 

centralized end-of-line quality reviews. According to SSA, the proposed 

quality reviews should help to hold adjudicators more accountable for 

their decisions and ensure that they consistently apply SSA’s policies as 

well as help the agency detect and amend those policy areas leading to 

inconsistent decisions. Table 2 provides a description of SSA’s proposed 

changes to improve the disability decision-making process. 

 14An appeal also significantly increases the time required to reach a decision. According to 

SSA’s Performance and Accountability Report for fiscal year 2003, the average number of 

days that claimants waited for an initial decision was 97 days, while the number of days 

they waited for an appealed decision was 344 days. However, the time a claimant waits for 

a decision should not impact the amount of benefits received, as benefits are based on the 

date the claimant becomes disabled. 

15U.S. General Accounting Office, Performance and Accountability Series: Major 

Management Challenges and Program Risks: Social Security Administration, 

GAO-03-117 (Washington, D.C.: January 2003). 
Page 13 GAO-04-656 SSA Disability Decisions 

Table 2: Newly Proposed Changes to the Disability Decision-Making Process 

Proposed change Purpose 

Centralize medical expertise from the states 

into regional offices and organize experts by 

medical specialty 

To make quick decisions on initial claims for individuals who are obviously disabled 

(e.g., those with aggressive cancers and end-stage renal disease) and to provide 

equal access to medical expertise for all adjudicators and more consistent medical 

review of claims. 

Require DDSs to fully document and explain 

the basis for their decisions 

To hold DDS adjudicators accountable for providing higher-quality and more 

consistent decisions and to have better information to identify and correct problem 

areas leading to incorrect decisions. 

Eliminate DDS reconsiderations To reduce time taken to process claims and avoid having claimants who are 

disabled drop out of the disability claims process. 

Create a reviewing official position To evaluate all appealed DDS decisions and prepare either (1) an on-the-record 

allowance, (2) a recommended disallowance detailing reasons for a denial, or (3) a 

prehearing report outlining the evidence needed to fully support the claim. 

Require ALJs to address the reviewing officials’ 

reports 

To improve accountability and consistency by having ALJs either (1) describe in 

detail the basis for rejecting the reviewing official’s recommended disallowance or (2) 

provide detailed information on the evidence used to support allowances made in 

response to the reviewing official’s prehearing report. 

Eliminate the claimant’s ability to appeal ALJ 

decisions to the Appeals Council 

To reduce processing time for claims and use resources more effectively. 

Use of in-line quality control To build quality into each level of the decision-making process rather than rely too 

heavily on case reviews performed by end-of-line quality reviewers or by the next 

adjudication level in response to claimants appealing denied decisions. 

Use of centralized quality control unit To perform end-of-line reviews for all decisions, thereby replacing regional reviews 

of DDS decisions, and to provide a more balanced review of both DDS and ALJ 

decisions to ensure that all adjudicators are consistently applying SSA’s policies and 

to detect and amend those policy areas leading to inconsistent decisions. 

Create oversight panels that include two ALJs 

and one Administrative Appeals Judge from the 

Appeals Council 

To review and either affirm or reverse an ALJ decision referred by the centralized 

quality control unit when the unit disagrees with the ALJ decision. 

Sources: SSA documents and agency interviews. 

Note: Under the new proposal, when the agency’s centralized quality control unit and oversight panel 

review an ALJ decision, a claimant cannot submit any new information to be considered by the 

agency. 

SSA does not plan to implement its proposed changes before it has 

successfully implemented its Accelerated Electronic Disability (AeDib) 

system. This major initiative should allow adjudication staff in states and 

throughout the agency, regardless of geographic location, to access case 

information electronically through the use of an electronic disability 

folder. The initiative is intended to reduce delays that result from mailing, 

locating, and organizing paper folders. SSA also expects this new system 

to provide critical management information for analyzing and reducing 

inconsistencies in disability decisions. SSA is implementing the new 

system and plans to give adjudicators time to adjust to this change before 
Page 14 GAO-04-656 SSA Disability Decisions 

implementing its new proposal. SSA’s implementation of the new 

proposal will therefore be no earlier than October 2005. In the meantime, 

SSA continues to discuss the proposal with stakeholders and plans to 

further refine it before implementation. 

SSA has partially implemented its process unification initiative. Although 

the agency initially made improvements in its policies and training 

intended to improve the consistency of decisions between adjudication 

levels, it has not continued to actively pursue these efforts. As part of the 

initiative, the agency also implemented a review of ALJs’ allowance 

decisions to identify additional ways to improve training and policies, but 

no new changes were made as a result of findings from the review. Finally, 

the agency also began two tests of process changes to help improve the 

consistency of decisions, but one ongoing test with design problems is not 

likely to lead to any conclusive results and the other test has been 

abandoned. 

While SSA initially made progress carrying out efforts to improve policies 

and training to better ensure the consistency of decisions, the agency has 

not continued to actively pursue these efforts. SSA quickly accomplished 

most of its planned efforts to clarify policy guidance. In 1996, SSA issued 

nine process unification rulings to clarify policy areas it found to be 

contributing to inconsistent decisions. For example, one ruling provided 

all adjudicators with guidance on how to weigh and document their 

evaluation of the treating physician’s opinions when making a disability 

decision.16 SSA successfully went through the regulatory process several 

years later and published three new regulations to strengthen its process 

unification rulings, but was unable to agree on a fourth regulation 

regarding the weight to be given to the treating physician’s opinion when 

evaluating a claim.17

SSA planned to develop a single presentation of policy guidance to replace 

the different sources used by each level, but has since abandoned full 

implementation of these plans in favor of a more limited approach. DDS 

 16Social Security Ruling 96-2: Policy Interpretation Ruling Titles II and XVI: Giving 

Controlling Weight to Treating Source Medical Opinions, effective July 2, 1996. 

17Since SSA rulings are binding only on SSA adjudicators and do not have to be followed by 

the courts, SSA planned to strengthen the impact of several rulings by creating regulations 

that would be followed by the courts. 

SSA Has Partially 

Implemented Its 

Process Unification 

Initiative 

SSA Made Early Progress 

Improving Policies and 

Training, but Has Not 

Actively Pursued These 

Efforts 
Page 15 GAO-04-656 SSA Disability Decisions 

adjudicators currently follow a detailed set of policy and procedural 

guidelines,18 whereas ALJs rely directly on statutes, regulations, and 

rulings for guidance in making disability decisions. To help ensure that 

inconsistent guidance was not contributing to inconsistent DDS and ALJ 

decisions, SSA began issuing guidance in the same wording to all 

adjudicators in 1996. Although SSA had also planned to address 

differences in policy guidance issued before 1996 and to eventually 

combine existing adjudication policy documents into a single document, it 

ultimately decided not to take these additional steps. According to SSA, 

further efforts to unify the policy guidance used by both levels would be a 

massive undertaking and not worth the cost because the guidance issued 

since 1996 had already addressed important policy areas that were leading 

to inconsistent decisions. While some stakeholder groups representing 

adjudicators tended to agree with SSA’s position, the Social Security 

Advisory Board and other groups still believe the agency should take 

additional steps to provide a unified policy guide to all adjudicators. 

Instead of creating one policy manual for all adjudicators, SSA told us that 

it plans to undertake a comprehensive effort to evaluate and improve its 

disability policies to make them less susceptible to differing 

interpretations and to ensure they are up to date. A more comprehensive 

approach could address key weaknesses in SSA’s disability program that 

we previously highlighted in our performance and accountability series, 

and thereby help to modernize federal disability programs to better meet 

the needs of Americans with disabilities. 

Early on, SSA also provided extensive cross-training of DDS and ALJ 

adjudicators, although the scope of its efforts has since diminished. To 

help all adjudicators understand how to appropriately apply process 

unification rulings, SSA provided extensive and mandatory training in 

1996 and 1997 to 15,000 disability adjudicators (including DDS examiners, 

physicians, ALJs, and quality assurance staff). The training was provided 

to adjudicators at all levels of the process in three of the most complex 

disability areas—assessment of symptoms, treatment of expert opinions, 

and assessment of claimants’ remaining capacity to work (i.e., residual 

functional capacity). While this training was intended to be ongoing, SSA’s 

training efforts have diminished significantly since 1997. Stakeholder 

groups representing DDS adjudicators told us that SSA’s training does not 

 18These guidelines—called the Program Operations Manual System (POMS)—contain, 

within an estimated 30,000 pages, interpretations of relevant statutes, regulations, and 

rulings and procedural information. 
Page 16 GAO-04-656 SSA Disability Decisions 

sufficiently cover process unification issues. In addition, our review of 

DDS and OHA participation in video training revealed inconsistent 

participation in training by adjudicators. To provide ongoing training to 

both adjudication levels and other components involved in the claims 

process, SSA has used interactive video technology. Almost all the state 

DDS sites and about 85 percent of OHA offices have this technology. 

However, in reviewing participation for two recent courses, we found for 

those sites with this interactive technology only 31 percent of DDS sites 

and 16 percent of OHA sites logged on for a course on the role of 

consultative examinations, and 18 percent of DDS sites and 4 percent of 

OHA sites logged on for a monthly disability hour training class.19

According to SSA, neither DDS nor OHA adjudicators are generally 

required to attend courses. In line with these findings, our recent report on 

the human capital challenges facing DDSs found gaps in the key 

knowledge and skills of their adjudicators in the same areas SSA had 

earlier identified as critical to making consistent decisions, and we 

recommended that SSA work with DDSs to close these gaps.20

Despite SSA’s early efforts to improve policy guidance and provide 

training, stakeholder groups representing state adjudicators told us that 

many states are not performing the additional development and 

documentation of decisions required by the process unification rulings. 

They also told us that the rulings have added significantly to the time, 

complexity, and subjectivity of the decision-making process, while 

insufficient resources have limited their ability to fully implement the 

rulings’ requirements. In addition, claimant lawsuits against three state 

DDSs have alleged that DDS adjudicators were not following SSA’s rulings 

or other decision-making guidance. In settling these lawsuits, SSA agreed 

to have these states fully develop and document cases. However, 

according to DDS stakeholder groups, SSA has not ensured that states 

have sufficient resources to meet ruling requirements, which they believe 

may lead to inconsistency in decisions among states. Furthermore, SSA’s 

quality assurance process does not help ensure compliance because 

reviewers of DDS decisions are not required to identify and return to the 

 19Additional DDS and OHA sites may have taped and viewed this training, but SSA’s 

monitoring of training provides only the number of sites logged on to the interactive video 

training. 

20U.S. General Accounting Office, Social Security Administration: Strategic Workforce 

Planning Needed to Address Human Capital Challenges Facing the Disability 

Determination Services, GAO-04-121 (Washington, D.C.: Jan. 27, 2004). 
Page 17 GAO-04-656 SSA Disability Decisions 

DDSs cases that are not fully documented in accordance with the rulings. 

SSA’s procedures require only that the reviewers return cases that have a 

deficiency that could result in an incorrect decision. 

As part of its initiative, the agency has also implemented a quality review 

of ALJ decisions, but the review has not proved useful for identifying any 

new changes to SSA’s policies or training that would help to address the 

inconsistency of decisions. This review—referred to as the ALJ Preeffectuation Review—involves a sequential review by SSA’s OQA and the 

Appeals Council of certain ALJ allowances that have not yet been finalized 

(i.e., the claimant has not yet been awarded benefits). In selecting 

allowances for review, OQA uses an error-prone profile developed from its 

analysis of errors detected when reviewing DDS allowances. SSA began 

testing the new review of ALJs’ decisions in 1996 and implemented it as an 

annual review in 1998. From fiscal years 1998 through 2002, OQA reviewed 

27,148 ALJ allowances and of these, OQA found fault with about 35 

percent and referred them to the Appeals Council.21 The Appeals Council 

screens the allowances for its own review and selects those in which the 

prior actions may not have been proper, fair, or in accordance with the law 

or the ALJ’s decision was not supported by substantial evidence.22 If the 

council finds fault with the ALJ’s decision, it will deny the claimant 

benefits or return the claim to the ALJ to have the identified problems 

corrected. If the council does not find fault with the ALJ’s decision, the 

claimant will be awarded benefits. 

In addition to identifying inappropriate ALJ allowances, SSA intended to 

use the new quality review to identify areas of inconsistency between 

adjudication levels and ways to improve policies and training to address 

those inconsistencies. Specifically, OQA identified cases where it found 

fault with the ALJ decision, but the Appeals Council, after screening them, 

did not accept them for review. OQA then forwarded these cases to a 

panel of staff from the various components involved in SSA’s claims 

process to determine whether the inconsistent assessment of these cases 

 21When reviewing an ALJ allowance, OQA uses the preponderance of evidence standard, 

which requires that the reviewer fairly consider all evidence and decide whether the weight 

of that evidence supports the allowance. 

22The Appeals Council uses a substantial evidence standard of review that requires the 

reviewer to determine that the evidence in the case is sufficient to convince a reasonable 

mind of the credibility of the allowance decision, and that there is no opposing evidence 

that clearly compels another finding or conclusion. 

SSA’s Review of ALJ 

Allowances Has Not 

Resulted in Improvements 

to Policy and Training 
Page 18 GAO-04-656 SSA Disability Decisions 

by OQA and the Appeals Council indicated the need to clarify policies, 

issue new policies, or provide training to improve the consistency of 

decisions. However, according to a SSA official, this review did not 

identify any new areas of inconsistency that required improvements to 

policy and training. Weaknesses in the design of the review may have 

contributed to SSA’s inability to identify new policy areas contributing to 

inconsistency. For example, rather than reviewing a random sample of all 

ALJ decisions, this review focused on allowances. Further, the review 

looked only at ALJ allowances that were selected using a DDS error-prone 

profile, i.e., a profile that is based upon cases in which quality reviewers 

did not agree with the DDS adjudicators’ decisions. As a result, SSA 

selected and reviewed nonrandom allowance decisions with case 

characteristics that the agency may have already suspected were 

associated with inconsistent decisions. In 1999, the panel was disbanded 

because members had other priorities needing attention. OQA told us that 

it continued to perform a limited review of cases viewed differently by 

OQA and the Appeals Council. More recently, OQA began an effort to 

summarize the results of its review and expected to issue a report of its 

findings in April 2004. As of April 2004, this report had not been issued. 

SSA began two tests of potential changes to the process to help improve 

the consistency of decisions, but neither test was successfully completed. 

The changes tested were (1) more fully developing and documenting 

decisions made at the initial level and (2) sending appealed cases that 

involve new medical information back to the initial level to be reevaluated. 

SSA wanted to test having DDSs more fully develop and document 

decisions because it believed that DDS decisions, especially denials, are 

often not well documented. SSA wanted to test whether better 

explanations of why benefits were denied would improve the accuracy of 

DDS decisions and consistency of decisions between adjudication levels. 

SSA first implemented a pilot of this change to explore alternatives for 

developing and documenting decisions. Then SSA tested this change, 

along with other process changes, in a larger test, called the prototype 

initiative. Concurrently, SSA tested other process changes, such as the 

elimination of a reconsideration step and a predecision DDS interview 

with the claimant. 

The prototype test had limitations for predicting the impact of 

documented decisions. For example, SSA’s decision to test several 

changes together left the agency without clear information on what impact 

fully developed decisions would have on the decision-making process 

SSA Began Two Tests of 

Process Changes to 

Improve the Consistency 

of Decisions, but Neither 

Test Was Successfully 

Completed 
Page 19 GAO-04-656 SSA Disability Decisions 

without the other process changes. SSA’s test design also did not build in 

an ALJ feedback mechanism to provide sufficient information on the 

usefulness of more fully documented decisions. SSA continues to test this 

change along with other changes and, despite limited information on the 

best approach for and impact of this change, currently plans to implement 

more fully documented decisions as part of the Commissioner’s new 

proposal to improve SSA’s disability programs. 

SSA also began, but ultimately abandoned, a test in which appealed cases 

with new medical information submitted prior to the hearing were to be 

sent back to the initial level so that the evidence could be evaluated by 

medical consultants residing at the DDSs. Since medical expertise resides 

in the DDS and not at the hearings level,23 SSA decided to test whether 

“remanding,” or sending cases to the DDS for evaluation, might result in a 

more consistent review of medical evidence. SSA believed that this 

change, in turn, could help improve the consistency of decisions because 

the new medical information might be contributing to ALJ allowances. 

However, the change also had the potential to increase the time claimants 

with remanded claims would have to wait for final decisions because 

claims that were not allowed by the DDSs had to be returned to OHA for 

hearings. SSA began remanding cases in July 1997, with a 1-year goal of 

remanding 100,000 cases, but after 10 months, it had remanded fewer than 

9,000. In implementing this test, SSA encountered several difficulties. For 

example, it had difficulty identifying the claims to be remanded and 

ensuring the ALJs, who had authority over the claims, would remand the 

claims to the DDSs. The ALJs’ resistance to remanding claims to the DDSs 

may be due in part to concerns that remanding would not lead to many 

allowances by the DDSs and would result in many claims being returned 

to OHA, thereby increasing the time many claimants would have to wait 

for a final decision from OHA. Realizing that the agency would not be able 

to reach its remanding goal, the agency decided to discontinue this test. 

 23At the hearings level, ALJs can purchase medical evidence, at their discretion. 
Page 20 GAO-04-656 SSA Disability Decisions 

SSA’s assessments have not provided the agency with a clear 

understanding of the extent and causes of possible inconsistencies in 

decisions between adjudication levels. The two measures SSA uses to 

monitor changes in the extent of inconsistency of decisions have 

weaknesses and therefore do not provide a true picture of the changes in 

consistency. In addition, SSA has not sufficiently assessed the causes of 

possible inconsistency. The agency conducted an analysis in 1994 that 

identified some potential areas of inconsistency. However, although SSA 

continues to collect information that would support this analysis, it has 

not repeated this initial effort, nor has it expanded on it by employing 

more sophisticated assessment techniques. 

SSA has made some efforts to monitor changes in the extent of 

inconsistency between the initial and hearings levels, including tracking 

trends in allowance rates at different levels and conducting special 

reviews of ALJ decisions. Together, according to SSA, these measures and 

assessments suggest that the consistency between levels has improved 

since the agency began implementing its process unification initiative. 

However, because of methodological weaknesses, these measures provide, 

at best, a partial picture of trends in the consistency of decisions between 

adjudication levels. 

SSA tracks trends in the proportion of all allowances decided at each level 

to assess the consistency of decisions between levels. The agency collects 

information on the number of allowances granted to claimants at each 

level of the process, tracks the proportion of claims allowed at the initial 

level relative to the hearings level, and looks at trends in these proportions 

over a period of several years. According to data from SSA, the proportion 

of overall allowances that occurred at the initial level has increased since 

process unification was implemented. As shown in figure 2, in fiscal year 

1996, 72 percent of all allowances were granted at the initial level. This 

proportion increased in most subsequent years, and by fiscal year 2003, 

77 percent of all allowances were granted at the initial level. Officials from 

OQA, the office responsible for reviewing, evaluating, and assessing the 

integrity and quality of the administration of SSA’s programs, view the 

relative shift toward earlier allowances as an indicator that consistency 

between adjudication levels has improved, and they believe that process 

unification efforts have contributed to these results. 

SSA Lacks a Clear 

Understanding of the 

Extent and Causes of 

Inconsistency 

between Levels 

SSA Attempts to Monitor 

Changes in the Extent of 

Inconsistency, but the 

Measures It Uses Provide 

an Incomplete Picture 
Page 21 GAO-04-656 SSA Disability Decisions 

Figure 2: SSA Allowances at Initial Level versus Hearings Level, by Proportion of 

Allowances 

Note: Hearings level allowances include allowances made by ALJs and judges from the Appeals 

Council. 

However, SSA’s measure of tracking yearly changes in the proportion of 

allowances at each level is a simplistic and inconclusive indicator of 

trends in the consistency of decisions because it does not control for the 

multitude of factors that can affect allowance rates at either adjudication 

level in any given year and over time. For example, SSA uses “snapshot” 

data in looking at the proportion of allowances granted at each level, 

meaning that it looks at the number of claimants and allowances at each 

level during a given year, rather than following a 1-year cohort of initial 

claimants through the entire process and capturing the proportion of 

allowances for that cohort decided at each level. Because SSA uses data 

that illustrate allowance rates at a given moment in time, it captures a 

different pool of claimants in the process at each level, and the resulting 

allowance rates are subject to a different set of demographic and case 

characteristics. Over time, the pool of claimants may change because of 

factors such as a downturn in the economy, which can cause more people 

with less severe impairments to claim benefits or appeal initial denial 

0

20

40

60

80

100

28 27 27 25 23 20 22 23

Percentage of all allowances

Source: SSA.

Fiscal year

Hearings level allowances

Initial level allowances

1996 1997 1998 1999 2000 2001 2002 2003

72 73 73 75 77 80 78 77
Page 22 GAO-04-656 SSA Disability Decisions 

decisions. In addition, snapshot data may be significantly affected by 

fluctuations in productivity at either adjudication level caused by process 

changes that are unrelated to process unification and that affect only one 

level. 

SSA has collected other data to further assess trends in the consistency of 

decisions. Since 1993, the agency has conducted a biennial case review as 

part of its Disability Hearing Quality Review process.24 This review 

consists of medical consultants and disability examiners in SSA’s central 

office evaluating a sample of ALJs’ decisions plus supporting 

documentation to determine whether the ALJ has adequately supported 

his or her decision. In evaluating the ALJ decisions, these medical 

consultants and disability examiners use the same standards as those used 

by initial-level adjudicators to adjudicate claims, which are from the 

official SSA program policy and operations guidance found in POMS. To 

some degree, therefore, the medical examiners and disability reviewers 

serve as a proxy for initial-level adjudicators, and their decisions are 

representative of how initial-level examiners should be deciding claims. 

While unpublished results from the biennial case reviews indicate an 

increase in supportable ALJ allowances, such findings focus on the ALJ 

level and therefore provide only a partial picture of trends in consistency. 

The reviews indicated that medical consultants and disability examiners 

have found that supportable ALJ allowances increased from 36 percent in 

fiscal year 1993-94 to 57 percent in fiscal year 1999-2000. OQA officials told 

us that this increase suggests an improvement in consistency between 

adjudication levels because it indicates that disability examiners using 

initial-level standards and ALJs increasingly agree on how like cases 

should be decided.25 However, SSA’s assessment provides only a partial 

picture because it does not reflect trend information on the extent to 

which ALJs have found DDS decisions to be supportable, to ensure that 

 24See the Social Security Administration, Office of Program and Integrity Reviews, 

Findings of the Disability Hearings Quality Review Process: An Assessment of the 

Quality of Hearing Decisions and Appealed Reconsideration Denial Determinations, 

September 1994. 

25An OQA official also said that results from another review bolster the conclusion that the 

quality of ALJ decisions has improved. Specifically, in the ALJ peer review, in which ALJs 

evaluate a random sample of other ALJs’ decisions, the reviewing ALJs have found an 

increasing percentage of ALJ decisions to be supportable. The percent of decisions found 

to be supportable increased from 81 percent to 90 percent from the reviews of decisions 

issued from fiscal years 1993 through 1994 compared with decisions from fiscal years 2001 

through 2002. 
Page 23 GAO-04-656 SSA Disability Decisions 

both levels are making more consistent decisions. Although the 1994 

report of findings from the initial biennial case review included the results 

of a special probe in which ALJs reviewed 165 DDS reconsideration denial 

decisions, the sample was not representative, and therefore results could 

not serve as a baseline for developing trend information. In 2003, SSA 

began another probe, in which ALJs reviewed 400 DDS reconsideration 

denial determinations, but the agency does not plan to release its findings 

until summer 2004. 

Although SSA has limited information on how ALJs view DDS decisions, 

other information collected by the agency suggests that consistency of 

decision making at the initial level might not be improving. For example, 

OQA reviewers routinely assess the accuracy and supportability of DDS 

decisions. A recent SSA study of these data shows that the accuracy of 

DDS denial decisions—those decisions most likely to be appealed to the 

hearings level—has declined by 4 percentage points over a 1- year period.26

Another review of DDS decisions by OQA reviewers also suggests a lack of 

improvement at the initial level. Specifically, the extent to which quality 

reviewers found that DDS reconsideration denials appealed to the 

hearings level were supported declined from 71 percent in fiscal year 

1993-94 to 68 percent in fiscal year 1999-2000. 

Despite some efforts to assess inconsistency in decisions, shortcomings in 

SSA’s analyses also limit its ability to identify areas and causes of possible 

inconsistency. Most notably, over the last 10 years, SSA has not updated 

its prior analyses of information from its initial biennial case review that 

helped identify problem areas. In addition, SSA has not improved on its 

case review and analysis by ensuring that reviewers assess all relevant 

case evidence used to make decisions, or performed more sophisticated 

analysis to identify the areas and causes of inconsistency in decisions. 

Other efforts—including the review of ALJ allowances and a probe of DDS 

reconsideration denials—have yet to yield useful information. 

In 1994, for its initial biennial case review report, the agency took its first 

step in identifying areas of possible inconsistency by identifying two 

characteristics about the claimants and their cases over which initial-level 

 26See the Social Security Administration, Office of Quality Assurance and Performance 

Assessment, Quality Assurance Group I Initial Denial Accuracy Report: August-October 

2002, July 2003. 

SSA’s Assessments Have 

Not Identified the Areas 

and Causes of Possible 

Inconsistency in Decisions 

between Adjudication 

Levels 
Page 24 GAO-04-656 SSA Disability Decisions 

reviewers tended to disagree with ALJs. Specifically, the 1994 report 

concluded that teams of reviewing medical consultants and disability 

examiners sometimes viewed cases involving mental impairments 

differently than the reviewing ALJs. In addition, these two sets of 

reviewers tended to have different views on the severity of claimants’ 

impairments and their resulting capacity to work. According to the official 

responsible for overseeing the review, the findings in this initial report 

provided important support for SSA’s process unification efforts as well as 

the agency’s efforts to redesign the disability claims process. 

SSA continues to conduct the biennial case reviews; however, the agency 

has not continued to analyze and identify areas that are viewed differently 

by different adjudication levels. Specifically, SSA no longer identifies the 

particular case characteristics over which reviewers from the two levels 

tend to disagree. As a result, SSA does not know whether previously 

identified problem areas are still present. Moreover, SSA no longer 

publishes any information from the medical consultant and disability 

examiner biennial case reviews, even though it has performed some 

limited analysis of the supportability of decisions made by adjudicators. 

By not continuing to publish its analysis and findings, the agency makes it 

difficult to ensure the reliability of its methods and results, and leaves 

stakeholders outside the agency, including disability groups, without a 

means for understanding SSA’s assessment efforts and progress in 

improving the consistency of decisions. The SSA office conducting the 

study has told us that, because of downsizing and competing priorities, it 

has no current plans to further analyze and publish these data. 

Further, in its ongoing biennial case reviews, SSA does not make full use 

of available case information that would be useful in identifying areas and 

causes of inconsistency. Specifically, medical consultants and disability 

reviewers do not listen to tapes of the hearings and therefore do not 

review the entire case as presented to the original ALJ. Although reviewing 

medical consultants and disability examiners read the ALJs’ explanations 

for their original decisions, which should include the most important 

factors behind the ALJs’ decisions, the reviewers do not evaluate the oral 

evidence independently. An SSA official with whom we spoke indicated 

that some evidence entered by witnesses at the hearing might not be 

accompanied by other hard copy sources of the same information. 

Therefore, reviewers would not consider information potentially relevant 

to the ALJ’s decision that could be used to identify areas and causes of 

inconsistency. 
Page 25 GAO-04-656 SSA Disability Decisions 

SSA also does not make full use of the information it collects because it 

has not employed analytical tools that would improve its ability to identify 

areas and causes of inconsistency. For example, SSA’s biennial case 

reviews provide a rich dataset that lends itself to regression analysis to 

identify areas and possible causes of inconsistency between levels. 

Regression analysis would allow the agency to better pinpoint any 

significant case characteristics affecting decisions and to more clearly 

identify the underlying causes of inconsistency. Specifically, among the 

data collected in this review are such variables as the types of impairments 

the claimant has, the types of relevant medical evidence, and additional 

impairments presented at the hearing. Multivariate analysis, such as a 

multiple regression model, could allow SSA to assess how these and many 

other factors, relative to one another, contribute to whether a case results 

in a similar outcome at both levels. However, SSA has not employed this 

more sophisticated multivariate technique, citing resource constraints and 

competing priorities. We recognize the methodological complexities of 

analyzing disability decisions, and we previously recommended that SSA 

establish an advisory panel of external experts from a range of disciplines 

to provide leadership, oversight, and technical assistance to the agency.27

Otherwise, in forgoing such analysis, the agency will continue to miss an 

opportunity to better pinpoint areas and some possible causes of 

inconsistency in decisions between the two adjudication levels, and to lay 

the foundation for further investigation. 

Another tool SSA has not sufficiently employed for identifying areas and 

causes of inconsistency is in-depth case studies involving both levels of 

adjudication. Case studies, in which different adjudicators review the 

same test case, can be a means for unearthing causes for inconsistency by 

getting adjudicators from both levels to acknowledge and address 

discrepancies in the ways they view cases. SSA has performed case studies 

in the past to ascertain differences in policy interpretation between DDS 

examiners and quality reviewers. However, SSA does not routinely have 

both DDS examiners and ALJs perform in-depth review of the same 

sample of cases, despite this method’s potential for helping identify causes 

 27In addition, the report recommended that SSA include cases appealed to its Appeals 

Council in the sample for its biennial case review to eliminate the systematic bias in that 

sample and make it representative of all cases that receive a decision from the hearings 

level. For more information on this recommendation and others, see U.S. General 

Accounting Office, SSA Disability Decision Making: Additional Steps Needed to Ensure 

Accuracy and Fairness of Decisions at the Hearings Level, GAO-04-14 (Washington, D.C.: 

Nov. 12, 2003). 
Page 26 GAO-04-656 SSA Disability Decisions 

of inconsistency between the two adjudication levels. OQA officials told us 

that case studies are a very resource-intensive tool because they need a 

sufficient number of cases from which to generalize. Therefore, the agency 

is reluctant to use this approach to help it understand the causes of 

inconsistency between adjudication levels. However, using multivariate 

analyses of the biennial case review data could help the agency to more 

effectively target its in-depth case studies on those areas found to be 

leading to inconsistent decisions and thereby increase its success at 

identifying the causes of inconsistency. 

SSA conducts other analyses of inconsistency between levels, but to date 

these efforts have yielded limited information concerning areas and 

possible causes of inconsistency. For example, as part of SSA’s ALJ PreEffectuation Review, two different levels of reviewers have evaluated 

thousands of cases. However, limitations in the review methodology, such 

as not using a random sample of ALJ decisions, do not allow the agency to 

use this review to identify the leading causes of inconsistency. SSA 

recently began an evaluation of this effort and plans to publish its findings 

and recommendations in April 2004. Another analysis currently under way, 

a special 400-case review, might help identify areas of inconsistency at the 

initial level, but it has yet to be completed. Begun in 2003, this review by 

ALJs of DDS reconsideration denial determinations is expressly aimed at 

assessing inconsistency between adjudication levels. SSA expects to gain 

some understanding of why about 60 percent of cases denied by the initial 

level and appealed to the hearings level are allowed. The agency plans to 

publish its findings in summer 2004. 

Some changes included in SSA’s new proposal to overhaul its disability 

claims process may improve the consistency of DDS and ALJ decisions, 

but challenges may hinder the implementation of the proposal. The new 

proposal includes several changes to the disability claims process that the 

agency and stakeholder groups representing adjudicators and claimant 

representatives believe offer promise for improving the consistency of 

DDS and ALJ decisions. However, past difficulties in improving the 

process, as well as stakeholder concerns about limited resources and 

other obstacles, indicate that some difficulties may arise in the 

development and implementation of SSA’s new proposal. 

SSA’s New Proposal 

Incorporates Efforts 

to Improve the 

Consistency of 

Decisions, but 

Challenges May 

Impede Successful 

Implementation 
Page 27 GAO-04-656 SSA Disability Decisions 

SSA told us that several aspects of the new proposal may improve the 

consistency of decisions, and although opinions varied among stakeholder 

groups, most thought the following four proposed changes have the 

potential to improve the consistency of decisions between adjudication 

levels: (1) requiring state adjudicators to more fully develop and document 

their decisions, (2) centralizing the agency’s approach to quality control, 

(3) providing both adjudication levels with equal access to more 

centralized medical expertise, and (4) requiring ALJs to address agency 

reports that either recommend denying the claim or outline the evidence 

needed to fully support the claim. 

Representatives from most stakeholder groups with whom we spoke told 

us that having state adjudicators more fully develop and document their 

decisions may help to improve the consistency of DDS and ALJ decisions. 

Specifically, stakeholders said that more developed decisions may provide 

ALJs with a better understanding of the DDS decision and enable them to 

more fully consider this information when evaluating a case. According to 

the agency and stakeholders, this change may contribute to a more 

consistent interpretation and application of SSA’s decision-making 

criteria. They also mentioned that well-developed decisions by DDS 

examiners could assist SSA in holding adjudicators accountable for case 

development and decisions, such as enabling quality reviewers to more 

effectively assess the appropriateness of the DDSs decisions. Unlike SSA’s 

earlier attempt at more fully developing decisions as part of process 

unification, SSA plans to incorporate a reviewing official into the process 

whose assessment of all appealed DDS decisions can provide feedback on 

the extent to which cases are being fully developed. 

In addition, the agency and many stakeholders told us that they believe 

centralizing the agency’s quality control system may help resolve some 

problems contributing to inconsistent decisions between the two levels. 

For example, they believed that it may help ensure a more consistent 

review of cases across the country and between adjudication levels. 

According to both stakeholders and other experts within and outside of 

SSA (including SSA’s Deputy Commissioner of Disability and Income 

Security and a consulting group that reviewed SSA’s quality assurance 

system),28 the current quality control and case review process encourages 

 28See the Lewin Group, Inc., et al., Evaluation of SSA’s Disability Quality Assurance (QA) 

Processes and Development of QA Options That Will Support the Long-Term Management 

of the Disability Program, (Falls Church, VA: Mar. 16, 2001), pp. 168 and E-9. 

The Agency and Most 

Stakeholders Viewed 

Several Aspects of the 

New Proposal as Offering 

Promise for Improving 

Consistency 
Page 28 GAO-04-656 SSA Disability Decisions 

adjudicators at the initial level to inappropriately deny cases, while 

encouraging adjudicators at the hearings level to inappropriately allow 

cases. Specifically, by overemphasizing a review of DDS allowances to 

help control the cost of benefits, the agency has unintentionally 

encouraged DDS examiners to deny cases. Conversely, SSA’s review of 

ALJ decisions consists mostly of SSA’s Appeals Council reviewing cases 

denied by ALJs, thereby providing an incentive for ALJs to allow cases. By 

centralizing the quality control system and making other changes to the 

process, SSA believes that it can remove the current incentives that 

contribute to inconsistency. 

The third proposed change that the agency and most stakeholder groups 

believe may improve consistency is SSA’s plan to provide both 

adjudication levels with equal access to more centralized medical experts, 

organized by clinical specialty. Although located in the regions, these 

experts should be able to review cases from across the country with the 

successful completion of SSA’s AeDib initiative—an electronic folder 

initiative for exchanging case information currently being implemented by 

SSA. By making experts in a range of specialties available to assist both 

levels of adjudicators in their decision making, SSA and stakeholders 

believe that adjudicators could more consistently apply SSA’s decisionmaking criteria, in addition to acquiring better medical evidence. 

Finally, the agency and most stakeholder groups told us that the 

requirement to have an ALJ’s decision address the recommendations from 

a reviewing official’s report to either deny or more fully develop the claim 

may increase consistency between levels. Under the new proposal, SSA 

plans to introduce a reviewing official into the process to evaluate all 

appealed DDS claims. The official will allow claims that meet SSA’s 

definition of disability and, for the remaining claims, will develop a report 

that either (1) contains reasons for denying the claim or (2) outlines the 

evidence needed to fully support the claim. The ALJ’s decision must 

address issues raised in the reviewing official’s report. Stakeholders 

believed that this change could, as intended by SSA, hold adjudicators 

more accountable for their decisions and provide adjudicators with 

feedback on the reasons decisions tend to differ between levels to improve 

the quality and consistency of their decisions. 

Although there was less agreement among stakeholder groups on the 

potential effect that other aspects of the new proposal may have on the 

consistency of decisions, some groups thought that other changes could 

result in improved consistency between DDS and ALJ decisions. For 

example, the Social Security Advisory Board and two groups representing 
Page 29 GAO-04-656 SSA Disability Decisions 

the DDSs thought that the proposed in-line quality control, if implemented 

effectively at all levels, could have a positive impact on consistency by 

ensuring that adjudicators adhere to the rulings and regulations 

throughout the decision-making process. One stakeholder group added 

that in-line quality control could also help the agency identify problem 

areas, including areas in which policy is applied inconsistently or where 

more training is needed. 

According to stakeholder groups—and based on SSA’s prior experience 

with making significant changes to its claims process—insufficient 

resources and other obstacles may prove to be major challenges for the 

agency in developing and implementing aspects of its new proposal. For 

example, experience with the process unification initiative has shown that 

limited state resources have hindered the agency’s ability to have state 

adjudicators fully document decisions. To address this issue, SSA plans to 

reduce the states’ workloads by decreasing the number of claims to be 

decided by the DDSs. Specifically, SSA expects that establishing regional 

expert review units to make quick decisions for claimants who are 

obviously disabled will substantially decrease the states’ workloads. 

However, SSA has not developed and provided stakeholders with 

estimates of the administrative cost for more fully documenting decisions 

and other planned changes, and stakeholder groups were not convinced 

that the reduction in claims was sufficient to offset resources needed to 

fully document their decisions. Although the agency has had some recent 

success in increasing its 2004 administrative budget, and is confident that 

it will be successful in acquiring the resources it needs to implement the 

proposal, the significance of stakeholders’ concerns about funding cannot 

be assessed until SSA fully develops its proposal and associated cost 

estimates. 

Experience has also shown that another proposed change, developing a 

centralized quality control system for both adjudication levels, could be a 

major challenge for the agency. In 1994, SSA began efforts to create a 

unified and comprehensive quality control system as part of its redesign 

efforts, but made little progress, in part because of considerable 

disagreement among internal and external stakeholders on how to 

accomplish this difficult objective. To get external assistance in 

developing an effective quality assurance system, SSA contracted with an 

independent consulting firm to assess SSA’s quality assurance practices 

used in the disability claims process. In 2001, concluding that SSA could 

achieve its quality objectives for the disability program only by adopting a 

broad, modern view of quality management, the consulting firm 

Resource and Other 

Constraints May Limit 

SSA’s Ability to 

Successfully Implement 

Some Changes in the New 

Proposal 
Page 30 GAO-04-656 SSA Disability Decisions 

recommended SSA abandon its current system and design a new quality 

management system focused on building quality into the process. The 

agency agreed that it was appropriate to transform the existing quality 

assurance system and established an executive work group to decide a 

future course of action. The agency is working with another consulting 

group to further develop the changes recently proposed by the 

Commissioner. However, after 10 years of efforts to develop a more 

unified quality review system, SSA has not yet formulated changes to its 

quality review system, beyond the brief and general descriptions provided 

in the Commissioner’s new proposal. 

Other obstacles also add to the complexity and difficulty of implementing 

the proposal. For example, stakeholder groups have raised concerns about 

SSA’s ability to successfully implement its proposed change to provide 

equal access for all adjudicators to more centralized medical expertise by 

removing medical expertise from the state DDSs and providing expertise 

in regional offices. Stakeholder groups were concerned that SSA would 

not be able to attract and retain sufficient medical experts to meet the 

agency’s needs. They told us that states are currently experiencing 

problems attracting medical experts because SSA’s compensation rates 

are too low. State adjudicators, who currently work with medical experts 

directly at DDS offices, were also concerned that removing these experts 

and placing experts in SSA regional offices would impair the states’ 

effectiveness and efficiency. By placing experts in regional offices, state 

disability examiners would no longer have on-site access to these experts 

who help facilitate the states’ adjudication of claims and provide on-thejob training and mentoring to DDS examiners. 

Stakeholders have also raised questions about SSA’s ability to ensure that 

ALJs’ decisions fully respond to the reviewing officials’ reports and the 

ultimate effectiveness of this change. Stakeholder groups representing 

ALJs and claimant representatives believed that the requirements may 

have the potential to impinge on an ALJ’s legal responsibility to ensure a 

claimant receives a fair hearing and an independent decision. Other groups 

have raised concerns about SSA’s ability to ensure that ALJs will 

adequately address recommendations in the reviewing officials’ reports to 

help ensure that this requirement leads to more consistent decisions. 

Although these concerns have been raised, the Commissioner has clearly 

stated that the intent of the proposal is to improve service to claimants, 

including providing fair and accurate decisions, and that changes will not 

impinge on the independence of ALJs. 
Page 31 GAO-04-656 SSA Disability Decisions 

In addition, several stakeholder groups also told us that staffing the new 

reviewing official positions with attorneys, as SSA intends to do, would be 

expensive. To the extent that SSA has difficulty filling these positions, the 

agency could create a slowdown or bottleneck in the process that could 

increase the time claimants must wait for a decision. Furthermore, 

according to one stakeholder group, SSA’s new quality assurance process 

will need to ensure that this new position does not create another source 

of inconsistent interpretation and application of SSA’s decision-making 

criteria. 

Several groups representing hearings level adjudicators and claimant 

representatives were also concerned about other aspects of the 

Commissioner’s new proposal, such as the proposed elimination of the 

Appeals Council and the claimants’ loss of the right to appeal an ALJ 

decision to the council. The Appeals Council currently reviews about 

100,000 appealed ALJ decisions annually. For these claims, the council 

provides an additional appellate step for addressing claimants’ objections 

to the ALJs’ decisions, reviewing new medical information on the claims 

and reducing the number of claims appealed directly to the federal courts. 

According to one stakeholder group, the council also performs other 

important functions, such as reviewing claims for surviving children or 

spouses of workers who were insured under the disability insurance and 

retirement program. The council also reviews cases remanded from 

federal courts. This stakeholder group also told us that as SSA refines its 

proposal it will need to articulate how all of the council’s functions will be 

handled under the new process. 

Adding to uncertainties about the proposal’s success is its dependence on 

the successful development and implementation of the AeDib system—a 

highly complex and as yet unproven system using electronic folders to 

share information with all entities involved in disability determinations. 

SSA does not plan to implement its newly proposed changes before it has 

completed a national rollout of its electronic disability system, scheduled 

to be completed by October 2005. The new electronic disability system 

represents an important step toward a paperless and more efficient 

sharing of information by multiple partners involved in the disability 

claims process, including SSA and state officials, as well as physicians and 

other members of the medical community who provide needed medical 

evidence. SSA also expects this new system to provide critical 

management information for analyzing and reducing inconsistencies in 

disability decisions. As we previously reported, SSA has made progress 

developing the new system. However, its approach involves risks that 

could jeopardize the agency’s successful transition to an electronic 
Page 32 GAO-04-656 SSA Disability Decisions 

disability claims process.29 For example, SSA recently began a national 

rollout of the electronic disability system without fully evaluating pilot test 

results or ensuring the resolution of all critical problems. Skipping such 

important steps in development and implementation leaves the new 

system vulnerable to problems in its performance and reliability. In 

addition, problems with implementation of this system could delay the 

implementation of SSA’s new proposal. 

SSA recognizes that transforming its massive and complex disability 

programs and achieving the benefits envisioned by the Commissioner will 

be a challenging undertaking. The agency is refining its proposal and, as 

part of this process, is actively seeking input from stakeholder groups. The 

Commissioner and her staff have met directly with stakeholder groups to 

understand and begin to address their concerns. As the agency refines its 

proposal, the significance of both stakeholder concerns and previous 

problems SSA has experienced improving its programs should become 

clearer. 

When SSA’s Commissioner announced her new proposal to overhaul the 

disability programs, the agency acknowledged the importance of making 

similar decisions on similar cases and making the right decision as early in 

the process as possible. SSA has good cause to focus on the consistency of 

decisions between adjudication levels. Incorrect denials at the initial level 

that are appealed increase both the time claimants must wait for a 

decision and the cost of deciding cases. Incorrect denials that are not 

appealed may leave needy individuals without a financial or medical safety 

net. Conversely, incorrect allowances at any adjudication level could 

substantially increase the cost of providing disability benefits. 

While the agency has made some effort to assess the inconsistency in 

decisions between levels, its efforts have not provided the agency with a 

clear understanding of the extent and leading causes of possible 

inconsistencies in the interpretation and application of disability guidance. 

 29For additional information on SSA’s progress and its risky strategy for implementing its 

accelerated disability claims system, see U.S. General Accounting Office, Electronic 

Disability Claims Processing: Social Security Administration’s Accelerated Strategy 

Faces Significant Risks, GAO-03-984T (Washington, D.C.: July 24, 2003); Social Security 

Administration: Subcommittee Questions Concerning Efforts to Automate the Disability 

Claims Process, GAO-03-1113R (Washington, D.C.: Sept. 5, 2003); and Electronic 

Disability Claims Processing: SSA Needs to Address Risks Associated with Its 

Accelerated Systems Development Strategy, GAO-04-466 (Washington, D.C.: Mar. 26, 2004). 

Conclusions 
Page 33 GAO-04-656 SSA Disability Decisions 

For example, SSA’s assessment of ALJ error-prone allowances has not 

proven to be effective at identifying new areas and causes of 

inconsistency. SSA also has not updated its more effective approach of 

analyzing its Disability Hearings Quality Review data to identify problem 

areas and help improve its understanding of the factors that may be 

contributing to inconsistency. Further, SSA’s analysis lacked sophisticated 

statistical techniques and in-depth analysis of cases by adjudicators at 

both levels, which together would have allowed SSA to better identify and 

address the areas and leading causes of inconsistency. Moreover, by not 

having examiners and medical consultants perform a complete review of 

all relevant information before an ALJ, SSA has limited its ability to 

understand the areas and causes of possible inconsistency. 

Without better information on the areas and causes of possible 

inconsistency, the agency cannot ensure that the Commissioner’s new 

proposal will help to resolve this complex and long-standing concern. By 

taking immediate actions to improve its understanding of the leading 

causes of possible inconsistency in decisions, the agency will have 

information needed to evaluate and possibly refine its new proposal, 

including its plans to build an effective quality assurance system that can 

both detect and prevent inconsistencies in decisions. This information will 

help the agency to target its limited resources and take decisive steps to 

build a claims process that provides claimants with the accurate, 

consistent, and timely decisions they deserve, as envisioned in the 

Commissioner’s proposal. 

To move successfully forward with agency efforts to make more 

consistent decisions, including efforts incorporated in the Commissioner’s 

proposal for an improved disability claims process and quality assurance 

system, we recommend that SSA quickly expand its assessment of the 

areas and causes of inconsistency in decisions between adjudication 

levels. In doing so, SSA should consider making near-term and costeffective enhancements to its current approach for assessing the 

consistency of decisions, including: 

1. Reestablish ongoing analyses of case characteristics as part of its 

biennial case review, in line with efforts undertaken for the review 

report published in 1994. 

2. Perform more sophisticated multivariate analysis on the biennial 

case review data in order to pinpoint the most significant case 

characteristics influencing allowance decisions and to distinguish 

Recommendations 
Page 34 GAO-04-656 SSA Disability Decisions 

factors that might be contributing either appropriately or 

inappropriately to allowance decisions. 

3. Expand the biennial case review by requiring disability examiners 

and medical consultants to review the hearing tapes to ensure that 

reviewers have the complete case before them (including the types 

and sources of testimonial evidence provided during the hearings) 

when evaluating the ALJs’ decisions. 

4. Have adjudicators and reviewers from each level study cases in 

depth to help pinpoint the causes of inconsistency, once potential 

areas of inconsistency between levels are identified. 

5. Publish the methods and findings of all analyses, to keep internal 

and external stakeholders aware of the agency’s efforts to assess 

consistency and demonstrate improvement over time. 

6. Use the information from these improved analyses to develop a 

more focused and effective strategy for ensuring uniform 

application of SSA’s guidance and to improve the consistency of 

decisions. To accomplish this, SSA should clarify guidance for 

making disability decisions and develop mandatory training for 

adjudicators on issues identified as contributing to inconsistency.

We provided a draft of this report to SSA for comment. SSA expressed 

several reservations about the recommendations, findings, and 

conclusions of our report. Primarily, SSA took issue with: (1) our 

characterization of the agency’s progress over the past several years in 

analyzing and reducing the inconsistency of decisions, (2) our 

recommendation that the agency incorporate multivariate analysis into its 

assessments, and (3) our finding that the agency has not acted on the 

results of its reviews of decisions. SSA indicated that it would reevaluate 

our recommendations as the design of its Commissioner’s new approach 

to disability decision making evolves. However, the agency did agree to 

pilot one recommendation—that quality reviewers assess hearing tapes 

when evaluating the ALJs’ decisions—as part of a quality review. 

One of SSA’s main concerns was that our report did not fully discuss the 

progress SSA had achieved in analyzing and reducing the inconsistency in 

decision making between adjudication levels. For example, SSA 

commented that our report dismissed the 21-percentage point increase in 

the quality reviewers’ support rate of ALJ decisions, conducted as part of 

Agency Comments 

and Our Evaluation 
Page 35 GAO-04-656 SSA Disability Decisions 

SSA’s biennial case reviews over the last 10 years. SSA also pointed to 

findings from its ALJ peer reviews as additional evidence that the quality 

and consistency of SSA’s decisions had improved. In addition, SSA 

asserted that its comparison of the relative proportion of allowances at the 

DDS and ALJ levels, along with high accuracy rates, indicated that 

adjudicators were making the right decisions sooner in the process—a 

goal of both process unification and the Commissioner’s new disability 

approach. Although our report incorporates results from the analyses 

cited by SSA, our conclusion about the improvement in consistency 

between levels is not as optimistic as SSA’s because of weaknesses in 

SSA’s assessments. As we reported, SSA’s analysis of the quality reviewers’ 

assessment of ALJ cases has been limited for 10 years to calculating ALJ 

support rates. SSA has not used available data to determine the potential 

areas of inconsistency between levels or the extent to which changes in 

the ALJ support rate is related to improvements in consistency of 

decisions between adjudication levels. SSA’s assessment also lacks a 

reliable method for determining whether DDS decisions are more 

consistent with ALJ decisions, for example, by having ALJs regularly 

review a statistical sample of DDS decisions. Lastly, as we pointed out, 

changes in the proportion of overall allowances made by the DDS and ALJ 

levels cannot serve as a reliable indicator for measuring the consistency of 

decisions between levels, because many factors can affect these 

proportions, such as significant fluctuations in the number of decisions 

made at each adjudication level. 

SSA also expressed its reservations about the benefits of multivariate 

analysis in its evaluation of decision making. SSA asserted that its analyses 

over the past 10 years have provided the agency with a solid 

understanding of how certain variables influence disability decision 

making and that the multivariate analyses we recommended would not 

identify the causes and effects of inconsistent decision making at different 

levels of this complex process. We agree with SSA that the disability 

decision-making process is complex and that multivariate analysis alone 

cannot establish all the causes and effects of inconsistent decision making. 

However, because multivariate analysis takes into account the influence of 

a number of relevant variables for each decision, this analytical technique 

can provide a more accurate understanding of areas and causes of 

inconsistency in decisions than methods previously employed by SSA. 

Such analyses, followed by in-depth case studies by adjudicators at both 

levels, which we also recommended, would bring SSA closer to 

understanding and resolving the inconsistency of decisions between 

adjudication levels. Therefore, we continue to believe that by performing 

the analyses we recommend, the agency will have a better understanding 
Page 36 GAO-04-656 SSA Disability Decisions 

of the extent and causes of inconsistency, and that SSA’s Commissioner 

should quickly implement our recommendations to ensure that her new 

approach effectively addresses the consistency of decisions between 

adjudication levels. 

Finally, SSA disagreed with our finding that it has not acted on the results 

of its reviews of decisions. SSA noted that it has made changes to address 

training needs that have been identified by its reviews. Specifically, SSA 

indicated that it has provided a series of interactive video training (IVT) 

sessions focusing on problematic areas noted in the ALJ peer review 

reports. We acknowledge that SSA has conducted ALJ peer reviews and 

used findings from its reviews to develop and provide training to ALJs. 

However, we did not include these findings in our report, because our 

objectives were limited to reporting efforts undertaken by SSA to assess or 

improve the consistency of decisions between adjudication levels or to 

implement its process unification initiative. SSA’s ALJ peer review is 

conducted to identify problems with the quality of ALJ hearing process 

and decisions, not to identify problems with the inconsistency of decisions 

between levels. Conversely, our report included information on SSA’s ALJ 

pre-effectuation review, because it was part of SSA’s process unification 

initiative. According to information provided to us by SSA during our 

audit, although this review was intended to help identify policy and 

training areas that were associated with inconsistent decisions between 

adjudication levels, it was not effective at identifying any new areas to be 

pursued by the agency. This finding, along with those provided throughout 

the report, supports our recommendations to SSA that the agency perform 

additional analysis to determine the causes of potential inconsistency 

between adjudication levels and to clarify guidance and provide 

mandatory training to address any identified causes. 

In addition, SSA provided several other general and technical comments 

about the draft report. These additional comments, as well as our response 

to them, are provided in appendix II. 

Copies of this report are being sent to the Commissioner of SSA, 

appropriate congressional committees, and other interested parties. The 

report is also available at no charge on GAO’s Web site at 
Page 37 GAO-04-656 SSA Disability Decisions 

http://www.gao.gov. If you have any questions about this report, please 

contact me at (202) 512-7215. Other contacts and staff acknowledgments 

are listed in appendix III. 

Sincerely yours, 

Robert E. Robertson 

Director, Education, Workforce, 

 and Income Security Issues 
Appendix I: Excerpt of SSA’s Testimony 

Announcing Its New Proposal to Improve Its 

Disability Decision-Making Process 

Page 38 GAO-04-656 SSA Disability Decisions 

In designing my approach to improve the overall disability determination 

process, I was guided by three questions the President posed during our 

first meeting to discuss the disability programs. 1

• Why does it take so long to make a disability decision? 

• Why can’t people who are obviously disabled get a decision 

immediately? 

• Why would anyone want to go back to work after going through such a 

long process to receive benefits? 

I realized that designing an approach to fully address the central and 

important issues raised by the President required a focus on two overarching operational goals: (1) to make the right decision as early in the 

process as possible; and (2) to foster return to work at all stages of the 

process. I also decided to focus on improvements that could be 

effectuated by regulation and to ensure that no SSA employee would be 

adversely affected by my approach. My reference to SSA employees 

includes State Disability Determination Service employees and 

Administrative Law Judges (ALJs). 

As I developed my approach for improvement, I met with and talked to 

many people—SSA employees and other interested organizations, 

individually and in small and large groups—to listen to their concerns 

about the current process at both the initial and appeals levels and their 

recommendations for improvement. I became convinced that 

improvements must be looked at from a system-wide perspective and, to 

be successful, perspectives from all parts of the system must be 

considered. I believe an open and collaborative process is critically 

important to the development of disability process improvements. To that 

end, members of my staff and I visited our regional offices, field offices, 

hearing offices, and State Disability Determination Services, and private 

disability insurers to identify and discuss possible improvements to the 

current process. 

Finally, a number of organizations provided written recommendations for 

changing the disability process. Most recently, the Social Security 

 1

This excerpt is taken from a statement by the Honorable Jo Anne B. Barnhart, 

Commissioner, Social Security Administration, Testimony before the Subcommittee on 

Social Security of the House Committee on Ways and Means, September 25, 2003. 

Appendix I: Excerpt of SSA’s Testimony 

Announcing Its New Proposal to Improve Its 

Disability Decision-Making Process 
Appendix I: Excerpt of SSA’s Testimony 

Announcing Its New Proposal to Improve Its 

Disability Decision-Making Process 

Page 39 GAO-04-656 SSA Disability Decisions 

Advisory Board issued a report prepared by outside experts making 

recommendations for process change. My approach for changing the 

disability process was developed after a careful review of these 

discussions and written recommendations. As we move ahead, I look 

forward to working within the Administration and with Congress, as well 

as interested organizations and advocacy groups. I would now like to 

highlight some of the major and recurring recommendations made by 

these various parties. 

The need for additional resources to eliminate the backlog and reduce the 

lengthy processing time was a common theme. This important issue is 

being addressed through my Service Delivery Plan, starting with the 

President’s FY 2004 budget submission which is currently before 

Congress. Another important and often heard concern was the necessity of 

improving the quality of the administrative record. DDSs expressed 

concerns about receiving incomplete applications from the field office; 

ALJs expressed concerns about the quality of the adjudicated record they 

receive and emphasized the extensive pre-hearing work required to 

thoroughly and adequately present the case for their consideration. In 

addition, the number of remands by the Appeals Council and the Federal 

Courts make clear the need for fully documenting the administrative 

hearing record. 

Applying policy consistently in terms of: 1) the DDS decision and ALJ 

decision; 2) variations among state DDSs; and 3) variations among 

individual ALJs—was of great concern. Concerns related to the 

effectiveness of the existing regional quality control reviews and ALJ peer 

review were also expressed. Staff from the Judicial Conference expressed 

strong concern that the process assure quality prior to the appeal of cases 

to the Federal Courts. 

ALJs and claimant advocacy and claimant representative organizations 

strongly recommended retaining the de novo hearing before an ALJ. 

Department of Justice litigators and the Judicial Conference stressed the 

importance of timely case retrieval, transcription, and transmission. Early 

screening and analysis of cases to make expedited decisions for clear 

cases of disability was emphasized time and again as was the need to 

remove barriers to returning to work. 

My approach for disability process improvement is designed to address 

these concerns. It incorporates some of the significant features of the 

current disability process. For example, initial claims for disability will 

continue to be handled by SSA’s field offices. The State Disability 
Appendix I: Excerpt of SSA’s Testimony 

Announcing Its New Proposal to Improve Its 

Disability Decision-Making Process 

Page 40 GAO-04-656 SSA Disability Decisions 

Determination Services will continue to adjudicate claims for benefits, and 

Administrative Law Judges will continue to conduct hearings and issue 

decisions. My approach envisions some significant differences. 

I intend to propose a quick decision step at the very earliest stages of the 

claims process for people who are obviously disabled. Cases will be sorted 

based on disabling conditions for early identification and expedited action. 

Examples of such claimants would be those with ALS, aggressive cancers, 

and end-stage renal disease. Once a disability claim has been completed at 

an SSA field office, these Quick Decision claims would be adjudicated in 

Regional Expert Review Units across the country, without going to a State 

Disability Determination Service. This approach would have the two-fold 

benefit of allowing the claimant to receive a decision as soon as possible, 

and allowing the State DDSs to devote resources to more complex claims. 

Centralized medical expertise within the Regional Expert Review Units 

would be available to disability decision makers at all levels, including the 

DDSs and the Office of Hearings and Appeals (OHA). These units would be 

organized around clinical specialties such as musculoskeletal, 

neurological, cardiac, and psychiatric. Most of these units would be 

established in SSA’s regional offices. 

The initial claims not adjudicated through the Quick Decision process 

would be decided by the DDSs. However, I would also propose some 

changes in the initial claims process that would require changes in the way 

DDSs are operating. An in-line quality review process managed by the 

DDSs and a centralized quality control unit would replace the current SSA 

quality control system. I believe a shift to in-line quality review would 

provide greater opportunities for identifying problem areas and 

implementing corrective actions and related training. The Disability 

Prototype would be terminated and the DDS Reconsideration step would 

be eliminated. Medical expertise would be provided to the DDSs by the 

Regional Expert Review units that I described earlier. 

State DDS examiners would be required to fully document and explain the 

basis for their determination. More complete documentation should result 

in more accurate initial decisions. The increased time required to 

accomplish this would be supported by redirecting DDS resources freed 

up by the Quick Decision cases being handled by the expert units, the 

elimination of the Reconsideration step, and the shift in medical expertise 

responsibilities to the regional units. 
Appendix I: Excerpt of SSA’s Testimony 

Announcing Its New Proposal to Improve Its 

Disability Decision-Making Process 

Page 41 GAO-04-656 SSA Disability Decisions 

A Reviewing Official (RO) position would be created to evaluate claims at 

the next stage of the process. If a claimant files a request for review of the 

DDS determination, the claim would be reviewed by an SSA Reviewing 

Official. The RO, who would be an attorney, would be authorized to issue 

an allowance decision or to concur in the DDS denial of the claim. If the 

claim is not allowed by the RO, the RO will prepare either a 

Recommended Disallowance or a Pre-Hearing Report. A Recommended 

Disallowance would be prepared if the RO believes that the evidence in 

the record shows that the claimant is ineligible for benefits. It would set 

forth in detail the reasons the claim should be denied. A Pre-Hearing 

Report would be prepared if the RO believes that the evidence in the 

record is insufficient to show that the claimant is eligible for benefits but 

also fails to show that the claimant is ineligible for benefits. The report 

would outline the evidence needed to fully support the claim. Disparity in 

decisions at the DDS level has been a long-standing issue and the SSA 

Reviewing Official and creation of Regional Expert Medical Units would 

promote consistency of decisions at an earlier stage in the process. 

If requested by a claimant whose claim has been denied by an RO, an ALJ 

would conduct a de novo administrative hearing. The record would be 

closed following the ALJ hearing. If, following the conclusion of the 

hearing, the ALJ determines that a claim accompanied by a Recommended 

Disallowance should be allowed, the ALJ would describe in detail in the 

written opinion the basis for rejecting the RO’s Recommended 

Disallowance. If, following the conclusion of the hearing, the ALJ 

determines that a claim accompanied by a Pre-Hearing Report should be 

allowed, the ALJ would describe the evidence gathered during the hearing 

that responds to the description of the evidence needed to successfully 

support the claim contained in the Pre-Hearing Report. 

Because of the consistent finding that the Appeals Council review adds 

processing time and generally supports the ALJ decision, the Appeals 

Council stage of the current process would be eliminated. Quality control 

for disability claims would be centralized with end-of-line reviews and ALJ 

oversight. If an ALJ decision is not reviewed by the centralized quality 

control staff, the decision of the ALJ will become a final agency action. If 

the centralized quality control review disagrees with an allowance or 

disallowance determination made by an ALJ, the claim would be referred 

to an Oversight Panel for determination of the claim. The Oversight Panel 

would consist of two Administrative Law Judges and one Administrative 

Appeals Judge. If the Oversight Panel affirms the ALJ’s decision, it 

becomes the final agency action. If the Panel reverses the ALJ’s decision, 

the oversight Panel decision becomes the final agency action. As is 
Appendix I: Excerpt of SSA’s Testimony 

Announcing Its New Proposal to Improve Its 

Disability Decision-Making Process 

Page 42 GAO-04-656 SSA Disability Decisions 

currently the case, claimants would be able to appeal any final agency 

action to a Federal Court. 

At the same time these changes are being implemented to improve the 

process, we plan to conduct several demonstration projects aimed at 

helping people with disabilities return to work. These projects would 

support the President’s New Freedom Initiative and provide work 

incentives and opportunities earlier in the process. 

Early Intervention demonstration projects will provide medical and cash 

benefits and employment supports to Disability Insurance (DI) applicants 

who have impairments reasonably presumed to be disabling and elect to 

pursue work rather than proceeding through the disability determination 

process. Temporary Allowance demonstration projects will provide 

immediate cash and medical benefits for a specified period (12-24 months) 

to applicants who are highly likely to benefit from aggressive medical care. 

Interim Medical Benefits demonstration projects will provide health 

insurance coverage to certain applicants throughout the disability 

determination process. Eligible applicants will be those without such 

insurance whose medical condition is likely to improve with medical 

treatment or where consistent, treating source evidence will be necessary 

to enable SSA to make a benefit eligibility determination. Ongoing 

Employment Supports to assist beneficiaries to obtain and sustain 

employment will be tested, including a Benefit Offset demonstration to 

test to effects of allowing DI beneficiaries to work without total loss of 

benefits by reducing their monthly benefit $1 for every $2 of earnings 

above a specified level and Ongoing Medical Benefits demonstration to 

test the effects of providing ongoing health insurance coverage to 

beneficiaries who wish to work but have no other affordable access to 

health insurance. 

I believe these changes and demonstrations will address the major 

concerns I highlighted earlier. I also believe they offer a number of 

important improvements: 

• People who are obviously disabled will receive quick decisions. 

• Adjudicative accountability will be reinforced at every step in the 

process. 

• Processing time will be reduced by at least 25%. 
Appendix I: Excerpt of SSA’s Testimony 

Announcing Its New Proposal to Improve Its 

Disability Decision-Making Process 

Page 43 GAO-04-656 SSA Disability Decisions 

• Decisional consistency and accuracy will be increased. 

• Barriers for those who can and want to work would be removed. 

Describing my approach for improving the process is the first step of what 

I believe must be—and will work to make—a collaborative process. I will 

work within the Administration, with Congress, the State Disability 

Determination Services and interested organizations and advocacy groups 

before putting pen to paper to write regulations. As I said earlier, and I say 

again that to be successful, perspectives from all parts of the system must 

be considered. 

Later today, I will conduct a briefing for Congressional staff of the Ways 

and Means and Senate Finance Committees. I will also brief SSA and DDS 

management. In addition, next week I will provide a video tape of the 

management briefing describing my approach for improvement to all SSA 

regional, field, and hearing offices, State Disability Determination Services, 

and headquarters and regional office employees involved in the disability 

program. Tomorrow, I will be conducting briefings for representatives of 

SSA employee unions and interested organizations and advocacy groups, 

and I will schedule meetings to provide an opportunity for those 

representatives to express their views and provide assistance in working 

through details, as the final package of process improvements is fully 

developed. 

I believe that if we work together, we will create a disability system that 

responds to the challenge inherent in the President’s questions. We will 

look beyond the status quo to the possibility of what can be. We will 

achieve our ultimate goal of providing accurate, timely service for the 

American people. 
Appendix II: Comments from the Social Security Administration 

Page 44 GAO-04-656 SSA Disability Decisions 

Appendix II: Comments from the Social 

Security Administration 

Note: GAO comments 

supplementing those in 

the report text appear at 

the end of this appendix. 
Appendix II: Comments from the Social Security Administration 

Page 45 GAO-04-656 SSA Disability Decisions 

See comment 4. 

See comment 1. 

See comment 2. 

See comment 3. 
Appendix II: Comments from the Social Security Administration 

Page 46 GAO-04-656 SSA Disability Decisions 

See comment 5. 

See comment 6. 

See comment 7. 
Appendix II: Comments from the Social Security Administration 

Page 47 GAO-04-656 SSA Disability Decisions 
Appendix II: Comments from the Social Security Administration 

Page 48 GAO-04-656 SSA Disability Decisions 

See comment 8. 

See comment 9. 

See comment 10. 
Appendix II: Comments from the Social Security Administration 

Page 49 GAO-04-656 SSA Disability Decisions 

Now on p. 11. 

Now on page 11, footnote 13. 
Appendix II: Comments from the Social 

Security Administration 

Page 50 GAO-04-656 SSA Disability Decisions 

1. We maintain that our report fully and fairly describes SSA’s progress in 

analyzing and addressing the underlying causes of inconsistent 

decisions between state DDS examiners and ALJs. Our research 

included an extensive review of agency documentation and interviews 

with SSA officials, as well as stakeholder groups for adjudicators and 

claimant representatives, to develop a complete understanding of the 

agency’s efforts to assess and improve the consistency of decisions 

between adjudication levels. Also, in agreement with our requestor, we 

sought to expand the review to include SSA’s new approach to 

improving its disability programs, so that we could provide the 

Congress with an understanding of how SSA’s future plans may help to 

address this issue. 

2. We provided information on the various reviews and analyses of 

disability decisions to assess the consistency of decisions between 

adjudication levels conducted by SSA over the last 10 years, but none 

of these reviews have clearly identified the causes of inconsistency in 

decisions between adjudication levels. 

3. Our report has not overlooked the data cited by SSA. Nevertheless, 

our conclusion about the improvement in consistency between levels 

indicated by the data is not as optimistic as SSA’s because of 

weaknesses in SSA’s assessments. As we reported, for 10 years SSA’s 

analysis of the quality reviewers’ assessment of ALJ cases has been 

limited to calculating ALJ support rates. SSA has not used available 

data to determine the potential areas of inconsistency between levels 

nor the extent to which changes in the ALJ support rate are related to 

improvements in consistency of decisions between adjudication levels. 

SSA’s assessment only provides a general indication of overall changes 

in consistency at one adjudication level. 

4. Our report recognizes that SSA’s disability decision-making process is 

complex. Because of this complexity, we believe that multivariate 

analysis is an appropriate assessment tool that would allow SSA to 

assess the effect of multiple factors. In recommending this 

sophisticated tool, we were careful not to imply that causes and effects 

of inconsistent decision making can be established with certainty. 

However, we believe that such an analysis will help SSA understand 

the relative importance of the variety of factors that affect its decisionmaking process. After identifying areas of inconsistency, SSA can 

target these areas with in-depth case analyses to pinpoint the causes of 

inconsistency and develop a more effective strategy for addressing 

inconsistency. On the basis of our review of SSA’s analyses to date, we 

do not agree with the implications of SSA’s comments that it has a 

GAO Comments 
Appendix II: Comments from the Social 

Security Administration 

Page 51 GAO-04-656 SSA Disability Decisions 

solid understanding of how certain variables influence disability 

decision making, and therefore does not need to conduct additional, 

more sophisticated analyses. 

5. We agree with SSA that the proportion of allowances made at each 

level can provide some insight into the allowance rate dynamic. 

However, as we reported, we do not believe that it can serve as a 

reliable indicator of the agency’s progress in achieving more consistent 

decisions between the DDS and OHA levels. The allowance data 

provided by SSA simply show that the relative proportion of 

allowances made at the DDS level increased in comparison with the 

OHA level, but SSA has not performed any additional analysis to show 

that these changes have any relationship to improved consistency in 

decision making between the two adjudication levels. Additional 

analysis is needed because a myriad of factors, such as changes in the 

economy, can affect allowance rates. Although SSA claims that over 

this period of time the economy has been “relatively stable,” without 

performing any additional analysis it cannot eliminate changes in the 

economy or demographics of claimants as an influence on the 

allowance rates at each level. In addition, SSA has not analyzed how 

other factors, such as changes in productivity and total number of 

decisions made at each level, may be influencing the allowance data. 

6. The allowance rate data provided by SSA in its comments is very 

similar to that provided by SSA earlier to us and included in our report 

in figure 2. The figures we reported for the proportion of allowances 

made by the DDS and OHA levels for fiscal years 1997 and 1998 vary in 

comparison with those provided by SSA by one percentage point. We 

have not changed the figures in our report because we believe that 

these slight differences simply reflect that we reported data based 

upon fiscal, not calendar, years. 

7. In our report, our statements that SSA has not made changes as a 

result of findings from its reviews were specifically related to SSA’s 

ALJ pre-effectuation review. We included information on this review 

because it was part of SSA’s process unification initiative and was 

intended to identify policy and training areas associated with 

inconsistent decisions between adjudication levels. During our review, 

we were told by an SSA official that the ALJ pre-effectuation review 

was not successful at identifying new areas of inconsistency to be 

addressed by SSA. In its comments, SSA cites a review unrelated to 

assessing the inconsistency of decisions between levels, the ALJ peer 

review, to assert that it has used reviews to identify training issues to 

improve the quality of decisions. The lack of success with the ALJ pre-
Appendix II: Comments from the Social 

Security Administration 

Page 52 GAO-04-656 SSA Disability Decisions 

effectuation review—along with other findings showing a limited 

understanding of the cause of inconsistency—supports our 

recommendations to SSA to perform additional analysis and to clarify 

guidance and provide mandatory training to address any identified 

causes of inconsistency between adjudication levels. 

8. We applaud SSA’s plans to use the electronic disability system to 

capture critical management information to address decisional 

variance or inconsistency, which could provide a wealth of useful 

information for the agency. We have adjusted our report’s text to 

reflect this additional purpose. We continue to believe that SSA should 

not wait for the development of this system, but should proceed to 

perform multivariate analysis, using available data from its biennial 

case reviews, to start identifying areas of potential inconsistency 

between adjudication levels. 

9. We applaud SSA’s deep commitment to improving the disability 

decision-making process, but believe that additional efforts to 

understand the causes of potential inconsistencies in decision making 

would help to inform the design of the Commissioner’s new approach 

and should, therefore, be undertaken immediately. 

10. We generally agree with the technical comments provided and changed 

the text accordingly. 
Appendix III: GAO Contacts and Staff 

Acknowledgments 

Page 53 GAO-04-656 SSA Disability Decisions 

Michele Grgich, Assistant Director (415) 904-2183 

Julie DeVault, Analyst in Charge (415) 904-2269 

In addition to the individuals mentioned above, the following staff 

members made major contributions to this report: Michael Morris, Corinna 

Nicolaou, Walter Vance, and Rebecca Woiwode. Douglas Sloane provided 

assistance with methodological issues, and Daniel Schwimer provided 

legal support. 

Appendix III: GAO Contacts and Staff 

Acknowledgments 

GAO Contacts 

Staff 

Acknowledgments 

(130219) 
The General Accounting Office, the audit, evaluation and investigative arm of 

Congress, exists to support Congress in meeting its constitutional responsibilities 

and to help improve the performance and accountability of the federal 

government for the American people. GAO examines the use of public funds; 

evaluates federal programs and policies; and provides analyses, 

recommendations, and other assistance to help Congress make informed 

oversight, policy, and funding decisions. GAO’s commitment to good government 

is reflected in its core values of accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and fulltext files of current reports and testimony and an expanding archive of older 

products. The Web site features a search engine to help you locate documents 

using key words and phrases. You can print these documents in their entirety, 

including charts and other graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site 

daily. The list contains links to the full-text document files. To have GAO e-mail 

this list to you every afternoon, go to www.gao.gov and select “Subscribe to e-mail 

alerts” under the “Order GAO Products” heading. 

The first copy of each printed report is free. Additional copies are $2 each. A 

check or money order should be made out to the Superintendent of Documents. 

GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a 

single address are discounted 25 percent. Orders should be sent to: 

U.S. General Accounting Office 

441 G Street NW, Room LM 

Washington, D.C. 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800 

U.S. General Accounting Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 

GAO’s Mission 

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Mail or Phone 

To Report Fraud, 

Waste, and Abuse in 

Federal Programs 

Public Affairs