GAO-05-100R SSA’s Disability Programs 

December 10, 2004 

The Honorable Jo Anne B. Barnhart 

Commissioner, Social Security Administration 

Subject: SSA’s Disability Programs: Improvements Could Increase 

 the Usefulness of Electronic Data for Program Oversight

Dear Ms. Barnhart: 

In 2003, we added the federal government’s disability programs to our 

high-risk list in part because of difficulties agencies faced in managing 

these programs and the expected growth in the rolls as baby boomers 

reach their disability-prone years. The Social Security Administration 

(SSA) manages the federal government’s two largest disability programs, 

the Disability Insurance (DI) program and the Supplemental Security 

Income (SSI) program, which together paid out $91 billion in federal 

benefits to 11.4 million individuals with disabilities in 2003. To help 

address management difficulties and prepare for expected growth in the 

rolls, SSA must have reliable administrative data from its disability 

decision-making process to adequately understand the population it serves 

and the possible effect of proposed program changes on this population. 

However, in a prior study, we identified potential problems with the 

reliability of SSA’s electronic administrative data.1

This report examines (1) the extent to which SSA collects useful and 

reliable electronic administrative data in order to effectively manage its DI 

and SSI programs and (2) whether ongoing and planned changes to SSA’s 

computer systems and internal controls will address any weaknesses that 

we identified. To determine the adequacy of SSA’s electronic 

administrative data for management of its disability programs, we 

reviewed SSA, Office of the Inspector General (OIG), and our prior reports 

regarding these data; interviewed users of these data; and performed a 

literature search on criteria for establishing information databases. To 

identify the nature and extent of data problems, we interviewed staff 

 1

GAO, SSA Disability Decision Making: Additional Steps Needed to Ensure Accuracy and 

Fairness of Decisions at the Hearings Level, GAO-04-14, Washington, D.C.: Nov. 12, 2003. 

United States Government Accountability Office

Washington, DC 20548 
Page 2 GAO-05-100R SSA’s Disability Programs 

responsible for managing and using the data to assess the controls and 

processes in the disability system, conducted data reliability tests on 

selected variables in the disability records for calendar year 2003, and 

reviewed relevant reports by OIG and SSA contractors for their 

assessment of the data quality. To determine the extent to which SSA has 

plans or efforts under way to address these problems, we interviewed SSA 

officials and reviewed SSA documents and OIG reports. We conducted our 

work between December 2003 and September 2004 in accordance with 

generally accepted government auditing standards. 

While most types of information collected by SSA are useful for program 

management purposes, the agency lacks sufficient internal controls to 

ensure that these data are reliable. In its electronic records, SSA collects 

information related to the claimant, process, and decision—all of which 

are important for informing aspects of program management. For 

example, SSA electronically collects information on the claimant’s 

impairment that is used to identify beneficiaries who should be reviewed 

for medical improvement and continuing eligibility for disability benefits. 

However, SSA does not collect some information that would enhance 

program oversight. For example, without an indicator on claimant’s race, 

SSA lacks a complete basis for assessing the equity of disability decisions. 

At the same time, SSA collects other information in its electronic disability 

record that the agency considers of limited value. For example, codes 

entered for the claimant’s prior occupation and industry are based on 

outdated lists, and the agency acknowledges that the value of that 

information is therefore limited. Although the types of information SSA 

collects in its electronic records generally contribute to the management 

of the disability programs, the accuracy of the records is unknown. For 

example, inadequate data entry controls permit the entry of invalid 

impairment codes—codes that do not reflect a meaningful impairment. In 

addition, inadequate data entry controls have resulted in missing data for 

some information, such as the claimant’s educational level, which is a 

critical factor in the disability decision and is therefore important in a 

study of the disability decision-making process. Most important, because 

SSA’s policy does not require that the electronic record be verified against 

the information in the case file, the agency does not know the extent to 

which codes that appear valid reflect the claimant’s actual information, 

such as the claimant’s actual impairment or education level. 

While SSA’s ongoing and planned changes to computer systems and 

internal controls may reduce the chance for some inaccuracies, SSA’s 

plans do not provide adequate assurance of the accuracy of its electronic 

Results in Brief 
Page 3 GAO-05-100R SSA’s Disability Programs 

administrative data. SSA’s ongoing transition from a paper-based disability 

system to an electronic one should reduce some of the inaccuracies we 

identified by limiting the amount of data re-entered at various levels in the 

process. However, SSA’s planned changes will not address data entry 

problems found by GAO that could be prevented with additional data entry 

controls. SSA’s current plans also do not include an internal control 

strategy for ensuring that electronic data match the information in the 

case file, nor do they provide for corrective action when inaccuracies are 

found. Finally, although SSA has proposed far-reaching changes to its 

disability decision-making process and is currently reassessing the 

processes for ensuring the quality of its disability decisions, the agency has 

not yet made any plans for evaluating the types of information it currently 

collects and whether other types of information would improve program 

management and oversight. 

To improve the value of SSA’s electronic administrative data for managing 

its disability programs, we recommend that the agency establish a costeffective internal control strategy to ensure that those data are reliable and 

that they accurately reflect the information in the case file. In addition, the 

agency should take steps to review the usefulness of the types of 

information collected and consider whether additional types of 

information could improve program oversight. 

In commenting on the draft of this report, SSA generally agreed with our 

conclusions and recommendations. SSA acknowledged that the report 

made valid points about steps SSA can take to increase the usefulness of 

electronic data for program oversight, and that the agency would consider 

how best to incorporate improvements in its data collection activities. 

DI and SSI are the two largest federal programs providing cash assistance 

to people with disabilities. Established in 1956, DI provides monthly 

payments to workers with disabilities (and their dependents) under the 

age of 65 who have enough work experience to qualify for disability 

benefits. Created in 1972, SSI is a means-tested income assistance program 

that provides monthly payments to adults or children who are blind or 

who have other disabilities and whose income and assets fall bellow a 

certain level. To be considered eligible for either program as an adult, a 

person must be unable to perform any substantial gainful activity because 

of a medically determinable physical or mental impairment that is 

expected to result in death or that has lasted or can be expected to last for 

a continuous period of at least 12 months. 

Background 
Page 4 GAO-05-100R SSA’s Disability Programs 

The disability determination process is complex, involving more than one 

office and decision maker. The process begins at a field office, where an 

SSA representative determines whether a claimant meets the programs’ 

nonmedical eligibility criteria. Claims meeting these criteria are forwarded 

to a state’s disability determination service (DDS) office to determine if a 

claimant meets the agency’s medical eligibility criteria. At the DDS, the 

disability examiner and the medical or psychological consultants work as 

a team to analyze a claimant’s documentation, gather additional evidence 

as appropriate, and approve or deny the claim. A denied claimant may ask 

the DDS for a reconsideration of its finding, at which point a different DDS 

team will review the claim. 2 If the claim is denied again, the claimant may 

appeal the determination to SSA’s Office of Hearings and Appeals (OHA), 

where an administrative law judge (ALJ) will review it. The ALJ usually 

conducts a hearing in which the claimant and others may testify and 

present new evidence. In making the disability decision, the ALJ uses 

information from the hearing and from the state DDS, including the 

findings of the DDS’s medical consultant. A claimant whose appeal is 

denied may request a review by SSA’s Appeals Council and, if the claim is 

denied again, may file suit in federal court. 

At each step of the disability determination process, information collected 

is entered into the claimant’s case file as well as into an electronic 

database. SSA field office staff create the case file, which is the legal 

record for the disability determination and contains all documents related 

to the case. If the claimant meets the nonmedical eligibility criteria, the 

field office staff forwards the case file to the appropriate state DDS office. 

DDS staff add the information collected at the DDS level and the initial 

disability determination to the case file and enter selected data from the 

case file into an electronic record known as the Form 831. Once the DDS 

submits these 831 records to SSA, the records become part of SSA’s 831 

file.3

 If the claimant asks for a reconsideration, another DDS team will 

place the new information in the case file and enter selected data from the 

case file into a new 831 record reflecting the reconsideration decision, 

which will also become part of SSA’s 831 file. If the claimant appeals the 

 2

While most claimants may request a reconsideration, at the time of our study, SSA was 

testing an initiative that eliminates the reconsideration step from the DDS decision-making 

process. 

3

SSA’s database of 831 records is called the National Disability Determinations Service 

System (NDDSS) but is generally referred to as the “831 file.” Therefore, in this report, we 

use the term “831 file” or “831 database” to refer to the NDDSS. 
Page 5 GAO-05-100R SSA’s Disability Programs 

reconsideration by requesting a hearing by an administrative law judge at 

the Office of Hearings and Appeals, the ALJ conducts a new review of the 

claimant’s file and any additional evidence the claimant submitted since 

the DDS determination. The ALJ may also hear testimony from medical or 

vocational experts and the claimant regarding the claimant’s medical 

condition and ability to work. As with the DDS, OHA staff place the new 

information in the case file and enter selected data from the case file into 

an electronic record. These electronic records become part of a 

nationwide electronic database referred to as the Case Control System 

(CCS) database. 

It has been well established that adequate internal controls over electronic 

data systems are necessary to effectively manage programs, such as SSA’s 

disability programs. In 1998, the Social Security Advisory Board noted that 

it is essential for policy makers have accurate, balanced, and objective 

information to help them determine the extent to which the program is 

meeting the long-standing objectives of social adequacy and individual 

equity, the nature and extent of changes that may be needed, and the 

impact of proposals for change. SSA stated in its 2003 Performance and 

Accountability Report that the agency was committed to providing clear 

and reliable data to those who use the data for managing, decision making, 

and oversight of SSA’s programs. Finally, GAO’s standards state that 

adequate internal controls should be established to provide reasonable 

assurance that agency objectives are being achieved. These internal 

control standards define the minimum level of quality acceptable for 

internal controls in the government and include controls over information 

processing and management review of performance. 

SSA is implementing significant changes to its disability determination 

process, and considering numerous others, in order to improve speed and 

efficiency. Significant changes currently under way include the Electronic 

Disability System (eDib)—which SSA considers to be a foundation for 

other initiatives currently under consideration. With eDib, all components 

involved in disability claims adjudication and review—including SSA field 

offices, DDSs, OHA, and its Office of Quality Assurance—will use an 

electronic disability folder that replaces the paper case file. The electronic 

folder will contain an electronic copy of everything that was formerly 

included in the paper case file, such as electronic images of medical 
Page 6 GAO-05-100R SSA’s Disability Programs 

evidence and other documents related to the disability determination.4 The 

electronic folder will also contain electronic images of the 831 or CCS 

records.5

 When the process is fully implemented, each component will be 

able to electronically view and process claims using the electronic 

disability folder. Other initiatives under consideration include centralizing 

medical expertise in regional expert review teams to expedite claims 

where the claimant is clearly disabled, requiring the DDSs to more fully 

document decisions and provide in-line quality review, eliminating the 

DDS reconsideration and Appeals Council decisional levels, and 

establishing end-of-line quality review at OHA. 

Although, for the most part, the types of information SSA collects in its 

electronic records contribute to the management of the disability 

programs, the accuracy of the records is unknown. In its electronic 

records, SSA collects information related to the claimant, process, and 

decision, which informs aspects of program management. However, SSA 

does not collect some information that would enhance program oversight. 

At the same time, SSA enters codes in its electronic disability records for 

the claimant’s prior occupation and industry, even though these codes are 

based on outdated lists of occupations and industries, and the agency 

acknowledges that the value of that information is therefore limited. While 

SSA has some internal controls to help capture reliable data, these 

controls are not sufficient to ensure those data are reliable. SSA’s policy 

does not require that the electronic record be verified against the 

information in the case file, and inadequate data entry controls have also 

resulted in some inaccurate and missing data. 

 4

Electronic images are like electronic photocopies of the documents. They can be retrieved 

or viewed in their entirety, but the elements that constitute the image—such as words or 

numbers—are not in a format that allows them to be identified using electronic search 

mechanisms or readily retrieved and manipulated for reporting or analytical purposes. 

5

Although these electronic images cannot be manipulated, data keyed into the 831 and CCS 

databases are in a format that allows for easy identification, retrieval, and manipulation for 

reporting or analytical purposes. 

While Most Types of 

Information Collected 

Are Useful, Unknown 

Accuracy Detracts 

from Their Value 
Page 7 GAO-05-100R SSA’s Disability Programs 

In its electronic disability records, SSA collects important information 

about the claimant, the process, and the decision. For example, SSA 

electronically collects information on individuals who apply for disability 

benefits, such as the applicant’s name, Social Security number, gender, 

date of birth, state of residence, and type of benefits requested. It also 

collects information on the various events in the disability determination 

process, such as the application and decision dates, as well as the office 

processing the application and the adjudicative level involved. Finally, it 

collects information on the disability decision, such as the date the 

disability began, the primary impairment used in the medical 

determination of eligibility, and the regulation basis code for the disability 

decision that indicates the specific DI or SSI program requirements and 

whether they were met. 

This electronic disability information is analyzed and used to manage the 

DI and SSI programs. For example, SSA uses the data to monitor the time 

it takes the agency to process a claim and to determine the number of 

cases allowed and denied at each adjudicative level. From the 831 and CCS 

databases, SSA selects (randomly or based on particular characteristics) 

cases for review to help the agency ensure the accuracy of disability 

determinations. For each selected case, staff from SSA’s Office of Quality 

Assurance obtain and review the paper case file to ensure that the DDS or 

ALJ disability decision is accurate.6

 SSA also uses the data to determine 

when beneficiaries should be reviewed for continuing disability and what 

type of review is cost-effective. The disability data may also be used to 

study specific aspects of its disability determination process, and other 

issues, such as the prevalence or handling of specific illnesses, like 

chronic fatigue syndrome. 

The electronic disability data are also used to inform the Congress and the 

general public about the administration of the DI and SSI programs. For 

example, SSA uses these data to prepare the Annual Statistical 

Supplement, which is a major data resource on SSA-administered 

programs that serves as a foundation for various research and policy 

analyses and helps SSA to respond to requests for program data from 

congressional committees, government agencies at all levels, and the 

research community. Further, SSA uses these data to provide the Congress 

 6

Once eDib is fully implemented, instead of reviewing a paper case file, quality assurance 

staff will review electronic documents in the electronic disability folder, including medical 

records and other information related to the disability determination. 

SSA Captures Important 

Information for Program 

Management, but 

Additional Information 

Could Improve Program 

Oversight 
Page 8 GAO-05-100R SSA’s Disability Programs 

with an annual report on the status of the Supplemental Security Income 

program, including 25-year projections of program participation prepared 

by SSA’s Office of the Chief Actuary. SSA’s Office of the Chief Actuary 

uses these data to analyze demographic trends in mortality and morbidity 

rates, in order to project program growth and financial status. 

SSA shares its electronic disability data with other entities for research 

and analysis purposes. For example, the Disability Research Institute, 

which is funded through a cooperative agreement between SSA and the 

College of Applied Life Studies at the University of Illinois at UrbanaChampaign, uses SSA’s data to conduct in-depth research to directly 

inform disability policy decision makers. In addition, SSA’s administrative 

and program data have also been used by outside researchers and 

academics to study relationships between and among various 

characteristics of people with disabilities, including their work histories, 

impairments, and outcomes.7

 SSA’s databases have the advantage of being 

large enough to allow researchers to make statistically meaningful 

distinctions among subgroups of disabled individuals. 

While SSA has many uses for the information it collects, there is some 

information not being collected that could enhance program oversight. For 

example, after identifying unexplained racial differences in decisions 

made at the OHA level, we recommended that SSA collect information on 

race so that the agency can better monitor the equity of its decisions.8

 In 

this same study, we also recommended that SSA collect information on the 

source of medical evidence (such as specialist, primary physician, 

emergency physician) and type of medical evidence (such as blood test, xray, observation, opinion), which could be used to analyze and better 

understand the role the agency and claimant representatives play in 

developing evidence for the disability determination. 

 7

Outside researchers and academics using SSA disability data include Kajal Lahiri, Denton 

R. Vaughan, and Bernard Wixon, “Modeling Social Security’s Sequential Disability 

Determination Using Matched SIPP Data,” Social Security Bulletin, 58, 3-42; Congressional 

Research Service Report to the Congress (1992), “Status of the Disability Programs of the 

Social Security Administration,” 92-691 EPW, the Library of Congress, Washington, D.C.; 

and Eddy Bresnitz, Howard Frumkin, Lawrence Goldstein, David Neumark, Michael 

Hodgson, and Carolyn Needleman, “Occupational Impairment and Disability among 

Applicants for Social Security Disability Benefits in Pennsylvania,” American Journal of 

Public Health 84 (November 1994), 1786-1790. 

8

GAO-04-14. SSA officials told us that the agency formed a work group in September 2003 

to develop a plan for collecting race/ethnicity information. 
Page 9 GAO-05-100R SSA’s Disability Programs 

An SSA contractor also found the agency’s electronic medical information 

lacking for other purposes. In 2001, PricewaterhouseCoopers (PwC) 

reported on the quality of electronic disability data used to target cases for 

continuing disability reviews (CDRs)—i.e., reviews of medical 

improvement of beneficiaries to determine their continuing eligibility for 

disability benefits. In its report, PwC stated that although SSA collects a 

large amount of medical evidence for SSA disability decisions, very little of 

this medical information is captured in SSA’s electronic disability 

databases. Specifically, PwC noted that only 8 of the 162 variables 

captured in the 831 record relate directly to medical condition. PwC stated 

that despite the importance of medical information to any decision on 

continuing eligibility, a relatively small number of medical variables are 

available in SSA’s 831 record for use in determining the most cost-effective 

method to target cases for CDRs.9

In addition to medical information, SSA lacks electronic data on claimants’ 

vocational and functional capacity, which are critical factors in the 

disability decision and are therefore important in a study of the disability 

decision-making process.10 Nearly 60 percent of disability decisions cannot 

be made based solely on medical considerations and must rely on these 

additional factors. Nevertheless, in the CCS database, we found that no 

variable is related to vocational and functional capacity. For the DDS level, 

we found one variable in the 831 database that provides limited vocational 

and functional information.11 The 831 database has no other variables that 

capture additional factors often considered in making disability decisions 

(such as mental conditions, fine motor skills, and stamina). Moreover, for 

DDS decisions made in 2003, we found that the one variable related to 

vocational and functional factors is captured in only 10 percent of the 

cases in which such factors had to be considered for the disability 

 9

In targeting cases for CDRs, SSA also uses information contained in the Master Beneficiary 

Record system and the Supplemental Security Record system, as well as information from 

Medicare claim forms and individual responses to prior CDR mailer forms. 

10Vocational factors are age, education and work experience. A claimant’s residual 

functional capacity is the individual’s maximum sustained capability for sedentary, light, 

medium, heavy, or very heavy work. 

11The vocational rule number variable indicates certain vocational and functional patterns 

that include functional capacity, age, work history, and education level. 
Page 10 GAO-05-100R SSA’s Disability Programs 

determination.12 In the remaining 90 percent of these cases, DDS staff did 

not capture any vocational or functional factors used in the decision. 

Without this type of information, SSA lacks electronic data on critical 

factors considered in the disability decision and important to any study of 

the disability decision-making process. 

Mathematica—a contractor hired by SSA to evaluate new strategies for 

promoting employment among people with disabilities—also noted that 

important vocational and functional information was not being collected 

electronically by SSA. Specifically, Mathematica reported that certain 

characteristics (such as household composition, occupation, industry, and 

the presence of functional limitations) are, according to the research 

community, important predictors of employment, but they are not 

available in SSA’s electronic databases. 

While SSA does not collect many types of information that contractors and 

others believe would be useful for managing the agency’s programs, SSA 

may be collecting other data that are not particularly useful. SSA officials 

told us that they believe information on the claimant’s prior occupation 

and industry could be useful, but because the lists of industries and 

occupations used by SSA are largely outdated, the agency considers these 

variables to be of limited value. Nevertheless, some DDS staff members 

still collect information on the claimant’s prior occupation and industry.13 

 12SSA uses a five-step disability determination process whereby the agency considers 

vocation and functional capacity in steps 4 and 5, i.e., when it has determined that the 

claimant has a severe, medically determinable physical or mental impairment, but the 

impairment is not severe enough for the individual to be determined eligible based on 

medical factors alone. In steps 4 and 5, SSA uses these additional factors to determine 

whether the claimant’s impairment(s) prevents the performance of his or her past work or 

other work in the national economy. Federal regulations contain rules (vocational rule 

numbers) that direct a finding of disabled or not disabled based on vocational factors (such 

as age, education, and work experience) in combination with the individual’s residual 

functional capacity. 

13In 1992, when the DDS workload was heavy, SSA informed the DDSs that the variables for 

industry, occupation, and years in occupation did not need to be filled in. Although this was 

intended to be a temporary measure, SSA’s incentive for re-instituting the policy 

diminished as the job classifications grew more outdated over time. 
Page 11 GAO-05-100R SSA’s Disability Programs 

While SSA emphasizes the importance of prompt and accurate disability 

determinations, the agency has not put sufficient emphasis on creating a 

positive control environment that would ensure a reasonable level of 

quality for its electronic disability data. SSA officials told us that although 

the DDSs and OHA are required to ensure that disability determinations 

are accurate, there is no specific requirement that electronic records 

provided to SSA be accurate. Regulations specify only that the DDSs must 

“provide the organizational structure, qualified personnel, medical 

consultant services, and a quality assurance function sufficient to ensure 

that disability determinations are made accurately and promptly.” SSA 

officials told us that while some DDS offices may verify the accuracy of 

electronic data, SSA does not require or suggest that the DDSs follow a 

specific strategy or format for doing so. Some DDS and OHA staff and 

supervisors told us that they were unaware of how SSA used the 831 and 

CCS records or why it was important that they be accurate. If SSA 

required verification on at least a sample of cases, it would indicate to 

DDS and OHA staff the importance of accuracy in the electronic data. 

SSA has some electronic controls in place to help ensure the quality of 

data. These front-end data entry controls help ensure data quality by 

limiting what can be keyed into certain variables. For example, some 

variables accept only a numeric entry, and others, such as the Social 

Security number, must be filled in or the record will be rejected. Many 

date variables require a month/date/year entry, and certain dates must fall 

within a prescribed range. For example, the application date must be on or 

after a person’s date of birth. In addition to these front-end data entry 

controls, SSA also reviews the 831 file by checking for unacceptable 

entries for certain variables. If SSA finds an increasing trend of 

unacceptable entries being keyed into those variables, the agency may 

issue additional guidance to staff in the field. On the basis of the results of 

this review, SSA officials told us that while the data may not be 100 

percent accurate, they believe the data are sufficiently accurate. 

However, while such controls may reduce data entry errors, we believe the 

data entry controls are inadequate. In reviewing SSA’s electronic records 

for disability decisions made by the DDSs and OHA in 2003, we found that 

even though information keyed into the record had not been rejected by 

the data entry controls, the record contained information that was clearly 

wrong because it did not agree with other information in the same 

SSA Does Not Have 

Adequate Internal Controls 

to Ensure That the 

Electronic Data Accurately 

Reflect the Facts of the 

Case 
Page 12 GAO-05-100R SSA’s Disability Programs 

record.14 This type of comparison was possible for only a few variables—

impairment code, body system, the code indicating the basis for the 

disability decision, and the disability decision itself—but these are 

important variables that contain information crucial to SSA’s management 

of its disability programs. Although our analysis identified only a small 

number of errors, the errors were in key variables. For example, GAO 

found that 

• More than 4 percent of DDS records for claims processed in 2003 

contained codes for the body system and impairment that were 

incompatible, indicating that either the impairment code or the body 

system code was incorrect. 

• For 211 claims allowed by the DDSs and 1,541 claims allowed by OHA, 

the impairment code indicated an unknown impairment for which 

there was no medical evidence. 

• For 239 claims allowed by the DDSs and 519 claims allowed by OHA, 

the impairment code indicated an unknown impairment for which 

there was insufficient medical evidence. 

• For nearly 4 percent of OHA’s electronic disability records for allowed 

claims, the impairment codes did not appear in the OHA list of 

impairment codes. 

• OHA’s records indicated that 2,367 applicants, who applied for both the 

DI and SSI programs, met SSA’s medical criteria for one program but 

not the other—a situation not allowed under SSA regulations, which 

require that the same medical criteria apply to both programs. 

Nevertheless, for each of these applicants, OHA recorded different 

decision codes, primary impairment codes, or regulation basis codes 

for the two programs. 

Inadequate controls have also resulted in missing data. In reviewing the 

831 and CCS records, we found that for many of the variables, leaving the 

variable blank is considered a valid entry. Therefore, when variables are 

not filled in, it’s not possible to determine if staff had the information but 

failed to key it in, or if staff did not have the information and correctly left 

 14We reviewed 3.9 million 831 records for disability decisions made in 2003, of which 31 

percent (1.2 million) were allowed. We also reviewed over 400,000 CCS records for 

disability decisions made in 2003, of which 70 percent (nearly 282,000) were allowed. 
Page 13 GAO-05-100R SSA’s Disability Programs 

it blank. Although for most variables we were unable to determine that the 

variable should have been filled in, the education variable was an 

exception. For cases in which the adjudicator is required to consider 

educational level in making the disability determination, the information 

should have been obtained and keyed into the record. 15 Yet, for 25 percent 

of such cases, that variable was left blank. This raises the question of how 

much of the missing data for other variables was available and should 

have been keyed into the record. 

Other organizations have also found weaknesses in SSA’s data entry 

controls. In its 2001 study for SSA, PwC reported that poor data quality 

could present a significant barrier to efficient CDR case selection. As a 

result, it recommended that the agency improve system data edits, stating 

that such data entry controls would improve the integrity of disability 

programs, reduce inaccuracies and inconsistencies in data collection and 

reporting, and reduce the risk of error or erroneous data collection and 

reporting. PwC estimated that between 2001 and 2005, poor and missing 

disability data would cause SSA to improperly target CDRs and thereby 

incur additional administrative and program costs. An OIG report in 2000 

reported that missing or invalid diagnostic codes had contributed to SSA’s 

failing to perform mandatory reviews of certain disability cases and 

prevented the agency from accumulating and disseminating more accurate 

disability statistics. 

Even when variables have been filled in and accepted by existing data 

entry controls, SSA does not know the extent to which the data reflect the 

information in the case file because the agency’s policy does not require 

that the electronic record be verified against the information in the case 

file. SSA’s case files contain all documents related to the claim (such as 

medical records) and represent the legal record of the claim. Although we 

did not perform a comparison of electronic records against actual case 

files, a study by SSA’s OIG in 2000 provides an example of the type of 

inaccuracies that may exist in disability records, even when the entries 

have been accepted by data entry controls. The OIG reviewed case files for 

a sample of 132 disability beneficiaries whose electronic records 

contained questionable impairment codes. It found 63 electronic records 

that contained an impairment code indicating, “diagnosis established—no 

predetermined list code of medical nature applicable,” although the case 

 15Over 50 percent of all claims processed by DDSs in 2003 required that educational level be 

considered in making the disability determination. 
Page 14 GAO-05-100R SSA’s Disability Programs 

file contained enough information to determine the proper impairment 

code. At the same time, the OIG sample was quite small and not intended 

to be representative. It is not cost-effective to check all, or even a large 

percentage of, electronic records against case files, and perfect reliability 

is not possible. However, without comparing a more representative sample 

of electronic records against case files, SSA cannot reliably estimate the 

true extent of inaccuracies in the electronic data. 

SSA’s ongoing and planned changes to computer systems and internal 

controls may reduce the chance for some inaccuracies. However, these 

planned changes will not address data entry problems found by GAO that 

could be prevented with additional data entry controls. SSA’s current 

plans also do not include an internal control strategy for ensuring that 

electronic data match the information in the case file, nor do they provide 

for corrective action when inaccuracies are found. Finally, although SSA 

has proposed far-reaching changes to its disability decision-making 

process and is currently reassessing the processes for ensuring the quality 

of its disability decisions, the agency has not yet made any specific plans 

for evaluating the types of information it currently collects and whether 

other types of information would improve program management and 

oversight. 

Although most of SSA’s ongoing and planned changes do not involve 

aspects of the electronic data systems touched on in this report, a few will 

alter the way information is transmitted and, in this way, could reduce the 

number of chances for unintentional errors. Under eDib, SSA field office 

staff keying in the disability history for an applicant will now use the 

Electronic Data Collect System (EDCS). Once the field office staff have 

keyed the Social Security number into the EDCS record, EDCS 

automatically verifies that there is a disability application in existing SSA 

databases. Information such as the application date, disability allegations, 

and disability onset date are automatically propagated into the EDCS 

record. This will improve the accuracy of the information keyed into the 

EDCS record, which goes into the electronic disability folder used at all 

levels of the disability determination process. Some additional data entry 

controls that are part of the new OHA case tracking software (the Case 

Processing Management System—CPMS) that was implemented this year 

as part of eDib may also reduce data entry errors. 

SSA’s Ongoing and 

Planned Changes May 

Reduce the Chance 

for Some 

Inaccuracies, but the 

Agency Has No Plans 

for Improving the 

Accuracy or 

Usefulness of 

Electronic Data 

SSA’s Ongoing and Planned 

Changes May Reduce the 

Chance for Some 

Inaccuracies 
Page 15 GAO-05-100R SSA’s Disability Programs 

SSA’s current plans do not include an overall internal control strategy for 

ensuring the accuracy of electronic disability data. While SSA has included 

some front-end data entry controls, it does not address problems with 

electronic data we found that could be prevented with additional controls. 

SSA also has no plans to improve its system for the back-end review of 

electronic disability records, so that the agency can estimate the accuracy 

of its electronic disability records and provide feedback to staff on 

accuracy. Further, the agency has no specific plans to evaluate the types of 

information collected during the disability determination process. 

Most of the changes currently under way—such as creating a shared 

electronic folder for each disability case, scanning evidence for inclusion 

in the electronic folder, providing new software systems, and improving 

interfaces between systems—were intended to improve the speed and 

efficiency of the disability claims process and not necessarily to improve 

the reliability or usefulness of the data in SSA’s 831 and CCS databases. To 

accelerate the rollout of eDib, SSA carried over the requirements from the 

old systems to the new systems. Although the agency did introduce some 

additional front-end data entry controls to reduce the entry of invalid data, 

the agency did not add sufficient controls to address the problem with 

missing and invalid data we found. An OHA official told us that even the 

additional data entry controls available in the new CPMS software are not 

adequate to prevent data entry errors we identified in this study. For 

example, under eDib, SSA’s data entry control still allows any four digits 

for the impairment variable in the CPMS, as well as the 831 file, rather than 

limiting data entry to one of the 240 acceptable impairment codes. These 

officials said that limited resources have prevented the agency from 

establishing and maintaining additional or more exacting controls—such 

as a current list of acceptable impairment codes. Moreover, SSA officials 

told us that additional data entry controls would require staff to determine 

the correct entry when errors were identified, which could slow case 

processing. 

SSA also has no plans to improve its system for the back-end review of 

electronic disability records, so that the agency can estimate the accuracy 

of its electronic disability records and provide feedback to staff on 

accuracy. SSA already has a process in place for reviewing a random 

sample of case files as part of its quality assurance for disability decisions, 

which allows the agency to make estimations of the accuracy of decisions 

at both the DDS and the OHA levels. In the course of this quality assurance 

SSA’s Plans Do Not Include 

Strategies for Ensuring 

Data Accuracy or 

Evaluating Types of 

Information Needed 
Page 16 GAO-05-100R SSA’s Disability Programs 

review, staff are supposed to verify the electronic 831 record against the 

information in the case file and note errors.16 If a significant error is found, 

staff are directed to fax a corrected 831 form to a central location so that 

the correction can be made in the 831 file. However, SSA officials told us 

that corrections don’t always end up in the 831 file, and the agency does 

not track input errors because such errors are low on the priority list. 

Because there is no tracking system that would collect statistical data on 

the number or type of errors, SSA is unable to determine the level of 

accuracy for this electronic data. A tracking system that provided 

feedback to the field would also send a message to staff that the items 

being tracked are important—a basic element in any quality assurance 

system. Without the statistical data and feedback to staff, the reviews for 

data quality affect only those records reviewed and do not affect the 

possible quality of the rest of the records. Finally, the back-end review 

covers only 831 records, not CCS records. 

SSA has not yet made concrete plans for evaluating the types of 

information it currently collects and whether other types of information 

would improve program management and oversight. Users of 

administrative data that we interviewed indicated that they have not been 

surveyed as to their data needs, and some indicated that their requests for 

additional data collection have not yet been implemented. However, the 

agency may take such steps after other priorities are accomplished. An 

SSA official stated that SSA’s first priority was to implement eDib, in order 

to modernize its core business process of taking and adjudicating claims. 

The official also stated that after eDib is fully implemented, SSA will 

assess whether the information collected is sufficient. 

In more recent interviews, high-level SSA officials validated the 

importance to the agency of reliable and useful data for program 

management purposes, and indicated a strong interest in developing a 

comprehensive information management plan that would allow the agency 

to systematically and regularly assess its information needs and make 

appropriate adjustments to its computer systems. According to agency 

officials, this effort would include a review of best practices in information 

management. However, the ideas expressed were still being formulated, 

and the agency has not yet taken any concrete actions. 

 16Under eDib, such a back-end review would involve matching the 831 or CCS record 

against the electronic images of medical records and other documents related to the 

disability determination. 
Page 17 GAO-05-100R SSA’s Disability Programs 

SSA lacks a comprehensive strategic plan for collecting useful and reliable 

data for effective oversight and program planning. Despite the fact that the 

reliability of its data has been an ongoing issue, the agency currently has 

no specific plans to implement additional front-end data entry controls or 

for a tracking and feedback system for the back-end verification of the 

electronic records. In addition, the agency has no specific plans for 

evaluating whether the types of information it currently collects continue 

to be useful for program oversight and whether other types of information 

would contribute to oversight. 

While stretched resources and other priorities may have prevented SSA 

from addressing these issues to date, SSA plans to make more changes to 

its disability processes, which will create additional opportunities and a 

sense of urgency for improving the usefulness and reliability of its 

electronic disability data. For example, in September 2003, SSA’s 

Commissioner proposed significant changes to its disability decisionmaking process—such as the proposed use of centralized medical 

expertise and full documentation of initial-level decisions—that will, if 

implemented, translate into new requirements for collecting administrative 

data to monitor and assess these changes. Moreover, as SSA continues to 

transition from a paper-based case-processing system to an electronic one, 

SSA will have additional opportunities to institute changes that would 

improve upon the usefulness and reliability of the data that it routinely 

collects. Finally, SSA has been considering fundamental changes to its 

quality assurance processes, at both the front and back ends of its 

disability determination process. Combined, these changes, each 

significant and interrelated, create a compelling case for SSA to broadly 

assess and improve the nature and reliability of the data it uses to manage 

its disability programs. 

To improve the value of SSA’s electronic administrative data for managing 

its disability programs, we recommend that the Commissioner of Social 

Security develop a comprehensive strategic plan to ensure the reliability 

and usefulness of the data the agency collects. In doing so, the agency 

should take the following steps: 

1. Establish a cost-effective internal control strategy for ensuring the 

reliability of data in the electronic disability records that would include 

both front-end controls on data entry and a tracking and feedback 

system for back-end verification of the electronic records. 

Conclusion 

Recommendations for 

Executive Action 
Page 18 GAO-05-100R SSA’s Disability Programs 

2. Take steps to review the usefulness of the types of information 

collected and consider whether additional types of information could 

improve program oversight. This effort could include a survey of users 

of electronic disability data. 

We provided a draft of this report to SSA for comment. SSA generally 

agreed with our conclusions and recommendations, acknowledging that 

the report made valid points about steps SSA can take to increase the 

usefulness of electronic data for program oversight and that the agency 

would consider how best to incorporate improvements in its data 

collection activities. 

SSA agreed in theory with our first recommendation and noted that the 

agency is continuing to examine future software enhancements to improve 

front-end controls on data entry and back-end review. SSA also noted that 

its back-end review of cases may provide some check on data entry. We 

agree and modified our report and first recommendation to further clarify 

this. However, SSA expressed concern regarding the cost and technical 

feasibility of implementing this recommendation. We agree that the agency 

needs to balance enhanced internal controls with operational costs and 

productivity, but we believe some of the specific concerns raised by SSA 

are unwarranted. For example, as discussed in our response to SSA’s 

comment in appendix I, we believe that the internal controls proposed by 

GAO would not require additional “structured” data or a data collection 

system “robust enough to catalog and aggregate the information.” 

In its response to our second recommendation, SSA agreed that the 

disability program would be well served by reviewing all the information 

that SSA could potentially collect on a disability claim and determining 

which of these data elements might be beneficial for program oversight 

and policy development. SSA also stated that the agency is drafting a 

statement of work to contract for an assessment of data needs for 

managing the disability programs, which will identify possible sources and 

methods for obtaining data and make recommendations for improving 

existing data and data retrieval. The agency further observed that it would 

have to consider the potential adverse effect of increased electronic data 

collection on productivity and timeliness. 

SSA provided additional general comments, which we have included 

(along with our responses to them) in appendix I and addressed in the 

body of our report where appropriate. SSA also provided technical 

comments that we have incorporated in the report as appropriate. 

Agency Comments 

and Our Evaluation 
Page 19 GAO-05-100R SSA’s Disability Programs 

We are sending copies of this report to the appropriate congressional 

committees and other interested parties. We will also make copies 

available to others on request. In addition, the report will be available at 

no charge on GAO’s Web site at http://www.gao.gov. 

If you or your staff have any questions concerning this report, please 

contact me or Michele Grgich, Assistant Director, at (202) 512-7215. You 

may also reach us by e-mail at robertsonr@gao.gov or grgichm@gao.gov. 

Other major contributors to this assignment were Ann T. Walker, Jill D. 

Yost, and Corinna Nicolaou. 

Sincerely yours, 

Robert E. Robertson 

Director, Education, Workforce, 

 and Income Security Issues 
Appendix I: Comments from the Social 

Security Administration 

Page 20 GAO-05-100R SSA’s Disability Programs 

Appendix I: Comments from the Social 

Security Administration 

Note: GAO comments 

supplementing those in 

the report text appear at 

the end of this appendix. 
Appendix I: Comments from the Social 

Security Administration 

Page 21 GAO-05-100R SSA’s Disability Programs 

See comment 3. 

See comment 1. 

See comment 2. 
Appendix I: Comments from the Social 

Security Administration 

Page 22 GAO-05-100R SSA’s Disability Programs 

See comment 6. 

See comments 1, 2, 6 

and 7. 

See comment 5. 

See comment 4. 
Appendix I: Comments from the Social 

Security Administration 

Page 23 GAO-05-100R SSA’s Disability Programs 

See comment 6. 

See comment 7. 
Appendix I: Comments from the Social 

Security Administration 

Page 24 GAO-05-100R SSA’s Disability Programs 

See comments 1 and 2. 
Appendix I: Comments from the Social 

Security Administration 

Page 25 GAO-05-100R SSA’s Disability Programs 

1. We acknowledge that the Disability Quality Branches’ (DQB) back-end 

review of cases may provide some check on data entry, and we have 

adjusted our recommendation and text in our report to clarify this. 

However, SSA officials told us that tracking of input errors is low on 

DQB’s priority list, and DQB changes don’t always end up in the 831 

file. Further, as SSA indicated, it does not track input errors for 

statistical purposes; therefore SSA does not know the level of data 

accuracy. A tracking system that also provides feedback to the field 

would also send a message to staff that the items being tracked are 

important—a basic element in any quality assurance system. Without 

the statistical data and feedback to staff, the reviews for data quality 

affect only those records reviewed, and do not affect the possible 

quality of the rest of the records. Finally, the back-end review by DQB 

covers only 831 records (i.e., DDS decisions), not CCS records (i.e., 

ALJ decisions). 

2. Our report recognizes that as part of the implementation of eDib, SSA 

is making software enhancements that may address some data entry 

problems. However, we believe that the eDib enhancements fall short 

of an effective overall strategy. Our previous comment explains why 

we believe SSA’s back-end reviews do not sufficiently ensure that 

electronic data match the information in the case file. 

3. We amended the report to include information about the work group. 

4. We have incorporated into our report the additional sources of 

information used in the CDR targeting process. 

5. We recognize that the agency has improved data capture policies and 

practices over the years, but in our study, we examined electronic 

disability data for 2003, the most recent year available, and found 

problems with that data. In addition, without an effective system for 

verifying the electronic record against the case file and a tracking 

system for errors, neither SSA nor we can determine the extent to 

which the electronic disability data are accurate. Further, 

PricewaterhouseCoopers’ projection that SSA would incur hundreds of 

millions of dollars in CDR administrative and program costs between 

2001 and 2005 because of poor and missing electronic disability data 

was made despite its acknowledgment that SSA’s electronic disability 

data had improved. In order to not overstate PwC’s findings, we 

adjusted the text of our report to say that the agency would incur 

“additional administrative and program costs” because of problems 

with the quality of its electronic disability data. 

GAO Comments 
Appendix I: Comments from the Social 

Security Administration 

Page 26 GAO-05-100R SSA’s Disability Programs 

6. We do not agree that additional “structured” data are necessary before 

data quality can improve. The 831 record is an electronic version of the 

SSA-831 form, in a structured data format. For cases processed prior to 

the use of the electronic case folder, a printout of the completed SSA831 form was placed in the paper case file to document the final 

decision. Under eDib, which uses an electronic case folder, an 

electronic image (“tiff” image) of the SSA-831 form is created and 

placed in the electronic case folder (along with other electronic images 

such as medical records) to document the final decision. Regardless of 

this 831 image, the original 831 record still exists as structured data 

and becomes part of a national database known as the “831 file.” SSA 

is still able to aggregate and report on these data. Therefore, it is 

possible for SSA to implement our recommendation that it compare 

the information in the electronic 831 record against the information in 

the case file or the electronic case folder, and correct the electronic 

831 record as needed. Even after the agency eliminates paper source 

documents and relies totally on electronic documents in the case 

folder, SSA will still be able to compare the electronic 831 record 

against the electronic images of medical records and other documents 

related to the disability determination that are in the case folder. 

7. We agree that it may be difficult to determine the primary impairment. 

However, our concern is that once a disability examiner makes the 

decision, the 831 record should accurately reflect the disability 

decision. For example, if the primary impairment is determined to be 

diabetes, then the 831 record should contain the four-digit impairment 

code for diabetes and the appropriate body system code for that 

primary impairment. SSA could develop data entry controls that would 

allow only acceptable impairment codes and the corresponding body 

system code. The agency would have to update those listings only 

when changes in policy dictated changes in impairment codes or body 

system codes. This type of front-end data entry control should not 

require a “data collection system robust enough to catalog and 

aggregate the information.” 

(130348)