Modernizing SSA Disability Programs: Progress Made, but Key Efforts Warrant More Management Focus
Released: 2012-07-19T00:00:00

Letter

Background

SSA administers two disability programs: the Social Security Disability Insurance (DI) program, enacted in 1956, and the Supplemental Security Income (SSI) program, enacted in 1972. In order to be eligible for DI or SSI benefits based on a disability, an individual must meet the definition of disability for these programs—that is, they must have a medically determinable physical or mental impairment that (1) prevents the individual from engaging in any substantial gainful activity and (2) has lasted or is expected to last at least 1 year or result in death.

To determine eligibility for both programs, SSA uses a five-step sequential process that is intended, in part, to expedite disability decisions when possible and limit administrative costs by conducting less intensive assessments at earlier steps (see fig. 1).

At steps 1 and 2 of the process, SSA determines whether an applicant is working and meets income thresholds, as well as the medical severity of impairments. If not working (or not meeting income thresholds) and with the determination of a severe impairment, the applicant moves to step 3 of the process.

At step 3, SSA examiners assess the applicant’s medical impairments against the Listings of Impairments, also known as the medical listings, which are organized into 14 major body systems for adults and reflect medical conditions that have been determined by the agency to be severe enough to qualify an applicant for benefits. When using the listings to determine eligibility, SSA generally relies on information on the applicant’s diagnoses, including laboratory findings, diagnostic tests, and symptoms, as well as some limited consideration of the applicant’s functional limitations. If the individual’s impairment meets or is equal in severity to one or more of those in the listings, the individual is determined to have a disability at step 3.

If a disability determination is not made at step 3, SSA performs an assessment of the individual’s physical and mental residual functional capacity. Based on this assessment, SSA determines whether the individual is able to perform past relevant work (step 4) or any work that is performed in the national economy (step 5). To inform determinations at steps 4 and 5, SSA uses a Department of Labor database—known as the Dictionary of Occupational Titles (DOT), which is an inventory of occupations performed in the national economy. At step 5, SSA also uses a set of rules and guidelines, referred to as the grid rules, to evaluate the combined effect of an individual’s physical residual functional capacity, age, education, and work experience.

While originally created for expediency, over time the medical listings used at step 3 were relied on less to make program determinations as they became increasingly outdated. In the early years of the program, more than 90 percent of cases were decided based on medical conditions specified in the listings; in 2010 only 47 percent of allowances were made at step 3. Experts attribute the decline in allowances based on the medical listings to changes in the program, workplace, and medical treatment that the medical listings had not kept pace with. Since the 1990s, we, along with SSA’s Office of Inspector General and the Social Security Advisory Board, have expressed concerns that the medical listings being used no longer provide current and relevant criteria to evaluate disability applicants’ inability to work. In 2003, we deemed SSA’s and other federal disability programs as high-risk areas, in part, because their programs continue to emphasize medical conditions in assessing work capacity, without adequate consideration of work opportunities afforded by advances in medicine, technology, and changes in the labor market.

In 2008, we reported that SSA had recently established a new process— referred to by SSA as the “business process”—for revising the listings to better incorporate feedback into its continuous updates. This process, which has been in effect since 2003, incorporates feedback from multiple parties, including medical experts and claims examiners, to update the medical criteria. Under this process, SSA gathers external feedback from comments associated with regulatory actions, such as the publication of advanced notices of proposed rulemaking (advanced notices) and notices of proposed rulemaking (notices) in the Federal Register.

In addition, this process includes conducting postimplementation reviews one year after a revision is made to assess the impacts of a revised listing, areas to improve, and whether expectations from the revisions have been achieved.

With respect to information on jobs in the national economy that supports SSA’s occupational criteria, we and others have reported that the DOT is outdated, although SSA still relies on it to assess eligibility at steps 4 and and 5 of the process. The DOT has not had a major update since 1977 the Department of Labor (Labor) replaced it with a new database in 1998 called the Occupational Information Network (O*NET). However, SSA determined that O*NET is not sufficiently detailed for evaluating DI and SSI disability claims, so SSA has begun developing its own occupational information system to better reflect the physical and mental demands of work in the national economy.

Beyond dated medical criteria and vocational information, numerous disability experts have expressed concern that SSA’s disability programs—which statutorily require that an adult’s medical condition prevents that person from engaging in substantial gainful activity—have historically tended to equate the severity of medical conditions with inability to work and thus are out of sync with modern concepts of disability. Modern concepts focus on an individual’s functional abilities in the workplace environment, including consideration of the presence or lack of assistance, for example, per the requirements for reasonable accommodation by the Americans with Disabilities Act of 1990. These modern views are reflected in the International Classification of Functioning, Disability and Health (ICF), which is the World Health Organization’s framework for assessing health and disability. This framework takes into account the interaction of health conditions and contextual factors, such as products and technology, attitudes, and services, on an individual’s functional capacity, rather than viewing disability solely as a medical or biological issue. Along these lines, experts have recommended that SSA incorporate more consideration of individual function in its medical listings when doing so can improve their use as a screening tool for determining inability to work. Further, several groups, such as the Social Security Advisory Board and the Urban Institute, have reported that SSA’s disability programs should focus more on whether an individual can work given appropriate environmental or other supports, and that SSA—through its demonstration authority or other means—could play a role in determining how. These suggestions are consistent with SSA’s long-term targeted outcomes, which include not only regularly updating regulations and policies to incorporate the most recent medical advances, but also making it easier for individuals with disabilities to return to work.

SSA Has Improved Its Process for Updating Medical Listings, but Still Faces Challenges with Timely Updates

SSA Has Improved Its Strategy to Update Medical Listings

Since our last review in 2008, SSA has made several changes that hold promise for improving its medical listings updates. First, the agency is using a two-tiered system for ongoing revisions to the listings. Under this system, SSA first completes a comprehensive listings update for a body system, which entails reviewing all the diseases and disorders listed within that system and making necessary revisions. Second, following a comprehensive revision, SSA will pursue a more targeted approach—that is, SSA will conduct ongoing reviews and updates of a smaller number of medical diseases or disorders within that body system. Agency officials told us that targeted updates should be completed more quickly than comprehensive updates, allowing them to focus on the most critical changes needed. As of early March 2012, SSA had begun the ongoing review process to identify opportunities for targeted revisions for 8 of the 14 adult body systems that were recently comprehensively revised.

Another change, according to agency officials, is that in 2010, the SSA Commissioner set a 5-year cycle for updating listings for each body system. listings under each body system, ranging from 3 to 8 years, but frequently extended them. SSA officials believe that conducting targeted reviews will generally allow the agency to conclude any necessary revisions prior to the 5-year period. Additionally, they expect that using the “business process,” which requires early public notification of changes and obtaining necessary data and feedback from internal and external parties, should help keep continuous reviews on track. See figure 2 for the status and expiration date by which the listings should be reviewed and updated, if needed, for the 14 body systems undergoing review for either comprehensive or possible targeted revisions, as of early March 2012.

The 5-year period will be applied to listings under a body system upon completion of their current revision. SSA began applying the 5-year period in 2011 when it comprehensively updated the endocrine body system listings. For other body system listings updated prior to 2011, SSA generally assigned periods extending beyond 5 years.

For the two-tiered revision process, the beginning point is the date of the last comprehensive or targeted revision.

SSA officials told us they have grouped the disorders in the special senses and speech listings into two broad sections—vision and hearing—and two standalone medical conditions. SSA has not yet comprehensively revised the two standalone medical conditions that cover speech and disturbances of labyrinthine-vestibular function. The two sections have undergone comprehensive revisions and are undergoing reviews that began in 2007 for the vision section and in 2010 for the hearing section. Special senses and speech listings are set to expire in 2015.

According to SSA officials, SSA published limited revisions for the hematological disorders in 1988. It also published final rules for limited revisions for the mental disorders listings in 2000 and the musculoskeletal system listings in 2001.

SSA has made another change by more extensively engaging the medical community to identify ways to improve the medical listings. For example, SSA contracted with the Institute of Medicine to study its medical criteria for determining disability and to make recommendations for improving the timeliness and accuracy of its disability decisions, resulting in a 2007 report with recommendations and a symposium of experts in 2010. SSA has addressed some of the Institute of Medicine’s recommendations, such as making better use of its administrative data to update criteria and creating a standing committee through the Institute of Medicine to provide recommendations for listings revisions. In addition, through the Institute of Medicine, SSA created consensus committees to conduct research and provide other assistance with updating SSA’s cardiovascular listings and its Human Immunodeficiency Virus (HIV) listing, which culminated in two reports with recommendations.

SSA Has Experienced Delays with Its Revision Process

SSA has made progress, but continues to face delays in completing both comprehensive and ongoing targeted updates. As of early March 2012, SSA officials told us they had completed comprehensive revisions of listings for eight body systems, some of which resulted in significant changes. For example, in 2011, SSA removed the endocrine body system listings for all the adult disorders, such as diabetes, because they found that they were now generally diagnosed early and treated very successfully. Nevertheless, according to SSA, the agency still needs to complete the comprehensive revisions for listings of the remaining six body systems—a process that has been ongoing for the last 19 to 33 years, with numerous extensions beyond the original expiration periods (see table 1). For example, it has been at least 27 years since SSA finalized comprehensive revisions for two of the six body system listings—mental and neurological disorders, which are among those SSA uses most frequently in its eligibility determination process. SSA has made progress on four of the six body systems set to expire in 2012, but will likely miss targeted time frames. SSA issued a notice of proposed rulemaking to revise its listings of mental disorders in 2010 and has told us that it plans to finalize this comprehensive revision by the end of 2012, after its current expiration date in July 2012. SSA is still developing notices for the neurological and two other body system listings, which are also set to expire in 2012. Because SSA has generally taken more than a year to move from publishing the notice to final revision and may also need additional time for any internal revision and review, as well as a review by the Office of Management and Budget, it is also unlikely that SSA will meet its planned 2012 time frames for updating these listings. See appendix II for details on progress updating these listings.

Updates using SSA’s targeted approach seem to be moving at a faster pace than the comprehensive revisions, but some delays have occurred here as well, and more are expected. For example, SSA has already extended the expiration date for its targeted review of the cardiovascular system. According to SSA officials, it may also need to extend expiration dates for listings under two other body systems set to expire in 2012. Extensions may be needed because SSA has not yet published the notice of proposed rulemaking for those two body system listings and may not have time to publish these notifications, respond to comments, and complete the final updates by their current expiration date.

At the same time that SSA has been experiencing delays completing timely revisions, agency officials reported challenges with other steps in the business process. Whereas SSA’s business process includes a postimplementation review of case data at the 1-year mark to determine whether expectations from a revision were met, several did not undergo this review, such as those for skin disorders, genitourinary impairments, and impairments affecting multiple body systems. SSA officials told us they only began conducting these reviews in 2010, and to date, SSA has completed just one in 2011 that involved a targeted sample of 175 cases. SSA officials told us they are conducting or planning to conduct two more reviews at the 1-year mark in 2012. While disability experts we interviewed spoke highly of SSA’s business and targeted review processes to obtain feedback early on and update the listings more promptly, ongoing delays raise questions about the agency’s ability to fully follow its current business process while completing continuous and timely revisions for all 14 body systems.

SSA officials offered two key reasons for the delays in updating the listings: (1) limitations in the number and expertise of staff and (2) the complexity and unpredictability of the regulatory process. According to SSA officials, revising the medical listings requires research, deliberation, testing, regulatory review, and consensus with many stakeholders, and consequently is difficult and time-consuming to achieve. According to an SSA official in the Office of Medical Listings Improvement, the office is short-staffed and there is a lack of expertise needed to perform this work. To address these constraints, SSA has contracted with the Institute of Medicine to review and develop recommendations for revising two of the body system listings. While not finalized as of March 2012, SSA officials reported that the agency plans to renew this contract after it expires in 2012 and extend it to 2015. However, SSA has not yet determined how it will use the Institute of Medicine to revise the listings or the extent to which the contract will address staffing shortfalls.

Also contributing to delays is the time required for internal review and public comment under the regulatory process, which depends largely on the number and the substance of comments received, according to an SSA official. Obtaining public comment is one way SSA receives critical information for identifying areas for revision and obtaining stakeholder consensus. As such, SSA reviews and responds to each comment that would result in a significant change, and the time required for doing so varies depending on the number of comments and resources. For example, SSA officials told us that the advanced notice of proposed rulemaking to the listings of mental disorders in 2003 resulted in 500 comments, which took SSA 5 years to incorporate into a draft notice of proposed changes for regulatory review. SSA officials reported they have considered options for automating and thereby speeding up the process of obtaining and reviewing public comment. For example, they told us that they eventually plan to use a web-based tool to obtain early public feedback on medical listing updates to help target their limited resources and more quickly make changes. According to an SSA official, the agency plans to retain its optional use of the advanced notice of proposed rulemaking because it can also help to identify appropriate areas to focus on to ultimately make timely updates.

SSA Has Begun an Ambitious Project to Develop Updated Occupational Information, but Many Uncertainties Remain

SSA is Designing a New Occupational Information System, but Has Fallen behind with Some Activities

In 2008, SSA began a multiyear project to research and design a new source of occupational information that will replace the outdated information currently being used to determine if claimants are able to do their past work or any other work in the national economy. Since the 1960s, SSA has been using the DOT, which contains a list of job titles found in the national economy and had a last major update in 1977. The DOT provides SSA with descriptions of the physical demands of work— such as climbing, balancing, and environmental requirements—for each of the more than 12,000 occupations listed. According to SSA, these descriptions have been essential to its evaluations of how much a claimant can do despite his or her impairment and whether this level of functioning enables the claimant to do his or her past work or any other work. After its last limited update, Labor decided to replace the DOT with O*NET, which has far fewer occupational titles compared with the DOT and has served Labor’s purposes more efficiently. According to an SSA report, after investigating potential alternatives, SSA decided that O*NET and other existing databases with occupational information were not sufficiently detailed and not able to withstand legal challenges for use in its decision-making process. SSA further decided to develop its own occupational information system (OIS), which would contain detailed information as in the DOT, but would also include additional information, such as the mental demands of work. In addition, SSA has determined that the OIS will (1) meet its legal, program, and data requirements; (2) be flexible enough to incorporate changes in its policies and processes; and (3) be able to be updated to reflect the evolving workplace environment.

In 2008, SSA began taking steps to guide the development of its OIS. SSA created an internal office and working group, as well as an Occupational Information Development Advisory Panel. While the number of panel members has fluctuated over time, in April 2012, 14 external experts were serving on the panel and represented various affiliations, such as medicine, disability law, rehabilitation, and industrial organizational issues. The advisory panel holds quarterly public meetings and has several subcommittees that review material and make recommendations to SSA on developing various components of the OIS. For example, in a 2009 report, the advisory panel supported the need for SSA to develop a new source of occupational information, rather than adapt O*NET, and recommended the type of data SSA should collect, as well as approaches for classifying occupations.

The Occupational Employment Statistics program produces employment and wage estimates for approximately 800 occupations. The U.S. Census Bureau’s household surveys include (1) the American Community Survey, which is an ongoing survey that provides annual data on demographics such as age, education, and disabilities, and (2) the Current Population Survey, which is primarily a labor force survey, conducted every month by the U.S. Census Bureau for the Bureau of Labor Statistics and provides data such as the national employment rate. explore specific subject areas, such as sampling issues. Besides working with Labor and U.S. Census Bureau officials, SSA officials and panel members have sought input from other experts and current users of the DOT, such as SSA disability adjudicators and external rehabilitation professionals. SSA officials conducted a user needs analysis in 2009 and have presented on the OIS project at events and conferences.

In July 2011, SSA published a detailed research and development plan outlining all activities related to researching, developing, and testing the key components of the OIS in order to implement it by 2016 at an estimated total cost of $108 million. For example, the plan includes several baseline activities to identify and study other occupational information systems and various approaches for analyzing occupations that may inform or could be leveraged in SSA’s OIS data collection. The plan also includes activities to identify the primary occupational, functional, and vocational characteristics of current beneficiaries. Other key components of the plan include developing descriptions of work requirements, such as the physical and mental demands for jobs; developing data collection and analysis strategies; and identifying the occupations, categories, and definitions that will constitute the structure of the new OIS. SSA also plans to develop a strategy for piloting how it would ultimately collect data for the OIS nationwide within this time frame.

As of May 2012, SSA had made progress on many of the baseline activities outlined in its research and development plan for the OIS. For example, according to an SSA official, its investigation of existing occupational information systems, now complete, resulted in useful information about design issues other organizations have confronted and mitigated when creating their own system. Additionally, SSA’s preliminary analysis of its own administrative data identified the most frequently cited occupations and functional and vocational characteristics of disability applicants. SSA officials told us the agency will target the occupations identified in this analysis for its pilot studies of the OIS. Also in 2011, SSA completed a comprehensive framework for assessing an individual’s capacity to work—key to informing the OIS content, according to SSA officials—which was based on recommendations of outside experts as well as SSA’s policy and program requirements.

While SSA has made progress on several key activities, agency officials delayed 2011 completion dates for certain activities and anticipate making additional changes to its timeline as a result of not meeting its staffing goals for fiscal year 2011 (see table 2). Some activities that were delayed by several months included finalizing reports for the baseline studies and conducting a literature review that would inform how occupations might be analyzed. SSA officials told us that they did not have enough staffing to complete all of the 2012 planned activities within the estimated schedule. Further, SSA officials said they did not have the budget to hire new staff in September 2011. To address this challenge, SSA officials hired consultants to meet some of their needs. SSA officials also met with the Office of Personnel Management to explore the possibility of an interagency agreement that would allow SSA to use one or two of the office’s industrial organizational psychologists to help on a part-time basis.

Our Cost Estimating and Assessment Guide identifies a number of best practices for effective cost estimating and scheduling that should result in reliable and valid cost estimates that management can use for making informed decisions. Per these criteria, the success of any program depends in part on having reliable cost estimates and a reliable schedule. A reliable cost estimate provides the basis for informed investment decision making, realistic budget formulation and program resourcing, meaningful progress measurement, proactive course correction when warranted, and accountability for results. A reliable schedule defines, among other things, when work activities will occur, how long they will take, and how they are related to one another. As such, the schedule not only provides a road map for systematic execution of a program, but also provides a means by which to gauge progress, identify and address potential problems, and promote accountability.

We compared SSA’s cost estimate to three best practices for assuring a reliable cost estimate and found SSA only minimally or partially met each of these (see table 3). First, we found SSA’s estimated cost of the OIS research and development phase—$108 million—was not well- documented. For example, after reviewing all of SSA’s documentation supporting this cost estimate as of December 2011, we did not find step by step documentation showing how the estimate was derived so that someone unfamiliar with the project could use the documentation to recreate the estimate and get the same results. Well-documented cost estimates are considered a best practice and without good documentation, SSA is not in a position to defend the reliability of its estimate. Additionally, SSA only partially met the best practice of having an accurate cost estimate. While SSA provided documentation showing how some of their estimates compared to actual costs and the reasons for any variances, SSA did not provide adequate documentation for us to determine, for example, if the estimate took inflation into account or if there were any calculation errors. Generally speaking, in the absence of a detailed cost model, third party reviewers cannot be certain that cost estimate calculations are accurate and account for all costs.

We also found the cost estimate was not comprehensive as it does not include any costs beyond the research and development phase, such as the costs of producing, maintaining, and updating the final data system, which could be significant. SSA officials told us that they are still in the process of determining what information will be included in the data system, how it will be collected, and how many occupations will be covered, each of which will influence the cost of developing and maintaining the OIS. As such, they maintain that it is too early to estimate the costs of the final system or the costs for maintaining the system. According to industry best practices, cost estimates should be comprehensive and include all costs necessary to achieve agency objectives and should be updated as the agency proceeds with the project and gains more information. At this point in the project, estimating total costs of options under consideration could enhance decision making by evaluating the potential tradeoffs of different designs. Without any estimate for the cost of producing the OIS, SSA risks designing a system that would not be a viable or affordable option to complete. Additionally, without maintenance cost estimates, SSA is at risk of designing a system that would be too costly to maintain on a regular basis resulting in outdated information. Other federal information systems provide some basis for estimating the cost of producing or maintaining an OIS. For example, Labor officials indicated that the cost of maintaining O*NET—an occupational information system that includes approximately 1,000 occupations and uses a paper or web survey to collect data—is roughly $6 million a year.

We also reviewed SSA’s master schedule for the OIS against nine best practices for a reliable schedule, and found that SSA did not meet four and minimally met five of these practices (see table 4). For example, among those minimally met, we found a significant number of activities within SSA’s schedule were not logically sequenced in the order that they were going to be carried out (best practice 2) and activities that were dependent on completion of a prior activity were not identified (best practice 5, 6, and 7). For example, SSA officials and experts acknowledged that activities associated with the “OIS work taxonomy” part of the research and development plan—activities that together determine which information should be included in the OIS—are an essential building block that will inform other OIS activities, such as developing the “OIS work analysis instrument,” which will determine how OIS information will ultimately be collected.

Nevertheless, when we tested SSA’s schedule for sequencing and linkages associated with the OIS work taxonomy activity, we found a significant delay of almost 1,000 days that should have significantly delayed the project actually barely affected the final completion date in the OIS project. Such missing links between key activities in the schedule represent broken logic that reduces the reliability of the forecasted dates. The OIS schedule also did not meet the best practice of conducting a schedule risk analysis (best practice 8), which is an essential tool for project managers to understand the most important risks to the project and focus on mitigating them. Based on the schedule provided to us, we do not have any indication that SSA has considered the vulnerability within its schedule for meeting time frames for individual activities which would in turn impact the time frames for the entire project. Without explicitly identifying risks to its schedule, SSA either does not know or is not conveying the probability of completing research and development activities on time, and its relationship to overall costs for this phase. However, in order to conduct a schedule risk analysis the schedule has to be properly sequenced and networked with all logic links in place for the analysis to produce credible results.

Beyond gaps in their cost estimate and schedule of specific activities, SSA faces broader challenges that could impede the success of the OIS design and implementation; however, SSA has not done a formal risk analysis of these challenges. While SSA officials said they had recently begun discussing various risks and how they might address them, they said it was premature to provide us with this information. Examples of potential challenges for OIS that experts and stakeholders we spoke with cited include:

SSA’s lack of expertise with designing an OIS. SSA does not have prior experience with designing a complex and, in some respects, unprecedented occupational information system. As such, SSA will need to depend on many outside experts and contractors to complete the system. While the Occupational Information Development Advisory Panel members can provide some expert counsel, most of its members lack the technical and scientific background essential to informing this complicated effort. To help design the OIS, SSA recently hired a research psychologist and plans to hire two more individuals in the near future, but may need additional resources. In March 2012, SSA put out a request for information and, according to an agency official, received ideas from knowledgeable experts about how to move forward with technical aspects of the project. Until the agency secures required and sufficient expertise, SSA may not be able to move ahead with key technical decisions, such as defining the number of occupations to include and the data collection methodology, which may cause further delays.

Cost of maintaining an OIS. SSA has not yet made design decisions on the OIS that will ultimately define the overall cost of producing, implementing, and maintaining the OIS system. For example, SSA officials told us that they were not sure how many occupations they would include in the OIS, although they believe the total number will be somewhere between the number of occupations in the O*NET and the DOT, or between approximately 1,000 and 12,000 occupations. A 2010 study conducted by the National Academy of Sciences reviewing Labor’s O*NET cited the three major cost drivers of an occupational information system as (1) size of sample, (2) number of occupations, and (3) frequency of updates. This study noted that as the agency increases any one of these cost drivers, it does so at the expense of the others or of the overall cost. For example, if SSA chooses to frequently update the entire OIS, the agency may need to make tradeoffs with the number of occupations or sample size it chooses if it wishes to contain costs.

Managing large, multiyear projects. In the past, SSA has experienced difficulties managing complex, multiyear efforts. In prior work, we reported that SSA has cancelled numerous demonstration projects due to limitations or weaknesses in design or implementation and lacked sufficient controls to ensure effective management of its demonstration projects. We have also reported on challenges that SSA faced managing two separate redesigns of its disability decision- making process. While the OIS project alone represents a complex effort, stakeholders have emphasized the importance of this project progressing in close concert with other ongoing research managed by another part of SSA, placing greater emphasis on broad and effective project planning. In light of past experiences and as SSA faces a potential change in leadership in 2013, the management shortcomings we identified increase the vulnerability of this complex project.

Given the range of challenges and potential risks they pose to the success of the OIS project, it is important to identify and carefully explore feasible alternatives that may mitigate these risks. Examples of potential alternatives suggested by experts and other observers in the disability field include:

Leveraging O*NET. In a report issued in 2010, SSA’s advisory panel found that it did not believe that O*NET would meet SSA’s needs, in part, because it is not detailed enough. However, the National Academy of Sciences also issued a report in 2010 and concluded that O*NET could be altered to better meet SSA’s needs for disability adjudication. SSA officials have told us that modifying O*NET would not result in savings for the agency, but they have not conducted any analysis to determine this. While most disability experts agree that O*NET in its current form would not be a suitable source of occupational information for SSA, the National Academy of Sciences study noted that there are potential linkages between O*NET and an SSA OIS that are worthy of more careful exploration on SSA’s part and that leveraging one system to serve broader purposes could be most cost-effective overall.

Adjusting the scope of the OIS. SSA could reduce costs by limiting the scope of the OIS. For example, the OIS could be designed to capture fewer occupations. SSA officials have told us that since they are early in their design, they do not know how many occupations will be included in their OIS. Since this decision is fundamental to informing the OIS data collection methods, feasibility, and cost, consideration of alternative scope should be given high priority attention.

Limiting data collection methods. Another key factor that SSA must consider is the method they will use to collect data for their OIS as some methods are more costly than others. For example, among the methods SSA is considering, conducting on-site job analyses of occupations would be more expensive than surveys, which are also more resource and time intensive than collecting information through telephone interviews.

Leveraging resources from other agencies and OIS users. Other federal agencies with experience in data collection and occupational information have resources that SSA could leverage. SSA officials told us that they have been considering how they could benefit from the infrastructure that other agencies such as Labor and the U.S. Census Bureau have for data collection, but that they have not identified specific ways to leverage those resources. Additionally, while SSA needs to assure the OIS serves its most basic needs, it may be appropriate for SSA to explore the possibility of cost sharing or applying users’ fees with organizations that currently rely on the outdated DOT and could ultimately benefit from using a more modern and comprehensive OIS database.

SSA Has Taken Other Steps to Modernize Criteria but Faces Limitations

SSA Has Taken Steps to Incorporate Modern Concepts of Disability into Eligibility Criteria

Although SSA’s adult disability programs were initially built upon the assumption that certain severe medical conditions equate to work incapacity, through its medical listings updates and ongoing research, SSA has taken steps to modify its eligibility criteria. SSA now is taking a more modern view of disability that looks beyond the claimant’s medical condition by giving greater consideration to his or her functional capacity, that reflects the International Classification of Functioning, Disability and Health (ICF) framework. According to the 2007 Institute of Medicine report, a modern concept of disability should recognize that disability is not just inherent in the individual and his or her medical condition, but is the result of complex interactions between the person, the person’s medical impairments, assistive devices to which they have access, and features of his or her socioeconomic environment, such as the presence or lack of accessible transportation and workplace accommodations. Under this concept, as described in this report, two people with the same impairment might have different degrees of work disability for a variety of reasons.

Updates to the medical listings have been one vehicle through which SSA can include an assessment of an individual’s functional abilities to determine whether an impairment prevents work. For example, as part of SSA’s comprehensive revision to the listings for the immune system, the agency included several functional criteria, such as performing activities of daily living, maintaining social functioning, and completing tasks in a timely manner despite deficiencies in concentration or persistence. Generally, SSA officials, adjudicators, and disability experts we spoke with support incorporating appropriate functional criteria into the medical listings to facilitate a more reliable assessment of an individual’s ability to work. However, some have also noted that as SSA continues to more broadly incorporate functional criteria into listings updates, such efforts may result in a more subjective assessment by adjudicators that could increase the difficulty of making step 3 disability determinations.

Since 2008, SSA has had an ongoing interagency agreement with the National Institutes of Health (NIH) to conduct short- and long-term research that has informed SSA’s efforts to incorporate functional information into the disability criteria. For example, in its 2011 annual report to SSA, NIH presented its findings on the use of functional criteria, as defined by the ICF, in the adult listings. The objective of the project was to quantify the comprehensiveness and consistency of functional criteria among the adult listings and any influence this has in determination of outcomes. NIH found that while the use of functional terms were particularly apparent in five body systems, nearly one-half of the 14 body systems did not capture the influence of health conditions and impairment on human functioning. NIH is currently working on another project, which examines the presence and consistency of functional terms in the listings criteria. It is estimated the project will be completed by 2013.

Besides exploring how medical listings could further incorporate functional considerations, SSA is also sponsoring longer term research through the NIH to develop an automated method to more quickly and comprehensively evaluate a claimant’s functional abilities for its disability determinations. Specifically, project researchers are developing a computer-based tool to rapidly and reliably assess the functional abilities of individual claimants considering their medical conditions. As envisioned, the claimant, a medical provider, or both would respond to a series of questions through the computer-based tool, which would likely take an hour to complete the entire battery, perhaps less if a more limited assessment is warranted. The project’s intended scope encompasses six areas of function consistent with the ICF activity domain: (1) mobility, (2) interpersonal/social interaction and relationships, (3) self-care, (4) communication, (5) general tasks and demands (such as multitasking and carrying out daily routines), and (6) learning and applying knowledge. NIH researchers said they plan to consider the use of common personal assistive devices, such as wheelchairs, in developing the tool. As of April 2012, NIH had completed preliminary testing of instruments in two of the six areas—mobility and interpersonal/social interaction and relationships—with a sample of claimants and providers and had begun analyzing the data. SSA and NIH officials anticipate several benefits from the functional assessment tool, such as providing information on the impact of impairments more consistently, comprehensively, and early in the disability determination process. This information would help adjudicators more accurately assess whether a person can perform certain kinds of work given his or her functional and occupational capabilities.

While this research is promising, there are several unknown variables. SSA officials said they have not yet determined when or how the tool will be integrated into the disability determination process. NIH officials indicated they believe that the tool may be most useful early in the determination process. SSA officials said they expect to pilot the functional assessment tool after all relevant domains have been tested and validated, which will likely be by 2016. Additionally, NIH researchers told us that, ultimately, this tool, which collects information on the applicant, would need to be aligned with the OIS, which collects information on the demands of work. SSA and NIH officials reported they have held meetings to share their progress to date, but both projects are in the early stages and additional coordination will be needed going forward.

SSA is in the preliminary stages of taking additional steps to broaden its use of functional criteria throughout the disability determination process in response to other findings by NIH. Specifically, NIH evaluated a number of forms that SSA uses during the disability application and determination processes to determine how well these forms captured information on claimant functional activity as described by the ICF. The researchers found that current SSA assessment processes had major gaps in covering the ICF concepts. For example, SSA captured only limited information relative to sensory experiences, such as watching or listening, learning and applying knowledge, communication, and interpersonal interactions and relationships, all of which NIH considered particularly relevant to work. NIH concluded that the gaps need to be addressed in order to characterize individual functioning more comprehensively in relationship to the demands of the workplace. Another step the agency reports that it plans to take is the issuance of a Federal Notice of Solicitation of Collaboration from federal agencies in developing a standard for coding functional capacity in federal disability programs based on the ICF. The notice will explain that SSA believes using the ICF would help standardize how agencies describe and measure different aspects of disability, improve the clarity and comparability of research findings, and strengthen the base of scientific knowledge that guides public policies and health practices. In addition, SSA has asked the Institute of Medicine to plan an international symposium focused on how best to use and assess function in the disability determination process. SSA officials also stated that the agency may ultimately revise the residual functional capacity forms based on criteria from the ICF.

SSA Faces Constraints to Incorporating Modern Concepts

Consistent with modern views of disability, a key consideration in assessing disability is that the environment can hinder or enhance an individual’s ability to function. While assistive devices and workplace accommodations can play a critical role in an individual’s ability to function in the work environment, SSA does not always consider them in its assessment of disability. Regarding the incorporation of assistive devices into the medical listings, SSA officials and experts we spoke with expressed concern about the extent to which specific technologies should be incorporated if they are not widely available. SSA officials told us that they currently incorporate assistive devices into the medical listings once these devices become standard in the medical community—a threshold that SSA officials described as generally involving some combination of availability, accessibility, and insurance coverage. After an assistive device, such as a prosthetic device for walking, is incorporated into a listing, adjudicators must evaluate the individual’s ability to walk with the device being used. For example, evaluations of people who have had amputations involving a lower extremity or extremities are to be done with the prosthetic device in place. When we asked if wheelchairs are considered standard in the medical community, and whether SSA considers how individuals with wheelchairs might function in today’s knowledge-based labor market, given their age and education, we heard conflicting information from SSA officials. Some explained that individuals in wheelchairs are generally allowed at step 3 on the basis of their underlying medical condition, without evaluating how their disability might be assessed at steps 4 or 5. Other officials maintained that according to SSA’s policy, an individual in a wheelchair would be evaluated for upper body strength or other medical issues. However, SSA officials we spoke with said they have not evaluated these types of allowances specifically.

Regarding workplace accommodations, SSA officials said their policy is to not consider them for several reasons. First, officials cited SSA’s inability to ensure that workplace accommodations are provided by employers—a concern shared by other disability experts we interviewed. SSA officials also indicated the agency would be unable to assess the effectiveness of workplace accommodations for claimants. Further, officials noted that SSA already faces resource constraints managing its disability claims workload and expanding the scope of individualized assessments would exacerbate those constraints. Finally, they noted that data on the availability and use of workplace accommodations are lacking.

SSA’s policy notwithstanding, some opportunities exist for SSA to learn more about the availability of workplace accommodations. For example, developing the OIS may provide an opportunity to obtain some, albeit limited, information on workplace accommodations for the disability determination process. Specifically, while SSA officials reported that they do not plan to collect specific information on workplace accommodations as they develop the system, they may collect more specific information on the physical requirements of different jobs, such as options for sitting or standing. Some experts we spoke with agreed that while information on workplace accommodations would be immensely useful to include in the OIS, given the current scope of that project, they agreed collecting this information via the OIS would likely be too great a task for SSA to accomplish with existing budget constraints and time frames. However, another potentially viable option for collecting more information about workplace accommodations might be the Disability Research Consortium, which SSA is creating pursuant to section 1110 of the Social Security Act, as amended. SSA envisions the consortium as a 5-year cooperative agreement that will serve as a national resource for fostering high quality research, communication, and education on matters related to disability policy, such as identifying or eliminating barriers encountered by people with disabilities in returning to or maintaining work.

Researchers and disability experts have commented on the limitations of SSA’s current disability program eligibility criteria to fully consider whether an individual can function in the workplace due to limited consideration of assistive devices, including those in the workplace—limitations that we have also noted. Moreover, officials we spoke with from an organization of vocational examiners expressed frustration when they see young individuals who could work with minor accommodations being provided disability benefits likely throughout their working life, rather than receiving support to pursue work. Representatives of the organization added that minor accommodations can include a stool for sitting or devices to assist with vision impairments. Although giving broad consideration of assistive devices and workplace accommodations may be difficult to incorporate into the current disability criteria and process, considering how common and inexpensive workplace supports may affect work disability seems feasible and reasonable.

Conclusions

SSA has taken important and concrete steps toward modernizing its disability criteria, but varied challenges may prevent the agency from achieving its goals. Specifically, SSA has a better process in place for updating medical listings but delays in meeting its own goals will likely continue unless the agency explicitly identifies and assigns the resources needed to achieve them. SSA is also taking bold and needed steps to replace outdated occupational information with a new OIS. However, especially in an era of governmentwide fiscal constraints, success of its OIS may hinge on SSA prioritizing its analysis of risks and feasible alternatives to address potential funding shortfalls and other significant challenges. Regardless of the shape and scope of the OIS, absent a complete, reliable, and transparent cost estimate and schedule, SSA risks schedule and performance shortfalls. Finally, SSA is sponsoring promising research to increase consideration of functional ability in its medical listings and other aspects of eligibility criteria—research that experts believe must be aligned with the design of SSA’s new OIS system at the earliest opportunity to best serve the disability determination process. On the other hand, SSA is not considering the full range of assistive devices and workplace accommodations available today. While SSA and others raise valid concerns as to their universal availability and other considerations, in the absence of studies on how certain assistive devices and workplace accommodations are playing a role in helping individuals with impairments to stay at work or return to work, and their costs in comparison with potentially many years of disability benefit payments, SSA may be missing an opportunity to potentially assist individuals with disabilities to re-engage with the workforce and to target finite resources efficiently and effectively.

Recommendations for Executive Action

1. To achieve the goal of updating listings for each body system within SSA’s 5-year time frame, we recommend that the Commissioner of Social Security explicitly identify the resources needed to achieve this goal, such as staff, contractors, and technology aids, and its plans to overcome any resource limitations. 2. To ensure that its work to revise occupational information is feasible and cost effective, and to improve its chance for success, we recommend that the Commissioner of Social Security: formally assess risks to the success of the OIS—addressing such challenges as related to controlling cost, acquiring expertise, managing project complexity, and coordinating with ongoing and related SSA research—and develop appropriate mitigation strategies, and develop a comprehensive and reliable cost estimate and schedule for the life cycle of the project, in accordance with best practices. 3. To help ensure that SSA’s disability decisions are as equitable and consistent with modern views of disability as possible, we recommend the Commissioner of Social Security conduct limited and focused studies on the availability and effects of considering more fully assistive devices and workplace accommodations in its disability determinations.

Agency Comments and Our Evaluation

We provided a draft copy of this report to SSA and the Departments of Commerce, Health and Human Services, and Labor for review and comment. The Department of Commerce did not provide comments. SSA, Labor, and the Department of Health and Human Services provided technical comments, which we incorporated as appropriate. SSA also provided written comments, which are reproduced in appendix V.

SSA agreed with the first two recommendations and disagreed with the third recommendation that the agency should conduct limited and focused studies on the availability and effects of considering more fully assistive devices and workplace accommodations in its disability determinations. SSA stated that conducting such studies is inconsistent with Congress’ intentions. Specifically, SSA noted that the Americans with Disabilities Act of 1990 (ADA), as amended, addresses requirements for workplace accommodations. SSA referred to a 1999 Supreme Court case which acknowledged the complexity of SSA involvement in determining the availability of reasonable accommodations and also noted that ADA determinations are separate from disability determinations under the Social Security Act. SSA asserts that because Congress has made no effort to change the balance between its disability programs and the ADA in the past 22 years, it would be inappropriate for SSA to spend its limited administrative resources on “an initiative that would undermine the balance Congress chose to strike.”

Notwithstanding SSA’s response, we continue to believe that SSA should conduct limited and focused studies on the availability and effects of considering more fully assistive devices and workplace accommodations in its disability determination process. Although SSA asserts that workplace accommodations are addressed by the ADA, this act serves a very different purpose than SSA’s disability programs. We do not think that the fact that workplace accommodations are addressed by the ADA would necessarily preclude SSA from potentially considering them in making disability determinations. Further, although we agree with SSA that Congress has not explicitly directed the agency to consider incorporating assistive devices and workplace accommodations into its disability determinations process, we also note that Congress has not explicitly prohibited SSA from making such considerations. Because the language of the Social Security Act is silent on this issue, and in the absence of clear Congressional intent, we think it would be reasonable for SSA to conduct limited studies on the feasibility of considering such factors.

SSA further cites the 1999 Supreme Court decision in Cleveland v. Policy Management Systems Corporation to point out that assessing reasonable accommodation may turn on highly disputed workplace-specific matters, and that an SSA misjudgment about that detailed matter could deprive a disabled person of the financial support the statute provides. While this is true, SSA fails to acknowledge that by not incorporating such factors, it may be providing benefits to individuals who are capable of working with accommodations, thereby potentially expending scarce government resources in a manner that may not be most appropriate. SSA’s response acknowledges that the agency’s resources are limited—a concern that we share. By conducting limited and focused studies on this issue, SSA would be in a better position to thoughtfully weigh the costs and benefits of these various policy options before deciding on an appropriate course of action.

As we discussed in the report, SSA has avenues to research the availability of select devices or accommodations and the impact of their inclusion on disability determinations, such as through the SSA funded Disability Research Consortium, which is charged to conduct research to identify and eliminate barriers encountered by people with disabilities in returning to work or maintaining their ability to work. SSA could also incorporate discussion of assistive devices and workplace accommodations into its planned symposium with the Institute of Medicine that will assess the inclusion of functional measures in the disability determination process. Through these efforts, SSA may be able to identify certain common and widespread workplace accommodations that could be incorporated into the disability criteria, just as it incorporates certain medical treatments and assistive devices into the medical listings when they become a medical standard. Despite SSA’s efforts to better address an individual’s functioning in its disability criteria, without at least evaluating the costs and benefits of considering more assistive devices and workplace accommodations, SSA is likely to face ongoing barriers in its efforts to integrate a more modern concept of disability into its determination process.

As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to the Commissioner of the Social Security Administration and the Secretaries of Commerce, Health and Human Services, and Labor; relevant congressional committees; and other interested parties. In addition, this report will be available at no charge on GAO’s website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-7215 or bertonid@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix VI.

Appendix I: Objectives, Scope, and Methodology

We were asked to assess the Social Security Administration’s (SSA) plans and efforts to revise its disability criteria. Specifically, we examined the status and management of efforts to update its medical listings to reflect current medical knowledge and develop a new occupational information system (OIS) to reflect labor market changes, and we identified other steps taken by SSA to incorporate a modern view of disability into its eligibility criteria. For this review, we narrowed our scope to criteria used for initial adult disability determinations that involve the medical listings at step 3 or a medical-vocational assessment at steps 4 and 5 of SSA’s five-step sequential process.

To determine the status of activities to revise the medical listings and to develop the OIS, we reviewed our prior reports on the subject and SSA Office of Inspector General reports, relevant federal laws and regulations, and program documentation. This documentation included policies, such as those listed in SSA’s Program Operations Manual System for examiners; strategic goals for fiscal years 2011 and 2012 presented in the Agency Performance Plan for 2012; and other guidance. We also interviewed SSA officials from relevant offices with direct responsibility for revising disability criteria, as well as those offices that provide support for these efforts, and key project contractors and stakeholders. To obtain contextual information on modernization, we reviewed relevant literature, including studies, position papers, and testimonies from disability groups and commissions and interviewed disability experts. We assessed the reliability of data used in this report and found it to be sufficiently reliable for our purposes.

To determine the extent to which SSA’s efforts to revise its medical listings and develop a new OIS were anchored in sound project management practices, we first identified sound project management practices by reviewing A Guide to the Project Management Body of Knowledge, our guidance on internal controls, and the Government Performance and Results Act of 1993. We also identified our recent work (for fiscal years 2008, 2009, and 2010) that evaluated federal planning efforts. Through these efforts, we identified six practices for sound project management which we used as a framework for evaluating SSA’s efforts to revise the medical listings and develop the OIS. Although there is no established set of requirements for all plans, we determined that these practices help implementing parties and decision makers effectively shape policies, programs, priorities, and resource allocations so that they can achieve desired results while ensuring accountability. While these practices may be organized in a variety of ways or use different terms, for the purposes of this report, we grouped them into six categories, from plan conception to implementation. Given the differences among SSA’s efforts to update the medical listings and develop the OIS, we selected broad, higher-level criteria that may apply to a wide variety of projects or plans (see table 5).

Additionally, we compared the cost estimate and schedule for completing SSA’s OIS and related documents with best practices in our Cost Estimating and Assessment Guide. We compared these practices to the OIS project because it requires a significant commitment of resources and time by SSA to complete and will result in an end product. For SSA’s OIS cost estimate and schedule, we scored each best practice as follows:

Not met—SSA provided no evidence that satisfies any of the criteria.

Minimally met—SSA provided evidence that satisfies a small portion of the criteria.

Partially met—SSA provided evidence that satisfies about half of the criteria.

Substantially met—SSA provided evidence that satisfies a large portion of the criteria.

Met—SSA provided complete evidence that satisfies the entire criteria.

We provided the results of our schedule and cost analyses to SSA officials and met with them to confirm the results. Based on the interviews and additional documentation provided by SSA officials, we updated the results of our analyses, as needed.

Appendix II: Progress Updating Six Body Systems since Last Comprehensive Revision

Appendix III: Description of Cost Estimating Best Practices

Characteristic Comprehensive

Characteristic description The cost estimate should include all costs necessary to achieve agency objectives including government and contractor labor costs as well as any necessary material or equipment costs. Comprehensive cost estimates should be structured in sufficient detail to ensure that cost elements are neither omitted nor double counted. Specifically, the cost estimate should be based on a standardized structure that allows a program to track cost and schedule consistently over time. Finally, where information is limited and judgments must be made, the cost estimate should document all cost-influencing ground rules and assumptions.

The cost estimate should be supported by detailed documentation that describes the purpose of the estimate, the program background and system description, the scope of the estimate, the ground rules and assumptions, all data sources, estimating methodology and rationale, and the results of the risk analysis. Moreover, this information should be captured in such a way that the data used to derive the estimate can be traced back to, and verified against, the sources.

The cost estimate should be based on an assessment of most likely costs (adjusted for inflation), documented assumptions, historical cost estimates, and actual experiences on other comparable programs. Estimates should be cross-checked against an independent cost estimate for accuracy, double counting, and omissions. In addition, the estimate should be updated to reflect any changes.

Appendix IV: Description of Scheduling Best Practices

Appendix IV: Description of Scheduling Best Practices Description A schedule should reflect all activities defined in the program’s work breakdown structure and include all activities to be performed by the government and contractor.

The schedule should be planned so that all activities are logically sequenced in the order they are to be carried out.

The schedule should realistically reflect the resources (i.e., labor material and overhead) needed to do the work, whether all required resources will be available when needed, and whether any funding or time constraints exist.

The schedule should reflect how long each activity will take to execute.

The schedule should be horizontally and vertically integrated—that is, it should link already sequenced activities with outcomes while also delineating the relation of supporting tasks and subtasks to upper-level milestones. Such mapping among levels enables different groups to work to the same master schedule.

The schedule should identify the critical path, or those activities that, if delayed, will negatively impact the overall project completion date. The critical path enables analysis of the effect delays may have on the overall schedule.

The schedule should identify float—the amount of time an activity can slip in the schedule before it affects other activities—so that flexibility in the schedule can be determined. As a general rule, activities along the critical path typically have the least amount of float.

The schedule should include a schedule risk analysis that uses statistical techniques to predict the probability of meeting a completion date. A schedule risk analysis can help management identify and understand the most important risks and focus on mitigating them.

The schedule should use realistic durations for activities and be monitored to determine when forecasted completion dates differ from the planned dates. This analysis can be used to assess whether schedule variances will affect future work.

Appendix V: Comments from the Social Security Administration

Appendix VI: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, Michele Grgich, Assistant Director; James Bennett; Kate Blumenreich; Tisha Derricotte; Jennifer Echard; Julie DeVault; Alex Galuten; Sheila McCoy; Patricia M. Owens; Carol Petersen; Karen Richey; Anjali Tekchandani; Kathleen Van Gelder; and Walter Vance made key contributions to this report. Also contributing to this report were Jaime Allentuck, Susan Bernstein, and Amy Frazier.