Social Security Administration: More Effort Needed to Assess Consistency of Disability Decisions
Released: 2004-07-02T00:00:00

Letter

Background

SSA operates the Disability Insurance (DI) and Supplemental Security Income (SSI) programs—the two largest programs providing cash benefits to people with disabilities. The law defines disability for both programs as the inability to engage in any substantial gainful activity by reason of a severe physical or mental impairment that is medically determinable and is expected to last at least 12 months or result in death. The programs have grown substantially, from 10.7 million beneficiaries and $61 billion in benefits in 1995 to 11.4 million beneficiaries and $91 billion in federal benefits to individuals with disabilities in 2003. While disability benefits account for only 15 percent of SSA’s total benefit payments for its Old-Age, Survivors and Disability Insurance (OASDI) programs, administering the disability benefits accounted for 45 percent of the agency’s annual administrative expenses. The relatively high cost of administering the DI program reflects the complex and demanding nature of making disability decisions. SSA estimates that the cost of the disability programs will rise substantially in the near future as the baby boom generation reaches its disability-prone years.

The disability determination process begins at a field office, where an SSA representative determines whether a claimant meets the programs’ non- medical eligibility criteria. Claims meeting these criteria are forwarded to the state DDS to determine if a claimant meets the agency’s definition of disability. At the DDS, the disability examiner takes the lead, or works as a team with the medical or psychological consultants, to analyze a claimant’s documentation, gather additional evidence as appropriate, and approve or deny the claim. A denied claimant may ask the DDS to reconsider its finding, at which point a different DDS team reviews the claim. If the claim is denied again, the claimant may appeal the determination to SSA’s Office of Hearings and Appeals (OHA), where it will be reviewed by an ALJ. The ALJ usually conducts a hearing in which the claimant and others may testify and present new evidence. In making the disability decision, the ALJ uses information from the hearing and from the state DDS, including the findings of the DDS medical consultant. A claimant whose appeal is denied may request a review by SSA’s Appeals Council and, if denied again, may file suit in federal court. Figure 1 provides an overview of SSA’s disability decision-making process and outcomes for 2003.

SSA uses a sequential evaluation process when determining disability. First, SSA field office representatives determine whether a claimant is performing substantial gainful work. If not, DDS or ALJ adjudicators will assess the severity of a claimant’s medical condition(s) to determine whether it meets or equals the criteria in SSA’s regulations (commonly referred to as the medical listings). For a claimant whose conditions do not meet or equal the listings, adjudicators then focus on the functional consequences of the claimant’s medically determined impairments—that is, whether the claimant can perform work he or she has done in the past, and, if not, whether the claimant can perform other work in the national economy.

Concerns about the rate of appeals for hearings, ALJs’ allowance rates, and the accuracy and consistency of ALJ decisions led the Congress to direct SSA to conduct a study in 1980 to determine the extent to which hearings decisions conformed to legal requirements and binding SSA policy. Since the allowance rates at the hearings level could be influenced by many factors, such as the introduction of new evidence, the purpose of the 1980 study was to present the same evidence on cases to different reviewers representing different adjudication levels. In determining the extent to which decision makers agreed on whether to allow or deny benefits, the study concluded that different levels of decision makers had significantly different allowance rates. Specifically, the ALJs decided to allow 64 percent of the cases, whereas the SSA’s central office quality assurance reviewers, comprising medical consultants and disability examiners, decided that only 13 percent of cases should be allowed. The study identified several possible causes of the disparity, including inconsistency in the standards and procedures, interpretation of the standards, and weight given to the evidence. The study also found that disability decisions are complex and necessarily involve some degree of subjectivity by adjudicators.

To help address concerns raised by this and other studies, SSA began its process unification efforts to ensure that both levels more consistently interpreted and applied SSA’s policy guidance. SSA’s plans for its process unification initiative were part of SSA’s larger effort to redesign its disability claims process and were modified over time. SSA’s process unification plans included six major efforts, as described in table 1.

In 1997, we reported on the possible reasons for the inconsistency of decisions between the initial and hearings levels. Our report found that differences in state DDSs’ and ALJs’ views on the claimants’ functional abilities was a key factor in explaining why ALJs allowed benefits on appealed cases. We also reported that poorly documented state DDS evaluations of the claims were of limited use to ALJs and SSA quality reviews did not focus on identifying inconsistency in decisions. To support SSA’s process unification efforts, the report recommended that SSA, using available systems and data collected so far, move quickly ahead to implement its quality assurance initiative to provide consistent feedback to DDS and ALJ adjudicators as soon as possible. In addition, we recommended that SSA expand its effort to return cases to a DDS for review when new evidence is introduced on appeal. Last, we recommended that SSA set goals for measuring the effectiveness of process unification in reducing inconsistent decisions.

More recently, the Social Security Advisory Board issued a 2001 report that identified many factors that could potentially affect the overall consistency of disability decision making between adjudication levels. Some of the factors the board suggested as potentially affecting consistency included: the fact that most claims are decided based on a paper review of case evidence without face-to-face contact with an adjudicator until a claimant has an ALJ hearing, involvement of attorneys and other claimant representatives at the ALJ hearing, the fact that claimants are allowed to introduce new evidence and allegations at each stage of the appeals process, differences in quality assurance procedures applied to initial- and differences in the training given to ALJs and state examiners, and lack of clear and unified policy guidance from SSA.

Despite SSA’s process unification efforts and related studies to improve the consistency of decisions, recent ALJ allowance rates—which declined after process unification began, but started increasing in 1999 to reach 61 percent in fiscal year 2003—still raise questions as to whether initial- and hearings-level decision makers are consistently applying the agency’s guidance. In addition to inconsistent application of SSA’s policy guidance, there are several other reasons why a large number of ALJ allowances are made. For example, some ALJ allowances should be expected because, by law, cases can remain open throughout the hearings process, allowing new evidence to be submitted that may not have been available to the state adjudicators. Such new evidence could show that the claimant’s condition has worsened and prohibits work. Also, SSA’s decision-making criteria require that a great deal of professional judgment be applied. As a result, some allowances at the hearings level could simply reflect the differing judgments of two adjudicators reviewing a case. While a claimant’s deteriorating health, changes in the characteristics of a claim over time, and the complexity of disability decisions may help to explain some of the ALJ allowances, studies have not sufficiently explained why consistently over half the cases appealed to the hearings level are allowed. Instead, studies indicate that systemic differences in the assessment of claims at both adjudication levels are contributing to the ALJ allowance rate. For example, our 1997 report noted a difference in state DDSs’ and ALJs’ views on the claimant’s functional abilities was a key factor in explaining why ALJs allowed cases on appeal.

Inconsistency in decisions may create several problems. High hearings allowance rates may create the perception that the hearings level is applying SSA’s criteria less strictly than the initial level and create an incentive for claimants to appeal to an ALJ for a more favorable decision. If deserving claimants must appeal to the hearings level for benefits, this situation increases the burden on claimants, who must wait, on average, almost a year for a hearing decision and frequently incur extra costs to pay for legal representation. In addition, to the extent that the ALJ allowance rates include inappropriate allowances, SSA could be incurring unwarranted program costs. Although SSA has tried to address these problems, its inability to resolve them has contributed to our decision to include federal disability programs on our list of high-risk government programs.

Renewing its effort to address long-standing and critical problems with the disability programs, SSA’s Commissioner recently announced a new proposal to improve these programs. (See app. I for an excerpt of the announcement that describes the newly proposed decision-making process.) In addition to proposing demonstration projects that provide work incentives and supports to help people with disabilities return to work, SSA has proposed significant changes to both the process of adjudicating disability claims and the structure and management of the agency’s quality management system to improve the timeliness, accuracy, and consistency of the disability decision-making process. The agency believes that several of these changes will help to improve consistency between DDS and ALJ decisions. For example, SSA plans to provide more centralized end-of-line quality reviews. According to SSA, the proposed quality reviews should help to hold adjudicators more accountable for their decisions and ensure that they consistently apply SSA’s policies as well as help the agency detect and amend those policy areas leading to inconsistent decisions. Table 2 provides a description of SSA’s proposed changes to improve the disability decision-making process.

SSA does not plan to implement its proposed changes before it has successfully implemented its Accelerated Electronic Disability (AeDib) system. This major initiative should allow adjudication staff in states and throughout the agency, regardless of geographic location, to access case information electronically through the use of an electronic disability folder. The initiative is intended to reduce delays that result from mailing, locating, and organizing paper folders. SSA also expects this new system to provide critical management information for analyzing and reducing inconsistencies in disability decisions. SSA is implementing the new system and plans to give adjudicators time to adjust to this change before implementing its new proposal. SSA’s implementation of the new proposal will therefore be no earlier than October 2005. In the meantime, SSA continues to discuss the proposal with stakeholders and plans to further refine it before implementation.

SSA Has Partially Implemented Its Process Unification Initiative

SSA has partially implemented its process unification initiative. Although the agency initially made improvements in its policies and training intended to improve the consistency of decisions between adjudication levels, it has not continued to actively pursue these efforts. As part of the initiative, the agency also implemented a review of ALJs’ allowance decisions to identify additional ways to improve training and policies, but no new changes were made as a result of findings from the review. Finally, the agency also began two tests of process changes to help improve the consistency of decisions, but one ongoing test with design problems is not likely to lead to any conclusive results and the other test has been abandoned.

SSA Made Early Progress Improving Policies and Training, but Has Not Actively Pursued These Efforts

While SSA initially made progress carrying out efforts to improve policies and training to better ensure the consistency of decisions, the agency has not continued to actively pursue these efforts. SSA quickly accomplished most of its planned efforts to clarify policy guidance. In 1996, SSA issued nine process unification rulings to clarify policy areas it found to be contributing to inconsistent decisions. For example, one ruling provided all adjudicators with guidance on how to weigh and document their evaluation of the treating physician’s opinions when making a disability decision. SSA successfully went through the regulatory process several years later and published three new regulations to strengthen its process unification rulings, but was unable to agree on a fourth regulation regarding the weight to be given to the treating physician’s opinion when evaluating a claim.

SSA planned to develop a single presentation of policy guidance to replace the different sources used by each level, but has since abandoned full implementation of these plans in favor of a more limited approach. DDS adjudicators currently follow a detailed set of policy and procedural guidelines, whereas ALJs rely directly on statutes, regulations, and rulings for guidance in making disability decisions. To help ensure that inconsistent guidance was not contributing to inconsistent DDS and ALJ decisions, SSA began issuing guidance in the same wording to all adjudicators in 1996. Although SSA had also planned to address differences in policy guidance issued before 1996 and to eventually combine existing adjudication policy documents into a single document, it ultimately decided not to take these additional steps. According to SSA, further efforts to unify the policy guidance used by both levels would be a massive undertaking and not worth the cost because the guidance issued since 1996 had already addressed important policy areas that were leading to inconsistent decisions. While some stakeholder groups representing adjudicators tended to agree with SSA’s position, the Social Security Advisory Board and other groups still believe the agency should take additional steps to provide a unified policy guide to all adjudicators. Instead of creating one policy manual for all adjudicators, SSA told us that it plans to undertake a comprehensive effort to evaluate and improve its disability policies to make them less susceptible to differing interpretations and to ensure they are up to date. A more comprehensive approach could address key weaknesses in SSA’s disability program that we previously highlighted in our performance and accountability series, and thereby help to modernize federal disability programs to better meet the needs of Americans with disabilities.

Early on, SSA also provided extensive cross-training of DDS and ALJ adjudicators, although the scope of its efforts has since diminished. To help all adjudicators understand how to appropriately apply process unification rulings, SSA provided extensive and mandatory training in 1996 and 1997 to 15,000 disability adjudicators (including DDS examiners, physicians, ALJs, and quality assurance staff). The training was provided to adjudicators at all levels of the process in three of the most complex disability areas—assessment of symptoms, treatment of expert opinions, and assessment of claimants’ remaining capacity to work (i.e., residual functional capacity). While this training was intended to be ongoing, SSA’s training efforts have diminished significantly since 1997. Stakeholder groups representing DDS adjudicators told us that SSA’s training does not sufficiently cover process unification issues. In addition, our review of DDS and OHA participation in video training revealed inconsistent participation in training by adjudicators. To provide ongoing training to both adjudication levels and other components involved in the claims process, SSA has used interactive video technology. Almost all the state DDS sites and about 85 percent of OHA offices have this technology. However, in reviewing participation for two recent courses, we found for those sites with this interactive technology only 31 percent of DDS sites and 16 percent of OHA sites logged on for a course on the role of consultative examinations, and 18 percent of DDS sites and 4 percent of OHA sites logged on for a monthly disability hour training class. According to SSA, neither DDS nor OHA adjudicators are generally required to attend courses. In line with these findings, our recent report on the human capital challenges facing DDSs found gaps in the key knowledge and skills of their adjudicators in the same areas SSA had earlier identified as critical to making consistent decisions, and we recommended that SSA work with DDSs to close these gaps.

Despite SSA’s early efforts to improve policy guidance and provide training, stakeholder groups representing state adjudicators told us that many states are not performing the additional development and documentation of decisions required by the process unification rulings. They also told us that the rulings have added significantly to the time, complexity, and subjectivity of the decision-making process, while insufficient resources have limited their ability to fully implement the rulings’ requirements. In addition, claimant lawsuits against three state DDSs have alleged that DDS adjudicators were not following SSA’s rulings or other decision-making guidance. In settling these lawsuits, SSA agreed to have these states fully develop and document cases. However, according to DDS stakeholder groups, SSA has not ensured that states have sufficient resources to meet ruling requirements, which they believe may lead to inconsistency in decisions among states. Furthermore, SSA’s quality assurance process does not help ensure compliance because reviewers of DDS decisions are not required to identify and return to the DDSs cases that are not fully documented in accordance with the rulings. SSA’s procedures require only that the reviewers return cases that have a deficiency that could result in an incorrect decision.

SSA’s Review of ALJ Allowances Has Not Resulted in Improvements to Policy and Training

As part of its initiative, the agency has also implemented a quality review of ALJ decisions, but the review has not proved useful for identifying any new changes to SSA’s policies or training that would help to address the inconsistency of decisions. This review—referred to as the ALJ Pre- effectuation Review—involves a sequential review by SSA’s OQA and the Appeals Council of certain ALJ allowances that have not yet been finalized (i.e., the claimant has not yet been awarded benefits). In selecting allowances for review, OQA uses an error-prone profile developed from its analysis of errors detected when reviewing DDS allowances. SSA began testing the new review of ALJs’ decisions in 1996 and implemented it as an annual review in 1998. From fiscal years 1998 through 2002, OQA reviewed 27,148 ALJ allowances and of these, OQA found fault with about 35 percent and referred them to the Appeals Council. The Appeals Council screens the allowances for its own review and selects those in which the prior actions may not have been proper, fair, or in accordance with the law or the ALJ’s decision was not supported by substantial evidence. If the council finds fault with the ALJ’s decision, it will deny the claimant benefits or return the claim to the ALJ to have the identified problems corrected. If the council does not find fault with the ALJ’s decision, the claimant will be awarded benefits.

In addition to identifying inappropriate ALJ allowances, SSA intended to use the new quality review to identify areas of inconsistency between adjudication levels and ways to improve policies and training to address those inconsistencies. Specifically, OQA identified cases where it found fault with the ALJ decision, but the Appeals Council, after screening them, did not accept them for review. OQA then forwarded these cases to a panel of staff from the various components involved in SSA’s claims process to determine whether the inconsistent assessment of these cases by OQA and the Appeals Council indicated the need to clarify policies, issue new policies, or provide training to improve the consistency of decisions. However, according to a SSA official, this review did not identify any new areas of inconsistency that required improvements to policy and training. Weaknesses in the design of the review may have contributed to SSA’s inability to identify new policy areas contributing to inconsistency. For example, rather than reviewing a random sample of all ALJ decisions, this review focused on allowances. Further, the review looked only at ALJ allowances that were selected using a DDS error-prone profile, i.e., a profile that is based upon cases in which quality reviewers did not agree with the DDS adjudicators’ decisions. As a result, SSA selected and reviewed nonrandom allowance decisions with case characteristics that the agency may have already suspected were associated with inconsistent decisions. In 1999, the panel was disbanded because members had other priorities needing attention. OQA told us that it continued to perform a limited review of cases viewed differently by OQA and the Appeals Council. More recently, OQA began an effort to summarize the results of its review and expected to issue a report of its findings in April 2004. As of April 2004, this report had not been issued.

SSA Began Two Tests of Process Changes to Improve the Consistency of Decisions, but Neither Test Was Successfully Completed

SSA began two tests of potential changes to the process to help improve the consistency of decisions, but neither test was successfully completed. The changes tested were (1) more fully developing and documenting decisions made at the initial level and (2) sending appealed cases that involve new medical information back to the initial level to be reevaluated.

SSA wanted to test having DDSs more fully develop and document decisions because it believed that DDS decisions, especially denials, are often not well documented. SSA wanted to test whether better explanations of why benefits were denied would improve the accuracy of DDS decisions and consistency of decisions between adjudication levels. SSA first implemented a pilot of this change to explore alternatives for developing and documenting decisions. Then SSA tested this change, along with other process changes, in a larger test, called the prototype initiative. Concurrently, SSA tested other process changes, such as the elimination of a reconsideration step and a predecision DDS interview with the claimant.

The prototype test had limitations for predicting the impact of documented decisions. For example, SSA’s decision to test several changes together left the agency without clear information on what impact fully developed decisions would have on the decision-making process without the other process changes. SSA’s test design also did not build in an ALJ feedback mechanism to provide sufficient information on the usefulness of more fully documented decisions. SSA continues to test this change along with other changes and, despite limited information on the best approach for and impact of this change, currently plans to implement more fully documented decisions as part of the Commissioner’s new proposal to improve SSA’s disability programs.

SSA also began, but ultimately abandoned, a test in which appealed cases with new medical information submitted prior to the hearing were to be sent back to the initial level so that the evidence could be evaluated by medical consultants residing at the DDSs. Since medical expertise resides in the DDS and not at the hearings level, SSA decided to test whether “remanding,” or sending cases to the DDS for evaluation, might result in a more consistent review of medical evidence. SSA believed that this change, in turn, could help improve the consistency of decisions because the new medical information might be contributing to ALJ allowances. However, the change also had the potential to increase the time claimants with remanded claims would have to wait for final decisions because claims that were not allowed by the DDSs had to be returned to OHA for hearings. SSA began remanding cases in July 1997, with a 1-year goal of remanding 100,000 cases, but after 10 months, it had remanded fewer than 9,000. In implementing this test, SSA encountered several difficulties. For example, it had difficulty identifying the claims to be remanded and ensuring the ALJs, who had authority over the claims, would remand the claims to the DDSs. The ALJs’ resistance to remanding claims to the DDSs may be due in part to concerns that remanding would not lead to many allowances by the DDSs and would result in many claims being returned to OHA, thereby increasing the time many claimants would have to wait for a final decision from OHA. Realizing that the agency would not be able to reach its remanding goal, the agency decided to discontinue this test.

SSA Lacks a Clear Understanding of the Extent and Causes of Inconsistency between Levels

SSA’s assessments have not provided the agency with a clear understanding of the extent and causes of possible inconsistencies in decisions between adjudication levels. The two measures SSA uses to monitor changes in the extent of inconsistency of decisions have weaknesses and therefore do not provide a true picture of the changes in consistency. In addition, SSA has not sufficiently assessed the causes of possible inconsistency. The agency conducted an analysis in 1994 that identified some potential areas of inconsistency. However, although SSA continues to collect information that would support this analysis, it has not repeated this initial effort, nor has it expanded on it by employing more sophisticated assessment techniques.

SSA Attempts to Monitor Changes in the Extent of Inconsistency, but the Measures It Uses Provide an Incomplete Picture

SSA has made some efforts to monitor changes in the extent of inconsistency between the initial and hearings levels, including tracking trends in allowance rates at different levels and conducting special reviews of ALJ decisions. Together, according to SSA, these measures and assessments suggest that the consistency between levels has improved since the agency began implementing its process unification initiative. However, because of methodological weaknesses, these measures provide, at best, a partial picture of trends in the consistency of decisions between adjudication levels.

SSA tracks trends in the proportion of all allowances decided at each level to assess the consistency of decisions between levels. The agency collects information on the number of allowances granted to claimants at each level of the process, tracks the proportion of claims allowed at the initial level relative to the hearings level, and looks at trends in these proportions over a period of several years. According to data from SSA, the proportion of overall allowances that occurred at the initial level has increased since process unification was implemented. As shown in figure 2, in fiscal year 1996, 72 percent of all allowances were granted at the initial level. This proportion increased in most subsequent years, and by fiscal year 2003, 77 percent of all allowances were granted at the initial level. Officials from OQA, the office responsible for reviewing, evaluating, and assessing the integrity and quality of the administration of SSA’s programs, view the relative shift toward earlier allowances as an indicator that consistency between adjudication levels has improved, and they believe that process unification efforts have contributed to these results.

However, SSA’s measure of tracking yearly changes in the proportion of allowances at each level is a simplistic and inconclusive indicator of trends in the consistency of decisions because it does not control for the multitude of factors that can affect allowance rates at either adjudication level in any given year and over time. For example, SSA uses “snapshot” data in looking at the proportion of allowances granted at each level, meaning that it looks at the number of claimants and allowances at each level during a given year, rather than following a 1-year cohort of initial claimants through the entire process and capturing the proportion of allowances for that cohort decided at each level. Because SSA uses data that illustrate allowance rates at a given moment in time, it captures a different pool of claimants in the process at each level, and the resulting allowance rates are subject to a different set of demographic and case characteristics. Over time, the pool of claimants may change because of factors such as a downturn in the economy, which can cause more people with less severe impairments to claim benefits or appeal initial denial decisions. In addition, snapshot data may be significantly affected by fluctuations in productivity at either adjudication level caused by process changes that are unrelated to process unification and that affect only one level.

SSA has collected other data to further assess trends in the consistency of decisions. Since 1993, the agency has conducted a biennial case review as part of its Disability Hearing Quality Review process. This review consists of medical consultants and disability examiners in SSA’s central office evaluating a sample of ALJs’ decisions plus supporting documentation to determine whether the ALJ has adequately supported his or her decision. In evaluating the ALJ decisions, these medical consultants and disability examiners use the same standards as those used by initial-level adjudicators to adjudicate claims, which are from the official SSA program policy and operations guidance found in POMS. To some degree, therefore, the medical examiners and disability reviewers serve as a proxy for initial-level adjudicators, and their decisions are representative of how initial-level examiners should be deciding claims.

While unpublished results from the biennial case reviews indicate an increase in supportable ALJ allowances, such findings focus on the ALJ level and therefore provide only a partial picture of trends in consistency. The reviews indicated that medical consultants and disability examiners have found that supportable ALJ allowances increased from 36 percent in fiscal year 1993-94 to 57 percent in fiscal year 1999-2000. OQA officials told us that this increase suggests an improvement in consistency between adjudication levels because it indicates that disability examiners using initial-level standards and ALJs increasingly agree on how like cases should be decided. However, SSA’s assessment provides only a partial picture because it does not reflect trend information on the extent to which ALJs have found DDS decisions to be supportable, to ensure that both levels are making more consistent decisions. Although the 1994 report of findings from the initial biennial case review included the results of a special probe in which ALJs reviewed 165 DDS reconsideration denial decisions, the sample was not representative, and therefore results could not serve as a baseline for developing trend information. In 2003, SSA began another probe, in which ALJs reviewed 400 DDS reconsideration denial determinations, but the agency does not plan to release its findings until summer 2004.

Although SSA has limited information on how ALJs view DDS decisions, other information collected by the agency suggests that consistency of decision making at the initial level might not be improving. For example, OQA reviewers routinely assess the accuracy and supportability of DDS decisions. A recent SSA study of these data shows that the accuracy of DDS denial decisions—those decisions most likely to be appealed to the hearings level—has declined by 4 percentage points over a 1- year period. Another review of DDS decisions by OQA reviewers also suggests a lack of improvement at the initial level. Specifically, the extent to which quality reviewers found that DDS reconsideration denials appealed to the hearings level were supported declined from 71 percent in fiscal year 1993-94 to 68 percent in fiscal year 1999-2000.

SSA’s Assessments Have Not Identified the Areas and Causes of Possible Inconsistency in Decisions between Adjudication Levels

Despite some efforts to assess inconsistency in decisions, shortcomings in SSA’s analyses also limit its ability to identify areas and causes of possible inconsistency. Most notably, over the last 10 years, SSA has not updated its prior analyses of information from its initial biennial case review that helped identify problem areas. In addition, SSA has not improved on its case review and analysis by ensuring that reviewers assess all relevant case evidence used to make decisions, or performed more sophisticated analysis to identify the areas and causes of inconsistency in decisions. Other efforts—including the review of ALJ allowances and a probe of DDS reconsideration denials—have yet to yield useful information.

In 1994, for its initial biennial case review report, the agency took its first step in identifying areas of possible inconsistency by identifying two characteristics about the claimants and their cases over which initial-level reviewers tended to disagree with ALJs. Specifically, the 1994 report concluded that teams of reviewing medical consultants and disability examiners sometimes viewed cases involving mental impairments differently than the reviewing ALJs. In addition, these two sets of reviewers tended to have different views on the severity of claimants’ impairments and their resulting capacity to work. According to the official responsible for overseeing the review, the findings in this initial report provided important support for SSA’s process unification efforts as well as the agency’s efforts to redesign the disability claims process.

SSA continues to conduct the biennial case reviews; however, the agency has not continued to analyze and identify areas that are viewed differently by different adjudication levels. Specifically, SSA no longer identifies the particular case characteristics over which reviewers from the two levels tend to disagree. As a result, SSA does not know whether previously identified problem areas are still present. Moreover, SSA no longer publishes any information from the medical consultant and disability examiner biennial case reviews, even though it has performed some limited analysis of the supportability of decisions made by adjudicators. By not continuing to publish its analysis and findings, the agency makes it difficult to ensure the reliability of its methods and results, and leaves stakeholders outside the agency, including disability groups, without a means for understanding SSA’s assessment efforts and progress in improving the consistency of decisions. The SSA office conducting the study has told us that, because of downsizing and competing priorities, it has no current plans to further analyze and publish these data.

Further, in its ongoing biennial case reviews, SSA does not make full use of available case information that would be useful in identifying areas and causes of inconsistency. Specifically, medical consultants and disability reviewers do not listen to tapes of the hearings and therefore do not review the entire case as presented to the original ALJ. Although reviewing medical consultants and disability examiners read the ALJs’ explanations for their original decisions, which should include the most important factors behind the ALJs’ decisions, the reviewers do not evaluate the oral evidence independently. An SSA official with whom we spoke indicated that some evidence entered by witnesses at the hearing might not be accompanied by other hard copy sources of the same information. Therefore, reviewers would not consider information potentially relevant to the ALJ’s decision that could be used to identify areas and causes of inconsistency.

SSA also does not make full use of the information it collects because it has not employed analytical tools that would improve its ability to identify areas and causes of inconsistency. For example, SSA’s biennial case reviews provide a rich dataset that lends itself to regression analysis to identify areas and possible causes of inconsistency between levels. Regression analysis would allow the agency to better pinpoint any significant case characteristics affecting decisions and to more clearly identify the underlying causes of inconsistency. Specifically, among the data collected in this review are such variables as the types of impairments the claimant has, the types of relevant medical evidence, and additional impairments presented at the hearing. Multivariate analysis, such as a multiple regression model, could allow SSA to assess how these and many other factors, relative to one another, contribute to whether a case results in a similar outcome at both levels. However, SSA has not employed this more sophisticated multivariate technique, citing resource constraints and competing priorities. We recognize the methodological complexities of analyzing disability decisions, and we previously recommended that SSA establish an advisory panel of external experts from a range of disciplines to provide leadership, oversight, and technical assistance to the agency. Otherwise, in forgoing such analysis, the agency will continue to miss an opportunity to better pinpoint areas and some possible causes of inconsistency in decisions between the two adjudication levels, and to lay the foundation for further investigation.

Another tool SSA has not sufficiently employed for identifying areas and causes of inconsistency is in-depth case studies involving both levels of adjudication. Case studies, in which different adjudicators review the same test case, can be a means for unearthing causes for inconsistency by getting adjudicators from both levels to acknowledge and address discrepancies in the ways they view cases. SSA has performed case studies in the past to ascertain differences in policy interpretation between DDS examiners and quality reviewers. However, SSA does not routinely have both DDS examiners and ALJs perform in-depth review of the same sample of cases, despite this method’s potential for helping identify causes of inconsistency between the two adjudication levels. OQA officials told us that case studies are a very resource-intensive tool because they need a sufficient number of cases from which to generalize. Therefore, the agency is reluctant to use this approach to help it understand the causes of inconsistency between adjudication levels. However, using multivariate analyses of the biennial case review data could help the agency to more effectively target its in-depth case studies on those areas found to be leading to inconsistent decisions and thereby increase its success at identifying the causes of inconsistency.

SSA conducts other analyses of inconsistency between levels, but to date these efforts have yielded limited information concerning areas and possible causes of inconsistency. For example, as part of SSA’s ALJ Pre- Effectuation Review, two different levels of reviewers have evaluated thousands of cases. However, limitations in the review methodology, such as not using a random sample of ALJ decisions, do not allow the agency to use this review to identify the leading causes of inconsistency. SSA recently began an evaluation of this effort and plans to publish its findings and recommendations in April 2004. Another analysis currently under way, a special 400-case review, might help identify areas of inconsistency at the initial level, but it has yet to be completed. Begun in 2003, this review by ALJs of DDS reconsideration denial determinations is expressly aimed at assessing inconsistency between adjudication levels. SSA expects to gain some understanding of why about 60 percent of cases denied by the initial level and appealed to the hearings level are allowed. The agency plans to publish its findings in summer 2004.

SSA’s New Proposal Incorporates Efforts to Improve the Consistency of Decisions, but Challenges May Impede Successful Implementation

Some changes included in SSA’s new proposal to overhaul its disability claims process may improve the consistency of DDS and ALJ decisions, but challenges may hinder the implementation of the proposal. The new proposal includes several changes to the disability claims process that the agency and stakeholder groups representing adjudicators and claimant representatives believe offer promise for improving the consistency of DDS and ALJ decisions. However, past difficulties in improving the process, as well as stakeholder concerns about limited resources and other obstacles, indicate that some difficulties may arise in the development and implementation of SSA’s new proposal.

The Agency and Most Stakeholders Viewed Several Aspects of the New Proposal as Offering Promise for Improving Consistency

SSA told us that several aspects of the new proposal may improve the consistency of decisions, and although opinions varied among stakeholder groups, most thought the following four proposed changes have the potential to improve the consistency of decisions between adjudication levels: (1) requiring state adjudicators to more fully develop and document their decisions, (2) centralizing the agency’s approach to quality control, (3) providing both adjudication levels with equal access to more centralized medical expertise, and (4) requiring ALJs to address agency reports that either recommend denying the claim or outline the evidence needed to fully support the claim.

Representatives from most stakeholder groups with whom we spoke told us that having state adjudicators more fully develop and document their decisions may help to improve the consistency of DDS and ALJ decisions. Specifically, stakeholders said that more developed decisions may provide ALJs with a better understanding of the DDS decision and enable them to more fully consider this information when evaluating a case. According to the agency and stakeholders, this change may contribute to a more consistent interpretation and application of SSA’s decision-making criteria. They also mentioned that well-developed decisions by DDS examiners could assist SSA in holding adjudicators accountable for case development and decisions, such as enabling quality reviewers to more effectively assess the appropriateness of the DDSs decisions. Unlike SSA’s earlier attempt at more fully developing decisions as part of process unification, SSA plans to incorporate a reviewing official into the process whose assessment of all appealed DDS decisions can provide feedback on the extent to which cases are being fully developed.

In addition, the agency and many stakeholders told us that they believe centralizing the agency’s quality control system may help resolve some problems contributing to inconsistent decisions between the two levels. For example, they believed that it may help ensure a more consistent review of cases across the country and between adjudication levels. According to both stakeholders and other experts within and outside of SSA (including SSA’s Deputy Commissioner of Disability and Income Security and a consulting group that reviewed SSA’s quality assurance system), the current quality control and case review process encourages adjudicators at the initial level to inappropriately deny cases, while encouraging adjudicators at the hearings level to inappropriately allow cases. Specifically, by overemphasizing a review of DDS allowances to help control the cost of benefits, the agency has unintentionally encouraged DDS examiners to deny cases. Conversely, SSA’s review of ALJ decisions consists mostly of SSA’s Appeals Council reviewing cases denied by ALJs, thereby providing an incentive for ALJs to allow cases. By centralizing the quality control system and making other changes to the process, SSA believes that it can remove the current incentives that contribute to inconsistency.

The third proposed change that the agency and most stakeholder groups believe may improve consistency is SSA’s plan to provide both adjudication levels with equal access to more centralized medical experts, organized by clinical specialty. Although located in the regions, these experts should be able to review cases from across the country with the successful completion of SSA’s AeDib initiative—an electronic folder initiative for exchanging case information currently being implemented by SSA. By making experts in a range of specialties available to assist both levels of adjudicators in their decision making, SSA and stakeholders believe that adjudicators could more consistently apply SSA’s decision- making criteria, in addition to acquiring better medical evidence.

Finally, the agency and most stakeholder groups told us that the requirement to have an ALJ’s decision address the recommendations from a reviewing official’s report to either deny or more fully develop the claim may increase consistency between levels. Under the new proposal, SSA plans to introduce a reviewing official into the process to evaluate all appealed DDS claims. The official will allow claims that meet SSA’s definition of disability and, for the remaining claims, will develop a report that either (1) contains reasons for denying the claim or (2) outlines the evidence needed to fully support the claim. The ALJ’s decision must address issues raised in the reviewing official’s report. Stakeholders believed that this change could, as intended by SSA, hold adjudicators more accountable for their decisions and provide adjudicators with feedback on the reasons decisions tend to differ between levels to improve the quality and consistency of their decisions.

Although there was less agreement among stakeholder groups on the potential effect that other aspects of the new proposal may have on the consistency of decisions, some groups thought that other changes could result in improved consistency between DDS and ALJ decisions. For example, the Social Security Advisory Board and two groups representing the DDSs thought that the proposed in-line quality control, if implemented effectively at all levels, could have a positive impact on consistency by ensuring that adjudicators adhere to the rulings and regulations throughout the decision-making process. One stakeholder group added that in-line quality control could also help the agency identify problem areas, including areas in which policy is applied inconsistently or where more training is needed.

Resource and Other Constraints May Limit SSA’s Ability to Successfully Implement Some Changes in the New Proposal

According to stakeholder groups—and based on SSA’s prior experience with making significant changes to its claims process—insufficient resources and other obstacles may prove to be major challenges for the agency in developing and implementing aspects of its new proposal. For example, experience with the process unification initiative has shown that limited state resources have hindered the agency’s ability to have state adjudicators fully document decisions. To address this issue, SSA plans to reduce the states’ workloads by decreasing the number of claims to be decided by the DDSs. Specifically, SSA expects that establishing regional expert review units to make quick decisions for claimants who are obviously disabled will substantially decrease the states’ workloads. However, SSA has not developed and provided stakeholders with estimates of the administrative cost for more fully documenting decisions and other planned changes, and stakeholder groups were not convinced that the reduction in claims was sufficient to offset resources needed to fully document their decisions. Although the agency has had some recent success in increasing its 2004 administrative budget, and is confident that it will be successful in acquiring the resources it needs to implement the proposal, the significance of stakeholders’ concerns about funding cannot be assessed until SSA fully develops its proposal and associated cost estimates.

Experience has also shown that another proposed change, developing a centralized quality control system for both adjudication levels, could be a major challenge for the agency. In 1994, SSA began efforts to create a unified and comprehensive quality control system as part of its redesign efforts, but made little progress, in part because of considerable disagreement among internal and external stakeholders on how to accomplish this difficult objective. To get external assistance in developing an effective quality assurance system, SSA contracted with an independent consulting firm to assess SSA’s quality assurance practices used in the disability claims process. In 2001, concluding that SSA could achieve its quality objectives for the disability program only by adopting a broad, modern view of quality management, the consulting firm recommended SSA abandon its current system and design a new quality management system focused on building quality into the process. The agency agreed that it was appropriate to transform the existing quality assurance system and established an executive work group to decide a future course of action. The agency is working with another consulting group to further develop the changes recently proposed by the Commissioner. However, after 10 years of efforts to develop a more unified quality review system, SSA has not yet formulated changes to its quality review system, beyond the brief and general descriptions provided in the Commissioner’s new proposal.

Other obstacles also add to the complexity and difficulty of implementing the proposal. For example, stakeholder groups have raised concerns about SSA’s ability to successfully implement its proposed change to provide equal access for all adjudicators to more centralized medical expertise by removing medical expertise from the state DDSs and providing expertise in regional offices. Stakeholder groups were concerned that SSA would not be able to attract and retain sufficient medical experts to meet the agency’s needs. They told us that states are currently experiencing problems attracting medical experts because SSA’s compensation rates are too low. State adjudicators, who currently work with medical experts directly at DDS offices, were also concerned that removing these experts and placing experts in SSA regional offices would impair the states’ effectiveness and efficiency. By placing experts in regional offices, state disability examiners would no longer have on-site access to these experts who help facilitate the states’ adjudication of claims and provide on-the- job training and mentoring to DDS examiners.

Stakeholders have also raised questions about SSA’s ability to ensure that ALJs’ decisions fully respond to the reviewing officials’ reports and the ultimate effectiveness of this change. Stakeholder groups representing ALJs and claimant representatives believed that the requirements may have the potential to impinge on an ALJ’s legal responsibility to ensure a claimant receives a fair hearing and an independent decision. Other groups have raised concerns about SSA’s ability to ensure that ALJs will adequately address recommendations in the reviewing officials’ reports to help ensure that this requirement leads to more consistent decisions. Although these concerns have been raised, the Commissioner has clearly stated that the intent of the proposal is to improve service to claimants, including providing fair and accurate decisions, and that changes will not impinge on the independence of ALJs.

In addition, several stakeholder groups also told us that staffing the new reviewing official positions with attorneys, as SSA intends to do, would be expensive. To the extent that SSA has difficulty filling these positions, the agency could create a slowdown or bottleneck in the process that could increase the time claimants must wait for a decision. Furthermore, according to one stakeholder group, SSA’s new quality assurance process will need to ensure that this new position does not create another source of inconsistent interpretation and application of SSA’s decision-making criteria.

Several groups representing hearings level adjudicators and claimant representatives were also concerned about other aspects of the Commissioner’s new proposal, such as the proposed elimination of the Appeals Council and the claimants’ loss of the right to appeal an ALJ decision to the council. The Appeals Council currently reviews about 100,000 appealed ALJ decisions annually. For these claims, the council provides an additional appellate step for addressing claimants’ objections to the ALJs’ decisions, reviewing new medical information on the claims and reducing the number of claims appealed directly to the federal courts. According to one stakeholder group, the council also performs other important functions, such as reviewing claims for surviving children or spouses of workers who were insured under the disability insurance and retirement program. The council also reviews cases remanded from federal courts. This stakeholder group also told us that as SSA refines its proposal it will need to articulate how all of the council’s functions will be handled under the new process.

Adding to uncertainties about the proposal’s success is its dependence on the successful development and implementation of the AeDib system—a highly complex and as yet unproven system using electronic folders to share information with all entities involved in disability determinations. SSA does not plan to implement its newly proposed changes before it has completed a national rollout of its electronic disability system, scheduled to be completed by October 2005. The new electronic disability system represents an important step toward a paperless and more efficient sharing of information by multiple partners involved in the disability claims process, including SSA and state officials, as well as physicians and other members of the medical community who provide needed medical evidence. SSA also expects this new system to provide critical management information for analyzing and reducing inconsistencies in disability decisions. As we previously reported, SSA has made progress developing the new system. However, its approach involves risks that could jeopardize the agency’s successful transition to an electronic disability claims process. For example, SSA recently began a national rollout of the electronic disability system without fully evaluating pilot test results or ensuring the resolution of all critical problems. Skipping such important steps in development and implementation leaves the new system vulnerable to problems in its performance and reliability. In addition, problems with implementation of this system could delay the implementation of SSA’s new proposal.

SSA recognizes that transforming its massive and complex disability programs and achieving the benefits envisioned by the Commissioner will be a challenging undertaking. The agency is refining its proposal and, as part of this process, is actively seeking input from stakeholder groups. The Commissioner and her staff have met directly with stakeholder groups to understand and begin to address their concerns. As the agency refines its proposal, the significance of both stakeholder concerns and previous problems SSA has experienced improving its programs should become clearer.

Conclusions

When SSA’s Commissioner announced her new proposal to overhaul the disability programs, the agency acknowledged the importance of making similar decisions on similar cases and making the right decision as early in the process as possible. SSA has good cause to focus on the consistency of decisions between adjudication levels. Incorrect denials at the initial level that are appealed increase both the time claimants must wait for a decision and the cost of deciding cases. Incorrect denials that are not appealed may leave needy individuals without a financial or medical safety net. Conversely, incorrect allowances at any adjudication level could substantially increase the cost of providing disability benefits.

While the agency has made some effort to assess the inconsistency in decisions between levels, its efforts have not provided the agency with a clear understanding of the extent and leading causes of possible inconsistencies in the interpretation and application of disability guidance.

For example, SSA’s assessment of ALJ error-prone allowances has not proven to be effective at identifying new areas and causes of inconsistency. SSA also has not updated its more effective approach of analyzing its Disability Hearings Quality Review data to identify problem areas and help improve its understanding of the factors that may be contributing to inconsistency. Further, SSA’s analysis lacked sophisticated statistical techniques and in-depth analysis of cases by adjudicators at both levels, which together would have allowed SSA to better identify and address the areas and leading causes of inconsistency. Moreover, by not having examiners and medical consultants perform a complete review of all relevant information before an ALJ, SSA has limited its ability to understand the areas and causes of possible inconsistency.

Without better information on the areas and causes of possible inconsistency, the agency cannot ensure that the Commissioner’s new proposal will help to resolve this complex and long-standing concern. By taking immediate actions to improve its understanding of the leading causes of possible inconsistency in decisions, the agency will have information needed to evaluate and possibly refine its new proposal, including its plans to build an effective quality assurance system that can both detect and prevent inconsistencies in decisions. This information will help the agency to target its limited resources and take decisive steps to build a claims process that provides claimants with the accurate, consistent, and timely decisions they deserve, as envisioned in the Commissioner’s proposal.

Recommendations

To move successfully forward with agency efforts to make more consistent decisions, including efforts incorporated in the Commissioner’s proposal for an improved disability claims process and quality assurance system, we recommend that SSA quickly expand its assessment of the areas and causes of inconsistency in decisions between adjudication levels. In doing so, SSA should consider making near-term and cost- effective enhancements to its current approach for assessing the consistency of decisions, including: 1. Reestablish ongoing analyses of case characteristics as part of its biennial case review, in line with efforts undertaken for the review report published in 1994. 2. Perform more sophisticated multivariate analysis on the biennial case review data in order to pinpoint the most significant case characteristics influencing allowance decisions and to distinguish factors that might be contributing either appropriately or inappropriately to allowance decisions. 3. Expand the biennial case review by requiring disability examiners and medical consultants to review the hearing tapes to ensure that reviewers have the complete case before them (including the types and sources of testimonial evidence provided during the hearings) when evaluating the ALJs’ decisions. 4. Have adjudicators and reviewers from each level study cases in depth to help pinpoint the causes of inconsistency, once potential areas of inconsistency between levels are identified. 5. Publish the methods and findings of all analyses, to keep internal and external stakeholders aware of the agency’s efforts to assess consistency and demonstrate improvement over time. 6. Use the information from these improved analyses to develop a more focused and effective strategy for ensuring uniform application of SSA’s guidance and to improve the consistency of decisions. To accomplish this, SSA should clarify guidance for making disability decisions and develop mandatory training for adjudicators on issues identified as contributing to inconsistency.

Agency Comments and Our Evaluation

We provided a draft of this report to SSA for comment. SSA expressed several reservations about the recommendations, findings, and conclusions of our report. Primarily, SSA took issue with: (1) our characterization of the agency’s progress over the past several years in analyzing and reducing the inconsistency of decisions, (2) our recommendation that the agency incorporate multivariate analysis into its assessments, and (3) our finding that the agency has not acted on the results of its reviews of decisions. SSA indicated that it would reevaluate our recommendations as the design of its Commissioner’s new approach to disability decision making evolves. However, the agency did agree to pilot one recommendation—that quality reviewers assess hearing tapes when evaluating the ALJs’ decisions—as part of a quality review.

One of SSA’s main concerns was that our report did not fully discuss the progress SSA had achieved in analyzing and reducing the inconsistency in decision making between adjudication levels. For example, SSA commented that our report dismissed the 21-percentage point increase in the quality reviewers’ support rate of ALJ decisions, conducted as part of SSA’s biennial case reviews over the last 10 years. SSA also pointed to findings from its ALJ peer reviews as additional evidence that the quality and consistency of SSA’s decisions had improved. In addition, SSA asserted that its comparison of the relative proportion of allowances at the DDS and ALJ levels, along with high accuracy rates, indicated that adjudicators were making the right decisions sooner in the process—a goal of both process unification and the Commissioner’s new disability approach. Although our report incorporates results from the analyses cited by SSA, our conclusion about the improvement in consistency between levels is not as optimistic as SSA’s because of weaknesses in SSA’s assessments. As we reported, SSA’s analysis of the quality reviewers’ assessment of ALJ cases has been limited for 10 years to calculating ALJ support rates. SSA has not used available data to determine the potential areas of inconsistency between levels or the extent to which changes in the ALJ support rate is related to improvements in consistency of decisions between adjudication levels. SSA’s assessment also lacks a reliable method for determining whether DDS decisions are more consistent with ALJ decisions, for example, by having ALJs regularly review a statistical sample of DDS decisions. Lastly, as we pointed out, changes in the proportion of overall allowances made by the DDS and ALJ levels cannot serve as a reliable indicator for measuring the consistency of decisions between levels, because many factors can affect these proportions, such as significant fluctuations in the number of decisions made at each adjudication level.

SSA also expressed its reservations about the benefits of multivariate analysis in its evaluation of decision making. SSA asserted that its analyses over the past 10 years have provided the agency with a solid understanding of how certain variables influence disability decision making and that the multivariate analyses we recommended would not identify the causes and effects of inconsistent decision making at different levels of this complex process. We agree with SSA that the disability decision-making process is complex and that multivariate analysis alone cannot establish all the causes and effects of inconsistent decision making. However, because multivariate analysis takes into account the influence of a number of relevant variables for each decision, this analytical technique can provide a more accurate understanding of areas and causes of inconsistency in decisions than methods previously employed by SSA. Such analyses, followed by in-depth case studies by adjudicators at both levels, which we also recommended, would bring SSA closer to understanding and resolving the inconsistency of decisions between adjudication levels. Therefore, we continue to believe that by performing the analyses we recommend, the agency will have a better understanding of the extent and causes of inconsistency, and that SSA’s Commissioner should quickly implement our recommendations to ensure that her new approach effectively addresses the consistency of decisions between adjudication levels.

Finally, SSA disagreed with our finding that it has not acted on the results of its reviews of decisions. SSA noted that it has made changes to address training needs that have been identified by its reviews. Specifically, SSA indicated that it has provided a series of interactive video training (IVT) sessions focusing on problematic areas noted in the ALJ peer review reports. We acknowledge that SSA has conducted ALJ peer reviews and used findings from its reviews to develop and provide training to ALJs. However, we did not include these findings in our report, because our objectives were limited to reporting efforts undertaken by SSA to assess or improve the consistency of decisions between adjudication levels or to implement its process unification initiative. SSA’s ALJ peer review is conducted to identify problems with the quality of ALJ hearing process and decisions, not to identify problems with the inconsistency of decisions between levels. Conversely, our report included information on SSA’s ALJ pre-effectuation review, because it was part of SSA’s process unification initiative. According to information provided to us by SSA during our audit, although this review was intended to help identify policy and training areas that were associated with inconsistent decisions between adjudication levels, it was not effective at identifying any new areas to be pursued by the agency. This finding, along with those provided throughout the report, supports our recommendations to SSA that the agency perform additional analysis to determine the causes of potential inconsistency between adjudication levels and to clarify guidance and provide mandatory training to address any identified causes.

In addition, SSA provided several other general and technical comments about the draft report. These additional comments, as well as our response to them, are provided in appendix II.

Copies of this report are being sent to the Commissioner of SSA, appropriate congressional committees, and other interested parties. The report is also available at no charge on GAO’s Web site at http://www.gao.gov. If you have any questions about this report, please contact me at (202) 512-7215. Other contacts and staff acknowledgments are listed in appendix III.

Appendix I: Excerpt of SSA’s Testimony Announcing Its New Proposal to Improve Its Disability Decision-Making Process

In designing my approach to improve the overall disability determination process, I was guided by three questions the President posed during our first meeting to discuss the disability programs.

Why does it take so long to make a disability decision?

Why can’t people who are obviously disabled get a decision immediately?

Why would anyone want to go back to work after going through such a long process to receive benefits?

I realized that designing an approach to fully address the central and important issues raised by the President required a focus on two over- arching operational goals: (1) to make the right decision as early in the process as possible; and (2) to foster return to work at all stages of the process. I also decided to focus on improvements that could be effectuated by regulation and to ensure that no SSA employee would be adversely affected by my approach. My reference to SSA employees includes State Disability Determination Service employees and Administrative Law Judges (ALJs).

As I developed my approach for improvement, I met with and talked to many people—SSA employees and other interested organizations, individually and in small and large groups—to listen to their concerns about the current process at both the initial and appeals levels and their recommendations for improvement. I became convinced that improvements must be looked at from a system-wide perspective and, to be successful, perspectives from all parts of the system must be considered. I believe an open and collaborative process is critically important to the development of disability process improvements. To that end, members of my staff and I visited our regional offices, field offices, hearing offices, and State Disability Determination Services, and private disability insurers to identify and discuss possible improvements to the current process.

Finally, a number of organizations provided written recommendations for changing the disability process. Most recently, the Social Security Advisory Board issued a report prepared by outside experts making recommendations for process change. My approach for changing the disability process was developed after a careful review of these discussions and written recommendations. As we move ahead, I look forward to working within the Administration and with Congress, as well as interested organizations and advocacy groups. I would now like to highlight some of the major and recurring recommendations made by these various parties.

The need for additional resources to eliminate the backlog and reduce the lengthy processing time was a common theme. This important issue is being addressed through my Service Delivery Plan, starting with the President’s FY 2004 budget submission which is currently before Congress. Another important and often heard concern was the necessity of improving the quality of the administrative record. DDSs expressed concerns about receiving incomplete applications from the field office; ALJs expressed concerns about the quality of the adjudicated record they receive and emphasized the extensive pre-hearing work required to thoroughly and adequately present the case for their consideration. In addition, the number of remands by the Appeals Council and the Federal Courts make clear the need for fully documenting the administrative hearing record.

Applying policy consistently in terms of: 1) the DDS decision and ALJ decision; 2) variations among state DDSs; and 3) variations among individual ALJs—was of great concern. Concerns related to the effectiveness of the existing regional quality control reviews and ALJ peer review were also expressed. Staff from the Judicial Conference expressed strong concern that the process assure quality prior to the appeal of cases to the Federal Courts.

ALJs and claimant advocacy and claimant representative organizations strongly recommended retaining the de novo hearing before an ALJ. Department of Justice litigators and the Judicial Conference stressed the importance of timely case retrieval, transcription, and transmission. Early screening and analysis of cases to make expedited decisions for clear cases of disability was emphasized time and again as was the need to remove barriers to returning to work.

My approach for disability process improvement is designed to address these concerns. It incorporates some of the significant features of the current disability process. For example, initial claims for disability will continue to be handled by SSA’s field offices. The State Disability Determination Services will continue to adjudicate claims for benefits, and Administrative Law Judges will continue to conduct hearings and issue decisions. My approach envisions some significant differences.

I intend to propose a quick decision step at the very earliest stages of the claims process for people who are obviously disabled. Cases will be sorted based on disabling conditions for early identification and expedited action.

Examples of such claimants would be those with ALS, aggressive cancers, and end-stage renal disease. Once a disability claim has been completed at an SSA field office, these Quick Decision claims would be adjudicated in Regional Expert Review Units across the country, without going to a State Disability Determination Service. This approach would have the two-fold benefit of allowing the claimant to receive a decision as soon as possible, and allowing the State DDSs to devote resources to more complex claims.

Centralized medical expertise within the Regional Expert Review Units would be available to disability decision makers at all levels, including the DDSs and the Office of Hearings and Appeals (OHA). These units would be organized around clinical specialties such as musculoskeletal, neurological, cardiac, and psychiatric. Most of these units would be established in SSA’s regional offices.

The initial claims not adjudicated through the Quick Decision process would be decided by the DDSs. However, I would also propose some changes in the initial claims process that would require changes in the way DDSs are operating. An in-line quality review process managed by the DDSs and a centralized quality control unit would replace the current SSA quality control system. I believe a shift to in-line quality review would provide greater opportunities for identifying problem areas and implementing corrective actions and related training. The Disability Prototype would be terminated and the DDS Reconsideration step would be eliminated. Medical expertise would be provided to the DDSs by the Regional Expert Review units that I described earlier.

State DDS examiners would be required to fully document and explain the basis for their determination. More complete documentation should result in more accurate initial decisions. The increased time required to accomplish this would be supported by redirecting DDS resources freed up by the Quick Decision cases being handled by the expert units, the elimination of the Reconsideration step, and the shift in medical expertise responsibilities to the regional units.

A Reviewing Official (RO) position would be created to evaluate claims at the next stage of the process. If a claimant files a request for review of the DDS determination, the claim would be reviewed by an SSA Reviewing Official. The RO, who would be an attorney, would be authorized to issue an allowance decision or to concur in the DDS denial of the claim. If the claim is not allowed by the RO, the RO will prepare either a Recommended Disallowance or a Pre-Hearing Report. A Recommended Disallowance would be prepared if the RO believes that the evidence in the record shows that the claimant is ineligible for benefits. It would set forth in detail the reasons the claim should be denied. A Pre-Hearing Report would be prepared if the RO believes that the evidence in the record is insufficient to show that the claimant is eligible for benefits but also fails to show that the claimant is ineligible for benefits. The report would outline the evidence needed to fully support the claim. Disparity in decisions at the DDS level has been a long-standing issue and the SSA Reviewing Official and creation of Regional Expert Medical Units would promote consistency of decisions at an earlier stage in the process.

If requested by a claimant whose claim has been denied by an RO, an ALJ would conduct a de novo administrative hearing. The record would be closed following the ALJ hearing. If, following the conclusion of the hearing, the ALJ determines that a claim accompanied by a Recommended Disallowance should be allowed, the ALJ would describe in detail in the written opinion the basis for rejecting the RO’s Recommended Disallowance. If, following the conclusion of the hearing, the ALJ determines that a claim accompanied by a Pre-Hearing Report should be allowed, the ALJ would describe the evidence gathered during the hearing that responds to the description of the evidence needed to successfully support the claim contained in the Pre-Hearing Report.

Because of the consistent finding that the Appeals Council review adds processing time and generally supports the ALJ decision, the Appeals Council stage of the current process would be eliminated. Quality control for disability claims would be centralized with end-of-line reviews and ALJ oversight. If an ALJ decision is not reviewed by the centralized quality control staff, the decision of the ALJ will become a final agency action. If the centralized quality control review disagrees with an allowance or disallowance determination made by an ALJ, the claim would be referred to an Oversight Panel for determination of the claim. The Oversight Panel would consist of two Administrative Law Judges and one Administrative Appeals Judge. If the Oversight Panel affirms the ALJ’s decision, it becomes the final agency action. If the Panel reverses the ALJ’s decision, the oversight Panel decision becomes the final agency action. As is currently the case, claimants would be able to appeal any final agency action to a Federal Court.

At the same time these changes are being implemented to improve the process, we plan to conduct several demonstration projects aimed at helping people with disabilities return to work. These projects would support the President’s New Freedom Initiative and provide work incentives and opportunities earlier in the process.

Early Intervention demonstration projects will provide medical and cash benefits and employment supports to Disability Insurance (DI) applicants who have impairments reasonably presumed to be disabling and elect to pursue work rather than proceeding through the disability determination process. Temporary Allowance demonstration projects will provide immediate cash and medical benefits for a specified period (12-24 months) to applicants who are highly likely to benefit from aggressive medical care. Interim Medical Benefits demonstration projects will provide health insurance coverage to certain applicants throughout the disability determination process. Eligible applicants will be those without such insurance whose medical condition is likely to improve with medical treatment or where consistent, treating source evidence will be necessary to enable SSA to make a benefit eligibility determination. Ongoing Employment Supports to assist beneficiaries to obtain and sustain employment will be tested, including a Benefit Offset demonstration to test to effects of allowing DI beneficiaries to work without total loss of benefits by reducing their monthly benefit $1 for every $2 of earnings above a specified level and Ongoing Medical Benefits demonstration to test the effects of providing ongoing health insurance coverage to beneficiaries who wish to work but have no other affordable access to health insurance.

I believe these changes and demonstrations will address the major concerns I highlighted earlier. I also believe they offer a number of important improvements: People who are obviously disabled will receive quick decisions.

Adjudicative accountability will be reinforced at every step in the process.

Processing time will be reduced by at least 25%.

Decisional consistency and accuracy will be increased.

Barriers for those who can and want to work would be removed.

Describing my approach for improving the process is the first step of what I believe must be—and will work to make—a collaborative process. I will work within the Administration, with Congress, the State Disability Determination Services and interested organizations and advocacy groups before putting pen to paper to write regulations. As I said earlier, and I say again that to be successful, perspectives from all parts of the system must be considered.

Later today, I will conduct a briefing for Congressional staff of the Ways and Means and Senate Finance Committees. I will also brief SSA and DDS management. In addition, next week I will provide a video tape of the management briefing describing my approach for improvement to all SSA regional, field, and hearing offices, State Disability Determination Services, and headquarters and regional office employees involved in the disability program. Tomorrow, I will be conducting briefings for representatives of SSA employee unions and interested organizations and advocacy groups, and I will schedule meetings to provide an opportunity for those representatives to express their views and provide assistance in working through details, as the final package of process improvements is fully developed.

I believe that if we work together, we will create a disability system that responds to the challenge inherent in the President’s questions. We will look beyond the status quo to the possibility of what can be. We will achieve our ultimate goal of providing accurate, timely service for the American people.

Appendix II: Comments from the Social Security Administration

GAO Comments

1. We maintain that our report fully and fairly describes SSA’s progress in analyzing and addressing the underlying causes of inconsistent decisions between state DDS examiners and ALJs. Our research included an extensive review of agency documentation and interviews with SSA officials, as well as stakeholder groups for adjudicators and claimant representatives, to develop a complete understanding of the agency’s efforts to assess and improve the consistency of decisions between adjudication levels. Also, in agreement with our requestor, we sought to expand the review to include SSA’s new approach to improving its disability programs, so that we could provide the Congress with an understanding of how SSA’s future plans may help to address this issue. 2. We provided information on the various reviews and analyses of disability decisions to assess the consistency of decisions between adjudication levels conducted by SSA over the last 10 years, but none of these reviews have clearly identified the causes of inconsistency in decisions between adjudication levels.

3.

Our report has not overlooked the data cited by SSA. Nevertheless, our conclusion about the improvement in consistency between levels indicated by the data is not as optimistic as SSA’s because of weaknesses in SSA’s assessments. As we reported, for 10 years SSA’s analysis of the quality reviewers’ assessment of ALJ cases has been limited to calculating ALJ support rates. SSA has not used available data to determine the potential areas of inconsistency between levels nor the extent to which changes in the ALJ support rate are related to improvements in consistency of decisions between adjudication levels. SSA’s assessment only provides a general indication of overall changes in consistency at one adjudication level. 4. Our report recognizes that SSA’s disability decision-making process is complex. Because of this complexity, we believe that multivariate analysis is an appropriate assessment tool that would allow SSA to assess the effect of multiple factors. In recommending this sophisticated tool, we were careful not to imply that causes and effects of inconsistent decision making can be established with certainty. However, we believe that such an analysis will help SSA understand the relative importance of the variety of factors that affect its decision- making process. After identifying areas of inconsistency, SSA can target these areas with in-depth case analyses to pinpoint the causes of inconsistency and develop a more effective strategy for addressing inconsistency. On the basis of our review of SSA’s analyses to date, we do not agree with the implications of SSA’s comments that it has a solid understanding of how certain variables influence disability decision making, and therefore does not need to conduct additional, more sophisticated analyses. 5. We agree with SSA that the proportion of allowances made at each level can provide some insight into the allowance rate dynamic. However, as we reported, we do not believe that it can serve as a reliable indicator of the agency’s progress in achieving more consistent decisions between the DDS and OHA levels. The allowance data provided by SSA simply show that the relative proportion of allowances made at the DDS level increased in comparison with the OHA level, but SSA has not performed any additional analysis to show that these changes have any relationship to improved consistency in decision making between the two adjudication levels. Additional analysis is needed because a myriad of factors, such as changes in the economy, can affect allowance rates. Although SSA claims that over this period of time the economy has been “relatively stable,” without performing any additional analysis it cannot eliminate changes in the economy or demographics of claimants as an influence on the allowance rates at each level. In addition, SSA has not analyzed how other factors, such as changes in productivity and total number of decisions made at each level, may be influencing the allowance data. 6. The allowance rate data provided by SSA in its comments is very similar to that provided by SSA earlier to us and included in our report in figure 2. The figures we reported for the proportion of allowances made by the DDS and OHA levels for fiscal years 1997 and 1998 vary in comparison with those provided by SSA by one percentage point. We have not changed the figures in our report because we believe that these slight differences simply reflect that we reported data based upon fiscal, not calendar, years. 7. In our report, our statements that SSA has not made changes as a result of findings from its reviews were specifically related to SSA’s ALJ pre-effectuation review. We included information on this review because it was part of SSA’s process unification initiative and was intended to identify policy and training areas associated with inconsistent decisions between adjudication levels. During our review, we were told by an SSA official that the ALJ pre-effectuation review was not successful at identifying new areas of inconsistency to be addressed by SSA. In its comments, SSA cites a review unrelated to assessing the inconsistency of decisions between levels, the ALJ peer review, to assert that it has used reviews to identify training issues to improve the quality of decisions. The lack of success with the ALJ pre- effectuation review—along with other findings showing a limited understanding of the cause of inconsistency—supports our recommendations to SSA to perform additional analysis and to clarify guidance and provide mandatory training to address any identified causes of inconsistency between adjudication levels. 8. We applaud SSA’s plans to use the electronic disability system to capture critical management information to address decisional variance or inconsistency, which could provide a wealth of useful information for the agency. We have adjusted our report’s text to reflect this additional purpose. We continue to believe that SSA should not wait for the development of this system, but should proceed to perform multivariate analysis, using available data from its biennial case reviews, to start identifying areas of potential inconsistency between adjudication levels. 9. We applaud SSA’s deep commitment to improving the disability decision-making process, but believe that additional efforts to understand the causes of potential inconsistencies in decision making would help to inform the design of the Commissioner’s new approach and should, therefore, be undertaken immediately. 10. We generally agree with the technical comments provided and changed the text accordingly.

Appendix III: GAO Contacts and Staff Acknowledgments

GAO Contacts

Staff Acknowledgments

In addition to the individuals mentioned above, the following staff members made major contributions to this report: Michael Morris, Corinna Nicolaou, Walter Vance, and Rebecca Woiwode. Douglas Sloane provided assistance with methodological issues, and Daniel Schwimer provided legal support.