Electronic Disability Claims Processing: SSA Is Proceeding with Its Accelerated Systems Initiative but Needs to Address Operational Issues
Released: 2005-10-24T00:00:00

Letter

Background

SSA’s Disability Insurance and Supplemental Security Income programs are the nation’s largest providers of federal income assistance to disabled individuals, with the agency making payments of approximately $113 billion to more than 14 million beneficiaries and their families in 2004. Yet, over the years, it has become more challenging for the agency to ensure an acceptable level of service—in terms of both the quality and the timeliness of its support to these individuals. In January 2003, we designated disability benefits programs across the federal government as high risk—in need of urgent attention and transformation.

The process through which SSA approves or denies disability benefits is complex and involves multiple partners at both the federal and state levels in determining a claimant’s eligibility. SSA’s 1,300 field offices are the initial points of contact for individuals applying for benefits. SSA also depends on 54 state DDS offices to provide crucial support to the claims process through their role in determining an individual’s medical eligibility for disability benefits. DDSs make initial determinations regarding disability claims in accordance with federal regulations and policies; the federal government reimburses 100 percent of all costs incurred by states to make disability determinations. Physicians and other members of the medical community provide the DDSs with medical evidence to help them evaluate disability claims. When disability claims have been denied by the DDSs, claimants can appeal to SSA’s OHA.

The process begins when individuals apply for disability benefits at an SSA field office, where determinations are made about whether they meet nonmedical criteria for eligibility. If the claimant is eligible, the field office forwards the application to the appropriate state DDS, where a disability examiner collects the necessary medical evidence to make the initial determination of whether the claimant’s condition meets the definition of disability. Once the claimant’s medical eligibility is determined, the DDS returns the claim folder to SSA for final processing.

A claimant who is initially denied benefits can ask the DDS to reconsider its determination. If the DDS denies the claim again, the claimant can request a hearing before a federal administrative law judge at an SSA hearings office and, if still dissatisfied, can request a review of the claim by SSA’s Appeals Council. Upon exhausting these administrative remedies, the claimant may file a complaint in federal district court. Each level of appeal involves multistep procedures for collecting evidence, reviewing information, and making the decision. Many individuals who appeal the initial determination on their claims will wait a year or longer—perhaps up to 3 years—for a final decision.

To address concerns regarding the program’s efficiency, in 1992, SSA initiated its Modernized Disability System project, intending to redesign the disability claims process emphasizing the use of automation to achieve an electronic (paperless) processing capability. This project, which in 1994 was renamed the Reengineered Disability System, was to automate the entire disability claims process—from the initial claims intake in the field office to the gathering and evaluation of medical evidence by the state DDSs, to payment by the field office or processing center. The system also was intended to automate the handling of appeals by SSA’s hearings offices. However, as our prior work noted, SSA encountered performance and other problems during its initial pilot testing of the system and, after spending more than $71 million, suspended this project in 1999.

In August 2000, SSA renewed its commitment to developing an electronic disability system by the end of 2005. The agency worked on this initiative through the spring of 2002, at which time the Commissioner of Social Security announced an accelerated electronic disability initiative— AeDib—to more quickly move to an automated process. Under the accelerated strategy, the agency planned to begin implementing its electronic disability system by January 2004. SSA anticipated that the electronic disability system would enable the disability offices to achieve processing efficiencies, improve data completeness, reduce keying errors, and save time and money. With technologically enhanced claims processing offices, the agency projected that it could realize benefits of more than $1 billion—at an estimated cost of approximately $900 million—over the 10- year life of the initiative. SSA reported actual AeDib costs of approximately $215 million through fiscal year 2004 for planning, hardware and software acquisition, maintenance, and personnel.

The AeDib strategy focuses on developing the capability to electronically process claimant information and large volumes of medical images, files, and other documents that are currently maintained in paper folders. Stored in electronic folders, this information could then be accessed, viewed, and shared among the disability claims processing offices. The initiative to achieve this electronic capability involves five key projects: an Electronic Disability Collect System that would provide the capability for SSA field offices to capture electronically, in fixed data fields, information about a claimant’s disability that previously had been contained on paper disability forms (structured data) and to store it in databases for later use by the SSA and DDS offices responsible for processing disability claims; a Document Management Architecture to provide a data repository and scanning and imaging capabilities that would allow unstructured claimant and medical data, such as images or information not found in fixed data fields (e.g., a hospital report, doctors’ notes, or an x-ray report), to be stored, indexed, and shared among the disability processing offices; Internet applications to enable the public to submit disability claims and medical information to SSA via the Internet (all data keyed into the Internet applications would be transmitted directly into the Electronic Disability Collect System); a systems migration and electronic folder software interface to position DDS offices to operate on a common IBM-series hardware platform and enhance their existing claims systems to process the electronic claims information and to enable the DDS systems to access information in the electronic folder; and a Case Processing Management System that would interface with the electronic folder and enable OHA’s staff to track, manage, and complete case-related tasks electronically.

According to SSA, the Electronic Disability Collect System and the Document Management Architecture are the two fundamental components needed to create the electronic disability folder. Via their claims processing systems, SSA and DDS users would be able to access and pull the structured and unstructured claimant data into appropriate computer screens, organized as electronic folders of information. The agency’s electronic disability claims processing system is depicted in figure 1.

By mid-January 2004, SSA had implemented all planned releases of the Electronic Disability Collect System and had completed and placed into production Internet applications to aid claimants in filing online for disability benefits and services. It also had enhanced the DDSs’ claims processing systems by migrating and upgrading hardware to allow these offices to operate on a common IBM-series platform and by upgrading the claims processing software in all but 3 state DDS offices that used the standard disability claims processing systems. In addition, SSA had begun pilot testing OHA’s Case Processing Management System in a standalone environment at five sites. Further, the agency was pilot testing the Document Management Architecture in three state DDS locations—North Carolina, Illinois, and California. However, it had not yet implemented the Document Management Architecture repository and scanning and imaging capabilities and related DDS software enhancements or the software to enable DDS and OHA systems to interface with the electronic folders. SSA began its national rollout of these remaining system components at the Mississippi DDS on January 26, 2004.

When we last reported on the initiative in late March 2004, SSA was proceeding with its implementation of its electronic disability system. However, our work had noted that the agency’s strategy for developing the system components involved risks that threatened the success of the project. For example, we determined that the agency (1) had begun the national rollout without conducting testing that was adequate to evaluate the performance of all system components collectively, (2) could not provide evidence that it was consistently applying established procedures to guide the AeDib software development or had developed risk mitigation strategies, (3) had not validated its analysis to ensure the reasonableness of estimated AeDib costs and benefits, and (4) had not articulated a comprehensive plan for ensuring that state DDSs’ concerns about the initiative were addressed. In view of the risks and the technological complexity, scope, and size of the initiative, we had recommended that the Commissioner of Social Security, before continuing with the national rollout of AeDib, ensure that all critical problems identified in pilot testing of the electronic disability system were resolved and that end-to-end testing of the interrelated systems was performed, ensure that the software that had been developed was approved and that the systems had been certified for production, establish a revised time frame for and expedite actions toward finalizing AeDib risk mitigation strategies, validate all AeDib cost and benefit estimates, and implement a communications plan to clearly and comprehensively convey SSA’s approach for effectively addressing disability stakeholders’ and users’ concerns and ensuring their full involvement in the AeDib initiative.

SSA Is Meeting Its Overall AeDib Implementation Schedule, but Operational and Other Concerns Could Impact the Success of the Initiative

SSA is proceeding with a national rollout of its electronic disability system and has generally met its schedule for implementing the remaining key components—the Document Management Architecture and the electronic folder interface software—that are required to process an entire disability case electronically. Nonetheless, the agency has considerable work to accomplish before it will be effectively positioned to fully process all disability claims in an electronic environment. Among the critical tasks that remain are certifying all state DDS offices and OHA sites to electronically process claims and addressing operational and other concerns that threaten to undermine the reliability and use of the system. Until SSA has effectively addressed these matters, it remains uncertain when and to what degree the agency will realize the full benefits of its electronic processing capability.

AeDib Implementation Schedule Was Generally Met through June 2005

The AeDib implementation schedule had called for all state DDSs and OHA sites to be equipped with the electronic disability claims processing capability by June 27, 2005, and October 3, 2005, respectively. Since beginning the national rollout of the Document Management Architecture and related DDS software enhancements and the electronic folder interface software in late January 2004, the agency has largely met its implementation schedule. As of late June 2005, SSA had fully or partially implemented the electronic disability system in 53 of the 54 state DDS offices and in 85 of the 144 OHA sites, as planned. The agency reported that it expected to finish implementing the electronic disability system in the one remaining DDS—New York—in October 2005.

SSA officials attributed the 4-month delay in the planned implementation at the New York DDS to the need for additional time to interface the electronic disability system with that state’s existing claims processing capabilities. New York and Nebraska are the only two DDSs in which the states’ claims processing capabilities are not supported by the common hardware platform that the majority of DDSs use and that have developed and rely on disability claims processing software that is unique to their processing environments. As a result of New York’s efforts to develop and test an electronic disability claims process, it had achieved a level of electronic processing, including the capability to scan medical evidence into its system, prior to SSA’s completion of the electronic disability system. SSA and New York DDS officials agreed to interface the new electronic disability system with portions of that state’s existing claims processing capabilities.

In addition, SSA officials reported, as of early July 2005, that they expected to meet the scheduled completion date of October 3, 2005, for 115 of the 144 OHA sites. They stated that the agency expected to complete implementation of the electronic disability system at the remaining 29 OHA sites approximately 1 month later, in November 2005. According to agency officials, 10 of the 29 sites support claims that are processed by the New York DDS. The agency delayed implementation at these sites in order to be more in step with New York’s revised implementation schedule and with anticipated time frames for when the DDS will be positioned to process disability cases using the electronic folder. Regarding the remaining 19 sites, officials explained that the agency did not wish to train staff and provide the electronic folder capability to OHA sites too far in advance of when these offices expected to receive electronic cases from the DDSs, believing that too much lag time between training and actual use of the system could result in the staffs’ losing some of the knowledge and skills they need to process cases electronically.

Operational and Other Concerns Could Impact the Electronic Disability System’s Reliability and Use

Although the roll out of the electronic disability system is moving toward completion, the agency still has considerable work to accomplish before it will be effectively positioned to process all disability claims in a fully electronic environment. After implementing the electronic system, each DDS must undergo an assessment of the quality and accuracy of its electronic processing capabilities and must be certified by SSA to use the electronic folder as its official disability claims record. This assessment, referred to as the Independence Day Assessment, is intended to validate that an office is ready to process 100 percent of the initial disability claims and any reconsiderations that it receives in the electronic environment and that the electronic folder can serve as the official disability claims record. According to SSA Operation’s staff, the assessment involves examining the disability office’s operations and claims processing tasks to ensure that (1) the business process (e.g., the way in which the disability claims office is organized to do its work) and the electronic processing environment are compatible; (2) existing claims processing systems have the necessary functionality to process electronic folders; and (3) staff can, when using the electronic disability system, produce complete information that equals what is contained in the paper folders.

As of early July 2005, SSA reported that only three state DDSs—Mississippi, Illinois, and Hawaii—had completed assessments and been certified to process all initial disability claims in a paperless electronic environment in which the electronic folder is recognized as the official disability claims record. SSA reported that it had certified Mississippi—the first state under the national rollout—to process all of its initial disability claims electronically by February 2005; the agency further reported that it had completed certifications in June 2005 for Illinois—one of the three states that had participated in a pilot test of the Document Management Architecture in 2003—and for Hawaii, one of the smallest states (consisting of 15 disability claims examiners), which completed its system’s implementation in February 2005.

In discussing Mississippi’s certification, the DDS director stated that the approximately 1-year time frame between the implementation and the certification of the office’s system had been devoted to such tasks as updating and testing software versions in order to give the office the full complement of functionality that it would need to use the electronic folder and to ensuring that the office’s business processes effectively supported electronic processing by, for example, familiarizing staff with the electronic capabilities, training them in using the capabilities, and testing the use of scanning equipment in the office. The director added that, since Mississippi was the first state to be assessed and certified, SSA had reviewed a substantially larger number of disability cases (approximately 300) than it intends to review in the assessments of other states using the same claims processing software.

SSA officials said that, following Mississippi’s certification, the Commissioner of Social Security had placed a moratorium on any additional certifications pending SSA’s review of the assessment that had been undertaken in that state. They said that the commissioner had wanted to capture lessons learned from Mississippi’s assessment to identify any needed improvements in the assessment process and to ensure that any business or user concerns about the assessments were resolved before they applied the process broadly across all DDSs and OHA sites. For example, according to the officials, they learned that a smaller sample of disability cases could be examined in subsequent offices using the same software as Mississippi’s without diminishing the integrity of the assessment and the related certification.

As of July 2005, SSA officials told us that they had resumed the assessments and that the agency’s plans called for a total of seven state DDSs to be certified to process claims electronically by the end of fiscal year 2005. However, as indicated in table 1, not all of the 54 state DDSs are expected to be certified to process initial disability claims and to use the electronic folder as the official record until January 2007.

According to OHA’s deputy director, SSA expects to certify each OHA site shortly after certifying the corresponding DDS office. The official noted that because all OHA sites will rely on the same standard system (the Case Processing Management System), their certification process is expected to be less complicated than the process for the DDSs. Until they are certified, offices that have already implemented the electronic disability system are expected to maintain paper folders as well as electronic ones for any initial disability cases that they process electronically. The paper folders will continue to serve as the official records for these cases.

Operational Concerns Exist

Even as the agency proceeds in certifying states’ electronic capabilities, however, operational concerns associated with the electronic disability system could undermine its reliability and use. Officials in seven of the nine DDS offices that we contacted (California, Delaware, Florida, Illinois, Mississippi, North Carolina, and South Carolina) stated that operational problems they had encountered while using the electronic disability system had affected its performance and raised doubts about its reliability in supporting their processing needs.

DDS officials stated, for example, that as SSA had brought the system online at the different DDS offices and/or added new software to enhance functionality, their staffs had encountered various operating problems that affected the performance of the system. They stated that their offices had experienced problems such as computer screen freezes, system slowdowns, and system access issues—all of which had disrupted the offices’ processing of claims. They described these problems as unpredictable and random because they did not always occur consistently among all of the offices using the same claims processing software or even among examiners in a particular office. The officials added that although SSA has been able to resolve many of the problems that affected their ability to process claims, additional instances of screen freezes, system slowdowns, and access issues have continued to occur throughout the system’s implementation.

According to the manager of the South Carolina DDS, which implemented the electronic disability system in March 2004, that office’s productivity had been adversely affected by system slowdowns that resulted from having inadequate network bandwidth to support its disability claims processing operations. In July 2005, the manager stated that SSA had recently made modifications to the office’s network architecture and had increased its bandwidth by installing two additional communications lines; at that time, the office was in the process of testing these enhancements. The manager said that all of the office’s disability claims examiners had begun processing all initial disability claims electronically and that, as a result of the enhancements, they expected the office’s claims processing efficiency and productivity to increase.

In addition, DDS officials in six offices (California, Delaware, Florida, Mississippi, North Carolina, and South Carolina) reported problems with the electronic forms that SSA had installed to facilitate the processing of disability claims. The officials explained that, while using the electronic forms, disability claims examiners had experienced slow system response times or system freezes that had contributed to increased claims processing times. Officials in California stated that they had stopped using the electronic forms as a result of the problems that their staff had encountered and instead were continuing to rely on paper forms. In a February 2005 survey, the National Council of Disability Determination Directors found that 22 DDSs had experienced problems with the electronic forms and had reported that the slow pace involved in loading and using these forms was barely tolerable. Further, based on its May 2005 quarterly evaluation of the electronic disability system, SSA reported that one of the systems-related problems that disability processing offices identified most frequently was the slow response times and lack of user- friendliness of the electronic forms.

Further, officials in five of the DDSs that we contacted among those that had implemented the electronic disability system (California, Illinois, Mississippi, North Carolina, and South Carolina) stated that their disability examiners faced difficulties in reading medical evidence on screen and performing certain case development or adjudication tasks because the size of the computer monitors that they had been provided to process medical evidence had proven inadequate. Users of the system—including managers, claims examiners, and medical consultants—reported difficulty with simultaneously viewing two documents on their monitors; some staff reported that they had resorted to printing out or toggling between documents to avoid using the split screen to review them. As a result of the inadequacies associated with using the existing monitors, the users reported that they needed longer periods of time to perform certain claims adjudication tasks and that they had been unable to complete as many cases per day as they could before they had the electronic disability system.

Beyond these concerns, officials in four of the DDSs that we contacted told us that, although their electronic capability had been implemented, they had not been provided certain software enhancements that they needed to fully process a claim electronically and that were critical to improving the efficiency of their offices’ claims processing capabilities. Specifically, Florida officials stated that their claims processing software did not provide the capability to electronically notify staff of the actions that were required to process a claim. As a result, the staff had to expend additional time notifying each other of actions needed to process the claim by, for example, sending e-mail notices. In addition, the director of the North Carolina DDS stated that that office lacked the necessary software to enable staff to electronically send claims files to SSA’s Disability Quality Branch, which is responsible for conducting quality reviews of the accuracy of DDSs’ disability determinations. The director added that staff could not yet respond electronically to SSA components that requested information on a particular claim that had been processed in their office. Further, in Nebraska, the DDS manager stated that that office was in need of additional software modifications to provide the functionality required to electronically import all required documents into the electronic folder. Finally, the manager of the California DDS stated that his office had lacked the capability to electronically process nonmedical claims data that were required to be included in the disability claims folders, such as a claimant’s work history. The manager stated that they also had lacked the functionality to electronically refer claims to medical consultants for consultative examinations.

In light of the operational and other concerns that have been encountered in using the electronic disability system, coupled with factors such as having to concurrently maintain paper and electronic claims folders while awaiting certification, both SSA and DDS managers acknowledged that the DDS offices had exercised wide discretion in their use of the new system. The President of the National Council of Disability Determination Directors stated that two key factors had affected some DDSs’ decisions about ramping up to full use of the system: (1) concerns about a drop in productivity in a fully electronic environment and (2) the instability of the electronic disability processing environment, particularly in terms of system performance and software reliability. In this regard, officials in eight DDS offices that we contacted—all of which had implemented the electronic disability system by the end of June—reported varying levels of usage, as shown in table 2.

Regarding the extent of their usage, managers in the Mississippi and Illinois DDSs acknowledged that problems had been encountered following their implementation of the electronic disability system but stated that they had nonetheless chosen to expedite efforts to achieve full electronic processing of disability claims in an attempt to minimize the inefficiencies associated with having to maintain both paper and electronic disability folders. Further, to help bring their states to full electronic processing, the managers of the Mississippi and Illinois DDSs stated that they had expended additional resources on overtime pay to disability examiners and on additional support from the software vendor to alleviate and/or establish workarounds for the operational problems that their examiners had encountered. While Mississippi officials said that they were unable to provide a dollar amount for their overtime usage, an Illinois DDS official provided documentation indicating that between September 2003 and May 2005, that office had spent over $2 million on overtime, which assisted them in processing disability claims in the electronic environment.

However, DDS managers in several other offices that we contacted stated that, as a result of the problems with and the resulting unpredictability of the electronic disability system, they had been reluctant to bring the system to full use. They expressed reluctance to increase their use of the system until a more reliable level of performance has been sustained, stating generally that the current problems with the system could hamper their ability to maintain their productivity levels.

For example, officials in the California DDS told us that the electronic disability system had not been used to fully process any of the approximately 450,000 initial claims that the office had received since it had implemented the system in October 2003. The officials stated that they had chosen not to ramp up the system until it proved to be more stable and all critical processing capabilities had been delivered. The manager believed that trying to use the system to process all of the office’s initial disability claims before the problems affecting their system’s operations were resolved and before all critical processing capabilities were delivered would prevent the office from maintaining its productivity levels. In mid- July 2005, California DDS officials stated that the vendor supporting their disability claims processing system had recently provided a software enhancement that gave them the capability to fully process claims and that the agency would begin a phased increase in the number of initial disability claims it would process in the electronic environment. According to the manager, each examiner would initially be assigned one disability case per week to process electronically.

In addition, in Florida, DDS officials stated that they had limited the number of claims being processed by their disability examiners until SSA was able to enhance their software to achieve better user efficiency. As a result, at the time of our review, only 109 of the office’s 487 disability examiners were using the electronic system to process cases, and only about 4 to 5 percent of initial disability claims were being processed electronically. Further, while the Nebraska DDS’s electronic capabilities were implemented in late June 2005, officials in that office stated that they did not plan to ramp up their use of the system until about September 2005. They explained that, lacking the software modifications required to electronically import all documents into the electronic folder, the office was reluctant to increase its use of the electronic disability system. They stated that doing so would require them to commit additional resources to scan documents that could not automatically be entered into the electronic system. The officials added that they expect to receive additional software modifications that they need to improve the efficiency of the office’s electronic processing capability in the September 2005 time frame.

Given the current status of the electronic disability system, neither SSA nor the state offices had yet been able to effectively assess or quantify benefits resulting from its use. All of the managers of the DDS offices that we contacted stated that they saw the potential for realizing substantial claims processing improvements from using the system; nonetheless, these managers—including the managers of the Mississippi and Illinois DDSs— stated that it was too early to determine whether and to what extent the electronic disability system would contribute to processing improvements in their offices. In their view, the system had not yet reached a level of maturity where it was feasible to quantify the benefits of its use, due in large measure to factors such as (1) the learning curve associated with using the system; (2) current inefficiencies involved in having to maintain paper folders until an office is certified to electronically process claims; (3) certain DDSs’ decisions to not fully utilize the system until further problem resolution; and (4) certain offices’ use of additional resources, such as overtime and temporary hires, to support their processing of claims following the system’s implementation. The managers added that processing claims electronically had thus far taken longer and consumed more resources than before the electronic system was implemented. In addition, because of ongoing system implementation in the OHA sites, along with the normal processing delays associated with bringing disability claims to the appeals stage, these sites had not yet accrued enough experience in using the electronic folder to make a reasonable assessment of processing improvements.

In speaking to the concerns that were raised about the reliability and use of the electronic disability system, SSA officials acknowledged the problems that had been identified by the DDSs, and that as a result, current use of the system among those offices varied considerably. However, these officials said they believed that the majority of the DDS offices would be able to bring their systems to full use with only a minimum of complications; they viewed California’s concerns, in particular, as not having been representative of other states’ experiences. Nonetheless, the officials said that the agency had initiated a number of measures to address the problems that had been encountered in using the system. For example, they stated that the agency had established a new help desk to more readily support those offices experiencing specific hardware and software problems while using the electronic disability system. In addition, they stated that SSA had assembled a work group to examine the DDSs’ use of the electronic forms, with the intent of determining whether a more suitable commercial-off-the-shelf product was available that could address the problems currently being encountered with these forms. Regarding the size of the computer monitors that disability examiners use, the officials stated that the agency planned to conduct a pilot test to address concerns with and identify a solution for ensuring that users have the monitors they need.

SSA’s actions to address the outstanding concerns with its electronic disability system represent a positive step toward achieving success in the use of the new system. However, as of July 2005, the agency did not have an overall strategy—articulating milestones, resources, and priorities—to guide its efforts in efficiently and effectively resolving the operational problems and system limitations being experienced with the electronic disability system. For example, although the agency had established a work group to explore options for resolving problems being encountered with the electronic forms, it had not yet established plans and a time frame for completing actions to address this concern. In addition, although the agency planned to conduct a pilot test of computer monitors, it did not yet have essential information to determine what type of equipment would best meet the needs of the electronic disability system users or the resources and time that it would need to devote to resolving this matter. Adequately resolving the concerns with and gaining full acceptance and use of its electronic capabilities will be essential to SSA’s achieving a more efficient means of delivering disability benefits payments to its increasing beneficiary population.

The Lack of Continuity of Operations Plans for the Electronic Disability System Places Claims Processing at Increased Risk

In addition to ensuring the immediate availability of the electronic disability claims processing system by preventing operational problems that could impact its performance and use, it is essential that SSA and the DDSs have plans for mutually ensuring the continuity of this vital disability benefits service in emergency situations. Federal law and guidance require that agencies develop plans for dealing with emergency situations involving maintaining services and protecting vital assets that could result from disruptions, such as localized shutdowns due to severe weather conditions, building-level emergencies, or terrorist attacks. Moreover, this guidance notes, a key element in developing a viable continuity capability is identifying interdependencies among agencies that support the performance of essential functions and ensuring the development of complementary continuity of operations plans by those agencies that provide information or data integral to the delivery of essential functions. Such planning would include developing and documenting procedures for continued performance of essential functions, identifying alternates to fill key positions in an emergency and delegating decision-making authority, and identifying vital electronic and paper records—along with measures for ensuring their protection and availability.

However, SSA and the DDSs currently lack continuity of operations plans to ensure that the DDS offices could continue to process disability claims in the event of a short- or long-term disruption to the electronic disability system. A September 2004 report, issued by SSA’s Acting Inspector General, noted that the agency’s existing continuity of operations plan did not address the information or the electronic disability claims processing systems managed by the DDSs. The report further noted that, in relying heavily on the DDSs, SSA would lack certainty about the availability of information from these offices in the event of a disaster. Based on its findings, the Acting Inspector General recommended that SSA implement a complete and coordinated continuity of operations plan for the agency.

Officials in the nine DDSs that we contacted further stated that their offices had not developed continuity of operations plans covering the electronic disability claims processing capabilities; yet, in discussing this matter, officials considered such plans to be vital to successfully ensuring the continued processing of disability claims. Officials in the Mississippi DDS stressed, for example, that in the event of a disruption to their system’s communications with SSA’s headquarters computer facilities, disability examiners would be unable to access the Document Management Architecture repository, send or receive faxes via the electronic system, or access the electronic forms they needed to support their work. In addition, they stated that medical examiners would be unable to perform tasks in support of disability determinations.

In discussing this matter, SSA officials acknowledged that their existing plan had not addressed the electronic claims processing functions of the DDSs. They stated that the agency had recently initiated actions to help resolve this limitation by having a contractor develop a business continuity planning strategy. According to the officials, the contractor began work on this strategy in May 2005 and is expected to deliver an initial report in September 2005. However, the officials did not articulate the agency’s specific plans or a time frame for ensuring that its continuity of operations plan addresses the electronic claims processing functions of the DDS offices or for ensuring that these offices develop and implement complementary plans for continuing essential functions to support the disability claims process in an emergency situation.

As SSA moves toward full implementation and use of the electronic disability system, the capability to continue essential electronic disability claims processing functions in any emergency or situation that may disrupt normal operations becomes increasingly important. In view of the fact that three states have already begun using electronic folders as official disability claims records, it is imperative that both SSA and the DDSs have plans that address the state systems’ interdependencies with the electronic disability claims processing components and that include preparations for continuing to provide critical claims processing services in the event of a disaster. Without continuity of operations plans, SSA will lack assurance that it is positioned to successfully sustain the essential delivery of disability benefits during unforeseen circumstances.

SSA’s Actions toward Implementing AeDib Have Addressed Some, but Not All, of Our Prior Recommendations for Improvement

As discussed earlier, in reporting on this initiative in March 2004, we recommended that the agency take measures to reduce the risks associated with its electronic disability strategy before continuing with its national rollout of this capability. These recommendations called for the agency to (1) resolve critical problems that it had identified in pilot testing of the electronic disability system and conduct end-to-end testing of the interrelated system components, (2) ensure that users approved the software being developed and that systems were certified for production, (3) finalize AeDib risk mitigation strategies, (4) validate AeDib cost and benefit estimates, and (5) improve communications with and effectively address the concerns of disability stakeholders and users involved in the initiative.

In proceeding with the implementation of its electronic disability system, SSA has taken actions related to three of the five recommendations. Specifically, SSA officials provided evidence indicating that the agency has taken measures to ensure that users approve new software and that it certifies its systems for production. For instance, we reviewed agency documentation reflecting disability system users’ approval of new software and SSA’s certification of over 50 cases where software was put into production from February 2004 (shortly after the national rollout of the electronic disability system began) through October 2004. By continuing to validate its software and certify its systems, SSA should be able to better ensure that its systems are ready for production and will be acceptable to their end users.

In addition, regarding our recommendation that it validate AeDib cost and benefit estimates, SSA has initiated studies, including quarterly evaluations of the initiative, that could help it assess the electronic disability system’s performance, costs, and processing times. The agency also has plans for conducting post-implementation reviews of the system, which include comparing baseline and current information to evaluate the system’s impact on performance, productivity, and cost—measures that if implemented fully and effectively could help validate AeDib’s costs and benefits.

Further, although the agency has not implemented a communications plan, DDS officials, including the President of the National Council of Disability Determination Directors, told us that SSA had improved its communications with these offices and had made progress in including DDS officials in AeDib decision making. Such action reflects a positive move toward involving stakeholders in the agency’s efforts. Nonetheless, we continue to advocate the importance of having a clear and comprehensive plan for communicating with stakeholders to sustain vital user acceptance and achieve full use of the electronic disability system.

However, SSA did not demonstrate any actions on two of the recommendations. As we previously noted, SSA did not take steps to resolve all of the critical problems that had been identified during pilot testing of the Document Management Architecture or to conduct end-to- end testing of the interrelated electronic disability system components before continuing with the national rollout of this system. Resolving all critical problems and conducting end-to-end testing of the interrelated system components prior to their implementation could have limited the problems that SSA and the DDSs have encountered with the electronic disability system’s operation. In the absence of such testing, as SSA moves to achieve certification and full use of the system, it will be essential that the agency work diligently to identify and alleviate the problems that could impact the successful outcome of this technically complex initiative.

Further, while our earlier work noted that the agency had identified the risks associated with the AeDib initiative and the related automation projects, SSA has not provided any evidence that it has yet completed risk mitigation strategies for these projects. Best practices and federal guidance advocate risk management—including mitigation strategies—to reduce risks and achieve schedule and performance goals. Among the high-level risks identified, SSA noted that the overall availability of the Document Management Architecture might not meet service-level commitments to its users. Because the DDSs could not effectively perform their work if the data repository or document scanning and imaging capabilities provided by the architecture were not available, it is critical that SSA have mitigation strategies in place to reduce this risk and to help ensure that the DDSs can meet their performance goals. In the continued absence of risk mitigation strategies, the agency lacks a critical means of ensuring that it can prevent circumstances that could impede a successful project outcome.

Conclusions

SSA is relying on its electronic disability system to play a vital role in improving service delivery to disabled individuals under its disability programs, and the agency has made considerable progress in implementing this system. However, even as the agency moves closer to achieving full systems implementation, important work still needs to be accomplished to ensure the system’s success. Among the agency’s critical tasks will be certifying that all of the SSA and state DDS offices are prepared to process all disability claims electronically. Yet, a number of DDSs have concerns about the operations and reliability of the electronic disability system, noting, for example, inadequacies in electronic forms and the computer monitors used to view claims information, as well as limitations in electronic processing capabilities—factors that they say have slowed system performance and impeded their productivity, and that have resulted in the levels of system usage varying among the DDSs. Further, as the agency moves to complete the system’s implementation, it will be essential that SSA and the DDSs have plans for mutually ensuring the continuity of this vital disability benefits service in emergency situations. The absence of a defined strategy or plans for ensuring that the electronic disability system will operate and will meet users’ needs as intended could threaten the continued progress and success of this initiative and make it uncertain when the agency will realize the full benefits of the AeDib initiative.

Recommendations for Executive Action

To further reduce the risks to SSA’s progress in successfully achieving its electronic disability claims processing capability, we recommend that the Commissioner of Social Security take the following two actions: develop and implement a strategy that articulates milestones, resources, and priorities for efficiently and effectively resolving problems with the electronic disability system’s operations, including (1) identifying and implementing a solution to improve the use of electronic forms, (2) identifying and implementing a solution to address concerns with existing computer monitors, and (3) ensuring that the DDSs have the necessary software capabilities to fully and efficiently process initial claims in the electronic processing environment; and ensure that the state DDSs develop and implement continuity of operations plans that complement SSA’s plans for continuing essential disability claims processing functions in any emergency or other situation that may disrupt normal operations.

Agency Comments and Our Evaluation

In written comments on a draft of this report, the Commissioner of Social Security expressed concern about our references to the agency’s testing of its electronic disability system and offered additional views regarding our discussion of the state disability determination services’ use of overtime to assist in electronically processing disability claims. In addition, the agency disagreed with one of our recommendations and agreed with the other.

Regarding the testing of its electronic disability system, SSA questioned why our report had concluded—months after the agency’s rollout of the system—that performing end-to-end testing of the interrelated system components was still critical to the initiative’s success. The agency believed that we should delete references to the criticality of such testing for this initiative so as not to lend confusion to and cast doubt on its rollout experience.

In discussing SSA’s decision not to conduct end-to-end testing before rolling out the electronic disability system, our report responds to one of the two stated objectives of our study—to determine the actions that SSA has taken in response to our prior recommendations on the AeDib initiative. In doing so, we did not conclude that it remains critical for SSA to perform end-to-end testing at this stage in the system’s implementation. Rather, in speaking to this issue, we described the agency’s response to our prior recommendation that it conduct end-to-end testing before proceeding with the national roll out, and we emphasized the importance of such testing as a means of limiting the types of problems that DDS officials told us they have encountered with the system’s operation. As our report stresses, in the absence of end-to-end testing, it is essential that SSA remain diligent in identifying and alleviating the problems that could impact the successful outcome of the AeDib initiative as the agency moves to achieve full certification and use of the electronic disability system.

Concerning the DDSs’ reported cost and use of overtime, SSA emphasized its belief that a number of factors other than the electronic disability system had contributed to the Illinois DDS’s increased use of overtime. For example, the agency said that overtime had been used to compensate for the loss of DDS employees that had accepted the state’s offer of early retirement. Based on our discussions with Illinois DDS officials, we understand that the office may not have used overtime solely to support its electronic processing of disability claims. However, as noted in our report, the Illinois officials told us that they had relied on overtime to assist in bringing their office to full electronic processing of disability claims using the new system. We have included language in the report to more clearly reflect this point.

Beyond these points of discussion, SSA disagreed with our recommendation that they develop and implement a strategy that articulates milestones, resources, and priorities for efficiently and effectively resolving problems with the electronic disability system’s operations, including (1) identifying and implementing a solution to improve the use of electronic forms, (2) identifying and implementing a solution to address concerns with existing computer monitors, and (3) ensuring that the DDSs have the necessary software capabilities to fully and efficiently process initial claims in the electronic processing environment. Specifically, the agency stated that it had substantially improved its electronic forms and already has a strategic project plan to address residual issues concerning them. It noted that a work group established in January 2005 had examined this issue and that many key recommendations had been adopted to improve the performance of and customer satisfaction with the electronic forms. SSA added that it is using a contractor to examine alternatives and determine if more robust software is available to better meet users’ needs, and that it may incorporate new software into the electronic disability claims process.

Our report recognized that SSA had established a work group to explore options for resolving problems with the electronic forms. However, during our study, SSA officials could not provide a time table for the work group’s efforts and, despite our inquiries, gave no indication that the agency had a defined strategy for addressing this area of concern. Further, SSA did not inform us of a specific contract to examine software alternatives or of the specific recommendations that have been made to correct problems with the forms and the improvements in performance and customer satisfaction that have been achieved. Thus, we did not have an opportunity to evaluate and comment on the agency’s actions in this regard. Given the DDSs’ expressed concerns about the electronic forms throughout the course of our study and as reflected in SSA’s own quarterly evaluation of the electronic disability system, we continue to stress the importance of SSA developing and implementing a strategy to guide its efforts toward efficiently and effectively resolving problems with this important electronic capability.

Regarding its existing computer monitors, SSA stated that it already has a plan in place to evaluate this issue and potential ergonomic solutions. The agency stated that it had awarded a contract to conduct a controlled test of the impact of various monitor configurations on ergonomics and productivity for all primary users of the electronic disability system, and that a final report is due in January 2007. It added that when final decisions are made regarding the appropriate monitor requirements, a business plan will be deployed if warranted. As noted in our report, SSA officials did inform us of their intent to conduct a pilot test to identify potential solutions for ensuring that users have appropriate monitors. However, the officials could not provide contractual and other pertinent documents explaining the pilot study and did not inform us of any plan that the agency had developed to guide this effort. Given that SSA does not anticipate its final report on its test of the monitor configurations until January 2007, and does not intend to consider the deployment of a business plan until final decisions are made regarding the monitor requirements, we believe that the agency could benefit from a strategy articulating clear milestones, resources, and priorities to guide its efforts toward finalizing decisions on its computer monitors and ensure that all users’ concerns are fully and effectively addressed.

Further, regarding the DDSs’ software capabilities, SSA stated that it had established a help desk to provide support for specific hardware and software issues associated with the electronic disability system, and that it had no information that there are outstanding issues concerning DDS software support. Moreover, the agency stated that our report had inaccurately described supposed issues related to implementing the electronic disability system in light of some states having not yet been certified to fully process cases electronically. For example, SSA commented that because Florida had not yet been certified to fully process cases electronically, it was premature to expect that the electronic disability system would notify staff of actions to process claims. Similarly, SSA stated that because Nebraska had not been certified to process cases electronically, it was premature to indicate that the state needed additional software modifications to electronically import all required documents into the electronic folder. Further, the agency said that it was inaccurate for us to report that the North Carolina DDS lacked the necessary software to enable staff to electronically send claim files to SSA’s Disability Quality Branch, since all but two states—New York and Nebraska—had this software. It added that the agency’s planning and electronic disability system roll out had addressed software capabilities and other issues impacting individual DDSs, negating the need for a formal strategy, as we have recommended.

We disagree that our report inaccurately described issues related to implementing the electronic disability system or that it prematurely highlighted limitations in certain states’ use of the system. A primary aspect of our review involved examining SSA’s progress in rolling out the electronic disability system, including the state DDSs’ experiences in implementing and using the system. In this regard, DDS officials apprised us of their electronic disability claims processing capabilities, and in certain instances, of their need for additional software capabilities that they deemed essential to improving their offices’ processing efficiency and sustaining productivity in the electronic environment. As of mid-July 2005, North Carolina DDS officials told us that they lacked the necessary software to enable staff to electronically send claim files to SSA’s Disability Quality Branch. We recognize that SSA does not expect to complete all states’ certifications until early 2007. However, in our view, until the agency ensures that all DDSs have full electronic processing capabilities, it will not be positioned to effectively assess the extent to which the electronic disability system can contribute to a more efficient and effective disability claims process.

SSA further stated that system availability issues had been addressed, as evidenced by its current fiscal year data on key electronic disability system components (e.g., the Electronic Disability Collect System, the Case Processing Management System, and the Document Management Architecture). However, our report does not convey that these key electronic disability system components were not available for use. Rather, the concerns discussed in our report pertain to the inefficiencies that DDS officials said they had encountered in using the electronic disability system. For example, DDS officials pointed to the continuing instances in which they had experienced processing slowdowns when using electronic forms, which ultimately had impeded their disability claims processing capability. Thus, we stand by our recommendation that SSA develop and implement a strategy to ensure that the DDSs have the necessary software capabilities to fully and efficiently process claims in the electronic environment.

Regarding our second recommendation, the agency agreed to ensure that the state DDSs develop and implement continuity of operations plans that complement SSA’s plans for continuing essential disability claims processing functions during emergencies or other disruptions to normal operations. In this regard, the commissioner stated that SSA is highly committed to providing uninterrupted services to the public and had hired a contractor to develop business continuity plans for the DDSs that document how these offices would respond to long- and short-range disruptive events. The actions that SSA stated that it is taking should help improve the agency’s and the DDSs’ ability to ensure the continuity of vital disability benefits services in emergency situations.

In addition to the aforementioned comments, SSA provided technical comments, which we have incorporated, as appropriate. Appendix II reproduces the agency’s comments on our draft report.

As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to the Commissioner of Social Security and the Director, Office of Management and Budget. Copies will also be available at no charge on our Web site at www.gao.gov.

Should you have any questions on matters contained in this report, please contact me at (202) 512-6240. I can also be reached by email at koontzl@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix III.

Objectives, Scope, and Methodology

Our objectives were to (1) assess the current status of SSA’s accelerated implementation of its electronic disability system—the initiative known as AeDib and (2) identify actions the agency has taken in response to our prior recommendations on this initiative. To assess the agency’s status in implementing its electronic disability system, we analyzed relevant project management documentation including schedules, project plans, and reports documenting the status of the system’s rollout to the 54 state disability determination service (DDS) offices and SSA’s 144 Office of Hearings and Appeals (OHA) sites. In addition, we reviewed technical documentation, such as software project scope agreements and software development plans, to assess the development, implementation, and operation of the electronic disability system. We also reviewed system release certifications to ensure that systems had been validated and certified.

To identify issues that arose during the AeDib implementation process, we reviewed problems reported by the DDSs via SSA’s Change Asset Problem Reporting System. We also reviewed the results of the National Council of Disability Determination Directors’ February 2005 survey of its member DDS offices on their experiences in implementing the electronic disability system, as a means of identifying any problems and issues that the states had encountered. In addition, we reviewed reports on the system’s implementation, performance, and capacity that had been prepared by the Council and the DDSs.

We supplemented our analysis with interviews of SSA officials in the Offices of Operations, Systems, Disability and Income Security Programs, and Hearings and Appeals. We also interviewed DDS officials in nine states: California, Delaware, Florida, Illinois, Mississippi, Nebraska, New York, North Carolina, and South Carolina. In addition, we interviewed the President of the National Council of Disability Determination Directors, an organization that represents the DDSs.

Our selection of the nine states was based on the following criteria: The Mississippi DDS was the first state to which the electronic disability system was rolled out, as well as the first state to achieve total electronic processing of initial disability cases.

The California, Illinois, and North Carolina DDSs had participated in initial pilot tests of the electronic processing system, which had included assessing use of the Document Management Architecture.

The Florida and South Carolina DDSs were states that received the electronic disability system early in the implementation schedule.

The New York and Nebraska DDSs posed potential unique challenges as the only two “independent” states, in which their existing claims processing capabilities were not supported by the common hardware platform that the majority of DDSs used and that had developed and were relying on disability claims processing software that was unique to their processing environments.

The Delaware DDS was managed by the President of the National Council of Disability Determination Directors.

We conducted site visits at two DDSs—Mississippi and South Carolina—to observe the electronic processing system in operation, and at OHA sites in these same states to discuss their experiences in implementing and using the electronic folder and their preparation for receiving appeals of initial claims that had been processed electronically in the respective state DDS offices.

To determine what actions SSA had taken toward implementing our prior recommendations on the electronic disability system, we obtained and reviewed software project scope agreements, software development plans, user validation and system certification plans, and AeDib component security risk assessment documentation. We also interviewed agency officials regarding the status of their actions on each of the recommendations made in our March 2004 report on AeDib. In addition, we discussed SSA’s efforts to improve communications on the initiative’s implementation with DDS officials in each of the offices that we contacted and with the President of the National Council of Disability Determination Directors.

We conducted our work at SSA’s headquarters in Baltimore, Maryland, and at selected DDS and OHA offices in Jackson, Mississippi, and Columbia, South Carolina, from October 2004 to July 2005, in accordance with generally accepted government auditing standards.

Comments from the Social Security Administration

GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, Valerie C. Melvin, Assistant Director; Michael A. Alexander; J. Michael Resser; and Eric L. Trout made significant contributions to this report. Neil J. Doherty and Joanne Fiorino also contributed to the report.