Legal Services Corporation: More Needs to be Done to Correct Case Service Reporting Problems
Published: 1999-09-29T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Pursuant to a congressional request, GAO discussed the two reviews that it has completed pertaining to case service reporting (CSR) by the Legal Services Corporation (LSC), focusing on: (1) what efforts LSC and its grantees have made to correct case reporting problems; and (2) whether these efforts are likely to resolve the case reporting problems that occurred in 1997.
section_title: What GAO Found
paragraphs: GAO noted that: (1) LSC issued a new, 1999 CSR Handbook and distributed other written communications intended to clarify reporting requirements to its grantees; (2) the 1999 Handbook, which replaced the 1993 edition, instituted changes to some of LSC's reporting requirements and provided more detailed information on other requirements; (3) in responding to a GAO telephone survey, most grantees indicated that the new guidance helped clarify LSC's reporting requirements, and virtually all of them indicated that they had or planned to make program changes as a result of the requirements; (4) many grantees, however, identified areas of case reporting that remained unclear to them; (5) the 1999 CSR Handbook included changes to: (a) procedures for timely closing of cases; (b) procedures for management review of case service reports; (c) procedures for ensuring single recording of cases; (d) requirements to report LSC-eligible cases, regardless of funding source; and (e) requirements for reporting cases involving private attorneys separately; (6) based on GAO's survey of executive directors of 79 grantees, GAO estimates that over 90 percent of grantee executive directors viewed the changes in the 1999 CSR Handbook as being clear overall, and virtually all of them indicated that they planned to or had made at least one change to their program operations as a result of the revised case reporting requirements; (7) although most of the grantee executive directors reported that the new LSC guidance helped clarify requirements, many of them also indicated that they were still unclear about certain requirements and that additional clarification was needed; (8) LSC sought to determine the accuracy of grantees' case data by requiring that grantees complete self-inspections of their open and closed caseload data for 1998; (9) on May 14, 1999, LSC issued a memo to all grantees instructing them to complete a self-inspection procedure by July 1, 1999; (10) according to LSC officials, about three-fourths of the grantees certified the accuracy of their 1998 case data; (11) as of August 26, 1999, LSC documents indicated that 199 of 261 grantees reported substantially correct CSR data to LSC; (12) the remaining 62 grantees did not certify to LSC that their CSR data were substantially correct; and (13) GAO's review raised some concerns about LSC's interpretation of the self-inspection results and about the accuracy of the data provided to LSC by grantees.

section_title: Letter
paragraphs: I am pleased to be here today to discuss two reviews that we have completed pertaining to case service reporting by the Legal Services Corporation (LSC). During the past year, both the LSC Office of the Inspector General (OIG) and we identified misreporting by grantees on both the number of cases they closed during calendar year 1997 and the number they had open at the end of that year. In a June 1999 report, we estimated that nearly 75,000 of the approximately 221,000 cases reported to LSC by 5 of its largest grantees were questionable. In light of these findings, we were asked to determine (1) what efforts LSC and its grantees have made to correct case reporting problems, and (2) whether these efforts are likely to resolve the case reporting problems that occurred in 1997. In a report issued earlier this month we found the following:
LSC revised its written case reporting guidance and issued a new handbook to its grantees to clarify case reporting requirements.
Grantees reported changing their policies and procedures to comply with LSC’s new reporting requirements.
Although most grantees indicated that LSC’s revised case reporting guidance has clarified reporting requirements for grantees, many grantees remained unclear about certain aspects of LSC’s reporting requirements.
Problems existed with LSC’s self-inspection, which sought to verify the accuracy of 1998 Case Service Reporting (CSR) data.
We believe that more needs to be done to correct reporting problems at LSC. We made eight recommendations to help LSC improve the accuracy of future CSR reports.
subsections: section_title: Background
paragraphs: LSC was established in 1974 as a private, nonprofit, federally funded corporation to provide legal assistance to low-income people in civil matters. LSC provides the assistance indirectly, through grants to about 260 competitively selected local programs. Grantees may receive additional funding from non-LSC sources. In fiscal years 1998 and 1999, LSC received appropriations of $283 million and $300 million, respectively. eligibility, clients’ income, in general, is not to exceed 125 percent of the federal poverty guidelines. LSC regulations require that grantees (1) adopt a form and procedure to obtain eligibility information and (2) preserve that information for audit by LSC. With respect to citizenship/alien eligibility, only citizens and certain categories of aliens are eligible for services. For clients who are provided services in person, a citizen attestation form or documentation of eligible alien status is required. For clients who are provided services via the telephone, documentation of the inquiry regarding citizenship/alien eligibility is required.
LSC uses a Case Service Reporting system to gather quantifiable information from grantees on the services they provide that meet LSC’s definition of a case. The CSR Handbook is LSC’s primary official guidance to grantees on how to record and report cases. LSC relies on such case information in its annual request for federal funding.
Audit reports issued by LSC’s OIG between October 1998 and July 1999 reported that five grantees misreported the number of cases they had closed during calendar year 1997 and the number of cases that remained open at the end of that year. The OIG found that all five grantees overstated the number of closed cases, while four overstated and one understated open cases.
section_title: Our Audit of Five LSC Grantees Found That One-Third of 1997 Cases Were Questionable
paragraphs: In June 1999, in response to Congress’ request for information on whether the 1997 case data of other LSC programs had problems similar to those reported by LSC’s OIG, we issued a report on our audit of five of LSC’s largest grantees: Baltimore, Chicago, Los Angeles, New York City, and Puerto Rico. We conducted a file review of a random sample of cases at each of these grantees to determine the extent to which they made overreporting errors in reporting cases closed during 1997 and cases open on December 31, 1997. We found similar types of reporting errors to those the OIG found and estimated that, overall, about 75,000 (+/- 6,100) of the approximately 221,000 cases that the five grantees reported to LSC for 1997 were questionable. Three grantees identified about 30,000 of their cases as misreported prior to our case file review. The primary causes for these self-identified overreporting errors were (1) improperly reporting to LSC cases that were wholly funded by other sources, such as states, and (2) problems related to case management reporting systems, such as grantee staffs’ difficulty in transitioning to new automated systems.
Our case file review deemed approximately 45,000 additional cases questionable for one of the following reasons:
The grantee reported duplicate cases for the same legal service to the same client.
Some case files did not contain any documentation supporting the grantee’s determination that the client was either a U.S. citizen or eligible alien.
For cases reported as closed in 1997, some case files showed no activity during the 12 months before the case was closed. For cases reported as open as of December 31, 1997, some cases showed no grantee activity during calendar year 1997.
Some case files did not contain any documentation that the grantee had determined that the client was financially eligible for LSC services. LSC regulations did not require specific documentation of these determinations in all cases. However, they required that grantees (1) adopt a form and procedure to obtain eligibility information and (2) preserve that information for audit by LSC.
LSC officials and executive directors of the five grantees told us that they had taken or were planning to take steps to correct these case reporting problems.
section_title: LSC’s Clarified Reporting Guidance Resulted in Program Changes, but Some Requirements Remain Unclear to Many Grantees
paragraphs: LSC issued a new, 1999 CSR Handbook and distributed other written communications intended to clarify reporting requirements to its grantees. The 1999 handbook, which replaced the 1993 edition, instituted changes to some of LSC’s reporting requirements and provided more detailed information on other requirements.
In responding to a GAO telephone survey, most grantees indicated that the new guidance helped clarify LSC’s reporting requirements, and virtually all of them indicated that they had or planned to make program changes as a result of the requirements. Many grantees, however, identified areas of case reporting that remained unclear to them.
subsections: section_title: LSC Issued New CSR Guidance
paragraphs: The 1999 CSR Handbook included changes to (1) procedures for timely closing of cases; (2) procedures for management review of case service reports; (3) procedures for ensuring single recording of cases; (4) requirements to report LSC-eligible cases, regardless of funding source; and (5) requirements for reporting cases involving private attorneys separately.
On November 24, 1998, LSC informed its grantees that two of the changes in the 1999 CSR Handbook were to be applied to the 1998 case data. The two changes pertained to timely closing of cases and management review of case service reports. The remaining new provisions of the 1999 CSR Handbook were not applicable to 1998 cases. For example, for 1998, there was no requirement for grantees to ensure that cases were not double counted. For 1999, LSC is requiring the use of automated case management systems and procedures to ensure that cases involving the same client and specific legal problem are not reported to LSC more than once.
For 1998, grantees could report only those cases that were at least partially supported by LSC funds. For 1999, LSC is requiring grantees to report all LSC-eligible cases, regardless of funding source. LSC intends to estimate the percentage of activity spent on LSC service by applying a formula that incorporates the amount of funds grantees receive from other funding sources compared with the amount they receive from LSC.
In addition to changing certain reporting requirements, the 1999 handbook also provides more detailed guidance to grantees than the 1993 handbook. For example, the 1999 handbook provides more specific definitions of what constitutes a “case” and a “client” for CSR purposes. The 1999 handbook also addresses documentation requirements that were not discussed in the 1993 handbook.
section_title: Grantee Directors Reported That They Are Implementing Changes to Comply With Reporting Requirements
paragraphs: Based on our survey of executive directors of 79 grantees, we estimate that over 90 percent of grantee executive directors viewed the changes in the 1999 CSR Handbook as being clear overall, and virtually all of them indicated that they planned to or had made at least one change to their program operations as a result of the revised case reporting requirements. These changes included revising policies and procedures, providing staff training, modifying forms and/or procedures used during client intake, implementing computer hardware and software changes, and increasing reviews of cases.
section_title: Many Grantees Remain Unclear About Certain Reporting Requirements
paragraphs: citizenship/alien eligibility documentation, single recording of cases, and who can provide legal services.
section_title: Most Grantees Certified the Accuracy of Their 1998 CSR Data, but Questions About Data Accuracy and Interpretation Remain
paragraphs: LSC sought to determine the accuracy of grantees’ case data by requiring that grantees complete self-inspections of their open and closed caseload data for 1998. Grantees were required to determine whether the error rate in their data exceeded 5 percent. According to LSC, about three-fourths of the grantees certified that the error in their data was 5 percent or less. LSC used the results of the self-inspections to estimate the total number of case closings in 1998. Our review of LSC’s self-inspection process raised concerns about the accuracy and interpretation of the results, and what the correct number of certifying programs should be.
subsections: section_title: LSC Grantees Conducted Self-Inspections of 1998 CSR Data
paragraphs: On May 14, 1999, LSC issued a memo to all grantees instructing them to complete a self-inspection procedure by July 1, 1999. The purpose of the self-inspection was to ensure that (1) grantees were properly applying instructions in the 1999 edition of the CSR Handbook that were applicable to the 1998 data, and (2) LSC had accurate case statistical information to report to Congress for calendar year 1998.
LSC provided detailed guidance to grantees on the procedures for the self- inspection. Each grantee was to select and separately test random samples of open and closed cases to determine whether the number of cases it reported to LSC earlier in the year was correct. Grantees were to verify that the case file contained a notation of the type of assistance provided, the date on which the assistance was provided, and the name of the case handler providing the assistance. Grantees were also to determine whether assistance had ceased prior to January 1, 1998; was within certain service categories as defined by the 1999 handbook; was provided by an attorney or paralegal; and was not prohibited or restricted. Finally, grantees were to verify that each case had eligibility information on household income, size, assets, citizenship attestation for in-person cases, and indication of citizenship/alien status for telephone-only cases. identified one or more problems in the random sample and corrected their entire 1998 database so that the problems no longer appeared. If, by correcting the problems, the error rate in the data was reduced to 5 percent or less, the grantees could resubmit their 1998 data along with a signed certification attesting to the substantial accuracy of the resubmitted data. In this way, grantees who were unable to certify at one point in time could certify at a later point in time.
section_title: Most Grantees Certified Their 1998 CSR Data
paragraphs: According to LSC officials, about three-fourths of the grantees certified the accuracy of their 1998 case data. As of August 26, 1999, LSC documents indicated that 199 of 261 grantees (76 percent) reported substantially correct CSR data to LSC. The remaining 62 grantees (24 percent) did not certify to LSC that their CSR data were substantially correct. According to LSC, 30 of the 50 largest grantees did not certify their 1998 data.
LSC officials told us they were surprised that such a large number of grantees certified their 1998 CSR data. They attributed the results to the following factors: (1) the self-inspection did not attempt to identify duplicate cases; (2) grantees received the new 1999 handbook in November 1998 and had already implemented some of the new requirements; and (3) grantees were less likely to report as cases telephone referrals in which no legal advice had been given and/or clients’ eligibility had not been determined because they were aware that the OIG identified this as a problem.
On the basis of the self-inspection results, LSC estimated that grantees closed 1.1 million cases in 1998.
section_title: Self-Inspection Results Raised Concerns
paragraphs: Our review raised some concerns about LSC’s interpretation of the self- inspection results and about the accuracy of the data provided to LSC by grantees. As a result, we could not assess the accuracy of LSC’s estimate of the number of certified programs and case closures for 1998.
LSC did not issue standardized procedures for grantees to use in reporting the results of their self-inspections. Grantees that could not certify their data wrote letters to LSC that contained varying degrees of detail about data errors that they found. Since LSC did not have a standard protocol for collecting the results of the self-inspections, LSC officials in some cases had to rely on their own interpretations of grantees’ descriptions of the problems they had discovered.
We are uncertain how many programs should have been counted as certified because we are uncertain if LSC applied a consistent definition of “certification.” Most programs that were on LSC’s certification list determined that they had error rates of 5 percent or less for both open and closed cases. However, LSC placed some programs on the certified list if the program’s overall error rate for closed cases was 5 percent or less, even if the overall error rate actually was higher than 5 percent. In two instances, executive directors told us that they did not certify their CSR data because their overall error rate exceeded 5 percent. However, these programs appeared on LSC’s list of certified programs. When we asked an LSC official about this, he told us that they advised grantees that if their closed case error rate did not exceed 5 percent, they should “partially certify” their data. In response to our inquiry, the official reviewed the certification letters submitted by nearly 200 grantees, and identified 5 certified programs whose error rates for open cases exceeded 5 percent. Given that some grantees submitted only an overall estimate of data error, we do not know how many programs qualified to be certified overall, just for closed cases, or just for open cases.
We are also concerned that LSC’s instructions to grantees on how to conduct the self-inspections may have led some of the smaller grantees to select too few test cases to make a reliable assessment of the proportion of error in their case data. Because these were smaller grantees, this limitation would have had little effect on LSC’s estimate of the total closed caseload. However, it could have affected LSC’s count of the number of certified programs.
LSC does not know how well grantees conducted the self-inspection process, nor how accurate the results are. We spoke with several executive directors who did not correctly follow LSC’s reporting requirements. Incorrect interpretations of LSC guidance may have resulted in some programs certifying their 1998 data when they should not have, and other programs not certifying their 1998 data when they should have. An LSC official told us that, although they have conducted CSR training sessions for grantee executive directors, thousands of case handlers in grantee offices have not received such training. The official acknowledged that written guidance and telephone contacts with grantees may not be sufficient to ensure correct and consistent understanding of reporting requirements, and that LSC plans to consider alternative ways of providing training to staff.
LSC officials told us that the self-inspection was valuable and that LSC plans to have grantees complete self-inspections again early next year as part of the 1999 CSR reporting process.
section_title: Conclusions
paragraphs: LSC’s 1999 CSR Handbook and other written communications have improved the clarity of reporting requirements for its grantees. However, many grantees remained unclear about and/or misunderstood certain aspects of the reporting requirements. LSC’s practice of disseminating guidance primarily by written or telephone communications may not be sufficient to ensure that grantees correctly and consistently interpret the requirements.
LSC sought to determine the accuracy of grantees’ 1998 case statistics by requiring grantees to conduct self-inspections. However, we do not know the extent to which the results of the self-inspection process are accurate. The validity of the results are difficult to determine because LSC did not standardize the way that grantees were to report their results, some of the grantees used samples that were too small to assess the proportion of error in their data, some grantees did not correctly follow LSC’s reporting guidance, and LSC had done no verification of the grantees’ self-inspection procedures.
We do not believe that LSC’s actions, to date, have been sufficient to fully resolve the case reporting problems that occurred in 1997.
section_title: Recommendations
paragraphs: develop a standard protocol for future self-inspections to ensure that grantees systematically and consistently report their results for open and closed cases; direct grantees to select samples for future self-inspections that are sufficient to draw reliable conclusions about magnitude of case data errors; and finally, ensure that procedures are in place to validate the results of LSC’s 1998 self-inspection, as well as of any future self-inspections.
In a written response to a draft of our report, the President of LSC generally agreed with our findings and noted that he plans to implement our recommendations to the fullest extent possible.
Mr. Chairman, this concludes my prepared statement. I would be pleased to answer any questions that you or other Members of the Committee may have.
Contacts and Acknowledgement For further information regarding this testimony please contact Laurie E. Ekstrand or Evi Rezmovic on (202) 512-8777. Individuals making key contributions to this testimony included Mark Tremba and Jan Montgomery.
The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.
U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.
Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touch- tone phone. A recorded menu will provide information on how to obtain these lists.