LEGAL SERVICES

CORPORATION

More Needs to Be

Done to Correct Case

Service Reporting

Problems

United States General Accounting Office

GAO Report to Congressional Requesters

September 1999

GAO/GGD-99-183

United States General Accounting Office General Government Division

Washington, D.C. 20548

Page 1 GAO/GGD-99-183 LSC Case Reporting Problems

B-283551

September 20, 1999

The Honorable Dick Armey

The Honorable Dan Burton

The Honorable Tom Latham

The Honorable Dan Miller

The Honorable Charles Taylor

House of Representatives

The Legal Services Corporation (LSC), operating through grantees,

provides legal assistance in civil matters to low-income individuals. In the

past year, LSC’s Office of Inspector General (OIG) and we have identified

misreporting by grantees on both the number of cases they closed during

calendar year 1997 and the number they had open at the end of that year.

The accuracy of the data is important because LSC has used case statistics

to seek increased funding for LSC, and Congress has considered these case

statistics in determining funding for LSC.

As agreed with your offices, we determined (1) what efforts LSC and its

grantees have made to correct problems with case service reporting and

(2) whether these efforts are likely to resolve the case reporting problems

that occurred in 1997.

LSC revised its written guidance and issued a new handbook to its

grantees to clarify case reporting requirements. Based on telephone

interviews with a sample of 79 LSC grantee executive directors, we

estimate that 90 percent of grantees viewed the new guidance as having

clarified reporting requirements, overall. Virtually all grantees said they

responded to the new requirements by making or planning to make one or

more changes to their program operations. However, many grantees

indicated that they were unclear about certain aspects of LSC’s reporting

requirements, particularly regarding (1) the specific information required

on client assets, (2) the information required for documenting

citizenship/alien eligibility for services provided over the telephone, (3) the

criteria for avoiding duplicate counts of cases, and (4) who can provide

legal assistance to clients in order for the service to be counted as a case.

LSC initiated a self-inspection procedure in which grantees were required

to review their 1998 case data and submit certification letters to LSC if

they found that the extent of error in their data was 5 percent or less.

Grantees who could not certify their 1998 data were required to develop

Results in Brief
B-283551

Page 2 GAO/GGD-99-183 LSC Case Reporting Problems

corrective actions that would address the problems identified. About 75

percent of the grantees submitted letters to LSC certifying that the error

rate in their 1998 data was 5 percent or less, while about 25 percent of the

grantees submitted letters to LSC indicating that they could not certify

their 1998 data. According to LSC, about 30 of the 50 grantees with the

largest caseloads were unable to certify their 1998 case data.

We could not assess whether the number of certified and uncertified

programs that LSC obtained for 1998 was correct, lower, or higher than it

should be. This is because LSC did not provide grantees with a

standardized way of reporting their self-inspection results, and LSC’s

instructions on how to conduct the self-inspections may have led some of

the smaller grantees to select too few cases to reliably assess the amount

of error in their case data. In addition, some grantees did not correctly

interpret LSC’s case reporting requirements. LSC is planning to consider

alternative ways of training grantee staff so that they gain a better

understanding of the reporting requirements.

For these reasons, we do not believe that LSC’s efforts to date have been

sufficient to fully resolve the case reporting problems that occurred in

1997. This report includes recommendations to LSC for clarifying its case

reporting guidance to grantees and improving its procedures for

conducting future self-inspections.

LSC was established in 1974 as a private, nonprofit, federally funded

corporation to provide legal assistance to low-income people in civil

matters. LSC provides the assistance indirectly through grants to

competitively selected local programs. LSC distributes funds to the

grantees on the basis of the number of low-income persons living within a

service area. Grantees may receive additional funding from non-LSC

sources.

During 1998, LSC funded 262 local grantees that operated through

approximately 900 neighborhood law offices employing about 3,600

attorneys and 1,400 paralegals. Each local program is governed by its own

board of directors and is required to spend at least 12.5 percent of its LSC

grant to encourage private attorney involvement (PAI) in delivering legal

services to low-income clients. In fiscal years 1998 and 1999, LSC received

appropriations of $283 million and $300 million, respectively.

LSC’s authorizing legislation restricts it from engaging in lobbying; political

activities; class actions except under certain conditions; and cases

involving abortion, school desegregation, and draft registration or

Background
B-283551

Page 3 GAO/GGD-99-183 LSC Case Reporting Problems

desertion from the military. Annual appropriations laws have placed

additional restrictions on the activities in which LSC grantees can engage,

even with non-LSC funds. In 1996, for example, grantees were prohibited

from engaging in challenges to welfare reform, litigation on behalf of

prisoners, representation in drug-related public housing evictions, and

representation of certain categories of aliens.

Grantees must serve only those clients who meet financial and

citizenship/alien eligibility requirements. With respect to client financial

eligibility, local programs are to establish their own criteria, which, in

general, require that clients’ income not exceed 125 percent of the federal

poverty guidelines. With appropriate documentation of the grantee’s

decision, clients who are between 125 percent and 187.5 percent of the

federal poverty level may be found eligible. LSC regulations require that

grantees (1) adopt a form and procedure to obtain eligibility information

and (2) preserve that information for audit by LSC. With regard to

citizenship/alien eligibility, only citizens and certain categories of aliens

are eligible for services. For clients who are provided services in person, a

citizen attestation form or documentation of eligible alien status is

required. For clients who are provided services on the telephone, grantees

must document that they made inquiries regarding the individuals’

citizenship/alien eligibility.

LSC uses a Case Service Reporting (CSR) system to gather quantifiable

information from grantees on the services they provide that meet LSC’s

definition of a case. The CSR Handbook is LSC’s primary official guidance

to grantees on how to record and report cases. According to the 1999 CSR

Handbook, which revised and expanded the guidance in LSC’s 1993

Handbook, information about cases is an important indicator of the

number of legal problems that programs address each year, and LSC relies

on such case information in its annual request for federal funding for legal

services.

Audit reports on five grantees issued by LSC’s OIG between October 1998

and July 1999 reported that all five grantees misreported the number of

cases they had closed during calendar year 1997 and the number of cases

that remained open at the end of that year. The OIG found that all five

grantees overstated the number of closed cases, while four overstated and

one understated open cases. The OIG attributed the overreporting to such

factors as (1) counting as cases telephone calls in which individuals were

not provided any legal assistance and only partial eligibility determinations

were made; (2) counting wholly non-LSC funded cases as LSC cases; (3)

double counting of the same cases; (4) reporting cases as closed during
B-283551

Page 4 GAO/GGD-99-183 LSC Case Reporting Problems

1997, or as still open at the end of 1997, when service ceased in prior years;

and (5) counting as cases the provision of services to over-income clients.

In June 1999, in response to Congress’ request for information on whether

the 1997 case data of other LSC programs had problems similar to those

reported by LSC’s OIG, we issued a report on our audit of five of LSC’s

largest grantees: Baltimore, Chicago, Los Angeles, New York City, and

Puerto Rico.1

 We found similar types of reporting errors at the five

grantees and estimated that, overall, 75,000 of the 221,000 open and closed

cases that the five grantees reported to LSC were questionable.2

 Interviews

that we conducted with LSC officials and executive directors of the 5

audited grantees indicated that they had taken or were planning to take

steps to correct the causes of these case-reporting problems.

Our objectives were to determine (1) what efforts LSC and its grantees

have made to correct problems with case service reporting, and (2)

whether these efforts are likely to resolve the case reporting problems that

occurred in 1997.

To address the objectives, we reviewed documents that contained LSC

case reporting guidance, interviewed cognizant officials at LSC

headquarters, and conducted structured telephone interviews with a

random sample of grantee executive directors. Specifically, we reviewed

LSC regulations and LSC’s 1993 and 1999 CSR Handbook, as well as

supplemental guidance that LSC distributed to grantees in the form of

program letters and frequently asked questions and answers about case

reporting. We also collected documentation and interviewed LSC officials

and grantee executive directors to gather information about a selfinspection process that LSC required all grantees to undertake in order to

determine the accuracy of their 1998 case data. In our telephone

interviews with executive directors, we asked if the directors viewed LSC’s

case reporting requirements as being clear, what changes they have made

or planned to make as a result of the requirements, and the results of their

self-inspection of 1998 case data. We also interviewed LSC officials and

grantee executive directors about areas of case reporting that they felt

needed further clarification.

 

1

Legal Services Corporation: Substantial Problems in 1997 Case Reporting by Five Grantees

(GAO/GGD-99-135R, June 25, 1999).

2

 Our estimate was based on file reviews of random samples of cases that the five grantees reported as

closed during 1997 and open at the end of 1997. In addition, we reviewed random samples of

potentially duplicate cases for four grantees. We estimated that, overall, 75,000 (+/-6,100) cases were

questionable.

Objectives, Scope, and

Methodology
B-283551

Page 5 GAO/GGD-99-183 LSC Case Reporting Problems

To identify the universe of LSC grantees, an LSC official referred us to the

list of LSC programs on the corporation’s Internet Web site. As of July 15,

1999, the Internet site listed 256 programs. From this list, we randomly

selected 80 programs for our sample. We developed the interview

instrument, pretested it with executive directors from two grantees,

revised the instrument based on the pretest results, and completed

approximately 1-hour-long structured interviews by telephone with the

executive directors of 79 of the 80 programs, for a 99 percent response

rate. (After numerous attempts over several days, we were unable to

contact the executive director of one LSC grantee.) Our sample was

designed so that we could generalize our findings with 95-percent

confidence and a maximum 10-percent margin of error to the universe of

LSC grantees.

We performed our work from July through September 1999 in accordance

with generally accepted government auditing standards. We requested

comments on a draft of this report from the President of LSC. LSC’s

comments are discussed at the end of this letter and included as appendix

I.

LSC issued a new CSR Handbook and distributed other written

communications intended to clarify reporting requirements to its grantees.

Most grantees indicated that the new guidance helped clarify LSC’s

reporting requirements, and virtually all of them indicated that they had or

planned to make program changes as a result of the requirements. Many

grantees, however, identified areas of case reporting that remained unclear

to them.

To address specific problems identified by the OIG and LSC’s own internal

program reviews, LSC has issued revised reporting guidance. In November

1998, LSC issued the 1999 CSR Handbook, which replaced the 1993 CSR

Handbook. The 1999 handbook instituted changes to some of LSC’s

reporting requirements and provided more detailed information on other

requirements than previously existed. LSC also distributed program letters

and a list of frequently asked CSR questions and answers that further

elaborated on points made in the 1999 handbook.

The 1999 CSR Handbook instituted several notable changes to case

reporting requirements. These included (1) procedures for timely closing

LSC Clarified CSR

Requirements and

Grantees Have Been

Making Program

Changes to Comply

With the Requirements,

but Some Grantees

Remain Unclear About

Certain Requirements

LSC Issued New CSR

Guidance
B-283551

Page 6 GAO/GGD-99-183 LSC Case Reporting Problems

of cases; (2) procedures for management review of case service reports;

(3) procedures for ensuring single recording of cases; (4) requirements to

report LSC-eligible cases, regardless of funding source; and (5)

requirements for reporting PAI cases separately. In addition, grantees were

required to use automated case management systems and procedures that

would ensure that program managers had timely access to accurate

information on cases and the capacity to meet their reporting

requirements.

On November 24, 1998, LSC informed its grantees that two of the changes

in the 1999 CSR Handbook were to be applied to the 1998 case data. The

two changes pertained to timely closing of cases and management review

of case service reports. The timely closing provision required grantees to

ensure that cases in which legal assistance had ceased in 1998, and was

not likely to resume, would be closed prior to grantees’ submission of case

service reports to LSC in March 1999. To the extent practicable, cases in

which the only assistance provided to the client was counsel and advice,

brief service, or referral after legal assessment were to be closed in the

year in which these types of service were provided. Cases involving other

types of service were to be closed in the year in which program staff

determined that further legal assistance was unnecessary, not possible, or

inadvisable and a closing memorandum or other case-closing notation was

prepared. The management review provision required the executive

director, or a designee, to review the program’s case service reports prior

to their submission to LSC in order to ensure their accuracy and

completeness.

The remaining new provisions of the 1999 CSR Handbook were not

applicable to 1998 cases. For example, for 1998, there was no requirement

for grantees to ensure that cases were not double counted. For 1999, LSC

is requiring the use of automated case management systems and

procedures to ensure that cases involving the same client and specific legal

problem are not reported to LSC more than once. For 1998, grantees could

report only those cases that were at least partially supported by LSC funds.

For 1999, LSC is requiring grantees to report all LSC-eligible cases,

regardless of funding source. LSC intends to estimate the percentage of

activity spent on LSC service by applying a formula that incorporates the

amount of funds grantees receive from other funding sources compared

with the amount they receive from LSC. For 1998, grantees were required

to report their LSC-funded PAI cases together with non-PAI cases. For

1999, PAI cases are to be reported separately.
B-283551

Page 7 GAO/GGD-99-183 LSC Case Reporting Problems

In addition to changing certain reporting requirements, the 1999 handbook

also provides more detailed guidance to grantees than the 1993 handbook.

For example, the 1999 handbook provides more specific definitions of

what constitutes a “case” and a “client” for CSR purposes. The 1999

handbook also addresses documentation requirements that were not

discussed in the 1993 handbook. For example, the 1999 handbook

indicates that, except for telephone service cases, the client’s file must

contain an attestation of citizenship or documentation of alien eligibility in

order for the case to be reported to LSC.3

 The 1999 handbook also imposes

requirements for documentation of information on client income and

assets. The handbook states that, for all cases reported to LSC, the

eligibility documentation must include specific information about income

and assets. The 1999 handbook also contains a requirement that legal

assistance, to be counted as a case, must be provided by an attorney or

paralegal.

On the basis of our survey, we estimate that over 90 percent4

 of grantee

executive directors viewed the changes in the 1999 CSR Handbook as

being clear overall, and virtually all of them planned to or had made at

least one change to their program operations as a result of the revised case

reporting requirements. Program changes that were cited included revising

policies and procedures, providing staff training, modifying forms and/or

procedures used during client intake, implementing computer hardware

and software changes, and increasing review of cases.

Nearly half of the grantees indicated that they planned to or had revised

their policies and/or procedures to comply with the 1999 handbook

requirements, and slightly less than a third planned to or had conducted

staff training on the requirements. Respondents told us that the focus of

their training was on such issues as the current definition of a case, how to

determine and document clients’ financial eligibility, how to determine and

document citizenship/alien eligibility, timely closing of cases, and

prevention of duplicate case reporting.

More than half of the grantees indicated that they planned to or had

changed their intake forms and/or procedures. Of the 41 respondents who

 

3

The requirement for documentation is contained in LSC regulations, at 45 C.F.R. 1626.6(a) and

1626.7(a). However, the 1999 CSR Handbook specified for the first time that the case could not be

reported to LSC if the documentation was missing.

4

All percentage estimates presented in this report have 95 percent confidence intervals with a margin of

error of 10 percent or less.

Grantee Directors Reported

That They Are

Implementing Changes to

Comply With Reporting

Requirements
B-283551

Page 8 GAO/GGD-99-183 LSC Case Reporting Problems

made this comment, 26 said they were making these changes in order to

document client income and assets.

Slightly over 40 percent of the grantees reported that they planned to or

had made computer changes. The computer changes included such actions

as adding new fields to their automated case management (e.g., to ensure

that client eligibility and acceptance information is recorded), making

programming changes (e.g., to identify duplicate cases, ensure that cases

are not assigned separate numbers for the same client with the same legal

problem, and better document client assets), and installing new software

(e.g., to generate reports so that they can track how long cases are open).

About 10 percent of grantee executive directors indicated that they could

better comply with CSR requirements if they had uniform case

management software. LSC, too, believes that grantees’ case management

systems should provide for more consistency in the collection and

processing of CSR information. According to an LSC official, LSC is

developing a strategy for modifying grantees’ case management systems so

that data errors can be detected and prevented. As part of the strategy,

LSC has hired an expert in case management systems and is working to

develop standard input requirements for these systems. LSC intends to

work with case management system vendors to modify the systems so that

they prevent cases from being accepted if grantees do not record the

required compliance information. LSC expects modifications to the case

management systems to be implemented in calendar year 2000 and

available for application to grantees’ 2000 CSR data. In the nearer term,

LSC plans to work with a contractor to develop customized case

management queries that would enable grantees to detect and remedy

errors in their 1999 case data. In 1999, LSC plans to pilot test the computer

queries at five programs that have the most commonly used case

management systems.

Nearly three-fourths of the grantees indicated that they planned to or had

increased their review of cases to comply with LSC requirements.

Respondents cited such review activities as more intense monitoring of

open cases, to ensure that they are closed in a timely manner, and more

thorough monitoring of PAI cases. Some grantees said they have directed

increased attention to reporting requirements during routine reviews of

cases, while others said they instituted more frequent reviews of cases.

One executive director reported to LSC that managing attorneys would

meet with all case-handling staff and reemphasize the importance of

keeping accurate activity records. These managing attorneys are to

monitor compliance with the requirement initially on a weekly basis and
B-283551

Page 9 GAO/GGD-99-183 LSC Case Reporting Problems

then on a random basis. If deficiencies are discovered, the managing

attorney is to direct the case handler to correct them immediately, and the

managing attorney is to recheck the file within 48 hours to ensure

compliance with the directive. Another executive director told us that his

program had not previously reviewed CSR data. He said that they now are

reviewing reports, along with listings of open and closed cases, to ensure

their compliance with LSC guidelines. Some executive directors said that

they planned to become more involved in reviewing case files and case

management reports.

Many grantees reported making other efforts to comply with reporting

requirements, such as disseminating the new handbook to case handlers,

developing compliance checklists, emphasizing the importance of

compliance to staff, sending written instructions or memos to staff,

holding meetings and discussions, providing more feedback to case

handlers, and more stringently enforcing requirements.

Although most of the grantee executive directors reported that the new

LSC guidance helped clarify requirements, many of them also indicated

that they were still unclear about certain requirements and that additional

clarification was needed. Among the areas of confusion or uncertainty that

executive directors identified were requirements pertaining to asset and

citizenship/alien eligibility documentation, single recording of cases, and

who can provide legal services.

• Asset documentation: The 1999 CSR Handbook states that, for all cases to

be reported to LSC, the eligibility documentation should include specific

information about income and assets. About 30 percent of the executive

directors indicated that LSC’s requirements for documenting client assets

were clear only to some, to little, or to no extent. Of the 24 respondents

who gave this response, 23 made comments to the effect that LSC should

clarify what it means by assets and asset limits and/or clarify its

documentation requirements for client assets. In a July 14, 1999, program

letter to grantees, LSC noted that although many grantees inquire about

applicants’ assets, they do not consistently document either the inquiries

or the applicants’ responses. In its program letter, LSC sought to clarify its

requirements for documenting and preserving the asset information

obtained from each applicant. Because we conducted our telephone

survey in late July and early August, we do not know how many of the

executive directors with whom we spoke had reviewed LSC’s July 14

guidance and felt that it sufficiently clarified their questions concerning

asset documentation. However, two respondents told us that LSC’s July 14

guidance was still unclear. One respondent said, for example, that the

Many Grantees Remain

Unclear About Certain

Reporting Requirements
B-283551

Page 10 GAO/GGD-99-183 LSC Case Reporting Problems

program letter left unclear the meaning of the term “household goods,”

whether exempted assets had to be documented, who was to value the

goods, and who was to determine what to count and what not to count as

part of assets. Our own analysis of LSC guidance on the asset issue

indicated that LSC has not been consistent in its directives to grantees

about the specificity of asset information they must have in order to

comply with CSR reporting requirements. For example, in its March 24,

1999, communication on frequently asked CSR questions and answers, LSC

instructed grantees to document, at minimum, the total amount of

household assets accessible to the client. In its July 14, 1999, program

letter, LSC instructed grantees to identify and document all of the liquid

and nonliquid assets of all persons who are resident members of a family

unit.

• Citizenship/alien eligibility documentation requirements for telephone

cases: The 1999 CSR Handbook does not explicitly address the citizenship

or eligible alien status documentation requirements for situations where

assistance is provided only over the telephone. Nearly one-fourth of the

executive directors indicated that LSC’s documentation requirements in

this area were clear only to some, to little, or to no extent. Of 19

respondents who gave this response in our survey, 15 said that more

clarification was needed on the grantees’ documentation responsibilities.

Respondents said, for example, that they were confused about exactly

when they needed to obtain a written attestation, whether it was sufficient

simply to record that questions about citizenship/alien eligibility had been

asked, whether certain types of service required documentation while

others did not, and whether the requirement changed if an individual

receiving assistance over the telephone came into the office to drop off

documents. In its July 14, 1999, program letter to grantees, LSC sought to

clarify its requirements for documenting citizenship/alien eligibility

information in telephone cases. LSC stated that it requires recipients to

make appropriate inquiry of every telephone applicant and record the

inquiry and response. All such documentation is to be maintained in the

client file. We do not know how many of the respondents to our survey

were familiar with LSC’s July 14 guidance when we interviewed them in

late July and early August.

• Single recording of cases: The 1999 CSR Handbook requires that programs

ensure that cases involving the same client and specific legal problem are

not recorded and reported to LSC more than once. Over one-fourth of the

executive directors indicated that the requirement for preventing duplicate

case reporting was clear only to some, to little, or to no extent. Of the 22

respondents who gave this response in our survey, 17 said that the
B-283551

Page 11 GAO/GGD-99-183 LSC Case Reporting Problems

distinction between specific and related legal problems is difficult to

determine. Several respondents cited examples in family law cases where

more than one problem can arise within one family case. Depending on the

situation, it was not clear to them at what point legally related issues

become separate enough to count as separate cases. One respondent

indicated that attorneys in his program have a hard time interpreting this

requirement, and that he got calls every week about this issue.

• Provider of legal assistance: The CSR handbook requires that legal

assistance be provided by an attorney or paralegal in order for a service to

be called a case.5

 Slightly over 40 percent of executive directors indicated

that LSC was clear only to some, to little, or to no extent about who can

provide such legal assistance. Respondents noted in our telephone survey,

for example, that the terms “case handler,” and “paralegal” are not clearly

defined, and that they did not know whether nonlawyers (e.g., intake

workers) who are supervised by lawyers can provide legal assistance.

Comments made to us by the executive directors revealed that grantees

held varying views about who can provide legal assistance to clients.

Several respondents specifically stated that this is an important issue that

LSC should address in its handbook. Our own analysis of LSC guidance on

this issue indicates that LSC has not been consistent in its advice to

grantees. For example, in its communication with grantees on frequently

asked CSR questions and answers, LSC stated that a telephone

conversation between an intake specialist and a caller who was accepted

for service can be counted as a case if the caller received some advice that

addressed a specific legal problem. LSC officials told us that LSC’s current

position on this issue is that the person giving legal advice must be

someone who is authorized to practice law or is under the supervision of

an attorney in accordance with local rules of practice. Therefore, legal

advice can be given by a (1) lawyer, (2) paralegal, or (3) intake specialist

or law student under the supervision of a lawyer, as long as the assistance

does not violate local rules of practice.

 

5

The 1999 CSR handbook does not explicitly state that legal assistance must be provided by an attorney

or paralegal. However, section VIII of the handbook states that, to be reported as a case, the legal

assistance must meet the definition of “case” in 45 CFR 1635.2(a). This regulation defines a case as a

form of program service in which an attorney or paralegal provides legal services.
B-283551

Page 12 GAO/GGD-99-183 LSC Case Reporting Problems

LSC sought to determine the accuracy of grantees’ case data by requiring

that grantees do self-inspections of their open and closed caseload data for

1998. Grantees were required to determine whether the error rate in their

data exceeded 5 percent. If the error rate was 5 percent or less, they could

certify that their data were substantially correct. If the error rate was

higher than 5 percent, they were to determine how the problems identified

could be addressed. LSC found that about three-fourths of the grantees

were able to certify to the substantial accuracy of their data. LSC used the

results of the self-inspections to estimate the total number of case closings

in 1998. Our review of LSC’s self-inspection process raised concerns about

the accuracy and interpretation of the results, and what the correct

number of certifying programs should be.

On May 14, 1999, LSC issued a memo to all grantees instructing them to

complete a self-inspection procedure by July 1, 1999. The purpose of the

self-inspection was to ensure that (1) grantees were properly applying

instructions in the 1999 edition of the CSR Handbook that were applicable

to the 1998 data, and (2) LSC had accurate case statistical information to

report to Congress for calendar year 1998.

LSC provided detailed guidance to grantees on the procedures for the selfinspection. Each grantee was to select and separately test random samples

of open and closed cases to determine whether the number of cases it

reported to LSC earlier in the year was correct. Grantees were to verify

that the case file contained a notation of the type of assistance provided,

the date on which the assistance was provided, and the name of the case

handler providing the assistance. Grantees were also to determine whether

assistance had ceased prior to January 1, 1998; was within certain service

categories as defined by the 1999 handbook; was provided by an attorney

or paralegal; and was not prohibited or restricted. Finally, grantees were to

verify that each case had eligibility information on household income,

household size, household assets, citizenship attestation for in-person

cases, and indication of citizenship/alien status for telephone-only cases.

According to LSC officials, the self-inspection was a single procedure that

was undertaken within a limited time period, and LSC did not expect the

self-inspection to resolve all case-reporting problems.

In requiring grantees to verify that their 1998 case files contained

information on client assets and an indication of citizenship/alien status

for telephone-only cases, LSC imposed stringent criteria on the selfinspections. The criteria were stringent in that LSC had not promulgated

explicit documentation requirements related to these issues until it

released the 1999 CSR Handbook and July 14 program letter. Grantees

Most Grantees

Certified to the

Accuracy of Their 1998

CSR Data, but

Questions About Data

Accuracy and

Interpretation Remain

LSC Grantees Conducted

Self-Inspections of 1998

CSR Data
B-283551

Page 13 GAO/GGD-99-183 LSC Case Reporting Problems

were not required to apply these new documentation requirements to their

1998 case data. Recognizing that in 1998 many grantees did not keep

sufficient documentation on assets or citizenship/alien eligibility to comply

with the stringent self-inspection requirements, LSC allowed grantees

some latitude in determining whether they were in fundamental

compliance with the requirement that legal assistance only be provided to

eligible individuals. In conversations with grantee executive directors

during the self-inspection, LSC officials advised them that if they had a

level of certainty that their program staff had asked questions about assets,

and if their certainty was sufficient for them to sign a form attesting to

this, that was an acceptable basis for asserting compliance with the

reporting requirement for assets. Similarly, if grantees were sufficiently

certain that, for telephone cases, their program staff had asked questions

about citizenship/alien status but had not documented the inquiry, that too

was acceptable. Finally, LSC allowed grantees some latitude with respect

to the requirement that an attorney or paralegal must be the provider of

legal assistance. LSC advised grantees during the self-inspection that it was

acceptable for them to count as valid cases instances in which legal advice

was given by a (1) lawyer, (2) paralegal, or (3) intake specialist or law

student under the supervision of a lawyer, as long as the assistance did not

violate local rules of practice.

If any single aspect of a case failed to meet LSC’s requirements, the case

was to be classified as an error for reporting purposes. If the grantees

found that their CSR case sampling had an error rate of 5 percent or less,

the program directors and policy board chairs were to sign a certification

form and return it to LSC. Grantees who could not certify to the

correctness of their 1998 CSR data were to submit a letter to LSC

describing (1) the problems they had identified during the self-inspection

process and (2) the corrective actions they had instituted to address the

problems. Grantees could resubmit their 1998 CSR data to LSC if they

identified one or more problems in the random sample and corrected their

entire 1998 database so that the problems no longer appeared. If, by

correcting the problems, the error rate in the data was reduced to 5

percent or less, the grantees could resubmit their 1998 data along with a

signed certification attesting to the substantial accuracy of the resubmitted

data. In this way, grantees who were unable to certify at one point in time

could certify at a later point in time. As of July 29, 1999, 26 grantees had

resubmitted their 1998 CSR data after having made corrections to the data,

and 20 of them had certified their data.
B-283551

Page 14 GAO/GGD-99-183 LSC Case Reporting Problems

According to LSC officials, about three-fourths of the grantees certified the

accuracy of their 1998 case data. As of August 26, 1999, LSC documents

indicated that 199 of 261 grantees6

 (76 percent) reported substantially

correct CSR data to LSC. The remaining 62 grantees (24 percent) did not

certify to LSC that their CSR data were substantially correct. On the basis

of the self-inspection results, LSC estimated that grantees closed 1.1

million cases in 1998.

LSC officials told us that they were surprised that such a large number of

grantees were able to certify their 1998 CSR data. They attributed the

lower-than-expected error rates to the following factors:

• The self-inspection did not attempt to identify duplicate cases. LSC

officials explained that, during most of 1998, LSC did not have a standard

regarding duplicate cases, and that they believed it would have been too

burdensome on grantees for LSC to require them to apply the new

standard retroactively. According to an LSC official, duplicate cases are

best caught at the time of intake. The official also noted that duplicate

cases had not in the past been found to be a major problem in comparison

with other identified problems.

• Grantees received the new 1999 CSR Handbook in November1998, and a

number of them had implemented the requirements that applied to their

1998 data by the time they submitted their 1998 case statistics.

• Grantees were aware of the problem that the OIG had identified with cases

coded as referral after legal assessment. That is, some programs had

inappropriately reported as CSR cases numerous instances of making a

telephone referral without providing legal advice and/or without

documenting an individual’s eligibility. Because grantees had been

sensitized to this issue, LSC officials believed that they were less likely to

count these referrals as cases in 1998.

According to LSC officials, CSR problems were more common at larger

grantees that had heavier case loads and multiple branches. They noted

that approximately 30 of the 50 grantees with the largest caseloads were

unable to certify their 1998 data. They hypothesized that larger programs

may have difficulty certifying if such programs have (1) numerous branch

offices, one or more of which are out of compliance with regulations and

causing reporting problems for the entire program, and/or (2) numerous

 

6

LSC funded 262 programs in 1998. Funding for one program was discontinued in 1999, and LSC has no

self-inspection results for this program.

Most Grantees Certified

Their 1998 CSR Data
B-283551

Page 15 GAO/GGD-99-183 LSC Case Reporting Problems

sources of funding and more compliance requirements, which could add to

the complexity of their reporting process. The officials acknowledged that

these hypotheses needed further exploration.

LSC officials also reported that noncertifying grantees identified two

principal problems. One pertained to the lack of citizenship attestations in

case files. The second pertained to the lack of information on client assets.

LSC officials also noted that some grantees had reported matters, such as

referrals, as cases, and some had reported cases as open when they did not

meet the timely closing requirement. In our telephone interviews with 79

executive directors, 24 reported that they did not certify their 1998 data to

LSC. Factors that the executive directors of noncertifying programs

believed greatly affected the errors in their 1998 data included lack of clear

guidance from LSC (cited by 11 respondents), computer problems (cited

by 7 respondents), and insufficient attention by their programs to

administrative matters (cited by 5 respondents).

Computer problems that were noted included difficulties merging

databases and using several different software packages, and problems

with upgrading the computer system.

Based on the self-inspection results, LSC estimated that its grantees closed

about 1.1 million cases in 1998. LSC arrived at this figure by subtracting the

total number of closed cases estimated to be in error (135,498) from the

total number of cases that grantees reported to LSC (1,260,351).7

 As of July

29, 1999, LSC estimated that its grantees’ total closed caseload for 1998

was 1,124,853.8

LSC intends to report only the estimated number of cases closed in 1998,

even though the self-inspections were to encompass both open and closed

cases. According to LSC officials, LSC has less confidence in the open-case

numbers than the closed-case numbers. One reason for this is that they

believe open cases are more likely than closed cases to have timely closing

 

7

LSC performed this calculation separately for certifying and noncertifying programs. For noncertifying

programs, LSC determined that 96,273 cases were reported in error out of grantees’ original submission

of 475,856 closed cases. For certifying programs, LSC subtracted 5 percent, or 39,225 cases, from

grantees’ original submission of 784,495 closed cases. LSC reduced the case statistics of certifying

programs by 5 percent to ensure that there would be no overstatement of cases for these programs.

8

In April 1999, prior to the self-inspection, LSC’s CSR database indicated that grantees closed 1,338,976

cases in 1998. The difference between the April 1999 figure and LSC’s July 1999 estimate of 1,260,351

closed cases—that is, the number reported closed prior to adjusting for the self-inspection results— is

attributable to the fact that some grantees who found data errors corrected them and resubmitted their

statistical results to LSC. Those who initially did not certify their data may have later certified them if

their corrections reduced their error rate to 5 percent or less.

LSC Used Self-Inspection

Results to Estimate Number

of Cases Closed
B-283551

Page 16 GAO/GGD-99-183 LSC Case Reporting Problems

problems. A second reason is that some grantees experienced problems

when converting to a new computer system. System conversions

sometimes caused dates to be lost or cases to be mistakenly coded as

being open.

Our review of LSC’s self-inspection results raised some concerns about

LSC’s interpretation of the results and about the accuracy of the data

provided to LSC by grantees. As a result, we could not assess whether the

number of certified programs and case closures that LSC estimated for

1998 is correct, lower, or higher than it should be.

Although LSC provided instructions to grantees on how they should select

test samples and what case information they should verify, LSC did not

issue standardized procedures for grantees to use in reporting the results

of their self-inspections. Grantees that could not certify their data wrote

letters to LSC in which they described the errors they uncovered. The

letters contained varying degrees of detail about the errors. Some

programs provided an overall error rate and did not separately report how

much error they found in open and closed cases. Others provided detailed

information on both the number of errors found in open and closed cases,

respectively, as well as the number of errors broken out by type. Since LSC

did not have a standard protocol for collecting the results of the selfinspections, in some cases LSC officials had to rely on interpreting grantee

letters that described the problems that were discovered. LSC officials

believe that their numerous contacts with grantees who called them to ask

questions about the self-inspections, combined with their analysis of the

grantee letters, enabled them to correctly determine the number of

certifying programs and estimate the number of closed cases.

We are uncertain how many programs should have been counted as

certified because we are uncertain if LSC applied a consistent definition of

“certification.” Most programs that were on LSC’s certification list

determined that they had error rates of 5 percent or less for both open and

closed cases. However, LSC placed some programs on the certified list if

the program’s overall error rate for closed cases was 5 percent or less,

even if the overall error rate actually was higher than 5 percent. We

encountered this situation in two instances in which executive directors

told us in telephone interviews that they did not certify their CSR data

because their overall error rate exceeded 5 percent. However, these

programs appeared on LSC’s list of certified programs. When we asked an

LSC official about this, he told us that they advised grantees that if their

closed case error rate did not exceed 5 percent, they should “partially

certify” their data. In response to our inquiry, the official reviewed the

Self-Inspection Results

Raised Concerns
B-283551

Page 17 GAO/GGD-99-183 LSC Case Reporting Problems

certification letters submitted by nearly 200 grantees, and he identified 5

certified programs whose error rates for open cases exceeded 5 percent.

Given that some grantees submitted only an overall estimate of data error,

we do not know how many programs qualified to be certified overall, just

for closed cases, or just for open cases.

In another instance, an executive director told us that she did not certify

her program because 8 of the 14 case-handling offices had error rates

exceeding 5 percent. Nonetheless, this program appeared on LSC’s

certified list. An LSC official explained that, after reviewing the

information provided by this program, the official agreed that the program

should not have been classified as certified. In a fourth instance, in which

an executive director reported to us that he did not certify his program, an

LSC official said that, although they thought the grantee’s data had been

corrected, the program had not yet certified its data and might need to do

additional sampling.

We are also concerned that LSC’s instructions to grantees on how to

conduct the self-inspections may have led some of the smaller grantees to

select too few test cases to make a reliable assessment of the proportion of

error in their case data. For example, LSC instructed grantees to select

every tenth case for review if the program handled less than 1,000 cases in

1998. This was to be done separately for open and closed cases. Based on

1998 case statistics that grantees submitted to LSC between January and

March 1999, several programs would have based their certification

determinations on reviews of a relatively small number of cases. Seven

grantees had fewer than 300 closed cases, and 43 grantees had fewer than

300 open cases. Three programs had fewer than 300 cases combined. The

smallest program would have based its self-inspection on 3 closed cases

and 1 open case. We believe that, in general, samples of 30 or fewer cases

are too small to provide reliable estimates of the total number of case data

errors.9

 Because these were smaller grantees, this limitation would have

had little effect on LSC’s estimate of total closed caseload. However, it

could have affected—by either overstating or understating—LSC’s count

of the number of certified programs.

LSC does not know how well grantees conducted the self-inspection

process, nor how accurate the results are. We spoke with several executive

 

9

To illustrate, a program that reviewed 30 closed cases and found a single error would have had a 3

percent error rate. However, the margin of error based on a sample size of 30 would range from about

0.1 percent to 17 percent at the 95 percent statistical confidence level. If the program found two errors,

it would have had a 7 percent error rate, with a margin of error ranging from 1 percent to 22 percent.

Statistically, the smaller the number of cases sampled, the larger the margin of error.
B-283551

Page 18 GAO/GGD-99-183 LSC Case Reporting Problems

directors who did not correctly follow LSC’s reporting requirements. In

one case, the executive director sought clarification from LSC

headquarters about the use of a CSR closure code and was given incorrect

verbal guidance. The situation concerned whether applicant files could be

closed as “client withdrew” cases when individuals who were accepted for

service and assigned to staff attorneys did not subsequently appear for a

meeting with the attorney. According to both the executive director and an

LSC official, the executive director was told that if the program attempted

to contact these individuals, by telephone or letter, to determine whether

they were still interested in obtaining legal assistance, they could be

counted as cases. The executive director told us that she would review her

entire database for this type of error and make corrections. This program

was included on LSC’s certified list, but we do not know whether the

program would stay on that list if all the errors were identified.

Another executive director told us that he was concerned that LSC did not

want grantees to count assistance over the telephone as a case. This is not

an entirely correct interpretation of LSC guidance since, under certain

conditions, LSC permits legal assistance over the telephone to be counted

as a case. Although this program was included on LSC’s certified list, any

valid telephone cases that were not counted would have erroneously

increased its error rate.

A third executive director told us that the timely closing rule required him

to close new cases in December. If the person sought assistance with a

similar problem in January, it would be treated as a new case because a

new reporting year had begun. In a March 1999 written communication

intended to supplement the 1999 handbook, LSC advised grantees to

exercise discretion about when to close cases that were opened near the

end of the year. LSC did not require grantees to close all cases at year-end.

These examples illustrate the possibility that incorrect interpretations of

LSC guidance may have resulted in some programs certifying their 1998

data when they shouldn’t have, and other programs not certifying their

1998 data when they should have. An LSC official told us that, although

they have conducted CSR training sessions for grantee executive directors,

there are thousands of case handlers in grantee offices who have not

received similar training. The official acknowledged that written guidance

and telephone contacts with grantees may not be sufficient to ensure

correct and consistent understanding of reporting requirements, and that

LSC may consider alternative ways of providing training to staff, such as

through videos.
B-283551

Page 19 GAO/GGD-99-183 LSC Case Reporting Problems

Incorrect interpretations of LSC guidance could also have affected the

accuracy of LSC’s estimate of closed cases. The Inspector General told us

that his office has completed audits of six grantees’ 1998 case data. One

additional audit of a grantee’s CSR data had not yet been completed. The

six completed audits included both certifying and noncertifying grantees.

According to the Inspector General, the results of the six completed audits

would be provided to Congress by September 29, 1999.

LSC officials told us that the self-inspection was valuable and that LSC

plans to have grantees complete self-inspections again early next year as

part of the 1999 CSR reporting process. We agree that a self-inspection

process can be valuable, provided that grantees have clear, consistent, and

appropriate guidance on the procedures for reviewing their case data,

determining whether to certify the data, correcting errors, and providing

their results to LSC in a standardized way that facilitates validation of the

results.

LSC’s 1999 CSR Handbook and other written communications have

improved the clarity of reporting requirements for its grantees. However,

many grantees remain unclear about and/or misunderstood certain aspects

of the reporting requirements. LSC’s practice of disseminating guidance

primarily by written or telephone communications may not be sufficient to

ensure that grantees correctly and consistently interpret the requirements.

LSC sought to determine the accuracy of grantees’ 1998 case statistics by

requiring grantees to conduct self-inspections. However, we do not know

the extent to which the results of the self-inspection process are accurate.

The validity of the results are difficult to determine because LSC did not

standardize the way that grantees were to report their results, some of the

grantees used samples that were too small to assess the proportion of

error in their data, some grantees did not correctly follow LSC’s reporting

guidance, and LSC had done no verification of the grantees’ self-inspection

procedures.

We do not believe that LSC’s efforts, to date, have been sufficient to fully

resolve the case reporting problems that occurred in 1997.

We recommend that the President of LSC

• clarify and disseminate information on the specific information on client

assets that grantees must obtain, record, and maintain;

Conclusions

Recommendations
B-283551

Page 20 GAO/GGD-99-183 LSC Case Reporting Problems

• clarify and disseminate information on the types of citizenship/alien

eligibility information grantees must obtain, record, and maintain for

clients who receive legal assistance only over the telephone;

• clarify and disseminate LSC’s criteria for single recording of cases;

• clarify and disseminate LSC’s policy concerning who can provide legal

assistance to clients for the service to be counted as a case;

• explore options for facilitating correct and consistent understanding of

reporting requirements, such as developing and disseminating a training

video for grantee staff;

• develop a standard protocol for future self-inspections to ensure that

grantees systematically and consistently report their results for open and

closed cases;

• direct grantees to select samples for future self-inspections that are

sufficient to draw reliable conclusions about magnitude of case data

errors; and finally,

• ensure that procedures are in place to validate the results of LSC’s 1998

self-inspection, as well as of any future self-inspections.

The President of LSC provided written comments on a draft of this report,

which are printed in full in appendix I. LSC generally agreed with our

findings and stated that it intends to implement our recommendations.

Concerning the issue of the clarity of case reporting guidance, LSC’s letter

stated that LSC is troubled by our finding that some grantees are

continuing to have difficulty in this area. LSC’s letter reiterated a point that

we made in the report; namely, that the overlap in time between our data

collection effort and LSC’s distribution to grantees of a July 1999 program

letter intended to clarify client eligibility documentation requirements may

have caused some of the executive directors not to factor in the new

guidance when they responded to our telephone interview questions.

Although, as we state on pages 9 and 10 of the report, some respondents

may not have been familiar with LSC’s July 1999 guidance, two

respondents told us that they were familiar with the new guidance and that

they were still unclear about the requirements dealing with assets. We also

state on page 10 of the report that our own analysis indicated that LSC was

not consistent in its guidance to grantees on what asset information they

needed to obtain from clients. Therefore, we do not believe that grantees’

Agency Comments
B-283551

Page 21 GAO/GGD-99-183 LSC Case Reporting Problems

lack of clarity concerning reporting requirements on client eligibility was

due solely to the overlap in the time period of our data collection and

LSC’s distribution of the July 1999 program letter.

With respect to grantees’ lack of clarity about how to determine duplicate

case counting, LSC indicated that it will consider providing additional

guidance to grantees through further revisions to the CSR handbook. LSC

also indicated that it will revise the handbook to clarify the issue of when

legal assistance by nonlawyers can be reported as a case.

LSC also reiterated that it is developing additional methods to supplement

written guidance, including (1) conducting training and technical

assistance, (2) developing case management standards that would detect

and prevent cases from being accepted and reported if the required

eligibility documentation was not obtained, and (3) developing database

queries for grantees to apply to their case management systems to identify

instances where cases lacked required compliance documentation.

Concerning the self-inspection results, LSC noted that its principal purpose

in requiring grantees to self-inspect their 1998 case data was to ensure that

case-reporting guidance was being properly applied. LSC stated that

assessing the accuracy of the 1998 case statistics that grantees submitted

to LSC in March 1999 was a secondary purpose of the self-inspection. LSC

believes that the self-inspection increased grantees’ awareness of reporting

requirements and prompted them to make changes in their case reporting

practices. Indeed, in LSC’s view, the self-inspection requirement resulted

in more program improvements than did the 1999 CSR handbook and other

written guidance issued by LSC because grantees that identified significant

problems were required to take corrective actions. LSC notes correctly

that, in relation to the self-inspection, our report focuses on the accuracy

of LSC’s determination of the number of grantees that certified their data,

and the factors that affected the accuracy of LSC’s estimate of the total

number of closed cases in 1998. Our review focused on the stated goals of

the self-assessment. Although we did not assess whether, relative to LSC’s

other efforts, the self-inspection had a greater effect on grantees’

awareness of and compliance with CSR reporting, we do report that 26

grantees made corrections to their 1998 data as a result of their selfinspections.

As arranged with your offices, unless you publicly announce the contents

of this letter earlier, we plan no further distribution until 7 days after the

date of this letter. At that time, we will send a copy of the report to the

Chairmen and Ranking Minority Members of LSC’s appropriations and
B-283551

Page 22 GAO/GGD-99-183 LSC Case Reporting Problems

legislative committees and to Mr. John McKay, the President of LSC. The

major contributors to this report are acknowledged in appendix II. If you

or your staff have any questions concerning this report, please contact me

or Evi L. Rezmovic on 202-512-8777.

Laurie E. Ekstrand

Director, Administration of Justice Issues
Page 23 GAO/GGD-99-183 LSC Case Reporting Problems
Appendix I

Comments From the Legal Services

Corporation

Page 24 GAO/GGD-99-183 LSC Case Reporting Problems
Appendix I

Comments From the Legal Services Corporation

Page 25 GAO/GGD-99-183 LSC Case Reporting Problems

Now on pp. 9-11.
Appendix I

Comments From the Legal Services Corporation

Page 26 GAO/GGD-99-183 LSC Case Reporting Problems
Appendix II

GAO Contacts and Staff Acknowledgments

Page 27 GAO/GGD-99-183 LSC Case Reporting Problems

Laurie E. Ekstrand, 202-512-8777

Evi L. Rezmovic, 202-512-8777

In addition to those named above, Mark Tremba, Kristeen McLain,

Jan Montgomery, David Alexander, Barry Seltser, Lemuel Jackson,

Brian Lipman, and Walter Vance made key contributions to this report.

GAO Contacts

Acknowledgments
Page 28 GAO/GGD-99-183 LSC Case Reporting Problems
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the

following address, accompanied by a check or money order made

out to the Superintendent of Documents, when necessary. VISA

and MasterCard credit cards are accepted, also. Orders for 100 or

more copies to be mailed to a single address are discounted 25

percent.

Order by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000 or by using

fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any list

from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to

obtain these lists.

For information on how to access GAO reports on the INTERNET,

send e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. G100

(182080)