Student Financial Aid: Schools' Experiences Using the National Student Loan Data System
Released: 1998-09-24T00:00:00

Letter

Background

The Department of Education administers four major student financial aid programs under title IV of HEA: FFELP, the William D. Ford Direct Loan Program (FDLP), the Federal Pell Grant Program, and the Federal Campus-Based Programs. These programs together will make available an estimated $47 billion to about 8 million individuals during the 1998-99 academic year—about 80 percent of it in student loans.

Schools are responsible for obtaining and evaluating the financial aid history of students to ensure that students are eligible for aid. During our review, we identified 10 tasks inherent to administering federal student aid programs through NSLDS or other methods. These tasks are related to four general processes and functions operational at the time of our survey that the Department made available to schools through NSLDS to help them administer the student financial aid programs more effectively. (See table 1.)

Once the school determines that a student is eligible, financial aid funds are disbursed to the student according to program requirements.

To support—as well as monitor—these student loan programs, the Department has developed a number of automated processing systems, including NSLDS. The budget for these systems is expected to be about $378 million for fiscal year 1999. (See app. III for descriptions of the major student financial aid systems.) Prior to NSLDS, the Department relied on a system—commonly referred to as the guaranty agency tape dump—to collect selected information from guaranty agencies on each federal student loan. The tape dump, developed in the late 1970s, was initially intended to be used by the Department primarily as an annual source of data for analysis of program trends. According to the Department, it did not expect that every guaranty agency would have historically collected all the data requested because each agency’s system was designed to meet the needs of that individual guaranty agency. The tape dump, according to the Department, was not designed to be used, for example, to prevent awarding loans to ineligible borrowers.

Under 1986 HEA amendments, the Secretary of Education was authorized to replace the tape dump and develop a computer system that would make national student loan data accessible to guaranty agencies; however, the Department could not require guaranty agencies to use the database before approving new loans. As a result, planning for the development of the new NSLDS was delayed for several years, when the Department was allowed, under the Omnibus Budget Reconciliation Act of 1989, to require guaranty agencies to use the system in determining student eligibility.

The scope of the database expanded when 1992 HEA amendments required the Department to integrate a national student loan database with other financial aid data systems. The 1992 amendments also stated that the Secretary of Education, in establishing a national database, should give priority to providing information on student enrollment and status, current loan holders, and servicers. In response to these legislative mandates, in January 1993, the Department awarded a 5-year, $39 million contract to develop and maintain NSLDS. As of March 1998, the costs for developing, implementing, and maintaining the system have totaled $96.5 million.

According to the Department, NSLDS contained about 118 million loan and grant records as of February 1998. These records were provided by guaranty agencies for the FFELP loans they guaranteed, by the contractor that services FDLP loans for the Department, and by schools for Pell grants and campus-based aid they awarded. As figure 1 illustrates, a significant portion of data stored in NSLDS—about 70 percent—related to FFELP loans as of March 1998.

The Department’s student financial aid data systems have suffered from data quality problems. For example, in 1996, the Department’s Office of Inspector General (OIG) found in its review of the September 30, 1992, tape dump that the number of FFELP loans in repayment was overstated by approximately 5.9 million loans. OIG also found that a significant number of loans that were incorrectly recorded in the tape dump remained incorrect in NSLDS, affecting the reliability of the new system. As we reported in 1997, poor quality and unreliable FFELP loan data remain in the Department’s systems, and inaccurate loan data were being entered into NSLDS. As a result, the Department cannot obtain complete, accurate, and reliable FFELP data, which, according to its OIG, hinders the Department’s effort to monitor borrowers and properly award aid to those who are eligible. The Department acknowledges that its student financial aid data systems have suffered from data quality problems and has initiated a number of actions to address data accuracy and integrity issues.

Regardless of its weaknesses, NSLDS is the most comprehensive departmental database on federal student loans that schools, lenders, and guaranty agencies can use. The Department has stated that it expected NSLDS to significantly reduce schools’ and students’ administrative burden of applying for and accounting for financial aid. With the exception of mid-year transfer students, the Department has not required schools to obtain paper financial aid transcripts since the 1996-97 award year. The Department also envisioned that NSLDS would simplify and enhance the process of updating SSCR information for schools. Prior to NSLDS, schools, for their FFELP loans, received SSCR rosters from every guaranty agency that guaranteed their student loans and had to manually verify and resubmit the rosters to the guaranty agencies. In addition, the Department believes that NSLDS has led to considerable improvements in identifying ineligible student aid recipients. It estimates that since the 1994-95 academic year, for example, NSLDS’ improved default matching capabilities may have prevented over $1 billion from being awarded to ineligible students.

Schools’ Use of NSLDS’ On-Line and Batch Processing Functions Is Limited

Our survey results indicate that a significant proportion—42 percent—of the schools participating in federal student financial aid programs were not using NSLDS’ on-line and batch processing functions, which first became available in November 1994. In addition, of the schools that were using these functions, most were not routinely using them for many of the tasks they were capable of performing.

Based on our survey results, we estimate that 58 percent of the 6,181 schools participating in student financial aid programs used NSLDS’ on-line or batch processing functions. (See fig. 2.) Of the schools that were not using these functions, they either had the ability to use it and simply had not done so (19 percent) or did not have the ability to use the system (23 percent).

When we asked schools about their use of NSLDS to accomplish the 10 tasks we identified as inherent to program administration, we found that schools were not routinely using NSLDS for most of them. As shown in figure 3, more than half of the schools rarely or never performed 7 of the 10 tasks using NSLDS.

The only task that a majority of schools routinely used NSLDS to accomplish was providing or updating SSCR information, which since February 1997, the Department has required schools—or third-party servicers on their behalf—to perform on the system. Specifically, our survey data show that 69 percent of the schools always or most of the time use NSLDS for SSCR processing. An additional 11 percent occasionally use NSLDS for SSCR processing, and another 5 percent rarely use the system for this task. Of the remaining schools—those that never use NSLDS for SSCR processing—13 percent process SSCRs through the Clearinghouse or other third-party servicers; the last 2 percent appear not to be meeting the requirement. The schools’ next most common uses of NSLDS’ on-line and batch processing functions were for correcting or updating student information (33 percent) and determining prior loan defaults (30 percent).

However, 43 and 47 percent of schools responded that they rarely or never use NSLDS for these two tasks, respectively.

The reason schools most frequently gave for rarely or never using NSLDS to perform any of the 10 tasks was that they use the Department’s Student Aid Reports (SAR) or Institutional Student Information Records (ISIR).Schools can identify loan status and determine student eligibility from information contained in SAR or ISIR, including prior defaults, types of loans, default dates, and outstanding balances. Other reasons schools often gave are that they used the Clearinghouse for SSCR processing, they did not know how to use NSLDS, and that they had experienced problems using the system.

Schools Encountered Few Problems or Data Inaccuracies Using NSLDS and Are Satisfied With Training and Customer Service

When we asked schools that were using NSLDS’ on-line and batch processing functions whether they experienced problems, such as transmitting or receiving data, or whether they encountered inaccuracies in NSLDS data, many reported that they rarely or never experienced problems or encountered inaccuracies in the data fields we specified. Many schools also responded that they are satisfied with the support they received from training and customer service.

Some schools, however, responded that they experienced problems more frequently with certain functions, including correcting and updating SSCR information or understanding error messages. Some schools said they occasionally found inaccuracies in data fields critical to correctly identifying students—such as Social Security numbers, last names, and date of birth—and other critical data fields, including enrollment status. While school responses indicate that these problems may not be widespread, they suggest that NSLDS does not yet offer the level of data accuracy expected by the Department.

Schools Rarely or Never Experienced Problems With Most Aspects of NSLDS, but Some Problems Did Exist

In general, schools’ experiences with most aspects of using NSLDS have been essentially problem free. (See fig. 4.) For example, 85 percent of schools that use NSLDS rarely or never experienced problems identifying multiple entries in the system of the same loans or grants.

However, some schools did experience problems with some aspects of the system. For example, about 23 percent always or most of the time had a problem correcting or updating SSCR information, a critical tool for effectively administering student financial aid. As we reported in 1997, some FDLP student loan borrowers have not started to repay their student loans after they are no longer enrolled in school. Reasons given were schools’ failing to report enrollment changes to the Department or the Department’s failing to accurately or promptly record the reported enrollment changes. Failure to record a borrower’s enrollment changes or status may result in borrowers not promptly repaying their student loans or in the loans becoming delinquent, increasing the likelihood of defaulting.

Although less than 10 percent of the schools found inaccuracies in NSLDS data fields always or most of the time, 7 to 29 percent of the schools occasionally found inaccuracies in these fields. (See fig. 5.) As we reported in 1997, recording a student’s correct Social Security number, which the Department considers its common student identifier, is critical for ensuring that aid is awarded to the correct individual and for identifying an individual’s data records.

NSLDS as well as other Department information systems also use combinations of unique elements—such as a student’s date of birth and the first two to three letters of the first or last name—to identify, access, and update a specific student’s record. Therefore, the Department’s information systems depend on these data fields to be as accurate as possible.

School responses indicate that inaccuracies in the data fields do not appear to be widespread, but they suggest that NSLDS does not yet offer the level of data accuracy expected by the Department. The Department depends on guaranty agencies, loan servicers, and other entities to provide many of these data and has initiated efforts to address these problems.

Schools That Use NSLDS Are More Satisfied Than Dissatisfied With the Support From Training and Customer Service They Received

In preparing schools for NSLDS’ implementation, the Department sent training materials to each school, made training sessions available to them, and established a customer service center to respond to questions and otherwise assist schools. Seventy-nine percent of schools using NSLDS said that they or their servicers received training or training materials from the Department. As figure 6 shows, 44 percent of the schools that received training or training materials were satisfied with the training, about 30 percent were neither satisfied nor dissatisfied, and 27 percent were dissatisfied.

About 80 percent of schools that use the system responded that they requested assistance or information from the NSLDS Customer Service Center. As figure 7 shows, more than two-thirds of the schools that received assistance responded that they were satisfied with the assistance provided by the NSLDS Customer Service Center.

Schools Had Mixed Views About Whether NSLDS Improved Student Financial Aid Administration

Schools using NSLDS’ on-line and batch processing functions had mixed views about whether these capabilities led to improvements in their ability to administer federal student financial aid programs. Although some schools identified several areas where NSLDS had improved their program administration, other schools thought NSLDS did not lead to improvements or make a noticeable difference in other areas. For example, half or more of the schools agreed or strongly agreed that NSLDS had improved the availability of student data, made student data easier to access, and reduced the amount of paper handled in administering financial aid programs. (See table 2.) However, a significant percentage of schools disagreed or strongly disagreed that NSLDS had reduced the staff required to administer student financial aid programs (49 percent) or reduced the time required to administer student financial aid (39 percent).

Schools Had a Variety of Reasons for Not Using NSLDS’ On-Line and Batch Processing Functions

Our survey results suggest that schools have a variety of reasons for not using NSLDS’ on-line and batch processing functions. Twenty-three percent of all schools lacked the ability to access the system, and 19 percent of all schools had the ability to do so but did not.

We estimate that—at the time of our survey—more than 1,000 schools lacked the ability to access the system; of these, a number planned to obtain access within 8 months. However, most had no plans to obtain access to NSLDS, had plans to obtain access but did not know when, or were unsure whether they would ever obtain access. Most of these schools were using the Clearinghouse or other third-party servicers to perform tasks, but several hundred schools appeared not to be accessing NSLDS either directly from their campus or through the Clearinghouse or other third-party servicer. Since these schools did not have plans to obtain access, they may not be meeting a requirement to have on-line access to NSLDS from their campuses as of January 1998.

While the schools that did not use NSLDS’ on-line and batch processing functions—either because they chose not to (19 percent) or did not have the ability to (23 percent)—had a variety of reasons for not using or having NSLDS, these reasons generally fell into one of three categories: use of alternative methods, such as the Clearinghouse, for obtaining and processing SSCR information; limitations in resources, personnel, or skills, such as lack of training; and lack of confidence in data reliability. (See fig. 8.)

The Department Has Taken Steps to Promote Greater School Use of NSLDS

The Department has efforts under way to increase schools’ use of NSLDS. While a small percentage of schools have not complied with requirements for SSCR processing and on-line access, the Department recognizes that to encourage schools to use NSLDS beyond these requirements, it must promote the system and ensure the accuracy of the system’s data. Therefore, the Department plans to provide additional NSLDS training to users at sites throughout the country and review and correct any inaccurate data in NSLDS. Although these actions were not developed specifically in response to the concerns schools had cited in our survey, they address many of the issues schools raised.

Actions to Increase Compliance With Requirements for Using NSLDS Are in Place

Schools are required to use NSLDS for SSCR processing. They may do so directly from their campuses or through the Clearinghouse or other third-party servicers. Our survey shows that 5 percent of all schools are not meeting this requirement. The Department has independently identified schools that have not complied with the requirement and taken a number of actions to increase compliance. For example, it has assessed fines ranging from $1,000 to $7,500 against 19 schools.

As of January 1998, schools are also required to have on-line access to NSLDS for their financial aid staff. The Department has estimated that 2 percent of all participating schools have not registered for on-line access. The contractor responsible for operating NSLDS is contacting these schools at the Department’s instruction to attempt to get them to register.

Additional NSLDS Training Has Been Scheduled

The Department plans to offer more training to increase participation among those schools that do not access NSLDS. According to Department officials, the Department already offers a variety of training opportunities and has no plans to alter the types of training it offers but will increase the number of training sessions. In February 1998, the Department began offering a series of workshops aimed at helping schools automate their financial aid offices. These workshops include components on using NSLDS. More recently, the Department began offering computer-based training sessions solely on how to use NSLDS. The Department expects that between August and October 1998, it will have offered 39 NSLDS sessions—each accommodating 40 participants—at its 11 regional training centers. If there is demand and sufficient funding, the Department plans to offer more training sessions in spring 1999. For school personnel who cannot travel to a training site, the officials said schools can use a self-paced computer program included with the NSLDS users’ manual.

Data Accuracy and Integrity Are Being Addressed

To ensure the overall accuracy of NSLDS data, the Department has initiated “serious and aggressive” data integrity efforts to review and correct any inaccurate or incomplete data, according to Department officials. These efforts include providing detailed technical instructions to data originators and providers (schools, lenders, and guaranty agencies); focusing on correcting inaccurate data; and addressing the issue of data quality in its annual performance plan.

The Department gave the major data providers detailed technical instructions that specify their responsibilities and the data they are required to provide. According to Department officials, data providers must correct data problems as quickly as possible, especially since they are the ones that usually identify the problems. Correcting data problems may require system changes or major data entry and research. The Department provides technical update bulletins on an as-needed basis, training sessions, and on-site technical reviews. Further, the Department tracks whether providers have corrected the data problems and sends monthly management reports to all providers. If a significant problem occurs when two or more data providers have conflicting information each believes is correct, the dispute must be formally adjudicated within the Department.

According to Department officials, to increase both voluntary use of and school satisfaction with NSLDS, schools must have confidence in the system’s data. The Department’s strategy focuses on increasing the accuracy of loan data as well as strengthening relationships with the financial aid community. For example, the Department’s data improvement efforts focus on deleting duplicate loans and loans with a “zero balance” (that is, loans that have been paid in full); identifying loans not in NSLDS that should be; resolving differences between NSLDS and lender databases; and conducting outreach efforts, such as holding regular workshops with other data providers. The Department is now taking these measures to address problems with FFELP and Perkins loan data. Department officials said that problems with FDLP data have been largely corrected.

To further demonstrate its commitment to improving the reliability of data on its financial aid programs, the Department has addressed the issue of data integrity in its long-range strategic and annual performance plans prepared in response to the Government Performance and Results Act of 1993. For example, in its fiscal year 1999 performance plan—its first annual plan—the Department acknowledges that its student financial aid delivery system has suffered from data quality problems that are severe enough to cause it to fail to receive an unqualified audit opinion. In its June 15, 1998, audit report, the Department’s OIG reported that, in its opinion, the consolidated financial statements present fairly, in all material respects, the financial position of the Department as of September 30, 1997. However, the reliability of data in NSLDS is still a material internal control weakness. Specifically, the audit report states that the Department’s ability to continue to prepare auditable loan estimates for its financial statements depends on establishing a reliable store of up-to-date historical loan data. The audit report notes that because of questionable data in NSLDS, the estimated liability for loan guarantees was based on data received from 10 large guaranty agencies, as opposed to NSLDS.

According to the Department’s performance plan, steps are being taken to improve the efficiency and quality of its student aid data. These include improving data accuracy by receiving individual student loan data directly from lenders rather than through guaranty agencies and by expanding efforts to verify the data reported to NSLDS and preparing a system architecture for the delivery of federal student aid by December 1998 that will help integrate the multiple student aid databases with NSLDS based on student-level data to improve the availability and quality of information on student aid applicants and recipients.

Agency Comments

The Department of Education provided written comments on a draft of this report in a letter from the Assistant Secretary for Postsecondary Education, dated August 14, 1998. Overall, the Department felt that the draft report was not balanced or written fairly and that it did not accurately portray the full extent of schools’ use of NSLDS. The comments expressed three specific concerns: (1) inappropriate emphasis placed on negative aspects of the survey results, (2) numerically distorted survey results, and (3) outdated audit report information in the background section that had no relation to the scope of the audit.

In addressing the Department’s comments, we revised the report, as appropriate, to clarify that our review focused on schools’ use of NSLDS’ on-line and batch processing functions rather than all school uses of NSLDS. To address the specific concerns about our presentation of survey results, we made other revisions, where appropriate, to better ensure that our results were presented objectively and fairly. However, we continue to focus the reader’s attention on those responses that show the extent to which schools were encountering shortcomings in using NSLDS’ on-line and batch processing functions. Finally, to address the concern about our use of previous audit reports, we added information on more recent activities, such as the Department’s efforts to improve the quality of NSLDS data. However, we retained a discussion of previous audit reports because we believe it is needed to establish the historical context and significance for creating NSLDS.

Department officials discussed these and other concerns and provided other comments in meetings with our staff on July 31 and August 7, 1998. (See app. IV for a discussion of these comments and our responses and a reprint of the Assistant Secretary’s August 14, 1998, letter.)

We are sending copies of this report to the Secretary of Education, appropriate congressional committees, and other interested parties. We also will make copies available to others on request.

If you have any questions about this report, please contact me at (202) 512-7104. Major contributors to this report are listed in appendix V.

Responses to Questions From GAO Survey on NSLDS Use

This appendix contains schools’ responses to our survey. All numbers are percentages, except for those in questions 2, 4, 7, 14, 27, 31, and 33. Percentages shown are based on the number of respondents answering each question. Percentages may not always add to 100 percent due to rounding.

Scope and Methodology

To address our objectives, we (1) surveyed a random sample of schools that participate in financial aid programs, (2) reviewed pertinent documents, and (3) spoke with Department of Education officials and members of the higher education community.

To obtain information on schools’ use of NSLDS, including problems encountered and benefits derived, we surveyed a random sample of postsecondary colleges and universities that participated in federal student financial aid programs as of August 1997. During the development of the survey instrument, Department officials informed us that all schools use NSLDS because data they receive on SARs and ISIRs are generated from the system; therefore, we focused our survey questions on the use of NSLDS’ on-line and batch processing functions. In addition, we reviewed NSLDS documents obtained from the Department and prior GAO reports.

To identify measures undertaken by the Department to ensure schools’ use of the system, we interviewed officials from the Department as well as staff from Raytheon/E-Systems, the contractor responsible for developing and maintaining NSLDS. We discussed NSLDS’ current operation, including the functions available for school use, the number of schools that have access to NSLDS, methods by which information is transmitted into NSLDS, and how users gain access to this information.

To aid in designing our survey instrument, we also contacted members of the higher education community. We interviewed officials from the American Association of State Colleges and Universities, the American Council on Education, the Coalition of Higher Education Assistance Organizations, the National Association of Independent Colleges and Universities, the National Association of Student Financial Aid Administrators, and the National Student Loan Clearinghouse.

We conducted our study between May 1997 and July 1998 in accordance with generally accepted government auditing standards.

Survey Design

To determine the extent of schools’ use of NSLDS’ on-line and batch processing functions, we developed a survey instrument and sent it to a randomly selected sample of postsecondary schools. The survey covered a variety of topics, including descriptive background data on each school, types of access to NSLDS, and actual experiences using NSLDS’ on-line and batch processing functions. For example, we asked schools to identify themselves as public or private and to provide the size of their student body. We also asked the schools that reported having access to NSLDS whether they used the system themselves or through a third-party servicer. In addition, we asked schools to provide information on their use of various features of NSLDS, including benefits derived and problems encountered. To determine the purposes for which schools use NSLDS, we reviewed NSLDS manuals and pretested our instrument with 12 schools.

Survey Universe and Response

We drew our sample of 600 postsecondary schools from 6,181 schools listed in the Department’s automated Postsecondary Education Participants System (PEPS) as of August 1997. We mailed our instrument to the 600 schools in November 1997. We did a follow-up mailing in December 1997 and again in January 1998.

Of the 600 schools, we determined that 11 were ineligible for our survey—because they no longer participated in federal student aid programs, were high schools rather than postsecondary schools, or had closed—resulting in an adjusted sample of 589 schools. Of these, 490 schools returned completed, usable survey instruments, which yielded a school response rate of 83 percent.

Our analyses are based on the 490 responses from 83 percent of the eligible schools sampled. All data are self-reported, and we did not independently verify their accuracy.

NSLDS Processes and Functions Included in Our Survey

We included in our survey four of the six NSLDS processes and functions available to schools (listed in the Department’s NSLDS users’ manual, NSLDS: The Paperless Link): prescreening for eligibility, borrower tracking, SSCR data, and financial aid transcripts. We did not include the remaining two functions—overpayment data and report selection—because the first was not available at the time of our review and the latter duplicated the four processes and functions included in our survey.

Prescreening for eligibility: The prescreening function allows schools to receive data on prior student financial aid recipients, enabling schools to determine the eligibility of financial aid applicants before funds are awarded and thereby reduce defaults.

Borrower tracking: This on-line NSLDS function is generally used by loan servicers and guaranty agencies attempting to locate a borrower who has defaulted on a student loan. NSLDS provides data on other organizations (such as schools and lenders) associated with the borrower, which servicers and agencies can contact to obtain the borrower’s current address.

SSCR: Schools are required to use this function to confirm and report the enrollment status of students who receive federal loans.

Financial aid transcript: NSLDS’ financial aid transcript function summarizes all previous title IV financial aid a student has received. Histories are received on students currently attending or transferring to an institution. Financial aid transcripts are reviewed by a financial aid administrator to determine current levels of aid, whether there is any information that would prevent awarding aid for the first time, or to continue aid to an enrolled or enrolling student.

Overpayment: NSLDS’ overpayment function—added since our survey—will enable schools to notify NSLDS that a student owes a refund of an overpayment on a Pell grant, State Student Incentives Grant, or SEOG grant, as well as a Perkins Loan. An overpayment notification to NSLDS notifies the entire student financial aid community because the actual overpayment data appear on all financial aid transcripts that are requested through NSLDS and through prescreenings of ISIRs and SARs.

Report selection: Reports and extracts are produced by NSLDS on both a regularly scheduled and on-request basis. Schools may query the system regarding the existence of reports, extracts, or both and may gain access to them via an on-line display or a file deposited to their wide area network mailbox.

Identified Administrative Tasks Included in the Survey

During the pretest, we identified 10 tasks inherent to administering federal student aid programs. Schools can perform these tasks by using NSLDS’ on-line and batch processing functions directly or using other methods. We also identified operations during which schools might encounter problems using the system and data fields in which they might encounter inaccuracies.

In the survey, we asked users of the system to identify the tasks for which they used NSLDS and any problems they had encountered, including what kinds of inaccuracies, if any, they had found in the data.

Respondent Use of NSLDS

For the purposes of this survey, we defined use of NSLDS as accessing it through on-line or batch processing functions designed to perform specific tasks. To focus on the direct use of these functional capabilities by school personnel, we instructed school officials completing the survey instrument not to consider their use of student eligibility information reports generated from the NSLDS database and sent by the Department to be a use of the system. Similarly, if schools only used the Clearinghouse for processing SSCR reports and did not use any of the system’s other functional capabilities, we did not consider those schools to be NSLDS users.

We directed our survey to the officials at the selected schools whom we determined to be the most knowledgeable about NSLDS use and federal student financial aid programs. To identify the appropriate respondent at each sample school, we sent a letter to the Director of Financial Aid, which both alerted the school to our survey and requested that the school return a postcard with the name and address of the appropriate recipient, if different from the financial aid office.

Sampling and Nonsampling Errors

All sample surveys are subject to sampling errors, that is, the extent to which the results differ from what would be obtained if the whole population had received the survey instrument. Since the whole population does not receive the instrument in a sample survey, the true size of this difference cannot be known. However, it can be estimated from the responses to the survey.

Using the number of respondents and the amount of variability in the data, we were able to estimate sampling errors for our survey. (See table II.1.)

Margin of error (percent)

In addition to sampling errors, surveys are subject to other types of systematic error or bias that can affect results. Bias can affect both response rates and the way respondents answer particular questions. We cannot assess the magnitude of the effect of bias, if any, on our survey results. Rather, possibilities of bias can only be identified and accounted for when interpreting results. One possible source of bias in our survey is inherent in all self-ratings and self-reports. Bias inherent in self-rating and self-reporting may impact survey results because integrity of the data depends upon respondents providing honest and accurate answers to survey questions. The results of this report are affected by the extent to which respondents accurately reported their school’s use or non-use of NSLDS.

We took several steps to minimize the impact of nonsampling errors. First, we examined responses for extreme values and inconsistencies. In a few cases, respondents had reported numbers incorrectly, and in these cases, we corrected the data or, if correction was not possible, we rejected the data known to be in error.

Major Student Financial Aid Systems

There are six major student financial aid systems that provide various types of information on student loans.

Campus-Based Programs System

The Campus-Based Programs System supports all data tracking and reporting functions associated with campus-based programs. This system uploads and edits data received from participating schools; calculates tentative and final school awards, notifying schools of their award levels; allocates funds; and reconciles school accounts. This system contains no student-level information; it uses only summary data by school.

Central Processing System The Central Processing System supports student financial aid applications and the determination of Pell grant eligibility; matches other databases for applicant eligibility; makes corrections to the records; and produces statistical analysis tables, student data rosters, and tapes for schools and state agencies.

Direct Loan Servicing System

The Direct Loan Servicing System services FDLP loans while the borrower is in school, in deferment status, or in repayment.

National Student Loan Data System

The National Student Loan Data System performs prescreening of student financial aid program applications, performs student status confirmation reporting, and tracks borrowers. The system contains information regarding loans made, insured, or guaranteed under title IV and selected Pell grant information. Its purposes are to (1) ensure that accurate and complete data on student loan indebtedness and institutional lending practices are available, (2) screen applications to identify prior loan defaults and grant overawards, (3) provide a database to research and identify trends and patterns, (4) support audits and program reviews, and (5) calculate default rates.

Pell Grant Recipient and Financial Management System

This system receives, evaluates, and processes student payment data and serves as the basis for obligations to schools.

Postsecondary Education Participants System

This system maintains data on school participation in student financial aid programs (such as eligibility, certification, address, and program participation); supports institutional reviewers and related activities; acts as the official source of information regarding schools and their associated school codes for all Department of Education systems; and supports the annual default rate calculation process for FFELP and FDLP.

Comments From the Department of Education and Our Responses

On July 31 and August 7, 1998, we met with Department of Education officials to obtain their comments on a draft of this report. In general, the Department commented that the survey results show NSLDS in a favorable light and that it is used universally for important operational purposes and without significant difficulty. The Department believes the report should convey such results.

Our meetings were supplemented by an August 14, 1998, letter from the Assistant Secretary for Postsecondary Education, which appears at the end of this appendix. Our responses to the comments raised in this letter are provided in the body of the report. Summaries of the comments Department officials provided during our meetings and our responses to these comments follow. 1. The Department said all schools use NSLDS, and that we were incorrect in reporting that almost half of the schools do not use NSLDS at all. In addition to accessing NSLDS through the on-line and batch processing functions, they said all schools (1) routinely use NSLDS for prescreening because the SAR or ISIR data they receive for this purpose are generated by NSLDS and (2) must submit SSCR data to NSLDS regardless of whether they do so themselves or use a servicer, such as the Clearinghouse. When such a servicer provides this service, it submits SSCR data to NSLDS in lieu of the schools doing so directly. Department officials said we failed to point out in our discussion of the 23 percent of schools that do not access the system at all that (1) this is due to limitations or constraints the schools face that are beyond the Department’s control and (2) schools use servicers or a third party to access NSLDS in order to meet SSCR processing requirements or receive ISIRs. The officials believe that the report should have stated that all schools use NSLDS (including those that access NSLDS indirectly) and schools do not need to use batch or on-line functions (except SSCR) if they are satisfied that the information provided by SARs or ISIRs in prescreening meets their needs.

GAO’s Response: We recognize that all schools use SAR and ISIR data generated by NSLDS and that in this way, all schools use NSLDS. Our review focused on schools’ use of NSLDS’ on-line and batch processing functions, and we have revised the report to further clarify this focus. As pointed out elsewhere in our report, the topic of NSLDS use was discussed extensively during our study design—with schools, Department officials, and others in the education community. To help school representatives in completing the survey, we explained on page 1 of the instrument that we defined “access” as on-line and batch processing functions (see app. I).

With regard to the number of schools we identified that were not accessing NSLDS at all, we have revised the report to clarify that these are schools that do not use the on-line or batch functions either themselves or through a servicer. As figure 8 shows, limitations and constraints at schools were among the more frequently cited reasons given by schools for not having this access. Figure 8 also shows that using servicers and the Clearinghouse for SSCR processing were frequently given as reasons for schools not using NSLDS’ on-line and batch processing functions. We did not independently obtain evidence on the extent of the Department’s ability to influence these factors and did not draw conclusions about the Department’s role. Our report, however, discusses how the surveyed schools are meeting the SSCR reporting requirement.

With regard to the Department’s comment that schools may not need on-line and batch processing functions when SAR or ISIR data meet their needs, we did not ask schools for details about their use of these or any other sources of financial aid data for determining student eligibility. It was our intent to find out from schools the extent of their use of NSLDS’ on-line or batch processing capabilities to perform a variety of tasks, such as borrower tracking or updating student information. We did not draw conclusions about the reported level of schools’ use of these capabilities. 2. The Department said that favorable responses we received to the survey instrument were not adequately reported. Department officials believe that there were numerous instances throughout our draft report where we deemphasized favorable results that reflect the majority of schools and emphasized corresponding unfavorable results. They cited as an example a statement in the draft that about one-fourth of schools had problems using NSLDS to correct or update SSCR information, and 20 to 30 percent of schools encountered occasional or frequent inaccuracies in certain data fields. They believe a more accurate portrayal of the survey results would be to state that 75 percent of the schools did not have any problems using NSLDS for SSCR processing and that 70 to 80 percent rarely or never encountered data inaccuracies.

GAO’s Response: Overall, we believe our draft report reasonably presented the results of our analysis. However, we made minor revisions, where appropriate, to further clarify our objectives and ensure the fairest possible presentation of our results. For example, in our discussion of schools’ views of the training and training materials they received, we added a statement that the Department mailed training materials to every school. However, we decided to continue to focus the reader’s attention on those responses that show the extent to which schools encountered shortcomings in using NSLDS’ on-line and batch processing functions because we believe it is important to note potential problem areas so the source of the problems may be explored and improvements to the program or operation can be identified. 3. Department officials said that the tone and balance of the draft were not true to the schools’ survey answers. For example, they said we cite a number of previous reports and studies illustrating data problems that they believe are outdated and are not relevant to the objectives of our review. If we feel strongly that these studies should be cited, the officials said we should also report more recent statistics on default matching, as well as numerous increases in the functional uses of NSLDS that have been developed. In addition, they believe the draft did not adequately address the work undertaken or under way to improve the quality of NSLDS data.

GAO’s Response: Our purpose in discussing prior GAO and OIG audit reports is to provide general background information, establish a historical context and significance for creating NSLDS, and discuss the basis for congressional interest in our review. As Department officials suggested, we have included more recent statistics on the Department’s use of NSLDS for conducting student loan default matches and expanded our discussion about its efforts to improve the quality of student financial aid data. 4. Department officials said that the draft report did not fully discuss the purposes and advantages of NSLDS, and that our treatment of original NSLDS goals needed clarification. They believe that it is important for us to distinguish between the original purpose of NSLDS and current efforts to expand its functionality. They suggested that the report should note that NSLDS was set up as a research database and that the Department is expanding its use to help improve the accuracy and availability of student aid data. More importantly, they said these improvements will ensure better accountability for student financial aid monies.

GAO’s Response: The report identifies the three main goals of NSLDS as they were presented and distributed to schools in the NSLDS users’ guide. Our study focused on schools’ use of NSLDS’ on-line and batch processing functions; we did not review the full spectrum of NSLDS’ purposes and functions. The Department’s explanation of these new functions is informative, and it appears that the Department is enhancing NSLDS to take advantage of many of its expanded capabilities. But many of these new functions were either not available to schools at the time we administered our survey instrument, were not related to on-line and batch processing functions, or were not designed to be used by schools. 5. Department officials were concerned about the development and timing of the survey instrument we administered. They said that it is not clear which school official completed the survey and that this is important because different offices in a school may use NSLDS for different purposes. Also, since the instrument was administered in late 1997, they believe some of the results may now be outdated and inaccurate. Finally, they believe that the report should note that the instrument was administered at a time when NSLDS was only 3 years old and that some portion of schools were not using it as a result of the time lags that occur in getting all schools to adapt to and welcome its use.

GAO’s Response: We made a concerted effort to direct the survey instrument to those school officials who were most knowledgeable about their school’s use of NSLDS’ on-line and batch processing functions. As discussed in appendix II, before we mailed the survey instrument, we sent a postcard to the student financial aid administrator of each school in our sample asking him or her to provide the name, title, and complete address of the school official who was best suited to complete the survey. About half of the schools provided this information; for those that did not, we mailed the survey to the director of student financial aid. In a letter accompanying the survey instrument, we further requested schools to ensure that those persons most knowledgeable about using NSLDS be involved in responding to the survey instrument.

The purpose of the survey was to record schools’ use of NSLDS at the time the survey was administered in November 1997, and this is noted in the report. We recognize that not all schools had access to NSLDS when they completed the survey instrument; as our results showed, many schools were still in the process of obtaining access. We would expect that, at this writing, more schools would be accessing NSLDS’ on-line and batch processing functions, but the report is intended to assess schools’ use of NSLDS’ on-line and batch processing capabilities and document the extent of this use at a point in time. We do not, nor did we intend to, draw any conclusions as to whether the level of school use is sufficient or indicative of the long-term utility of NSLDS as an administrative tool for federal student aid programs.

GAO Contacts and Staff Acknowledgments

GAO Contacts

Staff Acknowledgments

In addition to those named above, the following individuals made important contributions to this report: Paula Denman and Joan A. Denomme, Senior Evaluators; Carolyn S. Blocker and Edward H. Tuchman, Evaluators; and Deborah L. Edwards and Wayne M. Dow, Senior Social Science Analysts.

Related GAO Products

The Results Act: Observations on the Department of Education’s Fiscal Year 1999 Annual Performance Plan (GAO/HEHS-98-172R, June 8, 1998).

Department of Education: Information Needs Are at the Core of Management Challenges Facing the Department (GAO/T-HEHS-98-124, Mar. 24, 1998).

Student Financial Aid Information: Systems Architecture Needed to Improve Programs’ Efficiency (GAO/AIMD-97-122, July 29, 1997).

Department of Education: Multiple, Nonintegrated Systems Hamper Management of Student Financial Aid Programs (GAO/T-HEHS/AIMD-97-132, May 15, 1997).

Reporting of Student Loan Enrollment Status (GAO/HEHS-97-44R, Feb. 6, 1997).

High Risk Series: Student Financial Aid (GAO/HR-97-11, Feb. 1997).

Department of Education: Status of Actions to Improve the Management of Student Financial Aid (GAO/HEHS-96-143, July 12, 1996).

Student Financial Aid: Data Not Fully Utilized to Identify Inappropriately Awarded Loans and Grants (GAO/T-HEHS-95-199, July 12, 1995).

Student Financial Aid: Data Not Fully Utilized to Identify Inappropriately Awarded Loans and Grants (GAO/HEHS-95-89, July 11, 1995).

Federal Family Education Loan Information System: Weak Computer Controls Increase Risk of Unauthorized Access to Sensitive Data (GAO/AIMD-95-117, June 12, 1995).

High Risk Series: Student Financial Aid (GAO/HR-95-10, Feb. 1995).

Department of Education: Management Commitment Needed to Improve Information Resources Management (GAO/IMTEC-92-17, Apr. 20, 1992).

The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent.

U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.