> ![](media/image1.wmf){width="0.875in" height="0.875in"}
>
> **UNITED STATES DEPARTMENT OF EDUCATION**

OFFICE OF INSPECTOR GENERAL

Audit Services

September 25, 2008

> **Control Number**
>
> **ED-OIG/A09H0015**

Lawrence A. Warder

Acting Chief Operating Officer

Federal Student Aid

U.S. Department of Education

Union Center Plaza, Room 112G1

830 First Street, NE

Washington, D.C. 20002

Dear Mr. Warder:

This **Final Audit Report**, entitled *Federal Student Aid's Estimation
of* *Improper Payments in the Federal Family Education Loan Program*,
presents the results of our audit. The purpose of the audit was to
assess Federal Student Aid's (FSA) methodology for estimating improper
payments in the Federal Family Education Loan (FFEL) Program. Our review
covered the methodology used for reporting in the U.S. Department of
Education's (Department's) Performance and Accountability Reports (PARs)
for fiscal year (FY) 2006 and FY2007.

BACKGROUND

The Improper Payments Information Act of 2002 (IPIA) (Public Law
107-300) requires agencies to annually review and report on improper
payments in the programs and activities that they administer. The IPIA
specifies the annual three-step process that the agencies are to use to
meet the requirement. First, the agencies must review the programs and
activities and identify those that may be susceptible to significant
improper payments (risk assessment). Then, for each identified
risk-susceptible program and activity, the agencies must estimate the
annual amount of improper payments. Lastly, for programs and activities
with improper payment estimates exceeding a specified threshold, the
agencies are required to report on actions they are taking to reduce
improper payments. The IPIA provides definitions for the terms "payment"
and "improper payment."

The IPIA directs the Office of Management and Budget (OMB) to provide
agencies with guidance on implementing the requirements of the Act. On
August 10, 2006, OMB issued Appendix C to OMB Circular A‑123,
*Requirements for Effective Measurement and Remediation of Improper
Payments* (Appendix C).[^1] The OMB guidance requires agencies to
"institute a systemic method of reviewing all programs \[and
activities\] and identifying those which they believe to be susceptible
to significant erroneous payments." The OMB guidance defines
"significant erroneous payments" as annual improper payments in the
program "exceeding both 2.5 percent of program payments and
\$10 million." However, OMB may determine on a case-by-case basis that a
program or activity is subject to the annual PAR reporting requirement
even if it does not exceed the 2.5 percent and \$10 million thresholds
in OMB's definition of "significant erroneous payments."

The FFEL Program was identified as a program subject to the annual PAR
reporting requirement (i.e., estimation of the annual amount of improper
payments). For loan guaranty programs, such as the FFEL Program, the OMB
guidance provided the following clarification on the definition of an
improper payment.

> Under a loan guarantee program, an improper payment may include
> disbursements to intermediaries, third-parties for defaults,
> delinquencies, interest and other subsidies, or other payments that
> are based on incomplete, inaccurate, or fraudulent information. They
> may also include duplicate disbursements, disbursements in the
> incorrect amount, or any disbursements that are not in compliance with
> law, program regulations, or agency policy.

The OMB guidance on estimating the annual amount of improper payments
requires agencies to obtain a statistically valid estimate of the annual
amount of improper payments that is based on a random sample with a
specified confidence interval. The guidance states an agency may seek
OMB approval to deviate from the required steps or use an alternative
method. The OMB guidance also addresses the annual PAR reporting of the
estimate and corrective action plan for reducing improper payments.

The Department's Office of the Chief Financial Officer (OCFO) has
primary responsibility for issuance of the PAR. FSA is responsible for
providing information to the OCFO on programs it administers. FSA's
Financial Partners Services developed the methodologies for estimating
the amount of improper payments for the FFEL Program reported in the
PARs for FY2006 and FY2007.

AUDIT RESULTS

FSA used different methodologies for estimating the improper payment
rates for the FFEL Program that were reported for FY2006 and FY2007, and
plans to use another methodology for FY2008. FSA consulted with OMB
staff during the design and execution of the methodologies and generally
followed the IPIA definition and OMB guidance for loan guarantee
programs when it identified "payments" for the FFEL Program as the
payments made by the Department to lenders and guaranty agencies.
Nevertheless, there were several factors that affected the reliability
of the improper payment rates reported in the PARs. To improve the
reliability of the improper payment rates for the FFEL Program, we
recommend that FSA: (1) ensure that factors related to the use of
previously conducted audits and reviews are taken into account or their
effects mitigated; (2) ensure that all methods of payment are taken into
account when identifying the FFEL payments to be included in estimating
methodologies; (3) ensure that future methodologies use information on
improper payments for the fiscal year for the PAR or, if not available,
for the prior fiscal year; (4) when sampling is used, consider focusing
sample selection on higher‑risk entities and areas to increase the
likelihood of identifying improper payments; (5) finalize and revise the
estimated improper payment rate for FY2007 in future PARs;[^2] and (6)
disclose in the PARs when the rate is based on an interim calculation or
there are other limitations in the reported information on improper
payments.

We also found that the program outlays for the FFEL Program reported in
the Department's PARs for FY2006 and FY2007 and used to calculate the
estimated dollars of improper payments reflected different payment
universes. In addition, FSA used an estimated program outlay amount for
the current year when information was available on the actual outlays
for the year. We recommend that FSA develop and implement a revised
policy for identifying and reporting program outlays for the FFEL
Program in the PARs that provides consistent and comparable information
on outlays and dollars of improper payments reported and utilizes the
most currently available information on outlays.

In its comments on the draft report, FSA did not explicitly express
concurrence with our findings and recommendations, but did state it
would implement a methodology for estimating FFEL improper payments that
meets OMB requirements. FSA also stated that it would finalize and
report the estimated improper payment rate for FY2007, disclose in
future PARs when an interim calculation is used or other limitations in
reported information, and update its policy and procedures to ensure
consistency in information reported in future PARs. The comments are
summarized at the end of each finding and the full text of the comments
is included as an Attachment to this report.

**FINDING NO. 1 -- Several Factors Affected the Reliability of FSA's
Estimated Improper Payment Rates**

Several factors affected the reliability of the estimated improper
payment rates for the FFEL Program reported in the PARs for FY2006 and
FY2007. The reliability of the planned methodology for reporting in the
FY2008 PAR also could be impacted by several of the factors identified
in the earlier methodologies.

**Estimated Improper Payment Rate**

**Reported in the PAR for FY2006**

FSA provided an internal document that described the methodology used to
develop the estimated improper payment rate of 2.2 percent for FY2006.
The methodology extrapolated the monetary findings from prior reviews of
lenders, lender servicers, and guaranty agencies to the amount of
payments made to the lenders and guaranty agencies. To identify the
monetary findings, FSA's Financial Partners Services reviewed reports
issued in FY2005 for the following types of reviews: single audits,
program audits, OIG audits, and FSA program reviews of guaranty
agencies, lenders and lender servicers.[^3]

FSA compiled the identified monetary findings separately for
lenders/lender servicers and guaranty agencies by type of review, and
extrapolated the identified improper payments to all lenders and
guaranty agencies.[^4] FSA then added the calculated improper payments
for each type of review (totaled \$185,129,426 for all reviews) and
divided the total by the total payments to guaranty agencies and lenders
(\$8,625,964,796) in FY2005 to derive the estimated improper payment
rate of 2.2 percent for the FFEL Program.

Several factors affect the reliability of FSA's estimated improper
payment rate for FY2006. Sole reliance on audits and FSA program reviews
of guaranty agencies, lenders, and lender servicers excluded improper
payments identified by other sources. Also, amounts extracted from
reports on previously conducted audits and program reviews may not
reflect actual improper payments for a federal fiscal year. Other
factors were that the amounts used from the reports may be
unrepresentative of all lenders and guaranty agencies, duplicate amounts
included in another report, and contain amounts not applicable to the
PAR reporting period. Also, the identified amount of improper payments
was incomplete (i.e., exclusion of contested monetary findings and use
of final determinations of liabilities). We also found that FSA had
omitted certain payments to guaranty agencies from the total payments
used to calculate the estimated improper payment rate.

***Sole Reliance on Audits and FSA Program Reviews of***

***Guaranty Agencies, Lenders, and Lender Servicers***

***Excluded Improper Payments Identified by Other Sources***

The amount of improper payments used in FSA's methodology may be
incomplete because all sources for identifying improper payments were
not utilized in the methodology. FSA did not include monetary findings
from audits and reviews of institutions of higher education, guaranty
agency reviews of lenders, or OIG investigations. Also, all improper
payments self-identified by lenders or lender servicers were not
included in the methodology.

-   [Institutions of higher education reviews]{.underline}. Institutions
    of higher education that administer the FFEL Program are subject to
    audits conducted by independent auditors and the OIG, as well as
    program reviews conducted by FSA and guaranty agencies. These audits
    and program reviews may have findings related to students'
    eligibility for FFEL disbursements, amount of the disbursement, or
    amounts to be returned to the lender. For example, students enrolled
    in ineligible programs or at ineligible institutions are not
    eligible for loans under the FFEL Program. Student eligibility is
    significant because payments made on behalf of loans disbursed to
    ineligible students, such as interest and special allowance
    subsidies paid to lenders and claims paid to guaranty agencies, are
    improper payments.

-   [Guaranty agency reviews]{.underline}. Guaranty agencies are
    required by federal regulations to conduct biennial program reviews
    of certain lenders participating in the FFEL Program. Federal
    regulations define the criteria for selecting the lenders for
    review, but guaranty agencies may consider other factors, including
    complaints from schools, students or borrowers, and evidence of
    potential fraud or abuse in a lender's FFEL Program participation.
    Guaranty agencies reported that common program review findings at
    lenders included errors on lender invoices in origination fee
    calculations, interest, and special allowance reporting. These types
    of errors could result in improper payments.

-   [OIG investigations]{.underline}. Documentation provided by OIG
    investigations showed that improper payments of almost \$4 million
    were identified in FY2006 for all Student Financial Assistance
    programs, including but not limited to the FFEL Program. In one
    example, an OIG investigation identified \$650,000 that the
    Department paid a guarantor for fraudulently consolidated federal
    student loans.

-   [Lender self-reports]{.underline}. FSA included \$370,740 in
    self‑reported lender liabilities that was recorded in the
    Department's Postsecondary Education Participants System (PEPS), but
    did not include adjustments made by lenders on their invoices.
    Adjustments of amounts included in prior invoices may occur when
    updated borrower information was subsequently received that impacts
    the calculation of interest and special allowance on the loans.
    Adjustments may also occur when the lender or lender servicer has
    identified errors in their billings. The lender invoice does not
    identify the reason for an adjustment.

By not including these additional sources when compiling monetary
findings, FSA understated the improper payments used to develop its
estimated improper payment rate.

***Audit and Program Review Reports May***

***Not Reflect the Actual Improper Payments***

***For a Federal Fiscal Year***

The reports from previously conducted audits and program reviews may not
reflect the amount of actual improper payments. Audits and program
reviews generally cover a small sample or focus on selected program
requirements. Thus, there is a risk that the reviews may not detect
conditions that may have resulted in improper payments. For example, FSA
receives about 380 program audit reports annually. However, FSA
identified only 11 program audit reports issued during FY2005 that
contained monetary amounts for inclusion in the improper payment rate.

Even when the audits and reviews have detected reportable conditions,
the reports may not contain information on the actual amounts of
improper payment applicable to the period reported in the PAR. The
reports may not include a quantification that reflects the improper
payment amount, may be limited to the reviewed transactions, or be for a
period other than the federal fiscal year.

-   [Reported monetary impact may not reflect the improper payment
    amount]{.underline}. This situation was demonstrated by FSA's use of
    monetary amounts from program audits. Our review of 4 of the 11
    program audit reports confirmed that, for one audit report, an
    improper payment had occurred and that FSA had used an accurate
    amount in its development of the estimated improper payment rate.
    The other three audit reports included findings discussing
    conditions that may have resulted in improper payments. However, the
    amounts from the three reports that FSA used to develop its
    estimated improper payment rate were not a quantification of an
    improper payment made to a lender or guaranty agency. Instead, the
    amounts represented the principal balance of a loan or a loan
    disbursement.

-   [Quantification limited to reviewed transactions]{.underline}. Even
    when an audit or program review report includes a questioned cost
    (i.e., an improper payment), the amount is often limited to the
    specific erroneous transactions identified in the reviewed sample
    rather than an amount derived by projecting sample results across
    all transactions. Although not projecting sample results may have
    been valid (e.g., the sample was non‑statistical), using only sample
    results understates the improper payment amount.

-   [Period covered by quantification may be other than a federal fiscal
    year]{.underline}. The 12-month federal fiscal year is used for
    improper payments reporting. In contrast, the period covered by
    audits and program reviews may be an entity's fiscal year, multiple
    fiscal years, or other period of time. Thus, monetary findings in
    audit and program review reports may be linked to a period of time
    other than the federal fiscal year or a period that is longer or
    shorter than the federal fiscal year. Using the reported monetary
    findings could consequently result in either overstating or
    understating the improper payments.

FSA also recognized in its methodology documentation that the improper
payments were incomplete because monetary findings were not always
quantified by FSA in the final program review report. The OIG audit
report titled *Review of Financial Partners' Monitoring and Oversight of
Guaranty Agencies, Lenders, and Servicers* (ACN A04E0009), issued
September 2006, discussed this condition and recommended that FSA
develop a consistent policy for identifying, quantifying, and reporting
liabilities identified in program reviews.

***Findings in OIG Audits and FSA Program Reviews at***

***Individual Entities May Not Be Representative of***

***Improper Payments at Other Entities***

Findings from OIG audits and FSA program reviews may not be
representative of conditions effecting payments to all lenders and
guaranty agencies. Because these audits and program reviews are
generally selected on a judgmental basis that may consider risk,
extrapolating their results across loan balances for all lenders and
guaranty agencies may overstate the amount of improper payments. For
example, FSA included an improper lender payment of \$688,767 from an
OIG audit report[^5] and extrapolated the amount across loan balances
for all lenders. Thus, the one OIG audit, which covered less
than 0.4 percent of loan balances, accounted for \$181 million[^6] of
the \$185 million (97.7 percent) improper payments used to calculate the
estimated improper payment rate for the FFEL Program.

***Audits and Program Reviews***

***Could Identify the Same Improper Payments***

Audits and program reviews conducted by different organizations that
review the same entity could identify the same improper payments. For
example, program reviews conducted by FSA and the single audits and
program audits conducted by independent public accountants may cover the
same loan balances. FSA's documentation for lenders showed that the FSA
program reviews provided oversight for 26 percent of loan balances and
reported monetary findings of \$621,300. Additionally, FSA's
documentation showed that the single audits and program audits provided
95 percent coverage of the same loan balances and reported monetary
findings of \$864,400.

The likelihood of two reviewers identifying the same improper payment is
minimized by the limited nature of the audits and program reviews.
Auditors review an entity's internal controls and may limit their
testing to a small sample of loans. Program reviewers may also only
review documentation for a limited number of loans and may limit the
scope of their reviews to certain aspects of the FFEL Program. However,
if the auditors and program reviewers identify the same condition at a
lender or guaranty agency, the projection of the improper payments to
the total payments made to the entity could result in duplication of
improper payment amounts.

***Applicability of Identified Improper***

***Payments to the Reporting Period***

In its Appendix C guidance, OMB recommended that the annual estimate of
improper payments reported in the PAR coincide with the fiscal year
being reported, but allowed, in limited cases, that agencies report
using the previous year's data. Improper payments identified by audits
and program reviews may be for periods prior to both the fiscal year
being reported and the previous fiscal year. FSA used audit and program
review reports issued during FY2005 to identify the improper payments
used to calculate the estimated improper payment rate reported in the
PAR for FY2006. For example, the OIG audit report (ED‑OIG/A05E0017)
identified improper payments that occurred in FY2003 and FY2004. The
identified improper payments become less applicable to the PAR reporting
period as the time lengthens between the occurrence of the improper
payments and the fiscal year used for reporting in the PAR.

***Exclusion of Contested Findings and Use of***

***Final Determinations of Liabilities Understate***

***Improper Payments Identified in Audits and Reviews***

The amount of improper payments used in FSA's methodology is understated
due to FSA's exclusion of contested finding amounts (e.g., lender or
guaranty agency disagreed with the finding) and use of the liability
amounts from final determinations on reported findings.

-   [Contested findings]{.underline}. Improper payments identified in
    contested findings were not included in the improper payment total
    used to develop the estimated improper payment rate. The OIG audit
    report (ED‑OIG/A05E0017) with the \$688,767 improper payments that
    was mentioned in a prior section also included a finding that the
    lender may have been improperly paid \$18.4 million in special
    allowance payments for loans that were not eligible to be billed
    under the 9.5 percent floor. FSA only included the \$688,767 of
    improper payments, which related to a finding that the lender agreed
    with in its comments to the report.

-   [Final determinations of liability]{.underline}. FSA's document
    describing the FY2006 methodology showed that the amount of improper
    payments from FSA program review findings was obtained from the
    PEPS. FSA disclosed in its document that the PEPS data entry
    procedures, in effect at the time, delayed entry of the dollar
    amount of a program review finding into the PEPS until the review
    was closed on the system and monetary adjustments had taken place.
    FSA had delayed entering the amounts because the amounts in the
    final determination of a finding or after appeals may be different
    from the amount identified in the initial program review report.

Even when a contested finding is upheld or the final adjustment takes
place, the amount of the improper payments can exceed the amount of the
recorded liability. An OIG audit of a lender that was recently resolved
provides an example of the possible magnitude of the difference. The
audit

report disclosed \$278 million of improper payments.[^7] While the
finding was upheld, the Department determined a final liability of zero.
Even when the Department decides not to establish or recover a
liability, the improper payments should be included in determining the
estimated improper payment rate.

***Certain Payments to Guaranty Agencies***

***Were Not Included***

FSA omitted certain payments to guaranty agencies from the total
payments used to calculate the estimated improper payment rate. When
defining payments made to guaranty agencies, FSA only included
transactions that passed through the Department's payment system. For
example, for account maintenance fees, FSA included the \$160.7 million
in payments made through the payment system, but did not include the
additional \$103 million that guaranty agencies received by transferring
funds from their Federal fund to their Operating fund. FSA also did not
include the payments for default aversion fees in total payments.
Guaranty agencies calculate these fees and receive payment by
transferring funds from their Federal fund to their Operating fund.
Guaranty agencies reported transferring about \$86 million for default
aversion fees for FY2005.

**Estimated Improper Payment Rate**

**Reported in the PAR for FY2007**

FSA used a statistical sampling methodology to develop the estimated
improper payment rate of 0.032 percent for FY2007. FSA reviewed
statistically selected samples for two types of payments made during
FY2005: payments to lenders and payments to guaranty agencies. For each
payment type, FSA used a two-stage sampling process. In the first stage,
a sample of payments was selected by Financial Partners Services. In the
second stage, FSA selected a sub‑sample of loan‑level transactions from
the invoice supporting the payment selected in the first stage. FSA's
Financial Partner Eligibility & Oversight, a component of FSA's Program
Compliance unit, selected the second stage sub-samples and conducted the
associated reviews at the guaranty agencies, lenders, and lender
servicers.

Pilot reviews began in late November 2006. Lessons learned from these
reviews were incorporated into a 59-page review guide, which was
finalized in March 2007. The guide included provisions for consistency
of reviews, such as detailed electronic templates, and oversight over
reviews. The team leaders, who conducted the pilot reviews, conducted
training for the rest of the reviewers during February and March 2007.

-   [Payments from ED to Lenders]{.underline}: Lenders submit quarterly
    invoices to the Department via the Lender Reporting System (LaRS).
    Lenders invoiced the Department for interest and special allowance
    payments, which could be offset by loan origination and lender loan
    fees that lenders owed the Department. FSA identified
    11,264 payments, totaling \$3.2 billion, made in FY2005 to the
    approximately 3,200 lenders that participate in the FFEL Program
    (lender payment universe).

> In the first stage, FSA randomly selected 47 primary sample units,
> where a primary sample unit was a payment for a specific fiscal
> quarter tied to a specific invoice from a lender or a lender servicer.
> Depending on the number of lender servicers used by a lender, that
> lender could be associated with multiple invoices and multiple
> payments for any one fiscal quarter. The 47 payments selected in the
> first stage random sample were made to 47 different lenders.
>
> For the second stage sub-sampling, FSA obtained the lender or lender
> servicer data used to support the invoice that had been the basis for
> the Department's payment. The data was requested from the
> lender/lender servicer at the loan level, which linked each component
> of the invoice with a unique loan (i.e., a specific loan for a
> specific borrower.) In the second stage, FSA statistically selected
> unique loans and reviewed all transactions related to that loan which
> affected the selected invoice. For example, in an invoice, one loan
> might be associated with interest, special allowance, loan
> origination, and lender loan fee transactions, whereas another loan
> might only be associated with one of those transactions. The number of
> second stage sub-samples selected depended on the number of unique
> loans supporting the invoice.
>
> For each sampled loan, the review guide required the FSA reviewer to
> obtain electronic and hardcopy documentation from the lender that the
> reviewer used to examine the accuracy of data and calculations
> supporting the lender's invoice. The activities that FSA reviewers
> were instructed to complete and document for each sampled loan
> included:

-   Verify that totals on the lender's electronic back-up files agree
    with totals reported by the lender and paid through the LaRS
    process.

-   Verify the existence of a valid signed promissory note. Compare the
    signature date with the date of the first disbursement to verify
    interest rates and special allowance categories that are based on
    the date.

-   Verify that origination and lender fees due to the Department for
    loans originated during the quarter were paid timely and accurately.

-   Verify eligibility for interest and special allowance benefits
    during the quarter by examining lender documentation pertaining to
    loan amount, disbursement date, current status, payment amount,
    deferment, and forbearance.

-   Verify application of payments: Recalculate interest due on balance
    and verify that payment was applied correctly to interest, principal
    and late fees.

-   After recalculating the ending balance, recalculate interest accrued
    for the quarter on subsidized loans and payable by the Department,
    and compare to amount invoiced by lender.

-   Using transaction history, recalculate the average daily balance for
    the quarter and compare to amounts reported by lender to support
    special allowance payments by Department.

> We concluded from our evaluation of the documentation for five of the
> FSA lender reviews that the reviewers had adhered to the review
> instructions specified in the review guide. However, as we noted later
> in the finding, the review process did not consider subsequent
> transactions, did not verify all components of the special allowance
> category code, and did not confirm that lenders had satisfied due
> diligence requirements.
>
> After a cost-benefit discussion with OMB, FSA decided to cease the
> reviews before completion of all 47 lender reviews. FSA provided
> documentation showing that 42 of the 47 reviews were completed, of
> which 38 reviews were conducted on-site at lenders or lender servicers
> and 4 were conducted as desk reviews.[^8] FSA documented that the
> invoices for the 42 completed reviews totaled about \$12 million, and
> that the absolute value of the sub‑samples reviewed for those invoices
> totaled about \$29,243.

-   [Payments from ED to Guaranty Agencies]{.underline}: Guaranty
    agencies submit monthly invoices to the Department using the
    Guaranty Agency Financial Report (GAFR). Guaranty agencies request
    payments for various types of claims, such as claims for default,
    bankruptcy, death, disability, and closed schools. The amount
    invoiced would be offset by amounts the guaranty agency owed the
    Department for collections and refunds. FSA identified 488 payments,
    totaling \$3.5 billion, made during FY2005 to the 35 guaranty
    agencies that participate in the FFEL Program (guaranty agency
    payment universe).

> In the first stage, FSA randomly selected 44 primary sample units,
> where a primary sample unit was a payment for a specific month to a
> specific guaranty agency. The 44 payments selected in the first stage
> random sample were made to 27 of the 35 guaranty agencies.
>
> As had been done for the lender payments, FSA obtained the loan-level
> data used to support the invoice from the guaranty agency and used the
> data to randomly select the second stage sub‑sample. Then, FSA
> examined all transactions for the specific month that were related to
> the selected loan.
>
> Similarly, for each sampled loan, the review guide required the FSA
> reviewer to obtain electronic and hardcopy documentation from the
> guaranty agency that the reviewer used to examine the accuracy of data
> and calculations supporting the guaranty agency's invoice. The
> activities that FSA reviewers were instructed to complete and document
> for each sampled loan included:

-   Verify that totals on the guaranty agency's electronic back-up files
    agree with totals reported by the guaranty agency and paid through
    the GAFR process.

-   Analyze supporting documentation to determine the type of
    transaction being reviewed and determine if it was reported under
    the correct line item(s).

-   Verify the existence of a valid signed promissory note and compare
    it with loan and transaction records.

-   For claims, verify the accuracy of the invoiced amount by examining
    the supporting documentation pertaining to type of loan,
    disbursement/guarantee dates and amounts, and reinsurance rate.
    Verify that the claim payment date and amount paid to lender were
    correctly reported to the Department.

-   For collections, verify that payments received were reported timely
    and accurately by examining the supporting documentation pertaining
    to type of loan, first disbursement date, disbursed amount,
    reinsurance rate, receipt dates and amounts, payment attribution to
    all the borrower's loans, and amount attributed to the specific loan
    selected for the sub-sample.

> We concluded from our evaluation of the documentation for two of the
> FSA guaranty agency reviews that the reviewers had adhered to the
> review instructions specified in the review guide. However, as we
> noted later in the finding, the review process did not include a
> confirmation of required due diligence activities for default claims.
>
> As discussed above, FSA curtailed the planned reviews. FSA informed us
> that it completed 42 of the 44 guaranty agency reviews, and conducted
> all 42 reviews on-site at the guaranty agencies.[^9] FSA documented
> that the invoices for the 42 completed reviews totaled about
> \$271 million, and that the absolute value of the sub‑samples reviewed
> for those invoices totaled about \$846,835.

FSA reviewers identified a gross total (i.e., absolute value of over-
and under-payments) of \$42.37 for the lender reviews (improper payments
identified in 2 of 42 reviews) and \$163.26 for the guaranty agency
reviews (improper payments identified in 5 of 42 reviews.) For each
invoice reviewed, FSA divided the amount of any improper payments by the
total amount of the loan-level transactions reviewed for that invoice to
calculate an error rate for the reviewed loan-level transactions.
FSA then multiplied the error rate for the reviewed loan-level
transactions by the total amount of the invoice to extrapolate an amount
of improper payments for each invoice. FSA calculated the amount of
improper payments for the 84 lender and guaranty invoices at \$74,835.
While conducting the guaranty agency reviews, FSA identified an
additional \$18,157 of improper payments that were outside the specific
loan-level transactions reviewed. These two amounts totaled \$92,992.
However, due to a worksheet formula error when summarizing the amounts,
FSA calculated \$89,998 as the total improper payments. To calculate the
estimated improper payment rate, FSA divided the \$89,988 in total
improper payments by the total payments for the 84 invoices reviewed
(\$283,175,828). The resulting estimated improper payment rate of
0**.**032 percent was reported in the FY2007 PAR.[^10]

Several factors affected the reliability of FSA's improper payment rate
for FY2007. The methodology did not take into account determinations of
student eligibility and other loan-making activities. Also, an
estimating methodology that relies on the results of a review of sampled
payment invoices is impacted by the method of sample selection,
completeness of the payment universe, use of FY2005 payment data, and
design of the review process. Finally, the estimated improper payment
rate reported in the PAR was based on an interim calculation that did
not include all improper payments identified by the reviews.

***Methodology Did Not Take Into Account Determinations***

***of Student Eligibility and Other Loan-Making Activities***

Educational institutions often perform loan-making activities that
impact whether payments to lenders and guaranty agencies are proper. For
example, educational institutions determine the borrower's eligibility
for loans and loan amounts. They may also perform other loan-making
activities, such as explaining borrower's rights and responsibilities
under the loan. When certain loan-making activities are not performed
correctly, the federal guarantee on the loan may be impacted. In
addition, educational institutions and lenders rely on statements made
by borrowers

on their loan applications. Inaccurate application information could
result in an improper determination of student eligibility for the loan
and, thus, impact the federal guarantee on the loan. Payments made to
lenders for interest and special allowances and payments to guaranty
agencies for claims on loans that do not have a valid federal guarantee
are improper payments. In addition, activities performed by educational
institutions that take place over the loan's life‑cycle may impact
federal interest payments to lenders.

While student eligibility and other loan-making activities could effect
the payments to lenders and guaranty agencies that were reviewed under
FSA's methodology, it would not have been feasible for FSA to evaluate
these activities as part of its review. Loan-making activities are
completed by numerous educational institutions located throughout the
nation. Also, the student's preparation of the application and the
loan‑making and other activities performed by educational institutions
may have occurred several years earlier.

Even when audits and program reviews of educational institutions have
identified loans that were made to ineligible students, the lender and
guaranty agency may continue to receive payments on the ineligible
loans. When FSA identifies ineligible loans through its program reviews
at educational institutions, FSA requires the educational institution to
assume the liability for the interest subsidy cost and risk of loss from
default by the borrower for the ineligible amount of the loan.[^11] FSA
uses the estimated actual loss formula to determine the institution's
liability amount,[^12] and establishes a receivable to recover from the
school the estimated costs resulting from a school's disbursement of the
ineligible loan. According to FSA's procedures, the school is not given
the option of purchasing the ineligible loan from the lender.[^13] Since
FSA establishes the receivable directly with the school, the lender and
guaranty agency may have no knowledge of the loan's ineligibility. Even
though payments to the lender for interest and special allowance and the
guaranty agency for claims on the ineligible loans may be allowable
payments to those entities, they should be classified as improper
payments for the purpose of estimating the improper payment rate.

***Sample Size and Use of***

***Simple Random Sampling***

FSA used statistical software (EZ-Quant) to determine the number of
payments to be selected for review in the lender and guaranty agency
universes and the related loan-level transactions (i.e., sample sizes).
FSA's goal was to determine the sample sizes at a 95% confidence
level with a presumed 3 percent error rate and precision of
+/‑5 percent. However, FSA did not enter the correct amounts into the
EZ-Quant's input screens for the "desired maximum precision amount" when
it determined the sample sizes. FSA entered an amount equal to 5 percent
of the total dollar amount of the payment universe instead of 5 percent
of the presumed dollar amount of improper payments (i.e., 3 percent of
the dollar amount of the payment universe). For example, to determine

the sample size from the guaranty agency universe, FSA used
\$175,360,350 (5 percent of the total payments of \$3,507,207,010) for
the "desired maximum precision amount" instead of \$5,260,811 (5 percent
of the presumed improper payments of \$105,216,210).[^14] Due to the
procedural error, FSA used sample sizes that were significantly smaller
than those recommended by the EZ-Quant software. In the above example,
EZ-Quant recommended a sample size of 484 from the universe of 488
guaranty agency payments, basically a 100 percent review, to obtain
FSA's goal of a sample with a 95% confidence level with a presumed 3
percent error rate and precision of +/‑5 percent. FSA had determined a
sample size of 44. The impact of using smaller sample sizes will not be
known until a statistical projection of the actual sample results is
performed. The following table shows the amount of the payments made to
lenders and guaranty agencies in FY2005 that were selected for detailed
review.

+--------------------+----------+---------------+---------------------+
| **Table 1. Amounts |          |               |                     |
| In Payment         |          |               |                     |
| Universes and      |          |               |                     |
| Related Samples**  |          |               |                     |
+--------------------+----------+---------------+---------------------+
| **Payment Type**   | *        | **Payments    | **Payments for      |
|                    | *Payment | for Selected  | Selected Loans**    |
|                    | Un       | Invoices**    |                     |
|                    | iverse** |               | (Second Stage       |
|                    |          | (First Stage  | Sub-Sample)         |
|                    |          | Sample)       |                     |
+--------------------+----------+---------------+---------------------+
| Lender Payments    | \$3.2    | \$12 million  | \$29,243            |
|                    | billion  |               |                     |
+--------------------+----------+---------------+---------------------+
| Guaranty Agency    | \$3.5    | \$271 million | \$846,835           |
| Payments           | billion  |               |                     |
+--------------------+----------+---------------+---------------------+

Use of stratified sampling, rather than simple random sampling, could
improve the efficiency of sampling. FSA used a simple random selection
process to identify the invoices and loans in its samples. In its
Appendix C guidance, issued August 10, 2006, OMB suggested that "most
agencies will need to consult with a statistician to design an
appropriate sample that may involve . . . multiple stages of selection
or stratification (rather than a simple random sample)." Although FSA
conferred with a statistician and used a two stage sample selection
process, the samples in each of the two stages were selected by a simple
random process, which gave every item an equal chance of selection.
FSA's discussions with the statistician did not include consideration of
alternatives to a simple random sample selection process.
Stratification, where a universe is divided into strata, is an
alternative for sample selection that can be used to give special
emphasis to certain groups within the payment universes, such as
invoices of high dollar values or those with a great error potential.

***Completeness of the Payment Universe***

The lender and guaranty agency payment universes did not include all
types of payments. The lender payment universe was based entirely on
information in FSA's payment system which did not include "accounts
receivable" invoices. These are invoices where, after netting amounts
due from the Department against amounts due from lenders, the lenders
owed the Department. Because FSA did not include the "accounts
receivable" invoices in the lender payment universe, the amount payable
by the Department on those invoices was excluded from the improper
payment review process.

The guaranty agency payment universe, which was also based on
information in FSA's payment system, did not include amounts that
guaranty agencies received by transferring funds from their Federal fund
to their Operating fund. For example, the amounts transferred for the
default aversion fees mentioned in an earlier section of this report.

***Use of FY2005 Payment Data***

The lender and guaranty agency payment universes did not use the most
current payment information available in its methodology. FSA's use of
the FY2005 payment information exceeded the one year allowance outlined
in OMB's Appendix C guidance, issued August 10, 2006, and resulted in
the application of an estimated improper payment rate for FY2005 to
program outlays for FY2007 and later. We also noted that FSA's use of
FY2005 payment information provided no coverage of excess interest fees
during the improper payment reviews. However, the exclusions of the
excess interest fee likely had no impact on the estimated improper
payment rate since the fee became effective on April 1, 2006 and totaled
about \$1.5 million in FY2006.

***Review Process Did Not***

***Consider Subsequent Transactions***

FSA's review process did not consider subsequent transactions when
reviewing loan‑level transactions for lender payments. A transaction
would be considered accurate if it was accurate for the quarter
reviewed, even if it had been adjusted in a subsequent quarterly
invoice. As mentioned in a previous section of this report, subsequent
adjusting transactions are required when lenders receive updated
borrower information or identify errors in their procedures. When
examining the transactions for a loan for the sampled quarterly invoice,
FSA's review process did not include reviewing later invoices to
determine whether any subsequent adjustments had been made, and if so,
to determine the reason for the adjustment. Thus, FSA's process for
reviewing loan-level transactions may not have identified improper
payments.

***Review Process Did Not Verify All Components***

***of the Special Allowance Category Code***

FSA's review process for lender payments did not include verification of
all components of the special allowance category code, which is used to
calculate the special allowance subsidy payable by the Department. The
selection of the appropriate special allowance category code is
determined by four loan characteristics: loan type, date, borrower
status, and funding source (i.e., from either taxable or tax-exempt
sources). The review process included verification of the first three
loan characteristics, but did not include verification of the funding
source. FSA management stated that it was not practical to verify the
funding source, because funding documentation is maintained at the
lender whereas most of the reviews were conducted at the lender
servicer. Additionally, FSA management stated that the funding sources
were not confirmed because FSA reviewers had knowledge of the type of
funding source generally used by a lender and that separate on‑going
reviews were examining the funding source used to support special
allowance invoicing. Thus, FSA's review of sampled loans would not have
identified improper payments that could have occurred from lenders
placing loans in an inappropriate special allowance category for the
funding source. The OIG audit report (ED‑OIG/A05E0017), which was
included in the methodology used

for the FY2006 PAR, identified a substantial amount of improper payments
that resulted from a lender's use of the inappropriate special allowance
category for the funding source.

***Review Process Did Not Confirm That Lenders***

***Satisfied Due Diligence Requirements***

FSA's review process for lender payments and guaranty agency payments on
default claims did not include a confirmation that lenders performed
required collection activities (referred to as due diligence). The
federal guarantee on the loan may be lost if a lender fails to complete
the required activities within certain timeframes. If the guarantee on a
loan is lost, the lender also loses the right to collect interest and
special allowance until the lender completes the activities required to
reinstate the guarantee. Guaranty agencies may only submit default
claims for loans that have a valid federal guarantee. An FPE&O team
leader informed us that FSA had made a decision not to include due
diligence in the review process.

***Improper Payment Rate Reported in PAR***

***Was Based on Interim Calculation***

The estimated improper payment rate of 0.032 percent reported in the PAR
for FY2007 was based on an interim calculation that did not include all
improper payments identified by the reviews. FSA provided documentation
confirming that, as of September 30, 2007, reviews had been completed of
the 84 lender and guaranty agency invoices. However, FSA did not include
the results of some reviews in the calculation of the rate because, at
the time the calculation was prepared, FSA had not yet determined the
actual amount of the improper payments for those reviews. According to
the FSA staff responsible for conducting the calculation, the related
program review report had not yet been provided to him or FSA was
awaiting information from the lender or guaranty agency to determine the
improper payment amount.

In April 2008, FSA updated its calculation of the estimated improper
payment rate in order to respond to an information request from the OIG
auditors. The revised calculation identified a gross total of \$203
improper payments for the lender reviews (improper payments identified
in 3 of 42 reviews) and \$1,140 improper payments for the guaranty
agency reviews (improper payments identified in 11 of 42 reviews.) After
FSA extrapolated the identified errors to the related invoices, the
improper payment amount for the 84 invoices totaled \$569,850. FSA
increased the amount for the additional improper payments identified
outside the reviewed loan-level transactions to \$51,798. The two
amounts totaled \$621,648. Due to the same worksheet formula error that
we noted in the previous calculation, FSA calculated \$618,644 as the
total of estimated improper payments. To calculate the updated improper
payment rate for the 84 invoice reviews, FSA divided the \$618,644 in
total improper payments by the total payments for the 84 invoices
reviewed (\$283,175,828). The calculation resulted in an estimated
improper payment rate of 0.218 percent.[^15] The FSA staff stated that
the calculation may be further revised after FSA reviews responses and
additional documentation provided by lenders and guaranty agencies and
makes final determinations of the improper payments for the individual
reviews.

The Department did not disclose in the PAR that the reported rate was
based on an interim calculation that did not include all improper
payments identified in its reviews. Both the reported rate and the
recalculated rate are well below the benchmark (2.5 percent) for
significant erroneous payments. Nevertheless, the PAR should contain
complete and accurate information and disclose limitations in reported
information on improper payments. The PAR should also disclose the
statistical precision of the estimated improper payment rate.

**Improper Payments Methodology**

**Planned for FY2008**

FSA plans to use another methodology to estimate improper payments for
the FFEL Program for reporting in the FY2008 PAR. On February 27, 2008,
FSA signed an interagency agreement with the U.S. Department of Energy's
Oak Ridge National Laboratory (ORNL). The agreement's Scope of Work
states that "ORNL will evaluate available data sources and develop a
model that utilizes that data to evaluate the relative risk of erroneous
payments in the FFEL Program. The model shall also evaluate the relative
risk of \[the Department\] being billed inappropriately by lenders or
guaranty agencies. If enough relevant data can be extracted from
identified data sources, an estimated percentage of improper payments,
by payment types and/or type of recipient (GA or lender) will be
calculated." The project description stated that data on audits was
available in PEPS and from the Department's Audit Accountability
Resolution Tracking System. Other data for lenders and guaranty agencies
was contained in the Department's National Student Loan Data System and
Financial Management System (FMS). If ORNL develops a model that uses
the results of previously conducted audits and reviews, the reliability
of ORNL's risk assessment, and any estimated percentage of improper
payments, may be similarly impacted by the factors discussed in this
report for FSA's FY2006 methodology.

## Recommendations  {#recommendations .RptSubcaption}

We recommend that the Acting Chief Operating Officer for FSA---

1.1 Ensure that the design and implementation of any future improper
payment estimating methodologies that utilize results of previously
conducted audits and reviews take into account or mitigate the effects
of the following factors, when feasible: (1) improper payments
identified in reviews other than single audits, program audits, OIG
audits, and FSA program reviews of lenders and guaranty agencies
(including improper payments self-reported by lenders); (2) lack of
quantification of monetary impact of findings in the reports; (3) review
periods that do not correlate to the federal fiscal year; (4) not being
representative of all lenders and guaranty agencies; (5) duplicate
identification of improper payments; (6) the time between the occurrence
of the improper payments and the fiscal year used for reporting in the
PAR; and (7) improper payments identified in contested monetary findings
and findings without a final determination of liabilities.

1.2 Ensure that future methodologies appropriately include all payments,
such as payments made by transferring funds between accounts (i.e.,
guaranty agencies' transfer of funds from their Federal fund to their
Operating fund) and payments netted by larger amounts due to the
Department (i.e., accounts receivable invoices).

1.3 Ensure that future methodologies use information on improper
payments for the fiscal year used for reporting in the PAR or, if not
available, for the prior fiscal year.

1.4 If sampling is used in future methodologies, consider focusing
sample selection on higher‑risk entities or types of transactions, such
as subsequent adjustments and special allowance subsidies, to increase
the likelihood of identifying improper payments.

1.5 Finalize its calculation of the estimated improper payment rate for
FY2007 (including the measure of the statistical precision of the
estimated rate) and use the finalized rate and its statistical precision
when reporting the estimated improper payments for FY2007 in the PAR for
FY2008. The FY2008 PAR should also include an explanation of the change
from the rate reported in the FY2007 PAR.

1.6 In future PARs, disclose when the information presented on improper
payments is based on interim calculations and the limitations of the
reported information.

**FSA Comments**

FSA stated it will design and implement, in consultation with OMB, a
methodology for estimating FFEL improper payments that meets the
requirements of Circular A-123, Appendix C, and that the Department's
Office of the Chief Financial Officer and FSA have been meeting with OMB
regularly to reach agreement on a methodology for fiscal years 2009 and
beyond. FSA's comments did not address the specific actions contained in
Recommendations 1.1, 1.2, 1.3, and 1.4. In its comments on
Recommendations 1.5 and 1.6, FSA stated it will calculate and use the
final rate from the reviews to report the FY2007 improper payments in
the FY2008 PAR and will note the change from the rate used in the FY2007
PAR. FSA stated it will disclose in future PARs when information
presented is based on interim calculations and any limitations of
reported information.

**OIG Response**

In April 2008, the OIG provided FSA with a preliminary draft of this
finding for use in the design and implementation of the methodology for
FY2008. The execution of the methodology should be substantially
complete at this time. As FSA noted in its comments, decisions are
currently being made, in conjunction with OMB, on the design of the
methodology for FY2009. FSA should ensure that the actions in
Recommendations 1.1, 1.2, 1.3, and 1.4 are taken, if applicable, when
evaluating and reporting limitations, if any, in the FY2008 methodology,
and when designing, implementing, and reporting on the FY2009 and future
methodologies.

**FINDING NO. 2 -- FFEL Program Outlays Reported in PARs Represented
Different Payment Universes and Did Not Reflect the Most Current
Information**

The outlays for the FFEL Program reported in the Department's PARs for
FY2006 and FY2007 and used to calculate the estimated dollars of
improper payments reflected different payment universes. Also, the PARs
contained an estimated outlay amount for the current year when
information was available on the actual outlays for the year.

The annual PAR includes a table showing the improper payment reduction
outlook. The table contains improper payment information for each
risk-susceptible program for five fiscal years: prior year, current
year, and the next three years. The dollars of improper payment shown in
the table are calculated by multiplying the amount of the program
outlays by the estimated improper payment rate. The following table
shows the amounts that FSA reported in the PAR for FY2007.

+--------------------------+--------------+------------+--------------+
| **Table 2. Improper      |              |            |              |
| Payment Reduction        |              |            |              |
| Outlook**                |              |            |              |
|                          |              |            |              |
| **Reported in the FY2007 |              |            |              |
| PAR for the FFEL         |              |            |              |
| Program**                |              |            |              |
+--------------------------+--------------+------------+--------------+
|                          | **FY2006**   | **FY2007** | **FY2008**   |
|                          |              |            | (a)          |
|                          | **(Prior     | **(Current |              |
|                          | Year)**      | Year)**    | **(Current   |
|                          |              |            | Year+1)**    |
+--------------------------+--------------+------------+--------------+
| Program outlays for      | \$11,718     | \$5,861    | \$4,307      |
| fiscal year (b)          | million      | million    | million      |
+--------------------------+--------------+------------+--------------+
| Estimated improper       | 2.2 percent  | 0.0        | 0            |
| payment rate (c)         |              | 32 percent | .032 percent |
+--------------------------+--------------+------------+--------------+
| Estimated dollars of     | \$258        | \$2        | \$1 million  |
| improper payments        | million      | million    |              |
| for fiscal year          |              |            |              |
+--------------------------+--------------+------------+--------------+
| (a) The amounts for      |              |            |              |
|     FY2008 were also     |              |            |              |
|     used in the PAR to   |              |            |              |
|     report for FY2009    |              |            |              |
|     (Current Year+2) and |              |            |              |
|     FY2010 (Current      |              |            |              |
|     Year+3).             |              |            |              |
|                          |              |            |              |
| (b) The program outlays  |              |            |              |
|     for FY2006 reflect   |              |            |              |
|     the actual payments  |              |            |              |
|     made to lenders and  |              |            |              |
|     guaranty agencies in |              |            |              |
|     FMS (the             |              |            |              |
|     Department's         |              |            |              |
|     financial management |              |            |              |
|     system). The program |              |            |              |
|     outlays for FY2007   |              |            |              |
|     and later years      |              |            |              |
|     reflect the          |              |            |              |
|     estimated outlays    |              |            |              |
|     for FY2007 and       |              |            |              |
|     FY2008 reported in   |              |            |              |
|     the President's      |              |            |              |
|     Budget for FY2008.   |              |            |              |
|                          |              |            |              |
| (c) The rate used for    |              |            |              |
|     FY2006 was developed |              |            |              |
|     using monetary       |              |            |              |
|     findings identified  |              |            |              |
|     in audit and program |              |            |              |
|     review reports       |              |            |              |
|     issued in FY2005.    |              |            |              |
|     The rate used for    |              |            |              |
|     FY2007 and future    |              |            |              |
|     years was developed  |              |            |              |
|     from reviews of a    |              |            |              |
|     random sample of     |              |            |              |
|     payments made to     |              |            |              |
|     lenders and guaranty |              |            |              |
|     agencies in FY2005.  |              |            |              |
|     As we disclosed in   |              |            |              |
|     Finding No. 1, the   |              |            |              |
|     0.032 percent was    |              |            |              |
|     derived from an      |              |            |              |
|     interim calculation  |              |            |              |
|     that did not include |              |            |              |
|     all improper         |              |            |              |
|     payments identified  |              |            |              |
|     in the reviews and,  |              |            |              |
|     thus, understates    |              |            |              |
|     the estimated        |              |            |              |
|     improper payment     |              |            |              |
|     rate. Finding No. 1  |              |            |              |
|     provides additional  |              |            |              |
|     details on the two   |              |            |              |
|     methodologies.       |              |            |              |
+--------------------------+--------------+------------+--------------+

For the prior year's program outlays, FSA used an amount obtained from
FMS. The amount reflected actual payments made by the Department to
lenders and guaranty agencies during the fiscal year. For the current
and future years' program outlays, FSA used estimated outlay amounts
reported in the President's Budget. The amounts reflected the net
present value of the anticipated payments that will be made to lenders
and guaranty agencies for guaranteed loans disbursed by lenders during
the respective fiscal year. Thus, the program outlays in the President's
Budget represent amounts for payments that will be made to lenders and
guaranty agencies over an extended period of years in the future. The
payment universe reflected in the President's Budget is substantially
different from the payment universe reflected in FMS. For example, the
President's Budget for FY2008 reported actual outlays of \$27,132
million for FY2006. As shown in Table 2, the amount of actual outlays
from FMS was \$11,718 million.

FSA advised us that amounts obtained from FMS and the President's Budget
were used to comply with OMB requirements. OMB Circular A-123, Appendix
C instructs agencies to follow the format required by OMB Circular
A-136, *Financial Reporting Requirements* for reporting improper payment
information in the PAR. OMB Circular A-136, III.5.7, IV (a)(vi) provides
the following guidance on amounts to be used when reporting on improper
payments---

> \[A\]gencies are expected to report on CY \[current year\] activity,
> and if not feasible, then PY \[prior year\] activity is acceptable.
> (Future year outlay estimates (CY+1, +2, and +3) should match the
> outlay estimates for those years as reported in the most recent
> President's Budget).

While FSA complied with the instructions in the OMB Circular, its use of
different payment universes for program outlays resulted in dollars of
improper payments presented in the PAR that were not comparable among
the fiscal years.

Also, as noted in Table 2, FSA used estimated outlays from the
President's Budget for FY2008 for current year outlays reported in the
FY2007 PAR. Since the PAR, which was issued in November 2007, contained
the Department's financial statements for FY2007, the actual outlay
amounts for the current year should have been available for improper
payment reporting. The OMB Circular expects agencies to report current
year activity, when available.

## Recommendation  {#recommendation .RptSubcaption}

2.1 We recommend that the Acting Chief Operating Officer for FSA develop
and implement a revised policy for identifying and reporting program
outlays for the FFEL Program in the PARs that provides consistent and
comparable information on outlays and dollars of improper payments
reported and utilizes the most currently available information on
outlays.

**FSA Comments**

FSA stated that its operational policy and procedures are being updated
to include the FFEL payment universe definition, steps used to extract
the payment universe for outlay reporting, and queries to use for
improper payment reporting to ensure consistency in future PARs.

OBJECTIVE, SCOPE, AND METHODOLOGY

The purpose of the audit was to assess FSA's methodology for estimating
improper payments in the FFEL Program. Our review covered the
methodology used for reporting in the PARs for FY2006 and FY2007.

To gain an understanding of the methodologies and related internal
controls, we interviewed FSA management and staff in the Business
Operations unit and Program Compliance's Financial Partner Eligibility
and Oversight (FPE&O) unit. We also contacted FSA's offices of the Chief
Financial Officer and Enterprise Performance Management Services, as
well as the Department's OCFO, to understand their involvement in FSA's
IPIA efforts for the FFEL Program.

To gain a more complete understanding of the requirements of the IPIA,
we interviewed the OMB personnel who were involved with the development
of Appendix C to OMB Circular A‑123 and with whom FSA coordinated its
IPIA activities.

To assess the methodologies, we reviewed FSA's written policies and
procedures, written communications between FSA and OMB, and supporting
documentation for each methodology.

-   For the FY2006 methodology, our review was limited to review of
    FSA's documentation of its methodology and four program audits, and
    interviews with the staff who had implemented the methodology.

-   For the FY2007 methodology, we reviewed FSA's documentation of its
    methodology and assessed the quality of FPE&O's reviews during our
    site visits to its western and southern regional offices. At the
    western region, we examined all four reviews that had been completed
    by FPE&O at the time of our visit (reviews of one guaranty agency
    and three lender invoices). At the southern region, we judgmentally
    selected three reviews for examination (reviews of one guaranty
    agency and two lender invoices) to include reviews conducted by both
    the Dallas and Atlanta field offices and a review that had been
    conducted as a desk review. We compared source documentation from
    lenders/lender servicers and guaranty agencies against the data
    documented in FPE&O reviewers' summary worksheets. We used FSA's
    GAFR guide to verify the reinsurance rates and retention rates for
    the guaranty agency reviews, and FSA's LaRS guide and information
    provided by the National Council of Higher Education Loan Programs
    to verify the special allowance category codes and interest rates
    for the lender/lender servicer reviews. We reviewed the logic and
    consistency of formulas in the reviewers' electronic worksheets. We
    also interviewed the statistician, assigned to the Department's
    Institute of Education Sciences, National Center for Education
    Statistics, to gain an understanding of his role in reviewing FSA's
    methodology for selecting a statistically valid sample and for
    projecting the sample results.

We conducted our audit fieldwork at FSA's headquarters office in
Washington, D.C. and at its western (San Francisco) and southern
(Dallas) regional offices. We judgmentally chose San Francisco because
of its proximity to our Sacramento audit office and Dallas to meet with
the FPE&O staff who had obtained the supporting data for the stage one
samples and used it to select the stage two sub-samples for the FY2007
entity reviews.

We held an exit briefing with FSA officials on April 10, 2008. Our audit
was conducted in accordance with generally accepted government auditing
standards appropriate to the scope of the review described above.

ADMINISTRATIVE MATTERS

Corrective actions proposed (resolution phase) and implemented (closure
phase) by your office will be monitored and tracked through the
Department's Audit Accountability and Resolution Tracking System
(AARTS). Department policy requires that you develop a final corrective
action plan (CAP) for our review in the automated system within 30 days
of the issuance of this report. The CAP should set forth the specific
action items and targeted completion dates necessary to implement final
corrective actions on the findings and recommendations contained in this
final audit report.

In accordance with the Inspector General Act of 1978, as amended, the
Office of Inspector General is required to report to Congress twice a
year on the audits that remain unresolved after six months from the date
of issuance.

In accordance with the Freedom of Information Act (5 U.S.C. §552),
reports issued by the Office of Inspector General are available to
members of the press and general public to the extent information
contained therein is not subject to exemptions in the Act.

We appreciate the cooperation given us during this review. If you have
any questions, please call Gloria Pilotti at (916) 930-2399.

Sincerely,

/s/

Keith West

> Assistant Inspector General for Audit

Attachment

**ATTACHMENT**

## FSA's Comments on the Draft Report {#fsas-comments-on-the-draft-report .RptSubcaption}

![](media/image1.wmf){width="0.875in" height="0.875in"}

**\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_**

August 19, 2008

> \[Date stamped\]

**TO:** Keith West

> Assistant Inspector General for Audit Services
>
> Office of Inspector General

**FROM:** Lawrence A. Warder /s/

> Acting Chief Operating Officer

**SUBJECT:** Draft Audit Report -- "Federal Student Aid's Estimation of
Improper Payments in the Federal Family Education Loan Program," Control
Number ED-OIG/A09H0015

Thank you for providing us with an opportunity to respond to the Office
of Inspector General's (OIG) Draft Audit Report, "Federal Student Aid's
Estimation of Improper Payments in the Federal Family Education Loan
(FFEL) Program," dated July 9, 2008.

Federal Student Aid is committed to meeting the requirements of OMB
Circular A-123, Appendix C, for estimating improper payments in the FFEL
program. As you know, Federal Student Aid contracted with Oak Ridge
National Laboratory to provide an FFEL improper payment estimate for FY
2008. In addition, the Department's Office of the Chief Financial
Officer and Federal Student Aid are meeting with OMB regularly to reach
agreement by the end of this fiscal year on a methodology for fiscal
years 2009 and beyond that will satisfy OMB's requirements.

Our response to each of the recommendations follows:

**Finding 1, recommendation 1:** Ensure that the design and
implementation of any future improper payment estimating methodologies
that utilize results of previously conducted audits and reviews take
into account or mitigate the effects of the following factors, when
feasible: 1) improper payments identified in reviews other than single
audits, program audits, OIG audits, and FSA program reviews of lenders
and guaranty agencies (including improper payments self-reported by
lenders); 2) lack of quantification of monetary impact of findings in
the reports; 3) review periods that do not correlate to the federal
fiscal year; 4) not being representative of all lenders and guaranty
agencies; 5) duplicate identification of improper payments; 6) the time
between the occurrence of the improper

payments and the fiscal year used for reporting the PAR; and 7) improper
payments identified in contested monetary findings and findings without
a final determination of liabilities.

**Response**: Federal Student Aid will design and implement, in
consultation with OMB, a methodology for estimating FFEL improper
payments that meets the requirements of Circular A-123, Appendix C.

**Finding 1, recommendation 2:** Ensure that future methodologies
appropriately include all payments, such as payments made by
transferring funds between accounts (i.e., guaranty agencies' transfer
of funds from their Federal fund to their Operating fund) and payments
netted by larger amounts due to the Department (i.e., accounts
receivable invoices).

**Response:** Federal Student Aid will design and implement, in
consultation with OMB, a methodology for estimating FFEL improper
payments that meets the requirements of Circular A-123, Appendix C.

**Finding 1, recommendation 3:** Ensure that future methodologies use
information on improper payments for the fiscal year used for reporting
in the PAR, or if not available, for the prior fiscal year.

**Response:** Federal Student Aid will design and implement, in
consultation with OMB, a methodology for estimating FFEL improper
payments that meets the requirements of Circular A-123, Appendix C.

**Finding 1, recommendation 4:** If sampling is used in future
methodologies, consider focusing sample selection on higher-risk
entities or types of transactions, such as subsequent adjustments and
special allowance subsidies, to increase the likelihood of identifying
improper payments.

**Response:** Federal Student Aid will design and implement, in
consultation with OMB, a methodology for estimating FFEL improper
payments that meets the requirements of Circular A-123, Appendix C.

**Finding 1, recommendation 5:** Finalize its calculation of the
estimated improper payment rate for FY 2007 (including the measure of
the statistical precision of the estimated rate) and use the finalized
rate and its statistical precision when reporting the estimated improper
payments for FY 2007 in the PAR for FY 2008. The FY 2008 PAR should also
include an explanation of the change from the rate reported in the FY
2007 PAR.

**Response:** Federal Student Aid will use the final rate from the 2007
reviews to calculate the 2007 improper payments in the 2008 PAR and will
note the change from the rate used in the 2007 PAR.

**Finding 1, recommendation 6:** In future PARs, disclose when
information presented on improper payments is based on interim
calculations and the limitations of the reported information.

**Response:** Federal Student Aid will so disclose in future PARs.

**Finding 2, recommendation 1:** Develop and Implement a revised policy
for identifying and reporting program outlays for the FFEL Program in
the PARs that provides consistent and comparable information on outlays
and dollars of improper payments reported and utilizes the most
currently available information on outlays.

**Response:** Federal Student Aid is updating operational policy and
procedures to include the FFEL payment universe definition, steps used
to extract the payment universe for outlay reporting, and queries to use
for improper payment reporting to ensure consistency in future PARs.

Once again, we thank you for your recommendations and the opportunity to
review and respond to the report.

cc: Gloria Pilotti, Regional Inspector General for Audit

> Patrick J. Howard, Director, Student Financial Assistance Advisory and
>
> Assistance Team

[^1]: Appendix C consolidated requirements that were previously
    communicated to agencies in OMB memoranda issued on January 16,
    2003, May 8, 2003, and May 21, 2003, and clarified and updated the
    requirements.

[^2]: The estimated improper payment rate of 0.032 percent reported in
    the PAR for FY2007 was based on an interim calculation. In April
    2008, FSA updated its calculation of the estimated improper payment
    rate in order to respond to an information request from the OIG
    auditors. The recalculation resulted in an estimated improper
    payment rate of 0.218 percent.

[^3]: Single audits are audits conducted in accordance with OMB Circular
    A-133, *Audits of States, Local Governments, and Non-Profit
    Organizations.* Program audits are audits conducted in accordance
    with the Department's Audit Guide, *Compliance Audits (Attestation
    Engagements) for Lenders and Lender Servicers Participating in the
    Federal Family Education Loan Program*.

[^4]: For example, the FSA program reviews covered guaranty agencies who
    guaranteed \$174,034,845,527 of the \$300,142,555,788 in total loan
    balances. FSA identified \$533,440 of monetary findings for program
    reviews at the reviewed guaranty agencies. Using the following
    formula, FSA calculated \$919,978 as the amount of improper payments
    identified from program reviews at guaranty agencies:

    [\$300,142,555,788]{.underline}X \$533,440
    =\$919,978\$174,034,845,527

[^5]: *Special Allowance Payments to New Mexico Educational Assistance
    Foundation for Loans Funded by Tax‑Exempt Obligations*
    (ED-OIG/A05E0017), issued May 2005.

[^6]: FSA calculated the amount of improper payments identified from OIG
    audit reports using the following formula:

    [\$281,140,952,852]{.underline}X \$688,767
    =\$180,880,488\$1,070,544,497

[^7]: *Special Allowance Payments to Nelnet for Loans Funded by
    Tax‑Exempt Obligations* (ED-OIG/A07F0017), issued September 2006.
    Since the audit report was issued in FY2006, the report was not
    included in the reports reviewed to identify improper payments for
    development of the estimated improper payment rate for FY2006. The
    FY2006 estimated rate was based on improper payments identified in
    reviews of audit and program review reports issued in FY2005.

[^8]: FSA used a number of criteria to determine if desk reviews were
    appropriate, such as entity portfolio size, complexity of known
    issues, and the lender's ability to provide appropriate requested
    information.

[^9]: One of the two cancelled reviews was for a closed guaranty agency.

[^10]: The estimated improper payment rate would have been 0.033 percent
    if FSA had used the correct amount for the total of estimated
    improper payments (\$92,992) in its calculation.

[^11]: Except for certain exceptions, such as cases involving fraud or
    cases where the student never attended school.

[^12]: The formula takes into account a school's default rate and the
    amount of the ineligible disbursements to calculate an amount that
    includes the estimated actual loss to the government for which the
    school is now liable.

[^13]: FSA does not provide the school with the option of purchasing the
    ineligible loan from the lender and establishing a receivable from
    the borrower as this may deprive the borrower of certain benefits,
    such as consolidating their loans or requesting income contingent
    repayment plans. In addition, the loan agreement gives the borrower
    benefits, such as deferment and cancellation provisions, that may
    not be easily enforced against the school.

[^14]: ^?^ The presumed improper payment amount is calculated by
    multiplying the payment universe amount (\$3,507,207,010) by the
    presumed error rate (3%).

[^15]: The estimated improper payment rate would have been 0.220 percent
    if FSA had used the correct amount for the total of estimated
    improper payments (\$621,648) in its calculation.