Improper Payments: Selected Agencies Need Improvements in Their Assessments to Better Determine and Document Risk Susceptibility
Released: 2019-01-10T00:00:00

Letter

Background

Improper Payment Requirements

IPIA defines an improper payment as any payment that should not have been made or that was made in an incorrect amount (including overpayments and underpayments) under statutory, contractual, administrative, or other legally applicable requirements. It includes duplicate payments, any payment made to an ineligible recipient, any payment for an ineligible good or service, any payment for a good or service not received (except for such payments where authorized by law), and any payment that does not account for credit for applicable discounts. OMB M-15-02 also provides that when an agency’s review is unable to determine whether a payment was proper as a result of insufficient or lack of documentation, this payment must also be considered an improper payment.

IPIA also defines the scope of payments subject to improper payment requirements. Specifically, a payment is any transfer or commitment for future transfer of federal funds—such as cash, securities, loans, loan guarantees, and insurance subsidies—to any nonfederal person or entity that is made by a federal agency, a federal contractor, a federal grantee, or a governmental or other organization administering a federal program or activity.

Executive branch agencies are required to take various steps regarding improper payments under IPIA and as directed by OMB M-15-02. The steps include the following: 1. reviewing all programs and activities and identifying those that may be susceptible to significant improper payments (commonly referred to as a risk assessment), 2. developing improper payment estimates for those programs and activities that agency risk assessments, OMB, or statute identifies as being susceptible to significant improper payments, 3. analyzing the root causes of improper payments and developing corrective actions to reduce them, and 4. reporting on the results of addressing the foregoing requirements.

Figure 1 illustrates these steps, as well as the major components of conducting an improper payment risk assessment.

IPIA requires that agencies conduct improper payment risk assessments for all federal programs and activities at least once every 3 years and identify any program or activity that may be susceptible to significant improper payments. OMB guidance provides that programs that have been determined to be susceptible to significant improper payments and that are already reporting an estimate—or in the process of establishing an estimate—do not have to conduct additional improper payment risk assessments. IPIA defines “significant” improper payments as improper payments in the preceding fiscal year that may have exceeded either (1) 1.5 percent of program outlays and $10 million or (2) $100 million (regardless of the improper payment rate). OMB M-15-02 provides guidance for implementing the IPIA requirements and covers agencies’ responsibilities for improper payment risk assessments, estimation, and reporting.

OMB M-15-02 also lists steps that agencies should take when conducting improper payment risk assessments. Agencies must institute a systematic method of reviewing all programs and activities to identify those that may be susceptible to significant improper payments, as defined by IPIA. According to OMB M-15-02, this systematic method could be a quantitative evaluation based on a statistical sample or a qualitative method (e.g., a risk-assessment questionnaire). Prior to fiscal year 2018, at a minimum, agencies were required to take into account nine risk factors—seven specified in IPIA and two in OMB guidance—that are likely to contribute to improper payments, regardless of which method was used by the agency (see table 1).

In June 2018, OMB revised its guidance for improper payments in OMB Circular A-123, Appendix C, Requirements for Payment Integrity Improvement (OMB M-18-20). In the revised guidance, OMB no longer directs agencies to consider the two additional risk factors that were included in OMB M-15-02 in their risk assessments. Rather, OMB directs agencies to take into account those risk factors that are likely to contribute to a susceptibility of significant improper payments. The revised guidance also states that beginning in fiscal year 2020, agencies should use quantitative evaluations for programs or activities with outlays exceeding $5 billion. As specified in OMB M-18-20, the end goal of the systematic method of reviewing all programs, whether qualitative or quantitative, is to determine whether a program is susceptible to significant improper payments. Accordingly, OMB M-18-20 states that if a qualitative method is used, it must be designed to accurately determine whether the program is susceptible to significant improper payments.

Characteristics of Programs Reviewed

When conducting improper payment risk assessments, each federal agency, unless otherwise specified by OMB Circular A-11, after consultation with OMB, is generally authorized to determine the grouping of programs that most clearly identifies and reports improper payments for the agency. The five programs we reviewed serve a variety of purposes and are administered by various agencies across the federal government.

Head Start

HHS’s Head Start program was established in 1965 to deliver comprehensive educational, social, health, nutritional, and psychological services to low-income families and their children. These services include preschool education, family support, health screenings, and dental care. Head Start was originally aimed at 3- to 5-year-olds. The Head Start program makes grants directly to approximately 1,600 local organizations, including community action agencies, school systems, tribal governments and associations, and for-profit and nonprofit organizations.

The Head Start program has several primary eligibility criteria to enroll in the program—including that the child’s family earns income below the federal poverty level; the child’s family is eligible for or, in the absence of child care, would potentially be eligible for public assistance; the child is in foster care; or the child is homeless. Head Start services are to be provided free of charge to eligible families.

Prior to fiscal year 2013, HHS reported improper payment estimates for the Head Start program. However, as of fiscal year 2013, HHS, in consultation with its Office of Inspector General (OIG) and with approval from OMB, no longer reports annual improper payment estimates related to the program. According to HHS, Head Start’s fiscal year 2017 outlays were approximately $9.4 billion.

Interest on the Public Debt

Public debt is defined as Treasury-issued securities, primarily consisting of marketable Treasury securities (i.e., bills, notes, and bonds), and a smaller amount of nonmarketable securities, such as savings bonds and special securities issued to state and local governments. A portion is debt held by the public and a portion is debt held by federal government accounts.

Debt held by the public represents federal debt held by investors outside of the federal government, including individuals, corporations, state or local governments, the Federal Reserve, and foreign governments. Types of securities held by the public include Treasury bills, notes, and bonds and State and Local Government Series securities. Debt held by the public primarily represents the amount the U.S. government has borrowed from the public to finance cumulative cash deficits. As of September 30, 2017, total debt held by the public was $14.7 trillion.

Debt held by federal government accounts (intragovernmental holdings) represents balances of federal government accounts of certain federal agencies that are either authorized or required to invest excess receipts in Treasury securities. As of September 30, 2017, total debt held by federal government accounts was $5.6 trillion.

Interest calculations on the public debt differ depending on the types of securities, their associated terms, and average interest rates. According to Treasury, total interest paid on public debt for fiscal year 2017 was approximately $294.8 billion.

Home Affordable Modification Program

In February 2009, as part of a broader plan to stabilize the housing market and economy, Treasury established the Making Home Affordable Program to help struggling families avoid possible foreclosure. As part of this plan, Treasury announced a national modification program for first- lien mortgages, the Home Affordable Modification Program (HAMP). The program offered eligible homeowners who are at risk of foreclosure reduced monthly mortgage payments that are more affordable and sustainable over the long term. Homeowners who chose to participate in the program had to show (1) documented financial hardship and (2) an ability to make their monthly mortgage payments after a modification.

HAMP works by encouraging participating mortgage servicers to modify mortgages so struggling homeowners can have lower monthly payments and avoid foreclosure. It has specific eligibility requirements for homeowners and includes strict guidelines for servicers.

In December 2016, entrance into the Making Home Affordable program expired. However, payments for previously approved participants in HAMP will continue until approximately September 2023. According to Treasury, HAMP’s fiscal year 2017 outlays were approximately $4.1 billion.

Law Enforcement

For improper payment risk assessment purposes, DOJ has five mission- aligned program groups. The Law Enforcement group is the largest in terms of annual outlays and consists of the following five components: 1. the Bureau of Alcohol, Tobacco, Firearms, and Explosives; 2. the Drug Enforcement Administration; 3. the Federal Bureau of Investigation; 4. Offices, Boards, and Divisions; and 5. the United States Marshals Service.

According to DOJ, Law Enforcement’s fiscal year 2017 outlays were approximately $11.8 billion.

Agriculture Risk Coverage and Price Loss Coverage

The Agriculture Risk Coverage (ARC) and Price Loss Coverage (PLC) programs were authorized by the 2014 Farm Bill to provide farmers with protection against adverse changes in market conditions. Although ARC and PLC are considered two separate programs, they are grouped as one program for the purposes of conducting improper payment risk assessments. The programs are managed by the Commodity Credit Corporation, whose activities are primarily administered by USDA’s Farm Service Agency.

Within the ARC program, farmers have the choice of an individual-based option, known as ARC-Individual, or a county-based option, known as ARC-County. Both options provide revenue loss coverage to farmers when the legislative guarantee for a crop exceeds the actual year revenue.

PLC program payments are issued to farmers when a crop’s “reference price,” as specified in the 2014 Farm Bill, is in excess of an average price, which is determined at the national level each year for the covered commodities.

ARC/PLC statutes and regulations establish a series of eligibility criteria that farmers must meet in order to enroll in the programs. Among other things, to be eligible farmers must produce a certain quantity of at least 1 of the 21 covered commodities, actively engage in the farming process, meet income eligibility limits, and meet certain land conservation requirements. According to USDA, ARC/PLC’s fiscal year 2017 outlays were approximately $9.6 billion.

Four of the Five Risk Assessments Lacked Documentation to Support Their Risk Determinations, and Many of HHS’s Programs Were Not Assessed

HHS’s Improper Payment Risk Assessment for Head Start Lacked Documentation to Support Its Low Risk Determination, and Many Other Programs Were Not Assessed HHS’s Improper Payment Risk Assessment for Head Start Lacked Documentation to Support Its Low Risk Determination

In its fiscal year 2016 qualitative risk assessment, HHS assessed its Head Start program as at low risk of susceptibility to significant improper payments. However, HHS did not have sufficient documentation on how it developed its risk assessments, so we could not determine if the risk assessment process was designed to provide a reasonable basis for making risk determinations.

Although HHS did take into account the nine risk factors, among other factors, HHS did not document or effectively demonstrate how each specific risk factor affected Head Start’s susceptibility to significant improper payments. HHS’s improper payment risk assessment template included the nine risk factors, among other factors, and described how the divisions should consider each risk factor. However, HHS did not document how the descriptors or individual risk factors relate to the program’s susceptibility to significant improper payments. Further, although HHS used a risk assessment template to assess each of the risk factors, which included space for the divisions to provide additional information regarding the risk determinations, the division responsible for the Head Start program did not always provide sufficient documentation or support for us to determine how it arrived at its risk determinations for each risk factor. For example, see the following:

Eligibility determination: HHS considered the eligibility of initial Head Start payments that HHS made to the initial grantees—local organizations—as low risk. However, HHS did not consider the Head Start eligibility decisions that these organizations made at the subrecipient level—calling into question the reliability of HHS’s risk assessment. In the Head Start program, local organizations, not HHS, make the eligibility determinations for individuals to be enrolled in the program. In addition, local organizations, not HHS, are responsible for maintaining the documentation to substantiate the eligibility of enrollees. HHS did not consider the impact of these determinations in its improper payment risk assessment. Our analysis of improper payment estimates from paymentaccuracy.gov for fiscal years 2016 and 2017 indicates that the inability to authenticate eligibility is one of the largest root causes of improper payments.

Audit findings: HHS assigned a low-risk rating for findings from oversight agencies. However, in the risk assessment, it identified nine audit reports that the OIG issued pertaining to Head Start agencies with findings on unallowable costs, enrollment, and misuse of grant funds. According to agency officials, these OIG reports contained findings related to costs and misuse of grant funds that are specific to particular grantees and may not be indicative of widespread programmatic issues. However, HHS did not document the rationale for this assessment.

Program management report: HHS assigned a low-risk rating for findings related to program management reports. According to HHS’s Report to Congress on Head Start Monitoring for Fiscal Year 2015, “allowable and allocable costs” was the most commonly cited noncompliance issue in its fiscal reviews of grantees. Specifically, 8.8 percent of grantees included in a fiscal review were found to be noncompliant with regard to allowable and allocable costs. However, HHS did not document whether it considered the impact of noncompliance by grantees in its Head Start risk assessment.

According to HHS officials, divisions were required to maintain supporting documentation for their risk assessments, although submission of the related documents along with the risk assessment was not mandatory.

HHS officials stated that this policy was orally communicated to the divisions; however, it was not formally documented. Lack of a written policy for the divisions to maintain such information may have contributed to HHS’s inability to provide sufficient supporting documentation for its low risk determinations.

HHS’s qualitative risk assessment for Head Start also did not document or effectively demonstrate how the total score for all risk factors led to a determination that the program was not susceptible to significant improper payments. Our analysis of HHS’s risk assessment showed that for several of its risk factors, HHS did not score those factors as low risk. For example, HHS assigned a high-risk rating for three of the nine risk factors: (1) permanency of the program, (2) volume of payments made through the program, and (3) complexity per transaction. HHS’s risk assessment did not document or support how it determined Head Start to overall be at low risk for susceptibility to significant improper payments given the high-risk ratings for certain risk factors. Without supporting documentation, HHS cannot demonstrate, and we cannot determine, if HHS’s low risk determination for Head Start was reasonable.

Additionally, based on HHS’s risk assessment scoring template, a program could be considered “high risk” for all nine risk factors, but because of the assigned weight given to each of the nine risk factors, HHS’s final risk calculation would still not determine the program to be at high risk of susceptibility to significant improper payments. According to HHS officials, the agency has procedures to review the improper payment risk assessments that the individual divisions perform; however, these review procedures are not formally documented. HHS officials stated that while no risk assessment has identified all nine risk factors as high risk, if all nine risk factors were identified as high risk by a division, the agency would require supporting documentation from the division for review and could overrule the outcome calculated based on the risk assessment scoring template if necessary. Without documented procedures for this review process, HHS lacks assurance that this process, if applicable, would consistently take place.

According to HHS, the fiscal year 2016 improper payment qualitative risk assessment template used for Head Start was designed to calculate an overall risk rating of low, medium, or high based on program management responses to each individual risk factor. However, HHS did not have documentation to demonstrate how it determined the weighting of the risk factors or how the numerical risk level ranges from the risk assessment template related to a program’s susceptibility to significant improper payments. Additionally, HHS did not have documentation demonstrating the basis for its determination that specific risk factors do or do not lead to susceptibility to significant improper payments. HHS officials stated that OMB does not have specific guidance on establishing weights for each risk factor or assigning numerical risk level ranges to determine overall susceptibility to significant improper payments. HHS officials also stated that HHS developed its own numerical risk level ranges based on experience and data from previous risk assessments. When asked for documentation to support its weighting of the various risk factors, HHS officials stated that they did not document this analysis. Without documenting the basis for the assigned weights, HHS cannot demonstrate, and we cannot determine, that its process for determining Head Start’s susceptibility to significant improper payments was reasonable.

Federal internal control standards state that management should develop control activities to achieve objectives and respond to risks and implement control activities through policies. As part of these standards, management should clearly document internal controls and other significant events in a manner that allows the documentation to be readily available for examination. Additionally, management should periodically review policies, procedures, and related control activities for continued relevance and effectiveness. Further, federal internal control standards state that management should use quality information to achieve the entity’s objectives. As such, to reasonably determine if a program is susceptible to significant improper payments, agencies’ risk assessments would have a logical connection with, or bearing upon, the statutory definition of significant improper payments. Until HHS revises its risk assessment process to help ensure that it results in a reliable assessment, it will be uncertain whether Head Start may be susceptible to significant improper payments and therefore require an estimate of its improper payments.

HHS Did Not Conduct Risk Assessments for Many of Its Programs and Activities

During our agency and program selection process, we found that HHS did not assess many of its programs and activities at least once during the 3- year period from fiscal years 2015 through 2017, as required by IPIA. Although HHS conducted improper payment risk assessments for a total of 71 programs and activities during the 3-year period, based on our analysis of HHS-provided outlay data, HHS did not conduct the required risk assessment for at least 140 programs. For example, HHS did not assess its Block Grants for Prevention and Treatment of Substance Abuse program that had outlays of approximately $1.8 billion in fiscal year 2016. According to HHS officials, HHS has limited resources, so it took a risk-based approach when selecting programs to include in its improper payment risk assessment process. Further, HHS officials stated that HHS was transitioning in fiscal year 2015 to a new risk assessment process. As such, HHS’s procedures directed its divisions to select one program per division for fiscal year 2015 and two programs per division for fiscal years 2016 and 2017.

Federal internal control standards state that management should design control activities to achieve objectives and respond to risks and implement control activities through policies. Without properly designed control activities to help ensure that all programs and activities are assessed for susceptibility to significant improper payments at least once every 3 years, as required by IPIA, there is an increased risk that HHS may not identify all risk-susceptible programs and activities, resulting in incomplete improper payment estimates.

Treasury’s Improper Payment Risk Assessments for Interest on the Public Debt and HAMP Lacked Documentation to Support Its Low Risk Determinations

In its fiscal year 2017 qualitative risk assessments, based on fiscal year 2016 outlay data, Treasury assessed its Interest on the Public Debt and HAMP as at low risk of susceptibility to significant improper payments. However, Treasury did not have sufficient documentation for how it developed its risk assessments, so we could not determine if the risk assessment process was designed to provide a reasonable basis for making risk determinations.

Although Treasury did take into account the nine risk factors, among other factors, it did not document or effectively demonstrate how each specific risk factor affected the programs’ susceptibility to significant improper payments. Treasury’s risk assessment templates for these programs had 62 questions which required “Yes,” “No,” or “Not applicable” responses. Treasury did not document how each of the 62 questions related to each program’s susceptibility to significant improper payments. Further, the template did not require the bureaus responsible for the Interest on the Public Debt and HAMP risk assessments to provide documentation or support other than a check mark in response to these questions. Without descriptions of how to answer the questions or documentation to support the responses, we could not verify the reasonableness of the Interest on the Public Debt or HAMP improper payment risk assessments.

For example, the Interest on the Public Debt program’s risk assessment questionnaire was completed for 11 different payment types under the program. For the TreasuryDirect payment type, Treasury answered “No” to the question, “Are there risks due to a high volume of payments for TreasuryDirect?” Treasury did not provide documentation or other support for how the agency determined that there was no risk for this question. Further, since the template lacked descriptors, it is unclear if responses related to the number of transactions or the dollar amount of transactions. In fiscal year 2016, TreasuryDirect payments totaled almost $300 billion, representing about 7.8 percent of all the federal government outlays. In contrast, Treasury answered “Yes” to this same question for the HAMP program, for which payments were about 1 percent (about $4 billion) of the total payments made by TreasuryDirect.

Similarly, in the HAMP risk assessment questionnaire, Treasury answered “No” to the question, “Are payment or payment eligibility decisions made outside the agency?” However, under HAMP, financial institutions, not Treasury, determine whether borrowers are eligible for loan modification through the program. Treasury did not document why a “No” response was appropriate.

Treasury’s risk assessments for Interest on the Public Debt and HAMP also did not document or effectively demonstrate how the total scores for all risk factors led to the determinations that the programs were not susceptible to significant improper payments. For example, in its risk assessment, Treasury’s responses indicated several improper payment risks for Interest on the Public Debt, including (1) complexity of administering the payment type, (2) unmitigated risks relying on contractors to perform critical agency operations, and (3) payments being made to incorrect payees or ineligible recipients.

Further, based on total payments for the Interest on the Public Debt, Treasury would have to be over 99.97 percent accurate in its payments in order for the activity to not reach the $100 million threshold for significant improper payments. Treasury’s risk assessment did not document or support how it determined Interest on the Public Debt to be at low risk for susceptibility to significant improper payments considering these risks for improper payments.

Similarly, Treasury’s responses in its risk assessment questionnaire indicated several improper payment risks for HAMP, including (1) an emphasis on expediting payments, (2) risks resulting from recent changes in agency operations and personnel, (3) complicated criteria for manually computing payments, and (4) a high volume of payments. Treasury’s risk assessment did not document or support how it determined HAMP to be at overall low risk for significant improper payments considering these risks for improper payments. Without supporting documentation, Treasury cannot demonstrate, and we cannot determine, if Treasury’s low risk determinations for Interest on the Public Debt and HAMP were reasonable.

Additionally, based on our analysis of Treasury’s risk assessment template, a bureau could identify areas of risk related to each of the nine risk factors for a program, but because of the assigned weights given to each of the nine risk factors, Treasury’s final risk calculation would still not determine the program to be at high risk of susceptibility to significant improper payments.

According to Treasury officials, Treasury provides general instructions on how to complete the risk assessment templates, but the bureaus are responsible for assessing the risks. In addition, according to Treasury, the fiscal year 2017 improper payment risk assessment template used for Interest on the Public Debt and HAMP was designed to calculate an overall risk rating of low, medium, or high based on bureau responses to each individual question. However, Treasury did not have documentation to demonstrate how it determined the weighting of the risk factors or the numerical risk level ranges from the template related to the programs’ susceptibility to significant improper payments. Additionally, Treasury did not have documentation demonstrating the basis for its determination that specific risk factors do or do not lead to susceptibility to significant improper payments. According to Treasury officials, Treasury considered the severity of the impact on the program’s improper payments when developing its weights for each question. However, Treasury officials stated that they did not have documentary support for this analysis. Without documenting the basis for the assigned weights, Treasury cannot demonstrate, and we cannot determine, that its process for determining its programs’ susceptibility to significant improper payments was reasonable.

Federal internal control standards state that management should develop control activities to achieve objectives and respond to risks and implement control activities through policies. As part of these standards, management should clearly document internal controls and other significant events in a manner that allows the documentation to be readily available for examination. Additionally, management should periodically review policies, procedures, and related control activities for continued relevance and effectiveness. Further, federal internal control standards state that management should use quality information to achieve the entity’s objectives. As such, to reasonably determine if a program is susceptible to significant improper payments, agencies’ risk assessments would have a logical connection with, or bearing upon, the statutory definition of significant improper payments. Until Treasury revises its risk assessment process to help ensure that it results in reliable assessments, it will not be certain whether Interest on the Public Debt or HAMP may be susceptible to significant improper payments and therefore require an estimate of improper payments.

DOJ’s Improper Payment Risk Assessment for Law Enforcement Lacked Documentation to Support Its Low Risk Determination

In its fiscal year 2017 risk assessment, DOJ assessed its Law Enforcement program as at low risk of susceptibility to significant improper payments. However, DOJ did not have sufficient documentation for how it developed its risk assessments, so we could not determine if the risk assessment process was designed to provide a reasonable basis for making risk determinations.

Although DOJ conducted a quantitative evaluation as part of its improper payment risk assessment for its Law Enforcement program, the evaluation did not reliably indicate the program’s susceptibility to significant improper payments. Specifically, our analysis of Law Enforcement’s improper payment risk assessment found that the quantitative evaluation’s baseline was largely based on the prior fiscal year’s improper payment amount identified through recovery activities, which may not reliably represent the estimated improper payment amount that the agency incurred. For example, improper payment recovery activities do not include underpayments.

DOJ’s qualitative analysis on improper payments also did not document or effectively demonstrate whether the program may be susceptible to significant improper payments. Although DOJ’s risk assessment template did take into account the nine risk factors, among other factors, and descriptors of how the components should consider each risk factor, DOJ did not document or effectively demonstrate how each specific risk factor affected the program’s susceptibility to significant improper payments. Further, although DOJ used a risk assessment template to assess each of the risk factors, which included a voluntary comments section for each risk factor so that components can explain answers or justify the risk ratings, the components frequently left the comment sections blank. As such, DOJ did not always provide sufficient documentation or support for us to determine how the components arrived at their risk determinations for each risk factor.

DOJ’s risk assessment for Law Enforcement also did not document or effectively demonstrate how the total score for all risk factors led to the determination that the program was not susceptible to significant improper payments. For example, in its risk assessment, DOJ’s Offices, Boards, and Divisions’ responses indicated risks for contract payments related to (1) changes in funding, authorities, practices, or procedures; (2) results of monitoring activities; (3) results of recapture audit activities; (4) volume and dollar amount of payments; (5) inherent risks; and (6) capability of personnel. DOJ’s risk assessment did not document or support how it determined Law Enforcement to be at low risk for susceptibility to significant improper payments given the identified risks for certain risk factors. Without supporting documentation, DOJ cannot demonstrate, and we cannot determine, if DOJ’s low risk determination for Law Enforcement was reasonable. Additionally, based on our analysis of DOJ’s risk assessment template, a component could identify areas of risk related to each of the nine risk factors, but because of the assigned weight given to each of the nine risk factors, DOJ’s final risk calculation would still not determine the program to be at high risk of susceptibility to significant improper payments.

According to DOJ, the fiscal year 2017 improper payment qualitative risk assessment template used for Law Enforcement was designed to calculate an overall risk rating of low, medium, or high based on component responses to each individual risk factor. However, DOJ did not have documentation to demonstrate how it determined the weighting of the risk factors or the numerical risk level ranges from the template related to the program’s susceptibility to significant improper payments. Additionally, DOJ did not have documentation demonstrating the basis for its determination that specific risk factors do or do not lead to susceptibility to significant improper payments. Further, DOJ’s qualitative risk assessment template indicated that the overall risk determination does not relate to the program’s susceptibility to significant improper payments. For example, the template stated that “a risk rating of high risk for the purposes of this assessment does not mean that the payment type is susceptible to significant improper payments but may indicate that additional focus and testing should be placed on that payment type to better estimate the improper payment rate for the payment type.” DOJ officials stated that DOJ held internal discussions and considered the severity of the impact on the program’s improper payments when developing its weights for each risk factor. When asked for supporting documentation, DOJ officials stated that OMB guidance does not direct agencies to demonstrate how the weights for each risk factor or overall risk ratings relate to the definition of significant improper payments.

However, without documenting the basis for the assigned weights, DOJ cannot demonstrate, and we cannot determine, that its process for determining Law Enforcement’s susceptibility to significant improper payments was reasonable.

Federal internal control standards state that management should develop control activities to achieve objectives and respond to risks and implement control activities through policies. As part of these standards, management should clearly document internal controls and other significant events in a manner that allows the documentation to be readily available for examination. Additionally, management should periodically review policies, procedures, and related control activities for continued relevance and effectiveness. Further, although OMB does not direct agencies to demonstrate how the weights for each risk factor or overall ratings relate to the definition of significant improper payments, federal internal control standards state that management should use quality information to achieve the entity’s objectives. As such, to reasonably determine if a program is susceptible to significant improper payments, agencies’ risk assessments would have a logical connection with, or bearing upon, the statutory definition of significant improper payments. Until DOJ revises its risk assessment process to help ensure that it results in a reliable assessment, it will be uncertain whether Law Enforcement may be susceptible to significant improper payments and therefore require an estimate of improper payments.

USDA’s Improper Payment Risk Assessment for ARC/PLC Provided a Reasonable Basis for Its Risk Determination

USDA’s fiscal year 2017 improper payment risk assessment for ARC/PLC consisted of a qualitative analysis and a quantitative evaluation. Both assessments determined that the program was not susceptible to significant improper payments. We found that the quantitative evaluation, based on statistical sampling, provided a reasonable basis for USDA’s determination that the program was at low risk for susceptibility to significant improper payments. Specifically, based on its statistical sample, USDA estimated that ARC/PLC’s improper payment rate was 0.73 percent of program outlays with an estimated improper payment amount of $38.6 million. As such, the analysis clearly demonstrated that ARC/PLC did not meet the statutory definition of significant improper payments under IPIA—estimated improper payments that may have exceeded either (1) 1.5 percent of program outlays and $10 million or (2) $100 million (regardless of the improper payment rate).

Conclusions

Properly executed improper payment risk assessments are a cornerstone of government-wide efforts to estimate and reduce such payments. Although the qualitative risk assessments we reviewed for HHS, Treasury, and DOJ considered the nine risk factors required by IPIA or directed by OMB, none of them demonstrated how the factors affected a program’s susceptibility to significant improper payments. Additionally, despite the agencies identifying multiple factors as areas of risk in individual program risk assessments, each of the agencies’ overall determinations for the risk assessments we reviewed was “low risk,” and none of the agencies had documentation with which to explain the basis for their assessments.

Revising their processes for conducting improper payment risk assessments, including preparing sufficient documentation to support the assessments, would better position HHS, Treasury, and DOJ to demonstrate the reliability of the assessments. Without properly designed risk assessments, the departments will continue to be uncertain whether improper payment estimates should be prepared for most programs we reviewed, potentially affecting the completeness of their improper payment estimates and hampering efforts to reduce improper payments.

Recommendations for Executive Action

We are making the following four recommendations—two to HHS, one to Treasury, and one to DOJ: The Secretary of Health and Human Services should revise HHS’s process for conducting improper payment risk assessments for Head Start to help ensure that it results in a reliable assessment of whether the program is susceptible to significant improper payments. This should include preparing sufficient documentation to support its risk assessments. (Recommendation 1)

The Secretary of Health and Human Services should revise HHS’s procedures for conducting improper payment risk assessments to help ensure that all programs and activities are assessed for susceptibility to significant improper payments at least once every 3 years, as required by IPIA. (Recommendation 2)

The Secretary of the Treasury should revise Treasury’s processes for conducting improper payment risk assessments for Interest on the Public Debt and HAMP to help ensure that the processes result in reliable assessments of whether the programs are susceptible to significant improper payments. This should include preparing sufficient documentation to support its risk assessments. (Recommendation 3)

The Attorney General should revise DOJ’s process for conducting improper payment risk assessments for Law Enforcement to help ensure that it results in a reliable assessment of whether the program is susceptible to significant improper payments. This should include preparing sufficient documentation to support DOJ’s risk assessments. (Recommendation 4)

Agency Comments and Our Evaluation

We provided a draft of this report for comment to OMB, HHS, DOJ, Treasury, and USDA. DOJ and HHS provided written comments, which are reproduced in appendixes II and III, respectively. OMB, HHS, and Treasury provided technical comments, which we incorporated as appropriate. Treasury’s Acting Director of its Risk and Control Group notified us by email that Treasury concurred with the report and recommendation. A USDA management analyst notified us by email that USDA had no comments on the report.

In its written comments, HHS stated that it concurs with both recommendations and is committed to reducing improper payments in all of its programs. HHS also described actions it plans to take to address these recommendations, including (1) issuing a written policy directing divisions to maintain supporting documentation for risk assessments, (2) documenting the agency review procedures for risk assessments that the divisions perform and the rationale for assigning weights to the risk factors, and (3) developing an automated program identification process for monitoring and inclusion in risk assessments to help ensure that all programs and activities are reviewed. The actions described by HHS, if implemented effectively, would address our recommendations.

In its written comments, DOJ stated that it disagreed with our conclusions and recommendation. DOJ explained that its risk assessment methodology includes a qualitative evaluation and a quantitative analysis, and that it considers the nine risk factors likely to contribute to improper payments. Additionally, DOJ provided an overview of its risk assessment tool and guidance and stated that its methodology includes all steps required by OMB. We acknowledged in the draft report that DOJ did take into account the nine risk factors, among others, as directed by OMB and provided an overview of DOJ’s risk assessment template and process.

DOJ stated that it believes that some of our interpretations exceed the risk assessment requirements, and believes that its methodology complies with requirements and adequately demonstrates whether a program may be susceptible to significant improper payments. DOJ stated that the risk factor ratings summarized in its risk assessment provided a clear link of how the individual risk factor ratings support the overall assessed risk of significant improper payments. Further, DOJ stated that the risk assessment tool provides sufficient documentation for the formulas and logic for the risk rating conversions and weight-based summarization of risk factor scoring.

We disagree that our interpretations exceed the risk assessment requirements, and we continue to believe that DOJ’s risk assessment did not adequately demonstrate whether a program is or is not susceptible to significant improper payments. We believe that while agencies are not specifically directed to demonstrate how the weights for each risk factor or overall ratings relate to the definition of significant improper payments, management should use quality information to achieve the entity’s objectives as stated in federal internal control standards. As such, to reasonably determine if a program is susceptible to significant improper payments, agencies should have documentation to support how their risk assessments provided a logical connection with, or bearing upon, the statutory definition of significant improper payments. DOJ did not provide sufficient support for how it determined the weighting of the risk factors or the numerical risk level ranges. Because DOJ did not have sufficient documentation for how it developed its risk assessment template, we could not determine if the risk assessment was designed to provide a reasonable basis for the risk determinations.

DOJ stated that the report does not accurately portray DOJ’s risk assessment process. Specifically, DOJ stated that we incorrectly reported that DOJ’s quantitative evaluation did not include improper payments related to lack of documentation. Based on the information DOJ provided, we removed the lack of documentation example from our report.

DOJ also stated that it was misleading to report that although DOJ’s risk assessment template included a voluntary comments section for each risk factor for components to explain answers or justify risk ratings, the comment sections were frequently left blank. DOJ stated that its components only need to provide a comment when they believe it is necessary to qualify their responses and that obvious answers do not need to be explained. However, as previously noted, DOJ did not provide sufficient documentation or support for us to determine how the components arrived at their risk determinations for each risk factor. Without such documentation, DOJ cannot demonstrate, and we cannot determine, whether DOJ’s assessment for each risk factor was reasonable.

Further, DOJ stated that the Offices, Boards, and Divisions example was inaccurate and misleading. DOJ stated that the summary table in its risk assessment questionnaire documented that the risks identified were determined to be low risk and therefore supported the conclusions reached. DOJ also stated that its approach acknowledges that risks exist in every disbursement process and allows process owners to assess the level of risk that exists and determine whether a program may be susceptible to significant improper payments. We disagree that the Offices, Boards, and Divisions example is inaccurate or misleading. Although we recognize that DOJ’s summary table, or scoring template as referred to in the report, documented that the risks identified were determined to be low risk, we do not believe that it provided support for that determination. Specifically, the summary table was populated based on component responses and predetermined weights to calculate an overall risk rating of low, medium, or high; however, DOJ did not provide documentation to demonstrate how it determined the weights of the risk factors or the numerical risk level ranges involved in that calculation. Without documenting the basis for the assigned weights, DOJ cannot demonstrate, and we cannot determine, that its process for determining Law Enforcement’s susceptibility to significant improper payments was reasonable.

We continue to believe that our recommendation to DOJ is valid to help ensure that DOJ’s risk assessment reliably results in determining whether Law Enforcement may be susceptible to significant improper payments.

We are sending copies of this report to the appropriate congressional committees, the Director of the Office of Management and Budget, the Secretary of Agriculture, the Secretary of Health and Human Services, the Secretary of the Treasury, the Acting Attorney General, and other interested parties. In addition, this report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202) 512-2623 or davisbh@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV.

Appendix I: Objective, Scope, and Methodology

This report examines the extent to which certain federal agencies’ improper payment risk assessments for selected programs provided a reasonable basis for determining susceptibility to significant improper payments.

To address our objective, we reviewed improper payment risk assessment requirements in the Improper Payments Information Act of 2002, as amended (IPIA), and the related guidance in Office of Management and Budget (OMB) Circular A-123, Appendix C, Requirements for Effective Estimation and Remediation of Improper Payments (OMB M-15-02). We analyzed this statute and guidance to identify key criteria that agencies must meet when conducting improper payment risk assessments. IPIA identifies seven risk factors and OMB guidance includes two additional risk factors that agencies must consider, at a minimum, in their improper payment risk assessments to determine susceptibility to significant improper payments. IPIA also directs agencies to conduct risk assessments for all programs and activities at least once every 3 years. We also reviewed relevant internal control standards to determine the relevant processes and procedures needed to help ensure that agencies conduct effective improper payment risk assessments to determine the susceptibility to significant improper payments.

For this objective, we selected a nongeneralizable sample of 4 agencies and five programs to review. To select the agencies, we considered data for the 24 agencies subject to the Chief Financial Officers Act of 1990 (CFO Act). Specifically, we considered the timing of the agencies’ improper payment risk assessments, findings reported by the agencies’ inspectors general (IG), the number of programs and activities for which the agencies reported improper payment estimates for fiscal year 2017, the types of programs and activities that the agencies administered, and agency gross outlays in fiscal year 2017. To ensure we were including agencies that had most recently conducted improper payment risk assessments, we limited our selection to agencies that conducted improper payment risk assessments for any programs or activities in fiscal year 2017. In order to avoid duplicate efforts, we also eliminated agencies that reported IG findings related to risk assessments. We then selected a mix of agencies with and without improper payment estimates for fiscal year 2017, and ultimately selected 4 agencies based primarily on their fiscal year 2017 outlays for programs determined to be not susceptible to improper payments. Specifically, we selected one agency that did not report any improper payment estimates, one agency that reported a few improper payment estimates (for three or fewer programs or activities), and one agency that reported several improper payment estimates (for five or more programs or activities). We also selected one agency that administered eligibility-based programs in fiscal year 2017 because of the unique application and approval processes generally associated with eligibility determinations and their increased risk of improper payments.

We then selected up to two programs or activities at each agency, for a total of five programs. To facilitate our program selection, we requested a listing of all programs and activities at the selected agencies that underwent a risk assessment in fiscal years 2015 through 2017 (the most recent 3-year period at the time of our review) along with the gross outlay amounts associated with these programs and activities. Through our selection process, we noted that the Department of Health and Human Services (HHS) did not assess at least 140 of its programs and activities in the 3-year period from fiscal years 2015 through 2017, and therefore our program selection for HHS was limited to approximately 70 programs or activities.

To select programs, we considered outlay data, the timing of the most recent improper payment risk assessment conducted for each program or activity, and whether eligibility determinations were required for payments under each program or activity. Our selection was primarily based on the size of program and activity gross outlays reported for fiscal year 2017. We focused on outlays because the overall impact of any issues identified with an agency’s risk assessment process may be greater for programs and activities with higher gross outlays, as a higher volume of payments or higher payment amounts could potentially involve higher improper payments. Based on these data, we selected five programs for review. Our findings are limited to the five selected programs and cannot be generalized to all programs and activities at the 24 CFO Act agencies. The agencies and relevant programs selected for review are shown in table 3.

We interviewed officials at the selected agencies on their processes for conducting improper payment risk assessments and reviewed documented policies and procedures. We obtained the most recent improper payment risk assessments that the agencies conducted on the selected programs during the latest 3-year period at the time of our review (fiscal years 2015 through 2017). We then analyzed those risk assessments against relevant IPIA requirements, OMB guidance, and internal control standards to determine whether the agencies had evaluated the appropriate risk factors for improper payments, appropriately considered those risk factors in their risk assessments, and provided a reasonable basis for the risk determination. For any agencies that did not adhere to the improper payment risk assessment requirements, lacked supporting documentation for their risk assessments, or did not provide a reasonable basis for the risk determinations, we interviewed appropriate agency officials to determine the reasons they did not. We also interviewed OMB staff regarding their roles in developing risk assessment guidance.

We conducted this performance audit from December 2017 to January 2019 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Justice

Appendix III: Comments from the Department of Health and Human Services

Appendix IV: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments:

In addition to the contact named above, Matthew Valenta (Assistant Director), Stephanie Adams (Auditor in Charge), Marcia Carlsen, Pat Frey, Gina Hoover, Diana Lee, Zhen Li, and Charles Varga made key contributions to this report.