Executive branch agencies are required to take various steps regarding improper payments under IPIA, as amended by IPERA and IPERIA, and related OMB guidance. The steps include the following: 1. reviewing all programs and activities and identifying those that may be susceptible to significant improper payments (commonly referred to as a risk assessment), 2. developing improper payment estimates for those programs and activities that the agency identified as being susceptible to significant improper payments, 3. analyzing the root causes of improper payments and developing corrective actions to reduce them for those programs and activities that the agency identified as being susceptible to significant improper payments, and 4. reporting on the results of addressing the foregoing requirements. Figure 1 lays out these steps, as well as the major components of developing an improper payment estimate. IPERA also directs executive branch agencies’ inspectors general to annually determine and report on whether their respective agencies complied with six criteria listed in the law. On an annual basis, agencies are required to develop improper payment estimates for programs that they consider susceptible to significant improper payments. This generally involves selecting a sample of program payments (or other items, such as invoices) and reviewing them in order to determine whether the relevant payments were proper. OMB guidance for developing improper payment estimates focuses on the statistical nature of the estimates and provides agencies with flexibility in developing their estimates. IPIA, as amended, provides the definition of “improper payment” with IPERIA further instructing OMB to issue guidance requiring agencies to include in the estimate all improper payments, regardless of whether those payments have been or are being recovered. OMB incorporated this requirement into Appendix C to Circular No. A-123, Requirements for Effective Estimation and Remediation of Improper Payments. In accordance with these relevant laws and OMB guidance, agencies must apply “improper payment” in the context of their programs when developing improper payment estimates. The 10 programs we reviewed serve a variety of purposes and are administered by various agencies across the federal government. Table 2 summarizes each of these programs. IPIA, as amended, requires agencies to develop statistically valid improper payment estimates or estimates that are otherwise appropriate using a methodology approved by the Director of OMB. The six agencies we reviewed reported using either statistically valid or alternative sampling approaches for the 10 selected programs, and some agencies reported additionally incorporating actual improper payment amounts into their estimates, as shown in table 3. If an agency is unable to produce a statistically valid improper payment estimate, it can use an alternative approach if approved by OMB. For example, the Department of Education (Education) reported using an alternative methodology for the Direct Loan program after conducting a cost-benefit analysis comparing use of a statistical and an alternative methodology. Similarly, the Department of Health and Human Services (HHS) reported using an alternative methodology for Medicaid to better manage resources needed to conduct the required reviews. In addition to their statistical approaches, two agencies reported incorporating actual improper payment amounts into the estimates for 2 of the programs we reviewed. Officials at the Department of Defense (DOD) stated that the agency calculates its Military Pay improper payment estimate by adding the amount of debts due to DOD entered into its financial system based on overpayments (i.e., debts due to DOD by a recipient of an overpayment) identified during the fiscal year to a projected estimate of improper payments. Officials at the Office of Personnel Management (OPM) stated that the agency calculates its Retirement program improper payment estimate by adding the amount of debts due to OPM entered into its financial system based on overpayments (i.e., debts due to OPM by a recipient of an overpayment) identified during the fiscal year to a projected estimate of underpayments. To implement their sampling approaches, agencies select a sample of data to test from a larger, specified population of data. For the six agencies we reviewed, data sampled varied by program and include payments, claims, tax returns, and pay accounts. For example, according to their policies and procedures DOD samples invoices related to payments made from 12 financial systems for Defense Finance and Accounting Service Commercial Pay, HHS samples medical claims for Medicare Fee-for-Service, and DOD samples pay accounts for Military Pay. Agencies subject specific data populations to sampling, which may not include all payments made for a program. Reasons for sampling exclusions varied across programs, as shown by the examples in table 4. Some of the selected agencies reported sampling multiple sets of data. For example, for its Direct Loan improper payment estimate, Education officials stated that the agency reviews Program Review Reports to identify improper payments in originations and also samples loan consolidation and refund payments. According to agency officials, Direct Loan origination, consolidation, and refund transactions carry different risks of improper payment. To estimate improper payments for fiscal year 2017, the six agencies we reviewed reported sampling and testing data that varied in age from calendar year 2013 to fiscal year 2017. Figure 2 shows the range of data used. OMB guidance states that to the extent possible, data used for estimating improper payments should coincide with the fiscal year being reported, but agencies may use a different 12-month reporting period with approval from OMB. OMB staff acknowledged there are costs and benefits to sampling newer or older data. OMB staff stated that although they review agencies’ sampling and estimation plans, they defer to the agencies regarding the appropriateness of the age of data used to estimate improper payments. OMB staff stated that they approve the timeframe of the data used in alternative methodologies as part of the approval of the methodology overall, whereas OMB silence provides tacit approval (i.e., no communication to the agency) for statistically valid methodologies. After agencies determine what subsets of data and types of transactions to review, they generally test the data and calculate their improper payment estimates. Testing processes varied among the 10 programs, with some of the six agencies using processes designed specifically to estimate improper payments and others leveraging existing processes designed for other purposes. Some of the selected agencies reported using multiple testing processes and combining the results to develop a program’s improper payment estimate. For example, according to their policies and procedures the Direct Loan estimate comprises three component estimates for loan originations, consolidations, and refunds and the Medicaid estimate includes fee-for-service, managed care, and eligibility components. Table 5 summarizes the processes used by the six agencies we reviewed. Although agencies’ testing processes varied, most included steps to address aspects of eligibility of beneficiaries, goods, or services—a key component of determining the appropriateness of a payment—in their programs. For example, according to their policies and procedures for Medicare Fee-for-Service, reviewers examine the medical necessity, compliance with documentation requirements, and coding of services provided, among other things; for the Earned Income Tax Credit (EITC), auditors examine whether the taxpayer properly reported income and whether the taxpayer meets eligibility criteria, including income and qualifying child requirements, and auditors examine, among other things, whether the taxpayer is subject to a disallowance period on receiving EITC; for Medicaid, reviewers examine fee-for-service claims and managed care payments to determine the eligibility status of the beneficiary and the provider, as well as support for the medical necessity of fee-for- service claims, among other things; for Old-Age, Survivors, and Disability Insurance (OASDI), reviewers examine factors to support the beneficiary’s eligibility, including, among other things, citizenship, relationship (in the case of survivor benefits), and receipt of other government benefits; and although Education’s Direct Loan program reviews can vary in scope, they may include, among other things, steps to verify educational institution eligibility (such as licensing and accreditation) and student eligibility (such as enrollment status and satisfactory academic progress). In contrast, per their policies and procedures, eligibility is not tested for DOD’s Military Pay or the overpayment component of OPM’s Retirement estimate. DOD Military Pay. DOD reported using the results of monthly payment reviews to calculate a projected improper payment amount for Military Pay. However, DOD’s policies and procedures do not require a review of servicemember eligibility for special pay or allowances as part of these monthly reviews. DOD’s Standard Operating Procedures (SOP) direct reviewers to recalculate payments to servicemembers solely based on the pay account data included in DOD systems (i.e., to verify that components of servicemember pay were calculated appropriately). DOD’s SOP does not direct reviewers to verify that servicemembers were eligible for special pay or allowances by verifying the information included in the pay account (such as pay grade) with supporting documentation. According to DOD officials, reviewers may investigate potential inconsistencies in pay account data identified during their reviews—which may include eligibility issues—but this process is not consistently performed or documented. According to DOD officials, an example of a potential inconsistency is when a servicemember receives jump pay (a hazard pay for parachute jumps) but is located at a site where no jump activity occurred. According to DOD officials, to help compensate for the limitations of its monthly reviews, DOD calculates the final reported Military Pay improper payment estimate by adding actual debts due to DOD (related to overpayments) identified during the year to the projected estimate of the monthly reviews. DOD identifies the actual overpayments through various methods, including other postpayment reviews and servicemember self- reporting. Standards for Internal Control in the Federal Government states that management should identify, analyze, and respond to risks related to achieving the defined objectives. DOD has acknowledged internal control deficiencies related to the Military Pay program, which—if addressed in improper payment testing—could have an impact on the program’s improper payment rate. However, these deficiencies were identified through other internal control reviews not related to estimating improper payments. For the purposes of estimating improper payments, DOD has not fully assessed the risks in its Military Pay program and evaluated whether its approach for estimating improper payments effectively addresses these risks. As a result, DOD’s process for estimating Military Pay improper payments may not reflect significant risks of improper payment in the program, specifically whether servicemembers are eligible for the special pay or allowances they receive, calling into question the improper payment estimate and its usefulness for developing effective corrective actions. OPM Retirement. OPM relies on its existing Quality Assurance (QA) process to estimate Retirement underpayments. The QA process is designed to determine whether new Retirement claims (i.e., claims paid for the first time) have been adjudicated correctly. Therefore, only new Retirement claims are sampled and tested for accuracy. OPM applies historical results of QA testing to older claims; however, these historical results do not reflect any different risks of underpayment that the older claims may face. Although OPM’s QA process also produces an estimate of overpayments, the agency’s policies and procedures instead use actual debts due to OPM (related to overpayments) that were identified during the fiscal year as its overpayment amount (i.e., the overpayment amount does not reflect any testing of Retirement payments to verify eligibility or accuracy). These actual overpayments represent amounts that have been identified through various means, such as inspector general fraud referrals. OPM officials stated that the agency uses actual amounts because the QA estimate may overstate overpayments. However, the fiscal year 2016 QA overpayment estimate was lower than the actual amount of debts identified as due to OPM. Standards for Internal Control in the Federal Government states that management should identify, analyze, and respond to risks related to achieving the defined objectives. OPM has not fully assessed the risks of improper payments in its Retirement program—particularly related to the risk of underpayments in older claims and the risk of overpayments— and evaluated whether its approach for estimating improper payments effectively addresses these risks. As a result, OPM’s processes for estimating Retirement improper payments may not reflect significant risks of improper payment in the program, calling into question the improper payment estimate and its usefulness for developing effective corrective actions. OMB guidance. OMB issues guidance for agencies to implement various requirements of the improper payment laws. Specifically, OMB is required by IPERIA to issue guidance to set standards for agencies to follow in determining the underlying validity of sampled payments to ensure that amounts being billed, paid, or obligated for payment are proper. Although existing OMB guidance addresses requirements for sampling, it does not address how agencies test to identify improper payments, such as using a risk-based approach to help ensure that key risks of improper payments, like eligibility, are addressed through testing processes. Without such guidance, there is increased risk that agencies’ processes may not address key risks of improper payments in their programs—for example, the cases of DOD Military Pay and OPM Retirement described above—calling into question the improper payment estimates for such programs and their usefulness for developing effective corrective actions. According to OMB guidance, when an agency’s review is unable to determine whether a payment was proper because of insufficient or lack of documentation, the payment must be considered an improper payment. Among the six agencies and 10 programs we reviewed, treatment of insufficient documentation varied by program, as did the classification of these issues for root cause reporting in the AFRs. HHS’s programs were the only ones we reviewed that reported improper payments in the insufficient documentation root cause category for fiscal years 2016 or 2017, as shown in table 6. Some agencies stated that they report insufficient documentation in other root cause categories that they consider more appropriate. For example, Education officials stated that for the Direct Loan program, payments that lack sufficient supporting documentation may be placed in the “Administrative or Process Error Made by Other Party” root cause category. In these cases, a third party—such as a loan servicer—is unable to provide sufficient documentation supporting that the sampled payment was proper. OMB guidance states that in cases where the agency believes that more than one root cause category might be suitable, the agency should determine which category it believes to be the most appropriate. Additionally, some agencies stated that the “insufficient documentation” category was not always relevant when they recreated sampled cases to estimate a program’s improper payments. For example, according to officials, to complete an OASDI stewardship review of a sampled case, a Social Security Administration (SSA) quality reviewer reviews the documentation related to the original determination and then independently re-develops all factors of the payment and interviews the associated beneficiary. According to agency officials, insufficient documentation would not apply as all improper payments identified in the stewardship sample are supported by documentation and payment has been verified in all reviewed cases. As noted previously, the processes for estimating DOD Military Pay and OPM Retirement improper payments were limited, and these limitations may have an impact on the agencies’ ability to identify improper payments related to insufficient documentation. Treatment of cases of nonresponse. Some agencies contact outside entities—such as payees or beneficiaries—as part of their improper payment testing processes. Among the six agencies we reviewed, treatment of cases of nonresponse differed. For example: SSA officials stated that in cases where quality reviewers do not receive responses from OASDI beneficiaries they contact, they exclude the cases from review (unless the reviewer identifies an improper payment in the initial review that is completed prior to reaching out to the beneficiary). For EITC improper payment estimation purposes, the Internal Revenue Service (IRS) stated that the agency does not consider the sampled payment associated with a nonresponse case to be proper or improper. It sets the sampling weight of nonresponse cases to zero and adjusts the sampling weights of respondents upward to account for the nonresponse cases. IRS’s methodology assumes nonresponse and response cases have an equal likelihood of improper payment. For Medicare Fee-for-Service and Medicaid, HHS’s policies and procedures consider payments associated with nonresponse cases to be improper. OMB guidance states that when an agency’s review is unable to discern whether a payment was proper as a result of insufficient or lack of documentation, this payment must be considered an improper payment. However, it does not specifically address the appropriate treatment of nonresponse cases for improper payment estimation purposes. As a result, without clearer guidance there is increased risk that agencies’ improper payment estimates may be understated and that estimates for similar programs may not be comparable. When agencies identify improper payments, they must determine the amount of the payment that was improperly made. The six agencies we reviewed generally reported using the definition of improper payment in relevant laws and OMB guidance to determine the amount of improper payments identified. OMB guidance provides agencies with instructions on how to calculate the amount of improper payments. However, when developing its improper payment estimate for EITC, IRS subtracted overpayments that were paid out and later recovered. By subtracting recovered overpayments, IRS excluded them from the EITC improper payment estimate. For 2013—the tax year used to produce the fiscal year 2017 improper payment estimate—IRS estimated that $1.2 billion in EITC overpayments would be recovered. IPERIA directed OMB to provide guidance that requires agencies to include all improper payments in their improper payment estimates, regardless of whether they have been or are being recovered. Although the OMB guidance was revised in October 2014 to implement this requirement, IRS has not updated its estimation methodology for EITC. By not updating its guidance and continuing to remove EITC overpayments that may be subsequently recovered, IRS is understating its improper payment estimate and potentially limits its ability to address these types of improper payments before they occur. Improper payments are a long-standing, significant problem in the federal government. Estimation of improper payments is key to understanding the extent of the problem and to developing effective corrective actions to address it. Among the six agencies we reviewed, processes to estimate improper payments in their programs varied, and certain differences in these processes may affect the quality of the resulting estimates and consequently these agencies’ efforts to reduce improper payments. Specifically, policies and procedures for DOD’s Military Pay and OPM’s Retirement programs’ improper payment estimation methodologies do not address certain key risks, like eligibility, in part because these agencies have not fully assessed their processes. Further, although OMB guidance addresses requirements for sampling, it does not address how agencies test to identify improper payments. Without such assessments and guidance, there is increased risk that agencies’ processes may not address key risks of improper payments in their programs, calling into question the improper payment estimates for such programs and their usefulness for developing effective corrective actions. Additionally, for agencies we reviewed that contact outside entities as part of their improper payment estimation processes, the treatment of cases of nonresponse varied. OMB guidance does not specifically address the appropriate treatment of nonresponse cases for improper payment estimation purposes. Without clearer guidance there is increased risk that agencies’ improper payment estimates may be understated and that estimates for similar programs may not be comparable. Finally, although IPERIA directed OMB to provide guidance that requires agencies to include all improper payments in their improper payment estimates, regardless of whether they have been or are being recovered, IRS has not updated its processes to reflect the change. By not updating its guidance and continuing to remove EITC overpayments that may be subsequently recovered, IRS is understating its improper payment estimate and potentially limits its ability to address these types of improper payments before they occur. We are making two recommendations to the Director of OMB that have government-wide implications and specific recommendations to DOD, OPM, and IRS regarding their programs included in this review. The Director of OMB should develop guidance on how agencies test to identify improper payments, such as using a risk-based approach to help ensure that key risks of improper payments, such as eligibility, are addressed through testing processes. (Recommendation 1) The Director of OMB should develop guidance clarifying the appropriate treatment of nonresponse cases during improper payment testing. (Recommendation 2) The Under Secretary of Defense (Comptroller) should assess the processes for estimating Military Pay improper payments to determine whether they effectively address key risks of improper payments— including eligibility for different types of pay and allowances—and take steps to update the processes to incorporate key risks that are not currently addressed. (Recommendation 3) The Director of OPM should assess the processes to estimate Retirement improper payments to determine whether they effectively address key risks of improper payments—including eligibility and whether older claims face different risks of improper payments than new claims—and take steps to update the processes to incorporate key risks that are not currently addressed. (Recommendation 4) The Commissioner of IRS should update IRS’s improper payment estimation methodology to not exclude recovered overpayments from its EITC improper payment estimate. (Recommendation 5) We provided a draft of this report for comment to OMB, DOD, Education, HHS, Treasury, OPM, SSA, and USDA. OMB provided oral comments, which are summarized below. OPM, DOD, and IRS provided written comments, which are reproduced in appendixes II through IV, respectively. Education, HHS, SSA, and USDA did not provide written comments on the draft report. In addition, HHS, IRS, OMB, OPM, and SSA provided technical comments, which we have incorporated, as appropriate. In oral comments provided on April 30, 2018, a Senior Policy Advisor in OMB’s Office of Federal Financial Management stated that OMB partially agreed with our first recommendation and agreed with our second recommendation. Regarding the first recommendation, the Senior Policy Advisor stated that OMB should not have to develop more specific guidance as each program and activity has its own risks. Instead, inspectors general are better equipped and positioned to review the sampling and estimation plans as part of their annual IPERA compliance audits and that agencies, their statisticians, and inspectors general should work out the best testing procedures for their agencies. OMB could provide suggestions during OMB’s annual town hall meeting related to improper payments for areas that inspectors general may consider. Although we agree that programs and activities may face different risks of improper payment, we continue to believe that guidance from OMB on how agencies test to identify improper payments—such as directing agencies to take a risk-based approach in developing their testing procedures—could help ensure that agencies address the specific risks they identify when developing improper payment estimates. Further, such guidance could also help ensure that testing processes are designed to address an agency’s identified risks before the estimate is developed, whereas an inspector general’s review—as well as related recommendations for improvement—would generally occur after the agency’s improper payment estimate had been developed and reported. Regarding the second recommendation, the Senior Policy Advisor noted that OMB plans to update its guidance to direct agencies to treat nonresponse cases as improper payments and to include a new category for tracking such cases. In its written comments, OPM partially concurred with our recommendation to assess the processes to estimate Retirement improper payments to determine whether they effectively address the key risks of improper payments. OPM agreed to conduct an audit of older claims to determine if they face different risks than new claims. However, OPM did not agree with the part of the recommendation to assess the risk of improper payments related to eligibility in the estimation process. OPM stated that eligibility is determined before annuity or survivor benefits are fully adjudicated. However, the objective of an improper payment estimate is to determine whether payments were made properly. To do so, an agency should determine whether the payee was eligible for the payment that was made, among other things. As such, we continue to believe that the recommendation—including the assessment of the risk of improper payments related to eligibility—is warranted. In their written comments, DOD and IRS both agreed with our recommendations directed to them and described the steps they plan to take to implement them. We are sending copies of this report to the appropriate congressional committees; the Secretaries of Agriculture, Defense, Education, Health and Human Services, and the Treasury; the Director of the Office of Personnel Management; the Administrator of the Social Security Administration; the Director of the Office of Management and Budget; and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-2623 or davisbh@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix V. Table 7 lists the fiscal year 2017 improper payment estimates by agency and program, as reported by agencies in their fiscal year 2017 agency financial reports and compiled on the Office of Management and Budget’s payment integrity website, paymentaccuracy.gov. In addition to the contact named above, Phillip McIntyre (Assistant Director), James M. Healy (Auditor in Charge), Daniel Flavin, and Fabiola Torres made key contributions to this report.

Answer: Improper payments—which include payments that should not have been made or were made in an incorrect amount—are a long-standing, significant problem in the federal government, estimated at almost $141 billion for fiscal year 2017. Executive branch agencies are required to annually estimate improper payments for certain programs. Estimation of improper payments is key to understanding the extent of the problem and to developing effective corrective actions. Relevant laws and guidance provide agencies flexibility in developing estimates. This report describes agencies' processes to estimate improper payments in selected programs for fiscal year 2017 and the extent to which certain differences in these processes can affect the usefulness of the resulting estimates. GAO selected 10 programs across six agencies with the largest reported program outlays in fiscal years 2015 and 2016. For these programs, GAO reviewed relevant laws and guidance, analyzed agencies' policies and procedures, and interviewed officials at relevant agencies and OMB staff. The six agencies GAO reviewed reported taking various approaches related to key components of estimating improper payments—shown in the figure below—for 10 selected programs, which collectively reported outlays of over $2.5 trillion for fiscal year 2017. Sample selection. Eight of the 10 programs GAO reviewed reported using statistically valid approaches, and the remaining 2 reported using alternative methodologies approved by the Office of Management and Budget (OMB). The sampled data elements varied, including payments, medical claims, and tax returns. The age of the data used to develop fiscal year 2017 improper payment estimates also varied, ranging from calendar year 2013 to fiscal year 2017. Identification of improper payments. Some of the six agencies reported using processes designed specifically to estimate improper payments, whereas others reported leveraging existing reviews. These agencies' policies and procedures include a review of aspects of eligibility, except for those related to the Department of Defense's (DOD) Military Pay and the Office of Personnel Management's (OPM) Retirement overpayments. DOD and OPM have not fully assessed whether their estimation processes effectively consider key program risks. OMB guidance does not specifically address how agencies are to test to identify improper payments, such as using a risk-based approach to help ensure that key risks of improper payments are addressed. The six agencies also varied in the treatment of insufficient documentation, both in identifying and in reporting the root causes of improper payments. For the agencies that contact entities outside the agency to estimate improper payments, the treatment of nonresponse differed, with one agency including nonresponses as improper payments and another generally excluding the nonresponse cases from review. Although OMB guidance states that agencies should treat cases of insufficient documentation as improper payments, it does not specifically address the treatment of nonresponse cases. Calculation of the improper payment estimate. The six agencies generally reported using law and OMB guidance to calculate improper payment estimates for the selected programs, except for the Earned Income Tax Credit (EITC). The Internal Revenue Service (IRS) removed overpayments that were recovered when developing its estimate. OMB guidance requires agencies to include recovered amounts in their estimates. Removing these overpayments understates the EITC improper payment estimate and may limit IRS's ability to develop corrective actions to prevent improper payments. GAO recommends that OMB develop guidance on treatment of nonresponse cases and testing to identify improper payments, that DOD and OPM assess their estimation processes, and that IRS revise its methodology to not exclude recovered payments from its estimate. All of the agencies either agreed or partially agreed with the specific recommendations to them. GAO believes that the actions are warranted, as discussed in the report.