One of the five government-wide initiatives in the 2001 President’s Management Agenda (PMA) was improved financial management, which targeted improper payments as an area with opportunities for improvement. This initiative called for the administration to establish a baseline on the extent of improper payments. In July 2001, as part of its efforts to advance the PMA initiative, OMB revised Circular No. A-11 by requiring 16 federal agencies to submit data on improper payments, including estimated improper payment rates. Section 831 of the National Defense Authorization Act for Fiscal Year 2002 included the provisions commonly referred to as the Recovery Auditing Act (RAA).required, among other things, that all executive branch agencies entering into contracts with a total value exceeding $500 million in a fiscal year The RAA have cost-effective programs for identifying errors in paying contractors and for recovering amounts erroneously paid. Fiscal year 2011 marked the eighth year of the implementation of IPIA as well as the first year of implementation of IPERA. IPIA required executive agencies to (1) identify programs and activities susceptible to improper payments (typically referred to as risk assessments), (2) estimate the amount of improper payments in susceptible programs and activities, and (3) report these improper payment estimates and actions taken to reduce them. Among other things, IPERA amended IPIA by changing the definition of programs susceptible to significant improper payments and adding minimum risk factors for agencies to consider in identifying such programs. In addition, IPERA generally repealed the RAA and included a new, broader requirement for agencies to conduct recovery audits, where cost effective, for each program and activity with at least $1 million in annual program outlays. This IPERA provision significantly lowered the threshold for required recovery audits and expanded the scope for recovery audits to all programs and activities, including grant and loan programs. IPERA also added new accountability provisions. For example, in its improper payments reporting, an agency is to describe how it ensures that agency managers, programs, and states and localities (where applicable) are held accountable for meeting applicable improper payment reduction targets as well as establishing and maintaining sufficient internal controls to prevent improper payments and promptly detect and recover those improper payments that are made. The following sections describe key provisions of IPIA, the RAA, and IPERA. Under IPIA, executive agencies were required to annually review all programs and activities that they administer and identify any that may be susceptible to significant improper payments. OMB, in its 2006 guidance,to apply to only those programs and activities where the risk level was unknown. For those programs deemed not risk susceptible, risk assessments were required every 3 years. In addition, the guidance defined “significant improper payments”—the threshold at which agencies must perform an estimate for a program—as annual improper payments in the program exceeding both 2.5 percent of program payments and $10 million. interpreted this IPIA requirement for annual risk assessments IPERA changed several of the requirements associated with risk assessments. Specifically, IPERA did the following: Amended IPIA to require agency heads to review agency programs and activities during the year following IPERA’s enactment and at least once every 3 fiscal years thereafter to identify those that may be susceptible to significant improper payments. Defined “significant” in the law for the purpose of determining a program’s susceptibility to significant improper payments. IPERA defined “significant improper payments” as gross annual improper payments (i.e., the total amount of overpayments plus underpayments) in the program that may have exceeded either (1) both 2.5 percent of program outlays and $10 million, or (2) $100 million (regardless of the improper payment percentage of total program outlays). Included the following minimum risk factors likely to contribute to a susceptibility to significant improper payments that agencies are to consider in performing risk assessments: (1) whether a program or activity is new to the agency; (2) the complexity of the program; (3) the volume of payments made through the program or activity; (4) whether payment decisions are made outside of the agency, such as by a state or local government; (5) recent major changes in program funding, authorities, practices, or procedures; (6) the level, experience, and quality of training for personnel responsible for making eligibility determinations or certifying that payments are accurate; and (7) significant deficiencies in the audit report of the agency or other relevant management findings that might hinder accurate payment certification. Under IPIA, for each program or activity identified as susceptible to significant improper payments, the head of each agency was to (1) estimate the annual amount of improper payments and (2) submit those estimates to Congress before March 31 of the following applicable year, with all agencies using the same method of reporting, as determined by the Director of OMB. IPERA revised the IPIA requirements for estimating improper payments by directing agency heads to produce statistically valid estimates of their agencies’ improper payments, or an estimate that is otherwise appropriate using a methodology approved by the Director of OMB, and to include the annual improper payment estimates in their performance and accountability reports or in the agency financial reports. Under IPIA, annual reporting requirements for reducing improper payments included a description of the steps the agency has taken to ensure that agency managers (including the agency head) are held accountable for reducing improper payments. IPERA amended IPIA’s requirements for reporting on corrective actions. Specifically, it required that agencies’ annual reporting include the estimated completion dates of planned corrective actions. Also, it required agencies to report on steps taken to ensure that states and localities, where applicable, are held accountable for reducing improper payments in the federal programs they implement. OMB’s implementing guidance for IPERA also states that for those agency programs not implemented directly by federal or state agencies or governments, agencies may also consider establishing these accountability mechanisms. OMB encouraged agencies to leverage new technologies and techniques to assist them in preventing and reducing improper payments. Agencies implementing long-term, ongoing corrective actions should annually review their existing corrective actions to determine if any existing action can be intensified or expanded, resulting in a high-impact, high return-on-investment in terms of reduced or prevented improper payments. Recovery audits were not required under IPIA, but were required under the RAA. Specifically, agencies were required to carry out a cost-effective program of recovery audits to identify and recover improper payments to contractors, if they entered into contracts with a total value that exceeded $500 million in a fiscal year. IPERA generally repealed the RAA, expanded the scope for recovery audits beyond commercial payments to include all programs and activities, and lowered the threshold of annual outlays requiring agencies to conduct recovery audits—from $500 million in annual agency contracting to $1 million in annual program expenditures. Specifically, under the recovery auditing provisions of IPERA, agencies are required to identify and recover improper payments by conducting recovery audits, also known as payment recapture audits, for agency programs that expend $1 million or more annually, if such audits would be cost effective. In its November 2010 guidance, OMB required agencies to submit payment recapture audit plans by January 14, 2011, that described the agencies’ current payment recapture efforts under authorities that pre- dated IPERA and their planned payment recapture efforts based on the new authorities provided by IPERA. IPIA required, with respect to any program or activity of an agency with estimated improper payments that exceeded $10 million, the head of the agency to provide along with the estimate a report on what actions the agency was taking to reduce the improper payments, including a discussion of the causes of the improper payments identified, actions taken to correct those causes, and results of the actions taken to address those causes; statement of whether the agency had the information systems and other infrastructure it needed in order to reduce improper payments to minimal cost-effective levels; description of the resources the agency had requested in its budget submission to obtain the necessary information systems and infrastructure if the agency did not have such systems and infrastructure; and description of the steps the agency had taken to ensure that agency managers (including the agency head) were held accountable for reducing improper payments. For RAA, OMB guidance required that agencies include in their annual reporting, among other things, a general description and evaluation of the steps taken to carry out a recovery auditing program, the total amount of contracts subject to review, the actual amount of contracts reviewed, the amounts identified for recovery, and the amounts actually recovered in a current year. Further, OMB Circular No. A-136 required agencies to report cumulative amounts identified for recovery and amounts actually recovered as a part of their current year reporting. IPERA requires the reporting of estimates without regard to thresholds. IPERA and OMB guidance require agencies to report, as part of their agency financial reports, certain information regarding the improper payment estimation process and efforts to recover improper payments. These requirements include, among other things, gross estimates of the annual amount of improper payments (i.e., overpayments plus underpayments) made in the program and a description of the methodology used to derive those estimates; discussion of the root causes of the improper payments identified, actions planned or taken to correct those causes, the planned or actual completion date of those actions, and the results of the actions taken; and discussion of the amount of actual improper payments that the agency expects to recover and how these payments will be recovered. According to OMB’s recovery auditing guidance under IPERA,must continue to report information on improper contract payments reviewed, identified, and recaptured, according to instructions contained in OMB Circulars No. A-123 and A-136. In addition, agencies shall report agencies information on other types of recaptured improper contract payments. For instance, where applicable, agencies shall also identify and report information on improper contract payments recovered, if not already included in the annual reporting, including improper contract payments voluntarily returned to agencies by contractors prior to agency or payment recapture auditor identification; improper contract payments identified by the vendors, contractors, or agency staff, and used to provide offsets to future payments rather than returned to agencies; improper contract payments identified and returned through agency Office of Inspector General efforts such as audits, reviews, or tips from the public; improper contract payments identified and recovered through management postpayment reviews other than payment recapture audits; improper contract payments identified and returned or paid through contract closeout; and payment recapture targets and performance in meeting those targets on an annual and quarterly basis. DOD’s improper payment and recovery auditing policies are in two chapters of its Financial Management Regulation (FMR). DOD uses its FMR to govern financial management within the department by establishing the requirements, principles, standards, systems, procedures, and practices necessary to comply with financial management statutory and regulatory requirements applicable to the department. DOD’s FMR chapter on improper payments that was in effect during fiscal year 2011 was issued in 2008, before IPERA was enacted. In October 2011, the department issued a revised chapter on improper payments to implement the requirements of IPERA and associated OMB guidance.during fiscal year 2011, DOD components, which include the military services and defense agencies, are to perform risk assessments, statistically estimate improper payments, identify root causes and develop corrective actions, and report improper payment information annually to According to the FMR chapter on improper payments in effect the OUSD(C). The OUSD(C) is responsible for consolidating component information and preparing department-wide improper payment reports. DOD’s FMR chapter on recovery efforts that was in effect during fiscal year 2011 was issued in 2009, before IPERA was enacted. In October 2012, the department issued its revised FMR chapter on recovery audits to implement the requirements of IPERA and associated OMB guidance. The October 2012 recovery auditing chapter was published after fiscal year 2011 was completed, and we determined that the revisions did not affect the findings in this report. In July 2009, prior to the enactment of IPERA, we reported on DOD’s efforts to address improper payments under IPIA and the recovery auditing requirements under the RAA. In that report, we made 13 recommendations aimed at improving DOD’s efforts to strengthen its improper payment and recovery auditing processes. At that time, DOD did not concur with 12 of our 13 recommendations. However, as discussed in that report, we continued to believe that all 13 recommendations were critical for DOD to enhance its efforts to minimize improper payments and recover those that were made. Figure 1 lists the recommendations from our 2009 report. The department reported improper payment information for the following eight programs in its fiscal year 2011 AFR: Military health benefits are payments made to health care providers for services provided to active duty personnel and their family members, retirees and their family members, and family members of deceased service members through the TRICARE program. Military pay includes active duty pay (Army, Navy, Air Force, and Marine Corps) as well as the reserve components’ pay (Army Reserve, Army National Guard, Navy Reserve, Air Force Reserve, Air National Guard, and Marine Corps Reserve). Civilian pay includes civilian pay accounts from each of the components (Army, Air Force, Navy/Marine Corps, and defense agencies). Military retirement pay includes both payments to military retirees and the family members of deceased retirees (annuitants). Travel pay includes travel payments made through the Defense Travel System (DTS) for the military services and defense agencies as well as additional travel payments made by the Army, Navy, and Air Force for vouchers paid outside of DTS. DFAS commercial pay includes payments made by DFAS on behalf of DOD components to vendors and contractors. USACE travel pay includes travel payments made by USACE to employees. USACE commercial pay includes contract payments made by USACE. Figure 2 shows the total outlays, improper payment totals (sum of the overpayments and the underpayments), and total improper payments as a percentage of total outlays, as reported by DOD in its fiscal year 2011 AFR. The improper payment total shown for DFAS commercial pay was not a statistical estimate, but was limited to known improper payments. As shown in figure 2, DOD identified its programs for improper payment estimation and reporting in such a way that each program represents a category of disbursements made by the department. OMB’s guidance does not specify how agencies are to identify programs for improper payment estimation and reporting, but advises that agencies determine the grouping of programs that most clearly identifies and reports improper payments for their agency. DOD did not adequately implement key provisions of IPIA, IPERA, and OMB guidance related to estimating improper payments, identifying programs susceptible to significant improper payments, reducing improper payments through corrective actions, recovering improper payments, and reporting improper payment estimates and recovery efforts. Most important, we found that DOD’s improper payment estimates were neither reliable nor statistically valid. Also, DOD did not conduct a risk assessment for fiscal year 2011 in accordance with IPERA requirements. Further, although DOD had a corrective action plan for fiscal year 2011, the plan did not identify the underlying reasons or conditions that caused the errors to occur. Additionally, DOD did not conduct recovery audits nor did it determine that such audits would not be cost effective, as required by IPERA. Finally, the department did not have procedures to ensure that improper payment and recovery audit reporting in its fiscal year 2011 AFR was complete, accurate and compliant. DOD’s improper payment estimates reported in its fiscal year 2011 AFR were neither reliable nor statistically valid because of several deficiencies in the department’s procedures as documented in its sampling methodologies. Because of DOD’s long-standing and pervasive financial management weaknesses, the department did not have complete and accurate populations of payments from which to select statistical samples. We also identified deficiencies related to its (1) sampling methodologies and (2) maintenance of key documentation supporting its improper payment estimates. The foundation of reliable statistical sampling estimates is a complete, accurate, and valid population from which to sample. However, the department’s long-standing and pervasive financial management weaknesses precluded it from validating the completeness of its payment transaction populations. For example, DOD’s fiscal year 2011 Statement of Budgetary Resources (SBR) reported nearly $1,017 billion in gross outlays in fiscal year 2011. As previously shown in figure 2, the outlays for the eight programs for which the department reported improper payments totaled $617 billion. DOD attributed most of the difference between the SBR gross outlays and outlays for the eight programs to intragovernmental transactions and trust fund transfers, which IPERA exempted from improper payment estimation and reporting requirements. However, the department was unable to reconcile these two outlay amounts. DOD acknowledged in its fiscal year 2011 AFR that reported outlays for the eight programs could not be reconciled to gross outlays reported in the SBR. As a result, DOD could not ensure that all required outlays for improper payment reporting purposes were included in the sample populations. Although DOD had documented methodologies for developing improper payment estimates, DOD did not establish and perform key quality assurance procedures, such as reconciliations, on its program populations to validate that the populations were complete and accurate before selecting the statistical samples that were used to estimate improper payments. Standards for Internal Control in the Federal Government states that control activities such as reconciliations are an integral part of an entity’s planning, implementing, reviewing, and accountability for stewardship of government resources and achieving effective results. An effective reconciliation process would involve comparing transactions to supporting documentation, systems of record, or both to ensure the completeness, validity, and accuracy of financial information. An effective reconciliation process also involves resolving any discrepancies that may have been discovered and determining if unauthorized changes have occurred to transactions during processing. In addition to the lack of complete, valid, and accurate populations, deficiencies in DOD’s procedures as documented in its sampling methodologies further impaired DOD’s ability to produce reliable improper payment estimates. First, DOD’s sampling methodologies are based on the use of simple random samples to select payments to review for improper payments and thereby derive error rates. Using these methodologies, each transaction in the programs’ sample populations had an equal chance of selection without regard to the complexity of the transaction or its risk of being an improper payment. OMB guidance states that agencies will need to utilize complex sample designs to the extent their payment population contains wide-ranging dollar amounts, types of payments, or both. In addition, DOD did not use a sampling unit that was statistically appropriate for any of its programs. For example, the sampling unit for travel pay for fiscal year 2011 was the travel voucher. Each voucher had an equal chance of selection in the samples upon which improper payment estimates were based. However, DOD’s travel pay transactions range in complexity from an individual soldier’s relocation to payments made on travel vouchers involving multiple travel orders. As another example, DFAS commercial pay’s sampling unit was an individual invoice. As DOD reported in the fiscal year 2012 AFR, a $10 million invoice had the same chance of being sampled as a $100 invoice. Generally, higher dollar payments involve more complex transactions and thus are at greater risk of being an improper payment. If a population contains a few large invoices and many smaller invoices, equal probability sampling is unlikely to capture the large invoices. DOD’s sampling methodologies do not account for this risk. By not designing more complex sampling methodologies that utilize more statistically appropriate sampling units, such as dollars paid, DOD’s improper payment estimates could be significantly understated. Further, DOD provided evidence that it used its sampling methodologies to calculate statistically valid improper payment error rate estimates and related confidence intervals for military pay, civilian pay, and travel pay for fiscal year 2011, but did not provide such evidence for military health benefits, military retirement, USACE commercial pay, or USACE travel pay. Additionally, DOD did not generate statistically valid improper payment dollar value estimates for any of its programs. For instance, DOD did not use appropriate weights to calculate the reported dollar value estimates. Moreover, DOD did not derive confidence intervals for its improper payment dollar value estimates for any of its programs. Generally accepted statistical standards require the calculation and disclosure of confidence intervals around an estimate with a specified degree of confidence. Confidence intervals are a measure of the possible difference between the sample estimate and the actual population value, providing an idea of how close the sample estimate is to the actual population value. As previously mentioned, DOD did not statistically derive an estimate of improper payments for DFAS commercial pay for fiscal year 2011, but instead limited its reporting to known improper payments. Although DOD reported a statistically derived improper payment estimate— $100.1 million—for DFAS commercial pay for fiscal year 2012, the sampling methodology used to produce this estimate had deficiencies similar to the methodologies used for the department’s other programs. For example, to estimate DFAS commercial pay improper payments, DOD did not use a statistically appropriate sampling unit or a methodology that considered large dollar amounts or the level of complexity of the related payments. In its AFR for fiscal year 2012, DOD noted that the department had identified $318.3 million in known improper payments for DFAS commercial pay for fiscal year 2012. Further, DOD cited the sampling methodology as the main reason for the difference between the reported estimate of $100.1 million and the known amount of $318.3 million for DFAS commercial pay improper payments, which provides further evidence of how the deficiencies we identified in the sampling methodologies adversely affect the reliability of the resulting estimates. In our July 2009 report, we recommended that DOD develop and implement a statistically valid methodology to estimate and report commercial improper payments (contract and vendor over- and underpayments). This recommendation remains valid given the issues we have found during the course of this review. DOD did not have procedures in place to collect and maintain key supporting documentation needed to substantiate the improper payment estimates reported in its fiscal year 2011 AFR. For example, DOD did not maintain complete supporting documentation for the populations of transactions, from which statistical samples were selected, for most of the programs for which improper payment estimates were reported. This deficiency contributed to our determination that DOD’s reported improper payment estimates were not reliable. DOD officials stated that they were unaware of the extent of documentation necessary for the department to maintain to support its improper payment estimates. Standards for Internal Control in the Federal Government requires all transactions and other significant events to be clearly documented and the documentation Further, OMB guidance directs readily available for examination. agencies to retain documentation to support the calculation of their estimates. To enable auditors and other parties to substantiate reported improper payment estimates, we determined that the following documentation would generally need to be maintained: Description of the sampling methodology, including identification of the sampling units accompanied by an explanation of how the sampling units were determined. Schedules showing the total number of items and dollar value totals for each sample populationagency by program and month. for each military service and defense Descriptions and results of data reliability and quality assurance testing conducted to ensure that payment information in the sample populations was complete and accurate. Descriptions of how each sample was selected, including the random number and how it was generated, the software used to select the sample from the sample population, and copies of software program logs and related output files. The software program logs and related output files should have the details related to the sample population totals and samples selected, including the total number of items and dollar value totals in the sample population as well as a list of the items selected to be in the sample. In addition, a description of the method of selection of replacement items is needed. Descriptions of attributes and variablestransaction used to derive the total dollar value of improperly paid amounts, including source documents for each sampled transaction that support the conclusion of whether the sampled payment was improper. Calculations, spreadsheets (including cell formulas), and software programs used to evaluate the individual test results for every sample item tested, including the calculations used to derive each improper payment error rate and dollar value estimate and the related confidence intervals. Calculations, spreadsheets (including cell formulas), software programs, inputs and outputs used to aggregate the individual error rate and dollar value estimate and related confidence intervals from each sample to derive the improper payment estimates reported in the AFR. These documents should provide a clear trail showing how the results of each sample were aggregated, from the lowest level of sampling test results through to the estimates published in the AFR. Schedules listing all missing sample items and how these sample items were treated and related explanations. Schedules listing all items replaced and related explanations. The lack of complete supporting documentation precludes DOD and others from being able to determine the reliability of its reported improper payment estimates. DOD did not perform a risk assessment for fiscal year 2011 as required by IPERA, because DOD officials told us that they did not see any added value in doing a risk assessment. According to DOD officials, OMB directed the department to consider all of its programs as risk- susceptible—following its review of the department’s fiscal year 2006 improper payment reporting—because of the complex nature of the department’s business processes and the large dollar value of its annual payments. However, DOD officials were unable to provide documentation of this directive. Moreover, as discussed previously, IPERA laid out a clear statutory requirement to perform a risk assessment in fiscal year 2011, which would supersede an earlier OMB directive. Our executive guide describes characteristics of an effective risk assessment done for the purpose of determining an entity’s susceptibility to improper payments.comprehensive review and analysis of program operations to determine where risks exist and what those risks are, and then measuring the potential or actual impact of those risks on program operations. Once risk areas are identified, their potential impact on programs and activities should be measured and additional controls should be considered. As risks are addressed and controls are changed, the assessment should occasionally be revisited to determine where the risks have decreased and where new areas of risk may exist. A risk assessment is an activity that entails a By not doing a risk assessment for fiscal year 2011, DOD missed the opportunity to gain critical information for determining corrective actions needed to reduce improper payments. Periodic risk assessments are critical to ensuring that the department is identifying the root causes of improper payments and developing appropriate corrective actions. The information developed during a risk assessment forms the foundation or basis upon which management can determine the nature and type of corrective action needed. In addition, this information gives management baseline data for ensuring progress in reducing improper payments. Additionally, performing risk assessments may be more cost beneficial than estimating improper payments for each program. Given the time and resources needed to verify the completeness of populations, select and test samples, and evaluate and project the results of the samples, the department may be able to realize savings by first performing risk assessments. Moreover, if performed in a manner similar to that described in our executive guide, the information gained during the risk assessment may help DOD to determine the best sampling methodology to be used for each program, develop corrective actions, and guide recovery auditing efforts. DOD, FMR, Volume 4, Chapter 14, Improper Payments (October 2011). accordance with IPERA. For example, IPERA requires agencies to perform a risk assessment in the year after enactment (fiscal year 2011) and at least once every 3 years thereafter. However, the FMR chapter states that components are required to conduct risk assessments only for those programs or activities for which the risk level is unknown or is not currently measured and reported. DOD’s FMR does not provide a systematic approach to ensure that all programs and activities are reviewed to determine susceptibility to improper payments. Moreover, although DOD’s FMR states that components’ risk assessment methodologies must be documented and maintained, the FMR does not provide detailed requirements on what should be documented and maintained. As discussed previously, DOD’s lack of a risk assessment makes it difficult for the department to fully identify root causes and develop a comprehensive, effective, corrective action plan. While DOD has a policy for developing and reporting on corrective actions, it did not have detailed procedures for identifying root causes and related corrective actions. Also, the department’s corrective action plan, included in its fiscal year 2011 AFR, did not contain all elements of corrective action plans required by IPERA and OMB guidance, such as establishing accountability for reducing improper payments and including completion dates for implementing corrective actions. DOD’s corrective action plan reported reasons that improper payments occurred for all eight programs and included corrective actions to address them, but the reported reasons identified the type of errors that resulted in the improper payments, rather than the root causes—the underlying conditions that caused the errors to occur. DOD’s identified reasons do not consider possible underlying systemic causes of the errors, such as whether manual and automated controls were either not sufficient or not operating as intended. As a result, the related corrective action(s) addressed specific errors and not necessarily the underlying condition that gave rise to the error. Agencies, when developing corrective action plans, can use the results of risk assessments to ensure that the root causes leading to improper payments are identified. Also, the corrective actions reported were not sufficiently detailed to assess whether they would address the errors that were identified for DOD’s reported programs. For example, in fiscal year 2011, DOD reported that the corrective actions for military pay consisted of working with the military services to advise them of the results of payment reviews and the associated reasons for errors, including the provision of monthly reports on the reasons for individual improper payments and improper payment trends. While these corrective actions provide information on the reasons for improper payments, they do not indicate what, if any, actions the military services would take to address the causes of improper payments. DOD’s corrective action plan also did not describe the required steps for ensuring that responsible officials are held accountable for reducing improper payments, as required by IPERA. In addition, according to the United States Chief Financial Officers Council’s (CFOC) Implementation Guide for OMB Circular A-123, agencies should have procedures for tracking the status of corrective action plans. The implementation guide provides that corrective action plans should include measurable indicators of compliance and resolution for assessing and validating progress throughout the resolution cycle. However, DOD’s corrective action plan did not include (1) a timetable for when the corrective actions were to be implemented and (2) measurable indicators of compliance and resolution, which include follow-up tests to verify whether procedures and controls are working, to assess and validate progress in reducing improper payments. GAO-09-442. those actions to ensure that future improper payments will be reduced or eliminated. However, the FMR does not provide detailed procedures for the components to follow to identify root causes and develop corrective actions, and for the department to follow in monitoring the implementation of those corrective actions. DOD has not yet implemented our recommendations and told us that it was not planning any significant changes to its corrective action processes. Until DOD develops and implements detailed procedures that include the information required by IPERA and OMB guidance and recommended by best practices, DOD will continue to be hindered in its ability to (1) develop corrective action plans that address root causes, (2) effectively monitor and measure the progress made in taking those corrective actions, (3) hold individuals responsible for implementing corrective actions, and (4) communicate to agency leaders and key stakeholders the progress made toward remediating improper payments. We identified multiple deficiencies and omissions in DOD’s efforts to implement IPERA’s recovery audit requirements due to a lack of appropriate procedures. DOD neither conducted recovery audits in fiscal year 2011 nor determined that such audits would not be cost effective, as required by IPERA. Further, most DOD programs did not identify and collect cost information for their recovery efforts that would permit cost- effectiveness evaluations, and the programs that did collect this information did not subsequently evaluate the programs to ensure that they were, in fact, cost effective. We also identified deficiencies and omissions in the payment recapture audit plan that DOD submitted to OMB. DOD did not conduct recovery audits for the eight programs for which it reported improper payments in fiscal year 2011, nor has it determined that such audits would not be cost effective (i.e., that the government would not suffer additional financial losses because of ineffective recovery programs), as required by IPERA, because of outdated policy and a decision to rely on other recovery mechanisms. DOD’s FMR chapter on recovery efforts that was in effect during fiscal year 2011 was issued in 2009, before IPERA was enacted. As a result, the FMR chapter did not account for the expansion of recovery audits beyond commercial payments, as called for by IPERA. In addition, DOD cited its difficulties in tracing transactions back to source documentation as a major obstacle to conducting effective recovery audits. In lieu of conducting cost-effective recovery audits, DOD’s payment recapture audit plan stated that the department would rely on efforts such as random sampling of improper payments, DOD Inspector General (IG) and other auditor findings, self-reporting by recipients, and other activities, such as periodic independent reviews of commercial payments, to identify overpayments for potential recovery for seven programs. However, DOD did not describe any improper payment recovery effort in place for travel pay in the payment recapture audit plan. According to the fiscal year 2011 AFR, DOD estimated $238.2 million in overpayments related to travel pay during fiscal year 2011. Through procedures, including analysis of duplicate payments from fiscal years 2009 and 2010, DOD identified for recovery $1.6 million in travel pay overpayments, or less than 1 percent of the program’s estimated improper payments. DOD officials stated that they believe that a significant portion of the estimated improper overpayment amount was due to missing supporting documentation and did not represent funds owed to the federal government. However, DOD was not able to quantify how much, if any, of the overpayment estimate was due to missing documentation. Because DOD has not established recovery audits to recapture improper overpayments, and has not determined that such mechanisms would not be cost effective, DOD is at risk of forgoing the detection and recovery of potentially substantial funds owed to the government. GAO-09-442. establish the DOD program to implement the requirements of IPERA and associated OMB guidance with respect to recovery audits. The revised FMR directs all components with programs and activities with annual payments that exceed $1 million to determine if instituting recovery audits is cost effective. However, our review of the 2012 FMR chapter indicates that the guidance is still lacking some key elements that would enable DOD components to fully implement IPERA and OMB guidance. For example, the 2012 FMR chapter does not require the components to submit information to the OUSD(C) that OMB directs agencies to report, such as the amount of commercial pay recoveries that were used to offset future payments and the amount of improper overpayments identified through contract closeouts. Consequently, DOD has not yet fully implemented our 2009 recommendation. We found that DOD, with the exception of USACE, did not have procedures to identify and collect information on costs related to its payment recovery efforts. As a result, DOD did not determine if its ongoing recovery efforts, such as periodic independent reviews of commercial payments, were cost effective or if it would be cost effective for the department to establish and implement recovery audits for its programs. Even when DOD did determine the cost of certain improper payment recovery efforts, the department did not ensure that the efforts were cost effective. For example, USACE officials told us that the agency’s daily automated review system, which uses a data mining process to review contract payments for potential errors, costs $64,000 annually. However, USACE reported that as of December 2012, the system had only identified and recovered one improper payment of $20.79 since its implementation in May 2009. A USACE official stated that she believed that the data mining process was mandatory and that USACE was attempting to keep the cost as low as possible. By not assessing the cost- effectiveness of the daily automated review system, DOD is at risk of operating an improper payment recovery effort that is not cost beneficial. As stated previously, in October 2012, DOD issued an updated FMR chapter on recovery audits that directed all components with programs and activities with annual payments that exceed $1 million to determine if instituting recovery audits is cost effective. Further, the FMR chapter directed DOD components to report the total cost of their respective recovery audits and related recovery efforts. However, until the department establishes procedures to consistently identify and collect information regarding costs of recovery audits, DOD will be unable to implement the FMR policy and determine if recovery audits are cost effective to operate. OMB directed agencies to prepare and submit to both OMB and the agency’s IG a payment recapture audit plan by January 2011 that describes payment recapture efforts under both IPERA and authorities that pre-dated IPERA. developed, would help an agency manage its activities to maximize recovery of improper payments. DOD developed and submitted a payment recapture audit plan to OMB and the DOD IG in January 2011. In response to OMB and DOD IG comments, DOD revised and resubmitted its plan in November 2011. However, we found that DOD’s payment recapture plan did not include the following required elements: Payment recapture audit plans, if properly The quantity and dollar amount of payment reviews (except for USACE, which indicated quantity of items reviewed). Types of tools used to review payments (except for USACE, which disclosed an Oracle-programmed data mining tool). When the payments that were reviewed were made. A description of whether the payment recapture audit program focuses on programs or particular steps in a program’s payment process that are at higher risk of fraud, waste, and abuse. A description of the guidance that DOD provides to agency staff related to responsibilities and procedures to implement mechanisms to recover improper payments. Technology being used or planned that would assist in preventing and recapturing improper payments. By not developing and implementing a payment recapture audit plan that contains all elements required by OMB, DOD is not in compliance with OMB requirements and is hindered in its ability to effectively manage its recovery efforts. OMB, Memorandum M-11-04. DOD did not have documented procedures to ensure that improper payment reporting in the AFR was complete, accurate, and in compliance with statutory and regulatory guidance. We identified multiple reporting omissions in DOD’s fiscal year 2011 AFR. For example, the department did not include the following information required by IPERA and OMB guidance: corrective actions that would address the root causes of military health benefits improper payments; the actual or planned completion dates for corrective actions; the portion of the improper payment estimates attributable to insufficient supporting documentation or administrative errors; and whether the agency had the human capital, internal controls, and accountability mechanisms necessary to reduce improper payments. DOD also did not have documented procedures to ensure that recovery audit reporting in the AFR was complete, accurate, and in compliance with statutory and regulatory guidance. We identified instances where DOD’s reporting of efforts to recover improper payments did not include all information required by OMB guidance. For example, DOD did not disclose the following in the fiscal year 2011 AFR: the amount of contract payments that was voluntarily returned to DOD; the amount of improper contract payments identified by contract closeouts; the amount of commercial pay recoveries used to offset future payments rather than returned to DOD; improper payments identified as a result of DOD IG investigations, GAO audits, or reviews by DOD internal review offices, such as TMA’s Program Integrity Office, in its table of overpayments recaptured outside of payment recapture audits; and the total amount of and justification for identified improper overpayments that were determined to be uncollectible in fiscal year 2011. An OUSD(C) official told us that the OUSD(C) does not have standard operating procedures for the compilation, review, and reporting of improper payment and recovery audit information in its AFR. This OUSD(C) official stated that the department uses Appendix C of OMB Circular No. A-123, Requirements for Effective Measurement and Remediation of Improper Payments; OMB Circular No. A-136; and other relevant OMB instructions to prepare the improper payment addendum. However, as evidenced by the omissions in DOD’s improper payment and recovery audit reporting for fiscal year 2011, the department’s current process is not producing reports that comply with IPERA and OMB guidance. Standards for Internal Control in the Federal Government provides that internal control activities need to be clearly documented. documented procedures for the compilation and review of improper payment and recovery audit information, DOD is at risk of continuing to publish incomplete and inaccurate reports. Further, DOD has not yet implemented our 2009 recommendations that the DOD Comptroller perform oversight and monitoring activities to ensure the accuracy and completeness of the improper payment and recovery audit data submitted by DOD components for inclusion in the AFR. During our current review, we identified multiple instances when DOD’s oversight and monitoring of improper payment and recovery audit data submitted by components for inclusion in the AFR did not identify errors or omissions. For example, we identified a calculation error in the military health benefits improper payment estimate. Specifically, TMA used an improper denominator to calculate the improper payment rate for its sample. Instead of dividing the dollar amount of identified improper payments in the sample by the dollar amount paid to providers, which would provide a percentage of improper payments, TMA divided the dollar amount of identified improper payments by the dollar amount billed by providers for the services rendered. As a result, TMA’s improper payment rate of 0.24 percent, as reported in DOD’s fiscal year 2011 AFR, was incorrect. This calculation error was not identified by OUSD(C) personnel during their review of DOD component submissions of data for inclusion in the AFR. GAO/AIMD-00-21.3.1. In addition, OUSD(C) oversight and monitoring did not detect the overstatement of overpayment amounts identified and recovered for military retirement. Upon receiving a notification of death for a retiree or annuitant, DFAS records the entire amount of the last payment made to the retiree or annuitant as an overpayment identified and, after collecting the funds, as an overpayment recovered. However, the identified and recovered amounts recorded did not take into account that the retiree or his/her survivor may have been entitled to receive a portion, if not all, of the payment. As a result, the amounts that DFAS reported to the OUSD(C) as military retirement overpayments identified and recovered were overstated. This is one reason why DOD reported for fiscal year 2011 that the department had identified $67.6 million in military retirement improper overpayments for recovery, while estimating that only $18.8 million in military retirement improper overpayments had occurred. Further, the OUSD(C) oversight and monitoring did not identify omissions in TMA’s submission regarding overpayments identified outside of recovery audits. In its submission to OUSD(C), TMA did not include improper overpayments identified through its Program Integrity Office, which is responsible for prevention, detection, investigation and control of TRICARE fraud, waste, and abuse. As noted above, the OUSD(C) subsequently omitted this information from its fiscal year 2011 AFR reporting. Therefore, our prior recommendations for the DOD Comptroller to perform oversight and monitoring activities to ensure the accuracy and completeness of the improper payment and recovery audit data submitted by DOD components for inclusion in the AFR remain valid given the findings of this review. Although DOD reported estimated and known improper payments of over $1.1 billion for fiscal year 2011, this amount cannot be relied upon because of the deficiencies we found related to DOD’s procedures for identifying, estimating, reducing, recovering, and reporting improper payments. DOD’s long-standing history of pervasive financial management weaknesses, coupled with problematic sampling methodologies and the lack of adequate supporting documentation, contributed to improper payment estimates that were not reliable. Further, DOD has not established the procedures needed to effectively implement the improper payment and recovery auditing requirements included in IPERA and OMB’s implementing guidance. By not performing a risk assessment as required by IPERA, DOD did not reap the associated benefits, including the ability to better identify root causes and develop a comprehensive and effective corrective action plan to reduce improper payments. DOD’s lack of a detailed and effective corrective action plan also made it difficult for department officials to monitor and measure the extent of progress made to remediate causes, hold individuals responsible for implementing corrective actions, or communicate to DOD leadership and other key stakeholders the extent of the department’s progress in remediating the causes of improper payments. In addition, DOD did not comply with the IPERA requirement to either conduct recovery audits or provide justifications that such audits would not be cost effective. Finally, the department’s lack of key required information in its fiscal year 2011 AFR precludes DOD’s leadership and external stakeholders from determining whether DOD has the necessary human capital, internal controls, and accountability mechanisms to reduce improper payments. Until the department takes definitive action to address these deficiencies and thereby fulfills the requirements of IPERA and its implementing guidance, it remains at risk of continuing to make improper payments and wasting taxpayer funds. We recommend that the Secretary of Defense direct the Under Secretary of Defense (Comptroller) to take the following 10 actions: With regard to estimating improper payments: Establish and implement key quality assurance procedures, such as reconciliations, to ensure the completeness and accuracy of the sampled populations. Revise the procedures documented in DOD’s sampling methodologies so that they (1) are in accordance with OMB guidance and generally accepted statistical standards and (2) produce statistically valid improper payment error rates, statistically valid improper payment dollar estimates, and appropriate confidence intervals for both. At a minimum, such procedures should take into account the size and complexity of the transactions being sampled. Develop and implement procedures to collect and maintain the supporting documentation necessary to support improper payment estimates. With regard to identifying programs susceptible to significant improper payments, conduct a risk assessment that is in compliance with IPERA. With regard to reducing improper payments, establish procedures that produce corrective action plans that: Comply fully with IPERA and OMB implementation guidance, including at a minimum, holding individuals responsible for implementing corrective actions and monitoring the status of the corrective actions. Are in accordance with best practices, such as those recommended by the CFOC, and include (1) measuring the progress made toward remediating root causes and (2) communicating to agency leaders and key stakeholders the progress made toward remediating the root causes of improper payments. With regard to implementing recovery audits: Develop and implement procedures to (1) identify costs related to the department’s recovery audits and existing recovery efforts and (2) evaluate existing improper payment recovery efforts to ensure that they are cost effective. Monitor the implementation of the revised FMR chapter on recovery audits to ensure that the components either develop recovery audits or demonstrate that it is not cost effective to do so. Develop and submit to OMB for approval a payment recapture audit plan that fully complies with OMB guidance. With regard to reporting, design and implement procedures to ensure that the department’s annual improper payment and recovery audit reporting is complete, accurate, and in compliance with IPERA and OMB guidance. We provided a draft of this report to the Secretary of Defense for comment. In response, DOD provided written comments, in which it concurred with nine recommendations and partially concurred with one recommendation. In commenting on our report, DOD acknowledged that implementing our recommendations would further strengthen its program. DOD cited its planned actions, including (1) reviewing its sampling methodologies to ensure that they are appropriate and properly documented; (2) developing risk assessments and corrective actions in accordance with IPERA, OMB guidance, and best practices; (3) reviewing its recovery efforts to ensure that they are cost effective; and (4) ensuring that its reporting is complete, accurate, and in compliance with IPERA and OMB guidance. DOD partially concurred with our recommendation to revise the procedures documented in its sampling methodologies so that they (1) are in accordance with OMB guidance and generally accepted statistical standards and (2) produce statistically valid improper payment error rates, statistically valid improper payment dollar estimates, and appropriate confidence intervals for both. The department believes that its sampling methodologies are in accordance with OMB guidance and produce statistically valid improper payment rates and appropriate confidence intervals. However, as discussed in our report, we found that DOD produced statistically valid error rates and related error rate confidence intervals for only three of its programs. Additionally, we found that DOD did not produce statistically valid dollar estimates and appropriate dollar confidence intervals for any of its programs. However, DOD did state that it will review methodologies for all payment types and make modifications as appropriate. DOD also expressed concern that our characterization of the recommended improvements as “significant” does not account for its efforts to minimize improper payments, particularly through prepayment reviews. We acknowledge in our report efforts DOD has made to attempt to minimize improper payments. However, the deficiencies we identified related to DOD’s identifying, estimating, reducing, recovering, and reporting improper payments are significant and indicate that it has not yet established the detailed procedures necessary to effectively implement IPERA and OMB guidance and thus reduce the risk of making improper payments. Accordingly, we continue to believe that implementation of our recommendations is critical for DOD to enhance its efforts to minimize improper payments and to recover those that are made. DOD’s comments are reprinted in appendix II. DOD also provided technical comments on our draft report, which we incorporated as appropriate. As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the appropriate congressional committees, the Secretary of Defense, and other interested parties. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov. If you or your staffs have any questions about this report, please contact Asif A. Khan at (202) 512-9869 or khana@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix III. The objective of this engagement was to review the extent to which the Department of Defense (DOD) has implemented key provisions of the Improper Payments Information Act of 2002 (IPIA), the Improper Payments Elimination and Recovery Act of 2010 (IPERA), and related Office of Management and Budget (OMB) guidance. The scope for our engagement was DOD’s improper payments information presented in the department’s fiscal year 2011 Agency Financial Report (AFR), because this was the most current annual report available at the time of our review. As part of this objective, we assessed DOD’s plans and actions to estimate improper payments for commercial payments made by the Defense Finance and Accounting Service (DFAS) for fiscal year 2012, because DOD limited its improper payment reporting for this program to known improper payments for fiscal year 2011, rather than reporting a statistical estimate. Although DOD reported improper payment information for eight programs, statistical estimates were provided for only seven of those programs for fiscal year 2011. DOD’s improper payments reporting for DFAS commercial pay for fiscal year 2011 was limited to known improper payments. We interviewed officials from DFAS, the TRICARE Management Activity (TMA),obtain additional information about these methodologies. and the U.S. Army Corps of Engineers (USACE) to To assess the department’s plans and actions for estimating DFAS commercial pay improper payments for fiscal year 2012, we reviewed the department’s methodology for statistically estimating DFAS commercial pay improper payments for fiscal year 2012 and interviewed DFAS officials to obtain clarifications about this methodology. We conducted site visits at two DFAS processing center locations— DFAS-Columbus and DFAS-Indianapolis. We selected the DFAS- Columbus site because this facility processes the largest portion of DOD’s commercial payments and hosts the systems the department uses to track commercial pay improper payments. We interviewed DFAS-Columbus officials regarding how improper payments were identified and reported. We selected DFAS-Indianapolis because this facility houses the team that performed the reviews of selected sample transactions for military pay, civilian pay, military retirement for deceased retirees and annuitants, and travel pay. DFAS-Indianapolis compiles the results of the improper payment testing for all DFAS- tested programs, including DFAS commercial pay and military retirement pay, and reports these results to DOD’s OUSD(C). We examined the department’s corrective action plan and assessed it with the requirements in OMB’s implementing guidance for IPERA, OMB Circular No. A-136, and best practices suggested by the United States Chief Financial Officers Council (CFOC). We followed up with DOD officials, including the Improper Payments Project Officer, to obtain additional information about the department’s corrective action plan. We analyzed DOD’s payment recapture plan, DOD’s FMR chapter on and information in the AFR. We interviewed DFAS, recovery audits,TMA, and USACE officials, as well as the Improper Payments Project Officer, about the processes used to recover improper payments. To assess DOD’s implementation of the reporting requirements in IPERA and OMB’s guidance, we compared the improper payment information provided in DOD’s fiscal year 2011 AFR to the reporting requirements. We interviewed OUSD(C), DFAS, TMA, and USACE officials about the department’s process to compile the information reported in the AFR. To assess the reliability of data reported in DOD’s fiscal year 2011 AFR related to improper payments, we reviewed DOD’s supporting documentation and interviewed knowledgeable agency officials about the data. In the course of this assessment, we determined that DOD did not collect and maintain the supporting documentation necessary to substantiate the improper payment estimates reported in its fiscal year 2011 AFR. In addition, the department did not perform key quality assurance procedures, including reconciliations on all of the populations for its programs to validate that the populations were complete, valid, and accurate before selecting the statistical samples that were used to estimate improper payments. Therefore, we determined that the data were not reliable. These problems are discussed in our report. We conducted this performance audit from October 2011 to May 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. In addition to the contact named above, Cindy Brown Barnes (Assistant Director), Sharon Byrd (Assistant Director/Audit Sampling), Michael Bingham, and Sandra Silzer made key contributions to this report. Also contributing to this report were Francine DelVecchio, Justin Fisher, Wilfred Holloway, and Jason Kirwan.

Summary: DOD reported $1.1 billion in improper payments for fiscal year 2011, which marked the eighth year of implementation of IPIA, as well as the first year of implementation of IPERA. IPIA required executive branch agencies to annually identify programs and activities susceptible to significant improper payments, estimate the amount of improper payments for such programs and activities, and report these estimates along with actions taken to reduce them. IPERA amended IPIA and expanded requirements for recovering overpayments across a broad range of federal programs. GAO was asked to review the progress DOD has made to identify, estimate, and reduce improper payments. GAO's objective was to review the extent to which DOD has implemented key provisions of IPIA, IPERA, and OMB guidance. GAO reviewed improper payment requirements; analyzed agency financial reports, internal guidance and plans, and sampling methodologies; and interviewed cognizant officials. The scope for this engagement was DOD's reported improper payment information for fiscal year 2011 and DOD's plans and actions to estimate commercial pay improper payments for fiscal year 2012. The Department of Defense (DOD) did not adequately implement key provisions of the Improper Payments Information Act of 2002 (IPIA) and the Improper Payments Elimination and Recovery Act of 2010 (IPERA) and Office of Management and Budget (OMB) requirements for fiscal year 2011. Most important, GAO found that DOD's improper payment estimates reported in its fiscal year 2011 Agency Financial Report were neither reliable nor statistically valid because of long-standing and pervasive financial management weaknesses and significant deficiencies in the department's procedures to estimate improper payments. For example, DOD did not have key quality assurance procedures in place, such as reconciliations, to validate the completeness and accuracy of the populations used to estimate improper payments; develop appropriate sampling methodologies for estimating improper payments; maintain key documentation supporting its reported improper payment estimates. Also, GAO found significant deficiencies in DOD's policies and procedures to address other key improper payment requirements for fiscal year 2011. Specifically, DOD did not have procedures to identify root causes of improper payments and develop related corrective actions, conduct recovery audits for any of its programs or determine that these audits would not be cost effective, and have procedures to ensure that its annual improper payment and recovery audit reporting is complete, accurate, and in compliance with IPERA and OMB reporting requirements. DOD has taken some actions since fiscal year 2011, such as reporting a statistical estimate for Defense Finance and Accounting Service commercial pay and issuing revised Financial Management Regulation chapters on improper payments and recovery audits. However, until the department takes action to correct the deficiencies GAO found related to identifying, estimating, reducing, recovering, and reporting improper payments and thereby fulfills legislative requirements and implements related guidance, it remains at risk of continuing to make improper payments and wasting taxpayer funds. GAO is making 10 recommendations to improve DOD's processes to identify, estimate, reduce, recover, and report on improper payments. DOD concurred with 9 and partially concurred with 1 of the recommendations and described its plans to address them.