You are given two reports by the government agency.  

Report 1:
Executive branch agencies are required to take various steps regarding improper payments under IPIA, as amended by IPERA and IPERIA, and related OMB guidance. The steps include the following: 1. reviewing all programs and activities and identifying those that may be susceptible to significant improper payments (commonly referred to as a risk assessment), 2. developing improper payment estimates for those programs and activities that the agency identified as being susceptible to significant improper payments, 3. analyzing the root causes of improper payments and developing corrective actions to reduce them for those programs and activities that the agency identified as being susceptible to significant improper payments, and 4. reporting on the results of addressing the foregoing requirements. Figure 1 lays out these steps, as well as the major components of developing an improper payment estimate. IPERA also directs executive branch agencies’ inspectors general to annually determine and report on whether their respective agencies complied with six criteria listed in the law. On an annual basis, agencies are required to develop improper payment estimates for programs that they consider susceptible to significant improper payments. This generally involves selecting a sample of program payments (or other items, such as invoices) and reviewing them in order to determine whether the relevant payments were proper. OMB guidance for developing improper payment estimates focuses on the statistical nature of the estimates and provides agencies with flexibility in developing their estimates. IPIA, as amended, provides the definition of “improper payment” with IPERIA further instructing OMB to issue guidance requiring agencies to include in the estimate all improper payments, regardless of whether those payments have been or are being recovered. OMB incorporated this requirement into Appendix C to Circular No. A-123, Requirements for Effective Estimation and Remediation of Improper Payments. In accordance with these relevant laws and OMB guidance, agencies must apply “improper payment” in the context of their programs when developing improper payment estimates. The 10 programs we reviewed serve a variety of purposes and are administered by various agencies across the federal government. Table 2 summarizes each of these programs. IPIA, as amended, requires agencies to develop statistically valid improper payment estimates or estimates that are otherwise appropriate using a methodology approved by the Director of OMB. The six agencies we reviewed reported using either statistically valid or alternative sampling approaches for the 10 selected programs, and some agencies reported additionally incorporating actual improper payment amounts into their estimates, as shown in table 3. If an agency is unable to produce a statistically valid improper payment estimate, it can use an alternative approach if approved by OMB. For example, the Department of Education (Education) reported using an alternative methodology for the Direct Loan program after conducting a cost-benefit analysis comparing use of a statistical and an alternative methodology. Similarly, the Department of Health and Human Services (HHS) reported using an alternative methodology for Medicaid to better manage resources needed to conduct the required reviews. In addition to their statistical approaches, two agencies reported incorporating actual improper payment amounts into the estimates for 2 of the programs we reviewed. Officials at the Department of Defense (DOD) stated that the agency calculates its Military Pay improper payment estimate by adding the amount of debts due to DOD entered into its financial system based on overpayments (i.e., debts due to DOD by a recipient of an overpayment) identified during the fiscal year to a projected estimate of improper payments. Officials at the Office of Personnel Management (OPM) stated that the agency calculates its Retirement program improper payment estimate by adding the amount of debts due to OPM entered into its financial system based on overpayments (i.e., debts due to OPM by a recipient of an overpayment) identified during the fiscal year to a projected estimate of underpayments. To implement their sampling approaches, agencies select a sample of data to test from a larger, specified population of data. For the six agencies we reviewed, data sampled varied by program and include payments, claims, tax returns, and pay accounts. For example, according to their policies and procedures DOD samples invoices related to payments made from 12 financial systems for Defense Finance and Accounting Service Commercial Pay, HHS samples medical claims for Medicare Fee-for-Service, and DOD samples pay accounts for Military Pay. Agencies subject specific data populations to sampling, which may not include all payments made for a program. Reasons for sampling exclusions varied across programs, as shown by the examples in table 4. Some of the selected agencies reported sampling multiple sets of data. For example, for its Direct Loan improper payment estimate, Education officials stated that the agency reviews Program Review Reports to identify improper payments in originations and also samples loan consolidation and refund payments. According to agency officials, Direct Loan origination, consolidation, and refund transactions carry different risks of improper payment. To estimate improper payments for fiscal year 2017, the six agencies we reviewed reported sampling and testing data that varied in age from calendar year 2013 to fiscal year 2017. Figure 2 shows the range of data used. OMB guidance states that to the extent possible, data used for estimating improper payments should coincide with the fiscal year being reported, but agencies may use a different 12-month reporting period with approval from OMB. OMB staff acknowledged there are costs and benefits to sampling newer or older data. OMB staff stated that although they review agencies’ sampling and estimation plans, they defer to the agencies regarding the appropriateness of the age of data used to estimate improper payments. OMB staff stated that they approve the timeframe of the data used in alternative methodologies as part of the approval of the methodology overall, whereas OMB silence provides tacit approval (i.e., no communication to the agency) for statistically valid methodologies. After agencies determine what subsets of data and types of transactions to review, they generally test the data and calculate their improper payment estimates. Testing processes varied among the 10 programs, with some of the six agencies using processes designed specifically to estimate improper payments and others leveraging existing processes designed for other purposes. Some of the selected agencies reported using multiple testing processes and combining the results to develop a program’s improper payment estimate. For example, according to their policies and procedures the Direct Loan estimate comprises three component estimates for loan originations, consolidations, and refunds and the Medicaid estimate includes fee-for-service, managed care, and eligibility components. Table 5 summarizes the processes used by the six agencies we reviewed. Although agencies’ testing processes varied, most included steps to address aspects of eligibility of beneficiaries, goods, or services—a key component of determining the appropriateness of a payment—in their programs. For example, according to their policies and procedures for Medicare Fee-for-Service, reviewers examine the medical necessity, compliance with documentation requirements, and coding of services provided, among other things; for the Earned Income Tax Credit (EITC), auditors examine whether the taxpayer properly reported income and whether the taxpayer meets eligibility criteria, including income and qualifying child requirements, and auditors examine, among other things, whether the taxpayer is subject to a disallowance period on receiving EITC; for Medicaid, reviewers examine fee-for-service claims and managed care payments to determine the eligibility status of the beneficiary and the provider, as well as support for the medical necessity of fee-for- service claims, among other things; for Old-Age, Survivors, and Disability Insurance (OASDI), reviewers examine factors to support the beneficiary’s eligibility, including, among other things, citizenship, relationship (in the case of survivor benefits), and receipt of other government benefits; and although Education’s Direct Loan program reviews can vary in scope, they may include, among other things, steps to verify educational institution eligibility (such as licensing and accreditation) and student eligibility (such as enrollment status and satisfactory academic progress). In contrast, per their policies and procedures, eligibility is not tested for DOD’s Military Pay or the overpayment component of OPM’s Retirement estimate. DOD Military Pay. DOD reported using the results of monthly payment reviews to calculate a projected improper payment amount for Military Pay. However, DOD’s policies and procedures do not require a review of servicemember eligibility for special pay or allowances as part of these monthly reviews. DOD’s Standard Operating Procedures (SOP) direct reviewers to recalculate payments to servicemembers solely based on the pay account data included in DOD systems (i.e., to verify that components of servicemember pay were calculated appropriately). DOD’s SOP does not direct reviewers to verify that servicemembers were eligible for special pay or allowances by verifying the information included in the pay account (such as pay grade) with supporting documentation. According to DOD officials, reviewers may investigate potential inconsistencies in pay account data identified during their reviews—which may include eligibility issues—but this process is not consistently performed or documented. According to DOD officials, an example of a potential inconsistency is when a servicemember receives jump pay (a hazard pay for parachute jumps) but is located at a site where no jump activity occurred. According to DOD officials, to help compensate for the limitations of its monthly reviews, DOD calculates the final reported Military Pay improper payment estimate by adding actual debts due to DOD (related to overpayments) identified during the year to the projected estimate of the monthly reviews. DOD identifies the actual overpayments through various methods, including other postpayment reviews and servicemember self- reporting. Standards for Internal Control in the Federal Government states that management should identify, analyze, and respond to risks related to achieving the defined objectives. DOD has acknowledged internal control deficiencies related to the Military Pay program, which—if addressed in improper payment testing—could have an impact on the program’s improper payment rate. However, these deficiencies were identified through other internal control reviews not related to estimating improper payments. For the purposes of estimating improper payments, DOD has not fully assessed the risks in its Military Pay program and evaluated whether its approach for estimating improper payments effectively addresses these risks. As a result, DOD’s process for estimating Military Pay improper payments may not reflect significant risks of improper payment in the program, specifically whether servicemembers are eligible for the special pay or allowances they receive, calling into question the improper payment estimate and its usefulness for developing effective corrective actions. OPM Retirement. OPM relies on its existing Quality Assurance (QA) process to estimate Retirement underpayments. The QA process is designed to determine whether new Retirement claims (i.e., claims paid for the first time) have been adjudicated correctly. Therefore, only new Retirement claims are sampled and tested for accuracy. OPM applies historical results of QA testing to older claims; however, these historical results do not reflect any different risks of underpayment that the older claims may face. Although OPM’s QA process also produces an estimate of overpayments, the agency’s policies and procedures instead use actual debts due to OPM (related to overpayments) that were identified during the fiscal year as its overpayment amount (i.e., the overpayment amount does not reflect any testing of Retirement payments to verify eligibility or accuracy). These actual overpayments represent amounts that have been identified through various means, such as inspector general fraud referrals. OPM officials stated that the agency uses actual amounts because the QA estimate may overstate overpayments. However, the fiscal year 2016 QA overpayment estimate was lower than the actual amount of debts identified as due to OPM. Standards for Internal Control in the Federal Government states that management should identify, analyze, and respond to risks related to achieving the defined objectives. OPM has not fully assessed the risks of improper payments in its Retirement program—particularly related to the risk of underpayments in older claims and the risk of overpayments— and evaluated whether its approach for estimating improper payments effectively addresses these risks. As a result, OPM’s processes for estimating Retirement improper payments may not reflect significant risks of improper payment in the program, calling into question the improper payment estimate and its usefulness for developing effective corrective actions. OMB guidance. OMB issues guidance for agencies to implement various requirements of the improper payment laws. Specifically, OMB is required by IPERIA to issue guidance to set standards for agencies to follow in determining the underlying validity of sampled payments to ensure that amounts being billed, paid, or obligated for payment are proper. Although existing OMB guidance addresses requirements for sampling, it does not address how agencies test to identify improper payments, such as using a risk-based approach to help ensure that key risks of improper payments, like eligibility, are addressed through testing processes. Without such guidance, there is increased risk that agencies’ processes may not address key risks of improper payments in their programs—for example, the cases of DOD Military Pay and OPM Retirement described above—calling into question the improper payment estimates for such programs and their usefulness for developing effective corrective actions. According to OMB guidance, when an agency’s review is unable to determine whether a payment was proper because of insufficient or lack of documentation, the payment must be considered an improper payment. Among the six agencies and 10 programs we reviewed, treatment of insufficient documentation varied by program, as did the classification of these issues for root cause reporting in the AFRs. HHS’s programs were the only ones we reviewed that reported improper payments in the insufficient documentation root cause category for fiscal years 2016 or 2017, as shown in table 6. Some agencies stated that they report insufficient documentation in other root cause categories that they consider more appropriate. For example, Education officials stated that for the Direct Loan program, payments that lack sufficient supporting documentation may be placed in the “Administrative or Process Error Made by Other Party” root cause category. In these cases, a third party—such as a loan servicer—is unable to provide sufficient documentation supporting that the sampled payment was proper. OMB guidance states that in cases where the agency believes that more than one root cause category might be suitable, the agency should determine which category it believes to be the most appropriate. Additionally, some agencies stated that the “insufficient documentation” category was not always relevant when they recreated sampled cases to estimate a program’s improper payments. For example, according to officials, to complete an OASDI stewardship review of a sampled case, a Social Security Administration (SSA) quality reviewer reviews the documentation related to the original determination and then independently re-develops all factors of the payment and interviews the associated beneficiary. According to agency officials, insufficient documentation would not apply as all improper payments identified in the stewardship sample are supported by documentation and payment has been verified in all reviewed cases. As noted previously, the processes for estimating DOD Military Pay and OPM Retirement improper payments were limited, and these limitations may have an impact on the agencies’ ability to identify improper payments related to insufficient documentation. Treatment of cases of nonresponse. Some agencies contact outside entities—such as payees or beneficiaries—as part of their improper payment testing processes. Among the six agencies we reviewed, treatment of cases of nonresponse differed. For example: SSA officials stated that in cases where quality reviewers do not receive responses from OASDI beneficiaries they contact, they exclude the cases from review (unless the reviewer identifies an improper payment in the initial review that is completed prior to reaching out to the beneficiary). For EITC improper payment estimation purposes, the Internal Revenue Service (IRS) stated that the agency does not consider the sampled payment associated with a nonresponse case to be proper or improper. It sets the sampling weight of nonresponse cases to zero and adjusts the sampling weights of respondents upward to account for the nonresponse cases. IRS’s methodology assumes nonresponse and response cases have an equal likelihood of improper payment. For Medicare Fee-for-Service and Medicaid, HHS’s policies and procedures consider payments associated with nonresponse cases to be improper. OMB guidance states that when an agency’s review is unable to discern whether a payment was proper as a result of insufficient or lack of documentation, this payment must be considered an improper payment. However, it does not specifically address the appropriate treatment of nonresponse cases for improper payment estimation purposes. As a result, without clearer guidance there is increased risk that agencies’ improper payment estimates may be understated and that estimates for similar programs may not be comparable. When agencies identify improper payments, they must determine the amount of the payment that was improperly made. The six agencies we reviewed generally reported using the definition of improper payment in relevant laws and OMB guidance to determine the amount of improper payments identified. OMB guidance provides agencies with instructions on how to calculate the amount of improper payments. However, when developing its improper payment estimate for EITC, IRS subtracted overpayments that were paid out and later recovered. By subtracting recovered overpayments, IRS excluded them from the EITC improper payment estimate. For 2013—the tax year used to produce the fiscal year 2017 improper payment estimate—IRS estimated that $1.2 billion in EITC overpayments would be recovered. IPERIA directed OMB to provide guidance that requires agencies to include all improper payments in their improper payment estimates, regardless of whether they have been or are being recovered. Although the OMB guidance was revised in October 2014 to implement this requirement, IRS has not updated its estimation methodology for EITC. By not updating its guidance and continuing to remove EITC overpayments that may be subsequently recovered, IRS is understating its improper payment estimate and potentially limits its ability to address these types of improper payments before they occur. Improper payments are a long-standing, significant problem in the federal government. Estimation of improper payments is key to understanding the extent of the problem and to developing effective corrective actions to address it. Among the six agencies we reviewed, processes to estimate improper payments in their programs varied, and certain differences in these processes may affect the quality of the resulting estimates and consequently these agencies’ efforts to reduce improper payments. Specifically, policies and procedures for DOD’s Military Pay and OPM’s Retirement programs’ improper payment estimation methodologies do not address certain key risks, like eligibility, in part because these agencies have not fully assessed their processes. Further, although OMB guidance addresses requirements for sampling, it does not address how agencies test to identify improper payments. Without such assessments and guidance, there is increased risk that agencies’ processes may not address key risks of improper payments in their programs, calling into question the improper payment estimates for such programs and their usefulness for developing effective corrective actions. Additionally, for agencies we reviewed that contact outside entities as part of their improper payment estimation processes, the treatment of cases of nonresponse varied. OMB guidance does not specifically address the appropriate treatment of nonresponse cases for improper payment estimation purposes. Without clearer guidance there is increased risk that agencies’ improper payment estimates may be understated and that estimates for similar programs may not be comparable. Finally, although IPERIA directed OMB to provide guidance that requires agencies to include all improper payments in their improper payment estimates, regardless of whether they have been or are being recovered, IRS has not updated its processes to reflect the change. By not updating its guidance and continuing to remove EITC overpayments that may be subsequently recovered, IRS is understating its improper payment estimate and potentially limits its ability to address these types of improper payments before they occur. We are making two recommendations to the Director of OMB that have government-wide implications and specific recommendations to DOD, OPM, and IRS regarding their programs included in this review. The Director of OMB should develop guidance on how agencies test to identify improper payments, such as using a risk-based approach to help ensure that key risks of improper payments, such as eligibility, are addressed through testing processes. (Recommendation 1) The Director of OMB should develop guidance clarifying the appropriate treatment of nonresponse cases during improper payment testing. (Recommendation 2) The Under Secretary of Defense (Comptroller) should assess the processes for estimating Military Pay improper payments to determine whether they effectively address key risks of improper payments— including eligibility for different types of pay and allowances—and take steps to update the processes to incorporate key risks that are not currently addressed. (Recommendation 3) The Director of OPM should assess the processes to estimate Retirement improper payments to determine whether they effectively address key risks of improper payments—including eligibility and whether older claims face different risks of improper payments than new claims—and take steps to update the processes to incorporate key risks that are not currently addressed. (Recommendation 4) The Commissioner of IRS should update IRS’s improper payment estimation methodology to not exclude recovered overpayments from its EITC improper payment estimate. (Recommendation 5) We provided a draft of this report for comment to OMB, DOD, Education, HHS, Treasury, OPM, SSA, and USDA. OMB provided oral comments, which are summarized below. OPM, DOD, and IRS provided written comments, which are reproduced in appendixes II through IV, respectively. Education, HHS, SSA, and USDA did not provide written comments on the draft report. In addition, HHS, IRS, OMB, OPM, and SSA provided technical comments, which we have incorporated, as appropriate. In oral comments provided on April 30, 2018, a Senior Policy Advisor in OMB’s Office of Federal Financial Management stated that OMB partially agreed with our first recommendation and agreed with our second recommendation. Regarding the first recommendation, the Senior Policy Advisor stated that OMB should not have to develop more specific guidance as each program and activity has its own risks. Instead, inspectors general are better equipped and positioned to review the sampling and estimation plans as part of their annual IPERA compliance audits and that agencies, their statisticians, and inspectors general should work out the best testing procedures for their agencies. OMB could provide suggestions during OMB’s annual town hall meeting related to improper payments for areas that inspectors general may consider. Although we agree that programs and activities may face different risks of improper payment, we continue to believe that guidance from OMB on how agencies test to identify improper payments—such as directing agencies to take a risk-based approach in developing their testing procedures—could help ensure that agencies address the specific risks they identify when developing improper payment estimates. Further, such guidance could also help ensure that testing processes are designed to address an agency’s identified risks before the estimate is developed, whereas an inspector general’s review—as well as related recommendations for improvement—would generally occur after the agency’s improper payment estimate had been developed and reported. Regarding the second recommendation, the Senior Policy Advisor noted that OMB plans to update its guidance to direct agencies to treat nonresponse cases as improper payments and to include a new category for tracking such cases. In its written comments, OPM partially concurred with our recommendation to assess the processes to estimate Retirement improper payments to determine whether they effectively address the key risks of improper payments. OPM agreed to conduct an audit of older claims to determine if they face different risks than new claims. However, OPM did not agree with the part of the recommendation to assess the risk of improper payments related to eligibility in the estimation process. OPM stated that eligibility is determined before annuity or survivor benefits are fully adjudicated. However, the objective of an improper payment estimate is to determine whether payments were made properly. To do so, an agency should determine whether the payee was eligible for the payment that was made, among other things. As such, we continue to believe that the recommendation—including the assessment of the risk of improper payments related to eligibility—is warranted. In their written comments, DOD and IRS both agreed with our recommendations directed to them and described the steps they plan to take to implement them. We are sending copies of this report to the appropriate congressional committees; the Secretaries of Agriculture, Defense, Education, Health and Human Services, and the Treasury; the Director of the Office of Personnel Management; the Administrator of the Social Security Administration; the Director of the Office of Management and Budget; and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-2623 or davisbh@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix V. Table 7 lists the fiscal year 2017 improper payment estimates by agency and program, as reported by agencies in their fiscal year 2017 agency financial reports and compiled on the Office of Management and Budget’s payment integrity website, paymentaccuracy.gov. In addition to the contact named above, Phillip McIntyre (Assistant Director), James M. Healy (Auditor in Charge), Daniel Flavin, and Fabiola Torres made key contributions to this report.

 

 Report 2:
Coast Guard staffing for the TAP program reflects the organizational structure of its Health, Safety, and Work-Life Directorate, which oversees TAP policy. The Coast Guard’s TAP managers are assigned to 13 installations where Health, Safety, and Work-Life offices are located. One or two TAP managers are assigned to each of the Coast Guard’s nine districts, which often span multiple states and territories, and these TAP managers oversee operations both for the installation where they work and for units stationed throughout the region (see fig. 1). For example, the TAP manager assigned to Coast Guard Base Cleveland oversees TAP implementation both for that installation and for Coast Guard units serving in Coast Guard District 9—a region that encompasses portions of eight states and the Great Lakes area. The program manager in Coast Guard Headquarters manages Coast Guard’s Transition Assistance Program. The Coast Guard protects and defends over 100,000 miles of U.S. coastline and inland waterways, and consequently, TAP-eligible Coast Guard servicemembers sometimes work in small, widely dispersed units assigned to remote locations, including on Coast Guard vessels. One aspect of the Coast Guard’s mission—a first responder for maritime search and rescue in United States waters—can require Coast Guard servicemembers to respond to emergency situations at a moment’s notice. The Coast Guard, which is overseen by DHS, not DOD, generally oversees TAP implementation for its servicemembers. Federal law requires DOD and DHS to require eligible servicemembers under their respective command to participate in TAP, with some exceptions. In response to this statutory requirement, DOD has promulgated regulations and developed issuances which require that servicemembers complete the component parts of the TAP program, and that commanding officers ensure that servicemembers under their command complete these parts, with some exceptions. In contrast, according to Coast Guard officials, Coast Guard has not promulgated any regulations to implement TAP. Further, Coast Guard issued its most recent Commandant Instruction in 2003, approximately 8 years prior to TAP redesign in 2011. However, Coast Guard issued policy guidance in 2014 that made some limited updates to the Commandant Instruction. Coast Guard officials also said the Coast Guard plans to issue a new TAP Commandant Instruction in May 2018. Under the redesigned TAP, Coast Guard servicemembers—like their DOD counterparts—begin TAP by attending pre-separation or transition counseling where they are briefed on TAP requirements and available transition resources. Pre-separation or transition counseling can be delivered by TAP managers, uniformed career counselors, or online (see fig. 2). Coast Guard servicemembers are able to participate in TAP either through the Coast Guard or at a DOD installation, if space is available. During or at the end of pre-separation or transition counseling, participants register for and attend TAP courses. The core curriculum includes three required courses—the Department of Labor (DOL) Employment Workshop, unless exempt, and Department of Veterans Affairs (VA) Benefits Briefings I and II—and other courses that focus on aspects such as translating military skills and experiences into credentialing for civilian jobs and preparing a financial plan. Participants may also elect to attend additional 2-day classes either at a Coast Guard or DOD installation or online through the Joint Knowledge Online platform, according to agency officials. These additional 2-day classes include Accessing Higher Education, Career Technical Training, and Entrepreneurship. Federal law requires the Coast Guard to permit servicemembers who elect to take these additional 2-day classes to receive them. Federal law establishes a time frame within which servicemembers with anticipated separation or retirement dates should begin the program. According to federal law, retirees with anticipated separation dates are expected to begin TAP as soon as possible during the 24-month period preceding that date, but not later than 90 days before separation. Similarly, servicemembers with anticipated separation dates who are not retiring are expected to begin as soon as possible during the 12-month period preceding that date, but not later than 90 days before separation. Servicemembers who learn that they will separate or retire from the military fewer than 90 days before their anticipated separation or retirement date are expected to begin TAP as soon as possible within their remaining period of service. As we previously reported, officials from multiple federal agencies collaborate to deliver and assess TAP. The TAP interagency governance structure includes senior officials from DOD, VA, DOL, DHS, the Department of Education, the U.S. Office of Personnel Management, and the Small Business Administration (SBA), who participate in TAP Senior Steering Group meetings at least every month and TAP Executive Council meetings each quarter. Further, officials tasked to particular interagency working groups focus on specific elements of TAP (e.g., curriculum or performance measures), meet more frequently (typically at least once a month), and generally communicate weekly, according to agency officials. The TAP program manager for the Coast Guard told us that he participates in several of the working groups. One such working group is the performance management working group that oversees the interagency TAP evaluation plan, which includes monitoring performance measures related to TAP requirements, indicators of post-program outcomes, and formal evaluations sponsored by interagency partners. While DOD tracks TAP-specific performance measures, other interagency partners track indicators of how well veterans fare after leaving military service. For example, DOD tracks performance measures prior to servicemembers’ separation, such as TAP participation and credential attainment rates, while other agencies track post-separation indicators, such as unemployment rates among veterans ages 18 to 24. The performance management working group also reviews the formal evaluation efforts led by individual agencies and provides feedback to help shape their efforts in accordance with the TAP Evaluation Plan. The Coast Guard does not have complete or reliable data on participation levels in TAP. According to Coast Guard officials, a major reason why the data are not reliable is that the Coast Guard lacks an up-to-date Commandant Instruction that specifies when to record TAP participation data. Consequently, the data are updated on an ad-hoc basis, according to agency officials, and may not be timely or complete. For example, one TAP manager said she updates the list of TAP participants for her installation only once every few months because of her other duties. According to federal internal control standards, management should use quality information—including current and timely information— to achieve the entity’s objectives and to communicate quality information to external parties. Given the lack of timely and complete data, we determined the Coast Guard’s TAP data were not sufficiently reliable for an analysis of participation in TAP classes. Because it lacks policies and procedures governing reliable data collection, including when data should be entered and by whom, the Coast Guard cannot determine to what extent its servicemembers attend TAP, although federal law mandates that DHS ensure all TAP-eligible servicemembers participate in the program. In addition, the data collection system currently used to track TAP participation is not sufficient to ensure reliable data. For example, according to Coast Guard staff, TAP staff enter TAP participation data into a shared spreadsheet that all TAP managers can edit. Specifically, staff record the names of servicemembers they identify as TAP-eligible and whether these individuals completed required portions of TAP. Coast Guard officials said they are in the process of adopting a new data system, in October 2018, to more reliably track TAP participation and that they expect to fully adopt this system–DOD’s TAP-IT Enterprise System—after a new Commandant Instruction is finalized, in May 2018. In November 2016, DOD launched the new system to collect TAP-related data for servicemembers in the Army, Navy, Air Force, and Marine Corps. In addition to standardizing data collection and improving data completeness and accuracy, the TAP-IT Enterprise System is expected to track information related to the time frames of servicemembers’ participation. According to a senior DOD official, the military services will not be able to use the system to generate unit-level or installation-level reports until October 2018. According to our survey, the most common factor affecting TAP participation, cited at 11 of the 12 Coast Guard installations we surveyed, pertained to servicemembers assigned to geographically remote locations. The next three most commonly cited factors–each cited by 7 of the 12 installations surveyed—relate to the timing of TAP participation: rapid separation from the military, not being sufficiently aware of the need to attend TAP, and starting the transition process too late to attend. (See fig. 3.) Headquarters-based TAP officials identified additional factors that may affect servicemember participation, such as separating from the Coast Guard Reserves or retiring with no plans to work after leaving the military. However, the Coast Guard lacks participation data to verify whether participation rates for these groups are in fact lower than for other Coast Guard servicemembers. Coast Guard installations we surveyed did not indicate that unit commanders or direct supervisors affected participation in TAP’s required courses or additional 2-day classes. However, Coast Guard servicemembers and TAP officials we spoke with said unit commanders or direct supervisors sometimes prevented participation. All three TAP managers we spoke with (of 12 nationwide) told us that while commanders generally allowed servicemembers to register for TAP courses, they occasionally required them to return to their duties before completing the courses. We observed this during a TAP class at a Coast Guard installation we visited when a servicemember’s commander ordered her to return to the unit during TAP training and she missed a briefing she wanted to attend. Two of three TAP managers we interviewed also said commanders sometimes required servicemembers under their command to wait to take TAP classes until close to their separation date because of mission priorities. Two of three TAP managers interviewed said that commanders in the Coast Guard face unique challenges in ensuring TAP participation. They said commanders in all branches of the military must balance competing demands, including their primary mission and the training needs of the personnel they oversee. They said it can be particularly difficult for Coast Guard commanders to juggle these priorities because Coast Guard servicemembers are sometimes assigned to very small units or called to return to duty for emergency situations during scheduled TAP classes. One TAP manager said that a commander in a remote location had collaborated with her to provide a classroom-based TAP class for transitioning Coast Guard servicemembers within the commander’s unit, but rescue efforts occurred during the class which resulted in most of those servicemembers returning to their vessel to respond to the emergency. In addition, all three TAP managers we spoke with said there are limited resources for holding TAP in a classroom setting. Consequently, classroom-based TAP may not be offered frequently in remote locations, making rescheduling difficult. One TAP manager said that her installation typically offers three or four TAP classes a year and because classes are so infrequent, servicemembers are encouraged to start TAP as soon as possible prior to separation. Coast Guard staff we interviewed said that juggling competing priorities affected the Coast Guard’s ability to implement TAP. Both the frontline and headquarters staff who oversee TAP implementation said they oversee at least three other programs in addition to TAP at their installation and throughout their regions, including the Coast Guard’s relocation and spousal employment programs. The Coast Guard relies on online delivery of TAP information and classes for servicemembers who are rapidly separating and assigned to remote and geographically dispersed units, according to our survey results and several Coast Guard staff we interviewed. For example, all 12 installations we surveyed cited servicemembers facing rapid separations as a reason for accessing TAP training online, and 11 cited servicemembers being remotely stationed as a reason. Coast Guard staff also said online TAP was used for servicemembers interested in attending additional 2-day classes. The three TAP managers we interviewed also identified several reasons why installations had to rely on online TAP classes. For example, one manager corroborated our survey results, saying that many Coast Guard servicemembers worked in small units assigned to remote and geographically dispersed locations, making it difficult to convene a sufficient number of transitioning Coast Guard servicemembers to meet minimum class size requirements. In addition, all three managers said they used the online version of TAP for remotely stationed Coast Guard servicemembers because the Coast Guard lacked the resources for them to attend classes in person. Although they preferred that servicemembers participate in live, classroom-based TAP classes, all of the managers acknowledged that the online version of TAP played an integral role in ensuring that more servicemembers could participate in the program. However, two of them noted that while classroom delivery of TAP classes provided an interactive learning environment that allowed participants to ask questions and learn from their peers, online participants generally clicked quickly through the slides and had difficulty understanding the information being presented. Two managers told us that they regularly used the online version to deliver parts of the TAP curriculum. For example, one TAP manager said she required participants to complete the crosswalk of military and civilian occupations class online before attending required classes in person. Two managers noted that additional 2-day classes were available online, and one noted that some servicemembers attended these classes in a classroom setting either on a Coast Guard base or a DOD installation. Finally, all three TAP managers said that many participants in online TAP classes would benefit from participating in a real-time virtual version of TAP led by live facilitators. Two managers told us that having a remote facilitator delivering TAP in real time would give participants more opportunity to ask questions and better understand and absorb class content. Despite these challenges, TAP managers and separating Coast Guard servicemembers we interviewed provided generally positive feedback about the TAP program. All of the 25 Coast Guard servicemembers we spoke with said that the information they received during the courses was useful and they liked the instructors. One Coast Guard servicemember praised the classroom courses for being interactive, and several Coast Guard servicemembers said they wanted the opportunity to retake TAP before or shortly after they separated from the Coast Guard. However, many said the volume of information presented in a short period of time could be overwhelming and was like “trying to drink from a firehose.” The Coast Guard has not set a formal performance goal for TAP participation, according to a Coast Guard official, and as previously discussed, does not have complete, reliable data. Without reliable information, the Coast Guard cannot effectively monitor TAP implementation or measure program performance. DHS is mandated to ensure that all TAP- eligible servicemembers of the Coast Guard participate in TAP before leaving military service. However, without effective monitoring of program participation, the Coast Guard cannot know to what extent its servicemembers receive the required training they need to prepare for civilian life. According to federal internal control standards, management should consider external requirements—such as the laws with which the entity is required to comply—to clearly define objectives in specific and measurable terms. In addition, establishing goals can help agencies define expected performance and articulate results. A Coast Guard official said the Coast Guard’s long-term goal is for full compliance with TAP requirements, but in the interim, the Coast Guard uses DOD’s 85 percent VOW compliance goal as an informal benchmark against which to gauge the Coast Guard’s TAP performance. However, the Coast Guard has not communicated a specific, measurable goal to TAP staff implementing the program, or to Coast Guard commanders who oversee separating and retiring Coast Guard servicemembers, according to a Coast Guard official. Establishing and communicating a formal goal could help the Coast Guard define expected performance. The official also told us that, like DOD, the Coast Guard tracks the elements of TAP mandated under the VOW Act— transition or pre-separation counseling, VA Benefits I and II, and the DOL Employment Workshop. The Coast Guard does not monitor the (1) timeliness of participation in TAP, and (2) access to additional 2-day classes. A Coast Guard official said the Coast Guard does not currently monitor TAP beyond tracking whether separating servicemembers participate in the required courses, and currently lacks the capacity to undertake additional monitoring efforts. However, he said additional monitoring would be possible once the Coast Guard completed the move to the DOD TAP-IT Enterprise data system. According to a Coast Guard official, the Coast Guard does not currently monitor the timeliness of TAP participation although federal law prescribes time frames for servicemembers to begin TAP participation. Generally, separating servicemembers who are not retiring are to begin TAP participation no later than 90 days before their separation date. Without a systematic method for monitoring timeliness, the Coast Guard cannot know whether its servicemembers begin the program on time or account for the timeliness of TAP participation. As a result, the Coast Guard cannot know whether its servicemembers are starting TAP early enough to complete the training they need to adequately prepare for their transition to civilian life. The Coast Guard does not track which of its servicemembers participate in the additional 2-day classes, according to a Coast Guard official we interviewed, even though federal law requires that DHS ensure those who elect to participate are able to receive the training. By not tracking which Coast Guard servicemembers participate in 2-day classes or requiring transition staff to document when servicemembers ask to attend, the Coast Guard cannot determine the extent to which servicemembers who wished to attend these courses were able to do so, as required by law. Coast Guard commanders and TAP managers do not have clearly defined roles and responsibilities in implementing TAP because of the lack of an up-to-date Commandant Instruction, according to TAP staff we interviewed. As previously discussed, the Coast Guard’s last Commandant Instruction on TAP was issued in 2003, approximately 8 years prior to TAP’s redesign. According to federal internal control standards, to achieve the entity’s objectives, management should assign responsibility and delegate authority to key roles throughout the entity. Without an up-to-date Commandant Instruction, TAP managers and commanders may be unclear on who is ultimately responsible for ensuring servicemembers attend TAP. Moreover, two TAP managers also told us that an up-to-date Commandant Instruction might lead some commanders to place higher priority on ensuring TAP participation. Coast Guard officials said the Coast Guard was in the process of revising the TAP Commandant Instruction and anticipated issuing the new instruction in May 2018. The Coast Guard lacks the ability to share data with commanders, limiting its ability to monitor TAP participation and ensure servicemembers attend the program. According to a Coast Guard official, the Coast Guard’s current data collection system also cannot generate installation or unit- level participation rates to share with commanders who oversee transitioning and retiring servicemembers. Federal internal control standards state that management should share quality information throughout an organization to enable personnel to perform key roles, and we have previously reported that by regularly sharing useful performance information with leaders at multiple levels of an organization, agencies can help leaders make informed decisions. Without this information, individual unit commanders or the commanders’ supervisors cannot determine whether Coast Guard servicemembers under their command completed TAP or identify whether there is a need for corrective actions to ensure they do so. As we mention earlier in this report, the Coast Guard plans to adopt DOD’s TAP-IT Enterprise System, which according to officials, could help the Coast Guard ensure eligible servicemembers participate in the program. According to a Coast Guard official, once the system is fully implemented by the Coast Guard, commanders will be required to verify and document whether Coast Guard servicemembers under their command completed TAP, potentially making commanders more vested in the process. We previously reported that a senior DOD official said that the TAP-IT Enterprise System may be able to generate unit- and installation-level reports for the four DOD-led military services by October 2018, and a Coast Guard official said he would work with DOD to identify whether this capability could also extend to the Coast Guard. Once data reliability improves, sharing installation and unit-level TAP performance information with Coast Guard commanders could support monitoring efforts. The performance measures tracked by the TAP interagency working group do not reflect TAP implementation broadly across all five military services, according to a Coast Guard official we interviewed. The Coast Guard does not currently share TAP data it collects with DOD or other members of the interagency performance working group. While the benefits of interagency data sharing cannot be realized without the Coast Guard first improving the quality and completeness of its TAP data, we have identified leading practices for interagency collaboration, including that members of interagency working groups identify and share relevant agency performance data. Moreover, federal internal control standards call for management to communicate quality information to external parties. Because the Coast Guard does not share TAP data, the performance measures tracked by the interagency group do not reflect Coast Guard servicemembers’ experiences and thus do not provide a complete picture of TAP implementation across the five military services. More specifically for the Coast Guard, without such data sharing, future TAP evaluations may not be able to assess the effectiveness of TAP delivery, hindering the Coast Guard’s ability to make program adjustments to better prepare its servicemembers to successfully transition to life after military service. Coast Guard officials said migrating to DOD’s TAP-IT Enterprise System will facilitate information sharing with interagency partners and that improving data completeness and reliability is a top priority for 2018. Given the sacrifices servicemembers have made to serve their country, it is imperative they are afforded every chance to adequately prepare for civilian life before leaving military service. In order to make a successful transition, servicemembers need to be well-positioned to get a job or make an informed decision about whether to pursue additional education or start a small business. As such, the Transition Assistance Program (TAP) serves a critically important function—to give servicemembers the tools and information they need to successfully transition to life outside the military. Federal law requires that the Coast Guard ensure all eligible servicemembers participate in the program, but thousands of Coast Guard servicemembers may have transitioned without the support provided by TAP. Reliably tracking participation has proven to be a challenge for the Coast Guard, in part because it lacks a current Commandant Instruction that defines the roles and responsibilities of staff responsible for implementing TAP and ensuring complete and reliable data are collected. In preparing to issue an updated Commandant Instruction, the Coast Guard has taken a positive step toward addressing the limitations of its current TAP data, and will be better positioned to ensure compliance with VOW Act requirements using reliable data. In addition to collecting reliable data, the Coast Guard could further demonstrate its commitment to meeting TAP requirements by establishing formal performance goals that measure the extent to which Coast Guard servicemembers participate in TAP. By establishing interim performance goals, the agency would be able to show its progress towards achieving full compliance. Moreover, communicating performance goals to unit and installation commanders could enhance accountability and might spur progress toward meeting federal program requirements. By expanding its monitoring efforts beyond tracking participation in TAP’s required classes, the Coast Guard could enhance its ability to ensure other TAP requirements are met and that its servicemembers are able to access additional transition resources. Monitoring the timeliness of participation would help ensure Coast Guard servicemembers have adequate time to complete TAP before leaving the military. Further, by monitoring requests to participate in additional 2-day classes and 2-day class attendance, the Coast Guard would be in a better position to identify whether servicemembers who wish to attend the classes are able to do so, to determine whether more classes are needed, and to communicate this information to the interagency partners responsible for delivering these classes. Commanders can also play a key role in bolstering TAP participation. Having an up-to-date written Commandant Instruction that explicitly describes commanders’ roles and responsibilities could enhance commanders’ ability to ensure TAP’s proper implementation and compliance with VOW Act requirements. Moreover, once data quality improves, providing commanders a mechanism to readily determine whether servicemembers under their command have completed TAP could help them monitor the program to ensure that all TAP-eligible servicemembers receive the resources they need to successfully transition to civilian life. Finally, once more reliable data on Coast Guard servicemember participation are available, sharing this information with interagency partners could improve TAP implementation on a broader scale. Sharing reliable data, such as participation figures for the Coast Guard, would give TAP interagency partners a more complete picture of implementation across all five military services. Sharing such information would also enhance the interagency group’s ability to evaluate how well TAP serves the entire population of servicemembers. Improving the reliability of the Coast Guard’s TAP data will be essential for the benefits of data sharing to be realized. To ensure that all eligible Coast Guard servicemembers are provided the opportunity to complete the Transition Assistance Program (TAP), we recommend the Commandant of the Coast Guard take the following seven actions: Issue an updated Commandant Instruction that establishes policies and procedures to improve the reliability and completeness of TAP data by including when and by whom data should be recorded and updated. (Recommendation 1) Establish a formal performance goal with a measurable target for participation rates in VOW Act-mandated portions of TAP. (Recommendation 2) Monitor the extent to which Coast Guard servicemembers participate in TAP within prescribed time frames. (Recommendation 3) Monitor the extent to which Coast Guard servicemembers who elect to participate in additional 2-day classes are afforded the opportunity to attend. (Recommendation 4) Issue an updated Commandant Instruction that defines the roles and responsibilities of the personnel who administer the program and ensure servicemembers’ participation. (Recommendation 5) Once reliable data are available by installation or unit, enable unit commanders and the higher-level commanders to whom they report to access TAP performance information specifically for the units they oversee so that they can monitor compliance with all TAP requirements. (Recommendation 6) Once reliable data are available, share TAP information with DOD and other interagency partners, such as data on participation in required TAP courses and additional 2-day classes. (Recommendation 7) We provided a draft of this report to the Departments of Homeland Security, Defense, Education, Labor, and Veterans Affairs, the Office of Personnel Management, and the Small Business Administration for their review and comment. The formal written response of the Department of Homeland Security (DHS) is reproduced in appendix II. In addition, DHS provided technical comments from Coast Guard officials that we incorporated into the report as appropriate. The other agencies did not provide any comments. In its written comments, DHS agreed with all seven of our recommendations. If you or your staff have any questions about this report, please contact me at (202) 512-7215 or brownbarnesc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III. This report examines (1) what is known about the reliability of Transition Assistance Program (TAP) data on participation levels and the factors that affect Coast Guard servicemembers’ participation, and (2) the extent to which the Coast Guard measures TAP performance and monitors key areas of TAP implementation. To address these questions, we surveyed Coast Guard installations with full-time TAP operations; reviewed Coast Guard data on TAP participation for fiscal years 2012 to 2017; visited one Coast Guard installation and interviewed TAP managers from two additional Coast Guard installations selected for diversity in location, among other reasons; and interviewed Coast Guard officials responsible for overseeing TAP implementation for the Coast Guard. We also reviewed relevant federal laws, regulations, policies, documents, and publications. Information in this report is current as of the date GAO received formal agency comments from DHS. Our survey of Coast Guard installations with full-time TAP operations asked about how TAP was being implemented. The survey included questions about the accessibility of TAP components, challenges Coast Guard servicemembers faced in attending the components, and the level of commander support for participation. Our survey targeted front-line TAP managers, who could draw on the expertise of TAP course facilitators, transition counselors, career counselors, and other key TAP staff as necessary. After drafting the survey questions, we pretested them with a TAP manager to ensure (1) the questions were clear and unambiguous, (2) terminology was used correctly, (3) the survey did not place an undue burden on agency officials, (4) the information could feasibly be obtained, and (5) the survey was comprehensive and unbiased. We revised the content and format of the survey based on the feedback we received. We initially sent the survey to TAP managers at all 13 Coast Guard installations at which TAP staff were located. We removed one installation when we later found that the TAP manager position was vacant and revised the total to 12 Coast Guard installations. The survey was accessible online from October 31, 2016, through January 18, 2017, through a secure server that recipients were able to access using unique usernames and passwords. We sent an email announcement to TAP staff at all 13 Coast Guard installations at which TAP staff are located on October 24, 2016. We sent a second email on October 31, 2016 to notify participants the survey was available online, and provided their unique passwords and usernames. We sent two follow-up e-mails (November 14, 2016 and November 28, 2016) to those who had not responded. Finally, we contacted all remaining nonrespondents by telephone starting December 5, 2016. The survey was available online until we reached a 100 percent response rate. To increase our understanding into how TAP was being implemented at installations and supplement our survey findings, we visited one Coast Guard installation and interviewed TAP managers from two additional installations. We selected the installations based on several factors, including the size of the installation, proximity to Department of Defense (DOD) installations, and diverse locations in the United States. (See table 1.) At Coast Guard Base Elizabeth City in North Carolina, the installation we visited, we interviewed the TAP manager, uniformed career counselors, and senior installation leadership. During our interviews with TAP managers at all three installations, we asked about the extent to which Coast Guard servicemembers participate in TAP’s required and additional 2-day classes, including whether the servicemembers attended classes online or in a classroom setting, challenges to ensuring Coast Guard servicemembers participate in TAP, and the extent to which they monitor Coast Guard servicemembers’ participation in TAP. At Coast Guard Base Elizabeth City, we also interviewed 25 Coast Guard servicemembers (both officers and enlisted personnel) to get their perspective on how well TAP worked and any challenges they had participating. To help guide the interviews with the Coast Guard servicemembers, we asked them to complete a short questionnaire that asked about their experiences with the TAP program. We also interviewed TAP staff at Coast Guard headquarters to learn about TAP policy, monitoring efforts, and performance measures for the service overall. For example, we asked what policies and procedures guide installations’ TAP implementation; what performance measures the Coast Guard uses to monitor TAP; how performance results are reported and shared with different levels of Coast Guard leadership; and to what extent the Coast Guard uses results from TAP participant satisfaction assessments. We also asked whether the Coast Guard plans to shift to DOD’s new TAP-IT Enterprise System and how using the new system could affect its monitoring efforts in the future. In evaluating the Coast Guard’s performance measures, we focused on measures related to servicemembers’ transition experiences before leaving the military. We did not gather information on post-program evaluations and outcomes because they were determined to be outside the scope of this review. We reviewed DHS data on TAP participation for fiscal years 2012 to 2017. To assess the reliability of the Coast Guard’s TAP participation data, we interviewed agency officials knowledgeable about the data. We determined these data were not sufficiently reliable due to limitations with the Coast Guard’s data collection system. Specifically, the system lacks adequate controls to ensure TAP data are complete and accurate. We conducted this performance audit from February 2016 to April 2018 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. In addition to the contact named above, Meeta Engle (Assistant Director), Amy MacDonald (Analyst-in-Charge), James Bennett, Holly Dye, David Forgosh, Ying Long, Jonathan McMurray, Jean McSween, Andrew Sherrill, Benjamin Sinoff, and Timothy Young, made significant contributions to this report. Also contributing to this report were Susan Aschoff, Jessie Battle, Ramona Burton, Melinda Cordero, Elizabeth Curda, Dawn Hoff, Ben Licht, Serena Lo, Sheila McCoy, Almeta Spencer, Christopher Schmitt, James Whitcomb, and Jill Yost. Transitioning Veterans: DOD Needs to Improve Performance Reporting and Monitoring for the Transition Assistance Program, GAO-18-23. Washington, D.C.: November, 8, 2017. Transitioning Veterans: Improvements Needed in DOD’s Performance Reporting and Monitoring of the Transition Assistance Program, GAO-18-225T. Washington, D.C.: November 8, 2017. Department of Defense: Transition Assistance Program (TAP) for Military Personnel, GAO-16-302R. Washington, D.C.: December 17, 2015. Veterans’ Employment: Need for Further Workshops Should Be Considered before Making Decisions on Their Future, GAO-15-518. Washington, D.C.: July 16, 2015. Military and Veteran Support: DOD and VA Programs That Address the Effects of Combat and Transition to Civilian Life, GAO-15-24. Washington, D.C.: November 7, 2014. Veterans Affairs: Better Understanding Needed to Enhance Services to Veterans Readjusting to Civilian Life, GAO-14-676. Washington, D.C.: September 10, 2014. Transitioning Veterans: Improved Oversight Needed to Enhance Implementation of Transition Assistance Program, GAO-14-144. Washington, D.C.: March 5, 2014. Military and Veterans’ Benefits: Enhanced Services Could Improve Transition Assistance for Reserves and National Guard, GAO-05-544. Washington, D.C.: May 20, 2005. Military and Veterans’ Benefits: Observations on the Transition Assistance Program, GAO-02-914T (July 18, 2002).

Question: Write a one-page summary of the report 1.

Answer: Improper payments—which include payments that should not have been made or were made in an incorrect amount—are a long-standing, significant problem in the federal government, estimated at almost $141 billion for fiscal year 2017. Executive branch agencies are required to annually estimate improper payments for certain programs. Estimation of improper payments is key to understanding the extent of the problem and to developing effective corrective actions. Relevant laws and guidance provide agencies flexibility in developing estimates. This report describes agencies' processes to estimate improper payments in selected programs for fiscal year 2017 and the extent to which certain differences in these processes can affect the usefulness of the resulting estimates. GAO selected 10 programs across six agencies with the largest reported program outlays in fiscal years 2015 and 2016. For these programs, GAO reviewed relevant laws and guidance, analyzed agencies' policies and procedures, and interviewed officials at relevant agencies and OMB staff. The six agencies GAO reviewed reported taking various approaches related to key components of estimating improper payments—shown in the figure below—for 10 selected programs, which collectively reported outlays of over $2.5 trillion for fiscal year 2017. Sample selection. Eight of the 10 programs GAO reviewed reported using statistically valid approaches, and the remaining 2 reported using alternative methodologies approved by the Office of Management and Budget (OMB). The sampled data elements varied, including payments, medical claims, and tax returns. The age of the data used to develop fiscal year 2017 improper payment estimates also varied, ranging from calendar year 2013 to fiscal year 2017. Identification of improper payments. Some of the six agencies reported using processes designed specifically to estimate improper payments, whereas others reported leveraging existing reviews. These agencies' policies and procedures include a review of aspects of eligibility, except for those related to the Department of Defense's (DOD) Military Pay and the Office of Personnel Management's (OPM) Retirement overpayments. DOD and OPM have not fully assessed whether their estimation processes effectively consider key program risks. OMB guidance does not specifically address how agencies are to test to identify improper payments, such as using a risk-based approach to help ensure that key risks of improper payments are addressed. The six agencies also varied in the treatment of insufficient documentation, both in identifying and in reporting the root causes of improper payments. For the agencies that contact entities outside the agency to estimate improper payments, the treatment of nonresponse differed, with one agency including nonresponses as improper payments and another generally excluding the nonresponse cases from review. Although OMB guidance states that agencies should treat cases of insufficient documentation as improper payments, it does not specifically address the treatment of nonresponse cases. Calculation of the improper payment estimate. The six agencies generally reported using law and OMB guidance to calculate improper payment estimates for the selected programs, except for the Earned Income Tax Credit (EITC). The Internal Revenue Service (IRS) removed overpayments that were recovered when developing its estimate. OMB guidance requires agencies to include recovered amounts in their estimates. Removing these overpayments understates the EITC improper payment estimate and may limit IRS's ability to develop corrective actions to prevent improper payments. GAO recommends that OMB develop guidance on treatment of nonresponse cases and testing to identify improper payments, that DOD and OPM assess their estimation processes, and that IRS revise its methodology to not exclude recovered payments from its estimate. All of the agencies either agreed or partially agreed with the specific recommendations to them. GAO believes that the actions are warranted, as discussed in the report.