Viewed broadly, IDT refund fraud is composed of two crimes: (1) the theft or compromise of PII, and (2) the use of stolen (or otherwise compromised) PII to file a fraudulent tax return and collect a fraudulent refund. Identity theft. The sources of stolen identities are limitless, according to an official in IRS’s Criminal Investigation Division. Identity thieves can hack into government or commercial systems, recruit insiders (such as employees in the healthcare or education industries) to steal PII, or purchase or put pieces of PII together to create an identity (see figure 1). To successfully commit identity theft, thieves can exploit specific digital, physical, or social vulnerabilities (see sidebar). According to Department of Justice (DOJ) officials, the PII used in tax refund fraud can also involve compromised identities, where the real identity holder initially sells his identity to third parties. Identity Theft and Personally Identifiable Information (PII) Vulnerability: An Overview PII is vulnerable to theft and exploitation in three broad areas. Digital vulnerability: Criminals can access large amounts of digital information if it is inadequately safeguarded. For example, thieves can steal it through hacking and computer intrusion, can aggregate publicly available information, or can sell and buy PII from other criminals on the black market. In one case, a foreign national obtained PII from online databases and sold it to other criminals, resulting in 13,673 victims and $65 million claimed in refund fraud. Physical vulnerability: If insufficient attention is paid to the structures and tools used to store, maintain, and safeguard PII, such as hard drives, paper records, or unsecured mailboxes, thieves will exploit these vulnerabilities through computer theft and “dumpster diving.” Social vulnerability: Thieves can trick individuals into divulging their own PII or others’ PII, for example by impersonating IRS officials. Thieves may also recruit individuals with legitimate access to sensitive information. In one case, a ring of thieves used its employment access to steal identities from public and private databases, such as the U.S. Army, several Alabama state agencies, a Georgia call center and employee records from a Georgia company. IRS recognized the challenge of IDT refund fraud in its fiscal year 2014- 2017 strategic plan and increased resources dedicated to combating IDT and other types of refund fraud. In fiscal year 2015, IRS reported that it staffed more than 4,000 full-time equivalents and spent about $470 million on all refund fraud and IDT activities. The administration requested an additional $90 million and 491 full-time equivalents for fiscal year 2017 to help prevent IDT refund fraud and reduce improper payments. IRS estimates that this $90 million would help it protect an additional $612 million in revenue in fiscal year 2017, as well as protect revenue in future years. The Consolidated Appropriations Act, 2016, appropriated IRS an additional $290 million for improvements to customer service, IDT identification and prevention, and cybersecurity efforts. The IRS spending plan indicates that officials will use this funding to (1) reduce the wait times and improve the performance on IRS’s Taxpayer Protection Program/Identity Theft Toll Free Line, and (2) improve network security and protect taxpayer data from unauthorized access by identity thieves, among other things. To detect and prevent IDT refund fraud, IRS has developed tools and programs, including: IDT filters: IRS uses automated filters that search for IDT refund fraud characteristics to identify suspicious returns during processing and to confirm taxpayers’ identities before issuing refunds. These characteristics are based on both IRS’s knowledge of previous refund fraud schemes and clusters of returns with similar characteristics. Taxpayer Protection Program. The Taxpayer Protection Program (TPP) reviews returns that are flagged by IRS’s IDT filters. IRS asks taxpayers to authenticate their identities—either online or by phone— by answering questions that a legitimate taxpayer is likely to know, such as previous addresses, mortgage information, and data about family members. If the taxpayer fails to authenticate himself online or by phone, IRS instructs the respondent to authenticate his identity in person at an IRS Taxpayer Assistance Center. Identity Protection Personal Identification Number (IP PIN): IP PINs are single-use identification numbers sent to IDT victims who have authenticated their identities with IRS. If a return is electronically filed (e-filed) for a Social Security Number assigned an IP PIN, it must include the IP PIN or else IRS will reject the return. If a paper return has a missing or incorrect IP PIN, IRS delays processing the return while the agency determines if it was filed by the legitimate taxpayer. As a result of an ongoing security review, IRS temporarily suspended the IP PIN tool in March 2016 while it assesses how to further strengthen its security features. IRS also works with third parties, such as industry, states, and financial institutions, to try to detect and prevent IDT refund fraud. In March 2015, the IRS Commissioner convened a Security Summit with industry and states to improve information sharing and authentication. IRS officials said that 40 state departments of revenue and 20 tax industry participants have officially signed on to the partnership. IRS is investing $16.1 million for identity theft prevention and refund fraud mitigation actions that come out of the Security Summit. These efforts include developing an Information Sharing and Analysis Center where IRS, states, and industry can share information to combat IDT refund fraud. IRS monitors the extent of IDT refund fraud through its Taxonomy. This research-based effort aims to report on the effectiveness of IRS’s IDT defenses to internal and external stakeholders, help IRS identify IDT trends and evolving risks, and refine IDT filters to better detect potentially fraudulent returns, while reducing the likelihood of flagging legitimate tax returns. As shown in figure 3, IRS’s Taxonomy estimates the number of identified IDT refund fraud cases where IRS (1) prevented or recovered the fraudulent refunds (turquoise band), and (2) paid the fraudulent refunds (purple band). IRS breaks these estimates into categories corresponding to IDT detection strategies, which occur at three key points in the life cycle of a tax refund: before accepting a tax return, during return processing, and post refund. IRS creates the Taxonomy’s estimates through sources including IRS’s Refund Fraud & Identity Theft Global Report (Global Report) and a modeling data set composed of known IDT returns and potential identity theft returns. In response to our recommendation in January 2015, IRS began using the modeling data set to improve Taxonomy estimates for refunds it paid (Categories 4 and 5 in figure 3 above). According to IRS officials, the agency developed its modeling data set to explore IDT characteristics and build the models within its IDT filters to help identify and protect against IDT refund fraud. The modeling data set consolidates data on known and potential IDT returns from various IRS systems and programs. Figure 4 shows IRS’s estimates of attempted IDT refund fraud for 2014. IRS estimates that it prevented or recovered $22.5 billion in IDT refunds. For the cost of IDT refunds paid, IRS estimated a range of values; the $3.1 billion estimate for IDT refunds paid represents the upper bound of IRS’s range for IDT refunds paid. However, IRS recognizes that there is imprecision in these estimates. Further, there is uncertainty in these estimates, as will be discussed later. One of IRS’s key defenses in reducing the risk of IDT refund fraud is TPP, which is intended to verify the identities of suspicious filers. TPP has procedures that help IRS authenticate legitimate taxpayers by requiring filers to answer questions only legitimate taxpayers are likely to know, or in some instances, checking information reported on filers’ returns with information reported by third parties, such as W-2s. Figure 5 illustrates the TPP process. Of the 650,000 filers who responded to TPP notification letters, 450,000 (69 percent) attempted remote authentication—online or by phone— whereas 200,000 (31 percent) claimed to be victims of IDT who had not filed the selected returns. To pass remote authentication, filers first complete “identity proofing” by providing basic identifying information such as their names and dates of birth. Next, they are asked to answer knowledge-based authentication questions obtained from a third-party provider. Examples of authentication questions are “Who is your mortgage lender?” or “Which of the following is your previous address?” If filers pass knowledge-based remote authentication, then IRS releases those filers’ returns for further processing before issuing refunds. If filers cannot pass, IRS will not issue a refund unless those filers pass in-person authentication or IRS receives information return documents from third parties, such as W-2s, that match filers’ return data. Officials stated that TPP authentication poses a challenge to IRS because it must authenticate almost all taxpayers in the program remotely. According to a United States Digital Service (USDS) report, it is costly for fraudsters to attempt in-person authentication at scale because it requires human interaction. As a result, when compared to in-person authentication, fraudsters are incentivized to remotely authenticate because it allows for multiple attempts, allowing the fraudster more opportunity to access the taxpayers’ information more quickly and easily respond to authentication questions. IRS has conducted research both to evaluate the effectiveness of existing TPP authentication procedures and to identify options for strengthening those procedures. Based on research efforts, IRS made improvements to its phone authentication options for filing season 2015. For example, IRS created a more challenging High Risk Authentication (HRA) quiz, which requires taxpayers to recall information from past tax filings. Prior to the 2015 filing season, IRS’s HRA quizzes sometimes included simulated questions where IRS effectively had no data available to support correct answers other than “none of the above.” For example, a simulated question might ask a filer to identify the date of birth of a dependent even though that filer had no dependent. For the 2015 filing season, IRS eliminated these questions from HRA quizzes. IRS analysis has shown that simulated questions are easier to pass than questions based on taxpayer data. In addition, IRS required some respondents to answer a higher proportion of HRA questions correctly in the 2015 filing season. Of the about 1.6 million returns selected for TPP processing in filing season 2015, IRS estimated that it potentially paid about $30 million to IDT fraudsters who filed about 7,200 returns that passed TPP authentication. However, our analysis indicates that IRS underestimated how many fraudulent IDT returns passed TPP authentication. In developing its estimates, IRS first compared TPP selections to information returns provided by third parties, such as W-2s. IRS next identified which TPP selections passed authentication but had large mismatches with information returns. IRS then manually reviewed a sample of these returns to approximate how many returns that passed authentication were filed by likely IDT fraudsters. IRS used this finding to estimate the total number and value of refunds potentially paid to IDT fraudsters who passed TPP authentication. IRS likely underestimated how many fraudulent IDT returns passed TPP authentication because the agency did not include potential IDT returns that closely matched information returns. Though based on a nongeneralizable sample, past IRS research suggests that some IDT fraudsters are able to both file tax returns that closely match information provided by third parties and pass TPP authentication. By omitting some IDT returns from its estimates, IRS likely overestimated the effectiveness of TPP defenses. IRS officials told us that they did not include close matches in their analysis because it is challenging to determine how many of these returns are filed by IDT fraudsters, and IRS does not want to present estimates based on assumptions that could be inaccurate. In March 2016, IRS officials acknowledged the desirability of expanding their estimate to include a more generalizable sample of those who successfully passed authentication and said that they will consider doing so as staff are available to do so after the filing season. While we cannot quantify the specific amount by which IRS’s analysis underestimated the number of fraudulent IDT returns that passed TPP authentication, we conducted a scenario analysis to demonstrate the effect of omitting potential IDT returns on IRS’s estimates. If we assume that 5 to 10 percent of close matches passing authentication were filed by potential IDT fraudsters, we estimate that the value of refunds potentially paid to IDT fraudsters who passed TPP authentication could be as high as between $116 and $203 million in the 2015 filing season. We chose to not base our analysis on IRS’s past research (cited in the previous paragraph) because it used a nongeneralizable sample and because its methodology for identifying close matches changed from 2014 to 2015. Our analysis indicates that, even if a small proportion of close matches that pass TPP authentication are filed by IDT fraudsters, accounting for these selections can substantially affect IRS’s estimates because close matches represent about 91 percent of all returns filed by individuals who passed authentication. Further, the extent of IRS’s likely underestimation suggests that TPP’s authentication procedures may be at greater risk of exploitation by IDT fraudsters than suggested by IRS’s estimates. To verify taxpayers’ identities remotely, TPP uses single-factor authentication procedures that incorporate one of the following authentication elements: “something you know,” “something you have,” or “something you are.” TPP’s single-factor authentication procedures are at risk of exploitation because some fraudsters obtain the PII necessary to pass the questions asked during authentication. According to IRS officials, criminals can find personal information needed to pass authentication by searching records available through the Internet or purchasing it from websites designed to conceal their content. USDS has also reported that implementing effective authentication procedures has become more challenging because criminals are able to pass authentication checks at similar rates to legitimate users due to the wide availability of personal information. Similar to TPP, IRS used single-factor authentication procedures to authenticate users of its Get Transcript service, which fraudsters defeated in 2014 to 2015, as well as its IP PIN tool that IRS temporarily suspended due to security concerns in 2016. Both USDS and TIGTA have found that IRS needs to take a stronger approach to authenticating Get Transcript users. Though IRS is undertaking efforts to strengthen Get Transcript authentication, agency officials said they are still working to determine if improvements are necessary for TPP. Because IRS must ensure legitimate taxpayers can successfully authenticate, the agency faces challenges in making remote authentication more difficult for IDT fraudsters who often possess the PII needed to appear to be legitimate taxpayers. IRS officials said it was important for TPP to minimize delays in refund processing for large numbers of legitimate taxpayers and to avoid the appearance of discriminating against specific types of filers. For example, IRS could designate all TPP filers whose return information may be harder to verify for more challenging authentication; however, IRS officials said the agency wanted to avoid the appearance of discriminating against these filers who, on average, report lower income. In addition, IRS could delay refunds for these respondents until IRS could match these selections’ return data against information provided by third parties. Because delaying refunds is likely to burden taxpayers, IRS officials said large- scale delays were not feasible. Although IRS conducted a risk assessment for TPP authentication in October 2012, the agency has not updated this assessment to reflect the current threat of IDT refund fraud—specifically, the threat that some fraudsters possess the PII necessary to pass authentication questions. In conducting its risk assessment, IRS determined that improper authentication through TPP posed low or moderate risks to both the agency and taxpayers, and therefore required no more than single-factor authentication. Since IRS conducted its original risk assessment for TPP, TIGTA conducted a more recent risk assessment of Get Transcript and determined that Get Transcript should have required multi-factor rather than single-factor authentication. Given that both programs pose similar risks—fraudsters can use vulnerabilities in both Get Transcript and TPP to more easily obtain tax refunds—it seems likely that IRS would identify a higher authentication standard for TPP when updating that program’s risk assessment. In March 2016, IRS officials stated that they were planning to conduct a risk assessment and make improvements to TPP based on the results. However, their plans for a risk assessment are not documented yet because the Identity Assurance Office has prioritized improving authentication for Get Transcript service and the IP PIN tool before TPP. Office of Management and Budget (OMB) e-authentication guidance directs agencies to conduct risk assessments on information technology systems that remotely authenticate users and to identify appropriate assurance levels. Agencies then select authentication technologies based on the levels of assurance needed and e-authentication technical guidance provided by the National Institute of Standards and Technology (NIST). Senior IRS officials stated that they disagreed that OMB guidance and NIST e-authentication standards are applicable to TPP phone authentication. However, we believe the guidance and standards are applicable because TPP uses similar processes (e.g., knowledge- based authentication questions) to remotely authenticate taxpayers— whether taxpayers themselves type in answers to questions online or whether the taxpayer answers the questions over the phone and IRS Customer Service Representatives import the information into an Internet application to check those answers. Following a consistent standard for both online and phone authentication would also help prevent IDT fraudsters from shifting authentication attempts to the option that requires a less rigorous standard. In addition, federal internal control standards, best practices for risk management, and IRS’s own risk management guidance require or recommend that agencies regularly assess risks to their programs. Standards for Internal Control in the Federal Government state that agency management should assess the risks that the agency faces from both external and internal sources. Best practices in risk management recommend that fraud risk assessments generally include assessing risks’ likelihoods and impacts, determining the agency’s risk tolerance, and examining the suitability of existing fraud controls. In addition, they recommend that agencies plan regular fraud risk assessments, since allowing extended periods to pass between assessments could result in control activities that do not effectively address a program’s risks. IRS’s Enterprise Risk Management Program: Concept of Operations also states that IRS’s Office of the Chief Risk Officer is likewise committed to timely risk reporting. By conducting an updated risk assessment for TPP in accordance with e- authentication and risk management standards, IRS could identify appropriate opportunities to strengthen TPP authentication and prevent IDT fraudsters from passing and potentially receiving millions of dollars in refunds. Depending on the assessment’s results, IRS could implement stronger authentication procedures. For example, a multi-factor authentication standard for TPP’s remote authentication options would utilize a second element to authenticate filers, such as requiring filers to provide proof of “something they have” in addition to testing “what they know.” Strengthening TPP authentication could help IRS prevent millions of dollars from being paid to IDT fraudsters each filing season. In addition, strengthening TPP could improve IRS’s return on investment for fraud filters by ensuring that efforts to flag fraudulent returns result in fewer refunds paid to IDT fraudsters. Fewer legitimate taxpayers would also become victims of IDT refund fraud if TPP stopped more IDT refund fraud returns. In response to recommendations made in our previous report, IRS is working to improve Identity Theft Taxonomy (Taxonomy) estimates of IDT fraud. In that report, we found that IRS’s 2013 Taxonomy estimates met several GAO Cost Estimating and Assessment Guide (GAO Cost Guide) best practices, such as regularly updating the methodology to better reflect evolving fraud schemes. However, we also found limitations and recommended that IRS improve the estimates by (1) reporting the inherent imprecision and uncertainty of estimates, and (2) documenting the underlying analyses justifying cost-influencing assumptions. IRS reported that the agency is working to implement these recommendations by October 2016. Given the challenges inherent in estimating fraudulent activity and the evolving nature of fraud schemes, IRS’s efforts to improve Taxonomy estimates are likely to be ongoing. For example, to estimate potential IDT refund fraud paid, IRS compares the information reported on tax returns with data reported by third parties on information returns, such as Form W-2, Wage and Tax Statement (W-2). However, it is difficult for IRS to determine whether discrepancies between data reported on the tax return and information returns are due to IDT, a mistake made by the legitimate taxpayer, or other types of fraud committed by the legitimate taxpayer. Moreover, IRS cannot accurately estimate amounts of undetected fraud because of situations when it has no reported information to verify income. Furthermore, to better reflect evolving IDT refund fraud schemes, IRS updates the Taxonomy methodology over time. While these updates may result in more accurate estimates, these changes confound making comparisons between filing seasons. To assess IRS’s efforts to implement our past recommendations, we reviewed IRS’s 2014 estimates focusing on those best practices that we assessed as “partially met” or less in our review of the 2013 Taxonomy. While IRS is not required to follow the GAO Cost Guide best practices, it could help IRS meet OMB and its own information quality guidelines and improve the reliability of IDT refund fraud estimates. Our assessments— summarized in table 1—note places where IRS has taken steps to improve the estimates and places where IRS can take additional action to further improve its estimates. Our assessment ratings show IRS made progress in one area and took a step back in another area compared to 2013. The ratings remained unchanged in four areas. As noted in table 1 above, IRS improved the Taxonomy to meet one of the best practice characteristics—management review. By reviewing and approving the 2014 Taxonomy estimates and the new methodology, IRS management completed a vital step in verifying how estimates were developed. This step helps ensure that management understands the estimate’s underlying risks, data sources, and methods so that they are confident that the estimates are accurate, complete, and high in quality. In the following sections, we analyze in greater detail IRS’s efforts to meet best practices outlined in table 1 as well as Taxonomy estimates’ remaining limitations. IRS adopted a new methodology to improve 2014 Taxonomy estimates of refunds paid. This new methodology uses the modeling data set, which is based on individual return-level information, to estimate more precisely how much the agency paid to IDT fraudsters. The modeling data set is an improvement over previous data sources that were based on aggregated data. As a result of this improvement, officials can more precisely estimate known IDT refunds paid. Additionally, IRS uses the modeling data set when calculating estimates of likely IDT refunds paid. IRS defines likely IDT returns as returns where the information on the tax return does not match either (1) the current year’s information reporting (i.e., information on a W-2); or (2) specific prior-year tax return characteristics. While the data source used to estimate IDT refunds paid is an improvement from the previous Taxonomy, IRS’s methodology for calculating estimates for refunds paid excludes select categories of returns that can bias results. A key assumption IRS uses when building its modeling data set for IDT refunds paid is the amount of the refund. As part of its methodology, IRS omits some returns with refund amounts that fail to meet specific refund thresholds from its fraud estimates and does not include all relevant returns in its analysis. IRS officials said IRS uses the thresholds because it wants to prioritize IRS enforcement efforts. In February 2016, IRS officials stated that they did not know how many returns were excluded from the Taxonomy. In March 2016, IRS officials said that they are evaluating the extent to which omitted returns met other criteria associated with IRS’s definitions of known and likely IDT refund fraud. According to its Strategic Plan, IRS should identify trends, detect high-risk areas of noncompliance, and prioritize enforcement approaches by applying research and advanced analytics. Further, the GAO Cost Guide states that analysis should be regularly updated to reflect significant changes in the methodology and should include all relevant costs. While thresholds may help IRS prioritize enforcement efforts on likely IDT fraud schemes, they limit IRS’s ability to estimate the entire population of IDT refunds paid. Further, incomplete Taxonomy estimates could impede IRS and congressional efforts to assess the effectiveness of its IDT defenses over time. In response to our discussion, IRS officials said that they are considering removing some thresholds and including those returns when calculating estimates of IDT refunds paid for the 2015 Taxonomy estimates. We also found accuracy issues with IRS’s estimate of IDT refunds prevented that are likely to result in overestimates. To produce this estimate, IRS uses the Global Report, which overestimates the amount of IDT refunds prevented because it overcounts some IDT returns. Overcounting occurs because the Global Report aggregates return-level data to create a monthly inventory of confirmed IDT returns. According to IRS officials, the Global Report counts each time a return is caught by IRS defenses as a separate instance of refund fraud. For example, if an IDT return is flagged as IDT in both IRS’s Electronic Fraud Detection System and its Dependent Database, this return is counted as two IDT returns, even though it is the same return. E-file rejects are also overcounted because a single return can be rejected multiple times. IRS officials noted that there would be benefits of using return-level data to estimate refunds prevented in the Taxonomy, such as avoiding overcounting. However, officials said they use the Global Report to develop estimates of prevented IDT refund fraud because it represents IRS’s official record of IDT fraud and because IRS has invested substantial resources in improving the report. We agree that the Global Report is an important investment for monitoring the effectiveness of IRS’s many defenses against fraud, both individually and as a system; however, overcounting the incidence of fraud inflates IRS’s Taxonomy estimates of the cost of IDT refund fraud, and could potentially bias resource allocation and other decisions. For example, if IRS thinks it is catching 90 percent of estimated IDT refund fraud attempts, agency officials may decide to allocate resources differently than if IRS is, in fact, catching 50 percent. Our data reliability testing found that the Global Report’s counts for known IDT returns where IRS prevented the refund were larger than the counts from the modeling data set. Of the estimated $22.5 billion refunds prevented or recovered in 2014, the Global Report included 2.0 million returns worth $11.4 billion in its known IDT return population, whereas the modeling dataset included 1.6 million returns and $8.9 billion in its population. Officials acknowledged that they also believe this discrepancy is due to the overcounting in the Global Report and could also be caused by the modeling dataset’s exclusion of returns that fail to meet specific refund thresholds, as described above. As noted earlier, IRS’s Strategic Plan notes that IRS should “identify trends, detect high-risk areas of noncompliance, and prioritize enforcement approaches by applying research and advanced analytics.” It also states that IRS should strengthen refund fraud prevention by bolstering analytics capability, making full use of existing data sources, and exploring potential new data sources and techniques. Further, the GAO Cost Guide states that estimates be based on primary data sources and contain few mistakes. By using aggregated data to develop the Global Report, the agency’s official record of IDT returns is less accurate than if IRS used return-level data. Further, by using the Global Report to calculate Taxonomy estimates for refunds prevented, IRS may have overestimated the $22.5 billion in refunds prevented or recovered in the 2014 filing season. As described above, inaccurate Taxonomy estimates could impede decision makers’ ability to monitor the effectiveness of IDT defenses. In its Taxonomy documentation, IRS notes most—but not all— assumptions used to make estimates. For example, IRS does not document that refunds outside of thresholds, as described above, are excluded from IRS’s estimates of refunds paid by IRS. In addition, IRS does not always provide rationales or analyses to support the assumptions it does document. For example, IRS does not provide a rationale for the average refund value used to estimate the cost of electronically filed returns that IRS rejects (i.e., e-file rejects) and categorizes as IDT returns, which affects the total value of IRS’s refunds prevented estimates. Our analyses show that using different refund assumptions can affect the refunds prevented estimate by billions of dollars. Because IRS does not have reliable data on the refund values associated with e-file rejects, it uses the average refund value of returns detected by various IDT defenses. As noted in table 2, IRS’s estimate assumes the average refund value for all IDT defenses ($5,959), which results in $7.3 billion dollars prevented on 1.2 million e-filed returns. However, the average refund value of e-file returns detected by IRS IDT defenses varies— indicating uncertainty in the estimates. For example, if IRS used the different average refunds in table 2 to develop its e-file reject estimate, the total could range from $4.1 billion to $7.5 billion. We previously recommended that IRS document the analysis underlying the cost-influencing assumptions. As stated above, IRS officials told us they are working to implement this recommendation by October 2016. Given the evolving nature of IDT refund fraud, documenting Taxonomy assumptions and the rationales used to develop those assumptions in accordance with our prior recommendations would enable IRS management and policymakers to determine whether the assumptions remain valid or need to be revised or updated. IRS is still working to improve its reporting of the inherent imprecision and uncertainty of its Taxonomy estimates. Previously, we found that IRS presented 2013 Taxonomy estimates as point estimates, which did not represent the Taxonomy’s inherent uncertainty. We recommended that IRS report the inherent imprecision and uncertainty of the estimates and noted that one way IRS could do this would be to present a range of values for its Taxonomy estimates. High-quality cost estimates usually fall within a range of possible costs, with the point estimate between the best and worst case extremes. Having a range of costs around a point estimate is more useful to decision makers because it indicates the uncertainty in the estimates by conveying its level of confidence or by conveying the level of confidence of the most likely cost. Knowing the uncertainty related to Taxonomy estimates could affect different decisions about how to allocate resources to combat IDT refund fraud. For example, if there is 80 percent confidence in IRS’s estimates, then decision makers may make different decisions than if there is 50 percent confidence in the estimates. Under its revised methodology, IRS partially addressed our previous recommendation by presenting refunds-paid estimates as a range rather than a single point estimate to reflect the uncertainty in IRS’s estimate of the revenue lost to IDT refund fraud. In addition, IRS took steps to incorporate better quality data into its refunds paid estimate by utilizing both the modeling data set’s return-level information and results from a new sampling effort. However, these ranges may not give decision makers a truly accurate understanding of what IRS knows and does not know about IDT refund fraud because they are not derived from a cost risk and uncertainty analysis. Such an analysis accounts for the cumulative impact that multiple assumptions might have on IRS’s estimates. For example, ranges do not account for uncertainty regarding the extent to which IRS’s estimates account for all IDT fraud schemes. Additionally, IRS officials manually review some returns to determine whether or not the returns are IDT or non-IDT returns. IRS’s ranges also do not account for the uncertainty or the risk that manual reviewers may not accurately characterize returns as IDT returns and non-IDT returns. In addition, IRS does not conduct a sensitivity analysis for Taxonomy categories that include assumptions. A sensitivity analysis reveals critical assumptions and cost drivers that most affect estimate results, and can help managers take steps to ensure the estimates’ quality. By conducting a sensitivity analysis, IRS will know which assumptions and which factors affect the Taxonomy the most so IRS can devote resources to combating IDT refund in those areas and work to make the estimates more accurate in those areas. Until IRS addresses our prior recommendations and provides an indication of uncertainty in the Taxonomy estimates, the false sense of precision could affect decisions about how to allocate resources to combat IDT refund fraud. IRS officials told us in February 2016 that they plan to conduct a sensitivity analysis and a risk and uncertainty analysis for the assumptions that are used when IRS calculates the updated Taxonomy estimates for 2015. IRS’s continued efforts to improve TPP are critical to combatting IDT refund fraud. Though IRS has made improvements to TPP, evidence suggests that the agency’s efforts to authenticate taxpayers in filing season 2015 may not have kept pace with the evolving threat of IDT refund fraud. Since IRS last conducted a risk assessment for TPP, PII has become more widely disseminated, and IRS has changed TPP procedures. In addition, though IRS is undertaking efforts to strengthen Get Transcript, a program that poses risks similar to TPP, IRS has not determined whether authentication improvements are necessary for TPP. Documenting time frames and conducting an updated e-authentication risk assessment for TPP’s remote authentication options would enable IRS to identify opportunities and take actions to strengthen TPP authentication in accordance with appropriate standards. In turn, strengthened authentication would help IRS reduce revenue lost to IDT fraudsters, improve the efficiency of fraud filter investments, and reduce the number of legitimate taxpayers who become victims of IDT refund fraud. IRS’s monitoring of the extent of IDT refund fraud is key to supporting decision makers’ ability to determine how to combat IDT refund fraud. IRS has invested a considerable effort in monitoring and reporting the extent of IDT refund fraud through its Taxonomy estimates. However, the accuracy of IRS’s IDT refund fraud reporting in the Taxonomy estimates could be improved. For example, using return-level data, such as the modeling data set, could improve the accuracy of the Taxonomy’s refunds paid estimates. More accurate Taxonomy estimates would help IRS better understand how and to what extent IDT refund fraud is evading IRS defenses. This would allow it to focus attention on where the risk is greatest and improve the design of its IDT filters. Additionally, reducing overcounting and ensuring all relevant IDT returns—even those that fail to meet specific refund thresholds—are included in Taxonomy estimates could help IRS communicate more accurate information on the amount and cost of IDT refund fraud to decision makers. Finally, implementing our past recommendations will help IRS further improve the reliability of its estimates. To further deter noncompliance in the Taxpayer Protection Program, we recommend that the Commissioner of Internal Revenue take the following two actions in accordance with OMB and NIST e-authentication guidance: 1. conduct an updated risk assessment to identify new or ongoing risks for TPP’s online and phone authentication options, including documentation of time frames for conducting the assessment, and 2. implement appropriate actions to mitigate risks identified in the assessment. To improve the quality of the Taxonomy’s IDT refund fraud estimates, we recommend that the Commissioner of Internal Revenue take the following two actions: 1. remove refund thresholds from criteria used to develop IRS’s refunds- paid estimates, and 2. utilize return-level data—where available—to reduce overcounting and improve the quality and accuracy of the refunds-prevented estimates. We provided a draft of this product to the Commissioner of Internal Revenue, the Attorney General, and the Director of the Federal Bureau of Investigation for review and comment. In its written comments, reproduced in appendix III, IRS agreed with our TPP recommendations and neither agreed nor disagreed with our Taxonomy recommendations. IRS stated that it will conduct an updated risk assessment for TPP’s online electronic authentication application, in accordance with OMB and NIST guidelines. Regarding TPP’s phone authentication option, IRS reported that a portion of the telephone authentication option will be included in the assessment because IRS employees use a web interface. As noted in the report, we believe that following a consistent authentication standard for both online and phone authentication would help prevent IDT fraudsters from shifting authentication attempts to the option that requires a less rigorous standard. IRS officials stated that they will implement mitigation actions identified during the assessment, to the degree feasible. We continue to emphasize the importance of implementing appropriate actions to mitigate identified risks because doing so would improve TPP authentication and prevent additional fraudulent refunds from being issued. Consistent with our recommendation, IRS stated that it has reduced the lower threshold used to develop its IDT refund-paid estimate in its 2014 modeling dataset. IRS did not change its upper threshold. IRS also stated that the risk of this remaining threshold excluding relevant IDT returns is mitigated because IRS manually reviews such returns. We support IRS’s reduction of the lower threshold and its manual review of high-value refunds. With regard to our recommendation to use return-level data to reduce overcounting and improve the accuracy of the refunds-prevented estimate, IRS officials said that they are discussing the impact of the recommendation and determining if it is feasible to implement. As previously noted, it is important for IRS to provide accurate estimates of the IDT fraud it prevented or recovered. By not using return-level data, the Global Report overcounts some IDT returns. As a result, IRS is providing Congress and other stakeholders with overestimates of the amount of IDT refund fraud it prevented or recovered. The Department of Justice provided technical comments for itself and the Federal Bureau of Investigation, which we incorporated as appropriate. As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies to the Commissioner of Internal Revenue, the Attorney General of the United States, and the Director of the Federal Bureau of Investigation. In addition, the report will be available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-9110 or mctiguej@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix IV. This report (1) evaluates the performance of Internal Revenue Service’s (IRS) Taxpayer Protection Program (TPP) and (2) assesses IRS efforts to improve its estimates of identity theft (IDT) refund fraud costs for 2014. The report discusses IDT refund fraud and not employment fraud. Detailed information on IRS’s enforcement efforts was excluded from the report because of sensitivity concerns. To evaluate TPP’s performance, we reviewed IRS studies designed to identify and support ongoing identity authentication refinements to TPP. We compared specifics of IRS’s TPP against relevant guidance on enterprise risk management, electronic authentication, and internal controls. We assessed IRS’s TPP analysis by (1) reviewing relevant IRS documentation, (2) conducting manual testing to identify obvious errors, and (3) interviewing IRS officials. During the course of our work, we found that IRS likely underestimated the value of refunds issued to IDT fraudsters in filing season 2015 via TPP because the agency did not account for all refunds potentially paid to IDT refund fraudsters who passed TPP authentication. To assess how excluding potential IDT refunds affected IRS’s estimates of the amount potentially paid to IDT fraudsters who were able to pass TPP authentication, we conducted a scenario analysis. We chose not to base our scenarios on IRS’s past research because it used a nongeneralizable sample and because its methodology for identifying close matches changed from 2014 to 2015. Instead, we identified scenarios of 5 to 10 percent to illustrate the potential outcomes if relatively small percentages of these returns were actually IDT. To assess IRS’s efforts to improve its Identity Theft Taxonomy (Taxonomy) estimates of IDT refund fraud for 2014, we reviewed the Taxonomy’s methodology and estimates. We then evaluated them against selected best practices in the GAO Cost Estimating and Assessment Guide (GAO Cost Guide) that were applicable to the Taxonomy and consistent with IRS and Office of Management and Budget (OMB) information quality guidelines. These best practices are relevant because the Taxonomy is an estimate of the amount of revenue lost to IDT refund fraud—a cost to taxpayers. To develop this guide, our cost experts assessed the measures consistently applied by cost- estimating organizations throughout the federal government and industry; based upon this assessment, the cost experts then considered best practices for the development of reliable cost estimates. We focused our analysis on those best practices that we assessed as “partially met” or less in our review of the 2013 Taxonomy (see text box). In comparing 2014 estimates with 2013 estimates, we could not determine if differences in Taxonomy estimates between these years were due to changes in methodology, IDT fraud trends, or the efficacy of IRS’s IDT defenses. During our review of the 2013 Taxonomy, we discussed the GAO Cost Guide’s best practices with IRS officials who generally agreed with their applicability to the Taxonomy. Best Practices in Cost Estimating Used to Review the 2014 Taxonomy We assessed the Taxonomy against the following best practices for objective, reliable cost estimates: Include all relevant costs. Document all cost-influencing ground rules and assumptions. Include a sensitivity analysis. Include a risk and uncertainty analysis. Are not overly conservative or optimistic, and are based on most likely costs. Provide evidence that the cost estimate was reviewed and accepted by management. To analyze IRS’s Taxonomy against the best practices, we reviewed Taxonomy documentation, conducted manual and electronic data testing, reviewed coding for obvious errors, compared underlying data to IRS’s Refund Fraud & Identity Theft Global Report, and interviewed IRS officials to understand the methodology used to create the 2014 estimates and how that methodology changed from that used to develop the 2013 Taxonomy. We did not replicate IRS’s Taxonomy estimates using tax return data; rather, our focus was on IRS’s methodology for calculating the estimates. We developed an overall assessment rating for each best practice using the following definitions: Not met. IRS provided no evidence that satisfied any portion of the best practice. Minimally met. IRS provided evidence that satisfied a small portion of the best practice. Partially met. IRS provided evidence that satisfied about half of the best practice. Substantially met. IRS provided evidence that satisfies a large portion of the best practice. Met. IRS provided complete evidence that satisfies the entire best practice. We conducted this performance audit from March 2015 to May 2016 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. This report is the third in a series of our reports on identity theft (IDT) refund fraud. Since August 2014, we have issued two reports that included eight recommendations on actions the Internal Revenue Service (IRS) can take to monitor and combat IDT refund fraud. As of March 2016, IRS implemented three of the eight recommendations, and it is implementing the remaining five recommendations. Table 3 summarizes our prior recommendations and their implementation status. Earlier access to W-2s could help IRS match W-2 information to taxpayers’ returns and identify discrepancies before issuing billions of dollars of fraudulent IDT refunds. How IRS implements W-2 matching could affect the costs and benefits for itself and other stakeholders (e.g., logistical challenges for the Social Security Administration, which processes W-2 data before transmitting them to IRS). Implemented. In September 2015, IRS provided us with a document detailing the costs and benefits of W-2 acceleration. The document discussed the IRS systems and work processes that will need to be adjusted to accommodate earlier, prerefund matching of W-2s; the time frames for when these changes could be made; potential impacts on taxpayers, IRS, and other parties; and what other changes will be needed (such as delaying refunds) to ensure IRS can match tax returns to W-2 data before issuing refunds. Implemented. See description above. fully assess the costs and benefits of accelerating W-2 deadlines and provide information to Congress on potential impacts on taxpayers, IRS, the Social Security Administration, and third parties. fully assess the costs and benefits of accelerating W-2 deadlines and provide information to Congress on what other changes will be needed (such as delaying the start of the filing season or delaying refunds) to ensure IRS can match tax returns to W-2 data before issuing refunds. See description of benefits above. See description of benefits above. Implemented. See description above. provide aggregated information on (1) the success of external party leads in identifying suspicious returns and (2) emerging trends (pursuant to section 6103 restrictions). This feedback would help financial institutions know if the leads they provide to IRS are useful and would help them improve their own detection tools. develop a set of metrics to track external leads by the submitting third party. Implementation in progress. In November 2014, IRS reported that it would implement our recommendation by November 2015. In November 2015, IRS reported that it had developed a database to track leads submitted by financial institutions and the results of those leads. IRS also stated that it had held two sessions with financial institutions to provide feedback on external leads provided to IRS. In December 2015, IRS officials stated that the agency had sent a customer satisfaction survey asking financial institutions for feedback on the external leads process and was considering other ways to provide feedback to financial institutions. However, to date IRS has not provided feedback to the majority of relevant lead-generating third parties. Implementation in progress. See description above. This feedback would help financial institutions know if the leads they provide to IRS are useful and would help them improve their own detection tools. GAO-15-119 – We recommended that the Commissioner of Internal Revenue should follow relevant best practices outlined in GAO’s Cost Assessment Guide: Best Practices for Estimating and Managing Program Costs (GAO Cost Guide) by documenting the underlying analysis justifying cost-influencing assumptions. Given the evolving nature of IDT refund fraud, documenting the rationales for assumptions would help IRS management and policymakers determine whether the assumptions remain valid or need to be updated. Implementation in progress. In April 2015, IRS reported that it would implement our recommendation by mid- October 2016. In October 2015, IRS provided updated Taxonomy estimates for 2014. This new analysis and documentation noted most but not all assumptions. For example, it did not note that some returns resulting in paid refunds were excluded because they were outside thresholds. In addition, the rationales supporting some assumptions, such as the estimated refund values associated with e-file reject returns, were not documented. follow relevant best practices outlined in the GAO Cost Guide by reporting the inherent imprecision and uncertainty of the estimates. For example, IRS could provide a range of values for its Taxonomy estimates. Reporting the uncertainty that is already known from IRS analysis (and conducting further analyses when not cost prohibitive) might help IRS communicate IDT refund fraud's inherent complexity. While a point estimate might lead to one decision, a range that reflects the uncertainty may lead decision makers to a different decision. should estimate and document the costs, benefits and risks of possible options for taxpayer authentication, in accordance with Office of Management and Budget and National Institute of Standards and Technology guidance. Analysis of costs, benefits, and risks could help inform IRS’s and Congress’s decisions about whether and how much to invest in the various authentication options. Implementation in progress. In April 2015, IRS reported that it would implement this recommendation by mid- October 2016. In September 2015, IRS provided updated Taxonomy estimates for 2014 that presented the estimates for refunds paid and not recovered as ranges. While these ranges account for risk surrounding known IDT returns that were paid to actual fraudsters, these ranges do not take into account the cumulative impact of additional assumptions on the estimate. For example, IRS’s analysis does not account for the impact of how IRS defines the population of likely IDT returns. IRS should conduct additional analyses to understand the estimates’ uncertainty and report the imprecision and uncertainty of the estimates. Specifically, sensitivity analysis could help IRS understand how each assumption affects the estimates. A risk and uncertainty analysis could help IRS understand the cumulative impact of all assumptions on the Taxonomy estimates. Implementation in progress. In April 2015, IRS reported that it would implement our recommendation by November 2015. In late 2015, IRS officials told us that the agency has developed guidance for the authentication group to assess costs, benefits, and risks, and that its analysis will inform decision making on authentication-related issues.While IRS is making progress, it has yet to analyze the costs, benefits, and risks of the range of authentication options available and has not used analysis to select which authentication options to use for specific types of taxpayer interactions. We continue to monitor IRS’s progress. In addition to these eight recommendations, we also identified a matter for congressional consideration to help IRS combat IDT refund fraud. In August 2014, we reported that Congress should consider providing the Secretary of the Treasury with the regulatory authority to lower the threshold for electronic filing of the W-2, from 250 returns annually to between 5 to 10 returns, as appropriate. As discussed in table 3 above, earlier access to W-2s could help IRS match W-2 information to taxpayers’ returns and identify discrepancies before issuing billions of dollars of fraudulent IDT refunds. However, paper W-2s are unavailable for IRS matching until later in the year due to the additional time needed to process paper forms. The Social Security Administration estimated that to meaningfully increase the electronic filing (e-filing) of W-2s, the threshold would have to be lowered to include those filing 5 to 10 W-2s. In addition, the Social Security Administration estimated an administrative cost savings of about 50 cents per e-filed W-2. Based on these cost savings and the ancillary benefits they provide in supporting IRS’s efforts to conduct more prerefund matching, a change in the e-filing threshold is warranted. As of March 2016, Congress has not acted on this matter for consideration. James R. McTigue, Jr., (202) 512-9110 or mctiguej@gov. In addition to the individual named above, Neil Pinney, Assistant Director; Shannon Finnegan, Analyst-in-Charge; Lisette Baylor; Dawn Bidne; Amy Bowser; Sara Daleski; Deirdre Duffy; Michele Fejfar; Lauren Friedman; Robert Gebhart; Jason Lee; Dae Park; Jeffrey Daniel Paulk; Robert Robinson; and Albert Sim made key contributions to this report. Joanna Berry, Gary Bianchi, Mark Canter, Nina Crocker, Jeffrey Knott, Paul Middleton, Sabine Paul, Sara Pelton, Bradley Roach, and Julie Spetz also provided assistance. 2016 Annual Report: Additional Opportunities to Reduce Fragmentation, Overlap, and Duplication and Achieve Other Financial Benefits. GAO-16-375SP. Washington, D.C.: April 13, 2016. Information Security: IRS Needs to Further Improve Controls over Financial and Taxpayer Data. GAO-16-398. Washington, D.C.: March 28, 2016. Financial Audit: IRS’s Fiscal Years 2015 and 2014 Financial Statements. GAO-16-146. Washington, D.C.: November 12, 2015. Information Security: IRS Needs to Continue Improving Controls over Financial and Taxpayer Data. GAO-15-337. Washington, D.C.: March 19, 2015. High-Risk Series: An Update. GAO-15-290. Washington, D.C.: February 11, 2015. Identity Theft and Tax Fraud: Enhanced Authentication Could Combat Refund Fraud, but IRS Lacks an Estimate of Costs, Benefits and Risks. GAO-15-119. Washington, D.C.: January 20, 2015. Identity Theft: Additional Actions Could Help IRS Combat the Large, Evolving Threat of Refund Fraud. GAO-14-633. Washington, D.C.: August 20, 2014. Financial Audit: IRS’s Fiscal Years 2013 and 2012 Financial Statements. GAO-14-169. Washington, D.C.: December 12, 2013. Internal Revenue Service: 2013 Tax Filing Season Performance to Date and Budget Data. GAO-13-541R. Washington, D.C.: April 15, 2013. Identity Theft: Total Extent of Refund Fraud Using Stolen Identities is Unknown. GAO-13-132T. Washington, D.C.: November 29, 2012. Financial Audit: IRS’s Fiscal Years 2012 and 2011 Financial Statements. GAO-13-120. Washington, D.C.: November 9, 2012. Taxes and Identity Theft: Status of IRS Initiatives to Help Victimized Taxpayers. GAO-11-721T. Washington, D.C.: June 2, 2011. Taxes and Identity Theft: Status of IRS Initiatives to Help Victimized Taxpayers. GAO-11-674T. Washington, D.C.: May 25, 2011. Tax Administration: IRS Has Implemented Initiatives to Prevent, Detect, and Resolve Identity Theft-Related Problems, but Needs to Assess Their Effectiveness. GAO-09-882. Washington, D.C.: September 8, 2009.

Summary: IRS estimates that, in 2014, it prevented or recovered $22.5 billion in attempted IDT refund fraud, but paid $3.1 billion in fraudulent IDT refunds. Because of the difficulties in knowing the amount of undetected fraud, the actual amount could differ from these point estimates. IDT refund fraud occurs when a refund-seeking fraudster obtains an individual's identifying information and uses it to file a fraudulent tax return. Despite IRS's efforts to identify and prevent IDT refund fraud, this crime is an evolving and costly problem. GAO was asked to examine IRS's efforts to combat IDT refund fraud. This report (1) evaluates the performance of IRS's TPP and (2) assesses IRS's efforts to improve its estimates of IDT refund fraud costs for 2014. To evaluate TPP, GAO reviewed IRS studies, reviewed relevant guidance, and met with agency officials. Further, GAO conducted a scenario analysis to understand the effect of different assumptions on IRS's TPP analysis. To assess IRS's IDT cost estimates, GAO evaluated IRS's methodology against selected best practices in the GAO Cost Guide . Taxpayer Protection Program (TPP) . While the Internal Revenue Service (IRS) has made efforts to strengthen TPP—a program to authenticate the identities of suspicious tax return filers and prevent identity theft (IDT) refund fraud—fraudsters are still able to pass through and obtain fraudulent refunds. TPP authenticates taxpayers by asking questions only a real taxpayer should know; however, fraudsters can pass by obtaining a taxpayer's personally identifiable information (PII). IRS estimates that of the 1.6 million returns selected for TPP, it potentially paid $30 million to IDT fraudsters who filed about 7,200 returns that passed TPP authentication in the 2015 filing season; however, GAO's analysis suggests the amount paid was likely to be higher. Although IRS conducted a risk assessment for TPP in 2012, IRS has not conducted an updated risk assessment that reflects the current threat of IDT refund fraud—specifically, the threat that some fraudsters possess the PII needed to pass authentication questions. Federal e-authentication guidance requires agencies to assess risks to programs. An updated risk assessment would help IRS identify opportunities to strengthen TPP. Strengthened authentication would help IRS prevent revenue loss and reduce the number of legitimate taxpayers who become fraud victims. IDT Refund Fraud Cost Estimates . In response to past GAO recommendations, IRS adopted a new methodology in an effort to improve its 2014 IDT refund fraud cost estimates. However, the estimates do not include returns that fail to meet specific refund thresholds. IRS officials said the thresholds allow them to prioritize IRS's enforcement efforts. However, using thresholds could result in incomplete estimates. Improved estimates would help IRS better understand how fraud is evading agency defenses. The GAO Cost Guide states that cost estimates should include all relevant costs. Additionally, IRS's estimates of refunds it protected from fraud are based on the Global Report , which counts each time a fraudulent return is caught by IRS and thus counts some returns multiple times. IRS uses this data source because it is IRS's official record of IDT refund fraud. The GAO Cost Guide states that agencies should use primary data for estimates and the data should contain few mistakes. By using the Global Report , as opposed to return-level data, IRS produces inaccurate estimates of IDT refund fraud, which could impede IRS and congressional efforts to monitor and combat this evolving threat. GAO recommends that IRS update its TPP risk assessment and take appropriate actions to mitigate risks identified in the assessment. GAO also recommends that IRS improve its IDT cost estimates by removing refund thresholds and using return-level data where available. IRS agreed with GAO's TPP recommendations and will update its risk assessment. IRS took action consistent with GAO's IDT cost estimate recommendations.