Since 1973, IRS Policy Statement P-1-20 has prohibited using records of tax enforcement results to evaluate enforcement officers or to impose or suggest production goals or quotas. An enforcement officer is an employee who exercises judgment with regard to determining tax liability or the ability to pay. Enforcement officers include employees directly involved in collection, examination, or criminal investigation functions as well as appeals officers and reviewers. Each function has its own set of tax enforcement results. For example, for the examination function, tax enforcement results include data such as the number of hours spent per return, amount of dollars proposed per return, and number of cases closed. For the collection function, tax enforcement results include the number of liens, levies, and seizures; amount of dollars collected; and number of cases closed. As a general rule, the policy prohibited managers from using any statistic that measures quantity, time per case, type of disposition, or dollar value. Instead, employees are to be evaluated on the basis of the quality of the work, including the use of enforcement tools, timeliness of action, and application of tax law. However, the policy statement permitted IRS officials to use statistics on tax enforcement results when carrying out certain management activities, such as preparing long-range and financial plans, allocating resources, and evaluating program effectiveness. In addition, managers were permitted to use tax enforcement results when evaluating employee performance during a case review, for example by noting whether a revenue officer appropriately levied a bank account given the facts of the specific case. Managers were also permitted to discuss with employees the number of cases processed, the amount of time spent on cases, and the kind of results obtained, as long as the discussion was based on a review of the results that were appropriate for specific cases processed by the employees and goals and quotas were not involved. In 1988, along with other changes, the Taxpayer Bill of Rights also prohibited the use of records of tax enforcement results to evaluate employees, but the prohibition only applied to employees directly involved in collection activities and their immediate supervisors. The law specified that IRS would be in compliance with the legislative requirements as long as IRS followed its policy statement. To help ensure compliance, the law also required each district director to certify each quarter whether violations had occurred and the corrective action taken, if needed. Because IRS service centers also have collection responsibilities, IRS imposed the same certification requirement on its service center directors. Generally, the self-certification process requires managers to submit certifications to the next higher level for review. Quarterly, group managers are to submit certifications to their branch chiefs, branch chiefs are to submit certifications to their division chiefs, and division chiefs are to submit certifications to their directors. Directors, relying on lower-level managers’ certifications, are to submit their certifications to the Commissioner of Internal Revenue through the Assistant Commissioner (Collection). During hearings held by the Senate Committee on Finance in September 1997, witnesses alleged that IRS’ focus on enforcement statistics was encouraging enforcement officers to take unnecessary and illegal enforcement actions against taxpayers. Prompted by these hearings, IRS’ Internal Audit reviewed the use of enforcement statistics by collection personnel in 12 districts, as well as at national and regional offices, and found an atmosphere largely driven by statistical measures. Internal Audit also found similar conditions in its July 1998 report on the use of enforcement results by the examination functions in 12 district offices. The IRS Restructuring and Reform Act of 1998 (Public Law 105-206, July 22, 1998) repealed the Taxpayer Bill of Rights prohibitions, which only covered collection employees, and instead expanded the prohibitions to cover all employees. The act also expanded the quarterly certifications to cover each appropriate supervisor. To determine the extent to which IRS’ current certification process identified violations, we reviewed fiscal year 1996 and 1997 quarterly certifications filed by district and service center directors—a total of 288 for the district offices and 80 for the service centers. We also obtained summary information on the potential violations for the first quarter of fiscal year 1998. To obtain information on IRS’ quarterly certification policy and procedures, we interviewed officials from IRS’ National Office; the Western Regional Office; the Georgia, Kansas/Missouri, and Northern California District Offices; and the Atlanta and Fresno Service Centers. We visited these offices because of their close proximity to our offices. We also surveyed officials in all districts and service centers on the quarterly certification process they had in place as of the last quarter of calendar year 1997. Appendix I contains a summary of the violations reported in the quarterly certifications we reviewed. To determine IRS employees’ perceptions of the use of tax enforcement results, we surveyed a statistically representative sample of 1,104 of 20,974 IRS front-line employees and group managers in the collection and examination functions. The survey included questions on whether employees perceived that tax enforcement results influenced their evaluations, the basis for those perceptions, and the use of tax enforcement results to establish group goals. We also analyzed the comments provided voluntarily by IRS employees on the survey. Appendixes II through VI contain our sampling methodology and survey results. All results are reported at the 95-percent confidence level. Sampling errors for all of our estimates are less than plus or minus 10 percentage points unless otherwise reported. To determine how tax enforcement results were used in written employee evaluations, we reviewed the 2 most recent evaluations for a statistically representative sample of 300 of 18,292 examination and collection front-line employees. We reviewed the evaluations to determine whether tax enforcement results were used to rate the employee in violation of IRS guidance. We discussed potential violations with IRS officials and considered their opinions on whether they were violations. We also reviewed the evaluations to determine whether the narratives used tax enforcement results to generally characterize employee performance or to discuss specific case examples. Appendix II contains our sampling methodology and appendix VII contains the results of our review of employee evaluations. All results are reported at the 95-percent confidence level and, unless noted otherwise, the data have sampling errors of no more than 8 percent. To evaluate IRS’ efforts to revise the certification process, we discussed IRS proposals with senior-level officials who were involved in developing the new process. We reviewed the existing and proposed guidance provided to managers and identified the differences between the existing and proposed certification processes. We also met with National Treasury Employees Union (NTEU) officials to obtain their perspective on the existing and proposed processes. We compared the proposed changes with the results of our review of the current process to determine if the limitations we noted were being addressed. There were some limitations to our analysis. We did not verify employees’ responses to our survey. However, we discussed these responses with senior IRS officials. We also did not verify IRS’ personnel database from which we drew our sample. We did our work between December 1997 and August 1998 in accordance with generally accepted government auditing standards. We requested comments on a draft of this report from the Commissioner of Internal Revenue. The comments we received are in appendix VIII and are evaluated at the end of this letter. Our review of quarterly certifications for fiscal years 1996 and 1997 found that directors reported 11 potential violations in 368 quarterly certifications (see app. I). IRS determined that four of the incidents were actual violations, two were unfounded allegations, and five were reported because the directors believed the actions could be misconstrued as violations. However, there are some indications that the certification process may not have identified all violations. For example, for the first quarter of fiscal year 1998, directors reported 28 potential enforcement results violations, more than double the total reported for the prior 8 quarters. According to IRS officials, the increase in reported potential violations occurred as a result of the September 1997 Senate hearings on IRS abuses, which made managers more aware of their responsibilities for reporting potential violations. IRS had not completed its investigation of these potential violations at the time we completed our audit work. Also, we identified several systemic weaknesses that affected the reliability of the certification process. First, our analysis of the potential violations that were reported indicated that there was confusion among IRS officials about what constituted a violation. For example, one certification noted that a district director had asked the National Office official, who was responsible for providing guidance on the use of tax enforcement results, whether a group could establish a team goal to increase dollars collected by 10 percent as part of an organizational pilot project. Although the National Office official agreed that this was acceptable because neither awards nor individual evaluations would be based on achieving the goal, IRS’ Chief Counsel subsequently determined that the group goal was a production quota or goal and, as such, a violation. Second, IRS guidance did not specify how the certifications were to be supported. For example, IRS policy did not mandate what action directors should take to support their certifications, such as reviewing a sample of evaluations or indicating on the certification the number and type of documents reviewed. As a result, National Office officials may not have been in a position to determine whether directors performed enough oversight to validate their certifications. We found that officials at all levels with certification responsibility said that they reviewed some documents to determine whether managers complied with the tax enforcement results policy. However, the nature and scope of their review varied widely. For example, some officials reviewed all performance evaluations and supporting documents for violations while other officials reviewed a random sample or reviewed a small, judgmental sample. Third, IRS did not integrate the employee performance evaluation and certification processes. For example, although IRS procedures require managers to submit an evaluation to a higher level of management for review and approval before the evaluation is given to the employee, they do not require the reviewer to assess the evaluation for potential tax enforcement results violations. Although the preparers’ and reviewers’ signatures on the evaluation form signified that the evaluation complied with procedures for conducting evaluations, the procedures did not specifically refer to the tax enforcement results policy or the certification process. Such a requirement could heighten managerial awareness of policy and legal prohibitions at the time the evaluation is prepared. Fourth, IRS guidance did not clearly inform managers that they could be subject to sanctions or the kinds of sanctions that could be imposed if they either misused tax enforcement results or signed a false or misleading certification. Neither the policy guidance nor the certification included an acknowledgement that managers could be subject to disciplinary action for the improper use of tax enforcement records, so managers might not have known the importance or potential consequences of their actions. Furthermore, the lack of clear guidance could lead to inconsistent disciplinary actions. The Guide for Penalty Determinations listed in IRS’ Interim Handbook of Employee Conduct and Ethical Behavior did not specify the types of disciplinary actions that could be imposed on managers who violate tax enforcement results policy. According to an IRS official, the guide is not intended to be an exhaustive listing of all offenses, and one general category can be used for penalty determinations in matters not otherwise covered. The penalties for this general category ranged from a reprimand, to a 14-day suspension, to a removal. Disciplinary action for the four violations IRS identified in the fiscal year 1996 and 1997 certifications ranged from counseling to removing the manager from his/her position. Our employee survey showed a widespread perception on the part of collection and examination front-line employees that tax enforcement results are considered by managers when preparing performance evaluations. We estimated that, overall, about 75 percent of front-line employees (i.e., revenue agents and tax auditors in the examination function and revenue officers in the collection function) and 81 percent of examination and collection group managers believe that one or more enforcement measures were considered to some extent by their managers when preparing their most recent performance evaluation. For those revenue officers who indicated that tax enforcement results affected their evaluation, an estimated 58 percent of them cited “number of cases closed”—a measure of productivity—as a tax enforcement result influencing their evaluations. Revenue agents and tax auditors who indicated that tax enforcement results affected their evaluations cited measures of revenue production more frequently as factors that influenced their evaluations than did revenue officers. At least an estimated 61 percent of revenue agents and tax auditors said that “additional dollars proposed per return” influenced their evaluations, while an estimated 29 percent of revenue officers said that “average dollars collected per return” influenced their evaluations. Appendix III shows the point-estimates intervals for each type of enforcement result by type of employee and manager. An estimated 77 percent of collection group managers who indicated that tax enforcement results affected their evaluations believed that the number of cases closed affected their evaluations, compared with an estimated 39 percent who believed that the average dollars collected per return affected their evaluations. Examination group managers who indicated that tax enforcement results affected their evaluations cited workload measures about as frequently as revenue production measures. About 74 percent of examination group managers identified tax enforcement results dealing with the average hours per return and about 76 percent identified additional dollars proposed per return as affecting their evaluations. IRS’ quarterly certification process focused on the use of enforcement results in formal written performance evaluations. However, our survey results indicated that although many employees had multiple bases for their perceptions on the use of tax enforcement statistics, most based their perception on verbal information received from their managers. We estimated that about 70 percent of front-line employees, who believed that tax enforcement results affected their evaluations, based their perceptions on verbal communication with their managers during group meetings and performance feedback sessions. An estimated 36 percent based their perception on written information contained in their evaluations. Likewise, managers who believed that tax enforcement results affected their evaluations had multiple bases for their perceptions. An estimated 83 percent based their perception on verbal communication with their branch and division chiefs, while an estimated 51 percent based their perception on written communications. Appendix IV contains detailed information on the basis for front-line employee and group manager perceptions. IRS’ managers are prohibited from using tax enforcement results to establish group production quotas or goals because of the possibility that group goals will be interpreted as individual goals or expectations. About one-half of front-line employees and group managers indicated that performance goals were established for their groups. Where group goals were established, at least two-thirds of the front-line employees and managers indicated that the goals included one or more enforcement results. Appendix V provides more detailed information on which enforcement results were included in group goals. In voluntary written comments to our questionnaires, front-line employees and managers included additional reasons why they believe enforcement results influenced their performance evaluations. The comments generally centered on the following scenarios: (1) supervisors or managers implied that an employee’s performance evaluation was influenced by tax enforcement results; (2) the general culture or atmosphere of IRS implied that enforcement results affected performance evaluations; (3) higher-level managers pressured group managers to increase production or revenue yield; and (4) IRS’ business plan, reports, and other documents emphasized enforcement results as goals. Front-line employees and managers also made positive and negative comments about using enforcement and workload measures to establish group goals or in performance evaluations. Generally, the number of negative comments made by front-line employees and managers exceeded the number of positive comments. Appendix VI contains examples of voluntary comments received from front-line employees and managers. Using IRS guidance for determining whether an evaluation contained a violation, we estimated that 9 percent of front-line employees received at least one written performance evaluation containing a violation in the two most recent annual evaluations. The potential effect of the violations on future employee performance could vary, however, depending on how the enforcement results were used in evaluations. Some violations could influence employees to be more aggressive in their dealings with taxpayers. For example, the following two comments taken from evaluations could create an incentive for employees to more aggressively pursue fraud cases, liens, and levies. “You had one possible fraud case during this period. Keep trying for more fraud cases.” “You do not hesitate to lien or levy and have demonstrated innovative ideas in your efforts to comply with group goals. . . Your workload management produces a closure rate that demonstrates your commitment to group goals.” Both evaluations violations link a positive evaluation to achieving a higher number of enforcement results, rather than focusing on the appropriateness of the employees’ action. In other cases, however, the potential impact of the violations on employee performance was less clear. For example, many of the violations mentioned closing cases when summarizing employees’ accomplishments, as shown in the following two examples. “The effectiveness of your planning and time utilization is demonstrated by the fact that you closed 38 key cases during the year.” “You have selected the proper technique to detect unreported income and applied your technique to detect and recognize indications of fraud. For example, , during this fiscal year you have submitted and had accepted six civil fraud referrals.” For most of the violations we identified, IRS officials said that the comments would not have been considered violations under the guidance in effect at the time the evaluations were prepared. However, the officials agreed that the comments would be considered inappropriate under IRS’ revised guidance, which is discussed in the following section. The incidence of violations was relatively low, given that 36 percent of front-line employees who believed that tax enforcement results affected their evaluations based their perception on their written evaluations. Our analysis of employee evaluations found that an estimated 69 percent of the evaluations contained two types of narrative that employees could have interpreted as inappropriate references to tax enforcement results but which did not violate IRS guidance. The first type of narrative involved the use of general or case-specific references to enforcement-related activities, such as the amounts of revenue collected or use of collection tools. These references, which are not based on statistics such as group goals, are not violations. We estimated that 50 percent of employees received evaluations that contained this type of narrative. For example, one evaluation included the statement “You had one case that the property was in foreclosure and you were looking at seizing or redeeming the property but would only get around $23,000. Through your working with a third-party lending institution, you were able to secure $30,000. . . .” Another included “ . . . have done a very good job in locating assets even in unusual situations. An example of this would be the research you completed on a large dollar tax protestor assigned to your inventory.” Since the comments were based on specific cases reviewed, they are not considered tax enforcement results violations under IRS’ guidelines. However, IRS guidance cautions managers that discussing a tax deficiency or referencing a dollar amount collected may give the employee an improper perception that only the size of the deficiency or the amount collected was the basis for the employee’s evaluation. The second type of narrative that could create misperceptions about the use of tax enforcement results was the use of the terms “overage” (the age of the cases in the employee’s workload inventory) and “cycle time” (the number of cases worked within the established guidelines for closing cases). IRS views these statistics as measures of process time and responsiveness to taxpayers because they are indications of timely attention to taxpayer issues rather than measures of tax enforcement results. We estimate that 41 percent of employees received evaluations that included either general or case-specific references to overage cases and cycle time. For example, one evaluation included the statement “You are making excellent progress in closing out cases and specifically targeting overage and high cycle time to minimize these problems.” Another included the statement “You appear to be doing a better job of meeting the time frames for initial contact and follow-up action on your cases, but your high overage and potential overage rates still indicate a need to improve in this area.” Although neither cycle time nor overage cases are defined by IRS as enforcement results, IRS guidance cautions managers in their use of such terms because of the potential for either of the terms being interpreted as a prohibited statistic such as “hours per case.” Also, during our discussions with NTEU officials, they raised concerns about whether front-line employees and group managers clearly understood the distinction between the inappropriate use of productivity-related enforcement measures, such as hours per case, and the appropriate use of overage and cycle-time data. Appendix VII provides the detailed results for our review of employee evaluations. Because of Internal Audit reports, IRS has undertaken several steps to strengthen the certification process, including expanding the number of employees covered, instituting an independent review process of the certifications, revising guidance, and training managers. According to IRS officials, IRS has expanded the certification process to include all appropriate enforcement supervisors, not just those working in the collection function. Employees who are to be covered by the certification include revenue agents, tax auditors, appeals officers, and criminal investigators. The IRS Restructuring and Reform Act of 1998 also mandates quarterly certification of all appropriate enforcement supervisors. IRS is also instituting an annual independent review to verify the accuracy of the certifications. The reviews are to be performed annually by cross-functional management teams and must include a review of Employee Performance Folders and employee evaluations, which may include documentation related to the evaluation process—such as award narratives and case reviews. The review team may also consider other sources, such as local memos, minutes of meetings, and grievances submitted by employees. IRS has also implemented a new employee evaluation process effective September 1998 that includes a revised form to document a performance evaluation. The guidance on the new form sent to IRS managers in August 1998 states that by signing the form the rater and reviewer certify that enforcement statistics were not used in preparing the appraisal. In addition, IRS has revised the handbook that provides guidance on the appropriate and inappropriate use of tax enforcement results. The handbook provides guidance on which employees are considered enforcement officers and examples of common scenarios involving the proper and improper use of tax enforcement results for each job classification. The handbook also includes a decision table that leads managers through a series of four questions to help them determine whether they are using tax enforcement results appropriately. This new guidance may help clarify some of the confusion managers had on what constitutes a tax enforcement result violation. Furthermore, IRS trained managers on the new certification process and handbook. After providing background on the prohibitions, the training discussed the appropriate use of tax enforcement results and how the prohibitions affect supervisory activities, such as setting expectations as well as explaining the self-certification and independent review processes. To signal its commitment to addressing issues involving the use of enforcement statistics, IRS involved senior executives in the training. For example, IRS directed that district and service center directors would teach the section on self-certification and the independent review. IRS completed the training before the new certification process became effective in the last quarter of 1998. These initiatives address existing weaknesses that we have identified by clarifying which employees are covered and the steps directors should take to independently validate certifications. However, managers may be confused about how to apply the decision table contained in the revised guidance to the preparation of written evaluations. The scenarios included in the revised guidance focus primarily on the prohibited use of statistical reports to imply group goals with relatively few examples relating to the appropriate and inappropriate uses of tax enforcement results in written evaluations. Providing such examples is particularly important because, as noted earlier, IRS officials said that most of the violations we identified would be considered violations under the revised guidance but not under the guidance in effect before 1998. IRS officials agreed that expanding the guidance to include additional examples could be beneficial, especially since managers raised many questions about the appropriate use of tax enforcement results in written evaluations during the training on the revised guidance. Furthermore, IRS has revised its certification form to reflect that more employees are covered by the certification process, but the form contains only a vague statement of conformance. As shown in appendix IX, the certification does not specifically state that first-level managers have not personally misused records of tax enforcement results in written performance evaluations they prepared or to set individual production goals or quotas during their verbal communications with employees. In the case of second-level managers, the certifications do not specifically require that managers certify that tax enforcement results were not used in written performance evaluations they prepared or reviewed or during their verbal communications with employees. As a result, the form does not clearly explain to managers what they are certifying to. IRS has not informed managers about what specific sanctions can be imposed for misusing tax enforcement results or submitting misleading certifications. Neither the revised guidance nor the certification form refers to potential sanctions. In addition to these possible sources of confusion, IRS’ independent review process is to focus on reviewing documents and records to identify written violations. As a result, IRS does not have a mechanism, such as our employee survey, for monitoring employee perceptions of how often tax enforcement results are verbally communicated or whether their written evaluations are influenced by records of tax enforcement results. Consequently, without an employee survey, IRS cannot determine whether further clarifications of the guidance or additional training is needed. IRS directors reported few violations through the quarterly certification process in fiscal years 1996 and 1997, and we found an estimated 9 percent of employees received evaluations that were in violation of IRS’ revised guidance during our review of employee evaluation files. Nonetheless, our survey of IRS employees indicated a widespread perception that managers consider tax enforcement results when preparing performance evaluations. Most of the employees indicated that the violations occurred during verbal communications with their supervisors, such as staff meetings or performance feedback sessions, rather than in their written performance evaluations. Perceptions based on such verbal communications could encourage employees to focus on achieving statistical benchmarks rather than acting solely on the merits of the case. Furthermore, the use of overage and cycle-time data in evaluations may further employee perceptions that tax enforcement results affect their evaluations, because they may be misconstrued as an enforcement statistic, such as hours per case. Although IRS is taking steps to strengthen its reporting of violations, weaknesses remain. IRS’ revised guidance has few examples of the appropriate and inappropriate use of tax enforcement results in written evaluations. Furthermore, the certification form does not specifically require managers to certify that tax enforcement results were not used in written evaluations or used inappropriately during verbal communications with employees. IRS has not provided clear guidance on the sanctions for misusing tax enforcement results or submitting misleading certifications. Additionally, IRS does not have a mechanism for monitoring how often violations result from verbal communications or employee perceptions of whether records of tax enforcement results are influencing their evaluations. To better ensure managerial accountability for the proper use of tax enforcement results, we recommend that the Commissioner expand the guidance to include additional examples of the appropriate and inappropriate use of records of tax enforcement results in written evaluations, revise the quarterly certification form to specifically state that tax enforcement results were not used in any written employee evaluation prepared or reviewed, including appraisals, awards, or promotion justifications, and that the manager did not verbally communicate to employees that tax enforcement results affected their evaluations or were used to set individual production goals or quotas, revise the penalty guide to specifically list the disciplinary actions that can be taken for violations, and survey employees periodically to determine whether they perceive that tax enforcement results were used in written evaluations or verbally communicated by their supervisors and use the results to assess whether IRS needs to further clarify the guidance, provide additional training, or take any other appropriate action. Also, to avoid the potential inappropriate use of overage and cycle-time data, the IRS Commissioner should either designate overage and cycle-time data as prohibited tax enforcement results or emphasize in official policies or procedures to front-line employees and managers how overage and cycle-time data may be used appropriately. We obtained written comments on a draft of this report from the Commissioner of Internal Revenue. The Commissioner agreed with all of our recommendations and described the actions IRS plans to take in response to our recommendations. Regarding our recommendation that IRS should either designate overage and cycle-time data as prohibited tax enforcement results or emphasize in official policies or procedures to front-line employees and managers how overage and cycle-time data may be used appropriately, the Commissioner said that IRS would revise the Internal Revenue Manual to include examples of the appropriate use of these data. If effectively implemented, the actions described should help to resolve the problems we identified. In addition, the Commissioner pointed out that IRS has undertaken a number of initiatives to address the use of tax enforcement results in employee performance evaluations. These initiatives included (1) revising the Internal Revenue Manual provisions concerning the use of enforcement statistics; (2) providing training to all managers on the use of enforcement statistics, which included numerous examples of appropriate and inappropriate language for discussing tax enforcement results in evaluations; (3) providing an orientation course on the proper use of statistics for all IRS employees; (4) establishing a Support Panel to answer questions from managers or employees on the proper use of enforcement statistics; (5) establishing independent review teams that are obliged to review documentation in Employee Performance Files and Employee Evaluations and may, at the director’s discretion, look to other sources of information to ensure potential violations are uncovered, including group discussions, union comments or reports, grievances, and minutes of managers’ meetings; and (6) developing a balanced performance measure system to include measures of customer satisfaction, employee satisfaction, and business results. The business results measure is to focus on the quality and quantity of work done. The quantity measures are to consist exclusively of outcome-neutral production data, such as cases closed and time per closing. We are sending copies of this report to the Ranking Minority Member of the House Ways and Means Committee; the Ranking Minority Member of the Subcommittee on Oversight, Committee on Ways and Means; various other congressional committees; the Secretary of the Treasury; the Commissioner of Internal Revenue; and other interested parties. We will also make copies available to others on request. Please contact me at (202) 512-9110 if you or your staff have any questions. Major contributors to this report are listed in appendix X. This appendix is a summary list of all potential violations that were reported on quarterly certifications for fiscal years 1996 and 1997. No violations were reported in the quarters ending June 30, 1996, September 30, 1996, and December 31, 1996. The Internal Revenue Service (IRS) determined that the reported incidents numbered 1, 2, 5, and 11 were actual violations. 1. Quarter Ending 12/31/95. A branch chief discussed enforcement statistics in a group manager’s evaluation. According to a district office official, the violation, discovered during the next level review, was a statement to the effect that “Your group led the branch in the number of seizures.” Corrective action included assigning the individual to another position outside the Collection Division, assigning an acting branch chief until a permanent replacement could be brought in, and reviewing guidelines. 2. Quarter Ending 03/31/96. A proposal to redesign the revenue officer position under an IRS test project recommended that individual groups negotiate their own production goals. One group established a goal to increase the amount of dollars collected by 10 percent in the next year. The National Office Coordinator certified that this goal was not a policy violation because (1) it was established as a part of a test, and (2) awards or individual evaluations would not be based on the achievement of the goal. District officials were still uncertain about whether this goal was in conflict with policy and requested an opinion from the Office of Assistant Commissioner (Collection), who in turn requested legal advice from IRS’ Chief Counsel. The Chief Counsel ruled that the group’s goal was a violation because it was a production quota or goal and corrective action needed to be taken to be in compliance with policy. The Chief Counsel also cautioned that IRS managers must have the authority to void goals negotiated under the test project if the goals violated law or policy. 3. Quarter Ending 03/31/97. A local chapter of the National Treasury Employees Union (NTEU) raised a concern about a branch chief who discussed the number of seizures reflected on a monthly activity report with his group managers. Although this instance was reported on the quarterly certification, the incident was not considered a violation because, under the policy, managers can discuss recorded statistics with employees as long as they do not imply that goals or quotas are being established. Because the comments were open to misinterpretation, district management said they reinforced the policy guidelines. 4. Quarter Ending 03/31/97. An Automated Call Site manager discussed the low number of calls made during 1 week with his employees and requested them to quadruple the number of calls. The manager also commended the group on the number of closures they made and identified the number of closures he was “going for.” Although this instance was reported, it was not considered a violation because (1) Automated Call Site calls were not considered an enforcement statistic and (2) individual goals were not established. Nonetheless, the manager was counseled and requested to immediately meet with his team again to emphasize that it was the quality of the case work that resulted in the closures. 5. Quarter Ending 06/30/97. A group manager prepared a positive employee evaluation that was based on the amount of dollars collected. According to a district office official, this violation was discovered during a branch manager’s quarterly policy review. The group manager was counseled and the evaluation was revised. 6. Quarter Ending 09/30/97. A district reported a pending allegation in which an employee claimed he was forced to make seizures. According to a district official, based on subsequent district and regional offices’ investigations, the allegation was found to be unwarranted. 7. Quarter Ending 09/30/97. A district identified, during a division and branch chiefs’ sample review of employees’ performance binders, a few instances in which performance evaluations and documents contained enforcement results that could easily be misconstrued as violations (e.g., “His collection actions resulted in both the generation of substantial revenue and in the proper disposition of numerous cases”; “You have processed a number of Offer In Compromise recommendations”; etc.). Corrective actions included emphasizing the policy guidelines on the use of tax enforcement results and requiring more thorough reviews to assess adherence with policy and the law. 8. Quarter Ending 09/30/97. An acting branch manager distributed a memorandum to group managers indicating dollars collected were low and needed to be improved. Although not a violation, the acting branch manager was advised by the division chief that his memorandum was inappropriate and could be misconstrued. He was also instructed to schedule a meeting with the group managers on the real intent of his message to ensure they understood the focus was to identify reasons for the current level of performance and find ways to improve it. 9. Quarter Ending 09/30/97. According to the NTEU, a group manager shared with his group a report containing revenue officers’ statistical accomplishments, including a recap of the dollars collected over a 3-year period. As a corrective action, managers were given instructions that this information should not be shared at the group level and that branch statistics, comparing groups within a branch, should not be shared below the branch level. While this was not classified as a violation at that time, district management officials said they would consider it a violation under the new draft certification procedures. 10. Quarter Ending 09/30/97. The NTEU reported an allegation that, in the presentation of a manager’s award, a group manager mentioned that the revenue officer being honored had made the most seizures in the group. The manager denied making that statement, and the incident was not considered a violation. District officials said that, although not directly related to this incident, several actions were taken, including holding meetings and briefings to discuss the policy and the discontinuation of the distribution of statistics. 11. Quarter Ending 09/30/97. The NTEU and an outside party reported that a group manager placed a routing slip on employees’ desks indicating the amount of dollars that needed to be collected by each revenue officer each day, week, and month. The group manager was temporarily reassigned until an investigation and administrative determination was completed. Instructions were given to branch chiefs not to provide statistical information below their branch level. As a general corrective action applicable to these three incidents, a meeting was held to discuss the importance of adhering to the policy and law, and guidance was distributed to managers. This appendix discusses the sampling methodology we used to determine IRS employees’ perceptions of the use of tax enforcement results and how tax enforcement results were used in employee evaluations. To determine IRS employees’ perceptions of the use of tax enforcement results, we conducted statistically representative surveys of five groups of IRS employees: (1) revenue agents assigned to district office Examination Divisions and Employee Plans and Exempt Organizations Divisions, (2) tax auditors assigned to Examination Divisions, (3) revenue officers assigned to Collection Divisions, (4) group managers assigned to Examination Divisions and Employee Plans and Exempt Organizations Divisions, and (5) group managers assigned to Collection Divisions. To determine how tax enforcement results were used in employee evaluations, we reviewed performance evaluations for a separately drawn sample of revenue agents assigned to district office Examination Divisions and Employee Plans and Exempt Organizations Divisions, tax auditors assigned to Examination Divisions, and revenue officers assigned to Collection Divisions. Both samples were drawn from IRS’ personnel database as of January 3, 1998. Because the questionnaires for the employee surveys were not mailed until March 1998, the results do not include the opinions of any IRS employees who were in the groups of interest as of January 3, 1998, but were not in these groups at the time the questionnaire was sent. We assumed that the opinions of these employees, if they would have been included, would have been the same in the aggregate as those of employees who were included in the surveys. We sampled a total of 1,104 IRS employees from the 5 employee groups of interest from the population of 20,974 employees in the January 3, 1998, IRS personnel database. We allocated proportionately more of our sample to Collection Division revenue officers and managers, since they are directly engaged in collection activities. For the objective of determining how tax enforcement results were used in employee evaluations, we reviewed the evaluations of a sample of 300 IRS employees from the 3 employee groups of interest. By design, none of the 300 employees selected were the same as the 1,104 employees sampled for the first objective. IRS managers are not required to write performance narratives for every rating dimension for every employee each year. To review more narratives, the two most recent ratings were requested from IRS for each employee in the sample. The results presented in the report only reflect employees for whom two performance evaluations were received. For our survey of employees’ perceptions of the use of tax enforcement results, we received usable responses (those that indicated the employee was a member of one of the five target groups at the time the questionnaire was received and responded to at least some of the questions) from 814 employees, for a response rate of approximately 74 percent. The disposition of the sampled cases for our employee survey and review of performance evaluations is shown in table II.1 and table II.2, respectively. After weighting the responses to the employee survey to account for selection probabilities and nonresponse, we were able to estimate the percentage of IRS employees who perceived that certain tax enforcement results were or were not used in their evaluations. We also were able to estimate the percentage of IRS employees for whom tax enforcement results were used in at least one of their two most recent employee evaluations. Because we reviewed a statistical sample of employees, each estimate developed from the samples had a measurable precision or sampling error. The sampling error is the maximum amount by which the estimate obtained from a statistical sample can be expected to differ from the true population value being estimated. Sampling errors are stated at a certain confidence level—in this case, 95 percent. This means that the chances are 19 out of 20 that, if we surveyed all IRS employees in the groups of interest, the true value obtained for a question on these surveys would differ from the estimate obtained from our sample by less than the sampling error for that question. The sampling errors for our survey of the extent to which employees perceived that their ratings were based on enforcement results are less than plus or minus 10 percentage points, unless otherwise reported. The sampling errors for our sample of the extent to which enforcement results were discussed on employee evaluations are no more than plus or minus 8 percentage points. In addition to the reported sampling errors, the practical difficulties of conducting any survey may introduce other types of “nonsampling” errors. For example, differences in how a particular question is interpreted or the types of individuals that do not respond can introduce unwanted variability into the survey results. Our survey asked a representative sample of IRS employees the question “To what extent, if at all, do you perceive that each of the following factors was considered by your supervisor or managers in preparing your most recent performance evaluation?” The “factors” shown for this question were the tax enforcement results associated with each employee category. The responses are shown in tables III.1 through III.6. To compare the perceptions of front-line employees and managers, we consolidated the responses of front-line employees (revenue agents, tax auditors, and revenue officers) and managers (Examination Division and Collection Division group managers) as shown in table III.1. Average dollars collected per return Number of installment agreements obtained Number of installment agreements defaulted Number of cases closed as “currently not collectable” Average dollars collected per return Number of installment agreements obtained Number of installment agreements defaulted Number of cases closed as “currently not collectable” For survey respondents who perceived that their supervisor or manager considered one or more tax enforcement results in preparing their most recent performance evaluation, we asked “Why do you believe that one or more of the factors for which you checked box 2 through 5 [To some extent, To a moderate extent, To a great extent, To a very great extent] in influenced your rating?” They were asked to check all the reasons that applied. The responses are shown in tables IV.1 through IV.8. To compare the perceptions of front-line employees and managers, we consolidated the responses of front-line employees (revenue agents, tax auditors, and revenue officers) and managers (Examination Division and Collection Division group managers) as shown in table IV.1. To compare the importance of verbal and written communications, we also consolidated the responses indicating verbal communications (feedback and meetings) and written communications (performance expectations, performance evaluations, and award justifications). Two other categories—promotion patterns and a general “other” category—are reported separately. The responses of front-line employees and managers on the specific basis for their perception that tax enforcement results affected their ratings are given in tables IV.2 through IV.4. Front-line employees (percent) Managers (percent) Revenue agents (percent) Tax auditor (percent) Revenue officer (percent) Examination managers (percent) Collection managers (percent) One or more factors were mentioned in my performance plan (Form 9688) Our survey asked a representative sample of IRS employees “To your knowledge, were performance goals established for your group in calendar year 1997?” To compare the perceptions of front-line employees and managers, we consolidated the responses of front-line employees (revenue agents, tax auditors, and revenue officers) and managers (Examination Division and Collection Division group managers) as shown in table V.1. For those employees who had group performance goals, we asked “Did the performance goals established for your group in calendar year 1997 include any reference to any of the following factors?” The point estimates and confidence intervals for each employee group are shown in tables V.2 and V.3. Revenue agent (percent) Tax auditor (percent) Examination manager (percent) Revenue officer (percent) Collection manager (percent) Average dollars collected per return Number of installment agreements obtained Number of installment agreements defaulted Number of cases closed as “currently not collectable” This appendix gives some examples of voluntary written comments by front-line employees and group managers to our questionnaires on why they believe enforcement results influenced their performance evaluations. The comments generally involved the following scenarios: (1) supervisors or managers implied that an employee’s performance evaluation was influenced by tax enforcement results; (2) the general culture or atmosphere of IRS implied that enforcement results affected performance evaluations; (3) higher level managers pressured group managers to increase production or revenue yield; and (4) IRS’ business plan, reports, and other documents emphasized enforcement results as goals. Front-line employees and managers also wrote about positive and negative effects of using enforcement and workload measures to establish group goals or in performance evaluations. Generally, the number of negative comments made by front-line employees and managers exceeded the number of positive comments. “The district office provided monthly statistics to each branch, rating branches and groups based on ‘dollars per hour,’ ‘dollars per return,’ ‘no-change rate,’ ‘collections percent by exam division.’ . . . Even though performance goals were not established formally based on ‘dollars per return,’ etc., the statistics were mentioned at each group meeting and pressure was felt by each agent to assess the most tax at the least amount of time spent on the case. It was always felt that our evaluations were based indirectly on the factors mentioned in Item 19 (productivity and enforcement statistics).” “Our emphasis used to be on quality and customer service but when our district went under Boston, the emphasis changed to ‘dollars per hour.’ At each group meeting, we would hear about ‘stats’ and ‘dollars per hour.’ This kind of pressure does tend to carry over into the work product.” “Although numbers never mentioned, group manager discussed closure of cases, fraud referrals, and defaulted installment agreements in meetings and in appraisals so it gave the appearance that what he expected.” “We were told by management in group meetings and in town hall meetings that goals in the form of yield (dollars per hour) would no longer be set. Yet, at group meetings, our manager reads our yield from Table 37 and comments on whether it is good, bad, or fair. Our yield is obviously still very important to management.” “There is one important goal in any revenue agent’s life—dollars per hour. From the day I started with the Service until the present, that has been the only constant in the organization. Quality programs have come and gone, emphasis on taxpayer service ebbs and flows, lip service to auditing standards is periodically given. Through it all, any successful revenue agent knows that low time and high dollars will result in recognition, promotion, and awards.” “It is well known that these factors are considered by upper management to be important, although not all are mentioned in evaluations.” “Management has never been specific relative to the connection between promotion and enforcement. However, the perception is here that the more enforcement minded a revenue officer is, the greater the chance of promotion.” “You are expected (unsaid) to do a seizure every quarter.” “. . . was required to post statistics on most of these items on our library room wall up to just a few months ago. When Congress started seriously making noises about statistics, they were taken down. . . . would make reference to these statistics in a general way as pressure was put on him through his chain of command. The pressure relative to improving these statistics came primarily by passing word down from the chief of audit and his assistant chief that our statistics were not measuring up. . . .” “The manager only works toward faster closures and higher yield because it has been stressed by the district management.” “Time and dollars per hour, although less so within the past few years, has always been a major consideration in ‘informal’ discussions and meetings. Tables 36 and 37 were always utilized and disbursed to the group and discussed with agents, although seldom in writing.” “Statistics on dollars collected, number of returns , number of levies and liens were reported by group on a monthly basis to the division via the branch chief.” “Monthly statistics . . . were distributed regarding levies, dollars collected, liens, seizures, fraud referrals, currently not collectible cases, and cases closed.” “If a revenue agent is doing a good job, these factors (adjustments, etc.) are present. The revenue agent will have good adjustments that are technically correct and will also have no-changes. That is part of the job.” “Both time spent on a return and deficiencies are indicators of effectiveness and efficiency in the audit process. Excessive time may indicate complexity of issues or inefficiency and floundering in the audit. The deficiency amount will indicate effectiveness of audit as compared to similar audits or, potentially, a lack of issue identification. When used as indicators of efficiency or effectiveness, these can help direct employees toward improvement. It’s not all bad.” “Our job is to collect (hence ‘collection’) dollars and returns. It is ludicrous to think that a revenue officer who collected no dollars or returns would get a good performance evaluation. After all—that is our job!” “I definitely feel pressure to be productive in terms of dollars per hour, but realistically, that makes sense to me. We should spend time on issues which will most likely produce revenue and work them as quickly as possible.” “Statistics regarding ‘dollars per hour’ are disseminated at group meetings. Our group is compared to the branch and the district. . . . Ten years ago these kinds of statistics were never discussed at the group level. Due to the constant referring to these numbers, it is obvious that ‘dollars per hour’ is the most important concept to management. Consequently, agents will always strive to get the highest yield per case. This means that you will disregard adjustments in the taxpayers favor so as not to reduce the tax yield. Individual statistics have not been discussed. However, the constant reminder of the yield causes agents to do anything they can to get good yield on their cases.” “Even though it is made very clear in the . . . region that there are no individual or group enforcement goals, statistics are kept and very well known throughout the region. Specifically, our ‘mission’ is very goal oriented and very driven by the goals of the Chief of Collections. Very clearly those groups that have good statistics—specifically the number of seizures, cases closed, fraud referrals, etc.—receive special recognition. In some cases, managers receive performance rewards. The revenue officers who specifically take enforcement action to resolve cases are rewarded with promotion even though the enforcement action is not mentioned in the performance evaluation. Those revenue officers who are most aggressive using enforcement are those rewarded most quickly with promotion.” “National and regional management have used dollar goals and quotas for as long as I can remember. It is common knowledge among employees and first-line managers that ‘increase voluntary compliance’ equals ‘dollars per hour.’ You get what you measure.” “Within the last couple of years, ‘dollars collected’ was being conveyed from the division to the group level. This caused group comparisons and competition. While never conveyed to an individual within my group that dollars needed to be increased, it certainly was implied. While this didn’t change my case decisions, I cannot speak for everyone.” Table VII.1 provides the results of our review of employee evaluations in terms of whether the evaluations contained law or policy violations and case or general discussions of enforcement-related activity. Benjamin Douglas, Senior Evaluator The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists.

Summary: Pursuant to a congressional request, GAO reviewed the Internal Revenue Service's (IRS) managers' compliance with legislative and policy prohibitions against using enforcement statistics in employee evaluations and the requirement that managers certify each quarter whether violations had occurred, focusing on: (1) the extent to which IRS' certification process identified violations of law and policy; (2) IRS employees' perceptions of the use of tax enforcement results in their annual performance evaluations; (3) supervisors' use of tax enforcement results in written employee performance evaluations; and (4) IRS' efforts to revise the certification process. GAO noted that: (1) for fiscal years 1996 and 1997, district and service center directors submitted 368 quarterly certifications that reported 11 potential violations; (2) GAO identified several systemic weaknesses that affected the reliability of the certifications; (3) specifically, GAO found: (a) some confusion among IRS officials about what constituted a violation; (b) inadequate guidance about specific actions directors should take to identify violations; (c) a failure to integrate performance evaluations and the certification process; and (d) unclear guidance on sanctions that could be applied against managers for misusing tax enforcement results or submitting false certifications; (4) GAO's survey of a statistically representative sample of examination and collection employees showed a widespread perception that managers considered enforcement results when preparing annual performance evaluations; (5) GAO estimated that 75 percent of front-line employees and 81 percent of group managers perceived that tax enforcement results affected their most recent performance evaluation; (6) about 70 percent of front-line employees said they based their perception in part on information communicated to them verbally in staff meetings or performance feedback sessions with their managers; (7) and about 36 percent of front-line employees indicated tax enforcement results were used in their written performance evaluations; (8) 9 percent of employees received a written evaluation that contained a tax enforcement result and an estimated 69 percent contained narrative that employees could have interpreted as inappropriate references to tax enforcement results but which did not violate IRS guidance; (9) about 41 percent of the evaluations in GAO's sample mentioned process measures dealing with the age of the cases in the employee's workload inventory and the number of cases worked within guidelines established for closing cases; (10) IRS has undertaken several steps to strengthen the certification process; (11) although these actions address some of the weaknesses of the current system, IRS' revised guidance has few examples of the appropriate and inappropriate use of tax enforcement results in written evaluations; (12) the revised guidance does not clearly inform managers about potential sanctions for inappropriate use of tax enforcement results; and (13) IRS' new independent review process is geared toward identifying written violations and not violations communicated verbally.