SB/SE is one of IRS’s four business operating divisions. SB/SE is responsible for enforcement, taxpayer education, and account services for about 45 million taxpayers, including 33 million self-employed taxpayers and 7 million small businesses with assets of less than $10 million. SB/SE also performs some collection functions for other IRS operating divisions. SB/SE managers told us that the reorganization of IRS in 2000—including the creation of SB/SE—presented an opportunity for them to examine enforcement-related processes from a new perspective. Prior to this, the agency was organized around functional and geographic lines, with separate groups responsible for activities such as processing returns, audits, and collection in particular areas. The reorganization eliminated or substantially modified this national, regional, and district structure and established organizational units serving particular groups of taxpayers with similar needs. Officials told us that with the reorganization, they were now responsible for functions that they had not controlled so directly before. They said that there was general agreement among the managers of the newly created division that there were opportunities to make processes more efficient and effective, and that this led them to start several enforcement process improvement projects. They also distinguished between enforcement process improvement projects, which are generally incremental in their approach, and more far-reaching efforts to modernize IRS and transform processes through business systems modernization and other significant changes. We noted in our recent Performance and Accountability Series that IRS has made important progress in these larger efforts but its transformation continues to be a work in progress. Though many of the SB/SE projects include the word “reengineering” in their titles, SB/SE managers agreed that process improvement projects was a better description, given the scope of the changes these projects were making. As described in GAO’s Business Process Reengineering Assessment Guide, reengineering entails fundamentally rethinking how an organization’s work should be done while process improvement efforts focus on functional or incremental improvements. SB/SE managers explained that they purposefully avoided technology-driven changes of the sort under development in the IRS-wide business systems modernization effort. They said that their goal was to make shorter term, more SB/SE- focused changes in the meantime, while the more sweeping changes, and their longer planning and implementation horizons, were still years away from completion. In this report, we refer to the 15 SB/SE efforts under way as of November 2003 as “process improvement projects.” We have reported on declining enforcement trends, finding in 2002 that there were large and pervasive declines in six of eight major compliance and collection programs we reviewed, with the only exceptions in returns processing and in the automated underreporter program. In addition to these declines, we reported on the large and growing gap between collection workload and collection work completed and the resultant increase in the number of cases where IRS has had to defer collection action on delinquent accounts. In 2003, we reported on the declining percentage of individual income tax returns that IRS was able to examine each year, with this rate falling from .92 percent to .57 percent between 1993 and 2002. We also reported on enforcement productivity measured by cases closed per full-time equivalent employees, finding that IRS’s telephone and field collection productivity declined by about 25 percent from 1996-2001 and productivity in IRS’s three audit programs—individual, corporate, and other audit—declined by 31 to 48 percent. Improving productivity by changing processes is a strategy SB/SE is using to address these declining trends. As of November 2003, SB/SE had 15 ongoing process improvement projects under way, most of them in three broad enforcement areas—audit, collection, and compliance support. Audit projects entail changes to field and office examination processes. Collection projects include changes to automated collection programs, field collections, and other programs. Compliance support is the term SB/SE uses to describe processing functions related to audit and collection such as updating IRS information systems for the results of enforcement work and preparing examination closing letters and liens on taxpayer property. Compliance support projects include changes to technical services and case processing. We selected four SB/SE process improvement projects to review in detail for this report. Field Examination Reengineering includes changes to preaudit processes to better identify specific issues on tax returns for auditors to focus on, among other changes intended to improve examination efficiency and reduce taxpayer burden. The Compliance Support Case Processing Redesign project seeks to centralize data entry into online information systems that monitor the status of active audit and collection cases and their results from many different locations with widely variable workload to just a few with more consistent, predictable workload. The Collection Taxpayer Delinquent Account Support Project involves the development of two computer models to improve setting priorities for collections cases to assign to collections staff. The Collection Field Function Consultative Initiative seeks to improve timeliness on collections cases through regular managerial involvement as cases are being worked. Brief descriptions of all of SB/SE’s projects can be found in appendix IV. SB/SE process improvement project teams completed most of the steps we identified as key to SB/SE’s process improvement project planning, but none of the projects we reviewed completed all of the key steps. Guidance on project planning steps, such as our 20-step framework, could help ensure that key steps are followed more consistently. Also, SB/SE enforcement productivity data presented problems in that the data available to SB/SE managers to assess the productivity of their enforcement activities, identify processes that need improvement, and assess the success of their process improvement efforts are only partially adjusted for complexity and quality of cases worked. The planning for each of the four projects we reviewed included most of the key steps in our process improvement framework, but none of the projects included all of the steps. Figure 2 presents our findings, organized by project stages, for each of the four projects we studied. A full circle means a step was fully completed in project planning and a partial circle means that only part of a step was completed. Our basis for each “no” or “partial” finding is explained in appendix III. Following figure 2, we discuss our findings in more detail with selected examples from the four projects we reviewed. The four SB/SE projects we reviewed largely included the productivity baseline definition and process mapping steps under the “Decision to Change” stage, where SB/SE had to determine whether any of its processes should be improved. The Field Examination Reengineering project team and both collection project teams had baseline data showing that the time needed to complete casework was rising and all four project teams had extensive flowcharts mapping the details of current processes. By helping managers understand the strengths and weaknesses of current processes, such information contributes to more informed decisions about which processes to change. However, SB/SE did not as consistently include the complexity and quality of work being done in productivity baselines, compare productivity data to external benchmarks, identify root causes of productivity declines, or measure the gap between current and desired productivity. Weaknesses in these steps leave SB/SE managers without information that could be useful when making decisions about which processes to change. For example, on three of the four projects, productivity data were not adjusted for case complexity and only partially adjusted for quality. This could cause productivity trends to be misinterpreted, leaving SB/SE at risk of trying to redesign processes that are already working well or missing opportunities to fix processes with potential for improvement. Because GAO’s Business Process Reengineering Assessment Guide and our roundtable participants stressed the importance of complete productivity data and because this was a recurring issue we identified in our assessment of the four SB/SE projects, we discuss the importance of adjusting for case complexity and quality when measuring productivity in more detail in the next section of this report. Another example of not consistently following our key steps in the “Decision to Change” stage is found in the Field Examination Reengineering project. The project team sought the advice of many large, noted organizations to benchmark its productivity. However, the work did not lead to measuring how SB/SE’s productivity compared to others’ because the team did not believe that operations in other organizations were comparable. Without this benchmarking, the team did not know whether and by how much it could improve productivity by updating operations based on the experiences of other large organizations. Both GAO’s Business Process Reengineering Assessment Guide and our roundtable participants stressed that although processes may seem unique to government, they likely have counterparts at a process level in the private sector. Moreover, GAO’s Guide says that looking at dissimilar organizations can actually lead to the most fruitful improvements because it stimulates thinking about new approaches. During the “Target Process Development” stage, the projects we reviewed consistently included the steps that prepare for implementation. Planning on all four of the projects we studied included obtaining executive support, assessing barriers to implementing changed processes, and assessing resource needs and availability. The Compliance Support Case Processing Redesign team, for example, originally identified the need for a computer programming change to implement part of their process redesign. When the programming change could not be made immediately, they continued with a manual process in order to keep the project moving forward. However, SB/SE less consistently included key steps in this stage related to designing the new process. For example, in the Collection Taxpayer Delinquent Account Support project, SB/SE did not consider alternatives to achieving the project’s goal of identifying the best cases to assign to collections staff. Because options were not considered, the team ran the risk of missing a more effective approach than the one they took. Another team did not design the new process based on analysis of a gap between current and desired productivity. It is important at this stage for projects to include fact-based performance analysis to assess how to change processes that are in greatest need of improvement in terms of cost, quality, and timeliness. By analyzing the gap between an existing process’s performance and where that performance should be, projects can target those processes that are most in need of improvement, analyze alternatives, and develop and justify implementation plans. Using these steps can increase the likelihood of determining the best new process. During the “Implementation” stage, three of the four projects we reviewed had completed implementation plans and all three included key implementation steps. These steps focus on the challenge of turning project concepts into a workable program. For example, in the Collection Taxpayer Delinquent Account Support project, the team clearly defined who was responsible for updating the existing computer programs to select cases for priority collection action and who was responsible for evaluating the implemented process. We also found that three of the four teams conducted pilot tests and used their results to modify the new processes prior to implementation—steps important for ensuring that process problems are worked out prior to project implementation. SB/SE was less consistent, however, in establishing employee performance expectations for the new processes. In the Field Examination Reengineering project, SB/SE plans to implement changes to audit planning steps in order to streamline audits and reduce demands on taxpayers for additional information. SB/SE’s plan includes monitoring the deployment of the new process using measures such as the percent of personnel trained. However, SB/SE’s plan does not specify performance expectations for employees or how it will measure whether its auditors are using the new techniques properly. Two projects had completed plans for outcome assessments at the time of our review. One of these, the Collection Taxpayer Delinquent Account Support project, included an evaluation plan using available data to develop measures of how accurately the new models were working. The other two projects were in the process of developing evaluation plans—an important step to ensure that the correct data are available and collected once the change is implemented. Three of four initiatives incorporated change management principles throughout their initiatives. In the fourth, we agreed with SB/SE managers that change management key steps were not a factor because the changes to the method of prioritizing collection cases did not affect collections staff. These are key steps because successful process improvement depends on overcoming a natural resistance to change and giving staff the training to implement the changes. The three project teams where change management was a factor consistently completed all of the key steps in the “Change Management” stage. In the course of our discussions with SB/SE managers about the steps that their projects did and did not include, we learned that SB/SE does not have its own guidance or framework that describes the steps to be followed in planning process improvement projects. SB/SE managers said that projects had been planned and carried out without such a framework. Contractors provided substantial assistance in designing SB/SE’s process improvement projects, and managers told us that they relied in large part on the contractor staffs’ expertise and experience in planning the projects. A framework laying out the steps to be followed is an important internal control for projects such as these because it provides top managers assurance that the steps that the organization has determined to be important are either taken on each project or that project managers have explained why they should be omitted. GAO’s Business Process Reengineering Assessment Guide notes that an established framework is important for projects in that it defines in detail the activities the project team needs to complete and alerts the team to key issues that it must address. Without a process improvement framework and a consistent set of steps to follow, IRS runs the risk of future projects also missing key steps. This, in turn, exacerbates the risk of projects not addressing appropriate process problems, developing a less than optimal target process, ineffectively implementing the project, inaccurately assessing project outcomes, or mismanaging the change to the new process. A framework such as the one we developed for this report is an important internal control tool for SB/SE managers to guard against these risks. The internal control is needed whether process improvement is planned by SB/SE staff or contractors. Such a framework may also prove useful in other IRS units besides SB/SE. As with the 20-step framework we used to assess SB/SE’s approach, however, any such guidelines should allow for appropriate managerial discretion in cases where certain steps are not relevant. The data available to SB/SE managers to assess the productivity of their enforcement activities, identify processes that need improvement, and assess the success of their process improvement efforts are only partially adjusted for complexity and quality of cases worked. Productivity measures the efficiency with which resources are used to produce outputs. Specific productivity measures take the form of ratios of outputs to inputs such as cases closed or dollars collected per staff year. The accurate measurement of enforcement productivity requires data about the quantity of outputs produced and inputs used that are accurate and consistent over time and that link the outputs directly to the inputs used to produce them. The accurate measurement of productivity also requires good data on the relative complexity or difficulty of the cases and the quality of the work done by IRS staff. Case complexity can vary with the type of tax (employment vs. income), the type of taxpayer (individual vs. business) and the type and sources of income and expenses. A measure of productivity like cases closed per staff year that shows an increase may not indicate a real gain in efficiency if the mix of cases worked has shifted to less difficult cases or the quality of the work has declined. This problem of adjusting for quality and complexity is not unique to SB/SE process improvement projects—the data available to process improvement project managers are the same data used throughout SB/SE to measure productivity and otherwise manage enforcement operations. SB/SE managers used data on the number of cases completed and the time it takes to complete them to measure output. Such data were usually only partially adjusted for quality and only once were they adjusted for complexity. Opportunities to make more such adjustments were missed. An example of a complete adjustment for complexity is the Compliance Support Case Processing Redesign team’s use of a proxy for complexity. The project illustrates both the shortcomings of SB/SE’s productivity data and the feasibility of some adjustments using other currently available information. The team wanted to measure the work needed to enter examination and collection case data into the information system, holding complexity constant, but direct measures of complexity were not available. While developing their new process, the team knew that more complex cases were to be assigned to higher-grade clerks. The team used the grade of the clerk to adjust output for complexity. Although not a direct measure of relative complexity, the grade level of the clerks provided managers a means to adjust for complexity and better identify performance increases that were due to changes in productivity by holding complexity constant. Such an adjustment increases the credibility of the team’s estimate that IRS would save up to 385 positions from the proposed redesign. SB/SE has systems in place that measure quality against current standards but do not account adequately for changes in standards of quality. The Exam Quality Measurement System (EQMS) and the Collection Quality Measurement System (CQMS) use samples of audit and collection cases, respectively, to determine if IRS standards were followed and compute scores that summarize the quality of the case. Generally, the scoring is done on a numerical scale. For example, EQMS uses quality scores that range on a scale from 0 to 100. To SB/SE’s credit, most of the projects that we reviewed used EQMS and CQMS scores in an attempt to control for quality changes. Unfortunately, these scores may not adequately reflect changes in standards of quality. For example, the IRS Restructuring and Reform Act of 1998 placed additional documentation requirements for certain collection actions on SB/SE collections staff, such as certifications that they had verified that taxes were past due and that sanctions were appropriate given the taxpayers’ circumstances. SB/SE has changed the standards used in EQMS and CQMS to reflect the new requirements but has not changed its quality scale to account for the new, higher level of quality implied by the new standards. As a result, two exams with the same quality scores, one done before passage of the act and one after, may not have the same level of quality. If the way that SB/SE computes its quality scores does not adequately reflect such changes in quality standards, an increase in staff time needed to complete the additional requirement may be misinterpreted as a decline in productivity. Opportunities exist to improve SB/SE’s enforcement productivity data. Statistical methods that are widely used in both the public and private sectors can be used to adjust SB/SE productivity measures for quality and complexity. In particular, by using these methods, managers can distinguish productivity changes that represent real efficiency gains or losses from those that are due to changes in quality standards. These methods could be implemented using data currently available at SB/SE. The cost of implementation would be chiefly the staff time required to adapt the statistical models to SB/SE. Although the computations are complex, the methods can be implemented using existing software. We currently have under way a separate study that illustrates how these methods can be used to create better productivity measures at IRS. We plan to report the results of that study later in 2004. We recognize that better incorporating the complexity and quality of enforcement cases in enforcement productivity data could entail costs to SB/SE. Collecting additional data on complexity and quality may require long-term planning and investment of additional resources. However, as discussed in the previous paragraph, there are options available now to mitigate such costs. Existing statistical methods could be used in the short term, with currently available data on case complexity and quality to improve productivity measurement. In addition, IRS’s ongoing business systems modernization effort may provide additional opportunities for collecting data. Our roundtable participants stressed the benefits of productivity analysis. They said that an inadequate understanding of productivity makes it harder to distinguish processes with a potential for improvement from those without such potential. GAO’s Business Process Reengineering Assessment Guide also highlighted the importance of being able to identify processes that are in greatest need of improvement. SB/SE deserves recognition for embracing process improvement and for including so many key steps in planning the projects. To the extent that IRS succeeds in improving enforcement productivity through these projects, resources will be made available for providing additional services to taxpayers and addressing the declines in tax enforcement programs. While the SB/SE projects we reviewed included most of the key steps in our framework, putting guidance in place for future projects to follow would help ensure that all key steps are included and improve project planning. The 20-step framework that we developed for this report is an example of such guidance. More complete productivity data—input and output measures adjusted for the complexity and quality of cases worked—would give SB/SE managers a more informed basis for decisions on how to improve processes. We recognize that better productivity will mean additional costs for SB/SE and that, therefore, SB/SE will have to weigh these costs against the benefits of better data. GAO currently has under way a separate study, illustrating how data on complexity and quality could be combined with output and input data to create better productivity measures. This may prove useful to SB/SE managers as they evaluate the current state of their productivity measures. We will report the results of that review later in 2004. We recommend that the Commissioner of Internal Revenue ensure that SB/SE take the following two actions: Put in place a framework to guide planning of future SB/SE process improvement projects. The framework that GAO developed for this report is an example of such a framework. Invest in enforcement productivity data that better adjust for complexity and quality, taking into consideration the costs and benefits of doing so. The Commissioner of Internal Revenue provided written comments on a draft of this report in a January 14, 2004, letter, which is reprinted in appendix V. The Commissioner agreed with our recommendation that IRS develop a framework to guide future improvement projects. He notes that SB/SE used outside experts to help direct the projects we discuss in our report, and how the expertise gained from SB/SE’s projects puts the organization in a position to create a framework for future projects. In regard to our second recommendation, the Commissioner agreed in principle with the value of adding to current enforcement productivity data, but also expressed concerns about cost and feasibility. His letter also discusses initiatives in progress to improve program management and monitoring in the short term, as well as his intent to explore the use of statistical methods to improve enforcement program productivity measurement and to ensure that they are included in modernization projects. The careful consideration of costs and benefits and steps to improve measures in the long term are at the heart of our recommendation and we encourage his ongoing commitment to these efforts. The Commissioner’s letter also notes that employee performance goals— one of the steps in our framework—must not violate legal restrictions on the use of certain enforcement data to evaluate employee performance. We agree and clarified language in our report to make it clear that our framework step concerns employee performance expectations, not using enforcement data to evaluate employees or otherwise imposing production goals or quotas. In addition to commenting on our recommendations, IRS provided supplemental data on the results of some reengineering projects. Reviewing project results was not part of the scope of our review and time did not permit us to verify the supplemental data provided by IRS on project results. We conducted our work from September 2002 through November 2003 in accordance with generally accepted government auditing standards. As agreed with your offices, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days after its date. At that time, we will send copies of this report to the Secretary of the Treasury, The Commissioner of Internal Revenue, and other interested parties. This report is available at no charge on GAO’s Web site at http://www.gao.gov. If you or your staffs have any questions, please contact me at (202) 512- 9110 or David Lewis, Assistant Director, at (202) 512-7176. We can also be reached by e-mail at whitej@gao.gov or lewisd@gao.gov, respectively. Key contributors to this assignment were Tara Carter, Kevin Daly, Leon Green, Landis Lindsey, and Amy Rosewarne. The 20-step process improvement we identified is broken out into four broad stages, from deciding to make changes to assessing the results of the changes. A fifth stage deals with managing the changes being made and takes place throughout the latter part of the project. Figure 3 places the stages in their chronological order, with the change management stage shown taking place simultaneously with other stages. Within each of the stages of this framework are specific key steps that we developed based on GAO guidance and what we learned from managers in other organizations about the steps they took to ensure that they were embarking on the right projects, designing and implementing the projects appropriately, and accurately assessing their projects’ results. The sections below describe the nature and purpose of the key steps that fall under the different stages. We recognize that some steps may not be appropriate for some projects and that managers need to apply judgment in using this or any other process improvement framework. Development of this framework is described in appendix II. Organizations that base the decision to redesign processes on accurate productivity data and a clear understanding of current processes increase the likelihood that a project will avoid misdiagnosing a problem or designing a less than optimal outcome target. Six key steps are important to accomplishing this. Baseline data are information on the current process that provide the metrics against which to compare improvements and use in benchmarking. Productivity measures the efficiency with which an organization uses resources, or inputs, to produce outputs. Specific productivity measures generally take the form of a ratio of outputs to inputs. By establishing a baseline using such measures, a process improvement can be measured in terms of real efficiency gains. For example, the baseline could be used to measure the effect of a successful process improvement as additional output produced by the organization with the same or fewer inputs. Productivity measures may give misleading information if they do not account for the relative complexity and quality of outputs and inputs. A measure of productivity like cases closed per staff year that shows an increase may not indicate a real gain in efficiency if the mix of cases has shifted to less difficult cases or the quality of the work has declined. Besides accounting for complexity and quality, the organization must also choose the appropriate indicators of its outputs and inputs and measure them accurately. Organizations like IRS that are service providers have outputs that often consist of complex, interrelated activities that, in many cases, may require multiple indicators of outputs to accurately measure productivity. The specific data needed depend on the characteristics of particular IRS processes. For example, the number and type of output indicators appropriate for services that have direct contact with taxpayers, such as audits, may be larger and more varied (to reflect the full impact of these services on taxpayers) than those appropriate for other services with less or no direct contact, such as forms processing. However, factors like complexity and quality are necessary for accurate productivity measurement for any process in IRS, regardless of how the specific quantitative measures are defined. A benchmark is a measurement or standard that serves as a point of reference by which process performance is measured. During the “Decision to Change” stage, benchmarking is the solution-building component of process improvement through which an organization compares data used to measure existing internal processes with external data on similar processes in other organizations, or in other components of the same organization, to identify process improvement needs and outcome targets. Through benchmarking an organization is able to identify gaps between an organization’s process performance and that of other organizations or other components of the same organization. Benchmarking is a key tool for performance improvement because it provides “real world” models and reference points for setting ambitious improvement goals. Process mapping is a graphical representation depicting the inputs, outputs, constraints, responsibilities, and interdependencies of the core processes of an organization. Acceptable modeling tools and other analysis techniques include flowcharting, tree diagrams, fishbone diagrams, and business activity maps. It is important that a process map defines what the components of each process are, as well as the process’s boundaries, dependencies, and interconnections with other processes. If initial process mapping is done at a high level, more detailed modeling is necessary to identify all of the current process’s activities and tasks, staff roles and responsibilities, and links and dependencies with other processes, customers, and suppliers. Performance data (e.g., costs, time, throughput) for all activities and tasks should be included on the map, or made available elsewhere. The people who actually do the work as well as the process owner should validate the mapping. Regulations, policies, laws, and assumptions underlying the processes should be identified. Causal factors are the conditions that initiate the occurrence of an undesired activity or state. Causal factors that are within the span of control of an organization should be addressed during the Target Process Development stage. Causal factors that are beyond the span of control of an organization should be isolated when identifying a problem. Examples of causal factors are legal requirements, mix of inputs, quality of inputs, and staff constraints. An empirical basis for the decision to make a process change is an important step leading towards an improvement that is optimal and attainable. An empirical basis can be established by using productivity data to define the gap between where the organization currently is and where it wants to be. After deciding to undergo a process improvement, an organization can increase the likelihood of determining the best new process by using productivity data, assessing implementation barriers, and developing feasible alternatives. Solutions can be adapted from best practices found in other organizations. Best practices are processes and procedures that high-performing organizations use to achieve results. An organization should evaluate the pros and cons of each best practice and if possible, apply its own productivity standards. Ideally, this form of benchmarking should be done with an external organization. Many processes that seem unique to the government actually have counterparts in the private sector, especially in generic areas such as claims processing, loan management, inventory management, etc. Also, it is important to note that the other organizations do not have to be particularly similar, or even do similar work. For example, Xerox used L.L. Bean to improve order fulfillment. Looking at processes in dissimilar organizations can actually lead to the most fruitful improvements because it stimulates new thinking about traditional approaches to doing work. Alternatives are different process designs that would likely result in the same or a similar outcome. An organization’s analysis of alternative processes should consider benefits, costs, and risks. Performance results that each alternative could be expected to achieve should be determined. This can be done using methods such as prototyping, limited pilot testing, and modeling and/or computer simulation. In addition to performance, alternatives can be scored by any number of factors including, feasibility, budget, political appeal, implementation time, payback time, and risk. The team should explore each alternative thoroughly enough to convincingly demonstrate its potential to achieve the desired performance goals and fully describe the types of technical and organization changes necessary to support each goal, and if possible, test key assumptions. The selection of a target process from among alternatives needs an empirical basis in the form of some sort of quantitative analysis. The decision to improve and forming the target process should be linked by an analysis of productivity data that shows how the new process can close the gap between the productivity baseline and the desired outcome. Executive support should come in the form of an executive steering committee—a group headed by the organization’s leader to support and oversee the process improvement effort from start to finish. Executive involvement is important because they are in a position to build credible support among customers and stakeholders, mobilize the talent and resources for a reengineering project, and authorize the actions necessary to change agencywide operations. An executive steering committee’s roles include defining the scope of the improvement project, allotting resources, ensuring that project goals align with the agency’s strategic goals and objectives, integrating the project with other improvement efforts, monitoring the project’s progress, and approving the reengineering team’s recommendations. While carrying out these responsibilities the steering committee must also keep stakeholders apprised of the reengineering team’s efforts. Implementation barriers are obstacles that the organization will need to overcome to implement a new process. Examples of implementation barriers include political issues, entrenched workplace attitudes or values, an insufficient number of employees with the skills required for the redesigned roles, collective bargaining agreements, incompatible organization or physical infrastructure, current laws and regulations, and funding constraints. The impact of these barriers and the costs of addressing them (such as staff training, hiring, and relocation) need to be factored into the process selection decision. If the reengineering team determines that the risks and costs of implementing a preferred new process appear too great, they may need to pursue one of the less ideal, but more feasible alternatives. Prior to taking on a process improvement project, GAO guidance and the other organizations we consulted stress the importance of ensuring the availability of staff and other resources necessary to complete design and implementation of the changed process. Without adequate resources, an organization undertaking a change runs the risk of an incompletely implemented project. A carefully designed process improvement project needs a similarly well thought-out implementation in order to be successful. Pilot tests are trial runs of the redesigned process. Pilot testing is a tool used to move the organization successfully to full implementation. Pilot testing allows the organization to (1) evaluate the soundness of the proposed process in actual practice, (2) identify and correct problems with the new design, and (3) refine performance measures. Also, successful pilot testing will help strengthen support for full-scale implementation from employees and stakeholders. Postpilot adjustments are corrective actions taken to correct trouble spots prior to full implementation. Trouble spots can be pinpointed through the formal evaluation of pilot projects designed to determine the efficiency and effectiveness of the new process. Process owners are the individuals with the responsibility for the process being improved. Designating process owners is necessary to ensure accountability. New employee and/or team performance expectations should be established to account for changes in roles and career expectations caused by the new process. Measurable indicators that are currently being used to track and assess employee or team progress should be analyzed to determine if adjustments will be required after the new process is implemented. In the case of IRS enforcement activities, the agency must ensure that the expectations do not violate the legal prohibition on using tax enforcement results to evaluate employee performance or imposing or suggesting production quotas or goals. In 2002, we reported on IRS’s progress towards improving its performance management system; these changes were brought on, in part, by this requirement. Careful assessment of the results of a process improvement project is important in that it may lead to further changes in the process being addressed and may suggest lessons for other projects. An evaluation plan is a way to collect and analyze data in order to determine how well a process is meeting its performance goals and whether further improvements are needed. Good performance measures generally include a mix of outcome, output, and efficiency measures. Outcome measures assess whether the process has actually achieved the intended results, such as an increase in the number of claims processed. Efficiency measures evaluate such things as the cost of the process and the time it takes to deliver the output of the process (a product or service) to the customer. The data needed to conduct outcome assessments later on need to be identified during project planning to ensure that they are available and collected once implementation begins. Change management focuses on the adjustments that occur in the culture of an organization as a result of a redesigned process. Research suggests that the failure to adequately address—and often even consider—a wide variety of people and cultural issues is at the heart of unsuccessful organizational transformations. Similarly for process improvement efforts, redesigning a process is not only the technical or operational aspect of change, but also overcoming a natural resistance to change. Successfully managing change reduces the risk that improvement efforts will fail due to a natural resistance to change within an organization. An organization needs to establish a change management strategy that addresses cultural changes, builds consensus among customers and stakeholders, and communicates the planning, testing, and implementation of all aspects of the transition to the new process. Change management activities focus on (1) defining and instilling new values, attitudes, norms, and behaviors within an organization that support new ways of doing work and overcome resistance to change, (2) building consensus among customers and stakeholders on specific changes designed to better meet their needs, and (3) planning, testing, and implementing all aspects of the transition from one organization structure or process to another. Executive involvement is important for successful change management. Executive support helps strengthen upper management’s support for the project and serves to reinforce the organization’s commitment to the proposed changes. In a roundtable meeting held by GAO to obtain the perspectives of the private sector, one organization mentioned that providing continuous feedback to its employees is a critical element of a change management program. They also described the importance of consistently updating those employees who would be directly affected by a change initiative. Keeping employees informed of decisions and recognizing their contributions are important elements of developing positive employee attitudes toward implementing process improvement initiatives. Ongoing communication about the goals and progress of the reengineering effort is crucial, since negative perceptions could be formed and harden at an early stage, making the implementation of the new process more difficult to achieve. If change management is delayed it will be difficult to build support and momentum among the staff for implementing the new process, however good it might be. Establish a Transition Team A transition team is a group of people tasked with managing the implementation phase of process improvement projects. A transition team should include the project sponsor, the process owner, members of the process improvement project team, and key executives, managers, and staff from the areas directly affected by changeover from the old process to the new. Agency executives and the transition team should develop a detailed implementation plan that lays out the road to the new process, including a fully executable communication plan. The process owners responsible for managing the project will not effectively convey the goals and implementation strategy of the project if a viable mechanism is not set up by the transition team to keep employees and external stakeholders informed. Training and redeploying the workforce is often a major challenge and generally requires substantial preparation time. When a process is redesigned and new information systems are introduced, many of the tasks workers perform are radically changed or redistributed. Some positions may be eliminated or cut back, while others are created or modified. Workers may need to handle a broader range of responsibilities, rely less on direct supervision, and develop new skills. We began development of a process improvement framework by reviewing previously developed GAO guidance related to business process reengineering. We also reviewed guidance that GAO has recently issued on assessment frameworks for other major management areas. GAO’s Business Process Reengineering Assessment Guide recognizes that the steps for planning process improvement need to be adapted for the magnitude of the projects and the particular circumstances of an organization. To supplement the GAO business process reengineering guidance, we held a half-day roundtable meeting with the Private Sector Council and two of its member companies, Northrop Grumman (a $25 billion defense enterprise) and CNF (a $5 billion transportation services company). We also discussed process improvement planning with public sector managers with experience in revamping complex processes. Reviewing publicly available information and in discussions with SB/SE staff, we found that the tax agencies in the states of California, Minnesota, and Florida had gone through substantial process improvement efforts in recent years. Similarly, the Department of the Interior’s Minerals Management Service had carried out substantial process improvement projects. We interviewed officials from these organizations and reviewed documents that they provided. We then used all of this information to adapt GAO’s guidance to a 20-step framework appropriate to the SB/SE projects. We judgmentally selected 4 projects to study in detail from the 15 projects SB/SE had under way. Our goal in selecting projects for detailed review was to cover at least one project in each of the three main enforcement areas that IRS was revamping— audit, collection, and compliance support. We also looked for projects that were sufficiently far along that we considered it reasonable to expect to see either completed steps or plans for remaining steps for most of the project. We selected one project each in the audit and compliance support areas. We found that there were 2 projects underway in the collections area that were significantly far along, so we selected both of them for our detailed review. For the four projects we selected, we used the documentation previously provided to us to identify evidence that SB/SE managers had taken or were in the process of taking the key process improvement project steps we identified. We then discussed our initial findings with IRS officials responsible for the four projects and they provided additional evidence, both orally and in writing, concerning the elements we had identified as present or not in our initial document review. We then revised our initial assessments based on the additional evidence that the officials provided. Our assessments also included review by a GAO project design specialist, in addition to our usual quality control procedures. We also recognized the need for flexibility in the application of our criteria, in that not all of the steps we identified necessarily make sense for every project. Where a particular step did not logically apply to a particular project, we listed it as “not applicable” in our assessment. For instance, the Collection Taxpayer Delinquent Account Support project we reviewed in detail did not change processes that staff were asked to carry out, so we rated the step about developing a training plan as “not applicable.” Where a step was not fully completed but the project team did a number of elements of the step, we assessed that step as “partial” in our matrix. We did not evaluate the success so far or the likelihood of success for any of the projects we reviewed. We also did not evaluate the effectiveness with which project steps were completed. For example, we did not evaluate the quality of the pilot tests. To determine the usefulness of IRS productivity data as a basis for determining the direction and eventual success of SB/SE process improvement efforts, we reviewed the literature on productivity measurement in tax agencies and in the public sector generally. We also reviewed studies on productivity measurement in service industries with functions similar to IRS. The following four figures provide summaries of the evidence we used to make specific assessments of four selected SB/SE process improvement projects. SB/SE management capitalized on the opportunity presented by the IRS reorganization that created their operating division and saw declining productivity trends as an impetus to change. SB/SE had 15 distinct process improvement efforts under way as of November 2003, many with multiple subprojects. Table 1 provides descriptive information of the 15 projects. The General Accounting Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full- text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select “Subscribe to e-mail alerts” under the “Order GAO Products” heading.

Summary: In recent years, the Internal Revenue Service (IRS) has experienced declines in most of its enforcement programs, including declines in audits and in efforts to collect delinquent taxes. Increasing enforcement productivity is one strategy that can help reverse these declines. To this end, IRS is currently planning and has begun implementing enforcement process improvement projects. GAO was asked to assess the extent to which the planning for the projects followed steps consistent with both published GAO guidance and the experiences of private sector and government organizations. Specifically, GAO assessed the extent to which four judgmentally selected projects followed the 20 planning steps. Planning for the four enforcement process improvement projects GAO reviewed included most of the 20-step framework developed to assess the projects. This increases the likelihood that projects target the right processes for improvement, choose the best target process from among alternatives, effectively implement the project, accurately assess project outcomes, and properly manage the change to the new process. However, none of the projects completed all of the steps. For example, some projects did not fully identify the causes of productivity shortfalls, leaving a risk that the project did not fix the right problem. In the course of this work, GAO found that IRS managers do not have guidance about the steps to follow in planning process improvement projects, increasing the possibility of omitting steps. A recurring issue in the four projects was that IRS's enforcement data only partially adjust for the complexity and quality of cases worked. This issue is also a problem for IRS enforcement productivity data generally. Failing to adjust for both complexity and quality increases the risk that trends in productivity will be misunderstood. For example, a decline in the number of cases closed per employee at the same time that case complexity is increasing may not be a real decline in productivity. GAO recognizes that some options for improving productivity data could be costly. However, costs could be mitigated by using existing statistical methods and IRS complexity and quality data.