Tax Administration: Opportunities to Improve Compliance Decisions and Service to Taxpayers through Enhancements to Appeals' Feedback Project
Published: 2006-03-24T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Taxpayers disagreeing with Internal Revenue Service (IRS) compliance decisions can request an independent review by IRS's Appeals Office (Appeals). In 2004 the Commissioner requested that Appeals establish a feedback program to share the results of Appeals' reviews with the compliance programs. GAO was asked to assess whether (1) information on Appeals results would provide useful feedback to IRS operating divisions to benefit compliance programs, Appeals, and taxpayers through better case resolution and (2) the feedback project is being effectively managed to maximize its potential to improve IRS's performance and thereby reduce disputes with taxpayers.
section_title: What GAO Found
paragraphs: Appeals' case result information has the potential to help compliance programs improve taxpayer service, but realizing improvements requires investments in data collection and analysis that must be considered in light of the likely benefits. Based on a review of 153 Appeals cases, GAO estimates that 41 percent of the 102,623 cases closed in fiscal year 2004 were not fully sustained. Of these, about half were not sustained because Appeals applied a law or regulation differently than the programs. Lacking such information, officials could not assess whether actions like additional guidance were needed. However, identifying specific provisions that were interpreted differently would require data gathering and analysis. Because the differences span a host of laws and regulations, corrective action may only affect a small number of cases. Improved decision making, however, can benefit compliance programs, Appeals, and taxpayers. An initial data analysis, such as identifying programs with high nonsustention rates due to differences in applying laws or regulations, would help to target areas most likely to benefit from feedback. Appeals has taken several initial steps to launch the feedback project. During 2005, for example, Appeals and the compliance programs began to identify additional information needs. In addition, Appeals and the compliance programs could refine the feedback project's objectives to target the results-oriented improvements that are logical benefits of information sharing. Obtaining agreement between Appeals and the programs on objectives may not be easy because their perspectives differ on the steps needed to improve operations, but is necessary. Also, Appeals' plans to update its information system to provide additional data on case results will be hindered by inaccurate data. We found that several important data fields had error rates up to 14 percent. Appeals staff cited several reasons for this, including weak data verification procedures.

section_title: Letter
paragraphs: To help maintain the public’s confidence in the tax system, the Internal  Revenue Service (IRS) believes it needs to ensure that taxpayers receive a  fair, impartial, and uniform resolution of tax disputes.  If taxpayers disagree  with IRS decisions to assess additional tax or take collection action, they  can bring their cases before the IRS Appeals function (Appeals), which  independently reviews whether compliance decisions correctly reflect the  facts as well as applicable law, regulations, and IRS procedures.  With a  staff of about 1,900 employees and a budget of about $193 million in fiscal  year 2005, Appeals annually closes over 100,000 cases where taxpayers  disputed IRS’s compliance decisions.
Our prior reports have found that results-oriented organizations use  performance information to continuously identify performance gaps and  put that information to work to improve their operations.  One possible  source of this information in IRS is the results of Appeals cases.  Using this  information, IRS compliance programs may possibly identify whether  Appeals would agree that the case decision represents the best result for  IRS and the taxpayer.  Then, using Appeals case results information,  compliance program managers may possibly be able to identify  weaknesses in their programs and improve operations, providing better  service to taxpayers, and possibly resolving more cases before an appeal.   In line with this concept, the Commissioner recently requested that  Appeals establish a feedback project intended to maximize the benefits of  sharing Appeals decisions with the compliance programs.
Based on your request, this report’s objectives are to determine whether  (1) information on Appeals results would provide useful feedback to IRS  operating divisions to benefit compliance programs, Appeals, and  taxpayers through better case resolution and (2) the feedback project is  being effectively managed to maximize its potential to improve IRS’s  performance and thereby reduce disputes with taxpayers.
To make determinations in both areas, we reviewed Appeals and  compliance program documents and our prior work on performance  management, and interviewed Appeals and compliance program officials.   To determine whether information on the results of Appeals cases has the  potential to provide useful feedback, we reviewed a random sample of  Appeals cases closed in fiscal year 2004 to determine whether the cases  contained information that could be useful for improving case results.   Principally, we collected data on whether Appeals sustained or overturned  the compliance program’s decision in each case and identified the main  reasons for the Appeals decision.  The results from that sample have  related confidence intervals and precision estimates that are presented in  this letter and in appendix I.  We conducted our review from October 2004  through October 2005 in accordance with generally accepted government  auditing standards.
subsections: section_title: Background
paragraphs: America’s tax system is based on taxpayers voluntarily filing tax returns  that report the full amount of tax owed and paying any taxes that are due.   IRS has four operating divisions:   Wage and Investment Division (W&I) serves the vast number of  individual taxpayers including those who file jointly and only have wage  and investment income.
Small Business/Self-Employed Division (SB/SE) serves about 45  million small business, individual taxpayers with rental properties and  farming businesses, and individuals investing in businesses, such as  partnerships.  SB/SE also serves corporations and partnerships with less  than $10 million in assets and provides field collection services for the  other three IRS divisions.
Large and Mid-Size Business Division (LMSB) serves corporations,  subchapter S corporations, and partnerships with assets greater than  $10 million.  These businesses have a large number of employees, have  complicated tax and accounting issues, and often conduct business  globally.
Tax-Exempt and Government Entities Division (TE/GE) serves three  very distinct customer segments.  Employee Plans serves private and  public retirement plan customers.  Exempt Organizations serves  customers that are exempt from income taxes, such as charities, civic  organizations, and business leagues.  Government Entities serves  customers from federal, state, and local governments; Indian tribal  governments; and tax-exempt bond issuers.
These divisions are responsible for providing a full range of services to  these taxpayers.   Typically, these services would include assisting  taxpayers with filing returns, processing those returns and maintaining  their accounts, and examining suspected inaccurate returns.  Taxpayers  who are assessed additional tax and penalties or who have a pending  enforcement action to collect delinquent taxes, such as a proposed levy or  lien, have the right to request a hearing through an administrative appeal  before the assessment or collection actions are final.  IRS notifies the  taxpayers in writing of these pending actions and explains their appeal  rights.  Generally, the taxpayer has 30 days from this notification to request  an appeal.
subsections: section_title: The Appeals Process
paragraphs: Appeals’ mission is to independently resolve tax disputes prior to litigation  on a basis which is fair and impartial to both the government and the  taxpayer.  To assure their independence, Appeals’ staff cannot discuss  substantive case issues with compliance staff unless taxpayers or their  representative are present.  Generally, compliance staff does not directly  participate in an appeal or learn about the resulting decision.  To identify  whether the proposed compliance action should be sustained, Appeals  staff review the case file prepared by IRS’s compliance program and  determine whether that evidence demonstrates that the taxpayer and  compliance staff have followed the applicable law, regulation, and IRS  procedure.  If requested, the taxpayer may meet with Appeals staff and  provide additional evidence to support their appeal. To close an  examination case, Appeals may (1) agree with the examination program  and fully sustain its recommended assessment, (2) disagree and reduce the  recommended assessment to partially sustain the assessment, or (3) fully  concede to the taxpayer’s position and not sustain the assessment.  For a  collection case, Appeals may (1) agree with and sustain the proposed  enforcement action or (2) not sustain the proposed enforcement action by  modifying the proposed action (e.g., propose an installment agreement  rather than a levy), deferring collection, or fully conceding to the taxpayer’s  position.  If the taxpayer and IRS cannot reach agreement on the outcome  of the case through the Appeals process, the taxpayer may have the case  reviewed by the U.S. Tax Court, U.S. Court of Federal Claims, or a U.S.  district court.
In line with its mission to resolve cases prior to litigation, Appeals is also  authorized to review the facts of the case in light of the hazards that would  exist if the case were litigated.  Appeals is the only IRS organization  authorized to consider hazards of litigation when deciding whether to allow  taxes and penalties.  This means that Appeals may recommend a fair and  impartial resolution somewhere between fully sustaining or fully conceding  the examiner’s proposal that reflects the probable result in the event of  litigation.
If taxpayers do not reach agreement with IRS examiners on the proposed  deficiency, or if they choose not to contact Appeals, IRS will issue a notice  of deficiency.  This notice describes the deficiency and states that the  taxpayer has 90 days to file a petition with the court for a redetermination  of the deficiency.  However, even though Appeals may be initially bypassed,  it still has an opportunity to settle these cases.  Under IRS procedures  designed to encourage resolution of cases at the lowest possible level, the  attorney from the local IRS District Counsel’s office handling the court case  is required to refer the case to Appeals for possible settlement before it is  scheduled for trial.  Figure 1 summarizes IRS’s appeals system.
Appeals’ workload is organized into eight “workstreams” that reflect  similarities in the case workload rather than which of IRS’s four operating  divisions initiated the case.   Two of the eight workstreams relate to  collection issues and generally originate in two of IRS’s four operating  divisions responsible for collection issues (Collection Due Process and  Offer-in-Compromise workstreams).  Three of the eight workstreams  include a wide range of generally smaller examination and returns- processing-related penalty cases (Innocent Spouse, Penalty Appeals, and  Exam/TEGE).  The three other workstreams (Coordinated Industry Case,  Industry Case, and Other) cover a small number of complex examinations  from IRS’s LMSB programs as well as cases that do not fit into other  workstreams.  Appendix II includes definitions of Appeals workstreams,  identifies the related operating divisions for the workstream, and the  number of cases closed in each workstream during fiscal year 2004.
section_title: Strategic Approach
paragraphs: Results-oriented organizations consistently strive to improve their  performance through strategic planning.  As part of this approach, agencies  set objectives and measure performance to evaluate whether performance  has improved.  Specifically,    goals or objectives are the results that a program is expected to achieve,   performance measures are selected after goals or objectives are  developed, logically related to these goals or objectives, and used to  gauge progress toward them.
Other federal agencies have previously decided that developing and sharing  information on the results of appeals may help them measure performance  or at least serve as an indicator of whether their decisions are legally  correct.  For example, the Merit Systems Protection Board, an independent  quasi-judicial agency established to protect merit systems in the federal  workplace, has set a performance goal of maintaining or reducing its low  percentage of appealed decisions that are reversed or sent back to board  judges for a new decision.  The board’s performance plan for fiscal year  2005 contains an array of case-specific data to measure this performance  goal.
section_title: Feedback Information Could Be Useful for Improving Performance
paragraphs: Appeals overturned about 41 percent of the fiscal year 2004 cases we  reviewed and in about half of those cases Appeals disagreed with the way  compliance programs applied the law or regulations.  This suggests that  providing information on Appeals decisions could help compliance  program managers improve case results by fostering more proper and  consistent case decisions.  However, finding the source of possible  inconsistencies will require gathering and analyzing additional information  and systematic analysis.  Improved decision making, however, can benefit  compliance programs, Appeals, and taxpayers.
subsections: section_title: Feedback on Appeals Results Could Be Useful
paragraphs: Based on our case review, for cases closed in fiscal year 2004, we estimate  that Appeals did not sustain about 41 percent of compliance cases (about  42,075 of the 102,623) that year.  We identified six principal reasons for  those nonsustentions.  As shown in table 1, we estimate that Appeals did  not sustain compliance decisions in 52 percent of the cases not sustained  (21,879 cases) at least in part because Appeals disagreed with compliance  staff’s application of tax law or IRS regulations.  Providing feedback on  such disagreements could help compliance managers improve case results  by taking action to foster the proper and consistent application of tax laws  and regulations.  For example, compliance managers could assess whether  guidance or manuals, supervision, quality control, or other management  tools should be revised to ensure that cases are properly closed.   Identifying more specifically which laws or regulations were applied  differently by the compliance programs would require an investment to  gather and analyze additional data.  For instance, in table 1 we identified  the handling of a state tax refund as an example of differing applications of  tax laws and regulations.  To determine whether this is a common problem  or an isolated instance, officials would have to investigate the issue by, for  example, drawing a random sample of cases or questioning first-line  managers and staff.  Because these differing applications span a host of  laws and regulations across IRS’s compliance programs, the corrective  action that might be taken may only affect a relatively small number of  cases.  In complex cases, Appeals and the compliance managers may need  to work together to develop a mutual understanding of how laws and  regulations should be applied.
As also shown in table 1, we estimate that Appeals did not sustain  compliance decisions in 44 percent of the cases because the taxpayer  provided additional information to Appeals.  For cases in this category,  officials would need to investigate whether compliance staff could have  done more to obtain the information needed to resolve the tax before the  case was appealed.  For example, compliance managers might assess  whether staff clearly articulated the type and extent of information needed,  gave the taxpayer sufficient time to respond, or received the information  but did not use it appropriately to resolve the case.
Similar data gathering and analysis would be needed for the other reasons  we identified for Appeals not sustaining cases in order for the information  to be useful in improving compliance’s decision making.  For example, for  cases where Appeals had to perform original audit work or significant  rework, compliance managers would need to identify why their staff did  not perform the necessary work while the case was still their responsibility.    For cases where Appeals accepted a collection alternative, compliance  managers might assess whether it was because the taxpayer had not  requested an alternative, the taxpayer’s financial circumstances had  changed since compliance worked on the case, or a request for an  alternative was inappropriately rejected by compliance staff.   For cases  where taxpayers did not respond to compliance, compliance managers  might assess whether staff had made sufficient attempts to contact the  taxpayer.
section_title: Improved Decision Making Would Benefit Compliance Programs, Appeals, and Taxpayers
paragraphs: As shown in table 2, the appeal rate--the percentage of cases appealed-- varies across Appeals’ workstreams from 29 percent for LMSB’s  Coordinated Industry Case program (CIC) cases to one-tenth of 1 percent  for cases in the Penalty Appeals and Other workstreams.  Managers of  programs with high appeal rates told us that they would benefit from  Appeals feedback information in improving decision making.  For example,  with relatively high appeals rates and complex tax issues frequently  considered in both the Industry Case (IC) and CIC programs, LMSB  managers believe that Appeals case result information is important for  managing their programs to update policies and procedures, modify or  assess new training needs, or identify needed changes in the tax law.  Similarly, Offer-in-Compromise program managers say they have benefited  from working with Appeals staff on studies analyzing why cases were not  sustained by Appeals.  One study indicated that compliance managers and  Appeals needed to reevaluate or reinforce some of their policies as well as  be more consistent in following established procedures for assessing  financial information, such as calculating transportation expenses,  establishing the value of cars, and estimating future income.
Managers of programs with low appeal rates may not see as much benefit  in obtaining feedback from Appeals.  With an appeal rate of less than one- half of 1 percent, managers in W&I, the source of many cases in the  Exam/TEGE workstream, explained that they had limited interest in  devoting resources to analyzing Appeals feedback information, although  they would review any analysis Appeals provided to them.  Managers told  us that, given the challenges facing W&I, they needed to focus resources on  other issues.
However, although managers might not see much direct benefit for their  programs, reducing the appeal rate for compliance programs could benefit  Appeals.  As shown in table 2, the Exam/TEGE, Penalty Appeals, and Other  workstreams have appeal rates of less than 1 percent, but the cases from  these three workstreams make up about half of Appeals’ workload.  In  addition, the cases that are appealed are generally not fully sustained.  The  percentage of cases in our sample that were not sustained ranged from 73  percent for Exam/TEGE to 56 percent for the Other workstream.  From  Appeals’ perspective, improving case results in these workstreams could  represent a target of opportunity for reducing its case load and increasing  its efficiency.  Analysis of our sample found that across all workstreams,  Appeals cases that are fully sustained require about half of the staff hours  of cases that are not fully sustained.
If providing feedback to compliance programs improved their decision  making, taxpayers would benefit as well.  For example, if compliance  programs used feedback information to improve their understanding of  how to apply tax laws and regulations, they could reduce the number of  taxpayers requesting an appeal and therefore resolve cases more quickly  and with more uniform decisions.  Further, since Appeals managers said  some taxpayers decide not to pursue an appeal even though they disagree  with a compliance decision, more consistent application of the tax laws or  regulations could also improve the fairness and accuracy of their  outcomes.
The Exam/TEGE workstream can be used as a hypothetical example of the  potential effect of these benefits.  If the quality of compliance case  decisions were to improve and as a result the percentage of cases fully  sustained in Appeals were to increase from 28 percent to 38 percent,  Appeals would save an estimated 7 staff years. Another potential cost  saving would result if fewer taxpayers appealed because the quality of  compliance case decisions improved.  For example, if the number of cases from the Exam/ TEGE workstream that are appealed fell by 10 percent,  Appeals would save an estimated 17 staff years.
section_title: More Systematic Data Analysis Would Help to Identify Useful Feedback
paragraphs: Identifying which compliance programs would benefit most from feedback  is important given that Appeals hears a wide variety of cases, the cases are  spread across the operating divisions, and Appeals does not fully sustain  cases for a variety of reasons.   This dispersion means that in some  situations the costs IRS would incur to analyze Appeals data and devise and  implement improvements in operations may not be justified given how few  cases could be affected.
When analyzing our case sample, we found that overall (1) about half of all  not fully sustained cases cited either the application of laws and  regulations or additional information as the reason for nonsustention and  (2) certain workstreams have significantly higher nonsustention rates than  others.  As shown in table 3, by considering these two facts in combination,  we found that two workstreams–Penalty Appeal and Exam/TEGE–had a  large percentage of cases that were not sustained for these two reasons.   Other information already available might also be used to identify the most  promising areas in which to conduct feedback projects.  For example,  those cases that are most costly to Appeals to work on, measured for  instance by staff hours per case, may yield the most savings to Appeals if  the cases could be resolved in the compliance programs without an appeal  being made.  Appeals and compliance programs have been selecting their  projects more on the basis of manager judgment than through data  analysis, such as those cases described above.
As discussed earlier, Appeals and compliance program managers will need  to sort through possible reasons why some areas appear to have high levels  of nonsustained cases.  This may require several iterations of data analysis,  discussion, and manager judgment.
Once officials have identified the areas with the greatest potential for  improvement, Appeals and compliance programs can explore low-cost  avenues for using feedback information.  For example, in two of the three  workstreams with the highest percentage of appealed cases, Appeals and  compliance programs have completed some projects based on Appeals  case results.  The joint study on Offer-in-Compromise cases not sustained  by Appeals was conducted by a small team of compliance and Appeals staff  and involved the review of 113 cases in 1 week.   In contrast, Appeals and  LMSB concluded that jointly reviewing fully conceded issues in the CIC  program was too expensive because these cases can involve numerous  complex issues.  Rather, Appeals has started to provide to LMSB all  Appeals Case Memorandums (ACM) as a low-cost solution for providing  the information to target possible areas needing improvement.  However,  Appeals has not yet explored potential avenues for using feedback  information in other large worksteams, such as Penalty Appeals.
section_title: Appeals Can Build on Existing Efforts to Improve the Feedback Project
paragraphs: Appeals has taken several steps to launch and begin expanding the  feedback project.  As Appeals and compliance managers gain experience in  analyzing and using feedback information, Appeals, in partnership with  compliance managers, can build upon those efforts by identifying the  additional feedback information that needs to be shared and further  developing results-oriented objectives.  In addition, potentially useful  feedback data contain errors that undermine their usefulness.
subsections: section_title: Appeals Is Providing Some Feedback Based on Officials’ Initial Judgments of Its Needs
paragraphs: Appeals has taken several steps to launch and expand the feedback project.   For example, officials are sending ACMs to certain compliance programs.   During 2005, Appeals started to send ACMs to  LMSB Industry Case and Coordinated Industry Case programs,  W&I for the Innocent Spouse and EITC programs,  SB/SE for the Collection Due Process, Offer-in-Compromise, and  International Examination programs, and  TE/GE for some Exempt Organizations cases.
The Collection Due Process program also receives some summary-level  information on whether the taxpayer and Appeals agreed on the outcome  of an appeal as well as the Appeals inventory level.  Appeals and the  compliance programs are working together to determine which additional  programs should receive specific feedback information.
In addition, Appeals and compliance programs’ staff meet regularly  through advisory board meetings.  The advisory boards were created to  focus on important cross-functional issues, solve problems, identify new  issues arising in the compliance programs, and generally maintain close  working relationships.  For example, as previously discussed, Appeals and  the SB/SE collection staff jointly worked on a review of Offer-In- Compromise cases to determine why Appeals accepted some offers that  the compliance program rejected and are considering similar efforts with  other compliance programs.
Generally, decisions on what information to initially share with the  compliance programs have grown out of discussions between Appeals and  compliance staff and reflect their best judgment about the information that  likely will help the compliance programs improve case results.  Appeals  and the compliance programs are still determining what additional  feedback information should be shared.  Appeals, in coordination with the  compliance programs, is revising its case closing documents to provide  additional information describing the basis for the resolution of a case.  For  example, Appeals is working with the compliance programs to provide  information on whether additional information was considered by Appeals,  cases were closed based on hazards of litigation, or taxpayers did not  respond or delayed their responses.  Compliance program managers told us  that providing more detailed information tailored to their needs will help  them to improve their results.  Appeals plans to implement a revised  Appeals case closing system during 2006.   Appeals managers believe, and  we agree, that the compliance programs will likely identify additional  information needs in the future as they begin to analyze and use the  information.  For example, some compliance managers have told us that  information on sustention rates would be useful.  However, Appeals’ has no  plans to develop this information.
section_title: Partnership with Compliance Programs Could Help Ensure That Feedback Information Will Be Used to Achieve Desired Program Results
paragraphs: Although Appeals has worked with the compliance programs on many  aspects of the feedback project, Appeals developed the objectives and  performance measures for the feedback project with relatively little input  from the compliance programs.  After initially developing the objectives  and measures, Appeals distributed them to compliance program  representatives for comment but received little response.  Appeals officials  therefore concluded the compliance programs agreed with the objectives  and measures.
Best practices in strategic planning, of which setting objectives is a part,  call for the involvement of stakeholders.  In the case of the feedback  project, involving the compliance programs in establishing the project  objectives is particularly important because the programs themselves must  play active roles in the project to make any changes that will improve their  case results.  Appeals officials acknowledge that mutually agreed upon  objectives and measures would increase the likelihood that compliance  programs would use the feedback information provided.
However, gaining consensus may not be easy.  The following illustrates the  importance of involving the compliance programs in these decisions and of  the potential difficulties that could arise.  Appeals could adopt an objective  for the feedback project of improving the sustention rate for compliance  program cases that go to Appeals.  That is, if the quality of compliance  decisions is improved through feedback of Appeals case results, more  appealed cases should be upheld.  However, some of the compliance  program officials we interviewed do not want improvement in the  sustention rate to be an objective for a variety of reasons.  For example,  officials note that Appeals can change a case result for reasons that are out  of their control.  As discussed before, Appeals is authorized to close cases  based on hazards of litigation and compliance programs are not.  As a  result, compliance managers are concerned that including cases closed  based on hazards of litigation as part of a sustention rate would be unfair.   Other managers do not agree that Appeals always makes the correct  decisions on compliance cases.  Thus, the active involvement of  compliance program officials in the selection of objectives would be  important to determining the strengths and weaknesses of potential  objectives.
Further, the involvement of compliance programs in establishing objectives  and project measures may better ensure that the feedback project is  focusing on desired results. As defined by Appeals in April 2005, the  objectives of the case feedback project are to   build strong relationships between Appeals and operating divisions and   capture and share trend data,  analyze trend data–and provide meaningful commentary to the  operating divisions and functions, and   influence operating division policy and procedure.
Although these objectives indicate some of the activities that are integral to  feedback sharing and a desired outcome--influence on operating divisions’  policies and procedures--involving the operating divisions in considering  program objectives would provide an opportunity to build on these  objectives to more fully define the results intended for the feedback  project.  The Commissioner did not specify the benefits that he thought  should result from sharing of Appeals case information with the operating  divisions and their compliance programs.  However, as discussed earlier,  sharing this information has the potential to improve the operations of the  divisions and, consequently, the quality of their case decisions, potentially  increasing the case sustention rate and taxpayer satisfaction with the  Appeals process while also decreasing the time to complete an appeal.  In  addition, sharing information may also improve Appeals’ decision making  by, for example, clarifying IRS’s interpretation of new or particularly  complex tax laws so that both Appeals and compliance managers apply  them consistently.  By working with the compliance programs, Appeals  would have the opportunity to further refine the project objectives to more  specifically identify which of these possible results-oriented improvements  are being sought by the project.  For example, as mentioned earlier, the  Merit Systems Protection Board has set a performance goal of maintaining  or reducing its low percentage of appealed decisions that are reversed or  sent back to the board.  To the extent that new objectives are identified,  Appeals and the compliance programs would need to ensure that  appropriate performance measures are developed to track progress toward  those objectives.
section_title: Project Is Correcting Flaws in Data, but Sufficient Internal Controls to Monitor and Verify Data Not Yet Developed
paragraphs: When we compared data in Appeals’ Centralized Database System  (ACDS) to documentation in closed Appeal case files, we found  significant error rates related to data that would be used for a case  feedback project.  The highest error rates in the fields were related to the results of an appeal, such as in the revised tax, revised penalty amounts, or  case-closing code field.  For example, 14 percent of the cases contained  errors in the revised tax field.  These errors related to the outcome data  that likely would be included as part of any feedback information provided  to the compliance programs and would diminish the information’s  usefulness to compliance program managers.  Further, 12 of 165 cases   (7 percent) could not be analyzed because the files could not be located or  essential Appeals documents were not available.
On the basis of error rates identified, we reviewed internal controls for  processing case results data and identified several internal control  weaknesses that may have contributed to inaccurate data in ACDS.  For  example, we were informed by Appeals that some appeals officers, who are  responsible for working the taxpayers’ case, did not verify that ACDS data,  such as the amount of tax or penalty owed by the taxpayer, was entered  into ACDS accurately.  Appeals policy requires that the appeals officers  verify the key data in ACDS, such as the statute of limitations date, when a  case is received.  When a case is completed, Appeals procedures require  the case manager, who supervises the appeals officer, to review and sign  case-closing documents, which include data such as the amount of  proposed tax or penalty and case-closing code.  The closed-case data are  then entered into the ACDS information system by the Appeals Processing  Services staff.   According to Appeals, once the case is sent to Processing  Services for data entry, the appeals officer and case manager generally do  not see the case again and do not know whether the closing data have been  entered into ACDS accurately.  Appeals guidance does not require that the  appeals officer or Processing Services staff verify whether the data were  accurately entered.
According to Processing Services staff, appeals officers may not ensure  that case-closing documents are complete.  For example, data, such as the  amount of revised tax or penalty (the amount of tax or penalty as  determined by Appeals) or the closing code, may not be entered on the  closing document by the appeals officers.  Processing Services staff said  that in these cases, they must review the case file to determine the correct  closing data and enter that data into ACDS.  The staff stated that identifying  the correct data may be difficult in complex cases.
Other internal controls only partly compensate for the lack of data entry  verification.  Appeals performs an annual Inventory Validation Listing  process for open cases where critical fields in ACDS are verified and errors  identified are corrected in ACDS.  Since only open cases are reviewed,  fields with closing data, such as revised tax, revised penalty, and closing  code, are not reviewed.  These closing fields are critical to the feedback  loop process and without verification inaccurate data could be sent to the  compliance programs.
Appeals is making efforts to improve the accuracy of the data in ACDS.   Appeals, for the first time, completed a data reliability study of ACDS in  2005.  This study consisted of a random probability sample of 1,568 Appeals  cases where data fields that were considered critical or were used daily  were tested.  From the study, Appeals identified data accuracy and internal  control issues that were consistent with our findings.  Appeals found that  some fields in ACDS had lower than expected accuracy rates.  For  example, the revised tax field for the Innocent Spouse workstream had an  accuracy rate of 71.9 percent, while the revised penalty field for the Other  workstream was 78.1 percent.  Appeals also identified that improvements  were needed in (1) internal controls including training of Processing  Services staff on ACDS input procedures, (2) ACDS data fields with lower  than expected accuracy rates, and (3) Appeals’ section of the Internal  Revenue Manual, which includes guidelines for standard data accuracy  reviews.
Appeals has been revising its database and related data entry procedures to  improve the accuracy of the data in ACDS.  Case-closing documents are  being redesigned in a computer-based format so that only data which are  appropriate to the case under appeal could be selected, thus reducing the  potential for errors.  Although Appeals is making efforts to improve the  accuracy of the data in ACDS, it has not completed plans to address all of  the identified data accuracy issues.  Appeals will likely continue to  experience data accuracy issues unless it improves its internal controls to  verify, on an ongoing basis, the accuracy of case data entered into ACDS.
section_title: Conclusions
paragraphs: Using the results of Appeals case outcomes has the potential to improve  compliance programs’ case results and service to taxpayers with benefits  that could accrue to the divisions, Appeals, and taxpayers.  Nevertheless,  given the scope of Appeals’ work, careful targeting of investments to use  Appeals information is needed to ensure that the benefits will be significant  enough to justify the costs IRS incurs to collect and analyze Appeals data  and make changes in policies, procedures, or practices based on those  analyses.  Because relatively few compliance program cases may be  affected by the use of some Appeals feedback information, officials need to  be judicious in selecting topical areas to study.  Opportunities exist to move  beyond professional judgment in selecting these areas to a more data- driven approach.
However, to maximize the benefits of sharing Appeals information, as  intended by the Commissioner, the officials need to better define what the  program is intended to achieve and how results will be measured.  Appeals  and the compliance programs need to enter into an active partnership to  develop results-oriented objectives and associated performance measures  for the feedback project.  Finally, the feedback project must be built on  reliable data, which requires that better internal controls be instituted to  drive down the error rates in key data that will be provided to the  compliance programs.
section_title: Recommendations
paragraphs: We are making recommendations to the Commissioner of Internal Revenue  to ensure that the feedback project reaches its maximum potential in  improving case results.  Specifically, we recommend that the  Commissioner direct Appeals   in partnership with the compliance programs, to analyze Appeals case- results data, such as the workstream sustention rates, reasons for  nonsustention, or staff hours spent per case, to identify areas in which  improvements are likely to generate the greatest benefits to the  compliance programs, Appeals, and taxpayers;   in partnership with the compliance programs, to further investigate the  most promising areas and assess whether actions, such as additional  guidance or training, are needed to improve the quality of compliance  programs’ case decisions;   in partnership with the compliance programs, to further develop results- oriented objectives and associated performance measures for the  feedback project; and  to build upon its current efforts to improve the quality of Appeals  information for the feedback project by establishing internal controls to  verify, on an ongoing basis, the accuracy of the data entered into  Appeals information systems on case results.
section_title: Agency Comments and Our Evaluation
paragraphs: The Commissioner of Internal Revenue provided written comments on a  draft of this report in a March 6, 2006, letter which is reprinted in appendix  V.  The Commissioner agreed with our recommendations and said they will  help IRS develop a much stronger feedback program.
With regard to the first recommendation, the Commissioner said IRS would  continue quarterly meetings between the operating divisions and Appeals  and report national-level feedback data at least annually to identify specific  compliance programs where shared benefits would be realized.  We agree  that these actions would be a first step toward implementing the  recommendation.  However, as discussed in the report, more systematic  data analysis would help Appeals and the compliance programs identify  areas more likely to realize the benefits of using feedback data.  As  discussed in the Commissioner’s comments, this may involve reviewing  external data, such as the National Taxpayer Advocate’s reports, as well as  other data to identify areas that may yield the most savings to IRS if the  cases were resolved in the compliance programs without appeals.  The  analysis of existing data is a necessary step toward tailoring analyses to  each of IRS’s compliance programs.
As agreed with your offices unless you publicly announce its contents  earlier, we plan no further distribution of this report until 30 days after its  issue date.  At that time, we will send copies of this report to the Secretary  of the Treasury, The Commissioner of Internal Revenue, and other  interested parties. Copies will be made available to others upon request.   This report is available at no charge on GAO's Web site at  http://www.gao.gov.
If you or your staff have any questions, please contact me at (202) 512-9110  or Jonda Van Pelt, Assistant Director, at (415) 904-2186.  We can also be  reached by e-mail at brostekm@gao.gov or vanpeltj@gao.gov, respectively.   Contact points for our offices of Congressional Relations and Public Affairs  may be found on the last page of this report. Key contributors to this report  were Carl Barden, Evan Gilman, Leon Green, Shirley Jones, Laurie King,  Ellen Rominger, and Michael Rose.
section_title: Objectives, Scope, and Methodology
paragraphs: Our objectives were to determine whether (1) information on Appeals  results has the potential to provide useful feedback to the Internal Revenue  Service (IRS) operating divisions to benefit compliance programs, Appeals,  and taxpayers through better case resolution and (2) the feedback project  was being effectively managed to maximize its potential to improve IRS’s  performance and thereby reduce disputes with taxpayers.
To determine whether information on the results of Appeals cases has the  potential to provide useful feedback and whether the feedback project is  being effectively managed, we interviewed 24 Appeals executives,  managers, and staff who work with compliance program staff on feedback  issues, coordination, or information systems issues.  We also interviewed  58 compliance program executives, managers, and staff selected by  operating division liaisons to represent their compliance programs because  of their familiarity with Appeals issues.  We discussed with these officials  the type of feedback data that are being collected by Appeals and sent to  the compliance program officials as well as the type of feedback data  compliance program officials would like to receive from Appeals.  We  reviewed documents provided by Appeals on the feedback project.
We reviewed the Appeals Centralized Database System (ACDS) to  determine whether it contained sufficient case results information.  We  found that it did not contain sufficient information for our analyses, such as  whether Appeals agreed with the compliance decision.  Therefore, to  develop this information, we selected a random probability sample of case  files to review.  The sample was drawn from an initial population of 103,946  Appeals cases closed in ACDS for fiscal year 2004.  However, since Industry  Case (IC) and Coordinated Industry Case (CIC) cases, which originate from  IRS’s Large and Mid-Size Business Division (LSMB), are complex and the  supporting documentation is voluminous, we excluded these 1,323 cases  from the population.  Therefore, the final population size was 102,623  cases.
Of the 165 cases selected in our sample, we reviewed 153 cases to  determine the results of the cases.  The remaining 12 cases could not be  analyzed because the files could not be located or essential Appeals  documents were not available.  We assessed the known characteristics of  the 12 cases not received against those of the 153 received for potential  systematic differences.  Based on this nonresponse bias analysis, we  concluded that it was acceptable to treat the 12 cases as missing at random.
We reviewed documents, such as the Appeals Case Memorandum and the  Case Activity Record, and determined whether the cases were fully  sustained, partially sustained, or not sustained.  In determining the extent  to which a case was sustained, we based our decision on the determination  made by Appeals in the Appeals Case Memorandum using the following  scale: “fully sustained” indicated that in our judgment Appeals agreed with  compliance on all issues appealed by the taxpayer; “partially sustained”  indicated that Appeals agreed with at least one but not all of the issues; and  “not sustained” indicated that Appeals did not agree with any of the issues.   We also reviewed the cases to determine the reasons the cases were not  sustained by Appeals.  Since cases could include several compliance issues,  there may have been multiple reasons why a case was not sustained.  We  recorded each decision and the reason for the decision cited in the Appeals  case file for a case not being sustained on a data collection instrument  (DCI) that we developed.  The analysts who participated in reviewing the  case files and recording the information on the data collection instrument  were knowledgeable about the appeals process and how to interpret the  information in the case files.
To ensure that the data entered on the DCIs conformed to GAO's data  quality standards, each completed DCI was reviewed by at least one other  GAO analyst.  The reviewer compared the data recorded on the DCI to the  data in the case files to determine whether he or she concurred with the  interpretation of the case files and the way the data were recorded on the  DCI.  When there were differing perspectives, the analysts met and  reconciled them.
Tabulations of the DCI items were automatically generated using a  statistical software package to develop case outcome information.   For  these analyses, the computer programs were checked by a second,  independent analyst.
We developed case outcome information for each of the Appeals  workstreams except IC and CIC.  For the CDP, Exam/TEGE, and OIC  workstreams, our sample sizes were large enough to generalize the results  separately for each workstream, or to have a margin of error small enough  to produce meaningful workstream estimates.
Because we followed a probability procedure based on random selection,  our sample is only one of a large number of samples that we might have  drawn.  Since each sample could have provided different estimates, we  express our confidence in the precision of our particular sample’s results as  a 95 percent confidence interval, plus or minus 8 percentage points.  This is  the interval that would contain the actual population value for 95 percent of  the samples we could have drawn.  For example, Appeals did not sustain 41  percent of the cases in the sample, which has a 95 percent confidence  interval of 33 percent as a lower bound and 49 percent as an upper bound.    Workstream estimates come from subsets of the sample.  Thus  workstream-specific estimates have larger confidence intervals due to the  smaller sample size.  Tables 4, 5, and 6 present the confidence intervals for  sample data presented in the report.
To determine how effectively the feedback project was being managed, we  reviewed documents supplied by Appeals and compliance program  officials, such as meeting minutes for the advisory boards and strategic  planning documents.  We also interviewed these officials and reviewed our  prior work on best practices for developing information that can be used to  improve agency performance.
To compute appeal rates, we compared compliance cases closed in fiscal  year 2003 by worksteam to the Appeals cases closed in fiscal year 2004.   Since Appeals typically required about a year to complete a case, the 2004  Appeals closings were cases that were most likely closed by compliance  programs during 2003.  Further, IRS uses a similar approach to compute  audit rates.  To identify the number of compliance cases by worksteam, we  used data published in IRS’s fiscal year 2003 Databook.  Data on cases  closed for the Innocent Spouse and Offer-in-Compromise workstreams was  not available in the Databook and was provided by IRS staff.
subsections: section_title: Data Reliability
paragraphs: We assessed whether the case results data contained in ACDS were  sufficiently reliable for our use.  We selected the first 100 cases from our  random sample of 165 cases to make this determination.  We interviewed  knowledgeable Appeals officials about the data, performed electronic  testing of relevant data fields for obvious errors in accuracy and  completeness, and collected and reviewed documentation about the data  and the system.  We also reviewed prior Treasury Inspector General for Tax  Administration reports.
Of the 100 cases selected for our sample, we reviewed 92 cases from all of  the Appeals workstreams except as mentioned earlier, the IC and CIC  workstreams.  The remaining 8 cases could not be analyzed because  essential Appeal documents were not available.  We compared documents  in closed Appeals cases, such as the Appeals Case Memorandum, to data in  ACDS.  However, Appeals did not always provide documentation for the  basis of the compliance determination; therefore, in some cases, we were  unable to determine if data, such as the amount of tax proposed by  compliance, were accurate.
We had Appeals verify data errors in fields that were specific to case results  information, such as the amount of revised tax and penalty, as well as the  closing code.  Due to the high error rate of some data fields in our sample,  we reviewed internal controls used in the processing of case results data at  one Appeals area office.  This review consisted of observation and inquiry  of Appeals officials on Appeals’ case processing procedures and review of  Appeals documentation.  We also spoke to officials in Appeals  headquarters concerning weaknesses identified in Appeals’ internal  controls.
On the basis of our data reliability review of ACDS, we determined that  data in ACDS were not sufficiently reliable for our use.  Instead of relying  on that data, we used data developed from our sample of Appeals cases and  continued our analyses of Appeals’ internal controls.
We conducted our review at Appeals headquarters in Washington, D.C., and  one Appeals area office from October 2004 through October 2005 in  accordance with generally accepted government auditing standards.
section_title: Appeals Workstreams
paragraphs: Appeals’ workload is organized into eight workstreams.  These  workstreams include cases that have similar characteristics rather than  reflecting the IRS operating division where they originated.  For example,  cases in the Collection Due Process workstream include only appeals by  taxpayers under provisions of the IRS Restructuring and Reform Act 1998,  which authorizes an independent review by Appeals of proposed levies and  filed liens.  These cases could originate in either the Wage and Investment  Division or the Small Business and Self-Employed Division, since either  division could propose a levy or file a lien.  Other workstreams include a  wide range of cases from across IRS operating divisions.  The Exam/TEGE  workstream includes appeals for compliance actions, including  recommended assessments and proposed penalties originating from much  of IRS’s reporting and filing compliance program, with the exclusion of  LMSB cases.  These appeals can include diverse issues, such as  recommended assessments related to the Earned Income Tax Credit or  large charitable organizations, such as universities or hospitals.  During  fiscal year 2004, Appeals completed nearly 104,000 cases.  Table 7 describes  these workstreams and the IRS operating divisions where these cases  where proposed.
section_title: Appeal Rates
paragraphs: Taxpayers in each workstream requested Appeals of recommended  assessments or other compliance actions, such as proposed levies and filed  liens, at widely differing rates.  To compute the appeal rate for each  workstream, we compared the number of compliance cases closed for each  workstream to the number of cases Appeals closed.  We compared fiscal  year 2003 compliance case closings to fiscal year 2004 Appeals case  closings because Appeals averaged 260 calendar days during fiscal year  2004 to complete its work on a case.  For example, as reported in table 16  of the IRS Databook for 2003, during fiscal year 2003, IRS filed 548,683  notices of federal tax liens, served 1,680,844 notices of levy, and made 399  seizures for a total of 2,229,926 compliance actions.  Each of these actions  could be the basis for a CDP appeal.  During fiscal year 2004, Appeals  completed work on 32,226 CDP cases, for an Appeal rate of 1.445 percent  or 1 percent.
Other approaches could be used to compute appeal rates.  Our analysis  used compliance cases closed as the basis for measuring appeal rates  because (1) a uniform, published source of data was available and provided  data on six of the eight Appeals workstreams and (2) it broadly compares  IRS compliance programs to the Appeals program.  Another approach for  measuring appeal rates, for example, could use only cases closed where  IRS recommended an additional tax assessment and not include cases  where no tax was proposed, because taxpayers would not have a basis for  requesting an appeal.  In some programs this difference may be substantial.   For example, in the Offer-in-Compromise program, according to  unpublished data provided by IRS, 58 percent of fiscal year 2003 Offer-in- Compromise cases were closed by Compliance because the offer was not  processable or was returned to the taxpayer.  Accordingly, the taxpayer did  not have a basis for an appeal.  Eliminating these cases from the Offer-In- Compromise cases closed in 2003 would more than double the appeal rate  from 13 percent to 31 percent.
However, limited data are available to use other approaches for computing  appeal rates. For example, about 30 percent of the cases closed in Appeals’  second largest workstream, Exam/TEGE, originated from the Earned  Income Tax Credit and the Automated Underreporter programs.  Data were  not published on the proportion of these cases that were closed with  recommended assessments.
section_title: Related GAO Products
paragraphs: Tax Administration, Planning for IRS’s Enforcement Process Changes  Included Many Key Steps but Can Be Improved. GAO-04-287. Washington,  D.C.: January 20, 2004.
Tax Administration, IRS Needs to Further Refine Its Tax Filing Season  Performance Measures. GAO-03-143. Washington, D.C.: November 22,  2002.
IRS Modernization: IRS Should Enhance It’s Performance Management  System. GAO-01-234. Washington, D.C.: February 23, 2001.
Standards for Internal Control in the Federal Government. GAO/AIMD- 00-21.3.1. Washington, D.C.: November 1999.
Executive Guide: Measuring Performance and Demonstrating Results of  Information Technology Investments. GAO/AIMD-98-89. Washington, D.C.:  March 1998.
Executive Guide: Effectively Implementing the Government Performance  and Results Act. GAO/GGD-96-118. Washington, D.C.: June 1996.