metadata: authors: 0: author: Barbone, Luca
1: author: Das-Gupta, Arindam
2: author: De Wulf, Luc
3: author: Ha
docna: 0: docna: Reforming tax systems - the World Bank
            record in the 1990s
docty: Policy Research Working Paper
subtopic: Tax Administration,Macro-Fiscal Policy,Taxation & Subsidies,Economic Adjustment and Lending,Tax Law,Public Sector Economics,Tax Policy,International Trade and Trade Rules,Public Finance Decentralization and
            Poverty Reduction,Global Environment
teratopic: Finance and Financial Sector Development
abstracts: cdata!: The main constraint on world Bank
            operations in tax and customs administration is the
            Bank's inadequate institutional framework for
            accumulating knowledge from loan operations, concludes this
            review of the Bank's record on reform of tax systems in
            the 1990s. The Bank's theoretical basis for reforming
            tax and customs administration is still rudimentary. Recent
            theories stress the importance of institutions that harness
            voice and improve transparency and contestability, but there
            is little evidence that reform of these factors alone makes
            tax administration more effective. Improvements are needed
            in pre-project diagnosis and project design, especially for
            examining accountability, administration costs, managerial
            autonomy, performance incentives for staff, taxpayer equity
            and services, and environmental factors. Pre-project work
            could draw more systematically on lessons from previous
            experience. Institutional components of project design have
            been biased toward organization, manpower upgrading, and
            procedures related to information technology. Too little
            attention has been paid to improving accountability,
            administrative cost-effectiveness, and anticorruption
            institution-building. Projects have made inadequate use of
            different kinds of performance indicators, with little
            uniformity in those applied. Methods used to evaluate
            project outcomes could be better and more uniform.
            Suggestions for future Bank operations: 1) doing better
            background work and articulating a strategy and
            comprehensive framework for Bank involvement in reform of
            tax administration. 2) Possibly supporting and strengthening
            regional tax administration associations, which could serve
            as catalysts for change. 3) Strengthening partnering and
            supporting private sector consultant organizations, so they
            can manage major components of administrative reform. 4)
            Institutionalizing the accumulation of knowledge about tax
            administration (which might require changing staff
            recruitment, the mix of staff skills, and training plans).
            The authors provide recommendations for improving project
            diagnosis, design, performance indicators, and appraisal, as
            well as a short list of projects that serve as guides to
            good practice.
repnme: repnme: Reforming tax systems - the World Bank
            record in the 1990s
display_title: Reforming tax systems - the World
            Bank record in the 1990s
listing_relative_url: /research/1999/11/439484/reforming-tax-systems-world-bank-record-1990s
topicv3: Trade m1326924 698,Tax and Revenue Policy and Administration
            m1327008 720,Environment, Natural Resources and Blue
            Economy m1327391 566,Taxation m1327615 1121,Tax Policy m1327616 983
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Finance and Financial Sector Development
origu: Public Sector Governance (PRMPS)
no_of_pages: 44
model_extractions: | POLICY RESEARCH WORKING PAPER 2237

# Summary findings

The main constraint on World Bank operations in tax and customs administration is the Bank's inadequate institutional framework for accumulating knowledge from loan operations, concludes this review of the Bank's record on reform of tax systems in the 1990s.

The Bank's theoretical basis for reforming tax and customs administration is still rudimentary. Recent theories stress the importance of institutions that harness voice and improve transparency and contestability, but there is little evidence that reform of these factors alone makes tax administration more effective.

Improvements are needed in pre-project diagnosis and project design, especially for examining accountability, administration costs, managerial autonomy, performance incentives for staff, taxpayer equity and services, and environmental factors. Pre-project work could draw more systematically on lessons from previous experience.

Institutional components of project design have been biased toward organization, manpower upgrading, and procedures related to information technology. Too little attention has been paid to improving accountability, administrative cost-effectiveness, and anticorruption institution-building.

Projects have made inadequate use of different kinds of perforinance indicators, with little uniformity in those applied.

Methods used to evaluate project outcomes could be better and more uniform.

_Suggestions for future Bank operations:_

* Doing better background work and articulating a strategy and comprehensive framework for Bank involvement in reform of tax administration.

* Possibly supporting and strengthening regional tax administration associations, which could serve as catalysts for change.

Strengthening partnering and supporting private sector consultant organizations, so they can manage major components of administrative reform.

* Institutionalizing the accumulation of knowledge about tax administration (which might require changing staff recruitment, the mix of staff skills, and training plans).

The authors provide recommendations for improvin-g project diagnosis, design, performance indicators, and appraisal, as well as a short list of projects that serve as guides to good practice.

- This paper a product of the Public Sector Management Division, Poverty Reduction and Economic Management Network - is part of a larger effort in the network to draw on lessons of past Bank activity in order to pursue professional excellence and maximum client impact. Copies of the paper are available free from the World Bank, 1818 H Street NW, Washington, DC 20433. Please contact Luca Barbone, room J7-1 _19,_ telephone _202-473-2556,_ fax 202-473-8466, email address lbarbone@_worldbank.org. Policy Research Working Papers are also posted on the Web at w,vw.worldbank.org' research/workingpapers. The other authors may be contacted at oldmonk87@yahoo.com, ldewulf(@xworldbank.org or ahanssonl@worldbank.org. November 1999. (35 pages)

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to ger the findings out quickly, even if the presentations are less thain fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations. and conclusions expressed in this paper are entirely those of the authors. They do not ntecessarily represent the view of the World Banik, its Executive Directors, or the countries they represent._ Produced by the l'olicy Research Dissemination Center
## REFORMING TAx SYSTEMS: **THE** WORLD BANK RECORD IN THE 1990S

Luca Barbone, Arindam Das-Gupta, Luc De Wulf and Anna Hansson

# Tax Policy and Administration Thematic Group The World Bank

_The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the World Bank, its Executive Directors, or the countries they represent._
# REFORMING TAX SYSTEMS: TBE WORLD BANK RECORD IN THE 1990s

## **Summary**

This paper provides an assessment of the World Bank's record in helping its clients promote tax system reforms during the 1990s. World Bank advice and lending operations have been concentrated on both policy reform and reform of the administration of tax systems. This paper is primarily concerned with the latter. Its main conclusion is that substantial improvements can be expected in project design, execution, and, ultimately, effectiveness, if a more concerted approach to institutional analysis of the tax environment is adopted.

_Objectives and approach._ The use of institutional analysis in the design of operations in support to refonn of public institutions, of which tax and customs administration are important areas of concentration, is now an essential requirement in the World Bank. No formal methodology for institutional analysis has yet been developed in this area. However, institutional concerns have always been high in the preoccupation of task managers, who have striven to incorporate them even without formal methodological guidelines. This paper proposes a methodology for institutional analysis, based on ongoing work in the Bank and elsewhere, and then reviews tax and customs reforms operations during the 1990s to assess the extent to which they incorporate this methodology. As can be expected, since hindsight is 20/20, deficiencies are not hard to identify. The paper then proceeds to recommend steps to improve the institutional analysis methodology employed. It should be noted that the paper does not attempt, except in minor ways, to assess the effectiveness and sustainability of the operations reviewed, since many of them are still ongoing, or completion information is not available.

_Analysis in this review and major findings:_ Of about 120 lending operations in the 1990s with components seeking to strengthen some aspect of the tax system, 43 projects with an outlay of US$3.8 billion had major tax or customs administration reform components and 40 others had important components addressing other aspects of tax systems. Regional emphasis has gradually shifted from Latin America and the Carribean (LAC) and Africa (AFR) to East Europe and Central Asia (ECA) during the decade, and there has been a decline in the total outlay from US$5.1 billion during 1990-94 to US$3.6 billion during 1995-1998.

For the 43 operations with an administrative focus, major objectives cited most often were revenue enhancement (40%), strengthening administrative institutions (37%) and promoting macroeconomic stability and growth (28%). Improving accountability taxpayer education and services was a major objective in very few operations (7% and 5% respectively), while no project had a major focus on strengthening voice and participation.

The framework used in this paper to examnine pre-project diagnostic work of tax systems in these 43 operations assesses project documents according to how adequately they addressed key features of the operating environment (policy formulation, accountability, service delivery effectiveness and efficiency) according to their performance, capacity, institutional arrangements and exogenous constraints.
Turning to components included in project design, computerization and related procedural, organizational and manpower reforms were most popular, the former figuring in all technical assistance and related (TLSR) loans. On the other hand, besides the limited presence of anticorruption and institutions for accountability and budgetary resources, the conclusions for structural and sectoral adjustment loans (SALs/SECALs) in a 1997 Operations Evaluation Department (OED) study,' that a greater focus is needed on "conducive legal environment, taxpayer education, information systems (used by tax departments), audits, withholding and presumptive taxation and improvements in VAT administration" are confirmed for structural adjustment operations reviewed here and, excluding VAT administration and taxpayer audits, technical assistance operations as well.

The specification and use of performance indicators, especially quantitative indicators, was limited, especially in SALs/SECALs and there was little uniformity across loans. Furthermore, few operations established pre-project benchmarks for indicators, and there was limited correspondence between indicators used in pre-project and post-project appraisals. Insufficient infornation on the rationale for sequencing of different project components prevents any firm conclusion from being drawn here. Nevertheless, instances of both effective and poor sequencing are reported in post-project evaluation documents.

Only a handful of projects undertook quantitative project appraisal and risk assessment. Where these were done, methods used were not uniform across projects and appear capable of improvement.

Since many operations are still active, a relatively small set of post-project Implementation Completion and (OED) Project Audit reports were available. This sample, however, confirms the importance of Bank and borrower performance and quality at entry for project success found in earlier work in the Bank. In addition, lessons distilled from projects largely pertained to project implementation rather than to improving the content of projects at the design stage.

_Suggestions for future operations:_ Upstream work to draw up a strategy for Bank operations in the area of revenue administration, including the content and the process of such operations, is a major suggestion made here. Among process factors, outsourcing, strengthening of external partnerships and strengthening the skill mix of Bank staff are important components of the strategy.

The need for a standard diagnostic tool<sup>2</sup> , a greater focus on and integration of governance components in project design, the development and use of a standard set of administration performance indicators and also a standard appraisal methodology are other important suggestions made here. In addition, a formal hypothesis-testing framework to distill lessons about best practices in revenue administration is needed to strengthen the content of future Bank operations.

'Datta-Mitra (1997).

2 example, the tool developed within the Tax Policy and Administrative Group by Gill (1998).

ii
# REFORMING TAx SYSTEMS: Ti-E WORLD BANK RECORD IN THE 1990s

This review of Bank projects in the 1990s with tax or customs administration components aims to analyze the extent and nature of their institutional focus. A second objective is to identify and assess the performance indicators used. The purpose of the review is to draw lessons for future Bank work in the area of tax systems advice.

The paper discussed the rationale for involvement in tax reform; the nature of pre-project diagnostic work in projects; the composition of the 83 projects with tax systems components; key features of project design; the nature of performance indicators specified in projects; project appraisal; selected aspects of project implementation; and evaluation of project outcomes. Conclusions and suggestions for future Bank loans for administrative reform are then offered.

## 1. **The Why: Rationale for Bank Lending for Tax Systems Reform**

The corporate objective of the World Bank group is to reduce world poverty. Growth and macroeconomic stability are necessary pre-conditions to reduce poverty. The need to strengthen revenue generation has traditionally been a self-eviden **t** ask of development, which the Bank has chosen to pursue with a variety of means.

Whereas policy advice on revenue architecture has been and continues to be a prominent part of the Bank's analytical support to countries, there has been increasing realization that good policies alone are insufficient for sustained, reliable revenue generation, which in turn is a necessary condition for both macroeconomic stability and provision of adequate social services by the Government. In the medium to long term, evidence shows that strong institutions, particularly institutions of governance are as if not more importan **t** o achieve sustained growth and poverty reduction. There is also some evidence showing that this applies to revenue generation. Good institutions in government ensure effective policy making, accountability for their actions of the government and its functionaries to citizens, and effective and efficient service delivery. Key requirements for these outcomes are transparent procedures, mechanisms to involve citizens in decision making and evaluation, and enforcement of accountability especially to counter corruption and waste. Consequently **_institutional strengthening and governance capacity building_** have, of late, become integral to the Bank's development strategy and to its technical assistance (TA) operations.3 In the next few sections, we will examine the extent to which these concerns have been already incorporated in Bank work in the tax system reform area.

> 3 For example, the Operation Policy Department's _Handbook of Technical Assistance_ (1993) defines two kinds of TA, Institutional Development TA and Gap Filling TA and proceeds to devote the bulk of the Handbook to the former. Furthermore, it is critical of traditional approaches to gap filling TA. The list of core institutional functions listed in the Handbook include policy making; financial resource generation and allocation; managing personnel; communication; management; evaluation; and meeting citizen's needs.

1
# **2. The How: Features of World Bank Intervention in Tax System Reform**

_Magnitude._ There is no doubt that the World Bank, _de facto,_ is strongly committed to revenue system reforms among its member countries. During the 1990s, a total of about 120 loan operations in 67 countries had components dealing with reforming some aspect of the tax system<sup>4</sup> and involved an outlay of about US$13.9 billion (Table 1). In 37 of these loans, tax system reform was a component of a broader structural reform program. In the other 83 operations in 55 countries involving an outlay of US$8.5 billion, greater attention was devoted to the tax system. In 43 of these projects, modernization of tax or customs administration was a major or a principal component and involved a loan outlay of around $4.7 billion.

|**T**|**able 1. Projects**<br>(83 projects wit|**Classified b**<br>h a tax system|**y Type of**<br>component|**Lending Instrument**<br>during the199<sup>0</sup>s)||
|---|---|---|---|---|---|
||Number of<br>Countries|Number o|f Projects|Value (US$ million)|Average per Project<br>(US$ million)|
|SALs and SECALs|36|51||7673.7|150.5|
|TALs|17|20||351.9|17.6|
|LILs, SILs and RILs|9|12||457.3|38.1|
|All Projects|_55_|83||8482.9|102.2|
|SAL: Structural adjustm<br>SECAL: Sectoral Adjust<br>TAL: Technical Assista|ent Loan.<br>ment Credit.<br>nce Loan.||LIL: Learn<br>SIL: Speci<br>RIL: Reha|ing and Innovation Loa<br>fic Investrnent Loan.<br>bilitation Loan.|n.|

Table 2 lists the main objectives of the 43 loans with a major tax or customs modernization component. The list of possible objectives in the table was pre-specified in line with the institutional focus of this review (this explains the zero value for the objective _Strengthen citizen's voice and participation)._ The table shows that explicit institutional objectives were the focus of relatively few projects. Not surprisingly, the most common institutional development objective was direct strengthening of tax administration institutions (37% of projects), which mainly translates into strengthening organization, management, procedures, and information processing. Key institutional objectives such as strengthening accountability, anti-corruption and voice were major objectives in relatively few projects. Macroeconomic stability, revenue enhancement and, possibly, improving taxpayer compliance are the three non-institutional objectives in the list.

> 4 Here "tax system" is taken to include tax structure, the administration of taxes including customs administration, the institutional framnework of tax policy and the environmental and institutional framework of tax administration.

2
**Table 2. Major Project Objectives**

|(43 projects with a major|administration component)||
|---|---|---|
|Objective|Number of projects|Percentage of<br>projects|
|1. Revenue enhancement|17|39.5|
|2. Strengthen administrative institutions|16|37.2|
|3. Macroeconomic stability|12|27.9|
|4. Improve environmental institutions|8|18.6|
|5. Strengthen administrative capacity|7|16.3|
|6. Improve taxpayer compliance|6|14.0|
|7. Strengthen accountability|3|7.0|
|8. Strengthen anti-corruption|2|4.7|
|9. Improve taxpayer services|2|4.7|
|10. Improve taxpayer education|2|4.7|
|11. Strengthen voice and participation|0|0.0|
|12. Others|9|20.9|
|Number of major objectives in 43 projects|85||
|Notes: Each project can have more than one majo|r objective||

# **Basic Features of Projects With Tax System Components**

_Basic statistics of projects included in the sample used in this review are presented here. Projects are classified by loan type, by the importance of tac system components in loan design, by region and by the type of tax system components included in the project._ Table 1 showed that the bulk of loan activity for tax system reform was through SALs/SECALs and that these structural adjustment loans were, on average 5 to 8 times larger than the average TLSR loan (i.e. TALs, LILs, SILs and RILs). Almost all SALs/SECALs involved more than just tax system reform components. Information about loan fractions specifically for tax system reform was only available in a few cases. Instead, in Table 3, where the distribution of loans according to the importance of the tax system component is given, average loan amounts for projects where this component is the principal or a major component can be seen to be around $110 million for SALs/SECALs and $25 million for TLSR loans.

|**Table 3. Projects C**<br>(83 projec|**lassified by Importanc**<br>ts with a tax system compon|**e of Tax System Comp**<br>ent during the 1990s)|**onents**|
|---|---|---|---|
||Number of Countries|I Number of Projects|Value (US$ million)|
|**All Projects in the Sample**||||
|Major or Principal Component|31|43|4712.2|
|One of Several or Minor Component|24|40|3770.8|
|TALs, LILs, SILs and RILs||||
|Major or Principal Component|20|22_|550.2|
|One of Several or Minor Component|1<br>8|10|259.0|

Loan volume was lower during 1995-1998 compared to the first 5 years of the 1990s (Table 4) for both categories of loans. Furthermore, regionally, emphasis shifted from Latin America and the Carribean (LAC) and Africa regions (AFR) to Europe and Central Asia (ECA). No new loans 3
were made to the Middle East and North Africa (MNA) and South Asia region (SAR) during 1995-98, though at least 3 new loans are in the pipeline.

**Table 4. Projects Classified by Region**

|||(83 proj<br>**All Pr**|ects with a tax<br>**ojects**|system com|ponent durin<br>|g the 1990s)<br>**TALs, LILs, S**|**ILs and**RIL|s|
|---|---|---|---|---|---|---|---|---|
||1990|-1994|1995|-1998|1990|-1994|1995|-1998|
||Projects<br>(Number|Value<br>) (US$107)|Projects<br>(Number)|Value<br>(US$107)|Projects<br>(Number)|Value<br>(US$ 107)|Projects<br> (Number|Value<br>) (US$ 107)|
|AFR|21|1205.8|12|980.8|6|142.1|1|23.8|
|ECA|3|487.0|17|2108.2|2|37.0|9|175.2|
|EAP|4|203.0|1|25.0|4|203.0|1|25.0|
|LAC|16|2046.0|3|252.2|6|131.0|2|52.2|
|MNA|3|444.9|1|80.0|1|19.9|0|0.0|
|SAR|2|650.0|0|0.0|0|0.0|0|0.0|
|TOTAL|<br>49|5143.7|34|3553.2|19|533.0|13|276.2|

Disaggregating loan components firther, customs reform components were most frequent overall, though tax administration reform components predominated in TLSR operations (Table 5). Of special interest is the importance given to the VAT in tax structure modernization and to computerization in the area of administrative modernization.

**Table 5. Projects Classified by Type of Tax System Component**

(83 projects with a tax system component during the 1990s)

|||**All Projects**||**TALs, **|**LILs,SILs a**|**nd RILs**|
|---|---|---|---|---|---|---|
||Number<br>of<br>Countries|Number<br>of<br>Projects|Value<br>(US$ million)|Number<br>of<br>Countries|Number<br>of<br>Projects|Value<br>(US$ million)|
|With a Customs Reform Component|41|57|6203.3|11|14|269.6|
|With a Domestic Tax Administration<br>Component|45|64|5819.9|24|29|679.2|
|With a Domestic Tax Reform|30|46|4920.4|8|9|175.0|
|Component|||||||
|With a Tariff Reform Component|31|41|4773.9|2|2|78.0|
|With a VAT Component|22|31|3633.2|_5_|6|67.5|
|With a Computerization Component|31|38|3050.0|22|26|670.0|
|With a focus on particular taxes only*|6|7|479.3|1|1|24.3|

Notes: Projects may be counted more than once.

* Included are social security, property taxes, forest tax, petroleum tax, stamp tax and their administrations.

# **2. Building Sound Projects: Pre-project Diagnostic Work**

_After describing a framework for institutional assessment (Table 6), information on the presence of diagnostic work in different projects is presented Six broad areas are examined. tax policy institutions; tax structure; tax or customs administration effectiveness; tax or customs administration efficiency; the institutional and economic environment of administration; and the institutions and capacity of the tax administration. This discussion is then summarized and the_ 4
_comprehensiveness of diagnosis in the typical project discussed The section closes with an examination of the extent to which pre-project work discusses lessons from earlier experience._ In a recent Bank study, Evans and Battaile (1998) present evidence that good Bank performance is a key determinant of successful project outcomes, being dominated only by borrower performance. The most important ingredient of good Bank performance identified in the study is the quality-at-entry of a project. Thus the importance of thorough pre-project preparation and diagnostic work.

In order to evaluate the way in which Bank projects have incorporated institutional concerns in their design, we have analyzed existing documentation against a framework summarized in the matrix in Table 6. In this framework,<sup>5</sup> Performance is a function of Capacity, Institutions and Exogenous constraints. Furthermore performance is disaggregated into policy formulation, accountability and service delivery with the latter being divided further into effectiveness and efficiency. Effectiveness refers to the quality and quantity of services provided while efficiency deals with the unit cost of service provision.<sup>6</sup> Tables Al through A6 in appendix report the extent to which different diagnostic areas were examined in the 43 projects with major tax/customs administration components. A summary, which also shows the overall comprehensiveness of diagnostic exercises in the typical project, is in Table 7.

It should be made clear that, in compiling project data, no judgement was made about the adequacy or completeness of the diagnosis in any category in SARs or MOPs since other preproject documents, if any, were largely unavailable for this review.<sup>7</sup> On the basis of the data in table 7, we can infer the following conclusions:

- _Tax policy institutions and tax structure:_ Tax policy making and analysis institutions received attention in only about one quarter of projects in the sample. Regarding tax structure and revenue performance, diagnosis was typically limited to standard tax features such as the tax to GDP ratio; mix of taxes; tax rates, bases and exemptions; and their recent reform history. Within this set, disaggregated analysis was less common than aggregate information. Quantitative and formal analysis, such as analysis using relatively standard tools like effective rates, buoyancy and elasticity was relatively rare. Formal econometric modeling was reported in only 2 projects. Furthermore, the extent of analysis of these areas present in TLSR projects tended to be lower than in SALs/SECALs. Infirmities diagnosed were typically limited to only the major features that were against good practice, such as high and differentiated rates, narrow bases, multiple exemptions and input taxation (i.e. absence of a VAT). Most projects, furthermore, did not cite references to Bank or other work discussing "good" tax design.<sup>8</sup>

Inspired by Nick Manning and Brian Levy in various unpublished work.

> **6** Further disaggregation of the categories in the table is to be found in the diagnostic and project formulation framework of Gill (1998), though diagnostic categories are classified differently there. Other, less detailed, diagnostic frameworks are to be found in Bagchi, Bird and Das-Gupta (1995), Datta-Mitra (1997), Annex 4, and in Vazquez-Caro, Reid and Bird (1992).

> 7In point of fact, diagnosis in most SARs and MOPs, if not supplemented by other documents, was mostly inadequate and furthermore, coverage within different categories was typically incomplete. s See, for example, World Bank (1991), Khalilzadeh Shirazi and Shah (1991) and Gillis, Sicat and Shoup (1990).

_5_
- _Effectiveness:_ Analysis of effectiveness (meaning 'both raising adequate revenues and enforcing compliance, and providing a high level of services to taxpayers) was relatively limited and mostly qualitative, especially in SALs/SECALs. Quantitative examination of this key aspect of administrative performiance was absent in the bulk of projects.

- _Efficiency:_ The focus of analysis, especially in TLSR projects, was typically technology driven, examining the lack of sufficient automation and the consequent inefficiency of different activities or functions. Privatization of functions was mainly limited to discussion of tax collection through banks, withholding and use of pre-shipment inspection services. Manpower analysis focused on overstaffing and less commonly on the efficiency of deployment to different functions. A major weakness, especially of most SALs/SECALs was the lack of attention to the cost of tax administration and tax administration budgets.

- _Administration Environment:_ General economic conditions followed by taxpayer compliance levels and tax policy constraints were the environmental factors analyzed in the bulk of projects. Analysis of institutions was less common, with SALs/SECALs having a higher incidence, often by virtue of these operations having other loan components covering a broader area. However, discussion was disjoint, with attempt seldom being made to draw conclusions for the evaluation of tax or customs administration.

- _Institutions and Capacity:_ Analysis in this area focused, in most projects, on organization structure, technology, manpower quality, and tax administration procedures. Few projects paid attention to the presence and effectiveness of accountability institutions such as external audits or other external monitoring (e.g. voice and feedback) or to administrative autonomy and budgets. In sum, inadequate attention was paid in sample projects to institutional determinants of performance.

**_6_**
||**Table**<br>|**6. A Framework for Diagnosis of**<br>|**Tax**<sup>**System Weaknesses**</sup><br>|d il|
|---|---|---|---|---|
|Characteristic|Perfonnance (usually<br>quantitative indicators)<br>|Capacity<br>|Institutions<br> <br>|Exogenous an envronmenta<br>constraints<br> <br>|
|Policy<br>Formulation|*<br>Simplicity, equity and<br> <br>Comprehensiveness of<br>Administrative Law<br>*<br>Simplicity, equity and<br>Comprehensiveness of<br>taxpayer related procedures<br>*<br>Effectiveness of strategic<br>management decisions|*<br>Human resource quality and<br>quantity for policy<br>formulation and research<br>*<br>Training facilities<br>|*<br>Division of powers<br>*<br>Legal provisions governing<br>administration<br>*<br>Administrative autonomy<br>*<br>Budget determination<br>procedures<br>'<br>Organization structure<br>*<br>Political interference in<br>administrative decisions<br>Appropriateness of tax<br>structure<br>|*<br>Level of economic<br>development<br>*<br>Openness<br>*<br>Commercial development<br>*<br>Extent of transition to market<br>*<br>Accountancy profession and<br>standards<br>*<br>Financial and banking<br>development<br>Literacy<br>S<br>Legal conventions,<br>|
|Accountability|*<br>Corruption<br>*<br>Revenue loss due to<br>administrative lapses<br>*<br>Taxpayer survey<br>|*<br>Human and non-human<br>resource quality and quantity<br>in management inspection,<br>audit, vigilance and<br>anticorruption<br>*<br>Training facilities for<br>accountability staff<br>*<br>Civil society involvement|*<br>Internal and external audit<br>*<br>Ombudsman<br>*<br>Taxpayer survey process<br>*<br>Vigilance<br>*<br>Penalty structure and<br>effectiveness for administrators<br>*<br>Reward structure and<br>effectiveness<br>*<br>Performance evaluation<br>procedure and execution for<br>staff and administrative units<br>|constitutional law<br>*<br>Law and order<br>*<br>Compliance attitudes of<br>taxpayers<br>*<br>Credibility of govermnent<br><sup>*</sup><br>Government stability<br>*<br>Civil service conditions|
|Service<br>delivery:<br>Effectiveness|*<br>Revenue to potential<br>revenue indicators<br>*<br>Activity-wise effectiveness<br>indicators<br>|*<br>Human and non-human<br>resource quality and quantity<br><sup>*</sup><br>In administrative activities<br>*<br>Recruitment Career paths,<br>|0<br>Adequacy of procedures for<br>different internal activities and<br>taxpayer related activities<br>including:<br>||
|Service<br>delivery:<br>Efficiency|*<br>Revenue to cost indicators<br>*<br>Time indicators<br>Taxpayer survey<br> <br>|promotion and transfer<br>policies<br>Wages, bonuses and pay<br>determination<br>*<br>Training facilities<br>*<br>Automation<br>*<br>Budget size of administration<br>*<br>Financial, personnel,<br>equipment and activity<br>management<br>*<br>Feasibility and extent of<br>outsourcing and privatization|Compliance, valuation,<br>taxpayer identification, tax<br>audit, taxpayer education,<br>taxpayer services, tax<br>collection, tax sanctions, tax<br>appeals, border inspection and<br>clearance procedures, special<br>customs procedures,<br>warehousing, preshipment<br>formalities, post clearance<br>checks, information exchange<br>and sharing arrangements||

7
**Table** 7. **Comprehensiveness of Diagnosis**

|(43 projects with a major a<br>(Percentage of categories exa|dministration component)<br>mined: average per project)|||
|---|---|---|---|
|Area of Diagnosis|Number of sub-areas<br>examined for this review|TALs, LILs,<br>SILs, RILs|SALs and<br>SECALs|
|Taxpayer obligations, laws and procedures|1|57.1|61.9|
|Tax Structure Indicators and Analysis|13|48.3|61.6|
|_Of which buoyancies/ elasticities! otherformal analysis_|2|17.5|19.0|
|_recent reform history_|2|54.5|87.5|
|Environmental Indicators and analysis|8|47.7|34.9|
|Institutional Indicators and analysis|13|42.6|28.6|
|_Of which budget size and determination procedures_|2|7.1|14.3|
|Efficiency indicators and analysis|11|37.6|19.7|
|_Of which tcac collection cost indicators_|29.5||0.0|
|Effectiveness indicators and analysis|7|30.7|17.7|
|_Of which corruption indicators_|I|22.7|23.8|
|Tax Policy Institutions|I|22.7|28.6|
|**Average number of categories examined**|**63**|**39.2**|**34.1**|
|Notes: 1. See the notes to Table 4.3.||||

2. In case one or more areas not included in the basic inventory used for this study were examined in a project, a "normalizing" procedure whereby both numerator and denominator are increased by the number of "other" areas, was used to compute the diagnosis percentage for the project.

In sum, in the average project less than 40% of diagnostic categories considered in this review were included in the diagnosis. Furthernore, the failure to use quantitative perforrnance indicators is even more widespread.

One should note however that averages mask substantial differences. To highlight the "positives" available in our sample, two of a handful of exceptional projects are reviewed in Box 2 and a case study is offered in Box 3 below. In all three cases, it is evident that the project team took pains to ensure that the information available from the diagnostic work was first of all sufficiently comprehensive, and subsequently incorporated into the design of the operation.

# 5. **What Have We Tried to Accomplish? Key Features of Project Design**

_This section examines the extent to which different components are present in projects in the sample, the comprehensiveness or, conversely, degree offocus of the average operation and the sequencing of components. Corresponding to the framework of the previous section, components are divided into 4 broad areas, tax structure; the tax administration environment; different functions of tax administration; and institutional strengthening and capacity building. Special attention is given to trade tax and customs reform projects. The coordination of design with other projects and key features of implementation planning are then examined_ 8
<!-- Start of picture text --> Box 2: Pre-Project Diagnosis in ColombiaW and Jamaica;<br>Careful diagnosis of tax administration performance, tracing weaknesses to inadequate institutional, capacity or<br>environmental causes is important for sound project formulation. The tax administration reform project component in the<br>1993 Colombia Public Financial Management project and the 1994 Jamaica Tax Administration Refonn Project are among<br>the handfull of projects with relatively comprehensive diagnostic work. Importantly, in addition to the diagnostic work, both<br>projects drew on lessons learned from previous tax administration projects both for project design and for the<br>implementation. In both projects, diagnosis and lessons of experience were closely tied to project design.<br>Colonmia: The diagnostic work (Annex 9 of the Staff Appraisal Report, pp. 1-22) examines:<br>1. Administration reform history.<br>2. The economic base covering major phases in recent fiscal and economic history, economic growth and its sectoral<br>pattems, international capital flows, inflation, and the distribution of voluntary and enforcement based tax collection<br>according to categories of taxpayers.<br>3. Legislative developments in substantive tax laws and recent tax reforn history.<br>4. Reform of procedura l aws (e.g. filing and withholding obligations, tax penalties, tax amnesties, tax registration and<br>taxpayer documentation).<br>5. Organizational reforrns within the administration and stemmning from such things as out-sourcing tax collection to banks.<br>6. Induction of IT and associated managerial and procedural reforms.<br>7. Infrastructure for human resource development, personnel policies, developments and incentive structures.<br>8. Operational procedures including tax auditing, collection and internal administration.<br>9. Organizational structure of administration (including detailed organization diagrams at different levels starting with<br>inter-department coordination in the parent ministry).<br>On the basis of this diagnosis, key areas of weakness in the legal frnework, informnatics and resource and operations<br>management were identified and form the basis of project design.<br>Jamaica: The diagnostic work in the Staff Appraisal Report for this project is even more detailed than that for Colombia,<br>and is presented in 9 Annexes covering 61 pages. On the basis of the diagnosis key weaknesses, in the tax administration are<br>identified (Section B of the SAR) and a strategy for the reform of tax administration formulated (Section C of the SAR). The<br>major areas covered in the diagnosis are:<br>1. The tax structure including overall revenue performance and tax by tax discussion of rates, base, exclusions and revenue<br>performance.<br>2. The withholding tax structure and its management.<br>3. Formal estimates of revenue collection gaps by tax, by selected taxpayer types, by economic sector and activity wise<br>(e.g. non-filers versus under reporting).<br>4. A discussion of tax policy issues requiring resolution to increase taxpayer equity and reduce economic distortions.<br>5. Computerization of the tax admninistration<br>6. Training capacity and provision in the tax administration.<br>7. Taxpayer auditing.<br><!-- End of picture text --> 8. The legal and procedural framework: of tax administration (including resources organization and management of audits and investigation, equipment shortages and attitudinal concerns: searches and field operations; enforcement powers of administration; tax assessment; tax appeals; tax collection; administrative discretion; and harnonization of provisions in different tax statutes).

9. Management of tax collection.

From a comparison of the areas covered in the two projects it will be clear that there are areas that are not covered in each diagnosis. Furthermore, the depth of coverage is not uniform in all areas or across the two projects. Additional areas requiring diagnosis can also be identified, important anong them being institutional weaknesses in controlling corruption and support for economic evaluation of tax structures. The use of quantitative performance indicators is also somewhat limited. Finally, the review of lessons of experience fall well short of articulating either a clearly specified theory of effective and efficien **t** ax administration or a conceptual framework for project implementation. Nevertheless, these diagnostic exercises clearly pernitted the design of high value added reform projects.

Colombia Public Financial Management Project, Oct 21, 1993, Project ID CR-PE-6889. Jamaica tax Administration Reforn Project, May 24, 1994, Project ID JM-PE-7489.

9
# **Box 3. Balancing iComputers and Institution Building: The QAG Best Practice Project in Latvia**

Computerization often becomes and end instead of a tool in many tax administration reform projects. In contrast parallel institution building, necessary for sustainable reform, is often inadequately addressed. _7he_ **_State_** _Revenue Modernization_ **_Project in Latvia_** could have gone the same way as, originally, the Latvian government was primarily looking for assistance with information technology. However, witli help from the World Bank, the government integrated an "Institutional Development" component in the project design. The result was a 'Highly Satisfactory' and a 'Best Practice' ranking from Quality Assurance Group's (QAG) Quality At Entry Assessment' for 'sustainable institutional reform'. The balancing of institutional development and information technology is what is considered best practice and the major value addition by Bank.

# **_What_** _were thefactors_ **_fr the Best_** _Practice_ **_rating_**

_:_ Te Quality at Entry Assessment gave Latvia a highly satisfactory ranking in 7 out of 8 applicable dimensions. The most important factors mentioned by the QAG are:

# **_Project_** _Concept, Objectives and Approach_

- The Govemment of Latvia took the initiative. They only requested help from the Bank after considerable prior work and a Cabinet discussion of the issues. The Govemment showed commitment by providing almost 90% of project cost.

- . In preparatory work important institutional issues such as "vulnerability to corruption" and "institutional gaps" were addressed.

- The Govemrnent had clear objectives for both tax reform and capacity building to qualify for EU entry.

- The project utilized relevant lessons of experience in project design and implementation such as introducing perfornance indicators as benchmarks for revenue agency staff and managers, and complementing the ITdominated operational components with corrmponents aimed at the organizational and management aspects of revenue administration.

- I An alternative project design was considered and dropped because of clear criteria,

## _Institutional_ **_Capacity_** _Analysis_

- There was excellent balance between institutional and information technology needs.

- There was excellent explanation of the central importance of institutional development for sustainable development of capacity.

- A comprehensive analysis of the executing agency's capacity and incentives to carry out its mission was included. The diagnostic document is also best practice. It identifies gaps and solutions, and served as the base for project design.

Other dimensions considered for the overall ranking were: Environmental Aspect, Social and Stakeholder Aspects, Financial Management Aspects, Readiness for Irnplementation, Risk Assessment and Sustainability and Bank Inputs and Processes.

## _Stakeholder analysis_

A limitation of the project design process, according to the QAG report is its apparently limited assessment and incorporation of stakeholder views. It was considered too difficult to arrange the necessary dialogue to involve direct participation of taxpayers in the project, although the Project Appraisal Document (PAD) reports brief surveys of businesses, households, public officials and taxpayers, to incorporate their perceptions of the SRS in project development, The PAD reports that consultations were also carried out with major accounting firms, along with "selected" large taxpayers during pre-appraisal but notes that project design primarily reflecs work of SRS staff and management.

'Latvia, State Revenue Service Modernization P?roject, October 30, 1998, QAG, Quality at Entry Assessment 10
The previous section has documented the extent to which institutional concerns were incorporated in project diagnostic. Tables A6 to A9 in the appendix provide detailed information on the features of the projects under examination, which were, presumably, driven by the diagnostic.<sup>9</sup> The popularity and unpopularity of different components roughly parallels the pattern found in project diagnostic work:

- _Tax structure reform,_ which was present in most SALs/SECALs, tended to concentrate on basic broad-based taxes on income sales and imports.

- The most important tax reform in TLRS projects was VAT introduction or streamlining.

- Administratively convenient presumptive taxes received little attention. In three projects among those looking at turnover taxes, the objective was to abolish the tax and replace it with the less distortionary VAT.

- Roughly 50% of projects had components addressing one or more _tax policy institutions._

- _Environmental institutional components_ in TSRL projects tended to focus on laws and legal provisions relevant to tax administration and enforcement or on strengthening tax collection capacity of banks. SALs/SECALs, while addressing more institutional components on a wider front, seldom coordinated these components with tax or customs administration components.

- _Tax administration functional components_ tended to concentrate on the basic functions, identification, assessment, audit and collection, and for customs, cargo clearance.

- c Taxpayer education, improving taxpayer services were relatively neglected.

- c Contrary to expectations, components to set up or strengthen large taxpayer units, were present in less than half the projects. Thus, contrary to a widely held belief, Bank projects design followed the IMF's lead less than is believed to be the case OR large taxpayer units are far from _lde rigueur_ in IMF administration refomi packages. 10

- _Administrative institutions and capacity:_ On the other hand computerization components were found in all TSRL projects, though the emphasis given to it varied from being a minor part of the project with no expenditure on equipment, software or technical expertise, to being the primary focus of a project (irn 8 TSRL projects). Most projects with a strong computerization focus were accompanied by organizational, procedural and manpower reforms together with training activity. In fact, these institutional components were absent in some projects.

In compiling information on inclusion of project components, one problem to be tackled was the fact that a project could seldom be expected to have a comprehensive design, either because of borrower needs and priorities or because wide-ranging projects could end up straining the borrower's change absorption capacity (as happened in at least two projects in the sample). Yet, a picture of areas stressed and not stressed in Bank work could only be obtained if the reformn history encompassing, in particular, components of previous projects in a country was taken into account. To meet this need, components were deemed to be included in a project if they were included in parallel or past projects or were judged in diagnostic work to be adequate. Thus the frequency of inclusion of different components reflects how often components were ever included or were thought not to need reform. Low frequencies, therefore, imply a neglect of the concerned component in Bank project design. '0 Compare the third column of Table 5.6 below.

11
- Paralleling the pattern documented for project diagnosis, limited attention was paid to strengthening accountability and monitoring, to increasing administration autonomy, to improving budgetary processes or improving performance (wage) incentives for personnel.

- _Trade policy and customs project design:_ In order to provide a clearer picture of the design of trade policy and customs projects than is available from this analysis, Box 4 summarizes key features of around 15 projects with trade or customs reform components. Additionally, a key controversy in the area of tax (actually customs) adrninistration privatization, pre-shipment inspection is reviewed in Box 5.

# **Box 4: Design of Trade and Customs Reform Operations:** Case **Studiesl**

# **_Trade Liberalization_**

In **Argentina** in the mid-1980's the Bank supported the Goverment's policies of providing free trade status to Argentine exporters and to remove a large number of quantitative restrictions. In 1989 a new loan was granted to support Government's efforts to raise the exposure of domestic industrial production to import competition without QRs from 63% to _85%_ while reducing the average level of tariffs from 41% to 30% and the maximum tariff from _1_ 15% to 50%. The funds were disbursed in two tranches in line with the success achieved in obtaining these objectives.

The 1996 **Yemen and Jordan** programs also aimed at major tariff and quota reforms. In the case of Jordan there was also the objective of preparing the country for the commitments that were to be made in the context ofthe Association Agreement with the European Union and its intended accession to the WTO. in the case of Jordan, the project was prepared and followed up with the assistance of the USAID.

ISALs to **Cameroon** and to **Chad** assisted these Governments to adjust their tariff structure to the common UDEAC tariff structure as it was altered in light of the; 1994 devaluation of the common currency, the CFAF.

In **C6te d'Ivoire,** a succession of SALs airned to foster the supply response to the 1994 CEAF devaluation. These instruments contained measures of tariff reduction, quota relaxation and improved customs operation.

In the 1993 SAL for **Jamaica,** an explicit commitment was included to bring the tariff in line with the Caribbean Community Common External Tariff

# **_Revenue nobilization_**

A 1997 Technical Assistance project was provided to the Government **ofGuatemala,** with the major objective to raise its tax to GDP ratio above the 6-9% realized in recent years. This tax administration project complemented an IDB financed projiect to strength Customs Administration.

Several operation have aimed at modernizing the customs departments alongside with the Tax Department in transition economies. The 1995 AlbaniaTax Administration Modernization Project, had as its main objective such modernization. An agreement was reached between the Govemment and the IMF to use proceeds from the loan to compensate the IMF for supervising and managing key technical assistance aspects of the project. The 1993 Institution Building Loan forArmenia included a major componen **t** o (i) modernize customs laws and regulations, (ii) review and reform the organization and procedures of the customs operation, and (iii) provide training for customs classification and valuation. The Armenia project was prepared and implemented with the assistance of UNCTAD.

12
The 1994 **Lebanon** Revenue Enhancement Project, also aimed to strengthen customs procedures and management. The project financed reforms of customs procedures and the installation and commissioning of ASYCUDA++, as well as its translation into Arabic with the involvement of IJNCTAD in preparation and implementation. The work done in Lebanon, benefitedJordan which adopted the same system. The design and rationale of this project was independent of Government commitment or t decisions with respect to tariff reform as it was felt to be justified in its own right.

The 1996 Financial Sector Management Project in Turkey (see also Box 7), contained an important customs component aimed at shifting from ex ante, full coverage cargo examination to a more ex post and efficiency seeking stance. The project is relying heavily on training and the implementation of the SOFIX system for computerization of the customs procedures. French technical assistance has been crucial in adapting this system to the Turkish requirements and the IMF manages major aspects of the project.

# **_Trade Faciflaton_**

The 1993 Tax Computerization Project for the Philippinesaimed to provide faster and more reliable administration of revenues and in the process, facilitate private sector activities. For customs administration it focused on the computerization of the customs clearance process, involving brokers, importers and customs authorities. While its primary aim is to ensure reliable revenue generation, emphasis is being given to easing the administrative burden of the private sector from -often arbitrary - government interference. Crown Agents, selected though competitive bidding, worked with the Ministry of Finance and of the Bureau of Customs on the strategic plan and implementation of the modernization program. The IMF had extended substantial technical assistance in the areas of revenue procedures and institutional strengthening. With UNCTAD assistance , the ASYCUDA-H system for customs was refined to accommodate the specific Philippines requirements and was complemented with additional modules, while a private firm was hired to supply the required computer equipment and communication system. Observers suggest that these adjustments and development efforts have greatly contributed to making ASYCUDA++ a more versatile instrument for customs administration, and have benefited more recent adopters of the ASYCUDA++ system such as Lebanon and Jordan.

The **Ghana** project is an integral part of reforms required to reach Ghana's ambition to become a first class trade and investment center on the West African coast. The project --called the Gateway project-aims at improving the infrastructure of the maritime ports and the airport, establish Free Trade Zones under private sector operators, and streamline immigration procedures for investors and tourists. But foremost, the project aims at assisting Governmen **t** o bringing the Customs and Excises Department up to the standards of the International Chamber of Commerce and ISO 9000. Again, Crown Agents was selected, after a competitive bidding process, to help the Customs Departinen **t** o identify shortcomings of the Departnent and frame a reform program. From this followed the action program that aimed at (i) ! reducing to one the number of examination points and streamlining cargo documentation, (ii) reduce cargo examination from<sup>1</sup> 00<sup>0</sup> /o to I<sup>00</sup> /o for statutory free goods and<sup>2</sup> 00/a for all dutiable goods (iii) adopt the WTO valuation system, (iv) reduce the number of overland customs check points, and (v) start a specialized customs unit to facilitate the operation of export processing zones. The project will finance a business plan that streamlines clearance processes so as to reduce the clearance time for import and export cargo, re-equips and installs an adequate information technology infrastructure, and trains customs officials in these new procedures, including those for free trade zones. The Project will be accompanied by substantia **l** ong-term assistance as well as short-term technical assistance in a number of specialized areas.

IThis box is summarized from Luc De Wulf (1999) Trade Policies and Customs Operations: World Bank Interest, processed, Washington D.C: The World Bank.

13
# Box 5: Privatization in Customs: The Pre-Shipment Inspection Controversy

An issue that has raised some controversy was the decision of the Africa Department in the late 1980s and early 1i990s to require most countries that benefited from structural adjustment lending to sign up for a pre-shipment inspection (PSI) service. hle concern originally rested with the donor community that wanted additional assurance that the countries that obtained quick disbursing SAUSAC financial assistance at easy ter,ms would obtain "value for their money", that is, that they would import the quantity and quality of goods that were paid for by the grant and soft-loans, without any leakages. Gradually, however, revenue generation and anti-corruption concerns also led the donor community to look into the i use of PSI services. A Bank research paperl studied the contribution that could be made by PSI icompanies and presented some guidelines that, if followed, would help countries to benefits from PSI service. These recomimendations include competitive bidding for PSI services, mandating the recording of the PSI findings on the customs declaration, requiring a reconciliation of the findings of the PSI findings with those used by the customs department in its determination of the duties owed, and ensuring that a transparent and swift appeals procedture exists to settle disputes. Countries that have followed these recommendations have benefited from PSI services. Others have paid dearly for services that they did not use efficiently, either though less than transparent procurement or by simply ignoring potentially important findings. Currently, the Bank's approach to commercial PSI services depends very much on particular country circumstances, the Bank having abandoned the earlier blanket recommendation. Furthernore, recent allegations of corruption in the case of a leading PSI company have cast further doubt on the effectiveness of this strategy. An issue that remains on the front bumer though, is how to reduce over time the reliance on PSI services and build up national iiistitutional capacity in the customs departments.

Patrick Low (1995) "Pre-Shipment lnspection Services", World Bank Discussion Paper #278, Washington DC: The World Bank.

_Innovation:_ An additional interesting feature about the inclusion of project components, project docunents do not appear to incorporate very many new and innovative approaches to reform tried successfully in particular instances, Some examples: the use of receipts lotteries to enhance VAT compliance; privatized audit review or customs (second) inspection; performance linked bonuses; self-targeting in performance evaluation; joint returns for income and social security contributions; performance linked administration budgets. Perhaps the only major innovation seen in certain Bank projects is the creation of independent revenue authorities with discretion over resource induction and deployment. Adoption of more successful innovations holds out the potential to add significant value to Bank assistance but requires greater design effort.

_Sequencing._ The availability of information of sequencing of project activities in projects varies greatly. Several projects provide no info:urmation on project sequencing. Indeed, in one response to questionnaire's administered to task mnanagers as part of this review, for a project which did not have a project implementation plan, the respondent states that "There was an implicit sequence..." but "No plan was made to go about this implicit sequencing, and, therefore, after the first purchase of computers to _"solve_ all problems" the project came to a stop for about two years" (italics in original). Other projects provide detailed timelines for different project activities with or without a discussion of the rationale for the sequencing adopted (Box 5). However, as there is no unifornity of sequencing across projects and, as yet, limited information on 14
completed projects, no useful inferences can be drawn about the appropriate sequencing of activities in administration reform projects.

Box 6. Implementation Plan for VAT **Introduction** **_Mongolia Fiscal TechnicalAssistance_** _Project,_ **_19981_** The introduction of a VAT in Mongolia is part of a major tax _reform_ program of base broadening and rate lowering. Key requirements for successful VAT implementation have been identified as: (a) a sound tax administration and (b) well-informed taxpayers that know how to apply the new law.

To implemen **t** he reform. (a) a VAT management committee has been set up headed by the Director General of the General Tax Department (GDNT). (b) An interagency working group has been created for day-to-day project implementation with staff from the Ministry of Finance (MOF), the GDNT and the Customs General Administrationi (CGA). (e) Implementation is to be supported by an IMF-supervised lonig tenn VAT advisor.

I The project became effective in September 1998. Impleme-ntation started in March 1998. and will continue until March 2001. The detailed implementation plan has eight phases with a detailed plan of action having been prepared for each phase. The project has a detailed timeline and specifies implementation responsibilities for each component. The phases are:

- I . _Legislation and PublicAcceptance."_ To draft and pass a VAT Law through Parliament and start a public education program: MAOF and GDNT.

2. _Policy Interpretat_ **_ons:_** To develop interpretations of each provision, initiating contact with industry sectors on those issues and establishing a permanent policy interpretations unit in headquarters: GDNT.

3. _Registration:_ To develop registration forms and procedures, a strategy for identifying potential taxpayers and training for inspectors: GDNT.

4. **_Tax_** _Accounting:-_ The design, reporting procedures, control, and information required on VAT returns: GDiNT.

- _!5 Taxpayer information: GDNT._

- l6. _"Importationst._ To plan on how to deal with VAT on imports, the development of procedures for temporary imports, instructions to importers and information and procedures for Customs staff: CGA and GDNT.

7. _Refimds:_ To plan a general approach for refunds and information required in the VAT return as well as establish procedures witlh the Treasury Department regarding payment of refunds: GODNT and MOF.

8. _Training:_ A detailed plan speci6'ing how and when trainirng for the different components should be delivered: GDNT.

Provided implementation of this plan is not inflexible, the careful plan. detailed timetable and clear specification of responsibilities increase the probability of success of the project. Comparable implementation plans, which typically **_fiII_** within the ambit of Project Implementation Plan (PIP) documents along with detailed cost estimates of project components are available in only S or 6 other technical assistance projects in the sample.

lMongolia, Fiscal Technical Assistance Project, March 16, 1998. Project ID: MN-PE-5 1855 An interesting case study of sequencing of administrative reforms _across loan operations_ in a sequence of three Structural Adjustment Credits (SACs) to Uganda (Box 7). Starting with high 15
level institutional reform within tax administration, subsequent operations addressed selected tax functions followed by taxpayer service quality. Unfortunately, the available assessment of the success of this sequencing, though positive, is apparently flawed.

# **Box** 7: **Sequencing Institutional Reforms In Tax Administration: SACs in Ugandal**

Institutional reform and capacity building are, at the very least, medium term endeavours requiring carefui sequencingz There is, however, limited theoretical guidance about the appropriate sequencing of institutional and capacity building reforms in the area of tax administration. An evaluation of the sequencing and selection of tax administration reforms over 3 structural adjustment credits (SACs) and the Second Economic Recovery Credit (ERC II) in Uganda, spanning a 10 year period (1990-1999), could, therefo provide valuable insights.

Tax and administrative reforms undertaken in the three SACs were:

- _Second Economic Recovery Credit (EJC II), 1990, and SAC :_ Major reforms: Establishing the Uganda Revenue Authority (URA), reducing exemptions and the threshold of the income tax. Other reforms- Capacity and procedural reform to strengthen auditing, investigation, data and records management, monitoring systems, tax collection ; increased penalties for tax collection; improved training programss; excise and sales tax rate reformn.

- _SAC IT_ Major reforms: Taxpayer identification numbering; further curtailment of tax exemptions; move to a VAT in the medium term.

- _SAC III:_ Major reforms: Establishing an independent dispute resolution mechanism for tax grievances; improving tax policy analysis capacity in the Ministry of Finance; Further measures to eliminate income tax and VAT exemptions.

- Other reforms: Reformulating the role and increased independence for the URA; Strengthening performance monitoring in the URA; measures to enhance public perception of the integrity of URA Board members; reforming anti-corruption procedures.

Parallel reforms of public expenditure management, the civil service and the regulatory framework for the private sector were undertaken in all three SACs.<sup>2</sup> Among its actions, the URA effected significant personnel changes and raised staff salaries beyond civil service norms. There was also substantial induction of capital equipment financed by other donors.

Both SAC I and II were rated as having satisfactory outcomes that were likely to be sustained (ERC I1 was rated as marginally unsatisfactory primarily doe to limited success in civil service and parastatal reform), though institutional development was rated as moderate. The Proposal to the Managing Director of IDA for SAC III reports that "The reforn of tax administration in Uganda is widely regarded as highly successful" but goes on to report that the tax-GDP ratio is virtually the only performance measure on which this judgement is based, and points out that this is a "misleading indicator" on which to judge tax administration performanee. It points to several shortcomings, which fonn the basis of areas to be reformed under SAC III. A detailed, well designed, review at the time of closure of SAC III could, therefore, prove valuable.

Uganda Second Economic Recovery Credit, January 8, 1990, Project ID UG-PE-2949. Uganda Structural Adjustment Credit I, November 1, 1991, Project ID UG-PE-2947. Uganda Structural AdjustmentCredit ILApril i8, 1994, Project ID UG-PE-2967. Uganda Structural Adjustment Credit III, May 1, 1997, Project ID UG-PE-2987.

2 SAC III also had a reform component for sub-national taxes, while all three SACs had trade tax and customs reform components. These are not discussed here.

16
_Coordination and implementation planning:_ Table 8 presents information on discussion in project documents on the relation of projects with other Bank projects and with other agencies. Again the criterion used for the Table is existence of such discussion with no judgement being made as to its adequacy. If SAR/MOP discussion of coordination is taken at face value, then such coordination was adequate in the majority of projects where it is present. In fact, for one project, the ICR highlighted the inadequate coordination between the project and other donor financed and Bank projects as an important cause of its unsatisfactory outcome. An interesting feature documented in Table 8 is the relatively low incidence of coordination with the IMF in TLSR projects. Further examination of the causes and impact of this lack of coordination could prove useful. A final feature worth noting is the prevalence of dedicated client project teams and the relatively intensive use of consultants in TLSR project design.

**Table 8. Other Design Features**

||43 p|rojects with a major ad<br>(% of proj|ministration com<br>ects)|ponent||
|---|---|---|---|---|---|
|Type of Project|SAR/MOP<br>Discussion of<br>Relation with<br>other Bank<br>Projects|SAR/MOP<br>Discussion of<br>Coordination with<br>Other Agencies/<br>Projects (including<br>IhF)|Coordination<br>with IMF|Projects with<br>Dedicated Client<br>Country Project<br>Management/<br>Monitoring<br>Units|Projects with<br>One or More<br>Components<br>Implemented by<br>Consultants|
|TALs, LILs,<br>SILs, RILs|66.7|81.8|45.5|90.9|68.2|
|SALs and<br>SECALs|85.7|100|_95.2_|19.0|26.3*|
|All Projects|76.2|90.7|69.7|_55.8_|_48.8*_|

Notes: Details of 2 projects are not available.

# **6. How To Gauge Success: Performance Indicators**

_After discussion the rationale and design ofperformance indicators, their limited use and the lack of uniformity across projects is documented here._ _The nature and content ofperformance indicators:_ A distinction can be made between indicators to judge the performance of a tax system as a whole, a tax or customs administration, and a reformn project. A country should, in principle, be interested in the performance of its tax system as a whole and, within this, the tax and customs administrations. Thus, pre-project diagnostic work should use the same set of indicators as those needed to measure tax system performance. On the other hand, for project implementation such indicators are insufficient and, when there is a time lag between reforms and their impact, of little evaluative use during project execution. Besides outcome indicators, a system of PI for a project should therefore include input, process and output indicators."' In all cases, benchmarks or guidance at interpreting performance indicators are needed to make interpretation feasible:

- For outcome indicators - cross-country, previous year, or targeted performance benchmarks.

- For input indicators - quantity, timeliness (e.g. agreed dates), cost, and, if possible, quality benchmarks.

- A detailed discussion is in World Bank, Operations Policy Department (1996).

17
- For process indicators - coverage, duration and cost benckmarks.

- For output indicators - time, quantity and, where possible, quality benchmarks.

PIs are, by definition, _"quantitative_ indicators for use in the management, or assessing the performance, of an agency, program or activity".<sup>2</sup> While outcome PIs and PIs to judge quality are difficult to devise and while congruence between PIs an-d objectives may not be easily ascertained (Box 7), other PIs for inputs, processes and outputs are not difficult to devise or are already available. The World Bank, Operations Poiicy Department (1996) provides data pertaining to the use of Pls in TALs during 1993-1995 and points out that 39 out of 70 TALs were "Projects with comprehensive matrix, including PIs and quantitative measures" while 11 projects had no PIs. As pointed out earlier, Datta-Mitra (1997) identifies the absence of perfonnance indicators as a major weakness in the design of tax administration components in SALs/SECALs.

# Box 7: Non-congruence Between Pls and Objectives. Enforcement **Collections**

One indicator of administration effectiveness, present in some projects in the sample is the amount of enforcement collections or its ratio to total colections. Enforcement collections are collections of tax and, possibly, penalties and fines due to the efforts of the administration and exclude taxes paid voluntarily or t'compliance collections". Yet some refiection shows that an increase in the value of this indicator, in level or in ratio form, does not always indicalte improved effectiveness. If the effectiveness of an ineffective administration increases, it is of course likely that enforcement collections will increase. But excluding genuine mistakes? taxpayers will truithfully report their taxes due if the administration is very effective to ensure that they are not subjected to penalties or fines. So any increase in the effectiveness of an already effective admninistration should result in _adecrease_ in enforcement collections. Low enforcement collections could, therefore, indicate either a very ineffective or a very effective administration.

_PIs in the sam;ple._ Annex Table A2 lists performnance indicators used in the 12 TLSRs and 5 SALs/SECALs where perfornance indicators (other than compliance with covenants and the tax/GDP ratio) were specified. In addition, 1 SAC had a condition that the governnent would develop a set of performance indicators cdunng the project. The table shows that performance indicators specified in TLSRs were considerably more detailed than those in SALs/SECALs. Nevertheless, even among TLSRs, corrlprehensiveness varies and indicators specified lack uniformity across projects. There is also significant use of qualitative information in lieu of performance indicators. Judging from the table, the design weakness identified by Datta-Mitra (1997) in SALs/SECALs is only somewhat lower in TLSR loans. A case in which there is dramatic variation in the quality of "PIs", f.or customs and tax reform components, even in within the same project, is in Box 8. Box 8 also gives details of a tax administration reform project, where the set of PIs is relatively comprehensive even if some of them are "qualitative PIs".

> 2 World Bank, Operations Policy Department (1996).

18
Box **8. Performance** Indicators: Public Financial Management Projects in Turkeyl and Colombia<sup>2</sup> The sharp contrast between the detailed set of performance indicators for the customs administration and the tax administration components of the 1995 Public Financial Management Project (PPMP) for Turkey is instructive.

For customs administration, indicators are all base-lined with respect to pre-reform benchmarks, and accompanied by a detailed description of the intended results. The project also lists methods to be used in order to determine if the intended results are achieved. In most cases this involves a survey prior to and after reform. For example, the objective of improving service by reducing the physical inspection rate for shipments of imports and exports, was L00 percent before reforms and targeted to be reduced to no more than 20 percent of imports and 1-2 percent of exports by the end of the project. Indicators for the tax administration component on the other hand, are simply a list of the various project components and at what point in time they will be launched.

|WU<br>_NOW_:_iiEW0 _|XTIC"'-<br>_Tn_<br>v<br>-E||
|---|---|---|
||<br>_"A_<br>W MOaelrtttW'||
|M<br>m~~~idtndo a4||wb|
|g<br>*|~~<br>**.....**|~~~~~~|
|**OK**<br><br>|<br>||
|<br>;|||
|S|||

19
**_Colombia:_** By way of contrast, the performance indicator matrix specified in the Colombia **PFMP** is specified to include activity-wise indicatoTs, an assessment of the pre-project situation and a clearly specified end of proect outcome. The matrix has the following column heads.

For tax administration, indicators (summarized in Annex Table Al) are specified for:

# Operational subsystem s

1. Taxpayer services (reduced compliance cost based on surveys before and after the project).

2. "Fiscalization' (2 quantitative targ-ets for revenue and evasiow; one qualitative indicator).

3. Compliance collection (I qualitative, I quantitative).

4. Enforcement collection (I qualitative, I quantitative - revenue target).

## Administrative subsystems

- _S._ Inventories of goods (3 qualitative).

6. Financial management (I qualitative).

7. Personnel management (I qualitative).

8. Office management (1 qualitative).

9. Informatics management 0(qualitative).

## Control subsystems

10. Performance and interal audit (1 qualitatiive).

II1. Fiscal analysis and studies (I qualitative).

This set of performnance indicators can be tightened, particularly where only qualitative indicators are specified and fade more comprehensive so as to apply to administration _as_ a whole and not merely to components covered in the proect. However, the careful benchmnarking, identification of dates and implernentation responsibility and parsimony are key strengths.

'Turkey, Public Financial Management Project, August 29, _1995._ Project ID: TR-PA-35759. 1 Colombia, Public Financial Management Project October 21, 1993. Project ID: COePE-6889

# 7. Evaluation of **Project Appraisal Metrhodology in the Sample**

_A general discussion of diferent aspects qf quantitative examination of net project benefits and risks is first undertaken. Appraisal of benefit assessment and risk appraisal in different projects is next described and assessed_ 20
_Formal evaluation of expected project returns:_ The overall objective of a tax administration or customs reform project must ultimately be to increase the effectiveness and efficiency of the administration. No operational method to evaluate effectiveness benefits has, to our knowledge, been devised."<sup>3</sup> Under the assumption that tax administration improvements have limited effects on the rest of the economy (i.e., that the project is "small") and that revenues will remain below potential despite reforms, revenue increases due to the project can be evaluated at the incremental shadow value (or premium) of additional transfers from the private to the public sector. For customs control, other, non revenue, effectiveness indicators may be needed depending on the circumstances. Efficiency refers to real resources used in collecting revenues, per unit of revenue collected. Put another way it is the difference between the decrease in private sector resources per unit increase in the net revenue of the government. The difference between the two includes collection costs of the government and compliance costs of taxpayers. To the extent that horizontal and vertical equity of tax collection is important, shadow valuation of the cost to different taxpayer groups is needed. Leaving aside distributional considerations, efficiency benefits, unlike revenue increases, can be evaluated at their dollar value. While no method to evaluate administrative efficiency has been discussed in the literature either, deriving approximate indicators of efficiency for use in project appraisal is feasible. _No project in the sample has tried to formally assess benefits from administrative improvement in terms of benefits from improved effectiveness and efficiency ofadministration._<sup>_14_</sup> A number of projects carried out formal assessments of expected "economic returns" or "financial returns" from projects. Financial returns to projects were usually inappropriately judged in terms of additional (real or nominal) tax or customs revenues or increases in the tax to GDP ratio that would accrue to the government from reforms under the project. More surprisingly, most projects fail to even examine direct administrative costs. The conclusion of this review is that formal project appraisal of net projects benefits has been inadequate in every case where it was undertaken. That such analysis, even detailed and careful analysis, can be potentially misleading is the subject of the case study in Box 9.

On the other hand, a desirable feature found in 4 TSLR projects (for Bulgaria, Colombia, Hungary and Latvia) is the evaluation of alternative project designs. Aside from these projects, most projects failed to formally specify alternative designs for appraisal.

_Risk assessment:_ Here the identification of sources of risk and uncertainty and translation of this into changes in the benefit/cost stream of the project can be expected. Risks that arise are of three types: Inaccurate or uncertain project costs or benefits even if project activities are carried out adequately and assumptions about exogenous factors are borne out; failure to complete project activities; and exogenous risks. For quantifiable risks this implies the need for sensitivity analysis of the original project cash flows.

In Table 10 the incidence of formal project appraisal in the sample is reviewed though it bears repetition that appraisal methods are inadequate.

> **13** The Marginal Effective Cost of Funds methodology of Slemrod and Yitzhaki (1997) provides a framework which can be adapted to do this, but as yet only in principle.

> **4** Discussion addressing the same issue is to be found in McMahon (1997).

21
**Table 10. Methods Employed in Appraisal of Net Project Benefits** (43 projects withi a major administration component)

||Heuristic<br>appraisal|(% of projects)<br>Social/Economi<br>Returns|c Financial<br>returns|Heuristic<br>Risk<br>assessment|Quantitative<br>Risk<br>Assessment|
|---|---|---|---|---|---|
|TALs, LILs, SILs, RILs|lO0<br>_|8<br>|28.6|100|14.3|
|SALs and SECALs|100|)__<br>_|0|100|0|
|All Projects|100|2.4|14.3|1100|7.1|
|Notes: 1. Details are not av<br>2. For 2 projects rev|ailable for one p<br>enue impact was|roject.<br>estimated.||||

**Box 9. Cost Benefit Analysis** _The Philippines nax Computerization projecL, 19933_

The Philippines' Tax Computerization project is one of 7 projects in the sample with a formal analysis of net project benefits (others include technical assistance loans to Argentina, Bulgaria, Guatemala, Latvia, Mongolia and Thailand, though the methods of analysis are not uniform). Compared to other proects the analysis is detailed in the Staff Appraisal Report (SAR). Furthermore, estimates of additional tax collection are based on tax by tax evasion estimates made in three different studies.<sup>2</sup> The project's net financial benefits are calculated based on year by year (estimated) project expenses and year by year (estimated) incremental revenue collection due to the project. The source of incremental revenues is reduced evasion alone, the reduction in evasion being assumed to be to 40%/o of the initial level, translating to an increase in the tax-GDP ratio from the existing 15.5% to 19%K by 2001. The project duration is 1993-97 but cash flows for the period 1993-2001 are considered for net benefit calculations. Project costs after 1997 are assumed to be a constant (real) amount per year. Total tax collection in 1992 is estimated at 209 billion pesos and total tax; evasion at that point was estimated at 80 billion pesos. The project also assumed that tax collection would increase gradually over a period of up to 7 years. Given an estimated growth rate of GDP, incremental collection and net benefits were calculated year by year using three possible scenarios:

1. Benefits are achieved gradually during years 293

2. Benefits are zero in the first 4 years and then achieved gradually during years _5-9._ 3. Benefits are achieved gradually during yea rs2-54 Annual discount rates between 10% and 20% were used, without any comment, to discount future net benefits. The range of discounted present values for these calculations was 59 to 225 million pesos and the (discounted present value) benefit to (discounted present value) benefit-cost ratio ranged from 42 to I 110 million pesos. The internal financial rate of return (IFFR) was between 203% and 300%.

The assessment pointed out that intangible beinefits include lower future manpower requirements even if there was rapid growth in the number of taxpayers, and greater taxpayer and staff satisfaction. The assessment also claimed that an economic rate of return was "impossible to estimate" since the main effect of the project was the transfer of additional resources from the private to the public sector. Jt also noted-that additional collections were likely to be forthcoming from identified non-filers, even after the projectfs useful life was over.

_Evaluation:_ Taken on its own terms, the assumed real growth rate of GDP during 1994-2001 (7% per year) was overly optimistic given the 4.5% rate actually achieved during 1992-97, according to World bank data). The assumption that tax evasion would fall to 40% of its initial level appears similarly optimistic, especially since the project design does not explicitly address corruption of tax officials, 22
which available evidence suggests is substantial (the Transparency International 1998 Perceptions Index ranked the Philippines at 55 out of 85 countries, where rank I is the least corrupt country)5 In fact, till 1997 the highest tax-GDP ratio achieved (which was in 1997 itself) was 16.8%, making the achievement of a 19% tax effort by 2001 open to question. Though the SAR recognized that increased revenues are transfers from the private sector no attempt to take this into account in calculations was made. One way around the problem of estimating the (shadow) premium parameter of private to public transfers is to find the break even social premium as illustrated below. On the other hand, certain other assumptions are pessimistic. The increase in administration costs (in this case taken as equal to project outlays), which translates directly (absent distributional effects) into an added social burden, are overestimated. As the previous paragraph reports, reduced taxpayer compliance costs as well as reduced current and future administrative costs would have been likely, offsetting some of the direct administrativ **e** xpenditure increase.

As it stands, the projecs net benefit are probably positive (though lower than the SAR suggests) even if some of the over-optimism is corrected. For example, if (a) actual tax GDP ratios are used up to 1997 and projected to increase thereafter at the average increase during 1995- 1997 (2.4%), (b) if additional revenues due to the project were only 50% of the growth achieved (the rest being attributable to other reforms such as the recent VAT refonn), (c) no downward adjustment is made to costs, and (d) the discount rate is taken to be as high as 20%, the project breaks even provided the premium of each additional dollar transferred to the public sector is 0.05 pesos (a 5% premium). At a 10% premium, the net present value of the project ranges _from_ 1.5 billion to 3 billion pesos (and benefit-cost ratios between 3 and 4.2, IRR of 50.3%/o) for the 10% to 20% range of discount rates in the SAR.

i However, project net benefits could easily turn negative despite revenue increases, if additional transfers to the government have a lower premium, which could arise if, say, further public sector expansion had limited benefits so that, instead of increasing revenue, downsizing government was needed. On the other hand, if substantial cost saving were to be made possible by a revenue neutral project, net benefits would be expected to be positive.

'Philippines Tax Computerization, May, 1993, Project ID. PH-P&4599. - UNDP (1992), Venecia (1992) and Manasan (1988) full eitations are not given in the SAR I The cash flow was taken over 9 years rather than the 10 years claimed in the SAR.

4 The numerical counterpart of "gradually" is not reported in the SAR 5 Infact, the tax GDP ratio averaged 16.4% during 1993- 1997, reaching 17% in 1997. This implies that the incremental effort in 1997 was about 20% lower than assumed in the appraisal.

# **9. Evaluation Of Project Outcomes**

_The evaluation of project outcomes and impact in post-project documents is documented here. This is followed by an assessment of the basis of assessment of project outcomes and the extent to which lessons for future operations are derived The section closes with a discussion and an example of a possible strategy to increase the value of post-project evaluation for future operations._ As discussed earlier, only a handful of the projects examined in this review have closed and have undergone outcome evaluation. Boxes 10 and 11 respectively present case studies of a highly satisfactory and a highly unsatisfactory project.

23
# Box 10: **Computers and More: Thie Second Public Sector Management Project, Chile**<sup>**1**</sup>

Chile's Second Public Sector Management Project, commencing in 1991, was highly successful according to the end of project evaluation by the OED The project outcome (achievement of objectives), borrower and Bank performance were rated as highly satisfactory, institutional development as substantial and sustainability of achievenmenta **s** likely. In addition, the client-oriented public sector management vision of the Chilean govemment described in the loan document, the relatively low incidence of corruption<sup>3</sup> in and high institutional quality (Evans and Battaile, 1998) should also be noted. Thus, all 4 key project success factors identified by Evans and Battaile (1998) as also high institutional quality were present in PSM IL. A questionnaire response from a member of the initial appraisal mission, adds that ownership of the project by managerial and technical staff was a key success factor. Regarding the outcome itself, the respondent adds: "C:hile has the best tax administration in LAC and also perhaps in the developing world. It is probably on the level of high compliance EU countries. It is perhaps the most cost-effective administration in the world. T'he project helped Chile to further develop this path with a clear improvement of taxpayers services by the strategic and constant use of IT tools for an interactive tax administration." The project itself, was unabashedly a computerization project especially the tax administration component. Other sectors (customs, treasury, social security, treasury, policy analysis and legislature) also had dominant computerization components but all sectors had accompanying components dealing with suc things as organizational reform, training, skill upgrading, strengthening evaluation. The integrated design across different fiscal management activities, as opposed to a stand alone tax administration computerization project, can also be seen as a factor contributing to success. Furthermore, there was a clear recognition in the project design that the primary objective was _improvedinformation management_ and not merely induction of new technology. In fact the greatest hurdle faced in project implementation occurred with the equipment supply - the replicability across regions of software acquired through the project proved infeasible (according to the questionnaire respondent) but this was overcome via flexible management problem solving.

'Chile, Second Public Sector Management Project (PSM 11), September 11, 1991. Project ID: CL-PE-6669. 2 OED ICR Review -Evaluation Summary, 1998..

A questionnaire response from a member of the initial appraisal mission, however, reports that project implementation was not free of corruption.

# Box **11: Computers and Poor Supervision: Tax Computerization, Thailandl**

In 1992 the Government of Thailand introduced a major tax reform and requested Bank assistance to I finance an overhauling of its existing computer system. Its main objective was to establish a modern and decentralized tax administration system through the provision of appropriate hardware, software and technical assistance. At the time of Board approval (FYI991) this was the largest stand alone computer project ever supported by the Bank and also the largest integrated computer system in Thailand. However, at the closing date in December 1996 the loan had failed completely and was rated as highly unsatisfactory. No comprehensive data base rnanagement and processing system had been established and most of the applications software had not beern delivered.

The Implementation Completion Report lists the following as the main reasons for its failure: _f_ Lack of strategic analysis: 24
- | Poor project preparation: Technical requirements listed in the bidding document were not<sup>based on a</sup> description of the ehanges in the Revenue Department's business processes<sup>that were to be supported</sup> by computerization.

- The project objectives were narrowly focused on the computer system and not specific<sup>enough. For</sup> example it only listed a few services that were to be provided by the new system.

- The project lacked performance benchmarks.

# Weak project implementation and supervision:

- There was no detailed project implementation plan. The action plan listed the main project milestones, but gave no indication of how they were to be achieved. The action plan did<sup>not show the</sup> relationships and priorities between the various actions and tasks and who was responsible<sup>for them.</sup> Dates for the delivery of various outputs were repeatedly changed during supervision of the project.

- I The lack of detailed implementation plan made supervision difficult. Until January 1996,<sup>the project</sup> was rated satisfactory even though it had run into problems at an early stage of implementation.

Inadequate technical specifications

- The system specifications included in the Invitation For Bids (IFB) were for the hardware, and<sup>not</sup> specific enough for the applications software component of the project.

- In 1993 the software contractor abandoned work without having delivered the software<sup>applications</sup> in accordance with the contat. Furthermore, the computer hardware that had already been delivered was of insufficient capacity and required replacing.

lpoor prcureme rocess: i * TMere was no real bidding process for the turn-key contract. The Revenue Department wanted the software quickly, so instead of a two stage bidding process<sup>2</sup> software was provided through a twoenvelope process.

# Weaknesses in project management

* Despite an in-depth review by the Bank in January 1996 not much progress had been made by December 1996, the closing date of the project, and the Bank decided not to grant an extension<sup>of the</sup> loan.

I Thailand, Tax Computerization Projec January, 199 1, TH-PE-4774.

2Under a two stage biding process, unpriced technical proposals on the basis of a conceptual design or performance specifications are first invited, subject to technical as well as commercial clarifications and amendments.<sup>Theses are</sup> followed by amended bidding documents and the submission of final technical proposals<sup>and priced bids in the</sup> second stage. As opposed to two-stage bidding, in the two-envelope system, bidders are required to submit separate qualification, technical and price proposals at the same time, but which are opened sequentially in separate<sup>sessions</sup> and evaluated. The Bank no longer accepts the two envelope system.

# **Computerization: Adequacy and Sequencing**

_The computerization question:_ Two important questions with regard to IT are<sup>its role in good tax</sup> administration and the process of IT induction. Its role and the appropriate process of<sup>IT</sup> induction in the context of countries at different levels of development and with different characteristics is under-researched (for example in an Argentina versus a Vanuatu versus a 25
Mozambique).i5 Indeed, the role of IT and the induction process are poorly understood even in the United States, which, in the 1 960s, was a pioneer in the induction of IT in tax administration (Box 12). The following is a partial list of hypotheses, suggested by the experiences in the projects in Boxes 10-12 as well as others, on which reform experiences can possible shed some light:

- A prerequisite for cost-effective IT introduction is the availability of reliable power

- infrastructure and to a lesser extent, adequate modem communications.

- Use of IT is cost-effective for administration of the following taxes in economies above a (rather low) threshold size: income and corporation taxes, the VAT and sales taxes, customs duties and asset/property taxes.

- Cost-effectiveness of IT is greatest for the following functions (in descending order): taxpayer or asset identification and registration, tax collection, declaration processing, auditing or post-clearance controls, personnel management. Other functions are less costeffective.

- Institutional reforms and capacity creation, particularly re-organization, legal and procedural reforms, manpower upgrading, and performance monitoring, should precede or be undertaken simultaneously with IT induction or upgrading.

- v IT is most cost-effective if the administration is functionally organized.

- The effectiveness of a tax administration in combating non-compliance is increased when major functions are automated.

- IT aids in curbing corruption in functionally organized tax administrations when major functions are automated.

- It is cost-effective for tax administrat:ion to promote IT induction in related sectors (e.g.

- banks, exporters/importers, company accounts).

- It is cost-effective, given adequate use of IT in the economy, to privatize tax collection and, at a later stage, assessment (i.e. self-assessment via retum preparers as against official assessment).

- Modular introduction of IT, tax by tax and/or function by function does not impair its cost

- effectiveness.

- Pilot projects and careful involvement and education of tax department employees and management enable the gestation period for IT introduction to be shortened.

- A dedicated change management unit with appropriate powers is necessary for successful IT introduction.

Several of these hypotheses are, in principle, amenable to statistical testing.

# **Box 12. How** Not **to Automate Tax Administration? The (US) IRS saga**

It is not only developing countries that encounter probiems when trying to implement large technology investments. The Intemal Revenue Senrice (IRS) in the US has had continuing problems with the implementing itsTax Systems Modernization Program (TSM). A report of the National Commission on

> ' For a recent review of IT in developing and developed country tax administrations see, for example, Das-Gupta and Mookherjee (1998), Chapter 8. For a discussion of the process of IT induction see, for example, Clegg et. al. (1997).

26
Restructuring the Internal Revenue Service1 says that:"the IRS has experienced great difficulty in [ managing technology".

"bThe technology deficiencies are an outgrowth of management and governance problems and the agency's inability to pursue a long-tenn strategic vision in its business operations. Absent a strategic vision, no quantity or quality of technological modernization can be truly effective." TSM is the centerpiece of IRS's vision of virtually paperless tax processing to optimize operations and better serve tax payers. Between 1986 and 1997 the Internal Revenue Service (IRS) spent over $3.1 billion in Tax Systems Modernization (TSM),<sup>2</sup> and in December 1998 a technology contract was given to the Computer Science Corporation worth an estimated $5 billion over its 15-year life.<sup>3</sup>

# **_What went wrong?_**

A recent report from General Accounting Office (GAO) highlights reasons for the failure of the TSM project including- 'the IRS did not have a comprehensive business strategy to reduce paper tax return filings in a cost effective manner and had not fully put in place the requisite management, software development, and technical infrastructure necessary to successfully implement its ambitious systems modernization. They also lacked an overall systems architecture to guide the modemization's development and evolution.'<sup>4</sup> The report adds that "Senior technical leaders were noticeably absent and the organizational structure with the accountability and authority needed to manage modernization efforts was lacking below the Commissioner's office"'

# **_How to_** _deal_ **_with it_**

The GAO has recommended that the IRS (a) implements a process for investment management, (b) implements disciplined procedures for software development; and (c) completes and enforces an integrated systems architecture, including data and security sub-architectures. The IRS has agreed in principle with their recommendations and some progress has also been made in following the recommendations. For example, the IRS has created an investment review board to select, control, and evaluate its infornation technology investments. It also provided the first two levels of a four-level modernization blueprin **t** o the Congress in May, 1997.

On the other hand recent developments within the IRS are raising questions about whether it is actually moving in the right direction. A planned reorganization of the IRS will affect the very business processes and requirements that the blueprint is based on, raising questions about the blueprints validity and applicability. A second issue is the initiative to use contractors to modernize the systems since the IRS has a history of not being able to effectively manage its contractors. Finally the IRS' Annual Performance Plan explains the modernization efforts in very general terms without objective, quantifiable, and measurable performnance goals and does not specify measures for assessing progress toward the goals. I GAO therefore concludes that IRS still lacks the ability to etfectively modernize its tax systems and the TSM continues to be classified as a high risk project.

' A Vision For a New IRS, Report of the Commission on Restructuring the Internal Revenue Service, 1997 2 Tax Systems Modermization: Management and Technical Weaknesses Must be Corrected if Modernization is to succeed (Chapter Report, 07/26/95 GAO/AIMD-95-1 56).

_|_ Washington Post, Thursday, December 10, 1998; page BO1 4 GAO/OCG-99-14 Treasury Challenges.

27
# 10. Main **Conclusions**

The main conclusions of this review mainly relate to design and process issues of tax and customs administration projects rather than to their impact. This can be remedied in future work only if better information becomes available which, however, depends on project design and implementation planning incorporating measures to allow for improved impact assessment.

The maj or limnitation of Bank operations in the area of tax and customs administration pertains to the inadequate institutional framework For knowledge accumulation from loan operations. Unlike several other areas of operation, theoretical underpinnings for efficient and effective tax and customs administration are still rudimentary. The quality of administration can only improve in a sustained manner if there is accumulation of knowledge as to what the determinants of successful administration are. While recent theories stress the importance of institutions which lead to contestability and transparency and harness voice, evidence on the efficacy of such reforms in the specific context of tax administration is still lacking. Furthermore, these features only pertain to a part of the array of potential determinants of tax administration perfonnance identified in existing work.

Other major conclusions are:

1. Pre-project diagnosis and project design could be improved in most projects particularly in examining environmental, accountability, cost of administration, managerial autonomy, performance incentives for staff and taxpayer equity and services, and pre-project work could draw more systematically on lessons from previous experience.

2. In particular, institutional components in project design were biased towards organization, IT-related procedures and manpower upgrading, with insufficient attention to accountability and anti-corruption institution building and cost-effectiveness of administration.

3. There are a short list of projects which, despite sharing some of these problems, can serve as guides to good practice.

4. While no overall conclusion can be drawn about project sequencing nor about coordination across projects, instances of poor sequencing and coordination, as reported in ICRs, exist.

_5._ Projects made inadequate use of performance indicators of different kinds. Comparing indicators in projects where these were present, reveals a lack of uniformity in their choice.

6. Project appraisal methods were seldom used and when used were inadequate.

7. Methods used for evaluation of project outcomes are possibly capable of improvement and greater uniformity.

This review has the benefit of hindsight. Most projects in the sample predate the recent recognition of the importance of institutions and performance assessment in the Bank and so almost inevitably fail to meet standards set in the review. Indeed, the Latvia best practice project, which passes most tests in this review, dates from 1998.

# 11. Suggestions For **Future Bank Operations**

Background work is essential to improve .Bank assistance for revenue administration. A strategy needs to be articulated for the Bank, incorporating current perceptions of a best practice 28
framnework for tax and customs administration and their reform process. The nature of Bank involvement should also be clarified. For exarnple, one promising line of action, that has been suggested is for the Bank to support and strengthen regional associations of tax administrations which can act as catalysts of change. A second, may be to partner and support private sector consultancy organizations to enable them to manage major components of administrative reform. A third area of action is to formally institutionalize the knowledge accumulation process, an area which is particularly important in fields like tax administration. This will also have implications for the skill mix of staff and, therefore, staff recruitment and training plans. Regarding specific areas, a number of recommnendations are suggested by the analysis in this review.

# _Diagnosis_

1. Pre-project diagnostic work should be based on a comprehensive framework, which pays sufficient attention to institutions in addition to traditional concerns of tax administrators. Ideally, a single but flexible framework should be adopted throughout the Bank.

2. Diagnostic performance measurement should be done, where possible, quantitatively according to standard indicators and against preferably cross-country benchmarks.

3. Pre-project work should also include a review of (or a citation of) the key determinants of good administration and project implementation.

# _Design_

4. Project design should be based on a strategic vision of the administration, pay adequate attention to good governance but should, nevertheless be limited in scope given a country's implementation capacity. Alternative designs and sequencing should be analyzed and their rationale provided.

_5._ To generate long-term lessons, projects should specify hypotheses being tested.

# _Performance indicators_

6. A standard set of outcome performance (effectiveness, efficiency, accountability) indicators for tax and customs administrators should be specified, to be drawn on for all projects.<sup>16</sup> 7. Diagnostic work should give rise to pre-project base values of performance indicators for outcome assessment.

8. Outcome performance indicators for projects should not only permit project performance to be tracked, but should be chosen to permit hypotheses to be evaluated.

9. The use of taxpayer surveys should be an integral part of both diagnostic work and performance appraisal if not precluded by cost considerations.

10. Standard guidelines for quantitative input, process and output indicators should be laid down and mandated.

> 1<sup>6</sup> A set of performance indicators is being developed by the Tax Policy and Administration Thematic Group. A preliminary version has already been field tested. Similar work is also being done at the Inter-American Development Bank, the IMF and the OECD.

29
# _Appraisal_

- It. A standard, quantitative, appraisal framework or tool needs to be developed for tax administration projects. If possible, this should be linked to an economic impact assessment model such as the 1-2-3 model developed by DEC should be incorporated in this tool.

12. The tool should be designed to allow for risk assessment and sensitivity to key parameters like shadow values.

# _Post-project Evaluation_

13. The use of performance indicators specified in the project is essential if the evaluation is to have comparability across projects.

14. The assignment of outcome, sustainability, institutional development and Bank/borrower performance rankings should, as far as possible, be based on the quantitative indicators specified.

15. To facilitate the knowledge gathering and hypothesis testing role of projects (lessons learned) greater involvement of academic consultants could be desirable in diagnostic, design and evaluation phases.

30
# **Apppendix: Detailed Tables**

|**Table**|**Al. Inclusion in D**<br>(43 projects with a m<br>(|**iagnosis of Tax Structure Analysis**<br>ajor administration component)<br>% of projects)||
|---|---|---|---|
||TALs, LILs, SILs, R|ILs|SALs and SECALs|
|Major Taxes in Country|78.9|Tax to GDP ratio|100.0|
|Tax to GDP ratio|75.0|Major Taxes in Country|95.2|
|Recent Reform History: rates|63.6|Recent Reform History: base and<br>exemptions|90.0|
|Nature of Exemptions|61.1|Nature of Exemptions|85.7|
|Tax shares|58.8|Recent Reform History: rates|85.0|
|Rates|57.9|Base indicators|76.2|
|Base indicators|52.6|Tax shares|66.7|
|Other Analysis|47.4|Rates|66.7|
|Recent Reform History: base<br>and exemptions|45.5|Other Analysis|66.7|
|Effective Rates|15.8|Buoyancies/ Elasticities|28.6|
|Other Formal Analysis|15.8|Effective Rates|23.8|
|Buoyancies/ Elasticities|11.8|Other Formal Analysis|10.0|
|Tax to base ratio: other|10.5|Tax to base ratio: other|9.5|

Notes: 1. Projects are included if they examine an area, however sketchily and whether qualitatively or quantitatively.

2. Projects where the diagnosis category is not relevant are excluded from the denominator in calculating percentages.

|**Table A2. In**|**clusion in Diagnosis of**<br>(43 projects with a major<br>**(%/**of|**Effectiveness Indicators and**<br>administration component)<br>projects)|**Analysis**<br>|
|---|---|---|---|
||TALs, LILs, SILs, RILs||SALs and SECALs|
|Number of taxpayers|52.4|International comparison|38.1|
|Taxpayer service|38.1|Corruption|23.8|
|Tax Gap: Specific activities|33.3|Number of taxpayers|19.0|
|Overall Tax gap|25.0|Tax gap: specific taxes|14.3|
|International comparison|25.0|Overall Tax gap|9.5|
|Corruption|22.7|Tax Gap: Specific activities|9.5|
|Tax gap: specific taxes|20.0|Taxpayer service|9.5|
|Note: See the notes to Table 4|.3.|||

|**Table A3. Inclusion**<br>(43 proj|**in Diagnosis o**<br>ects with a major<br>(%of|**f Efficiency Indicators and Analys**<br>administration component)<br>projects)|**is**|
|---|---|---|---|
||TALs, LILs,<br>SILs, RILs|<br>_|SALs and<br>SECALs|
|Technology deployment|85.7|Activity wise efficiency|40.0|
|Activity wise efficiency|66.7|Technology deployment|33.3|
|Manpower|57.1|Extent of Outsourcing/ Privatization|33.3|
|Extent of Outsourcing/ Privatization|47.6|Manpower|28.6|

31
|Taxpayer compliance cost/time|42.9|Tax processing time|23.8|
|---|---|---|---|
|Tax processing time|42.9|Wage cost|23.8|
|Wage cost|23.8|Budgetary cost of tax administration|14.3|
|Budgetary cost of tax administration|14.3|Tax by tax efficiency|10.0|
|Cost/ revenue ratio|14.3|Taxpayer compliance cost/time|9.5|
|Tax by tax efficiency|10.0|Overall tax collection cost|0.0|
|Overall tax collection cost|4.8|Cost/ revenue ratio|0.0|
|Note: See the notes to Table 4.3.||||

|**Table A4. Inclusion in**<br>(43 proje|**Diagnosis of E**<br>cts with a major<br>**(%**of p<br>|**nvironmental Indicators and Analysis**<br>administration component)<br>rojects)|<br>|
|---|---|---|---|
||TALs, LILs,<br>SILs, RILs||SALs and<br>SECALs|
|Economic conditions|76.2|Economic conditions|95.2|
|Taxpayer compliance|52.4|Taxpayer compliance|61.9|
|Tax policy constraints|38.1|Tax policy constraints|47.6|
|Legal system|33.3|Legal system|47.6|
|Civil service and general government|23.8|Financial sector and banks|47.6|
|Financial sector and banks|23.8|Civil service and general government|33.3|
|Other|18.2|Law and order|33.3|
|Extent of market penetration|17.6|Other|28.6|
|Law and order|10.0|Extent of market penetration|15.8|
|Note: See the notes to Table 4.3.||||

|**Table A5. Inclusion in Diagnos**<br>(43 project|**is of Institu**<br>s with a majo<br>(% of|**tional and Capacity Indicators and Ana**<br>r administration component)<br>projects)|**lysis**|
|---|---|---|---|
|**l_______________________________**|TALs, LILs,<br>lSILs, RILs|<br> __SECALs|SALs and|
|Basic organization|77.3|Taxpayer Obligations, Laws and Procedures|61.9|
|Computerization extent|72.7|Organizational structure|57.1|
|Organizational structure|68.2|Recent reform history|47.6|
|Recent reform history|59.1|Basic organization|38.1|
|Manpower quality|59.1|Computerization extent|38.1|
|Taxpayer Obligations, Laws and<br>Procedures|57.1|Organizational autonomy|33.3|
|Policy making/analysis and legal analysis<br>capacity|54.5|Policy making/analysis and legal analysis<br>capacity|33.3|
|Training facilities|50.0|Budget size|19.0|
|Quality and quantity of data disseminated|40.9|Manpower quality|19.0|
|Organizational autonomy|33.3|External/Internal audit quality|19.0|
|Budget size|14.3|Quality and quantity of data disseminated|19.0|
|External/Internal audit quality|9.1|Other|19.0|
|Other external monitoring|9.1|Budget determination procedures|9.5|
|Other|4.5|Training facilities|9.5|
|Budget determination procedures|0.0|Other external monitoring|9.5|
|Note: See the notes to Table 4.3.||||

32
**Table A6. Percentage of Projects with Various Tax Structure Reform Components** 43 projects with a major administration component

(% of relevant projects)

|TALs, LILs, RILs, S|ILs|SALs and SE|CALs|
|---|---|---|---|
|Sales/VAT|57.1|Income|85.7|
|Income|42.9|Corporation|85.7|
|Corporation|38.5|Sales/VAT|85.7|
|Import tariffs|23.1|Import tariffs|81.0|
|Other national|23.1|Export tariffs|52.6|
|Social Security|16.7|Excise|52.4|
|Presumptive tax - business|14.3|Other national|36.8|
|Turnover|11.1|Presumptive tax - individual|19.0|
|Other Subnational|9.1|Presumptive tax - business|19.0|
|Export tariffs|8.3|Other Subnational|16.7|
|Excise|7.7|Tumover|14.3|
|Property|7.7|Property|14.3|
|Presumptive tax - individual|7.1|Social Security|14.3|
||**(b) Tax**|**Policy Institutions**||
|50.0||40.0||

Notes: 1. The table only shows that the component is present in the project - it does not indicate whether the component itself is comprehensive.

2. Projects where the component is not relevant are excluded from the denominator in calculating percentages.

**Table A7. Percentage of Projects with Tax Administration Environmental Components** 43 projects with a major administration component (% of relevant projects)

|TALs, LILs, RI|Ls, SILs|SALs and|SECALs|
|---|---|---|---|
|Legal system|_50.0_|Other environental|76.2|
|Financial system|27.3|Financial system|57.1|
|Other environmental|22.7|Civil service|52.4|
|Civil service|18.2|Legal system|42.9|
|Judicial system|4.5|Judicial system|4.8|

Notes: 1. See the notes to Table _5.1._ 2. The four main "Other environmental" components are, in descending order of importance, budget and treasury management, privatization and parastatal reforn, non-tariff trade liberalization and the private sector regulatory framework.

**Table A8. Percentage of Projects with Tax Administration Functional Components** 43 projects with a major administration component (% of relevant projects)

|TALs, LILs, RILs, SILs|SALs and S|ECALs|
|---|---|---|
|Cargo clearance<br>90.0|Tax assessment|90.5|
|Tax assessment<br>81.8|Audit|85.7|
|Audit<br>81.8|Tax collection|85.7|
|Taxpayer identification<br>72.7|Taxpayer compliance|81.0|
|Tax collection<br>72.7|Tax base valuation|61.9|

33
|Trade facilitation|72.7|Penalties/ prosecution|57.1|
|---|---|---|---|
|Taxpayer compliance|68.2|Taxpayer identification|57.1|
|Tax base valuation|59.1|Tax filing|50.0|
|Penalties/ prosecution|54.5|Cargo clearance|44.4|
|Taxpayer service quality|50.0|Arrears|42.9|
|Taxpayer education|45.5|Trade facilitation|38.9|
|Tax filing|45.5|Large taxpayers|38.1|
|Large taxpayers|45.5|Small taxpayers|23.8|
|Arrears|28.6|Taxpayer service quality|19.0|
|Small taxpayers|18.2|Taxpayer education|9.5|
|Note: See the notes to Table 5.1||||

|**Table A9. Percentage of P**<br>43 p<br>|**rojects with**<br>rojects with a<br>(% o<br>|**Administrative Institutional or Capacity Building**<br>**Components**<br>major administration component<br>f relevant projects)<br>|
|---|---|---|
|TALs, LILs, RILs, SIL|s|SALs and SECALs<br>|
|Computerization|100.0|Organizational structure<br>76.2|
|Organizational structure|77.3|Manpower rationalization<br>71.4|
|Procedural manuals|77.3|Wages and incentives<br>61.9|
|Manpower rationalization|72.7|Computerization<br>52.4|
|Training facilities|72.7|Recruitment and evaluation<br>52.4|
|Other capital/ infrastructure|68.2|Basic Organization<br>47.6|
|Policy making! analysis and legal<br>analysis capacity|50.0<br>=|Training facilities<br>42.9|
|Manpower quality|50.0|Procedural manuals<br>38.1|
|Basic Organization|45.5|Other capital/ infrastructure<br>38.1|
|Recruitment and evaluation|40.9|Manpower quality<br>38.1|
|Quality and quantity of data<br>disseminated|36.4|Organizational autonomy<br>33.3|
|Wages and incentives<br>____________________________|31.8<br>___________|Policy making! analysis and legal<br>33.3<br>______<br>analysis capacity|
|Privatization|27.3|Extemal/Intemal audit quality<br>33.3|
|Other|22.7|Other extemal monitoring<br>28.6|
|Organizational autonomy|18.2|Privatization<br>23.8|
|External/lnternal audit quality|18.2|Budget size<br>19.0|
|Other external monitoring|13.6|Quality and quantity of data<br>19.0<br>disseminated|
|Budget determination/financial<br>mgmt procedures|9.1<br>|Budget determination/financial<br>14.3<br>_mgmt procedures|
|Budget size|4.5|Other<br>9.5|
|<br>Notes: 1. See the note to Table 5.1.<br>2. While no project has an explicit an<br>corruption as a secondary project obje|ti-corruption<br>ctive.|component, 2 TALs (9%) and 5 SALs (23%) mention reducing|

34