metadata: authors: 0: author: Gokcekus,Omer
1: author: Manning, Nick
2: author: Mukherjee,Ranjana
count: World
docna: 0: docna: Public officials and their institutional
            environment - an analytical model for assessing the impact
            of institutional change on public sector performance
docty: Policy Research Working Paper
subtopic: Consumption,Fiscal & Monetary Policy,Labor Markets,Administrative & Civil Service Reform,Public Sector Administrative & Civil
            Service Reform,Inequality,Democratic Government,Public Sector Administrative and Civil
            Service Reform,De Facto Governments
teratopic: Poverty Reduction
abstracts: cdata!: To perform well, public officials must
            be confident enough about the future, to be able to see a
            relationship between their efforts, and an eventual outcome.
            Their expectations are shaped by their institutional
            environment. If the rules are not credible, or are unlikely
            to be enforced, of if they expect policies to be
            contradicted, or resources to flow unpredictably, results
            will be uncertain, so there is little point in working
            purposefully. The authors present an analytical framework,
            used to design a series of surveys of public officials'
            views of their institutional environment, and to analyze the
            information generated in fifteen countries. They describe
            how survey results help map public sector's strengths,
            and weaknesses, and offer an approach to identifying
            potential payoffs from reforms. The framework emphasizes how
            heterogeneous incentives, and institutional arrangements are
            within he public sector. It emphasizes how important it is
            for policymakers to base decisions on information (not
            generalizations) that suggests what is most likely to work,
            and where. In building on the premise that public
            officials' actions - and hence their
            organization's performance - depend on the
            institutional environment in which they find themselves,
            this framework avoids simplistic anti-government positions,
            bur doesn't defend poor performance. Some public
            officials perform poorly, and engage in rent seeking, but
            some selfless, and determined public officials, work hard
            under extremely difficult conditions. This framework offers
            an approach for understanding both bad performance, and
            good, and for presenting the results to policymakers in a
            format that leads to more informed choices, about public
            sector reform. Types of reforms discussed include
            strengthening the credibility of rules for evaluation, for
            record management, for training, and for recruitment;
            ensuring that staff support government policy; preventing
            political interference, or micro-management; assuring staff
            that they will be treated fairly; and, making government
            policies consistent.
repnme: repnme: Public officials and their institutional
            environment - an analytical model for assessing the impact
            of institutional change on public sector performance
display_title: Public officials and their
            institutional environment - an analytical model for
            assessing the impact of institutional change on public
            sector performance
listing_relative_url: /research/2000/08/693243/public-officials-institutional-environment-analytical-model-assessing-impact-institutional-change-public-sector-performance
topicv3: Education, Skills Development and Labor
            Market m1326135 1484,Poverty m1326687 695,Individual Performance Management m1326767 2058,Macroeconomic and Structural Policies
            m1327380 439,Jobs m1503931 2670
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Poverty Reduction
origu: Poverty Reduction & Economic Management Network
no_of_pages: 74
author_keys: 0: author_key: 435778
1: author_key: 390611
model_extractions: POLICY RESEARCH WORKING PAPER 2427

# Summary findings

To perform well, public officials must be confident enough about the future to be able to see a relationship between their efforts and an eventual outcome. Their expectations are shaped by their institutional environment. If the rules are not credible or are unlikely to be enforced, or if they expect policies to be contradicted or resources to flow unpredictably, results will be uncertain, so there is little point in working purposefully.

Manning, Mukherjee, and Gokcekus present an analytical framework used to design a series of surveys of public officials' views of their institutional environment and to analyze the information generated in 15 countries. They describe how survey results help map a public sector's strengths and weaknesses and offer an approach to identifying potential payoffs from reforms.

The framework emphasizes how heterogeneous incentives and institutional arrangements are within the public sector. It emphasizes how important it is for policymakers to base decisions on information (not generalizations) that suggests what is most likely to work, and where.

In building on the premise that public officials' actions-and hence their organizations' performancedepend on the institutional environment in which they find themselves, this framework avoids simplistic antigovernment positions but doesn't defend poor performance. Some public officials perform poorly and engage in rent seeking, but some selfless and determined public officials work hard under extremely difficult conditions. This framework offers an approach for understanding both bad performance and good and for presenting the results to policymakers in a format that leads to more informed choices about public sector reform.

Types of reforms discussed include strengthening the credibility of rules for evaluation, for record management, for training, and for recruitment; ensuring that staff support government policy; preventing political interference or micromanagement; assuring staff that they will be treated fairly; and making government policies consistent.

This paper-a product of the Poverty Reduction and Economic Management Network-is part of a larger effort in the network to develop practical strategies for the reform of public sector institutions. The surveys were funded by the World Bank-Netherlands Partnership Program. Copies of the paper are available free from the World Bank, 1818 H Street NW, Washington, DC 20433. Please contact Claudia Nolan, room MC4-562, telephone 202-473-0030, fax 202-522-7132, email addresscnolan@worldbank.org. Policy ResearchWorkingPapers are also posted on theWeb atwww.worldbank.org/ research/workingpapers. The authors may be contacted at nmanning@worldbank.org, rmukherjee@worldbank.org, or ogokcekus@worldbank.org. August 2000. (65 pages) _The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the view of the World Bank, its Executive Directors, or the countries they represeni._ Produced by the Policy Research Dissemination Center
# **Public officials and their institutional environment:** An analytical model for assessing the impact of institutional change on public sector performance

Nick Manning, Ranjana Mukherjee and Omer Gokcekus Nick Manning is Senior Public Sector Reform Specialist in the Public Sector group of PREM network. Ranjana Mukherjee is a consultant in the Public Sector group of PREM network. Omer Gokcekus is Assistant Professor of Economics at North Carolina Central University and Scholar-in-Residence at Economics Department of Duke University.

The authors wish to specially thank Gary Reid (ECSPE) and Yasuhiko Matsuda (LCSPS) for their review of this paper and their very helpful suggestions. The report benefited from discussions with Meta de Coquereaumont (DECRA), Steve Knack (PRMPS), Jeff Rinne (PRMPS) and Adeel Malik (PRMPS). Yasuhiko Matsuda (LCSPS) contributed to the conceptualization of the institutional environment in Section 2.
Abbreviations and Acronyms

|BNPP<br>|Bank Netherlands Partnership Program<br>|
|---|---|
|CAPAM<br>|Commonwealth Association for Public Administration and Management<br>|
|DECRA<br>|Research Advisory, Development Economics and Chief Economist<br>|
|ECSPE<br>|Europe and Central Asia, Poverty Reduction and Economic Management<br>|
|HRMIS<br>|Human Resource Management Information System<br>|
|INE<br>|Instituto Nacional de Estadistica<br>|
|IRR<br>|Institutional Rate of Return<br>|
|LCSPS<br>|Latin America and Caribbean Region, Public Sector<br>|
|OED<br>|Operations Evaluation Departnent<br>|
|OLS<br>|Ordinary Least Squares<br>|
|OECD<br>|Organization for Economic Cooperation in Development<br>|
|NGO<br>|Non Governmental Organization<br>|
|PREM.<br>|Poverty Reduction and Economic Management<br>|
|SAFCO<br>|Sistema de Administraci6n Financiera y Control Gubernamental<br>Bolivian Law 1178 on Financial Management and Control<br>|
|SURE|Seemingly Unrelated Regression Equations|

ii
# Table of Contents

|1.<br>Introduction ...........................................................|1|
|---|---|
|2.<br>Conceptual Framework ..........................................................|3|
|2.1 What is meant by institutional environment? ...................................................|.... 3|
|2.2 How can institutional environment be measured? ............................................|.... 7|
|2.3 How can performance be measured? .........................................<br>................|.<br>8|
|2.4 Whose performance? - different units of analysis ...............................|**1**...............<br>I|
|3.<br>Survey methodology ..........................................................|14|
|3.1 Questionnaire design .........................................................|14|
|3.2 Sampling methodology .........................................................|15|
|4.<br>Construction of Indicators for Institutional Environment ..................... .............|..... 17|
|5.<br>Construction of Indicators for Performance .........................................................|20|
|6.<br>The Connection between Performance and Institutional Environment .................|... 22|
|7.<br>Presenting Opportunities for Reforms .........................................<br>................|<br>24|
|7.1 Step 1 - it's the institutions not the people .......................................................|.. 24|
|7.2 Step 2 - encourage contestation in interpretation ................................... .........|... 25|
|7.3 Step 3 - challenge prior assumptions ..........................................................|25|
|7.4 Step 4 - encourage informed speculation on performance pay-offs ........... ...|...... 26|
|8.<br>Summary ..........................................................|34|
|Selected Bibliography ..........................................................|36|
|Appendices .........................................................|41|
|Appendix 1: Illustrative questions to probe for dimensions of the institutional<br>environment ..........................................................|41|
|Appendix 2: Institutional environment indicators in the East Caribbean States ...|.... 46|
|Appendix 3: Illustrative questions to probe for dimensions of performance ........|.... 48|
|Appendix 4:<br>Performance indicators in the East Caribbean States ........................|... 54|
|Appendix 5: Connection between performance and institutional environment ....|.... 56|
|Appendix 6: Econometric and analytical issues ....................................................|.. 59|
# Index of Figures

|Figure 1. Analytical model for BNPP-funded surveys ............................................. ......._3_|
|---|
|Figure 2. Credible rules, credible policy and predictable resourcing constitute the<br>institutional environment within any perspective ...................... .<br>8|
|Figure 3 The main components of government employment .................... .................... 11|
|Figure 4. Officials say that the whole government is much more corrupt than their own<br>organization ............................................................<br>_15_|
|Figure_5._Construction of indicators for institutional environment in East Caribbean<br>states ...........................................................<br>17|
|Figure 6. Construction of indicators for performance in East Caribbean states .............. 20|
|Figure 7. Performance and Institutional Environment in sampled Bolivian public sector<br>organizations ...........................................................<br>23|
|Figure 8. Rule complexity affects performance in Albania ....................... .................... 25|
|Figure 9. Expected impact of reforms on components of performance in Bolivia ......... 32|
|Figure 10. Effect of institutional environment on results focus in Bolivia ....................... 56|
|Figure 11. Effect of institutional environment on accountability in Bolivia .......... .......... 57|
|Figure 12. Effect of institutional environment on employee morale in Bolivia ......... ...... 58|
|Figure 13. Expected impact of reforms on different components of performance in Bolivia<br>Central Administrations ....................................................<br>_65_|

# Index of Tables

|Table 1 Illustrative prior assertions tested against survey evidence ............. ............|....<br>26|
|---|---|
|Table 2 Reform Interventions and anticipated performance pay offs in the Bolivian<br>public sector .............................................................|31|
|Table 3 Reform Interventions and anticipated performance pay offs in Bolivia Centr<br>Administrations ...................................................|al<br>64|
|Statistical Table I Rule credibility indicator and components (East Caribbean) ..........|. 46|
|Statistical Table 2 Policy credibility indicator and components (East Caribbean) .......|. 46|
|Statistical Table 3 Indicator of resource adequacy and predictability and components<br>(East Caribbean) ............................................|46|
|Statistical Table 4 Indicator of overall institutional environment and components (Ea<br>Caribbean) ...........................................................|st<br>47|
|Statistical Table 5 Results focus indicator and components (East Caribbean)<br>........|......<br>54|
|Statistical Table 6 Accountability indicator and components (East Caribbean) ......... .|.. 54|
|Statistical Table 7 Employee morale indicator and components (East Caribbean) .......|. 54|
|Statistical Table 8 Overall performance indicator and components (East Caribbean)|..._55_|
|Statistical Table 9 Calculating the IRRs (Bolivia) ........................................................|63|
# 1. **Introduction**

This paper lays down the analytical framework that was used in designing a series of surveys of the views of public officials concerning their institutional environment and in analyzing the information that was generated in 15 countries. It describes how the survey results help to challenge preconceptions, map the strengths and weaknesses of a public sector, and identify potential pay-offs from reforrn interventions.

The need for an analytical framework emerged from the lessons of past experience in the World Bank. The Operations Evaluation Department reported that during 1980-97, only one-third of the Bank's closed civil service reforrn interventions had successful outcomes.' Other reviews of the Bank's public sector reform efforts have identified shortcomings of the Bank's approach in this area, pointing out the risks of a narrow and 'technocratic' view of what is needed for public sector reform, and of a reliance on 'best practice' models that have not been feasible in the particular country setting.<sup>2</sup> The Bank's most recent strategy for reforning public institutions has identified that for the approach to be effective: "... _we need to work with our partners to understand and address the broad range of incentives and pressures - both inside and outside of government - that affect public sector performance."'_<sup>_3_</sup> The strategy paper also points out that for the analytic work to be useful: _"We need to start with a thorough understanding of what exists on the ground and emphasize goodfit rather than any one-size-fits-all notion of best practice. And we need to work with our clients and other partners to develop and apply analytic tools effectively. "_ The framework laid out in this paper addresses both these needs. Public officials are not inherently rapacious rent-seekers; they respond to the incentive structure they face. There is ample evidence, both theoretical and empirical, to suggest that the performance of public officials is greatly determined by the institutional environment that they find themselves in. The framework recognizes that incentive systems are different in different countries - and vary across types of organizations and types of officials in the same country. Recognizing this, it offers a method for designing surveys that uncover the sanctions and rewards that drive behavior from those who may be subjected to them - the public officials themselves.

Surveys were designed to cover several areas. They elicit a contextual description of the public sector, including characteristics of respondents, their reasons for joining the public sector and the length of time worked in government. Second, they offer the possibility of I World Bank (1999).

2 World Bank (2000). http://www-wbweb.worldbank.orgfprem/prmps/ 3 Ibid, p.4-5.

1
presenting policy-makers with robust confirmation of the theory-based assertion that it is the institutional environment in general, and rule credibility, policy credibility, and resource adequacy and predictability in particular, which drive performance. Third, they provide an opportunity to inform in-country reform discussions with current data.

Fourth, the surveys also allow country-specific hypotheses to be tested. Many widely held views on public officials are often repeated but without substantive evidence - more akin to "urban myths" than to empirical observations. The surveys allow such assertions to be tested and supported or refuted.

Finally, the surveys allow some assessment of _which_ aspects of institutional environment are impacting on performance. It suggests those reform actions that seem most promising for higher performance, given the country's existing public sector conditions.

The surveys and analyses were financed under the Bank-Netherlands Partnership Program (BNPP). Surveys of public officials funded by the BNPP have been completed in Albania, Argentina, Bangladesh, Bolivia, East Caribbean States, Guyana and Indonesia. Surveys are in progress or are being initiated in Bulgaria, Cameroon, India, Kenya, Macedonia, and Moldova. This program has provided funds for data analysis of a separate survey of public officials in Armenia, and another data analysis exercise is also planned for an ongoing survey in Benin. The survey instruments were based on a model designed in collaboration with Professor Bert Rockman of the University of Pittsburgh. World Bank staff working on a particular country tailored this basic questionnaire according to country-specific background and issues in public sector reform.

This paper summarizes the premises on which the surveys were designed and analyzed, and the analytical framework. In explaining that analytical framework, examples have been used whenever it was considered their use would help make ideas clear. The examples have been drawn from survey experience during 1999 and 2000, and country data in Albania, Armenia, Bolivia, Guyana, Macedonia, and five countries in the East Caribbean region. The selection of examples was guided by their usefulness in illustrating the steps in the analytical framework, and not to point out any strengths or weaknesses of any country's public sector.

Section 2 lays out the conceptual framework. Section 3 summarizes the survey methodology. Section 4 describes the approach for measuring the institutional environment using the East Caribbean example as an illustration. The approach for measuring aspects of performance is described in section 5, again using the same example of East Caribbean states. The relationship between performance and institutional environment is described in section 6, using the illustration of the Bolivian public sector. The Bolivian example was also used in Section 7 to show how survey results can be used to identify promising prospects for reform.

Section 8 concludes with an overview of how the analytical framework is being used in practice.

2
# **2. Conceptual Framework**

Public officials in developing countries are often viewed as unskilled, poorly educated and poorly motivated to perform their official tasks. The assumption is frequently made that public officials are primarily motivated to exploit any official privileges that their positions give them to engage in opportunistic behaviors. Given the poor performance of public bureaucracies in many developing countries, and the undeniable reality of the low public sector salaries, such an assumption may not be wholly unrealistic. Yet, there are many public officials who, despite such a poor incentive environment, may perform their tasks without shirking, stealing or subverting in spite of overwhelming incentives for opportunism. For every rent-seeking or poorly motivated public official, there may be another who is providing a valuable service under extremely adverse circumstances. There is perhaps more to be learnt from an examination of why some public officials perform against the odds than from reciting the standard litany of failing public services.

This paper starts from the premise that public officials' actions - and hence the performance of their organizations - depend upon the institutional environment in which they find themselves. In this way the analytical framework avoids the anti-government bias that pervades much of the literature on the public sector in developing countries, without attempting a defensive justification for poor performance.<sup>4</sup> The analytical model allows verification of the assumption that it is the institutional environment that impacts on the performance of organizations in the public sector rather than the waywardness or malfeasance of officials. Accordingly, this model suggests that reform interventions that improve any or all aspects of the institutional environment will result in higher perfornance of some or all of the areas in which performance is measured.

# **Figure 1. Analytical model for BNPP-funded surveys**

- Institutional environment: Performance: * policy credibility i * results focus * rule credibility * accountability . resource adequacy and * employee morale predictability

# _2.1 What is meant by institutional environment?_<sup>_5_</sup>

Institutions are the humanly devised constraints, or set of relational contracts that guide public officials' activities. They are made up of formal constraints (e.g. rules, laws, constitutions), informal constraints (e.g. norms of behavior, conventions, codes of conduct), and their enforcement characteristics. For public officials, formal rules are laid 4 See Tendler (1997) for a summary of these perspectives. This section was developed in collaboration with Yasuhiko Matsuda of LCSPS.

3
down in their code of conduct and operation manuals, in the budget documents, and in the many decrees, directives and instructions through which policy is conveyed. The informal rules are what the officials collectively understand as appropriate behavior, 'how we do things around here'. For example, not vigorously implementing the minister's newly announced scheme might result in a transfer to a position in a remote and inaccessible area.

Institutions provide the incentives that provoke or prohibit certain actions. Rules and regulations, formal and informal, together define the incentive structure of public officials within their organization, or within their peer group, or across the public sector as a whole.<sup>6</sup> Fundamentally then, this institutional environment shapes the expectations of public officials. If there is a rule about the management of records in the organization, or about methods of performance appraisal, then behavior will vary according to whether the official believes that breaches of these rules really will be punished. Similarly, willingness to gear actions to support Ministerial policies is somewhat greater if officials believe that policies will remain in force for a period of time, and will not be undermined by other policies of equal force. Expectations that policies are likely to be soon reversed lead, at best, to second-guessing of what the next ones might look like. At worst they lead to cynical disregard for any announced policy.

_Theprincipal-agent perspective_ assumes incompatibility of interests between the principal and the agent.! The consequent pursuit of private interests can take various forms including shirking (i.e., under-production of outputs desired by the principal), stealing (i.e., embezzlement and other forms of bureaucratic corruption), and sabotage/subversion (i.e., pursuit of objectives that are blatantly against the organization's goals).

The relationship between the principal and the agent thus conceptualized is a contractual one, whereby the agent agrees to perform certain tasks, which he/she would not have done without the "contract," in exchange for a set of positive incentives to be provided by the principal. The principal enforces the contract through the monitoring mechanisms. In this perspective, an organization is a chain of voluntary contracts in a hierarchical structure.<sup>8</sup> Suggested performance improvements emerging from this perspective tend to refer to increases in official salaries, greater contestability in public service provision, and stiff penalties for rule breaking.

6 See Horn (1995) 7 The principal-agen **t** heory is not the only application of economic theories to the study of organizations. For a more thorough and nuanced treatment of the 'new economics of organization,' see Moe (1984). For a systematic application of the transactions cost approach to explaining structures of public bureaucracies, see Horn (1995). More generally, for an application of principal-agen **t** heory to the subject of state reform, see Przeworski (1999).

8 The logic of principal-agent relations can be extended beyond a particular organizational boundary to include an analysis of the relationship between a bureaucratic organization (as represented by its head) and government (i.e., minister) or politicians (i.e., president, legislature). Examples include Weingast and Moran (1983), McCubbins and Schwartz (1984), and McCubbins, Noll, and Weingast (1987, 1989). 9 See, for example, Klitgaard (1997).

4
While persuasive, there are some limitations to this model. First, the economic theories tend to focus on formal institutions (e.g., official wages, as opposed to career possibilities, as an incentive; formal auditing, as opposed to social/peer pressure, as a monitoring mechanism).'" Second, crude economic theories may draw our attention disproportionately to the "minimal state" agenda with its accompanying mantra of "reduce bureaucratic discretion". Third, the whole rationale of the principal-agent theory breaks down if we accept that part of the bureaucracy's functions is to protect long-term public interests by not being fully responsive to particular short-term concerns of the government of the day.<sup>1</sup> Finally, unlike profit-seeking private firms that presumably operate with a clear ownership pattern and an unambiguous line of accountability, public bureaucracies usually operate under supervision from multiple principals.1<sup>2</sup> In sum, _the principal-agent perspective_ emphasizes:

1. Monetary incentives;

2. Mechanisms for monitoring contracts; and 3. Clarity of agency mission.

Critics complain that such economic theories offer relatively little in the way of explanations for non-opportunistic behavior in the public bureaucracy.<sup>3</sup> Such seemingly selfless actions are perplexing within the principal-agent perspective. However, the perspective is also limited by its focus on performance that can be observed and measured. Some other tasks require such a high level of discretion with virtually no possibility of effective monitoring that the principal cannot rely on economic incentives. Other non-monetary incentives, such as identification with the agency's mission or with the agency itself as well as esprit de corps, must be considered.

_The principled-agent perspective_ purports to offer more empirically accurate account of organizational dynamics and bureaucratic behavior, but at the same time its analytical power is somewhat more limited than that of the principal-agent theories. Leadership is seen to be key'<sup>4</sup> , as is "a strong sense of mission, effective managerial practice, and high expectations about employee performance".,<sup>5</sup> The perspective builds on the sociological traditions that have tended to see organization as an organic entity and have emphasized '° See Heyman (1988) and Miller (1994).

> 3' A large-scale survey-based study of senior bureaucrats and politicians in five western countries found that "policymaking by bureaucrats is characterized by continuity, stability, and predictability" in clear distinction to the views of politicians who see themselves as "advocates, partisans, and tribunes" (Aberbach, Putnam, and Rockman, 1981, p. 25 6-7).

12 See Moe (1984), Moe (1990), Simon (1991), and Fukuyama (1995).

1 3 They are "far better at explaining why bureaucrats shirk, subvert, or steal than they are at explaining why bureaucrats behave as 'principled agents' - workers who do not shirk, subvert, or steal on the job even when the pecuniary and other tangible incentives to refrain from these behaviors are weak or nonexistent" (Dilulio, 1994, p. 277). Herbert Simon also provides a critique of the principal-agent variant labeled transactions cost analysis (Simon 1991).

14 Dilulio (1994). Other influential organization theorists including Selznick (1957) and Wilson (1989) have also underscored critical roles of leadership in accounting for organizational performance. IS See Grindle (1997, p.491).

_5_
culture, leadership, and styles of authority and power'<sup>6</sup> , and that emphasize the limitations of formal contracts. **17** There is a sizeable literature that sees officials' identification with organizational objectives as key to their "principled" behaviors. Another source of motivation is public officials' long-term career paths. When bureaucratic career paths are fairly well defined and predictable, they induce a particular range of behaviors from public officials who are interested in improving their career prospects. Recruitment and promotion processes work as socialization mechanisms that selectively preserve particular types of individuals.'<sup>8</sup> One key to performance within the principled-agent perspective is that career paths are long term, allowing reputational rewards to be developed.'<sup>9</sup> Thus _the principled-agent_ perspective emphasizes:

1. Clarity of agency mission;

2. Organizational culture and trust; and 3. Long-term career paths.

Some political scientists see public sector organizations, not exclusively in developing countries, as webs of _patron-client_ networks.<sup>20</sup> This perspective observes the patronclient relationship as based more on trust and loyalty than short-term, transaction-specific utility-maximizing calculation and bargaining by both parties.

Unlike a typical principal-agent relationship marked by incentive incompatibility between the two actors, a patron-client relationship is based on the recognition of mutual (albeit unequal) benefits that the two actors derive from the relationship. Furthermore, to the extent that incentive incompatibility threatens the possibility of mutual gains, the personal and long-term nature of the relationship between the patron and the client reduces the monitoring costs of the client's behavior, and thus mnitigates the agency problems. The client is bound by the sense of loyalty and obligation as well as motivated by the expectation of present and future benefits from the continued participation in the relationship.

> 16 See Weber (1946), Selznick (1957), and Simon (1974).

> **17** An interesting point is that Williamson's (1975) seminal study of the economics of organization finds the superiority of internal organizations (hierarchies) over markets because of the former's greater ability to control opportunism.

18 Aberbach, Putnam, and Rockman (1981) provide an example of this impact on a range of attitudes and behaviors observed among bureaucrats at senior levels, and Schneider (1993) provides an example of the reverse.

19 OECD (1 994a) also notes that in some countries "the official doctrine is that public servants receive a means of subsistence to enable them to serve the state". The strongest expression of this particular incentive logic is to be found in the German pay policy for established civil servants, _Beamte._ For this group, traditionally, pay "is not regarded as remuneration for work performed or for performance of function, but as a means of livelihood linked to office. (The remuneration) is intended to enable _Beamte, to_ give full commitmen **t** o duty of service to state and to live at appropriate standard for service rank and (to) be independent".

> **20** See Dresang (1974), and Schneider (1991).

6
This perspective assumes that it is more than likely that the loyalty to the patron will overwhelm the client's identification with the organizational objectives. This militates against effective organizational performance if the patron utilizes employment opportunities in the public sector as a means to reward political supporters. This perspective also requires that career paths are long term, although in this case the reason is that the currency of jobs in which the patron rewards the client holds little value if jobs can be withdrawn at any moment.

Patron-client relations require extensive informality. Comprehensive, formal rules with clear policy objectives and restraints on the ability of the principal to micro-manage would remove the ability of the patron to reap the rewards from having placed their clients in key positions.

The _patron-client_ perspective emphasizes:

1. Long term career paths; and

2. Informality.

# _2.2 How can institutional environment be measured?_

These perspectives emphasize different sets of arrangements and consequently they direct attention to different potential levers for improving performance. Whatever perspective is adopted on the motivation of public officials, three particular sets of concerns stand out as fundamnental - the nature of the rules that constrain the behavior of public officials, the nature of the policies that they are asked to implement, and the manner in which they are provided with resources. As shown in Figure 2, these are always the ingredients in the incentive mix, whatever the assumed recipe.<sup>21</sup> Within these dimensions, the institutional environment shapes expectations of future constraints and incentives. An official, who has come to expect that rules will not be enforced, works in an environment of low rule credibility. His or her behavior is shaped by the expectation that the rules for personnel and budget management will not be enforced in the future.<sup>22</sup> The three dimensions of the institutional environment - _rule credibility; policy credibility;_ and _resource adequacy and predictability -_ represent measures of the expectations of public officials concerning the future.

21 See Manning and James (2000) for a discussion of credible regulation within the public sector. See Evans and Manning (2000) for a discussion of policy credibility. 22 This point that institutional arrangements impact individuals' actions in the present by shaping their expectations about the future is made extensively in the institutiona **l** iterature. See, for example, Bendor and Mooklierjee (1987) and Mnookin and Kornhauser (1989). It is consistent with an assertion that the problem facing both public and private sector managers is one of maintaining their collective reputation as a relevant actor among their staff (See Seabriglt 1993). More generally, the literature on cooperation in the absence of third-party enforcement emphasizes that "the shadow of the future" (i.e., the degree to which actors expect to interact again under similar circumstances) significantly determines behavior in the present (See Axelrod and Keohane, 1985).

7
**Figure 2. Credible rules, credible policy and predictable resourcing constitute the institutional environment within any perspective**

|||expecta<br>rule enf|tions of<br>orcement|expecta<br>|tions of po|licy||expecta<br>future r<br>flows|tions of<br>esource|
|---|---|---|---|---|---|---|---|---|---|
|princip|al-~~<br>moetr<br>agency~|~<br>emh-epa<br>**~4**<br>|**44e5**<br>~~~~|**o**<br>C6|siedsiedsie<br>-~~~~~<br>~~~~~~~~~~~~|~~~~~~~~~~~~C|**04'~**<br>|||
|principal-<br>agent<br>|monetary<br>incentives<br>|empha-<br>sized||||||empha-<br>sized||
|perspective|contract<br>enforcemens|empha-<br>tsized|||||||empha-<br>sized|
|principled-|claot tof<br>tyagen<br>mission<br>||||empha-<br>sized||empha-<br>sized|___|empha-<br>sized|
|agent<br>perspectiv|cultureand<br>eMUtual trUSt<br>|**empha-**<br> sized<br>|empha-<br>sized<br>|empha-<br>sized||empha-<br>sized||emnpha-<br>sized<br>|empha-<br>sized|
|patron-|long term<br>arrsm|empha-<br>sized|empha-<br>sizedpri|||||exnpha-||
|<br>client<br>perspectivec|<br>informality<br>_<br>_<br>_<br>_<br>_|<br> opposed<br>_<br>__<br>_|_<br>_<br>_<br>_|_<br>_<br>_|opposed||opposed|||

Against this context, the study has adopted a framnework for measuing the institutional environment built around measurement of these three factors that constitute the institutional environment. Accordingly, indicators are constructed to measure _how much_ rule credibility, policy credibility and resource adequacy and predictability the institutional arrangements are providing.

# _2.3 How can perfiormance be measured?_

In addition to the well-recognized difficulties in assessing organizational performance, quantifying performnance of public organizations runs into two main problems. **23** First, unlike private organizations, public organizations have no single performance indicator that can compare across different types of organization and products, such as profits earned or market share. In the public sector, only a few organizations work for profit.

**23** Acs and Gerlowski **(1996)** list the classic difficulties including measurement errors, random effects, and factors outside the control of the organization. Carter, Klein, and Day _(1995)_ lists ownership, trading/non-trading, competition, accountability, heterogeneity, complexity, and uncertainty as the organizational dimensions and argue that these dimensions show huge variation among public organizations. They also point out the difficulty in measuring the quality of service delivery.

8
Outputs of many organizations such as the audit body or the planning division of the Ministry of Finance are used only by other organizations within the public sector.

Second, public sector organizations are often dealing in goods of both low contestability and measurability.<sup>24</sup> In such circumstances it is generally impossible to find performance measures of the public sector that satisfy the ideal qualities of consistency, comparability, clarity, controllability, comprehensiveness, boundedness, relevance, and feasibility.<sup>25</sup> For instance, when public agencies' perfornance is measured, the metrics are distinctly organization or service-specific. The waiting time for a patient to see a doctor is not readily compared to performance in primary school enrolment. The current extensive debate on performance in the public sector provides illustrations of benchmarks that can be applied over time, but has done little to solve the problem of comparability between diverse agencies and sectors.<sup>2 6</sup> Nevertheless, performance needs to be measured. The analytical framework described in the following pages introduces a new approach to measure performance of public organizations. In particular, it suggests that there are some common assertions that lie behind currently used diverse measures of performance. In particular it identifies three dimensions that are key to any conceptualization of performance: how focused officials have been on results, their accountability for adherence to formal rules, and their morale.<sup>27</sup> _Results focus_ is prima facie evidence that public officials are striving to achieve organizational goals and clearly lies behind consideration of organizational efficiency and effectiveness. It is a reasonable, although empirically testable, proposition that rewarding good performance and punishing bad performance encourages a results focus.

_Accountability_ is perforrnance in the distinctive sense of having adhered to the formal rules and so enabling actual behavior to be tested against mandated standards. For public sector organizations, which work with public funds, accountability is an important element of performance. Corruption is a symptom of poor accountability performance - but the notion of accountability adopted for this study is wider than corruption.<sup>28</sup> Accountability in this sense rests on the past enforcement of regulations. This measure is of course different from an assessment of the existence of credible and appropriate rules and regulations, which is an aspect of the institutional environment. This notion of accountability includes transparency in decision-making and allocation of government resources. This refers, for example, not only to the audit of public accounts and making

- 24 See Girishankar (1999).

- 25 See Shand (1997).

26 See for example OECD (1994b), Public Service and Merit Protection Commission, Australia (1998), Gore (1993) and Osborne and Plastrik (1997).

- 27 A study by the authors of this paper is in progress to empirically examine the link between these performance measures and conventional output and outcome indicators, and its findings will be available in 2000.

2S See World Bank (1998).

9
the audit report available to legislature, but also to a history of regular consultations with the private sector, citizens' groups and NGOs.

Finally, _employee morale_ is also identified as a component of organizational performance. This is both a performance goal in its own right, and a contributor to organizational effectiveness and efficiency as identified in Australian and Canadian studies of public employees of the 1980s.<sup>29</sup> Employee morale is raised by job satisfaction and reduced by the orientational and attitudinal disconnect that managers have with their staff. **30** This latter concept of a disconnect between the managers of organizations and other officials working in them has been usefuilly termed 'vertical solitude'. **31** It is the orientational and attitudinal disconnect that managers have with officials below them. As managers rise in the hierarchy, two factors cause them to become "impersonal." First, they become increasingly aware that they are no longer managing front-line staff, but other people who think like them. Therefore, their thought patterns and worldview increasingly diverge from that of their staff Second, as they rise, the units they manage become larger until it becomes impossible to personally know everyone who works for them.<sup>32</sup> As a result, communication between the managers and general officials in an organization deteriorates. The general officials are in more regular contact with the public and client groups, but do not clearly understand the expectations of their ministers and manager who set the tone and direction in their departments.<sup>33</sup> It should be emphasized that these are intermediate measures of performance. Given the heterogeneity of organizational outputs it is only at this intermediate level that measures can be found that contribute to how effectively those heterogeneous outputs are produced in different organizations, but which are homogenous enough to permit measurement and comparability across disparate public agencies. The key feature of these measures of performance is that they refer to perceptions of past events. While the institutional environment shapes expectations of the future, performance refers to past behavior.

The conceptual framework described above proposes that performance is a function of the institutional environment. To test this, indicators were constructed that allow measurement of the degree of results focus, accountability and employee morale. It thus becomes a testable proposition that performance improves as the institutional environment improves."

- 29 See Jans and Frazer-Jans (1989) and Zussman and Jabes (1989). 30 See also Schein (1996).

- **31** See Zussman and Jabes (1989).

- 32 Schein (1996).

- 33 Zussman, and Jabes (1989).

- 34 Leadership as a determinant of organizational performance has been considered in a 1999 study of US Federal Executives (PricewaterhouseCoopers 2000) and a 1987 study of the Canadian federal public service (Zussman and Jabes 1989).

10
_2.4 Whose performance? - different units of analysis_ In addressing the key question about whose performance we are interested in, this analysis slices the public sector in three related ways. First, it examines some performance questions at the whole of government level. Second, it examines what needs to be done to get a particular group of officials to perform more effectively. Third, it asks what can be done to improve the performance of some particular agencies or organizations within the public sector. Disaggregating the public sector by type of official and by type of agency is a pragmatic judgment, and ultimately in this and the country reports the determination was made largely on the basis of custom and practice - how comrnentators refer to units or groups within government in this setting.

However, it is worth exploring briefly the challenges that lie behind this simple classification exercise.

# _Groups of officials_

The terms 'public sector employees' and 'civil service' are often used to describe anyone who is paid by government. However, this is a very heterogeneous group and in reality comprises: an inner core of administrators; members of the armed forces; officials employed by productive enterprises that are majority-owned, directly or indirectly, by the state; health workers of any level of government; primary, secondary, or university teachers paid by government; employees of sub-national governments, etc.

**Figure 3 The main components of government employment**<sup>**3 5**</sup>

<!-- Start of picture text --> TOTAL PUBLIC S E CITO R<br>rtat.okownd ote GEERm ALee ete eoltet te|rts<br>Fo uther, detil se Cross-N1,AtinlDt nGLrnn mlomn n ae,o h ol<br>em ploy-es ~ ~ ~ ~ ~ _. t P. .- r -- p -Ml<br>Bank Administrativoe an Civilto ~117Y'111111 VERNM1NSe|rvc Reforom websitel at !tolI<br>htot-/wl b--rltban_{ rglpb elicetrtstcvlevbcosh Se als Stvn (1994).|<br><!-- End of picture text -->

35 For ffirther details see Cross-National Data on Governument Employment and Wages, on the World Bank Administrative and Civil Service Reformi web site at http://wwwl.worldbank.org/publicsector/test/civilservice/cross.htm. See also Stevens **(1994).**

**11**
The surveys seek a country-specific approach to distinguishing between groups of officials. In the case of Bolivia, for example, officials from central administrations, decentralized institutions, regional administrations were surveyed and the analysis distinguished between consultants and other public officials. In Bangladesh, the analysis distinguished between officials of ministries, autonomous corporations and district-level officials. In the East Caribbean states, perceptions of managers, middle-level officials and general officials were differentiated.

# _Types of agency_

The characteristics of the goods that the agency produces, or the features of its tasks, are likely to have important effects on the way public officials behave. Some tasks naturally involve a greater level of discretion by front-line officials, while others are more amenable to strict standard operating procedures. Wilson (1989) has developed a typology of agencies according to varying characteristics along two dimensions: whether agency _outputs_ are observable or not, and whether agency _outcomes_ are observable or not. Agency outputs may not be observable simply because the agency's bureaucratic operators conduct their duties out of view of the manager (e.g., police officers on the beat, teachers in classrooms). They may also be "unobservable" because what the operators do is esoteric (e.g., a professor developing a hypothesis in a shower).

# In more formal terms, four broad categories of agency can be identified.

_Ministries_ are in a legal or constitutional sense indistinguishable from the state as they often have no specific primary legislative basis and their assets are the general property of the state. Their functions and objectives are multi-purpose complex tasks and traditionally defined by legislation (Continental European traditions) or determined incrementally by Cabinet (UK and other Anglophone traditions). Their source of funding is almost entirely the state budget and they usually have nil or very minor revenue earnings. As they have no corporate or legal identity separate from the state, they have generally no legal competence to enter into corporate contractual relationships with suppliers.

The term autonomous agencies can be quite confusing because it may refer to two very different things: "executing agencies" and "statutory commissions and independent regulators". _Executing agencies_ reporting to Ministries (or reporting to Ministers but under day to day supervision of a Ministry) are often in the same legal or constitutional sense indistinguishable from the state. However, the relevant Minister generally defines their objectives, perhaps in framework agreement, and their source of funding can include some revenue earnings. They rarely have legal competence to enter into contractual relationships with suppliers.

12
Some agencies are _statutory commissions_ in that they do have<sup>a separate legislative</sup> existence. They can be non-asset owning (legally distinct but not able to<sup>own assets in</sup> any sense) or asset owning (legally distinct and the owner of its assets). Their objectives are often enshrined in a charter and their functions are defined by<sup>the legislation,</sup> including the Minister's powers of direction. They can often be budget dependent (subvented) with significant revenue earnings. They can have legal competence<sup>to enter</sup> into contractual relationships. Such bodies tend to be established for regulatory purposes, although in some presidential systems they can also be service providing.

_Trading bodies_ also have a separate legislative existence, and can be established under commercia **l** aw in which case their charter and their articles of association<sup>define</sup> functions. They can be budget dependent, subsidized, but with significant<sup>revenue</sup> earnings. They generally have full legal competence to enter into contractual relationships - they can assume corporate liability and can be sued.

In the present series of BNPP surveys, those in Bangladesh, Bolivia,<sup>East Caribbean and</sup> Guyana covered officials from the first three types of organizations.

13
# **3. Survey methodology**

The surveys were designed to elicit the perceptions of public sector officials. All employees employed in the organizations being surveyed were eligible and no control group was included. As no 'before and after' information was sought, a cross sectional design was selected.

# _3.1 Questionnaire design_<sup>_36_</sup>

In drafting and pre-testing the questionnaires, care was taken to avoid: ambiguous words and phrases; questions asking two or more opinions simultaneously; words or phrases that could be expected to trigger off emotional responses; and information that could manipulate the respondent.<sup>37</sup> Most pre-testing exercises pointed to excessive length of the questionnaires. The rule of thumb applied was that it would be unlikely that a public official could afford to spend longer than 45 minutes with an interviewer even assuming that the surveyor could hold the official's attention for longer.

Although the responses had to be converted to numbers for analysis, verbal response scales were selected over numeric scales in most questionnaires. This happened because during pre-testing, responses on a numerical scale tended to show responses clustered close to the middle of the scale.<sup>38</sup> The advantage of using descriptive words for each category of response was that each category could be relatively well defined. The number of categories was kept low, to avoid loss of meaning in translation. The middle category was excluded to force respondents to express preferences even when they are indifferent. Pre-testing showed that including a middle category biased responses towards the middle category even when respondents had a slight preference one way or other. 'Don't know' as a response category did not appear in any survey questionnaire except in Indonesia. Thus the number of respondents expressing an opinion was biased upwards.

Filters and branches were used. Although filter questions added to the length of the questionnaire, they were used to ask whether or not something was a problem before asking the degree of the problem. Most questionnaires began with 'easy' questions like how long the respondent had been working with government or in the same position, or details about the personnel management system in place in the respondent's organization. Questions on the same theme were grouped together. Listing all categories in the same order minimized any existing order effect within questions. In most questions categories were listed from low to high, and from bad to good.

Most surveys asked about corruption and other sensitive questions such as what an official did when he or she did not agree with a supervisor's decision. Recognizing that respondents could be unwilling to admit incriminating behavior in face-to-face

> 36 The survey instruments are available on http://wwwl .worldbank.org/publicsector/test/civilservice/survey.htm. 3' **/** See Rea and Parker (1997).

> 38 See Recantini, Wallsten, and Xu (2000).

14
interviews, the first few surveys introduced a self-administered component (containing sensitive questions) besides the main in-person interview. However, this method ran into logistical problems at the data entry stage. In the later surveys, sensitive questions were asked towards the end of the interview by which time greater rapport would have developed between the surveyor and respondent. Even if this made the respondent suspicious in any way or unwilling to answer further questions, it did not bias the whole interview.

Some less direct questions were asked, such as "How serious is corruption in your organization?" instead of the more direct "What percent of officials in your organization take bribes?" Interestingly, in all surveys analyzed so far, officials admitted that their. own organization was corrupt, but less corrupt than the rest of government.

**Figure 4. Officials say that the whole government is much more corrupt than their own organization.**

<!-- Start of picture text --> 100'/<br>80V.<br>60%!<br>94% 98%<br>20%' 40 444%<br>0°/.- ___ _____<br>my organiz?.ion government my or anization government<br>Armenia Bolivia<br><!-- End of picture text -->

# _3.2 Sampling methodology_

Probability sampling was the method employed in Bangladesh, Bolivia and Guyana. Stratified random sampling was utilized with the strata being the types of organizations described in Section 2.4.3 In East Caribbean countries, quota sampling was the method chosen because the number of employees at each level was known accurately from the 1999 budget estimates available for these countries.<sup>4 0</sup> Focus groups were mostly utilized to pretest draft questionnaires.

**39** However, in Bolivia, the strata employed were not the types of organizations but the hierarchical levels in each organization. The number of different types of organizations chosen was "purposive" to maintain some representativeness, and to ensure that certain agencies were included. **40** Fink (1995).

15
Sample sizes in different countries had to be determined within the constraints of cost, time and the number of officials willing and available for interviews. Samples varied in size from 62 in St. Kitts and Nevis (population 41,000; size of public sector 3,000) to 823 in Bangladesh (population 126 million; size of public sector more than 100,000). In determining sample size, the number of subgroups to be included, and a 0.05 level of significance of the null hypothesis for the mean of the population, were both kept in mind. The sampling unit was the individual official, while the unit of analysis was an organization. In the case of East Caribbean countries, the unit of analysis was the whole public sector.

The response rate in the data sets received thus far was between 80% and 100%. There was some item non-response in several countries. For example, item non-responses were 50% in Bangladesh when officials were asked what they did when they disagreed with the opinions of their supervisors.

16
# **4. Construction of Indicators for Institutional Environment**

The survey findings provide an opportunity to enrich the factual basis of national debates on public sector reform, and to support or refute prior assertions and hypotheses in the light of the available data. Some other uses of the data require that indicators of the institutional environment and performance be constructed.

As explained in Section 2 outlining the conceptual framework, the institutional environment was measured by considering officials' expectations of the incentives and constraints that would apply in the future in terms of _rule credibility, policy credibility_ and _resource adequacy and predictability._ Indicators were constructed with rule credibility, policy credibility and resource adequacy and predictability measured in terms of these indicators on scales that ranged from 0 to 10, with 0 being the worst and 10 being the best. Questions addressed both personnel and expenditure management.

The figure below shows how responses to questions were grouped in constructing the institutional environment indicators in East Caribbean countries.<sup>4</sup> "

**Figure 5. Construction of indicators for institutional environment in East Caribbean states**

<!-- Start of picture text --> institutional environment<br>rule credibility policy credibility resource<br>adequacy &<br>predictability<br>in record policy seasonal<br>- management - consistency _ absences<br>in internal policy necessary<br>_ audit - coordination l skills<br>in performance political institutional<br>_ appraisal (of micro- _ capacity<br>individual officials) management<br>in project<br>_ evaluation<br><!-- End of picture text -->

> **41** In the East Caribbean states, public officials were surveyed in Antigua & Barbuda, Grenadines, St. Kitts & Nevis, St. Lucia and St. Vincent & Grenadines. The same questionnaire was administered in all these countries. In all settings, the rules for allocating questions were:

Does this question capture expectations of current or future constraints and incentives? If not, then it is not suitable for inclusion in an indicator of the institutional enviromnent. If it does, then does the question capture expectations of: (i) rule enforcement in relation to budget or personnel management; (ii) binding policy direction; or (iii) adequacy and predictability of financial and personnel resources?

17
Relevant and related questions were grouped under the same category as shown in the above figure. Since the responses were all on a verbal scale, they were converted to the same numerical scale. For uniformity, qualitative responses from all questions were converted into numbers on the scale of 0 to 10. For example, yes/no questions were converted into "O" and "10". For questions with four qualitative responses such as always (1), generally (2), occasionally (3), almost never or never (4), the following formula was used for the conversion:<sup>4 2</sup>

# Scaled response = 40/3 - (Un-scaled response)* 10/3

After the conversion, the following steps were taken to calculate rule credibility, policy credibility, resource adequacy and predictability, and eventually the institutional environment indicators. **43** First, the simple arithmetic average of all responses regarding the same specific aspect of the institutional environment (i.e. average of all questions in the same box in Figure 5 above) were calculated. Second, after converting responses into numbers on a 0-10 scale measures of policy consistency, policy coordination, and political micro-management were derived by taking the simple average of the aspects of the institutional environment under the same component. Finally, the institutional environment indicator for the whole public sector was derived as a simple arithmetic average of the three components of the institutional environment.

In more detail, in the East Caribbean example shown in Figure _5, rule credibility_ was measured by creating an indicator covering the existence and perceived quality of formal rules in four areas: record management, internal audit, performance appraisal, and project evaluation.' Six questions from the survey were utilized. Questions used in that and other surveys to probe for the dimensions of the institutional environment are set out in Appendix 1. Appendix 2 shows how the environment indicators were constructed for the East Caribbean states.

42 For consistency, always the positive, e.g., useful, helpful, working, responses were converted to "10," and negative, e.g., not useful, not helpful, not-working, responses were converted into "O". For example, in the question "Are job openings advertised?" "Yes" was converted into "10," and "No" was converted into "O." Similarly, in the question "How consistent are the various policies your organization?' "Very consistent" was converted into "10"; "more consisten **t** han consistent" into "6.7"; "more inconsisten **t** han consistent" into "3.3"; and "very inconsistent" into "O".

> 43 Aggregating data for indicators entails a delicate tradeoff. All data sets on public official perceptions are unreliable individually to some degree - and by aggregating several subjective measures from various sources, we can reduce this measurement error somewhat. However, the gain in reliability from aggregation comes at the expense of a loss in conceptual precision. In other words, the tradeoff is between emphasizing individual data sets that may have inaccuracies, and aggregating them to gain accuracy but with the risk that they become more blunt. This paper seeks to balance the risks involved by aggregating the data within indicators at this point, but subsequently to use the disaggregated data, albeit cautiously, in considering potential reform payoffs (see section 7.4) The risks of aggregation are in any case less severe in this approach as it entails using data from different questions but within the same survey.

> 44 Later surveys are emphasizing the degree to which the rules are structured to require contestability.

18
_Policy credibility_ was measured by creating an indicator covering three dimensions: whether policies were consistent; whether they are coordinated between the units of government e.g. ministries; and whether political interference/micro- management was felt likely. Responses from six questions were used to calculate the policy credibility indicator as described in Appendix 2.

_Resource adequacy and predictability_ was measured by creating an indicator compounded from six questions about: unpredictable seasonal absences of personnel due to severe weather conditions e.g. hurricane; anticipated supply of necessary skills; and about more general anticipated capacity. Appendix 2 provides details.

Using a similar method, an overall institutional environment indicator was also constructed on a scale of 0 to 10 (O = worst, 10 = best) by taking the simple arithmetic mean of the rule credibility, policy credibility and resource adequacy and predictability indicators. In this way, as Appendix 2 illustrates, an overall institutional environment indicator was calculated for the entire public sector of each of the East Caribbean states.

Using this approach, it is possible to measure rule credibility, policy credibility or resource adequacy and predictability _of a particular organization_ (e.g. Finance Ministry) or by _types of organizations_ (e.g. regional administrations) when sufficient data is available for each type of organization. This was possible in the Argentina, Bolivia, Guyana and Indonesia data sets, and such calculations were made.

19
# **5. Construction of Indicators for Performance**

Performance was measured in terms of results focus, accountability and employee morale by asking public officials questions that probed their perceptions of these dimensions of performance. To measure _how much_ results focus, accountability and employee morale, relevant indicators were constructed with scales which ranged from 0 to 10 - with 0 being the worst and 10 being the best.<sup>45</sup> As with the institutional environment indicators, the performance indicators drew on considerations of both personnel management and expenditure management. Although combined within the indicator, they permit disaggregated analyses to be undertaken subsequently.

**Figure 6. Construction of indicators for performance in East Caribbean states**

<!-- Start of picture text -->
|performance|<br>|results-focus aocouritability epoe<br>geared to enforceability satisfaction<br>organization's a i regulations t<br>objectives<br>up-to-date training<br>informiation w cseffetie in record an pnsmtct<br>management r evaluation<br>.n_indictor of eformance in internal in performaneo<br>audit __ appraisal<br>i aernt-based o o delega ation<br>|reward and|ll<br>|punishment|ll<br>.|to tho public<br><!-- End of picture text -->

In the East Caribbean, the degree of _results-focus_ was measured by creating an indicator cpvering: whether an organizanjob'satisatio.es were geared to its objectives; whether the organization was considered efficient; and whether a merit-based reward and punishment system was in place. Seven questions were utilized. Questions used in that and other 45 In all settings, the rules for allocating questions were: Does this question capture perceptions of past performance? If not, then it is not suitable for inclusion in an indicator of performance. If it does, then does the question capture perceptions of: (i) the degree to which a concern for results or outputs shaped behavior; (ii) rule-based behavior in relation to budget or personnel management; (iii) morale and job satisfaction. In addition, a measure of vertical solitude was constructed from the differences in perceptions concerning the environment between managers and their staff?

20
surveys to probe for the dimensions of performance are set out in Appendix 3. Appendix 4 shows how the performance indicators were constructed for the East Caribbean states.

_Accountability_ was measured by using 15 questions to create an indicator covering, inter alia, enforcement of regulations; demonstrated accountability to the public at large, and to civil society and parliament. In assessing the institutional environment, the existence of credible rules was tested in four areas: record management, project evaluation, internal audit and performance appraisal. In measuring performance, the past enforcement of these rules was tested in the same four areas.

Appendix 4 shows how the performance indicators, including that for accountability,. were constructed for the East Caribbean states.

Employee morale was measured by creating an indicator covering employee satisfaction and vertical solitude. Four questions were used to calculate satisfaction indicator as shown in Appendix 4.

Vertical solitude was defined in Section 2.3 above. The indicator for vertical solitude was calculated by first scoring managers' perception of each aspect of the institutional environment and then the general officials' perceptions of the same. Then, the (absolute value) difference of these scores in each aspect was averaged to yield an indicator of vertical solitude.46 The overall performance indicator was calculated by taking the simple arithmetic mean of the results-focus, accountability, and employee morale.

46 This formula was adopted from suggestions by Jeff Rinne of PRMPS.

21
# 6. **The Connection between Performance and Institutional Environment**

To describe and quantify the relationship between performance and institutional environment, a simple model was used. Aggregate performance and its three components, i.e., results focus, accountability for formal rules, and employee morale were regressed on institutional environment. Accordingly, performance, Yi, was modeled as a function of institutional environment (EO), and an error term ei<sup>47</sup> :

## Y= a + b Ei + ei

On current assumptions, for the Bolivian public sector there is a statistically significant positive relationship between institutional environment and performance indicators. The estimation result for Bolivia was the following:<sup>48</sup> Aggregate Performance = 0.47 + 0.74 (Institutional Environment) t-value = 5.54 R<sup>2</sup> = 0.70 Number of observations = _15_ Figure 7 is the result of cross-tabulation of institutional environment and aggregate performance indicators for 15 different public organizations in Bolivia. It shows the relationship between performance and institutional environment. Similarly, the effect of institutional environment on each performance area was tested. Strong relations existed between the institutional environment and each area of performance. The associated charts are provided in Appendix 5.

> 47 An error term was attached to capture measurement errors, random variations, as well as other determinants that have been omitted from the equation.

> 48 It is important to note that the aggregation of questions to construct the indicators is a question of judgment, although simple and explicit rules are used to minimize variations. These results may change as the country papers are prepared if the grouping of questions is altered.

22
Thus there is empirical evidence that:

- Institutions do indeed matter;<sup>49</sup> and

- Some performance areas are more susceptible to the institutional environment than others ."

**Figure** 7. **Performance and Institutional Environment in sampled Bolivian public sector organizations**

<!-- Start of picture text --> PERFORMANCE: INSTITUTIONAL ENVIRONMENT<br><!-- End of picture text --> <!-- Start of picture text --> 8 ----- - ---- __ __ __ __ __<br>y 0.7429x + 0.4709<br>R= 0.7026<br>* audit banking-regulati n<br>7<br>/ + central bank<br>06<br>A tax 4 + ine t evteleom-regulation<br>agrarian refotennstiution<br>Santa Cru2 *<br>components5ofperformance-whic agrepresentednAppendiculture<br>Te e t customs<br>labor road<br>4 .<br>5 ~~~~~6 7 8 9<br>institutional environment<br><!-- End of picture text --> 49 This assertion was based on differences in the estimated coefficients, i.e., slope, and in goodnessof-fit measures, i. e. R2, for the relationships between institutional enviromment and three different components of performance-which are presented in Appendix 5. _50_ This same relationship is being tested using data sets from the public sectors of Armenia, Bolivia, East Caribbean states, and Guyana. Results of these tests will provide more empirical evidence.

23
# 7. **Presenting Opportunities for Reforms**

To offer some options for consideration by policy-makers, four steps were taken as described below. The first step emphasizes the importance of presenting policy-makers with robust confirmation of the theory-based assertion that it is the institutional environment in general, and rule credibility, policy credibility, and resource adequacy and predictability in particular, which drive performance.

Having made this point and having attracted attention to the question of how and when that environment can be changed, the second step is to maximize access to the data to encourage policy-makers and advisers to contest possible interpretations and to suggest locally appropriate reform interventions. The contextual description of the public sector, including characteristics of respondents, their reasons for joining the public sector and the length of time worked in govemment, enrich the factual basis of these discussions.

The third step is to examine any prior assertions and topical reforms in light of the available data. In testing country-specific hypotheses, myths that have been consolidated through repetition rather than observation can be supported with evidence or refuted.

Finally, where feasible, as an encouragement to further debate, a particular heuristic approach is employed to move beyond theory into an empirical investigation of which institutions matter in particular and where might the largest "performance pay offs" be found.

# _7.1 Step 1 - it's the institutions not the people_

In all settings where the surveys have been undertaken, a summary country report sets out the evidence that supports the theory-based assertion that it is the institutional environment in general, and rule credibility, policy credibility, and resource adequacy and predictability in particular, which drive performance.

In making this point, the intention is to deter simplistic denigration of government or public officials. Although there is every reason to be concerned about poor performance and rent-seeking by some public officials, there are as many reasons to be impressed by the selflessness and determination of others working under conditions of extreme difficulty. Setting out the evidence offers an approach for understanding both good and bad performance and for presenting the results to policy makers in a format that lead to more informed choices about reform interventions.

24
**Figure 8. Rule complexity affects performance in Albania**

<!-- Start of picture text --> 10<br>9 Wocal-budget<br>con rl hr * central inst-budget<br>8<br>personnel<br>cZ7 ginlbdet<br>ional- * local-personnel<br>4; 6 *-persucntral inst-<br>i 4 -judca-<br>3 - personnel<br>2-<br>1<br>0<br>0 1 2 3 4 5<br>complexity of rules<br><!-- End of picture text --> In Figure 8 the horizontal axis shows the perception of officials of the complexity of budget management and personnel management rules in their respective administration (local governments, central institutions, regional administrations, judicial bodies and the control branch). The vertical axis shows the perception of these same officials on the enforceability of these rules. Both complexity and enforceability of rules have been ranked on a scale 0 to 10 with 0 being the worst possible, and 10 being the best possible.<sup>5</sup>

# _7.2 Step 2 - encourage contestation in interpretation_

The country reports also provide a summary of the descriptive data, presented in a format that facilitates comparisons between countries. In parallel, appropriately anonymized survey data is being placed on Internet sites. Open discussion fora are being facilitated to encourage policy makers, academics and other commentators to test alternative interpretations of the findings, to contest possible interpretations and to suggest locally appropriate reform interventions.

# _7.3 Step 3 - challenge prior assumptions_

The country reports then examine any prior assumptions or topical reforms already under consideration in the light of the available data. These prior assumptions have been derived from country dialogue with government officials and business and civil society representatives. They tend to fall into two categories. Some are assertions of facts (e.g. most civil servants would rather work in the private sector if there were jobs available)

> 51 Acknowledgments are due to Monica Das Gupta of the World Development Report 2000 who contributed to this figure.

25
and some are assertions of causal relationships (e.g. the way in which they were recruited affects the degree to which officials support government policy). Table 1 offers some examples of the assertions tested.

**Table 1 Illustrative prior assertions tested against survey evidence**

|**country**|**factual assertion**|**assertion of causal relationship**|
|---|---|---|
|Albania|there is wide variation in<br>understanding of the proposed civil<br>service law|performance is particularly highly<br>correlated with officials' support<br>for government policy|
|Eastern<br>Caribbean|managers feel that they have<br>inadequate autonomy in budgetary<br>matters|unpredictable availability of<br>personnel as a result of natural<br>disasters is damaging to<br>performance|
||records management systems are<br>unreliable|performance is undermined by the<br>lack of incentives linked to the<br>attainment of agency objectives|
||there is intense resistance to<br>regional initiatives|donor funds for technical<br>assistance lead to lack of incentives<br>and hence to poor performance|
||Public Accounts Committees are<br>not considered strong|automatic renewal of contracts<br>results in poor accountability for<br>performance|
|Bolivia|the SAFCO law is not considered<br>by officials to have improved<br>accountability|political patronage and interference<br>in day-to-day activities undermines<br>performance|

The factual assertions are supported or refuted by simple reference to the data. Some assertions of a causal relationship are sufficiently precise to allow simple testing by reference to the data. Other such assertions, particularly those that suggest that performance is in some general sense undermined by some aspect of the environment, require that some indicators of performance be drawn from the conceptual model set out in section 2.

# _7.4 Step 4 - encourage informed speculation on perfornance pay-offs_

Where feasible, as an encouragement to further debate, a particular heuristic approach is employed to move beyond theory into an empirical investigation of which institutions matter particularly and where might the largest "performance pay offs" be found. The intention of this final step is to move beyond the general finding that institutions matter for performance, and to open up consideration of which institutions matter particularly.

In making this step, three key points should be borne in mind. First, given potential problems of data quality, there is a risk of drawing over-elaborate conclusions that have a veneer of statistical robustness but are unlikely to be replicable. Second, at the general 26
level, interpretation of the survey data is strongly guided by theory. As section 2 indicates, theory gives us every reason to believe that institutions affect performance and that rule credibility, policy credibility, and resource adequacy and predictability are strongly implicated. However, theory provides no guidance on which of these institutional arrangements matter most and the findings at this disaggregated level are purely empirical. Finally, the model that has been developed is plausible but only one from very many that could have been constructed. In the country studies that offer detailed results, the model is only used to develop one set of interpretations where dialogue with government and World Bank operational staff suggest that this is useful.

The key stages in making an informed speculation on performance pay-offs are as follows:

# **Stage 1 - Develop an appropriate model**

# Model I

Performance = f (Institutional Environment).

This model has already been estimated in the previous section. It showed that predicted correlations are high when simple averages of the elements of institutional environment is regressed against aggregate performance as well as its three main components.

However, this information by itself is not enough for decision-makers in governments. They already know that performance is sometimes low because of weaknesses in the institutional environment; and if they could improve the whole institutional environment, the overall performance would improve. They need more precision. The logical next step is to move from Model I towards one in which the institutional environment is disaggregated into its components.

# Model II

# Performance = g (Rule Credibility, Policy Credibility, Resource Adequacy and Predictability).

Estimating Model II will also not give enough detailed information. Decision-makers in government will agree that improving policy credibility can improve performnance, but they will still ask what specifically they need to do to improve performance. Accordingly, a third model is required to capture the relationship between performance and particular elements of the institutional environment. This third model needs to be disaggregated enough to provide input for reform interventions. Improving the institutional environment as a method for improving performance is a practical strategy only if we can assess which particular elements of the institutional environment really matter. In other words, decision-makers are looking for reliable answers on the following questions: 27
- _Which reform efforts are most likely to succeed?_

- _Will the same intervention be equally effective across the whole public sector or even within the same sorts of organizations?_

To illustrate by returning to the analysis of the Bolivian public sector, in that context 11 components of the institutional environment were identified: six components captured rule credibility; four referred to policy credibility; and one referred to resource adequacy and predictability.

Rule credibility in the Bolivian public sector's institutional environment comprises: existence of rules in recruitment, existence of rules in evaluation, existence of rules in training, existence of rules in recording, fair treatrnent, and predictable career path. Policy credibility comprises four specific aspects of the institutional environment, i.e., policy consistency, policies communicated clearly to employees, policies supported by officials, political interference/micro-management. Just one question examined the predictability of resources.52 Accordingly, for the Bolivian public sector, a third model was written where performance is a function of eleven different components of the institutional environment.

# Model III

Performance = h (Existence rules in recruitment, Existence of rules in evaluation, Existence of rules in training, Existence of rules in recording, Fair treatment, Predictable career path, Policy consistency, Policy communication, Policy support, Political interference/micromanagement, Resource predictability) To unpack the relationship between institutional environment and performance, or in other words to quantify the relationship between performance and different potential reform efforts, an econometric model was utilized to estimate model I _.3_ Its estimated coefficients were used to derive appropriate indicators. For example, to find how much perfornance changes upon changing the institutional environment via making policies consistent, the following formula was utilized:

# _Rate of performance_ **_improvement_**

A _performancelApolicy consistency polkyComillmy performance,_ 52 In each country where BNPP surveys were administered, survey managers customized the survey instrumen **t** o country-specifi **c** oncerns, stressing some areas over others. As a result, not all surveys provided questions to capture all these aspects of the institutional environment or of performance. _53_ In a linear model, the estimated coefficient for a specific aspect of the institutional environment variable is the partial derivative of the performance with respect to that aspect of the institutional environment.

28
_where, AperformancelApolicy consistency = P foa , and apolicy consistency_ _performanceo = initial performance level._

# **Stage 2 - Examine institutional rates of return**

The illustrative equation below shows the percentage change in performance<sup>through</sup> making policies consistent. In that sense, it is the rate of return on reform<sup>effort to make</sup> policies consistent. Therefore, it is called the institutional rate<sup>of return of policy</sup> consistency (or policy consistency IRR): _IRpolicycDnsistency_ A _performance /A policy consistency performanceo_ In general, a comparatively high-value IRR for any reform effort means<sup>that the</sup> intervention is more likely to have a bigger impact on performance than another<sup>with</sup> low-value. A comparatively low value would indicate that the effort<sup>is less likely to have</sup> impact and should not be taken up if considerable results are desired. IRRs<sup>can be</sup> calculated for each of the eleven reform interventions listed above in the Bolivian examnple.

A detailed econometric model specification, the estimation method, regression<sup>results,</sup> and how the estimated coefficients were used in deriving IRRs for the Bolivian<sup>public</sup> s.ector (based on this formula which was described above) are all presented<sup>in Appendix</sup> 6. Since all the input variables are capturing institutional environment,<sup>it is theoretically</sup> possible that the input variables are correlated. For example, rule credibility<sup>could be</sup> associated with resource predictability. However, two factors ensure that multicollinearity is not a serious problem. First, as the model is offered as<sup>a heuristic</sup> device for attracting policy-makers attention to particular points of entry to<sup>reform, this is</sup> not a particular drawback provided that any correlation is positive. If policy<sup>credibility</sup> were to deteriorate as resource predictability improved, the resulting combined<sup>effect</sup> would be hard to interpret. However, in all the survey results analyzed to<sup>date the</sup> correlation where it exists has been positive. Second, multicollinearity<sup>is a sample</sup> phenomenon as well as a theoretical one. A good rule of thumb is to find variables that are theoretically relevant (for meaningful interpretation) and that are also statistically<sup>non</sup> multi-collinear (for meaningful inference). As the correlation matrix in Appendix<sup>6</sup> shows, the correlation coefficients between the explanatory variables<sup>in the Bolivia data</sup> are low.<sup>54</sup> The term "Institutional Rate of Return" has been coined deliberately to echo<sup>the more</sup> common Internal Rate of Return. Its purpose is to draw attention to the heterogeneous 54 In addition to presenting this matrix, Appendix 6 further explains how survey responses were converted to a binary mode thus lowering correlation among the explanatory variables.

29
results that are likely to be obtained from varying different elements of the institutional environment. It should not however be taken to imply greater accuracy than is actually possible at the present stage of development of this methodology. Given the imprecision of the measure, IRRs are used to rank order expected payoffs by type of intervention or change in institutional arrangements or practices, rather than to predict exact performance improvements.

# **Stage 3 - Identify anticipated performance pay offs**

As noted, IRRs are intended to draw attention to the varying results that are anticipated from different interventions, but given their probable imprecision it is the general magnitude of the pay off rather than the exact value that is relevant for policy makers.

The following table presents the potential reform interventions to improve institutional performance in the public sector of Bolivia. Only those reform interventions where IRRs were statistically significant for one-tailed test are identified. (For details, see Appendix 6.) Reform interventions with statistically significant IRRs are classified according to whether they are likely to impact on one, two or three areas of performance.

30
# **Table 2 Reform interventions and anticipated performance pay offs in the Bolivian public sector'**

|**Reform Intervention**|**Imp**|**rovement**|**in Perfor**|**mance**|
|---|---|---|---|---|
||Number<br>of areas<br>affected|Results<br>focus|Accoun<br>tability|Employee<br>Morale|
|Strengthening the credibility of rules for<br>evaluation|3|V|/|V|
|Strengthening the credibility of rules for<br>record management|3||/<br>_/||
|Ensuring that staff support government<br>policy|3||||
|Strengthening the credibility of rules for<br>training|2||/||
|<br>Preventing political interference/micro-<br>management|2||/|/|
|Assuring staff that they are treated fairly|2|/||V/|
|Strengthening the credibility of rules for<br>recruitment|1|||/|
|Making government policies consistent||1||~~~~/|

Figure 9 demonstrates how these anticipated payoffs accumulate to distinguish between the degrees to which each reform intervention is expected to impact on performance.

55 See footnote 48.

31
Figure 9. Expected impact of reforms on components of performance in Boliviae <!-- Start of picture text --> Reform Interventions affectine three nerformance areas.<br>I M results focus °accountability *employee morale<br>Reform Intervention<br>ensuring staff support for<br>government policies<br>strengthening credibility of<br>niles for record managemnent<br>strengthening credibility of<br>rules for evaluation<br>0% 10% 20% 30% 40%<br>Percentage increase in present level of performance<br>Reform Interventions affectins two performance areas.<br>assuring staff of fair<br>preventing political I<br>Reform Intervention intervention/nmcro- 15% 12%<br>moanagement<br>stengLhening credibility<br>of rules for uaining<br>0% 10% 20% 30%<br>Percentage increase in present level of performance<br>Reform Interventions affectine one Derformance area.<br>Reform I<br>Intervention strengthening credibility of<br>rules for recruitment<br>0% 3% 5%<br>Percentage increase in present level of performance<br><!-- End of picture text --> 56 **The** unit of measurement for the improvement in the performance due to a reform intervention is the percentage change in the present level of performance. For details, see Appendix A6.4.

32
Decision-makers in government having limited resources to invest in public sector reform will want to be in a position to choose interventions that will probably result in better performance. The information presented in Table 2 and Figure 9 allows them to prioritize among possible interventions. Available resources can be concentrated on those interventions that affect all three or at least two areas of performance. Bolivian decisionmakers would probably choose to strengthen rules in performance evaluation and record keeping and work to ensure that officials are committed to supporting government policies.

This same analysis can be performed for a type of organization (e.g. centralized administrations in Bolivia), a particular organization (e.g. the Ministry of Finance or a particular group of officials such as consultants). Appendix 6 presents the findings for central administrations in Bolivia.

5' For IRRs of all types of organi7ations and groups of officials, see forthcoming Bolivia country report by Yasuhiko Matsuda and the authors of this paper.

33
**8. Summary**

The public sector is a diverse environment in which to work. Although there are some general conclusions that can be drawn, staff perform better or worse in different agencies for highly diverse reasons. The Anna Karenina principle applies: "all well-performing agencies are alike; there are so many preconditions for effective performance that every dysfunctional agency is dysfunctional in its own way"." To perform well, public officials need to be confident about the future - not to the point of smugness, but certainly to the point that they can see the relationship between their efforts and any eventual outcome. The institutional environment within which they are working shapes these expectations. If the rules are not credible, with little prospect of enforcement, if they expect policies to be contradicted or resources to flow unpredictably, then they cannot envision any relationship between their effort and public sector performance. Rationally, there is little point in working purposefully as the results are so uncertain.

This paper sets out an analytical framework that was used in designing a series of surveys of the views of public officials concerning their institutional environment and in analyzing the information that was generated. It describes how the survey results help to map the strengths and weaknesses of a public sector, and offers an approach for identifying potential pay-offs from reform interventions. Above all, the framework emphasizes the heterogeneity of incentives and institutional arrangements _within_ the public sector - and the need for policy makers to have information that moves beyond generalities and indicates what is most likely to work and where.

In building on the premise that public officials' actions - and hence the performance of their organizations - depend upon the institutional environment in which they find themselves, this framework avoids any simplistic anti-government positions, without attempting a defensive justification for poor performance. Although there is every reason to be concerned about poor performance and rent-seeking by some public officials, there are as many reasons to be impressed by the selflessness and determination of others working under conditions of extreme difficulty. This framework offers an approach for understanding both good and bad performance and for presenting the results to policy makers in a format that lead to more informed choices about reform interventions.

The methodology depended on use of indicators to measure both institutional environment and performance, but the measures need to be interpreted with caution. As in the design of _any_ performance indicators, it is important to ask: who wants the indicators, who uses them and why. The reliability and the usefulness of indicators depend on the credibility of the information on which they were built, on the accuracy and comprehensiveness of the data. Finally, indicators are not the ends, they are a means for decision makers to raise questions and highlight issues for further discussion and investigation in the light of local, country-specific knowledge. Indicators are not precise 59 Tolstoy's original words were that "Happy families are all alike; every unhappy family is unhappy in its own way". Gary Reid pointed out the relevance ofthe observation for public sector performance.

34
measures, they do not provide comprehensive coverage of all areas of<sup>an individual's or</sup> agency's work, and they can not tell the whole story.

35
**Selected Bibliography**

- Aberbach, Joel D., Robert D. Putnam, and Bert A. Rockman. 1981. _Bureaucrats and Politicians in Western Democracies,_ Boston, MA.: Harvard University Press.

- Acs Z. J., and D. A. Gerlowski. 1996. _Managerial Economics._ New Jersey, Prentice Hall.

- Australian Public Service and Merit Protection Commission, 1998. _State ofthe Service Report_ 1997-9, Canberra: Public Service Commissioner.

- Axelrod, Robert, and Keohane, Robert. 1985. "Achieving Cooperation Under Anarchy: Strategies and Institutions". _World Politics._ 38-1: 226-254.

- Bendor, Jonathan, and Mookherjee, Dilip. 1987. "Institutional Structure and the Logic of Ongoing Collective Action". _The American Political Science Review._ 81-1: 129-154.

- Carter, N, R. Klein, and P. Day (1995) _How Organizations Measure Success: The Use of Performance Indicators in Government._ London: Routledge.

- Dilulio, Jr. John D. 1994. "Principled Agents: The Cultural Bases of Behavior in a Federal Government Bureaucracy." _Journal of Public Administration Research and Theory, 3:_ 277-318.

- Evans Gord and Manning Nick. 2000. "A practical approach to assessing central government policy-making institutions in cabinet government: learning lessons from recent World Bank institutional analyses". Unpublished World Bank paper, (http://wwwl.worldbank.org/publicsector/toolkits.htm)

- Fink, Arlene. 1995. _How to Sample Surveys. The Survey Kit,_ California: Sage Publications.

- Girishankar, Navin. 1999. _Reforming Institutions for Service Delivery: A frameworkfor Development Assistance with an Application to the Health, Nutrition and Population Portfolio,_ Policy Research Working Paper No. 2039, Washington D.C., The World Bank.

- Gore, Al. 1993. _Creating a Government that Works better and Costs Less: The Report of the National Performance Review._ New York: Plume.

- Greene, William H. 2000. _Econometric Analysis,_ Fourth Edition, New Jersey: Prentice Hall.

36
- Grindle, Merilee S. 1997. (Editor) _Getting Good Government: Capacity Building in the Public Sectors of Developing Countries._ Harvard Studies in International Development. Boston: Harvard Institute for International Development.

- Heyman, David B. 1988. "Input Controls in the Public Sector: What Does Economic Theory Offer?" _IMF Working Paper,_ Washington D.C.: International Monetary Fund.

- Horn, Murray J. 1995. _The Political Economy of Public Administration: Institutional Choice in the Public Sector._ New York: Cambridge University Press.

- Jans, Nick and Frazer-Jans Judy. 1989. _Careers and Culture in the SES,_ Canberra: Australian Government Publishing Service.

- Kaufmann, Daniel, Sanjay Pradhan and Randi S. Ryterman. 1998. _New Frontiers in Diagnosing and Combating Corruption._ World Bank PREM Note No. 7: Washington, DC.

- Kaufmann, Daniel, Aart Kraay and Pablo Zoido-Lobaton. 1999. _Aggregating Governance Indicators,_ and, _Governance Matters._ World Bank Policy Research Working Papers Nos. 2195 & 2196: Washington D.C. www.worldbank.org/wbi/gac.

- Klitgaard, Robert E. 1997. "Cleaning Up and Invigorating the Civil Service." _Public Administration and Development_ (U.K.) 17: 487 - 509.

- Knack, Steve and Nick Manning. 2000. _Policy attributes._ Unpublished World Bank document.

- McCubbins, Matthew D., and Thomas Schwartz. 1984. "Congressional Oversight Overlooked: Police Patrols versus Fire Alarms." _American Journal of Political Science,_ 28: 165-79.

- McCubbins, Matthew D., Roger G. Noll, and Barry R. Weingast. 1987. "Administrative Procedures as Instruments of Political Control." _Journal of Law, Economics, and Organization,_ 3: 243-77.

- . 1989. "Structure and Process, Politics and Policy: Administrative

- Arrangements and the Political Control of Agencies." _Virginia Law Review, 75:_ 431-482.

- Manning, Nick and Oliver James. 2000. _Regulation within the public sector- draft background paper._ Unpublished World Bank document.

37
- Manning, Nick, Ranjana Mukherjee, and Omer Gokcekus. March 2000. _The experience ofpublic officials in Bolivia: What works in a weak institutional environment?_ Unpublished World Bank document.

- Milgrom, Paul, and J. Roberts. 1992. _Economics, Organization, Management,_ New Jersey: Prentice Hall.

- Miller, Gary J. 1992. _Managerial Dilemmas: The Political Economy of Hierarchy._ New York: Cambridge University Press.

- Mnookin and Komhauser. 1988. "Bargaining in the Shadow of the Law: The Case of Divorce," _Yale Law Journal_ 88: 950-972

- Moe, Terry M. 1984. "New Economics of Organization." _American Journal of Political Science,_ Vol. 28, No. 4. pp. 739-777.

- . 1990. "The Politics of Structural Choice: Toward a Theory of Public

- Bureaucracy." In Oliver E. Williamson (ed.), _Organization Theory: From Chester Barnard to the Present and Beyond._ New York: Oxford University Press.

- Nunberg, Barbara, and John Nellis. 1995. Civil Service Reform and the World Bank. _World Bank Discussion Papers;_ No. 161.

- OECD, 1994a. _Public Service Pay Determination and Pay Systems in OECD Countries,_ Paris: Organization for Economic and Cooperative Development.

. 1994b. Performance Management in Government: Performance Measurement and Results- Oriented Government, _Public Management Occasional Papers_ 1994 No. 3.

- . 1999. Elements of Model Civil Service Laws for Developing Nations at

- http://www.oecd.org/puma/sigmaweb/acts/civilservice/cslaws.htm; Civil Service Legislation: Checklist on Secondary Legislation (and other Regulatory Instruments) at http: //www. oecd. org/puma/sigmaweb/pubs/ENGPDF/sp 14. pdf, both from OECD's SIGMA program.

- Osbome, David and Peter Plastrik. 1997. _Banishing Bureaucracy: The Five strategies for Reinventing Government._ New York: Plume.

- PricewaterhouseCoopers Endowment for the Business of Government. 2000. _Results of the Government Leadership Survey: A 1999 Survey of Federal Executives_ at http://www.pwcglobal.com/gx/eng/ins-sol/main/index.html

- Przeworski, Adam. 1999. "On the Design of the State: A Principal-Agent Perspective " in Luiz Carlos Bresser Pereira and Peter Spink (eds.), _Reforming the State: Managerial Public Administration in Latin America._ Boulder. Colo.: Lynne Rienner.

38
- Rea, Louis and Richard Parker. 1997. Designing and Conducting<sup>Survey Research: A</sup> Comprehensive Guide, San Francisco: Jossey-Bass Publishers.

- Recantini, Francesca, Scott Wallsten and Lixin Xu. 2000. "Surveying<sup>Surveys and</sup> Questioning questions: Learning from the World Bank Experience".<sup>_World Bank_</sup> _Policy Research Working Paper_ No. 2307, Washington D.C.: The<sup>World Bank.</sup>

- Seabright, Paul. 1993. "Managing Local Commons: theoretical issues in incentive design". _The Journal of Economic Perspectives._ 7-4: 113-134.

- Schein, Edgar H. 1996. "Three Cultures of Management: The key to organizational Learning" in _Sloan Management Review,_ Fall 1996.

- Selznick, Philip. 1957. _Leadership in Administration: A Sociological_<sup>_Interpretation,_</sup> University of California Press (1984, reprint version).

- Simon, Herbert A., "Organizations and Markets," _Journal of Economic_<sup>_Perspectives,_</sup> Vol. 5, No. 2 (Spring 1991): 25-44

- Stevens, Mike 1994. _Preparingfor Civil Service pay and Employment:_<sup>_a Primer in_</sup> _Rehabilitating Government: Pay and Employment Reform in Africa_ ed.<sup>David L.</sup> Lindauer and Barbara Nunberg, Washington D.C.: The World Bank.

- Shand, D. 1997. _The Role ofperformance Indicators in Public Expenditure_<sup>_Management,_</sup> Unpublished IMF document.

- Simon, Herbert A. 1974. _Administrative Behavior: A study of decision-making_<sup>_processes_</sup> _in administrative organization,_ New York: Free Press.

- . 1991. Organizations and Markets. _Journal of Economic Perspectives,_<sup>_Vol._</sup>

- 5, No. 2 (Spring 1991): 25-44.

- Tendler, Judith. 1997. _Good government in the tropics._ Baltimore, Md.: Johns<sup>Hopkins</sup> University Press.

- Weingast, Barry R. and Mark J. Moran. 1983. "Bureaucratic Discretion or Congressional Control? Regulatory Policymaking by the Federal Trade Commission"<sup>in</sup><sup>_Journal_</sup> _of Political Economy_ (U. S.); 91:765-800.

- Williamson, Oliver E. 1975. _Markets and Hierarchies, analysis and antitrust implications: a study in the economics of internal organization,_ New<sup>York: Free</sup> Press.

- Wilson, Jamnes Q., 1989. _Bureaucracy: what government agencies_<sup>_do and why they do it,_</sup> New York: Basic Books.

39
World Bank, 1994. _Governance: the World Bank's Experience._ Washington D.C.: The World Bank.

. 1998. _New Frontiers in Diagnosing and Combating Corruption,_ PREM Notes Number 7, October 1998. Washington D.C., The World Bank.

- . 1999. Report No. 19211 _Civil Service Reform A Review of World Bank_

- _Assistance_ April 27, 1999 Operations Evaluation Department.

. 2000. Reforming Public Institutions and Strengthening Governance: A. World Bank Strategy. http://www-wbweb.worldbank.org/prem/prmps

- Zussman, David and Jabes, Jak. 1989. _The Vertical Solitude: Managing in the Public Sector,_ Halifax: The Institute for Research on Public Policy

40
# **Appendices**

_Appendix 1: Illustrative questions to probefor dimensions of the institutional environment_<sup>_9_</sup> The test applied in grouping these questions was that these questions capture expectations of current or future constraints and incentives - and thus the degree to which officials are discounting the future and management is maintaining its reputation in relation to:

1. rule enforcement in relation to budget or personnel management;

2. binding policy direction; 3. adequacy and predictability of financial and personnel resources.

## _Rule Credibility_

_a. personnel management rules_

- Within an organization, different people are hired for different reasons. This survey is interested in learning about general hiring trends. In your organization would you say that employees were hired more often for: 1. good performance at entry

2. test qualifications 3. met job requirements 4. personal connections 5. political connections 6. family status in community 7. payment of gift to some 8. public authorities 9. knew a senior official (Guyana) Does your organization have a system where annual objectives/ targets that are measurable are agreed to between emnployees and supervisors at the beginning of every year? (yes/no) (Guyana) What is your assessment of the existing formal, written guidelines related to personnel mnanagement? Please tell us the extent to which you agree with the following statements. Answer on a scale of 1 to 4, where 1 corresponds to 'strongly disagree' and 4 corresponds to 'strongly agree' <!-- Start of picture text --> (Guyana)<br><!-- End of picture text -->

1. relatively simple to understand and implement 2. relatively flexible, providing our organization with a lot of discretion in their implementation 3. helpful for effective personnel management Now, we will ask you a series of questions about all types of payments in addition to basic salary. Please answer for each of the following types of payments. This type of payment is distributed on the basis of 1. Clear rules 2. Rules that require/allow for some discretion 3. Full discretion, not on the basis of any rules (i) public transportation allowance (ii) food allowance

- (iii) holiday payment (K-15), (iv) payment for overtime work,

- (v) bonuses for innovation and rationalization, (vi) membership in Boards of Managers

(vii) other bonuses or premia <!-- Start of picture text --> (Macedonia)<br><!-- End of picture text --> <!-- Start of picture text --> (Macedonia)<br><!-- End of picture text --> 59 The questions themselves were developed by individual survey task managers and they may have drawn from several different sources in preparing them; The sources are not all known and thus exact citations can not be provided for each question.

41
<!-- Start of picture text --> Are positions like yours generally advertised to the public? (yes/no)<br>(Bolivia)<br>In your organization, would you say that the employees are hired more often for:<br>1. Performance on an examination<br>2. Personal connections<br>3. Political connections<br>4. Famnily relations<br>5. Payment or gift to some public authorities<br>6. Good CV<br>(Bolivia)<br>Does your organization have Individual Annual Operations Plans (POAI) for its employees? (yes/no)<br>(Bolivia)<br>Can an employee appeal a dismissal? (yes/no)<br>(Bolivia)<br>Does your organization have a Human Resource Management Infornation System (HMIS) containing records of<br>employees? (yes/no)<br>(East Caribbean States)<br>As far as you know, does your organization have a stated performance appraisal system? (yes/no)<br>(East Caribbean States)<br>Are the criteria for appraisal known to you? (yes/no)<br>(East Caribbean States)<br>b. budget management rules<br>For each of the following aspects of budget mnanagement, does your organization have formal, written guidelines<br>describing how these functions are to be canied out. If so, are they carried out in full, in part, or not at all? Please<br>answer on a scale of I to 4, where 1 conresponds to "not at all" and 4 corresponds to "yes, in full"<br>0 budget preparation<br>0 procurement<br>0 budget execution<br>O audit<br>0 production of fnal end-of-year accounts<br>(Macedonia)<br>What is your assessment of the existing formal, written guidelines related to budget managernent? Please tell us the<br>extent to which you agree with the following statements. Answer on a scale of I to 4, where I corresponds to 'strongly<br>disagree' and 4 corresponds to 'strongly agree'<br>* relatively simple to understand<br>* relatively flexible<br>* helpful for effective and efficient planning of programs/activities<br>(Macedonia)<br>Are there rules in place, which govem security, movernent, and storage of financial records?<br>(Macedonia)<br>Does your organization have an intemal audit function? (yes/no)<br>(East Caribbean States)<br>c. general rules<br>Are there rules in place, which govem security, movement, and storage of records? (yes/no)<br>(Macedonia)<br>Is there a fornal system of evaluating programs/ projects being implemented by your organization? (yes/no)<br>(East Caribbean States)<br>Policy Credibility<br>Agencies receive instructions and policies from different parts of the government, some comnplement each other, others<br>conflict. Overall, how do you see the relationship between these various instructions/ policies?<br>I. very consistent<br>2. more consistent than inconsistent<br><!-- End of picture text --> 42
|3.<br>|more inconsistent than consistent<br>|
|---|---|
|4.|very inconsistent<br>|
||(Guyana)|
|Ho|w are changes in policy usually conmmunicated to you?|
|I.|Written correspondence|
|2.|Orally|
|3.|staff meetings|
|4.|management|
|5.|not at all|
||(Guyana)|
|Ho<br>|w often do you disagree with the policies that your organization is<br>|
|ask|ed to implement?|
|1.|Often|
|2.|Occasionally|
|3.|Rarely|
|4.|Not at all|
||(Guyana)|
|In y|our opinion, how would you rate the incidents of political interference that exists in your organization?|
|1.|very frequently|
|2.|frequently|
|3.|infrequently|
|4.|Almost never|
||(Guyana)|
|Do|you believe your budget institution has a clear set of strategic objectives? (yes/no)|
|(No<br>faci|te to interviewer examnple of a clear strategic objective might be: (a) for an agency regulating drug safety, to<br>litate the introduction of safe medicines within the domestic market; (b) for a ministry of education, to build a labor|
|for|ce with the skills and adaptability required for a modern economy<br>(Macedonia)|
|Wh|at are the reasons for imperfect implementation of formal, written policies, guidelines and regulations related to|
|per|sonnel management?|
|An|d among these potential reasons what are the two most important reasons?|
|<br>1.|,<br>the guidelines are overruled by other directives within our organization or from other government units|
|2.|<br>contradictory among themselves|
|3.|<br>guidelines are not known or disseminated<br>(Macedonia)|
|Wh<br>|at are the reasons you believe account for imperfect implementation of formal, written guidelines related to budget<br>|
|ma|nagement?|
|An|d arong these potential reasons, what are the two most imnportant reasons?|
|1.|the guidelines are overruled by other directives within our organization or from other government units|
|2.|contradictory among themselves|
|3.|<br>Guidelines are not known or disseminated<br>(Macedonia)|
|Ag|encies receive instructions and policies from different parts (of government), some of which are incompatible with|
|eac|h other. How compatible (consistent) are the instructions and policies which your agency receives with each other|
|1.|Very Consistent|
|2.|Consistent|
|3|Inconsistent|
|.<br>4.|Very Inconsistent<br>(Bolivia)|
|In y<br>int|our view, which of these measures would contribute to improving the performance of your organization?<br>er alia...|

(Bolivia) Protection from political interference (Bolivia) Which of these measures do you believe can be successfully implemented successfully in your organization? inter alia Protection from political interference (Bolivia) 43
How strongly do you agree with the following statement which is about the public administration in general not about specifically your organization.

I =strongly disagree 4=strongly agree

[for Ministers/Deputy-Ministers only]

- The process of formulating the national budget involves close consultation among members of the Cabinet. (1) (2) (3) (4)

The process of formulating the national budget involves close consultation between the Ministry of Finance and the line ministries.

(1) (2) (3) (4) The process of formulating municipal budgets involving close consultation with the Ministry of Finance (1) (2) (3) (4) The process of formulating the organizational budgets involves close consultation between the budget managers and department/division managers .

(1) (2) (3) (4)

|(Bolivia)<br>ob, are methods<br>(Bolivia)<br>king of public sector<br>problem of political|
|---|
|(East Caribbean States)<br>s it would pursue if|
|(East Caribbean States)|

If decrees, executive orders, or policies interfere with the ability of your organization to do its-job, are methods available to have them changed?

Political rnicro-management or interference by political leaders and ministers in day to day working of public sector entities can sometimes hamper the efficiency of these organizations. In your organization, is the problem of political micro-management

1. Serious 2. Present, but not serious 3. Not present 4. Helping your organization perform better

- Before an election, does each party clearly outline in its election manifesto the national priorities it would pursue if voted to power! (yes/no)

- After a govenmment is formed, to what extent are policies formulated within the previously announced priorities I. Most ofthe time

2. Some time 3. A few time 4. Very rarely (East Caribbean States)

# _Resource Adequacy and Predictability a. budget resources_

- Do you think that actual funds approved for next year's budget will exceed or fall short of budgeted requirements? 1. exceed

2. fall short 3. other [specify] In what way do you expect that actual funds will differ from budgeted Funds? I. a large amount 2. a moderate amount 3. a small amount 4. other Do you expect that actual funds for your organization will differ from budgeted funds this year? (yes/no) (Guyana) (Guyana) (Macedonia)

- Do you expect that actual funds will differ from budgeted funds by a large amount or a small amount? (1) (2) (3) (4)

- 1=a small amount; 4=a large amount

|(Macedonia)|
|---|

How typical, in your experience, is it for actual funds to differ from budgeted funds? Would you say that it is rare, not typical, fairly typical very typical?

- (1) (2) (3) (4) 1 =rare; 4=very typical

|(Macedonia)|
|---|

44
To what extent does the public administration and your organization have adequate financial resources to provide an appropriate level of service? Please answer on a scale of 1 to 5, where 1 corresponds to 'completely inadequate' and<sup>5</sup> corresponds to 'completely adequate'.

Are the following resources in short supply for your organization? Please also identify the two resources in most short supply.

Number of staff, skills of staff, computers, office supplies (excluding computers), other goods<sup>and services, mnanagerial</sup> support, other (specify) (Macedonia) Here is a list of attributes which people mention for an organization to work efficiently. If your organization lacks any of them, write it down in your list, please. inter alia...

Disbursement of budget resources without delay (Bolivia)

# _b. personnel resources_

How many weeks per year are lost from regular duties because you have to work on emergencies like natural<sup>disaster?</sup> 0 1-2 3-4 More than 4 (East Caribbean States) Some people consider that emigration of educated and trained persons from the Caribbean to US, lUK and Canada drains the public sector of skilled persons. How severe is this problem in your organization? 1. Very severe 2. Severe 3. Moderate 4. Mild (East Caribbean States) 45
_Appendix 2: Institutional environment indicators in the East Caribbean States_<sup>_6 0_</sup>

# **Statistical Table 1 Rule credibility indicator and components (East Caribbean)**

||Record<br>managemen|Internal<br>t audit|Performance<br>appraisal|<br>Project<br>evaluation|Rule<br>Credibility|
|---|---|---|---|---|---|
|Antigua and Barbuda|8.4|3.5|4.3|3.7|_5.0_|
|Grenada|8.8|3.6|9.3|6.1|6.9|
|St. Kitts and Nevis|7.1|6.5|5.6|_5.0_|6.1|
|St. Lucia|8.6|5.5|8.1|4.1|6.6|
|St. Vincent And Grenadines|8.9|6.9|4.2|_5.1_|6.3|
|East Caribbean|8.6|5.2|6.6|4.7|6.3|

# **Statistical Table 2 Policy credibility indicator and components (East Caribbean)**

||Consistency|Policy<br>coordination|Micro-<br>management|Policy<br>Credibility|
|---|---|---|---|---|
|Antigua and Barbuda|4.1|_5.3_|4.2|4.5|
|Grenada|3.7|5.4|3.8|4.3|
|St.Kitts andNevis|7.3|6.9|3.3|5.8|
|St. Lucia|6.0|6.4|3.5|5.3|
|St. Vincent And Grenadines|4.2|_5.1_|3.8|4.4|
|East Caribbean|4.8|5.7|3.7|4.8|

# **Statistical Table 3 Indicator of resource adequacy and predictability and components (East Caribbean)**

||Lost man-day<br>from natural<br>disasters|s skills<br>available|Institutional<br>capacity|Resource<br>adequacy and<br>predictability|
|---|---|---|---|---|
|Antigua and Barbuda|4.1|5.5|7.7|_5.8_|
|Grenada|3.9|7.4|7.9|6.4|
|St. Kitts and Nevis|5.3|5.9|7.4|6.2|
|St. Lucia|5.3|6.5|6.2|6.0|
|St. Vincent And Grenadines|5.3|6.1|6.3|5.9|
|East Caribbean|4.5|6.3|7.1|6.0|

60 See footnote 48.

46
Statistical Table 4 Indicator of overall institutional environment and components (East Caribbean)

||Rule<br>Credibility<br>from Statistica<br>Table I|Policy<br>Credibility<br>l from Statistica<br>Table 2|Resource<br>predictability<br>l from Statistica<br>Table 3|Overall<br>Institutional<br>l Environment|
|---|---|---|---|---|
|Antigua and Barbuda|5.0|4.5|5.8|5.1|
|Grenada|6.9|4.3|6.4|5.9|
|St. Kitts and Nevis|6.1|5.8|6.2|6.0|
|St. Lucia|6.6|5.3|6.0|5.9|
|St. Vincent And Grenadimes|6.3|4.4|5.9|_5.5_|
|East Caribbean|6.3|4.8|6.0|5.7|

47
_Appendix 3. Illustrative questions to probefor dimensions ofperformance_<sup>_6_</sup> _'_ Test applied:

# These question capture perceptions of past performance and of

1. the degree to which a concern for results or outputs shaped behavior,

2. rule-based behavior in relation to budget or personnel management; 3. morale and job satisfaction.

# _Results Focus_

Are there documented work / performance standards for each function in your organization? (yes/no) Are they generally achieved?

(Guyana)

|In<br>0 to|your opinion, to what extent would you say that your organization has realized its mnission?<br>25%|
|---|---|
|26<br>51|% to 50%<br>% to 75%|
|76|% to 100%<br>|
|In t|(<br>he past year have staff in your organization been rewarded for any of the following reasons?|
||1.<br>Taking prompt action on customers / Clients/ end users' requests or complaints|
||2.<br>Providing outstanding service|
||3.<br>Improving qualifications|
||4.<br>Excellent attendance record|
||_5._<br>Consistent courtesy|
|Ov|erall, do you think these rewards were justified?|
||1.<br>always|
||2.<br>generally|
||3.<br>occasionally|
||4.<br>almost never|
||5.<br>never|
|Acr|(<br>oss individuals within your budget institution, all types of bonuses and similar payments are|
||**I.**<br>very unequally distributed|
||2.<br>unequally distributed|
||3.<br>equally distributed|
||4.<br>distributed on the basis of performance<br>(Mac|
|In t<br>per|he past year, has anyone in your organization been the subject of the following types of sanctions for**poor**<br>formance or unprofessional conduct?|
|1.|Dismissal|
|2.|Demotion|
|_3._|Salary decrease for a specified length of time|
|4.|Suspension of duties with withdrawal of salary|
|_5._|<br>Postponement of promotion or pay increases|
|6.|Public censure and reprimand|
|7.|Warning|
|8.|Initiating criminal proceedings|

<!-- Start of picture text --> (Guyana)<br><!-- End of picture text --> (Guyana) (Macedonia)

|(Macedonia)|
|---|

> 61 The questions themselves were developed by individual survey task managers and they may have drawn from several different sources in preparing them. The sources are not all known and thus exact citations can not be provided for each question.

48
|What were the causes of the disciplinary actions?<br>**1**<br>**f**||
|---|---|
|**.**<br>**Poor work perormance**<br>2<br>Unauthorized absence||
|.<br> <br>3.<br>Working on a second job during official work hours||
|<br>4.<br>Soliciting or accepting bribe||
|5<br>Embezzlement||
|.<br><br>6.<br>Conflict with supervisor||
||(Macedonia)|
|During the past two years, has your organization received any sanctions or measur<br>|es for not performing well?|
|If yes, which of the following mneasures?||
|1<br>Our budget was reduced||
|.<br> <br>2.<br>Our budget was increased||
|<br>3.<br>Responsible staff was disciplined||
|4.<br>We received a warning<br> <br>||
|5.<br>other (specify)||
||(Macedonia)|
|In the past year, have staff in your organization been recognized/rewarded for any<br> <br>|of the following reasons?|
|1.<br>Taking prornpt action on consurer requests or complaints||
|2.<br>Providing outstanding service||
|3.<br>Other||
|4.<br>No recognition/reward||
||(Bolivia)|
|How many employees of your organization with excellent performance have not r|eceived recognition/reward?|
|<br>1.<br>Many||
|2.<br>Some<br>3.<br>A few||
|4.<br>No one has received recognition||
||(Bolivia)|
|Overall, do you think that these rewards were justified?||
|1.<br>Always<br> <br>||
|2.<br>Generally||
|3.<br>Occasionally||
|4.<br>Almost never or never||
|At the end of the year, are physical outputs of your organization measured and ev|(Bolivia)<br>aluated against the expenditure|
|incurred?<br> <br>||
|1.<br>Most of the time||
|2.<br>Frequently<br> <br>||
|3.<br>A few tirnes||
|4.<br>Very rarely||
||(East Caribbean States)|
|In the past year, how mnany ermployees in your organization received?||
|Reward or recognition for good performance?||
|*<br>0%||
|*<br>about 10%||
|*<br>about 25%||
|*<br>about 50%||
|Some kind of penalty for non-performance?||
|<br>*<br>0%||
|*<br>about 10%||
|<br> <br>*<br>about 25%||
|*<br>about 50%||
||(East Caribbean States)|
|In your performance evaluation, how much importance is laid on your achievemn<br>**I.**<br>a lot|<br>ent of your targets|
|2.<br>some<br> <br>||
|3.<br>a little<br>4<br>not at all||
|.<br>|(East Caribbean States)|

49
# _Accountability_

<!-- Start of picture text --> When you first joined the government, were you assured that you would get the job? (yes/no)<br>(Guyana)<br>Before you applied for your current job were you assured that you would get the job? (yes/no)<br>(Guyana)<br>When was your performance last appraised by your supervisor?<br>I . last quarter<br>2. mid year<br>3. end of first quarter<br>4. one year ago<br>5. two years ago<br>6. three years ago<br>7. more than three years ago<br>8. performance not appraised<br>(Guyana)<br>How difficult is it to obtain adequate and timely information from the records management system in your<br>organization?<br>[ very difficult D difficult D very easy D easy<br>(Guyana)<br>In your opinion, to what extent is corruption a significant problem in the public sector as a whole?<br>1. very significant<br>2. significant<br>3. somewhat significant<br>4. not significant at all<br>(Guyana)<br>How often are cases of corruption, generally, reported to the proper authorities?<br>1. Almost always<br>2. Frequently<br>3. Occasionally<br>4. Almost never<br>(Guyana)<br>In the past year, has anyone in your organization been the subject of the following types of sanctions for poor<br>performance or unprofessional conduct?<br>1. Dismissal<br>2. Demotion<br>3. Salary decrease for a specified length of time<br>4. Suspension of duties with withdrawal of salary<br>5. Postponement of promotion or pay increases<br>6. Public censure and reprimand<br>7. Warning<br>8. Initiating criminal proceedings<br>(Macedonia)<br>What were the causes of the disciplinary actions?<br>I . Poor work performance<br>2. Unauthorized absence<br>3. Working on a second job during official work hours<br>4. Soliciting or accepting bribe<br>5. Embezzlement<br>6. Conflict with supervisor<br>(Macedonia)<br>For each of the following aspects of budget management, does your organization have formal, written guidelines<br>describing how these functions are to be carried out. If so, are they carried out in full, in part, or not at all? Please<br>answer on a scale of 1 to 4, where 1 corresponds to "not at all" and 4 corresponds to "yes, in full".<br>(Macedonia)<br>Is there a problem of finding important financial records in your organization?<br>(1) (2) (3) (4) (5)<br>1 =extremely problematic 5==no problem at all<br>(Macedonia)<br><!-- End of picture text --> _50_
Please evaluate the following opposing statements about practice in monitoring commitments and mnanaging aggregate expenditure.

In your organization, a system for monitoring and controlling the total stock of expenditure commrritments incurred is * not present * present * present and moderately effective * present and highly effective In your organization the total stock of expenditure commitments is monitored frequently throughout the year and controls are in place to ensure that total commitments do not exceed total budget authorization (1) (2) (3) (4) (1) totally disagree (4) totally agree (Macedgnia) How easy is it to find necessary records in your department? Please answer on a scale from 1 to 4, where 1 corresponds to "extremely difficulf' and 4 corresponds to "very easy". (Macedonia) In general, cases of cormption are always, frequently, occasionally or never reported to proper authorities? 1. Always 2. Frequently 3. Occasionally 4. Almost never or never

||(Macedgnia)|
|---|---|
|How easy is it to find necessary records in your department? Please answer on a scal<br>corresponds to "extremely difficulf' and 4 corresponds to "very easy".<br>In general, cases of cormption are always, frequently, occasionally or never reported<br>1.<br>Always<br>2.<br>Frequently|e from 1 to 4, where 1<br>(Macedonia)<br>to proper authorities?|
|3.<br>Occasionally<br>4.<br>Almost never or never|(Bolivia)|
|Would you say that, in your organization, corruption is a||
|1.<br>very serious problem,||
|2.<br>a serious problem,||
|3.<br>a somewhat serious problem, or||
|4.<br>not a serious problem at all?||
||(Bolivia)|
|How difficult is to obtain information from those records?||
|1. Very difficult||
|2.<br>Difficult||
|3.<br>Easy||
|4.<br>Very easy||
||(Bolivia)|
|How reliable is the information contained in your organization'sHRMIS?||
|1.<br>Very reliable||
|2.<br>Reliable||
|3.<br>Not reliable||
|4.<br>Completely unreliable||
||(East Caribbean States)|
|Does your organization's HRMIS contain the following?||
|1 .<br>Updated records of all training you attended||
|2.<br>The dates when you got increment or moved from one scale to another||
|3.<br>Perfornance evaluation reports||
|4.<br>The date you will retire from service||
||(East Caribbean States)|
|When were your own records last updated on HRMIS?||
|1 .<br>In 2000||
|2.<br>In 1999||
|3.<br>In 1998||
|4.<br>Do not know when the records were updated||
||(East Caribbean States)|
|How easy or difficult is to obtain for official purposes records of employees in your|organization?|
|<br>I .<br>Very easy||
|<br>2.<br>Easy||
|3.<br>Difficult||
|4.<br>Very difficult||
||(East Caribbean States)|

51
<!-- Start of picture text --> Have you ever met with associations / groups of citizens who are the users, consumers of the services produced by<br>your organization? (yes/no)<br>(East Caribbean States)<br>Are you obliged to share official infonnation, especially that related to performance of your organization, with them?<br>(yes/no)<br><!-- End of picture text --> <!-- Start of picture text --> (East Caribbean States)<br><!-- End of picture text --> <!-- Start of picture text --> (East Caribbean States)<br><!-- End of picture text -->

- Is your organization's audit report required to be presented to parliament? (yes/no) If yes, how effective is the internal audit in ensuring compliance with internal control systems? 1. Very effective. 2. Effective. 3. Partially effective. 4. Ineffective How long ago was your last appraisal conducted? 1. Less than 6 months ago 2. Less than a year ago 3. More than a year ago 4. I have never been appmised

<!-- Start of picture text --> (East Caribbean States)<br><!-- End of picture text --> <!-- Start of picture text --> (East Caribbean States)<br><!-- End of picture text --> <!-- Start of picture text --> (East Caribbean States)<br><!-- End of picture text -->

# _Employee Morale_<sup>_62_</sup>

|Is your|salary sufficient to meet your living expenses? (yes/no)<br>(Guyana)|
|---|---|
|Would<br>I.<br>2.<br>3.|you describe the quality of life that you have maintained over the past year as:<br>Unacceptable<br>Acceptable<br>Good<br>(Guyana|
|During<br>a means<br>streamli|periods of economic difficulty and austerity, some people talk about decreasing the size of the government as<br>of saving money. Decreasing the size of the government would, in sorne areas, entail organization<br>ning and staff cutbacks. Given that, how worried are you about losing your job sometime in the near future?|
|1.|Very Worried|
|2.|Somewhat Worried|
|3.|Not At All Worried<br>(Guyana|
|How oft|en do you receive salary or wages late?|
|1.|very often|
|2.|a few times|
|3.|sornetimes|
|4.|no time at all<br>(Guyana|
|How wo<br>1.|rried are you about losing yourjob some time in the near future?<br>very worried|
|2.|<br>somewhat worried|
|3.|not very worried|
|4.|not at all worried|

<!-- Start of picture text --> (Guyana)<br>(Guyana)<br><!-- End of picture text --> <!-- Start of picture text --> (Guyana)<br><!-- End of picture text --> <!-- Start of picture text --> (Guyana)<br><!-- End of picture text -->

|(Macedonia)|
|---|

|In what year did you receive your last promotion?|
|---|
|When do you expect to be considered for your next promotion?|
|*<br>Don't know|
|*<br>Don't expect to be considered for promotion<br>(Macedonia|
|Would you describe the quality of life that you have raintained over the past year as unacceptable, acceptable, good<br>or excellent?|

(Macedonia) (Macedonia) 62 To further assess performance, **a** measure **of vertical solitude** was **also constructed from the** differences in perceptions conceming **the** environment **between managers and their staff.** 52
Are your earnings from your job in the public administration (including base salary and other payments) sufficient to pay your living expenses? (yes/no)

|(Bolivia)|
|---|

- How probable is it that in the near future you will become unemployed? 1. Very probable

2. Somewhat probable 3. Not very probable 4. Not at all probable Do you feel that you are contributing to your organization's performance? (yes/no) (Bolivia) (East Caribbean States) Do you enjoy your work? (yes/no) (East Caribbean States)

- How probable is it that in the near future you will lose your job? 1. Very likely 2. Likely 3. Unlikely 4. Very unlikely

|(East Caribbean States)|
|---|

53
# _Appendix 4: Performance indicators in the East Caribbean States_<sup>_63_</sup>

# **Statistical Table** 5 **Results focus indicator and components (East Caribbean)**

||Geared to<br>organization's<br>objectives|Efficiency|merit based reward R<br>& punishment|esults focus|
|---|---|---|---|---|
|Antigua and Barbuda|6.0|3.6|0.7|3.6|
|Grenada|6.7|4.9|1.7|5.1|
|St. Kitts and Nevis|7.4|5.4|2.0|5.2|
|St. Lucia|6.9|4.1|1.3|_5.0_|
|St. Vincent And Grenadines|6.9|4.5|1.2|5.2|
|EastCaribbean|.~~~~~~~~~~~~~~~~~~~~<br>6.8|~~~~~~~~~~~~~~~~~~~~~<br>4.4|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>1.3|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>4.9|

# **Statistical Table 6 Accountability indicator and components (East Caribbean)**

||Enforceme|nt of regul|ations in|_<br>_<br>**CD**<br>18-~<br>~<br>|A^<br>**0**<br>~<br>O<br>8<br>'~~~~~|<br>w<br>|Accounta<br>~~8|bility to<br>e~<br>g"|Accounta<br>~~~~~~~~~~~~bility<br>~~<br>from<br>previous<br>columns|
|---|---|---|---|---|---|---|---|---|---|
|Antigua and<br>Barbuda|5.4|7.7|4.6|5.6|5.8|5.0|4.4|6.2|5.4|
|Grenada|4.7|8.1|6.0|7.4|6.5|4.5|4.6|7.3|5.7|
|St. Kitts and<br>Nevis|6.1|9.2|7.4|5.7|7.1|5.9|4.6|7.3|6.2|
|St. Lucia|_5.1_|7.9|_5.5_|6.5|6.3|5.9|4.4|7.2|5.9|
|St. Vincent<br>And<br>Grenadines|4.8|7.6|5.4|4.5|5.6|_5.5_|4.1|7.4|5.7|
|East Caribbean|<br>_5.1_|8.0|5.7|6.1|6.2|5.3|4.4|6.2|5.5|

# **Statistical Table** 7 **Employee morale indicator and components (East Caribbean)**

|Country|Employee morale|
|---|---|
|Antigua and Barbuda|8.0|
|Grenada|7.8|
|St. Kitts and Nevis|8.5|
|St. Lucia|7.6|
|St. Vincent And Grenadines|8.2|
|East Caribbean region|8.0|

63 See footnote 48.

_54_
# **Statistical Table 8 Overall performance indicator and components (East Caribbean)**

||Results-focus<br>from Statistical<br>Table 5|Accountability<br>from Statistical<br>Table 6|Employee Morale<br>from Statistical<br>Table 7|Performance|
|---|---|---|---|---|
|Antigua and Barbuda|3.6|5.4|8.0|5.7|
|Grenada|_5.1_|5.7|7.8|6.2|
|St. Kitts and Nevis|5.2|6.2|8.5|6.6|
|St. Lucia|_5.0_|5.9|7.6|6.2|
|St. Vincent And Grenadines|5.2|_5.7_|8.2|6.4|
|East Caribbean|4.9|5.5|8.0|6.1|

No measure of vertical solitude was calculated<sup>in the East Caribbean. The very</sup> considerable differences in the perceptions of managers<sup>prevented the establishment of</sup> any single reference point against which other<sup>staff s perceptions could be tested.</sup> However, in other country reports where indicators<sup>could be calculated at organizational</sup> level, an indicator of vertical solitude was constructed.

_55_
# _Appendix 5: Connection between performance and institutional environment'_

Figure 10. Effect of institutional environment on results focus in Bolivia <!-- Start of picture text --> 7<br>result focus =0.54 19 IE + 0.7997<br>R 2 = 0.2853<br>6 ____________________ bW king-regula ion<br>tax<br>5 ;c<br>] + ~~~~~~~~Santa Cruz central bank<br>;l jzglaria a~~~~~cu reform<br>4 __iinance __tele rn-regulation<br>* road<br>* agriculture<br>3 __Chuquisac i<br>5 6 7 8 9<br>institutional environment<br><!-- End of picture text -->

> 64 See footnote 48.

_56_
Figure 11. Effect of institutional environment on accountability in Bolivia <!-- Start of picture text --> 9<br>accou itability = 1.043 IE - 1.0788<br>R2 = 0.741<br>*audit<br>t central bank<br>7 . ;~~~~~~~~~~~~~~~ tele DM-regu atien<br>6<br>5 +7f~~~~~saci t labor<br>* customs<br>5 6 7 8 9<br>institutitonal environment<br><!-- End of picture text --> 57
Figure 12. Effect of institutional environment on employee morale in Bolivia <!-- Start of picture text --> 8<br>mnwale = 0.6631 IE + 1.5892<br>R2 = 0.7606<br>biking-regula on<br>7<br>. ine / t + c~~~~~~~~~~~~~~rentral bank<br>4 tax _m-regulation */e telec<br>institutiomsl envion<br>/ + Santa Cnuz<br>5 6 a 7 8i 9orm<br>Chuguisa L<br>4 .... ....<br>5 6 7 8 9<br>institutional environment<br><!-- End of picture text --> 58
_Appendix 6: Econometric and analytical issues_ This appendix has five sections. Section A 6.1<sup>presents the econometric models. Section</sup> A 6.2 discusses the data and estimation problems<sup>and describes how these problems were</sup> tackled. Section A 6.3 includes the estimation results<sup>for the entire Bolivian public</sup> sector. Section A 6.4 briefly explains the construction<sup>of the IRRs for Bolivia. Section</sup> A6.5 has the IRR tables for centralized organizations.

# A 6.1. Econometric models

Econometric Model I:

**Yij=** a + b E +ei,

where Y = performance indicator, with j = aggregate performance,<sup>accountability,</sup> employee morale, and results focus; IE - institutional environment; and e = a well-behaved stochastic error term.

A linear functional form was chosen to keep the<sup>analysis simple.</sup> This model was estimated by using data points from<sup>16 Bolivian organizations. The</sup> estimation results are presented in Section 6 and in Appendix<sup>3.</sup>

# Econometric Model III:

11 where, Yj = performance-j, with j = aggregate performance, accountability<sup>(ACC),</sup> employee morale (MOR), and results-focus (RFO); X k= component-k of the institutional environment,<sup>with k=l,2, ..., 11; and</sup> **uj=** a classical error term.

In particular, xl = Existence of rules in recruitment x2 = Existence of rules in evaluation x3 = Existence of rules in training x4 = Existence of rules in recording x5 = Fair treatment 59
x6 = Predictable career path x7 = Policy consistency x8 = Policy communication x9 = Policy support xlO = Political interference/micro-management xl 1 = Resource predictability

# A 6.2. Dealing with data and estimation problems: the example of Bolivia

# _Number ofobservations_

Having 1 1 independent variables (and a constant term), and only 15 organizations, i.e., data points was a problem. In order to calculate IRRs for different organizations and different groups of public officials, responses from each public official were utilized in running regressions. That basically gave (after some data cleaning) 690 observation points for the entire Bolivia; and 240 for central administration, and so on.

# _Collinearity among independent variables_

Rather than doing nothing or changing the functional form, explanatory variables were transformed into binary responses: an indicator which is less than 5 out of 10 points were converted into zero, otherwise into one. This transformation also allowed the interpretation of the estimated coefficients in a more sensible way. Instead of referring to increasing policy credibility by one or by a certain amount, it was possible to refer to making policies credible. Similarly, instead of referring to two public officials who believe that rules in place 72.4% or 39.8%, it is possible to refer to distinguish between public officials who believe and don't believe respectively that rules are in place.

**Correlation coefficients for the (converted) institutional environment indicators (Bolivia)**

||_xl_|_x2_|_x3_|_x4_|_x5_|_x6_|_x7_|_x8_|_x9_|_xIO_|_xlI_|
|---|---|---|---|---|---|---|---|---|---|---|---|
|xl|1.00|||||||||||
|x2|0.13|1.00||||||||||
|x3|0.05|0.03|1.00|||||||||
|x4|0.09|0.05|0.05|1.00||||||||
|x5|0.04|0.10|0.07|0.05|1.00|||||||
|x6|0.21|0.08|0.09|0.12|0.20|1.00||||||
|x7|0.12|0.06|0.03|0.07|0.03|0.14|1.00|||||
|x8|-0.09|0.00|0.00|-0.03|0.01|-0.03|-0.02|1.00||||
|x9|0.11|0.05|0.13|0.09|0.06|0.19|0.13|-0.04|1.00|||
|xlO|0.31|0.10|0.08|0.12|0.14|0.47|0.09|-0.07|0.23|1.00||
|xl1|0.14|-0.01|0.07|0.03|0.03|0.10|0.13|-0.04|0.03|0.09|1.00|

60
_Correlation among the error terms of different performanice regression equations_ In the left hand side of Model III, there are three specific performnance indicators, namely Results-focus (RF) Accountability (ACC), and Morale (MOR) as a function of eleven different aspects of the institutional environment (iE). In other words, we are saying that much of the systematic component of these three different performances will consist of the same type of variation-variation due to the aspects of the institutional environment. But, it is also likely because of the matching across public officials that random components of the performances will be related. Thus, some knowledge of the random component of RF variations would give us some predictive insight into the random component of ACC variations. Consequently, it is reasonable to expect that the errors of these three regression equations--RF= a(IE), ACC= v(IE), MOR= c(IE)--may be correlated across equations. However, when the errors are correlated across the equations, OLS results in biased variance estimates. Therefore, to improve efficiency, Seemingly Unrelated Regression Equations (SURE) approach was used in the estimations. If the errors of the equations were not correlated, then SURE results and OLS results became the same.

61
A 6.3 SURE Estimation results for the Bolivian public sector

|Estimates for equation:|**Results-focus**|||N=690|
|---|---|---|---|---|
|Variable<br>Coefficient<br>Sta|ndard Error|z=b/s.e.|P[IZI_z]|Mean of X|
|Xi<br>0.1843148|0.23181|0.795|<br>0.42654|<br>0.4725|
|X2<br>0.6025640|0.27018|2.230|<br>0.02573|<br>0.2072|
|X3<br>-0.7502945E-02|0.27814|-0.027|<br>0.97848|<br>0.1913|
|X4<br>0.5985908|0.27651|2.165|<br>0.03040|<br>0.8058|
|X5<br>0.3981092|0.22865|1.741|<br>0.08165|<br>0.6232|
|X6<br>-0.2858045|0.26078|-1.096|<br>0.27309|<br>0.6420|
|X7<br>-0.5335056E-01|0.31301|-0.170|<br>0.86466|<br>0.8536|
|X8<br>-0.1540673|0.31894|-0.483|<br>0.62905|<br>0.8667|
|X9<br>0.7550640|0.24320|3.105|<br>0.00191|<br>0.6870|
|X1o<br>0.5296796E-02|0.25926|0.020|<br>0.98370|<br>0.4130|
|X1i<br>-0.3912180E-01|0.26093|-0.150|<br>0.88082|<br>0.7681|
|Constant<br>3.411840|0.50980|6.692|<br>0.00000||
|Model test: F[ 11,<br>678|]=<br>2.52,|<br>Prob v|alue=<br>|0.00421|

|Estimates for equatio|n:**Accountability**||||
|---|---|---|---|---|
|Variable<br>Coefficient|Standard Error|z=b/s.e.|P[IZI_z]|Mean of X|
|Xi<br>0.3022615E-|01<br>0.12985|0.233|<br>0.81593|<br>0.4725|
|X2<br>0.6108728|0.15134|4.036|<br>0.00005|<br>0.2072|
|X3<br>0.4409731|0.15580|2.830|<br>0.00465|<br>0.1913|
|X4<br>0.7030247|0.15489|4.539|<br>0.00001|<br>0.8058|
|X5<br>0.1116405|0.12808|0.872|<br>0.38338|<br>0.6232|
|X6<br>0.5904624|0.14607|4.042|<br>0.00005|<br>0.6420|
|X7<br>0.1345138|0.17533|0.767|<br>0.44296|<br>0.8536|
|X8<br>-0.2312646|0.17865|-1.294|<br>0.19550|<br>0.8667|
|X9<br>0.3926405|0.13623|2.882|<br>0.00395|<br>0.6870|
|X10<br>0.9710046|0.14522|6.686|<br>0.00000|<br>0.4130|
|X1i<br>0.1574662|0.14616|1.077|<br>0.28133|<br>0.7681|
|Constant<br>4.051039|0.28556|14.186|<br>0.00000||
|Model test: F[ 11,|678] =<br>22.58,|<br>Prob v|alue=<br>0.|00000|

|Estimates|for equation:|**Employee Morale**||||
|---|---|---|---|---|---|
|Variable|Coefficient<br>|Standard Error|z=b/s.e.|P[|Z|_z]|Mean of X|
|Xi|0.2087146|0.70065E-01|2.979|0.00289|0.4725|
|X2|0.3327698|0.81663E-01|4.075|0.00005|0.2072|
|X3<br>|-0.6659126E-01|0.84071E-01|-0.792|0.42831|0.1913|
|X4|0.4759933|0.83577E-01|5.695|0.00000|0.8058|
|X5|1.095540|0.69109E-01|15.852|0.00000|0.6232|
|X6|0.4592120|0.78821E-01|5.826|0.00000|0.6420|
|X7|0.1867915|0.94608E-01|1.974|0.04834|0.8536|
|X8<br>|-0.8713628E-01|0.96401E-01|-0.904|0.36605|0.8667|
|X9|0.4057072|0.73509E-01|5.519|0.00000|0.6870|
|X1o|0.1470672|0.78362E-01|1.877|0.06055|0.4130|
|X1i|0.2771287E-02|0.78869E-01|0.035|0.97197|0.7681|
|Constant|4.136974|0.15409|26.848|0.00000||
|Modeltest:|F[_11,_<br>67|8]<br>54.36,|Prob va|lue=<br>0|.00000|

62
# A 6.4 Constructing the IRR indicators

The following is a demonstration of how an IRR was constructed by using the estimated coefficients-which were reported in the previous section.

Based on the following formula, which was explained in section 7, particularly for policy consistency, two pieces of informnation were required to construct a specific IRR.

# A _performance / Apolicy consistency_

_IRRpolicyconsistency performance_ For instance, to construct Results-focus IRRs

- (1) we need estimated partial derivatives of the results focus with respect to particular potential reform interventions-i.e., the coefficient of the related variable from Model III. Yet, for statistical significance of the constructed IRR, only the estimated coefficients, which were statistically significant for one-tailed test, were used in the construction of the indicators.

- (2) we need the initial level of the results-focus indicator.

As the reported regression results show, four of the estimated coefficients were statistically significant for one-tailed test. The initial level of the result focus-indicator was 4.481. Thus, the IRRs were constructed as shown in the following table.

# **Statistical Table 9 Calculating the IRRs (Bolivia)**

|_Institutional_<br>_environmentk_|8_(results focus)_<br>_(_<br>_a(institutional_<br>_environment,,)_|_Resultsfocuso_|_Results focus-_<br>_IRRk_|
|---|---|---|---|
|Rules in evaluation|0.603|4.481|13.4%|
|Rules inrecording|0.599|4.481|13.4%|
|Fairtreatment|0.398|4.481|8.9%|
|Policy support|0.755|4.481|16.9%|

63
A6.5 Reform interventions and improvements in performance: Bolivia's Central Administrations<sup>65</sup>

**Table 3 Reform Interventions and anticipated performance pay offs in Bolivia Central Administrations**

|**Reform Intervention**|**Imp**|**rovement**|**in Perfor**|**mance**|
|---|---|---|---|---|
||Number<br>of areas<br>affected|Results<br>focus|Accoun<br>tability|Employee<br>Morale|
|Ensuring resource predictability|3|V|_V_||
|Ensuring that staff support government<br>policy|3|/|**_V/_**|**_V/_**|
|Strengthening the credibility of rules for<br>record management|3|/|/|/|
|Strengthening the credibility of rules for<br>evaluation|3|/||**_V_**|
|Preventing political interference/micro-<br>management|2|**_V/_**|**_V_**||
|Strengthening the credibility of rules for<br>training|1||**_V_**||

Figure 13 then demonstrates how these anticipated payoffs accumulate to distinguish between the degrees to which each reform intervention is expected to impact on performance.

> 65 See footnote 48.

64
**Figure 13. Expected impact of reforms on different components of performance in Bolivia Central Administrations'**

<!-- Start of picture text --> Reform Interventions imyactin2 three performance areas.<br>* results focus 3accountabilit employee morale<br>ensuring resource<br>predictability<br>ensuring staff support<br>for government 8%<br>Reform policies<br>Intervent'sVengthening<br>credibility of rules for I14% T 8%<br>record management<br>strengthening<br>credibility of rules for 10%i<br>evaluation l<br>. ~ ~ ~ III ~ ~ ~~I<br>0% 10% 20% 30% 40% 50%<br>Percentage increase in present level of performance<br>Reform Interventions imnactine one or two performance areas.<br>Reform<br>Intervention l<br>strengtheiing credibility<br>of rules for training 9<br>preventing political<br>intervention/micro- fvJ 'ILI15%K LI 20%<br>management<br>0% 10% 20% 30% 40%<br>Percentage increase in present level of performance<br><!-- End of picture text --> 66 As explained above, the unit of measurement for the improvement in the performance due to a reform intervention is the percentage change in the present level of performance.

65
**Policy Research Working Paper Series**

|**Title**<br>|**Author**|**Date**|**Contact**<br>**for paper**|
|---|---|---|---|
|WPS2403 Can the World Cut Poverty in Half?<br>How Policy Reform and Effective<br>Aid Can Meet International<br>Development Goals|Paul Collier<br>David Dollar|July 2000|E. Khine<br>37471|
|WPS2404 The Distribution of Mexico's Public<br>Spending on Education|Gladys Lopez-Acevedo<br>Angel Salinas|July 2000|M. Geller<br>85155|
|WPS2405 Marginal Willingness to Pay for<br>Education and the Determinants of<br>Enrollment in Mexico|Gladys Lopez-Acevedo<br>Angel Salinas|July 2000|M. Geller<br>85155|
|WPS2406 How Mexico's Financial Crisis<br>Affected Income Distribution|Gladys Lopez-Acevedo<br>Angel Salinas|July 2000|M. Geller<br>85155|
|WPS2407 Utility Privatization and the Needs of<br>the Poor in Latin America: Have We<br>Learned Enough to Get It Right?|Antonio<sup>Estache</sup><br>Andres Gomez-Lobo<br>Danny Leipziger|August 2000|G. Chenet-Smith<br>36370|
|WPS2408 What Makes Banks Special? A Study<br>of Banking, Finance, and Economic<br>Development|Biagio Bossone|August 2000|E. Mekhova<br>85984|
|WPS2409 How Did the World's Poorest Fare<br>in the 1990s?|Shaohua Chen<br>Martin Ravallion|August 2000|P. Sader<br>MC3-556|
|WPS2410 Is Functional Literacy a Prerequisite<br>for Entering the Labor Market?<br>An Analysis of Determinants of Adult<br>Literacy and Earnings in Ghana|Niels-Hugo<sup>Blunch</sup><br>Dorte Verner|August 2000|H. Vargas<br>37871|
|WPS2411 Natural Openness and Good<br>Government|Shang-Jin Wei|August 2000|H. Sladovich<br>37698|
|WPS2412 Urbanization without Growth:<br>A Not-So-Uncommon<br>Phenomenon|Marianne Fay<br>Charlotte Opal|August 2000|A. Abuzeid<br>33348|
|WPS2413 Foreign Direct Investment in Services<br>and the Domestic Market for Expertis|James Markusen<br>e Thomas F. Rutherford<br>David Tarr|August 2000|L. Tabada<br>36896|
|WPS2414 Pension Reform and Capital Market<br>Development: "Feasibility" and<br>'Impact" Preconditions|Dimitri Vittas|August 2000|A. Yaptenco<br>31823|
|WPS2415 Infrastructure Restructuring and<br>Regulation: Building a Base for<br>Sustainable Growth|Ian Alexander<br>Antonio Estache|August 2000|M. Salehi<br>37157|
**Policy Research Working Paper Series**

|**Title**|**Author**|**Date**|**Contact**<br>**for paper**|
|---|---|---|---|
|WPS2416 The Swiss Multi-Pillar Pension<br>System: Triumph of Common Sense?|Monika Queisser<br>Dimitri Vittas|August 2000|A. Yaptenco<br>31823|
|WPS2417 The Indirect Approach|David Ellerman|August2000|B. Mekuria<br>82756|
|WPS2418 Polarization, Politics, and Property<br>Rights: Links between Inequality<br>and Growth|Philip Keefer<br>Stephen Knack|August 2000|P. Sintim-Aboagye<br>37644|
|WPS2419 The Savings Collapse during the<br>Transition in Eastern Europe|Cevdet Denizer<br>Holger C. Wolf|August 2000|I. Partola<br>35759|
|WPS2420 Public versus Private Ownership:<br>The Current State of the Debate|Mary Shirley<br>Patrick Walsh|August 2000|Z. Kranzer<br>38526|
|WPS2421 Contractual Savings or Stock Market<br>Development-Which Leads?|Mario Catalan<br>Gregorio Impavido<br>Alberto R. Musalem|August 2000|P. Braxton<br>32720|
|WPS2422 Private Provision of a Public Good:<br>Social Capital and Solid Waste<br>Management in Dhaka, Bangladesh|Sheoli Pargal<br>Daniel Gilligan<br>Mainul Huq|August 2000|S. Pargal<br>81951|
|WPS2423 Financial Structure and Economic<br>Development: Firm, Industry, and<br>Country Evidence|Thorsten Beck<br>Ashl Demirguc,-Kunt<br>Ross Levine<br>Vojislav Maksimovic|August 2000|K. Labrie<br>31001|
|WPS2424 Global Transmission of Interest<br>Rates: Monetary Independence and<br>the Currency Regime|Jeffrey Frankel<br>Sergio Schmukler<br>Luis Serven|August 2000|E. Khine<br>37471|
|WPS2425 Are Returns to Investment Lower for<br>the Poor? Human and Physical<br>Capital Interactions in Rural Vietnam|Dominique van de Walle|August 2000|H. Sladovich<br>37698|
|WPS2426 Commodity Price Uncertainty in<br>Developing Countries|Jan Dehn|August 2000|P. Varangis<br>33852|