metadata: authors: 0: author: Hasnain,Zahid
1: author: Manning, Nick
2: author: Pierskalla,Jan Henryk
count: World
docna: 0: docna: Performance-related pay in the public sector
            : a review of theory and evidence
docty: Policy Research Working Paper
subtopic: Health Care Services Industry,Labor Markets,Administrative & Civil Service Reform,Public Sector Administrative & Civil
            Service Reform,Democratic Government,Public Sector Administrative and Civil
            Service Reform,De Facto Governments
teratopic: Private Sector Development,Macroeconomics and Economic Growth,Social Protections and Labor,Education
abstracts: cdata!: The objective of this paper is to
            provide a review of the theoretical and, in particular,
            empirical literature on performance-related pay in the
            public sector spanning the fields of public administration,
            psychology, economics, education, and health with the aim of
            distilling useful lessons for policy-makers in developing
            countries. This study to our knowledge is the first that
            aims to disaggregate the available evidence by: (i) the
            quality of the empirical study; (ii) the different public
            sector contexts, in particular the different types of public
            sector jobs; and (iii) geographical context (developing
            country or OECD settings). The paper's main findings,
            based on a comprehensive review of 110 studies of public
            sector and relevant private sector jobs are as follows.
            First, we find that overall a majority (65 of 110) of
            studies find a positive effect of performance-related pay,
            with higher quality empirical studies (68 of the 110)
            generally more positive in their findings (46 of the 68).
            These show that explicit performance standards linked to
            some form of bonus pay can improve, at times dramatically,
            desired service outcomes. Second, however, these more
            rigorous studies are overwhelmingly for jobs where the
            outputs or outcomes are more readily observable, such as
            teaching, health care, and revenue collection (66 of the
            68). There is insufficient evidence, positive or negative,
            of the effect of performance-related pay in organizational
            contexts that that are similar to that of the core civil
            service, characterized by task complexity and the difficulty
            of measuring outcomes, to reach a generalized conclusion
            concerning such reforms. Third, while some of these studies
            have shown that performance-related pay can work even in the
            most dysfunctional bureaucracies in developing countries,
            there are too few cases to draw firm conclusions. Fourth,
            several observational studies identify problems with
            unintended consequences and gaming of the incentive scheme,
            although it is unclear whether the gaming results in an
            overall decline in productivity compared to the
            counterfactual. Finally, few studies follow up
            performance-related pay effects over a long period of time,
            leaving the possibility that the positive findings may be
            due to Hawthorne Effects, and that gaming behavior may
            increase over time as employees become more familiar with
            the scheme and learn to manipulate it.
repnme: repnme: Performance-related pay in the public
            sector : a review of theory and evidence
display_title: Performance-related pay in the
            public sector : a review of theory and evidence
listing_relative_url: /research/2012/04/16235004/performance-related-pay-public-sector-review-theory-evidence
topicv3: Education, Skills Development and Labor
            Market m1326135 1484,Individual Performance Management m1326767 2058,Public Administration m1327723 723,Jobs m1503931 2670,Health m1504661 3480
publishtoextweb_dt: 2012-04-01T00:00:00Z
historic_topic: Private Sector Development,Macroeconomics and Economic Growth,Social Protections and Labor,Education
no_of_pages: 57
author_keys: 0: author_key: 167706
1: author_key: 1161816
model_extractions: WPS6043 Policy Research Working Paper 6043 Performance-related Pay in the Public Sector A Review of Theory and Evidence _Zahid Hasnain Nick Manning Jan Henryk Pierskalla_ The World Bank East Asia and Pacific Region Poverty Reduction & Economic Management Sector Department Poverty Reduction and Economic Management Network Public Sector Governance April 2012
Policy Research Working Paper 6043

# **Abstract**

The objective of this paper is to provide a review of the theoretical and, in particular, empirical literature on performance-related pay in the public sector spanning the fields of public administration, psychology, economics, education, and health with the aim of distilling useful lessons for policy-makers in developing countries. This study to our knowledge is the first that aims to disaggregate the available evidence by: (i) the quality of the empirical study; (ii) the different public sector contexts, in particular the different types of public sector jobs; and (iii) geographical context (developing country or OECD settings). The paper’s main findings, based on a comprehensive review of 110 studies of public sector and relevant private sector jobs are as follows. First, we find that overall a majority (65 of 110) of studies find a positive effect of performance-related pay, with higher quality empirical studies (68 of the 110) generally more positive in their findings (46 of the 68). These show that explicit performance standards linked to some form of bonus pay can improve, at times dramatically, desired service outcomes. Second, however, these more rigorous studies are overwhelmingly for jobs where the outputs or outcomes are more readily observable, such as teaching, health care, and revenue collection (66 of the 68). There is insufficient evidence, positive or negative, of the effect of performance-related pay in organizational contexts that that are similar to that of the core civil service, characterized by task complexity and the difficulty of measuring outcomes, to reach a generalized conclusion concerning such reforms. Third, while some of these studies have shown that performance-related pay can work even in the most dysfunctional bureaucracies in developing countries, there are too few cases to draw firm conclusions. Fourth, several observational studies identify problems with unintended consequences and gaming of the incentive scheme, although it is unclear whether the gaming results in an overall decline in productivity compared to the counterfactual. Finally, few studies follow up performance-related pay effects over a long period of time, leaving the possibility that the positive findings may be due to Hawthorne Effects, and that gaming behavior may increase over time as employees become more familiar with the scheme and learn to manipulate it.

This paper is a product of the Poverty Reduction & Economic Management Sector Department, East Asia an Pacific Region; and the Public Sector Governance and Poverty Reduction and Economic Management Network. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at zhasnain@worldbank.org and nmanning@worldbank.org _The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Performance-related Pay in the Public Sector: A Review of Theory and Evidence**

Zahid Hasnain<sup>1</sup> , Nick Manning<sup>2</sup> , and Jan Henryk Pierskalla<sup>3</sup>

## **JEL Codes: H11, H83, I18, I28**

_Acknowledgements_ : This paper has benefitted greatly from comments by Mike Stevens, Willy McCourt, Mariano Lafuente, Gary Reid, and Svetlana Proskurovska.

> 1 World Bank (zhasnain@worldbank.org)

> 2 World Bank (nmanning@worldbank.org)

> 3 Duke University (jhp5@duke.edu)
# **Contents**

|1. Introduction .......................................................................................................................................... 1|
|---|
|2. Theoretical debates ............................................................................................................................... 4|
|Expectancy and reinforcement theory ............................................................................................ 4|
|Incentive and principal-agent theory .............................................................................................. 5|
|Behavioral economics —intrinsic versus extrinsic motivation ...................................................... 7|
|3. Organizing the empirical evidence: ―Craft‖ and ―Coping‖ jobs ........................................................... 9|
|Methodological approaches .......................................................................................................... 11|
|4. The empirical literature reviewed ....................................................................................................... 13|
|Observational studies ................................................................................................................... 13|
|Public sector coping jobs ....................................................................................................... 13|
|Public sector craft jobs ........................................................................................................... 14|
|_Tax administration, job placement ................................................................................... 14_|
|_Teaching............................................................................................................................ 16_|
|_Health care jobs ................................................................................................................ 17_|
|Private sector: Craft or coping jobs ....................................................................................... 19|
|Experimental studies .................................................................................................................... 21|
|Meta-studies ........................................................................................................................... 21|
|Laboratory experiments ......................................................................................................... 22|
|Public sector craft jobs ........................................................................................................... 23|
|_Tax administration, job placement ................................................................................... 23_|
|_Teaching............................................................................................................................ 24_|
|_Health sector ..................................................................................................................... 25_|
|Private sector: Craft or coping jobs ....................................................................................... 26|
|5. Assessing the evidence ....................................................................................................................... 26|
|6. Summary ............................................................................................................................................. 28|
|Appendix A: List of empirical PRP studies reviewed ............................................................................ 33|
|Appendix B: List of High Quality Studies of Craft and Coping Jobs .................................................... 37|
|References .............................................................................................................................................. 40|

|**Figures**|
|---|
|Figure 1: Aggregate findings on performance-related pay........................................................ 27|
|Figure 2: Findings by internal and external validity............................................................... 27|
|Figure 3: Findings by job type........................................................................................ 28|
|Figure 4: Findings for craft and coping tasks by research quality and country context....................... 29|

|**Tables**|
|---|
|Table 1: James Q. Wilson‘s classification of job type ................................................................................ 11|
|Table 2: Studies by country environment, methodology, and job type....................................................... 13|
|<br>Table 3: Findings of high quality craft and coping studies by sector and country context ......................... 29|

# **Boxes**

|Box 1: The elements of pay flexibility.......................................................................................................... 3|
|---|
# **1. Introduction**

Performance-related pay (PRP) has been introduced in many countries as a possible tool for improving the productivity and accountability of the public sector. Over the past fifteen years, a majority of OECD countries have implemented PRP in the central administration (core civil service), in specialized entities such as revenue administration, and for key service delivery staff such as teachers and medical personnel. Middle income countries, and to some extent low income countries, perhaps drawing on the OECD example, have also experimented with PRP in an attempt to inject performance-orientation in otherwise dysfunctional bureaucracies. A vast theoretical and empirical literature has analyzed various dimensions of PRP, and there is now a small but growing body of robust evidence on the impact of PRP that is shedding new light on what is achievable and under what specific conditions.

The objective of this paper is to provide a review of the theoretical and in particular empirical literature on PRP in the public sector spanning the fields of public administration, psychology, economics, education, and health with the aim of distilling useful lessons for policy-makers in developing countries. This is by no means the first comprehensive review of this literature; but it is, to our knowledge, the first that aims to disaggregate the available evidence by (i) the quality of the empirical study; (ii) the different public sector contexts, in particular the different types of public sector jobs; and (iii) geographical context (developing country or OECD settings). The intention in so doing is to ensure that the findings from the empirical literature are appropriately nuanced.

PRP is a compensation arrangement in which the final salary of an employee is a function of some form of measured ―performance‖, where how performance is measured, who measures it, and how it is linked to salary can all vary considerably and are key aspects of the design of the scheme. Performance can be based on qualitative assessments or quantitative measures of inputs (effort allocation, attendance, voluntary contributions at the workplace and skills acquisition), outputs (completion of pre-agreed tasks or number of clients/cases served), or outcomes (student test scores, official client evaluations, service utilization rates or revenue creation). Salary can either be wholly a function of performance, for example as piece-rate pay in a manufacturing setting or commission-based salary in a sales environment, or a combination of base pay and one-off bonuses or merit increases of base pay. Bonuses and merit increases can be awarded on an individual, small team or larger departmental basis. Evaluations can be implemented by direct supervisors, human resource specialists, peer panels or outside agencies. Once performance has been measured, it has to be evaluated against a performance standard. This standard can be based on individually pre-agreed goals, absolute performance against minimum or scaled standards, relative improvements against past performance, rank-order of performance in a tournament evaluation or relative performance measured against co-workers, other teams of co-workers, other schools or agencies nationally or regionally.

PRP in the form of bonuses or merit increases to basic pay has been used more frequently in the OECD in recent years. According to one estimate, approximately two-thirds of OECD countries have introduced PRP in some form or the other (OECD 2005). The United Kingdom, Switzerland and the Czech Republic apply PRP more extensively than countries such as New Zealand, Austria and the Netherlands. In Finland, for example, the proportion of basic salary that PRP can represent can amount to over 40% of the total. In the US, while PRP is of limited use in the core public administration, it is at the center of efforts to improve teacher accountability over the past decade in the context of the No Child Left Behind Act. The literature suggests that there are similar movements underway in middle income countries and, perhaps more sporadically, in lower income countries where PRP is more often referred to in the health and education sectors than in the core administration.

1
Drawing on contract theory and the problems of moral hazard, a significant chunk of the academic literature has examined the impact of PRP on increasing effort and reducing shirking. In situations where effort is unobservable, fixed pay contracts provide little ability for employers to influence employee effort. This is especially likely to be the case in traditional civil service jobs characterized by uniform pay for jobs in similar grades, pay increases based largely on seniority, and negligible probability of termination. Contracts that tie observable outputs, which are correlated with unobservable effort, to desired pay incentives can mitigate such problems. Some of the literature has attempted to move beyond effort and to introduce the concept of ―engagement‖ as a measure of an employee‘s emotional and intellectual commitment to their employing organization and its success, encompassing both commitment (―I like working here‖) and organizational citizenship (―I am prepared to go the extra mile‖), with the question then being what impact can PRP have on staff engagement. Contract theory also suggests that PRP can help address the problem of adverse selection by encouraging high ability individuals who will do better under a performance pay scheme to join the agency and similarly discouraging low ability individuals (the ―sorting‖ effect).

Critics counter that PRP does not work when tasks are multi-dimensional as it results in ―gaming‖ behavior whereby effort is only allocated towards what is observed and measured which may not improve overall outcomes. A huge literature has examined the challenges in effectively designing these schemes, particularly in public sector contexts where tasks are complex and outcomes cannot be easily measured. The psychological and behavioral economics literature in addition has argued that individuals are also motivated by intrinsic concerns about the inherent social value of the job — particularly in the public sector — and that PRP which explicitly focuses on extrinsic benefits might crowd-out intrinsic motivation thereby reducing worker productivity.

In reviewing the empirical evidence, 153 studies in total, the paper is explicit in focusing largely on job types that are relevant to the public sector and characterized by task complexity and unobservability of effort thereby excluding studies that examine PRP for simple manufacturing jobs or other repeatable tasks. These jobs, borrowing James Q. Wilson‘s typology, are classified as ―craft‖ and ―coping‖ jobs. This narrows the list to 110 studies. Reviewing these, the paper draws the following conclusions. First, a majority (93 of the 153) of all studies find some positive effect of PRP, and a majority (65 of 110) of studies of craft and coping jobs show positive findings. Limiting the analysis to craft and coping also shows while early case studies found largely inconclusive evidence on the impact of PRP on staff morale, effort, and productivity, work over the last 10 to 15 years that has been based on more systematic observational studies and experimental evaluations in the laboratory and in the field have generally found that explicit performance standards linked to some form of bonus pay can improve, at times dramatically, desired service outcomes. 46 of the 68 high quality studies of craft and coping jobs showed a positive effect of PRP.

Second, however, these more rigorous studies are usually for jobs where the outputs or outcomes are more readily observable, such as revenue collection, teaching and health care. There is simply not enough robust evidence, positive or negative, of the effect of PRP in organizational contexts that that are similar to that of the core civil service, characterized by task complexity and the difficulty of measuring outcomes to reach a generalized conclusion for reform. To be specific, of the 68 high quality studies, only 2 were for contexts similar to core civil service jobs. Third, while some of these studies have shown that PRP can work even in the most dysfunctional bureaucracies, there is limited evidence in developing country contexts (only 10 high quality studies), with considerable discretionary as opposed to rule-based behavior and with significant politicization that can negatively affect the overall credibility and legitimacy of the incentive scheme. Fourth, several observational studies identify problems with unintended consequences and gaming of the incentive scheme, although it is unclear whether the gaming 2
results in an overall decline in productivity compared to the counterfactual. Finally, few studies follow up PRP effects over a long period of time, leaving the possibility that the positive findings may be due to Hawthorne Effects<sup>4</sup> , and that gaming behavior may increase over time as employees become more familiar with the PRP scheme and learn to manipulate it.

This literature review focuses almost entirely on the individual incentive effects of PRP as this is what the bulk of the literature has emphasized. It should be noted that there are, at least in the policy literature, potential agency-level and public sector wide effects of PRP that have to date been underexplored in the academic literature. PRP can provide a mechanism for conveying to staff the increasing expectations of agency performance, in effect changing the agency-level culture of performance (―this is how hard we work around here‖) through the multitude of individual performance appraisal discussions (Marsden 2004). PRP can also have wider public sector impacts on the fiscal sustainability of the wage bill, public sector pay competitiveness, and social objectives, such as gender equity. PRP may contribute to cost containment by limiting pay increases to less costly performance bonuses (Marsden and French 1998). Some studies have also suggested that PRP may have a gender dimension as men are more likely to see the arrangement as fair and reasonable. Finally, PRP in the OECD countries been introduced in the context of other changes in pay policy, in particular moves towards differentiated pay arrangements whereby pay for similar jobs varies across government agencies and occupational groups, and the delegation of pay setting authority from central human resource ministries to line ministries and agencies (see Box 1). What remains unexplored are the potential linkages between these reforms and whether or not PRP caused or was the effect of these other changes in pay policy.

# **Box 1: The elements of pay flexibility**

OECD countries have move towards flexible pay arrangements in the public sector which is in essence some combination of three key features, or design elements:

- **Performance-related pay:** Enabling pay to differ for civil servants doing the same job by linking a portion of the civil servants‘ pay to individual or group effort or performance with performance targets set either at the individual level and/or the group level.

- **Differentiation:** Pay differences within and across government ministries, departments, and agencies for civil servants doing the same job - based on the need to attract and retain qualified staff for those jobs, or to persuade staff to accept new working arrangements. The need for special incentives for attraction or retention can be based on labor market conditions and/or cost of living in the localities where agencies operate or as a function of the specific skills that the agency competes for in the labor market, based for example on job evaluations and labor market surveys.  Differentiation can affect: (i) all staff paid by the entity, with the result that there are agency-specific pay scales; or (ii) particular occupational groups or cadres **;** (iii) specific individuals, with scarce but vital skills; or (iv) specific locations entailing particular hardship.

- **Delegation:** Transferring authority over pay setting and human resource management from a central civil service agency to ministries, agencies, or departments. The transfer may entail transferring from the central to the entity level (or a mixture): pay negotiations (if any); setting the overall wage bill; and design of pay scales.

While the three dimensions of pay individualization, differentiation and delegation describe analytically distinct elements of pay flexibility, reform examples in the real world often combine all elements. There are three reasons why components of pay flexibility tend to go together: a shared set of assumptions that underpin them; a complementarity in the objectives that they are aiming for; and sequencing. First, many of these reforms took place in the context of New Public Management that emphasized giving managers the authority to manage in exchange for tighter accountability standards. This move reflected wider changes in the economy where there has been a significant move away from industry and sector-wide negotiating arrangements with significant government involvement towards more local and agency-specific arrangements.

> 4 Hawthorne Effect is the phenomenon whereby a subject modifies his or her behavior simply in response to the fact that they know they are being studied, and not because of the particular treatment of interest.

3
Second, the three components of pay flexibility likely complement each other in producing outcomes at the individual, agency, and public-sector wide level. For example, differentiated pay-setting is widely assumed to allow the agency to set pay at a level which is appropriate for the given task in the specific labor market in which the agency operates. However, to the extent that it also disrupts sector-wide pay negotiations, allowing government as employer to limit pay increases to the minimum necessary to maintain performance without the ―leveling up‖ effect of a common pay framework, it can also have some effect in reducing pressures on the aggregate public sector wage bill. Delegated pay setting by giving managers authority to engage their staff in determining how best to keep the agency functioning within the hard constraints is also likely to have some effect on the culture of performance at the agency level.

There are may also be a logical sequencing between the components. For example, delegation can lead to PRP since if pay-setting authority and accountability for results have been transferred to agency managers, they will look for mechanisms that maximize the likelihood that the results will be achieved at minimum cost. Delegation also automatically leads to differentiation to the extent that ―local supplements will be used as a means of topping up the earnings of those professional staff in demand from private sector employers rather than of those for whom the state acts as a near monopsonist employer‖ (Grimshaw 1998a, <u>p.7)</u> This paper is organized as follows. The next section details the main theoretical arguments for and against PRP and their hypothesized impact on staff effort and productivity. The bulk of the paper — sections 3 to 5 — focuses on the empirical evidence. The presentation of the empirical evidence is done with the aim of differentiating the findings by both the methodology of the study and the relevance of the study to different types of public sector jobs, such as the core civil service, tax administration jobs, teaching, and health sector jobs. Sections 6 summarizes the main findings and the final section points to the underexplored agency and public sector wide potential effects of PRP and provides suggestions for future research.

It should also be noted here what this paper is not about. It is clear that pay policy is only one factor affecting staff incentives and public sector performance; other human resource management considerations including recruitment, personnel management, training, and organizational management practices are undoubtedly important. The paper reviews the empirical evidence from research that is limited to PRP, assuming that all else is constant; this limitation is both to keep the task manageable and in order to examine a variable which has a particular relationship with other aspects of human resource reform. It cannot reach a conclusion that when PRP ―works‖ that it is more important than other variables. These are important issues which merit a different study.

# **2. Theoretical debates**

Theoretical debates on PRP have been evolving in the context of private businesses (Prendergast 1998, 1999), the general public sector (Dixit 1999; Burgess and Ratto 2003; Perry, Mesch, et al. 2006) and on specific occupations such as teachers (Neal 2011). The theoretical arguments can be roughly divided between early psychological theories on human motivation and training, popular in public administration research, on the one side and core economic theories on incentive structures and principal-agent problems on the other, with behavioral economics building a bridge between the two.

## **Expectancy and reinforcement theory**

Public administration research on PRP usually relies on what is often called ―expectancy‖ (Vroom 1964; Porter and Lawler III 1968) and ―reinforcement‖ theory (Skinner 1969; Luthans 1973). Expectancy theory builds on psychological insights about repeated behavioral patterns and learning under positive and negative stimuli. In its simplest form the theory suggests that explicit incentives in the form of 4
performance pay work under two conditions: First, employees need to believe that increased effort leads to increased performance, second increased performance leads to desired outcomes and is recognized by management. If the two conditions are met, employees form a behaviorally salient expectation about a future reward and adjust their work effort upwardly. Reinforcement theory stresses the effect of cultivating a behavioral norm of high work effort through reinforcing behavior with positive rewards. Apart from the direct link between performance and individual rewards, advocates in the field of public administration highlight secondary effects associated with performance pay: it helps to recruit and retain highly-skilled and/or motivated staff who presumably would do better under such an arrangement; increases the awareness for organizational goals by defining explicit performance standards; weakens the power of public sector unions; makes managers more responsible; signals core organizational goals to outside actors; increases the link between individual and organizational job goals; reduces the overall wage bill by moving away from automatic pay increases; and leads to an increase in overall job satisfaction through the individual recognition of employee efforts (Marsden 2004; OECD 2005b; Marsden 2009).

Critics of performance pay point out that the two conditions of expectancy theory are not always met and it is in principle difficult to design performance pay schemes that work as intended. Furthermore, critics argue humans do not always approach work effort and the assessment of salary in an entirely rational way and thus invalidate simple theoretical models based on rational-actor assumptions. In addition, measuring performance in the public sector is often fraught with difficulty. Many core public servants perform services on a daily basis that are hard to measure or are non-measurable, or produce outputs that are not market-priced. For example, early critics of test score based school and teacher evaluations argued that teacher performance cannot be neatly summarized by mechanic student test scores and such practice usually invites behavior that contradicts the overall goals of the teaching profession (Murnane and Cohen 1986). Interestingly, these criticisms were already leveled against early forms of test-based performance pay in British schools in the 19th century (Gratz 2009). Using explicit and objective performance measures can induce tunnel vision, myopia and measure fixation (Propper and Wilson 2003). Additionally, there is a lack of clarity about who determines and evaluates performance, and civil servants often work in large teams under the supervision of multiple managers, complicating the attribution of performance and responsibility of evaluation. Some authors see as a necessary condition the presence of high levels of trust and transparency between employees and management, to avoid arbitrary implementation and worker dissatisfaction (Kellough and Lu 1993).

An influential article (Kerr 1975) identified scenarios in which well-intended incentive schemes ended up favoring behavioral responses by employees that fulfill performance criteria, but taken to the extreme contradict overall organizational goals and standards. A different strand of criticism focuses on other motivations underlying public servants' effort. Apart from pure monetary rewards as expressed in salary, civil servants, it is argued, are motivated by notions of altruism, prosocial behavior and commitment to institutional goals (Perry and Hondeghem 2008), which are seen to compete with or even stand in conflict with explicit monetary incentives.

# **Incentive and principal-agent theory**

Several surveys have synthesized the theoretical literature on pay incentive systems for the private sector (Prendergast 1998, 1999), for the public sector (Dixit 1999; Burgess and Ratto 2003) and specialized occupations such as teaching (Neal 2011). The most basic argument for incentive pay is founded in a simple microeconomic principal-agent model of labor relations, in which a principal (the employer) wants to induce an agent (the employee) to perform a certain task. Such principal-agent relationships are commonly affected by two problems (Dixit 1999): moral hazard and adverse selection.

5
Moral hazard describes a scenario in which the agent's actions affect the principal's payoffs, but the action is not directly observable to the principal. This situation arises naturally in the workplace setting (public or private), i.e. the employee's effort at work is not directly observable, but influences productivity and outcomes, which the employer cares about. Contracts that tie observable outputs, which are correlated with unobservable effort, to desired pay incentives can mitigate inefficiencies in the principal-agent relationship from the perspective of the employer. Offering fixed pay contract gives the employer little leverage to influence employee effort after hiring decisions have been made. The incentive problem is exacerbated if employees are hard to fire. Bonus or merit pay schemes are therefore one way of designing incentive schemes that address moral hazard.

In the case of adverse selection the agent has access to private and valuable information at the time of contract signing. To induce the agent to reveal this private information, the principal must offer attractive contract terms. Adverse selection in the public sector plays an important role in civil service recruitment, where low and high-skill applicants are hard to distinguish based on public information. Public agencies need to offer contracts that induce high-quality applicants to apply and deter low quality applicants from misrepresenting their qualifications. Merit pay systems are argued to alleviate this sorting problem and attract higher quality personnel that expect to perform well under system of merit pay, while traditional fixed salary scales are seen to attract low grade applicants (Delfgaauw and Dur, 2008).

Avoiding moral hazard and adverse selection are often used to advocate for forms of performance pay in the public sector. Such incentive schemes fundamentally require the ability to measure some relevant outputs, design a scheme that properly links unobserved actions to outcomes and offer bonuses that induce agents to increase effort. Incentives work best if the agent's actions are tightly linked to observable outcomes, i.e. when the random noise is not overpowering the incentive effects. Incentives schemes are also affected by the risk-aversiveness of employees. Since an incentive scheme links outcomes that are only partially under the control of the agent, making final pay outcome-dependent decreases the utility of risk-averse employees, who usually demand an upward adjustment of average pay to compensate for the increase in risk. Even with very simple models, the optimality of the incentive scheme is sensitive to important design aspects, like the schedule of bonuses (linear, stepwise or other) and depends on the particularities of the employee's task.

A well-known criticism of performance pay arrangements is that when tasks are multi-dimensional incentivizing only some tasks that are observable and measurable will not necessarily improve overall outcomes, but rather lead to a substitution of effort allocation from the unobservable to the observable tasks, which under some circumstances — depending on whether the different tasks are complements or substitutes, and the nature of the functional relationship between the tasks and the outcomes — can even lead to worse outcomes (Holmstrom and Milgrom 1991). For example, the task of teaching can involve both instruction based on sound curricula and coaching on test-taking strategies, and poorly designed incentive schemes can encourage teachers to re-allocate effort to the latter and away from the former (―teaching to the test‖) to the detriment of human capital accumulation. Since it is generally hard to accurately measure and evaluate any aspect of public sector jobs, it is hard to devise effective civil service schemes that have the intended consequences with regard to final outcomes.

The problem of selecting appropriate performance measures to address this problem has spawned its own theoretical and empirical debates (Courty, Heinrich, et al. 2005). A problem related to the multi-tasking argument deals with the issue of gaming or cheating incentive systems. Typical examples are outright manipulation of results, cream-skimming, i.e. the manipulative selection of clients to improve program effects (Heckman, Heinrich, et al. 1997) or even the provision of high-caloric food to students during test days (Figlio and Winicki 2005). The problem of gaming performance standards is equally relevant in a 6
dynamic context, requiring ongoing adjustments by the principal (Courty and Marschke 2003). Luck during early periods of the evaluation period can induce increased slack for the rest of the time, a form of incentive gaming that has been found to be empirically relevant, as an influential study of Navy recruiters has shown (Asch 1990). To counteract excessive gaming of incentive schemes, it has been suggested, in the context of student test scores and teacher merit pay, to use evaluation systems independently from output measurements, i.e. in the case of teacher evaluations, to measure teacher contributions in tests different in form and content from instruments designed to track overall student and school progress or national exams (Neal 2011). Additionally, relative performance schemes in which employees are ranked against each other, potentially in a formal tournament setting are much harder to manipulate (Barlevy and Neal 2011; Neal 2011).

However, (Marsden 2009) notes that the gaming is not necessarily restricted to ingenious behaviors on the part of staff. Managers required to implement performance-pay arrangements might conclude that, in the bargaining with their staff concerning efforts and results, it is tempting and doubtless easier to: ―collude with their subordinates: to go through the motions and fill in the forms for goal setting and appraisal, but not to worry about the reality‖ (Marsden 2009, p.5) As outlined above, incentive schemes do not necessarily have to reward individual performance, but can focus on rewarding teams. Rewarding team performance can have certain advantages, ranging from reduced evaluation costs, to avoiding harmful competition between employees. However, basing rewards on team outputs can also lead to problems of free-riding where some team members willfully reduce their efforts in the expectation of relying on the work of others. The strength of free-riding problems depends on the size of the team and internal monitoring and punishment norms (Dixit 1999).

Picking the correct size of bonus brings its own challenges. Small bonuses will have little incentive effects and fall short of expectations, while large bonuses can lead to employees to treat incentive schemes as pure lotteries, especially if outcomes are strongly stochastic (e.g. student test scores (Neal 2011)), and to encourage cheating.

A reverse problem can arise if employees have to deal with multiple principals, a common feature of public service hierarchies. If different principals value different outputs, have different information available and have little ability or incentive to coordinate, separately designed incentive schemes are likely to fail (Dixit 1999).

# **Behavioral economics —intrinsic versus extrinsic motivation**

Building on the argument that worker motivation is also driven by intrinsic concerns behavioral economists have advanced a line of argument that casts additional doubt on the feasibility of performance pay arrangements, specifically in public service settings. Introducing explicit monetary incentives for employees with strong intrinsic motivation can have the effect of crowding-out these intrinsic affects, i.e. workers change their perception about the organizational goals and values, leading to an overall reduction of effort. Even small changes in the pecuniary reward structure can induce a change in attitudes, switching from seeing the task as a partially voluntary contribution to a low paid contract service without any ownership stake. Crowding-out can be especially salient if performance pay is introduced using antagonistic framing and can stifle creativity and collaboration (Frey and Osterloh 1999). While the theory of intrinsic motivation has been proposed by psychologists, formal treatments of the trade-off between extrinsic and intrinsic motivation have been developed (Kreps 1997; Benabou and Tirole 2003, 2006) and integrated in the wider context of public goods provisions (Besley and Ghatak 2004).

7
The debate around the weight of intrinsic motivation within overall incentives has been crystallized in the debate led by Le Grand about whether public service workers are ―knaves or knights‖ (Le Grand 2003). Le Grand argues that post-war public administration theory in the UK (and in Europe more generally) saw public servants as public-spirited altruists, a misleading interpretation of reality which was recognized in, although not adequately redressed by, the 1980s introduction of various New Public Management reforms.

On the opposing side of the argument, (Pink 2009) has developed the critique of monetary and other extrinsic incentives into a broader theory, hypothesizing that they are both counterproductive, as they frequently undermine intrinsic incentives, and unnecessary, as intrinsic incentives can be harnessed and used to maximize individual productivity. His theory suggests that tasks can be constructed to: (i) maximize an individual‘s sense of _autonomy_ (drawing inter alia on the ―self determination theory‖ expounded in the overview paper by (Ryan and Deci 2000) and the cross-country attitude survey undertaken by (Chirkov, Ryan, et al. 2003)); (ii) _mastery_ (continuous incremental learning and improvements rather than distant targets citing inter alia (Sauermann and Cohen 2008) who use data on over 11,000 industrial scientists and engineers to show that intrinsic motives, particularly the desire for intellectual challenge, appear to benefit innovation more than extrinsic motives such as pay); and (iii) _purpose_ (drawing inter alia on (Niemiec, Ryan, et al. 2009) who show from a follow up study of 246 students who graduated from two US colleges that those who met intrinsic aspirations for personal growth, close relationships, community involvement, and physical health had better scores for psychological satisfaction and health than those who pursued attainment of the extrinsic aspirations for money, fame, and image.

Another psychological argument, known as the ―Yerkes-Dodson law‖, highlights the phenomenon of ―choking under pressure‖ (Ariely, Gneezy, et al. 2009). If individual salary is subject to high-stakes pressure, individuals experience increased arousal and shift from automatic to controlled behavior, a narrowing of attention and a pre-occupation with the reward, all lessening the chances of success. The argument is that performance has an ―inverse U‖ relationship with the level of the incentive payment, with performance improving at low and moderate levels of incentive payments as compared to no payments, but then being worse at very high levels of payment compared to moderate, low, or even no payments.

Lastly, behavioral economists have identified the possibility of satisficing instead of maximizing behavior. Employees might exert effort until a certain minimum level of reward is reached and then substitute additional labor supply for increased leisure or idle time. A study of New York cab drivers identified the prevalence of satisficing behavior and questions the effectiveness of performance pay schemes, if the satisficing threshold is hit quickly (Camerer, Babcock, et al. 1997).

The proponents of ―self-determination theory‖ clearly have a strong and intuitively attractive point to make, but it is not evident that the case for intrinsic incentives overwhelms consideration of extrinsic incentives. Three areas of uncertainty hang over conclusions in this area. First, autonomy, mastery and purpose are more feasible in some tasks than in others. Second, intrinsic motivation can take two forms — being motivated by the inherent nature of the job or being motivated to earn the respect of one‘s peers — and group-based performance schemes, to the extent that they encourage teamwork, could increase intrinsic motivation. Finally, the theory is largely addressing the moral hazard problem inherent in principal-agent relationships, suggesting that staff can sense that they are being treated instrumentally through the use of extrinsic incentives and so will be motivated to cheat. There are different implications for the self-selection/sorting argument if the general population can be divided between predominantly intrinsically and extrinsically motivated people, since under those circumstances an explicit system of performance pay will attract extrinsically motivated applicants to the civil service.

8
# **3. Organizing the empirical evidence: “Craft” and “Coping” jobs**

To summarize, the main argument for PRP is that linking incentives to inputs, outputs and outcomes can: (i) ameliorate moral hazard, inducing more effort, including the selection of better working methods to the extent that the individual has some autonomy; and (ii) address adverse selection by encouraging high ability individuals who will do better under a performance pay scheme to join the agency and similarly discouraging low ability individuals. Both of these linkages suggest contemporaneous and over-time effects of incentive schemes. Comparatively little research focuses on the self-selection or sorting effects of competitive performance schemes; the larger part of the literature deals with the stringent context conditions necessary for performance incentives to engender increased effort.

These contextual factors can be disaggregated into two categories: First, variables that are characteristics of the job that the individual performs and second, variables that are characteristics of the technical design of the performance scheme itself. The key ―job‖ variables identified in the literature are:

- _Measurability of the goal or outcome from the job_ :  In some jobs outcomes are easier to measure than on others. Examples are production work on the factory assembly line or sales jobs, to be contrasted with <mark>managerial jobs in line ministries or</mark> large private bureaucracies;

- _Multi-dimensionality of the actions that produce the outcome_ :  The overall outcome of the job may depend on a single or multiple actions or activities. Then performance pay can create perverse outcomes by encouraging effort allocation towards the actions that positively impact the performance measure being used but negatively impact the overall outcome;

- _Observability and measurability of the actions that produce the outcome_ (Wilson, 1989):  Most public sector jobs involve multi-dimensionality of tasks; however, what distinguishes some tasks from others is whether or not the actions are observable.  As James Q. Wilson noted, in some jobs it is much easier to measure whether or not an action was performed — e.g. health safety regulations were drafted — than whether or not the outcome, improved occupational safety, occurred.  As (Dixit 1999) notes, the principle terrain of incentive theory are those jobs where actions are not observable;

- _Controllability of the outcomes_ (Bruns et al, 2011): The extent to which the outcome is a function of the efforts of the individual or is influenced by other factors beyond the individual‘s control.

Some of the main design choices in incentive schemes are:

- _Predictability of the incentive_ (Bruns et al 2011):  the probability of the agent receiving the incentive if the measured outcome is achieved. If the probability is either close to 0 or 1, then the incentive will have no impact. The incentive is clearest for example, for piece-work settings, common in sales or blue-collar settings in the private sector;

- _The size and nature of the incentive payment_ : While the incentive effect should theoretically increase with the size of the bonus, the Yerkes-Dodson Law points to the consequences of very large bonuses. Large bonuses also can create incentives for cheating. Group based schemes could encourage team work but also result in free-riding behavior;

- _The nature of the performance evaluation and the performance standard_ : Whether this should this be an objective evaluation based on quantitative performance targets or a subjective evaluation, what is the benchmark against which it is evaluation, and who should be responsible for the evaluation.

9
The general criticism is that performance pay cannot be implemented in the public sector because political difficulties in selecting appropriate design features to address the complexity of the job variables — for example, giving bonuses to everyone thereby rendering it a completely predictable salary supplement and not an incentive payment. However, this criticism glosses over the many different organizational contexts within the public sector. For example the outcomes of schools and tax authorities are more measurable than those of central policy or administrative units in ministries and departments. Moreover, many large private sector bureaucracies also approximate public bureaucracies on some of these variables — while there is the profit ―bottom line‖, the controllability of outcomes can be low and the tasks that generate the outcome can be multidimensional. In examining the empirical evidence therefore, it is very important to distinguish between these different contexts in order to better understand under what conditions these schemes can or cannot work.

This paper borrows and slightly modifies Wilson‘s typology (Table 1) to organize the empirical evidence so as to present these contextual nuances.<sup>5</sup> Jobs can be characterized by whether or not the job‘s outputs are easily measurable and whether or not the actions in the job to produce the output, or the internal production process, are observable. The matrix provides a framework within which to organize the empirical evidence by job type, with the simplifying assumption that jobs with multiple dimensions are located within the cell that represents the most complex of those dimensions. The top left box describes ―Production Jobs‖ in which outcomes are easily measurable, the production process consists of repeatable, mechanical tasks that are observable to an outside monitor, and controllability is likely to be high. Typical examples are manufacturing factory-floor jobs, sales jobs, and municipal services like garbage collection. If the production process is not directly observable, but outputs remain measurable, such jobs are termed ―Craft Jobs‖. With recent advances in measuring learning outcomes, teaching can be classified as a public service in which the exact process of production is hard to fix, but, at least to a certain degree, desired outcomes are quantifiable. Similarly some of the outcomes of healthcare, particularly preventative services like child immunization, are also more measurable. Other examples include tax collection, job placement services, and auditing.

In the bottom row are ―Procedural Jobs‖ and ―Coping Jobs‖. Both are characterized by difficult to measure outcomes, but again differ in the observability of the production process to an outsider. Procedural jobs like the military have clearly defined inputs, whereas administrative jobs in general policy units of the central government neither produce easily measurable outputs, nor have transparent production processes. Coping jobs present the most challenging functional contexts for PRP.

The general literature on performance pay is quite positive in relation to production jobs. Much of the earlier literature on incentive payments in the private sector looked at the impact of piece-rate compensation on productivity in production organizations involving manufacturing jobs or on similar repetitive tasks. (Stajkovic and Luthans 2003) conducted a meta-analysis of 72 empirical studies that investigate the impact of financial, as well as other, incentives on various measures of performance in organizational settings. These studies were conducted in a variety of private organizational settings, and

> 5 Wilson had originally used this framework to classify organizations and not jobs, the implicit assumption being that organizations were homogenous in the tasks that they performed.  This adaption of the framework to jobs does not change the logic of the typology, and is consistent with the proposal made by (Pritchett and Woolcock 2004) to classify decision-making functions in the public sector according to the discretion inherent in the task (vs. simple rule-following) and the number of the transactions necessary to deliver the service. Tasks which are necessarily discretionary and transaction intensive are “practice” tasks in Pritchett’s logic and “coping” tasks in Wilson’s view – but essentially refer to the same type of job – and, as Pritchett notes: “(t)he provision of key, discretionary, transaction-intensive services through the public sector is the mother of all institutional and organizational design problems” (Pritchett and Woolcock 2004, p.196).

10
included both industrial and service sector organizations. The study showed that financial incentives alone improved task performance by 23 percent, whereas financial incentives combined with social recognition and positive feedback from superiors increased task performance by 45 percent.

**Table 1: James Q. Wilson’s classification of job types**

|||**Actions or intern**|**al production process of the job**|
|---|---|---|---|
|||**Observable**|**Not observable**|
||**Relatively**<br>**easily**<br>**measurable**|**_Production job_**: Simple<br>repetitive stable tasks,<br>specialized skills.|**_Craft jobs_**: Application of general sets of<br>skills to unique tasks, but with stable, similar<br>outcomes.|
|**rom the job**||Examples: Manufacturing,<br>sales, simpler municipal<br>services (garbage collection).|Examples: Auditing; revenue collection;<br>teaching; medical practice; Job placement<br>work|
|**Outputs f**|**Not easily**<br>**measurable**|**_Procedural job_**: Specialized<br>skills; stable tasks, but unique<br>outcomes|**_Coping job_**: Application of generic skills to<br>unique tasks, but outcomes cannot be<br>evaluated in absence of alternatives|
|||Examples: Military|Examples: Administration; managerial jobs<br>in large private sector organizations|

A well-known study by (Lazear 2000) uses individual-level worker data from a glass company and estimates a 44% increase in productivity after switching to a piece-rate salary schedule. In a structurally similar setting, the introduction of piece-rate pay in a Canadian tree planting business has also been found to strongly affect worker productivity and profits (Paarsch and Shearer 1999).

However, production jobs, as well as procedural jobs, are of limited relevance to the public sector. Rather, given that the loci for incentive schemes, following the principal-agent literature, are mostly in contexts where actions are not easily observable but outcomes may or may not be, the focus of this paper will be on jobs in the right hand column of Table 1. Note that some of the studies of the private sector that have focused on incentives for rank-and-file ―knowledge workers‖ in large organizations can also be classified in this right hand column and will be included in the discussion.

# **Methodological approaches**

The vast empirical literature on PRP utilizes a range of methodological approaches. Early studies on performance pay in the public sector were largely qualitative case studies. As elements of civil service reform and incentive schemes were introduced in OECD countries throughout the 1980s and 1990s, academics and policy experts wrote initial studies and reports that summarized the main descriptive feature of reform attempts, and chronicled the reform process and implementation. Overall evaluation of the success of incentive schemes was based on qualitative impressions of practitioners. Later studies 11
approached questions of performance pay slightly more systematically, comparing several reform cases with each other and using convenience samples of employees and senior management, subject to performance pay, to collect data on self-assessed motivation and satisfaction with newly introduced performance pay. While attempting to provide a systematic evaluation of the successes and failures of performance pay reform, to a large extent these studies rely on weak research designs. Despite their descriptive value, inferences based on the single or comparative evaluation of reform cases, without the use of counterfactuals, cannot establish the existence or absence of potential program effects. Furthermore, utilizing convenience samples and using survey instruments that only measure self-assessed motivation suffer from selection and perception bias problems, while ignoring the information on actual outcomes.

Although the tenor of this initial wave of research was rather discouraging about the results on the effectiveness and popularity of performance pay among staff, incentive schemes for the public sector did not lose their appeal to policy makers (Marsden 2009). An intense empirical debate on teacher incentive was sparked in the context of local American school reforms and a national debate on test-score based school accountability. A series of papers by education economists used the opportunity of local reform attempts to collect data on samples of schools and students subject to new performance pay arrangements and comparable control populations. By measuring detailed public service outcomes, student's grades and test scores, drop-out rates and attendance records, plus detailed information on bonus programs, the quantitative evaluation of this new data allowed a more stringent test of the competing theoretical arguments. The advantage of these studies compared to prior work is their ability to disentangle the effect of teacher effort from other factors that also determine student outcomes.

While an improvement, many of these quantitative observational studies and similar work on private companies and the public service still fall short of an ideal research design for causal inference on program effects. The gold standard for program evaluation is the use of randomized-controlled trials (RCT), in which treatment assignment to subjects is randomized and unrelated to other observable and unobservable characteristics. This randomization allows the estimation of the treatment effect by a comparison of treated and control units. Observational studies on the other hand rely on treatment and control groups created not by controlled random assignment, but produced by social processes, often related to the research question at hand. Issues of selection bias and confounding factors can undermine the internal validity of such studies.

Utilizing the power of a randomization framework, several behavioral economists have used laboratory experiments to test hypotheses with regard to incentive schemes. Laboratory experiments offer at least two distinct advantages over observational studies: the researcher can use randomization to ensure unbiased and consistent estimation of treatment effects; and researchers can design their experiment to directly relate to theoretical questions at hand. The review of the theoretical literature has identified the importance of design details and the plethora of possibilities when it comes to performance pay arrangements. Observational studies have to rely on bonus pay schemes that have been implemented in real life, which do not necessarily cover all interesting variations and often mix different elements that conflate theoretical questions of interest. Being able to design a clean and tailored experiment to trace the effect of intrinsic versus extrinsic motivation in a linear bonus pay scheme for example is a huge advantage which laboratory experiments offer. A wave of articles in the late 1990s and 2000s explores various issues of performance pay. While offering certain advantages over observational studies, laboratory experiments often use notoriously small samples and student subjects that share few characteristics with actual workers or public servants. Furthermore, laboratory experiments can hardly ever replicate real work place settings or offer bonus schemes that remotely approach bonus sizes common even in only moderately incentivized performance pay schemes. This raises concerns from a sampling perspective and the representativeness of the subject pool, as well as the comparability of 12
laboratory treatments and real-world bonus programs. Researchers generally confer low external validity to laboratory experiments and caution against the isolated interpretation of results derived from a single experiment.

The most recent attempts at addressing the issue of proper causal inference on performance pay and increasing the representativeness of results are RCTs. In an RCT researchers are able to randomize key features of an actual policy program that services the actual population of interest. The advantage of such a field experiment is the similarity of the target population, the structure of the incentive program, paired with the randomization of treatment. Although field RCT are time and resource intensive studies, several teams of researchers have implemented similar studies in different contexts around the world, considerably adding to the empirical understanding of performance pay.

# **4. The empirical literature reviewed**

Given that the literature has analyzed incentive pay for a broad set of jobs and utilizing an array of methodological approaches, the following presentation of the empirical literature in this paper is sorted by (a) the nature of the job, using Wilson‘s typology and (b) the methodological approach. In total 153 empirical studies of PRP (see the Appendix for the full list) were considered in this review, of which 110 are for craft and coping jobs (Table 2). The research to date on the subject has largely focused on advanced countries — in the review 127 studies are in OECD contexts, and only 26 are in developing country settings. The literature has also focused largely on craft jobs and production jobs, with no experimental studies to date on coping jobs.

**Table 2: Studies by country environment, methodology, and job type**

|**Country and methodology**|||**Types**|**of Jobs**|||
|---|---|---|---|---|---|---|
||**_Productionjobs_**|**_Proceduraljobs_**|**_Coping jobs_**|**_Craft jobs_**|**_Unclassified_**|**_Total_**|
|**_OECD study_**|**27**|**0**|**15**|**72**|**13**|**127**|
|Observational|14|0|15|59|13|101|
|Field RCT|7|0|0|13|0|20|
|Lab. experiment|6|0|0|0|0|6|
|**_Developing country study_**|**1**|**0**|**1**|**22**|**2**|**26**|
|Observational|0|0|1|15|2|18|
|Field RCT|0|0|0|6|0|6|
|Lab. experiment|1|0|0|1|0|2|
|**Total**|**28**|**0**|**16**|**94**|**15**|**153**|

## **Observational studies**

## **_Public sector coping jobs_**

Observational studies about performance-related pay for public sector coping jobs suggest somewhat limited effectiveness of this component of pay flexibility for these activities — they also focus overwhelmingly on OECD settings.

A series of OECD reports and associated discussion papers chronicle the type and extent of pay related civil service reforms in advanced industrialized countries (OECD 1993, 1996, 1997b; Kim 2002; Burgess and Ratto 2003; OECD 2004a, 2005a, b; Perry, Mesch, et al. 2006; Ketelaar, Manning, et al. 2007; Rexed, Moll, et al. 2007; OECD 2008, 2009; Perry, Engbers, et al. 2009). Generally, these refer to coping jobs in that they refer to, or imply that they are referring to, managers with complex tasks.

13
(Cardona 2007) reviews incentive programs in the US, particularly the Performance Management and Recognition System, the UK's Inland Revenue Service performance scheme and similar attempts in Australia. The study documents several common issues in the implementation of performance pay: employees are hardly ever scored less than satisfactory in their evaluations, bonus systems were designed so that only very few employees actually received any payments and the majority of staff found the system de-motivating and inciting jealousies. (Straberg 2010) highlights the problem of perceived unfairness following the introduction of performance pay in an OECD country, although also notes that there was no empirical linkage between pay justice perceptions and workplace behaviors. Managers equally found little positive changes resulting from the introduction of performance pay. As context, multiple studies confirm the political and operational difficulties of successfully introducing any major program of pay reform within the public service (World Bank 1999; Kiragu and Mukandala 2003; Independent Evaluation Group 2008).

Brudney and Condrey present a pair of typical studies drawing on non-representative surveys of federal managers and government employees in the US, that were subject to performance-pay arrangements (Condrey and Brudney 1992; Brudney and Condrey 1993). While largely descriptive in nature, they find that 17% of managers report an increase in motivation, but prior attitudes about merit pay affect this result. It is unclear though whether the documented increase of motivation is solely an effect of performance pay, since no control units are used to rule out the influence of other possible confounders or if self-reported motivation is in any way linked to actual work outputs. Similar work on performance pay for civil servants in the US state of Georgia also finds highly critical opinions of staff members with regard to explicit evaluation and selective bonus pay (Kellough and Nigro 2002). More recent studies using survey data of US city managers finds higher satisfaction of employees when performance pay is used (Stazyk 2010). The research does however highlight the complexity of the issue as it suggests that there is a crowding-out effect on extrinsic motivation for staff with a distinctively high level of intrinsic motivation — but ultimately this does no harm to effort or to job satisfaction.

(Dowling and Richardson 1997) evaluate the effect of performance pay on UK National Health Service managers, a study, since it focuses on management rather than physicians, that is more closely related to core public administration jobs than healthcare jobs. Using self-reported data from a survey, they find a modest positive effect of pay incentives on manager motivation and effort.

Whether PRP should be used is one thing — when and where it can be used is another. (Dahlstrom and Lapuente 2009) theorizes, using transaction cost economics, that when senior officials share a career path with elected politicians and thus are unlikely to act impartially, then PRP is less often used as those officials are unable to provide credible commitments to their staff. Why would one work hard to achieve a goal if it transpires that the goalposts might be arbitrarily moved for political reasons at the last moment? They find empirical support for this prediction.

# **_Public sector craft jobs_**

Observational studies about performance-related pay for public sector craft jobs are more optimistic concerning their effectiveness. There is more evidence to draw on from developing countries, particularly in relation to teaching and health care.

# **Tax administration, job placement**

Revenue authorities are examples of public sector agencies that can be classified as craft organizations where outputs — number of audits conducted and tax fines collected — are more easily measurable and there is a clearer link between the efforts of, for example, individual tax auditors and revenue collection.

14
A good example of detailed pay PRP and high-powered performance pay incentives comes from the federal Brazilian tax collection agency. In 1988 the Brazilian government created a bonus program for tax officials that rewarded the identification of tax violations. Base salary was augmented on a monthly basis on an individual and group basis. The group reward was calculated based on the relative performance of one local agency versus others, with relative performance measured based on total fines collected, attainment of pre-defined quotas (total tax collection, number of inspections, collection of overdue taxes) and the size of the agency. Individual rewards were based on monthly evaluations by the direct supervisor, which combined objective performance criteria and managerial discretion and rated employees on a scale from zero to 70. Each employee that scored more than 21 points was entitled to an individual reward with the value being determined by the overall availability of funds (which are proportional to the collected fines) and the performance of co-workers. It was not unusual for total bonus payments to reach 200% of base pay.

(Kahn, De Silva, et al. 2001) found that this incentive scheme resulted in a 75% increase in fines per inspection. At the same time, they also found substantial regional variation, with responses ranging from 19% to 145%. The authors do caution that diverse management techniques resulted in some regions targeting wealthier sources (such as corporations), more aggressively, which points to the potential negative effects of such high powered incentive schemes which may encourage extortion. Unfortunately, limited data prevented the authors from examining these social costs further.

Another study of revenue authorities is by (Bertelli 2006), which explores the potential tradeoff between intrinsic and extrinsic motivation in the Internal Revenue Service in the US. The IRS implemented a paybanding system that imposed high-powered performance incentives on supervisors, but not on nonsupervisory personnel. Using data from the 2002 Federal Human Capital Survey, the author showed that the incentive scheme crowded in intrinsic motivation at the lowest pay levels, and crowded out at the highest levels.

(World Bank 2001) used survey data from revenue departments in 14 low, middle and high income countries, and detailed case study evidence from 7 of those, to review the effectiveness of bonus and salary supplement systems as a means to enhance effectiveness in revenue departments. They concluded that the ―circumstantial evidence‖ suggests that bonus systems do indeed seem to have an impact on organizational effectiveness. They note that in a number of countries the introduction of bonus systems have had a measurable impact on recruitment and retention of employees. However, they note that the success of bonus systems relies heavily on ―legitimacy‖, i.e. the internal and external ―acceptance‖ of the bonus system.

A set of studies of performance incentives in a similar organizational context is that of the US Job Training Partnership Act (JTPA). Under the Act, 620 semi-autonomous training centers were responsible for implementing job training programs for the indigent, and were given financial incentives tied to labor market outcomes — employment status, earnings — of the trainees. These bonuses were given to the training centers thereby augmenting their budgets but could not be used to supplement staff salaries. (Courty and Marschke 2004) find evidence of the prevalence of gaming among the agency staff in the choice of termination date of the training for the participants, which while increasing organizational bonuses imposed a cost to the participants in terms of earnings. Similar effects have been found by related studies of the program (Heckman, Heinrich, et al. 1997).

An early quantitative observational study of performance pay in the public service was implemented by (Asch 1990), who collected data on the behavior of Navy recruiters, subject to a point-based performance system. The incentive consisted of a point-scheme for the quality of recruited candidates, a fixed time frame for evaluation and a minimum threshold of points needed to qualify for a bonus. Asch shows that 15
the incentive scheme did increase the effort of recruiters and led to the recruitment of more high-quality candidates, but also induced recruiters to exhibit a form of gaming behavior, i.e. increasing recruiting efforts early in the cycle and once an expectation of reaching the bonus level was achieved, a reduction of effort for the remaining evaluation time followed.<sup>6</sup>

# **Teaching**

The literature on performance pay for teachers, an essentially craft job, also shows a variety of research designs. Some researchers use qualitative case studies (Murnane and Cohen 1986) or perception surveys of teachers subject to performance pay to measure effects on performance (Heneman III and Milanowski 1999; Kelley 1999). These studies often show a low degree of satisfaction with bonus systems and explicit evaluation. By contrast, studies focusing on actual outcomes often find rather encouraging results. A series of papers use the introduction of performance pay for teachers to quantitatively assess the effects on student outcomes. By now studies have utilized various data sources and structures to assess the effects of performance programs.

In the American context researchers have evaluated the effects of teacher quality on student outcomes (Goldhaber and Brewer 2000; Hanushek and Rivkin 2006; Clotfelter, Glennie, et al. 2007). Since performance pay is argued to be one important tool for attracting and retaining highly-qualified teachers, it is important to determine the effectiveness of merit systems in that regard. (Clotfelter, Diaz, et al. 2004; Clotfelter, Glennie, et al. 2008) show, using detailed data from North Carolina's schools, that accountability and performance pay systems contribute positively to retaining quality teachers. The introduction of merit pay can also be linked to student test scores, but with varying empirical robustness. (Cooper and Cohn 1997) find for a sample of over 500 South Carolina classes a positive effect of merit awards for teachers on mathematics and reading test score achievements.

Cross-sectional studies using data from the American National Educational Longitudinal Survey have been used to show a positive link between individual merit awards for teachers and student test scores (Figlio and Kenny 2007). Positive effects of performance pay have also been found in Arkansas kindergartens (Winters, Ritter, et al. 2009). To mitigate problems of causal inference, sometimes researchers can use difference-in-difference estimation, due to the geographic difference of the phasing-in of reforms (Eberts, Hollenbeck, et al. 2002; Atkinson, Burgess, et al. 2004). While (Eberts, Hollenbeck, et al. 2002) find no effects on student test scores and even slightly negative effects on other outcomes, their analysis relies on student-level data from only two schools. A more thorough and systematic differencein-difference analysis by Atkinson et al. finds clear positive effects of performance pay for British schools. By utilizing particular features of Tennessee's Career Ladder System and the Project STAR field experiment, (Dee and Keys 2004) are able to link teachers' quality assessments, as expressed in the career ladder grouping, to student test scores. They find that the official career ladder system had only mixed success in rewarding teachers with the highest test score gains, but nonetheless teachers with merit awards had positive effects on student's math scores. They found however no statistically significant effects on reading scores.

> 6 An early observational study which revealed significant gaming is noted in (Wilms and Chapleau 1999) who note that performance-based pay began in the UK in about 1710, with salaries based on test scores in reading, writing and arithmetic. The rationale was that it would help keep students from poor families in school, where they could learn the basics. In reality, the incentives encouraged teachers to narrow the curricula to include only easily assessed subjects, and cheating by both inspectors and teachers made the system ultimately untenable. The system was dropped in the 1890s. A similar scheme was introduced briefly in Canada in 1876, but it ran into similar difficulties and was terminated around the same time.

16
Several studies from the context of American school reform also document the role of unintended side effects of explicit accountability programs. Large-scale testing of students as part of the 2001 No Child Left Behind Act ties student test scores to important resource allocations for schools, giving schools incentives to improve student learning, but also to increase pure test-taking ability or to engage in outright cheating (Jacob and Levitt 2003; Jacob 2005). Quite surprisingly, (Figlio and Winicki 2005), using daily lunch menu data from a random sample of 23 school districts in Virginia, show that even the caloric content of school lunches was adjusted upwardly to improve cognitive ability on test days.

An interesting study of private schools in India assesses the role of teacher unionization on student outcomes (Kingdon and Teal 2008). While not explicitly evaluating performance pay, unionization of teachers represents an increase in job security and uniform, higher pay, without being linked to explicit performance standards. (Kingdon and Teal 2008) find strong negative effects of unionization on student outcomes, utilizing a within-pupil across subject variation fixed effects design. The study by (Ladd 1999), mentioned above, on school accountability in Dallas, uses panel data and finds positive effects of merit pay on student performance and dropout rates.

A set of observational studies (Lavy 2008, 2009) uses data from an Israeli policy experiment with tournament based teacher competition for bonuses. Using regression discontinuity and difference-indifference designs to approximate random treatment assignment, the study shows significant gains in student achievements. The studies also assess potential mechanisms for the link between performance pay and improved test scores, identifying a change in teaching methods, enhanced after-school teaching and increased teacher responsiveness over test-score manipulation as important causal channels.

A comprehensive study using cross-national data on performance systems in schools and PISA test scores also finds an positive association between pay-for-performance type reforms, improved teacher quality and student test scores (Woessman 2010).

# **Health care jobs**

A fairly large literature has dealt with the role of performance pay in improving health care delivery and services. General literature reviews (Petersen, D., et al. 2006; C. and N 2009) reflect the diversity in empirical approaches and the strong focus on OECD country experiences. In particular performance incentives in the UK NHS system and several U.S. insurance systems have received attention in the research literature.

A series of studies has evaluated the potential effects of financial incentives for primary care physicians. The British NHS introduced performance-pay elements into the remuneration of family practitioners in 2004. (Doran, Fullwood, et al. 2006) use data on over 8000 family practices and evaluate the effect of performance pay on patient outcomes and find overall high performance in the first year of the incentive scheme, but also evidence of ``gaming‘‘ through the exclusion of patients. (Campbell, Roland, et al. 2005; Campbell, Reeves, et al. 2007) also analyze the role of financial incentives in a stratified random sample of British general practices focusing on care for coronary heart disease, asthma and type 2 diabetes, finding a substantial effect of financial incentives introduced in 2004. (Steel, Maisey, et al. 2007) find positive effects on asthma and hypertension treatment, as do (Vaghela, Ashworth, et al. 2009).

(Chalkley, Tilley, et al. 2010), using evidence derived from a natural experiment in the UK publicly funded dental care system, analyzed the effects of using incentive pay that provided explicit rewards for increased service provision against the alternative of offering an employment-like relationship. They found that dentists who were moved from quasi-employment arrangement to an activity-based incentive contract increased their activity in the publicly funded service by 26%. They also found evidence of 17
considerable variation between suppliers, which suggests that factors such as an individual‘s intrinsic motivation, professional standards, and preferences were important moderators of financial incentives.

In the U.S. context, several insurance providers and health maintenance organizations have experimented with elements of financial incentives for care providers in various states. An early study of health maintenance organizations (HMO) managers‘ views on financial incentives found mixed support for the effectiveness of performance pay (Hillman, Pauly, et al. 1991) in the eyes of managers.

More recently, several studies have found fairly positive effects of performance incentives on health services, patient outcomes and satisfaction. (Safran, Rogers, et al. 2000) use a cross-sectional study of Massachusetts adults to assess the effects of various health-maintenance organizations (HMO) and their specific contract elements on primary care. One of the results links financial incentives for physicians to patient satisfaction. An evaluation of performance pay pilot program for physicians found meaningful improvements for diabetes patients compared to the control group (D. and Horrigan 2005). Small positive or mixed effects were also found studies by (Amundson, Solberg, et al. 2003; Casalino, Gillies, et al. 2003; McMenamin, Schauffler, et al. 2003; Levin-Scherz, DeVita, et al. 2006; Coleman, Reiter, et al. 2007; Felt-Lisk, Gimm, et al. 2007; Mandel and Kotagal 2007; Young, Meterko, et al. 2007).

(Rosenthal, Frank, et al. 2005) analyze a natural experiment, comparing quality improvements in two physician groups in the U.S. from 2001 to 2004. They find improvements in cervical cancer screenings but not other outcomes, largely rewarding practices with a high baseline performance. In a cross-sectional sample of primary care physicians that contracted with Medicaid managed care organizations in 2002 in California found a partially positive effect of incentive pay on STD care (Pourat, Rice, et al. 2005).

(Lindenauer, Remus, et al. 2007) analyze the effects of public reporting and pay-for-performance in hospital care in a Medicare/Medicaid demonstration project. Hospitals participating in the performance scheme show a significant improvement in overall measures of patient care quality, including care for heart failure, acute myocardial infection and pneumonia by up to 16%, compared to the control group. In a related, but patient-level study (Glickman, Ou, et al. 2007) evaluate the largest pay-for-performance pilot project in the U.S., finding no conclusive effects for several treatments and patient outcomes.

Similarly, (Pearson, Schneider, et al. 2008) find that pay-for-performance elements in physician contracts in Massachusetts did not add any significant gains above and beyond secular improvement in a time period from 2001-2003. In a study of public community health centers in Houston, (Gavagan, Du, et al. 2010) also find no effects of performance pay, while (Chung, Palaniappan, et al. 2010) find no effects for primary care physicians in California. Mirroring the results found in other areas, while financial incentives can improve particular behavioral responses of staff members, it is difficult to design an incentive scheme that does not also produce unintended consequences and rewards unwanted behavior. (Shen 2003) provides evidence of gaming and selection effects of financial incentives for substance abuse care providers. (Li, Hurley, et al. 2011) utilize a natural experiment in Ontario, assessing the effect of performance-related pay on physician behavior and targeted primary care provision. They do find positive results for some, but not all financial incentives, providing a cautionary message with regard to the potential impact of performance pay.

Importantly, all these studies evaluate performance pay for specific, easily measurable tasks, in a highly institutionalized environment with powerful monitoring capabilities. Observational studies dealing with performance pay in the health care sector in the developing country context are much fewer in number and can draw on fewer large-scale experiences. Despite the lack of widespread use of incentive pay in the developing world, incentive focused reforms for doctors and nurses have received some attention. (Vujicic 2009) outlines the potential benefits of performance pay for health services staff, but also 18
identifies the risks of supplier-induced demand and cost-explosions. He also correctly distinguishes between performance pay within health care units and the general contracting-out of health services to NGOs. While contracting of services though often includes overall performance targets and entails the introduction of staff performance incentives within contracted facilities, this review focuses explicitly only on performance pay elements in health care facilities.<sup>7</sup> Studies focusing on staff-level performance pay in low and middle income countries generally find positive results, but largely illustrate the lack of systematic findings and evidence. (McNamara 2005) discusses overall six cases of payment for quality in the health services sector across developed and developing countries, with cases in Nicaragua and Haiti having had had a positive effect. The Nicaraguan reform efforts though combined decentralization of decision-making authority, increased local accountability with explicit performance agreements, and while being judged to have led to an overall improvement (Jack 2003), it is hard to disentangle the effects of each reform element.

Similarly, in a recent study by (Witter, Zulfiqur, et al. 2011) a pay-for-performance arrangement in a NGO-led health project in the Battagram district of Pakistani was evaluated and found to have improved general services provision, but with an unclear effect of the performance-based elements. The study highlights though the weak link between bonus pay and performance, as well as the low amount of monetary incentives in relative terms.

A study of health care reform efforts in two Rwandan districts shows that the use of performance elements paired with increased autonomy seems to offer a viable and cost-effective way to improve health care delivery (Meessen, Musango, et al. 2006). In a later study, (Meessen, Kashala, et al. 2007) evaluate the performance of 15 health centers in Kabutare, Rwanda. They document a sharp increase in staff productivity after the introduction of output-based bonuses. (Soeters, Habineza, et al. 2006) highlight the potential applicability of the Rwandan experience in sub-Saharan Africa more generally.

Similarly, efforts to improve health services provision in Haiti using performance-based payment for NGOs in a USAID pilot project showed encouraging effects on immunization coverage and organizational behavior (Eichler, Auxila, et al. 2001).

A recent book by (Eichler and Levine 2009) outlines the general argument for the use of explicit incentive schemes in the provision of health services in developing countries. Apart from demand-driven financial incentives through conditional cash transfer programs, they discuss the role of performance-pay and contracting-out of services provision. They review evidence from various contexts, in particular experiences with contracting NGOs in Afghanistan and project evaluations in Haiti and Nicaragua, overall advocating the increased use of financial incentives.

# **_Private sector: Craft or coping jobs_**

Observational studies on performance-related pay in private sector craft or coping jobs are generally suggestive that PRP has a positive impact – but highlights the importance of careful design.

There is a newer and growing empirical literature that has looked at incentive payments in private organizational settings that are closer to that of ministries and departments, characterized by low measurability and low controllability of tasks. These organizational settings are akin to craft or, in some

> 7 (Loevinsohn and Harding 2005) review the success of ten contracting-out projects in the developing world, finding largely encouraging results.

19
cases, coping organizations. Analyses of incentive schemes in such organizational settings also exhibit a large variation in the research designs. (Beer and Cannon 2004) study the failure of 13 incentive plans at Hewlett Packard using interviews and internal documents, finding that managers abandoned the programs due to the perceived costs. Providing slightly more comprehensive evidence, in a worldwide survey of 205 top managers (Beer and Katz 2003) document the weak support of incentive schemes among management. Both studies echo some concerns identified in similar public sector studies, but suffer from weaknesses in their research design.

A number of quantitative observational studies improve upon prior work by using more representative samples. (Belfield and Marsden 2003) use panel data from a large UK work place survey, which include both piece-rate jobs and ―knowledge work‖ jobs in which performance is based on achievement of previously agreed goals, and find strong effects of individual pay-for-performance, but only conditionally on the monitoring regime. Another study uses the British Household Panel Survey to distinguish the productivity and sorting effects of performance pay, finding that jobs with performance-related pay attract workers of higher ability and induce workers to provide greater effort (Booth and Frank 1999).

(Blasi, Freeman, et al. 2008) econometrically examine the relationship between various forms of profit sharing and stock options on staff turnover, absenteeism, effort, and other productivity measures. The analysis is based on two large surveys of private sector firms, and finds statistically significant positive linkages between these shared capitalism schemes and perceptions of workplace performance such as turnover, loyalty, and worker effort. These incentives have the strongest impact when combined with other organizational variables such as competitive wages, training, and employee involvement in workplace policies.

(Hochberg and Lindsey 2010) is one of the few empirical examinations of the impact of stock options on company rank-and-file on firm performance (as opposed to the impact of options on top executives, on which there is a large literature). Since stock options are a group incentive based on overall company performance, incentives to free-ride are very high. However, alternative literature also suggests that nonexecutive compensation may also increase cooperation and encourage mutual monitoring among coworkers. Using a large database for a broad set of firms, and explicitly controlling for endogeneity, the study shows that stock options exert a positive effect on firm performance. The study also finds that this effect is higher on smaller firms, consistent with the free-riding hypothesis, and also higher in firms with higher growth opportunities where the monetary incentive is higher.

(Aboody, Johnson, et al. 2007) also examine the impact of executive and non-executive stock options on firms‘ operating performance, based on an empirical investigation of a sample of 1300 firms and contrasting between firms that re-priced their options to make them an attractive financial incentive versus those that did not. The study finds that while firms that repriced their options had a larger increase in operating income and cash flows compared to non-repricers, this impact was entirely due to executive stock options. The repricing of non-executive stock options had no impact, consistent with the free-rider argument, and in contrast to the (Hochberg and Lindsey 2010).

Some studies from the private sector have also highlighted other conditional variables, such as strength of social networks, that impact pay incentives. (Bandiera, Barankay, et al. 2005) use data from a farm labor operation to compare the effects of piece-rate versus relative incentive schemes, showing that individual piece-rates enhance effort irrespective of social relations among workers, whereas the effects of relative incentive schemes, which impose negative externalities among workers, is diminished when workers have stronger social ties. On the other hand, if incentives explicitly recognize team efforts, team rewards can outperform individual piece-rate wages, as one study on team work in a garment plant has shown 20
(Hamilton, Nickerson, et al. 2003). Data from a large group incentive scheme at Continental Airlines even found positive effects in the presence of strong free-rider incentives (Knez and Simester 2001).

# **Experimental studies**

Among experimental studies a distinction has to be made between laboratory and field experiments, with each having their own advantages and disadvantages. Laboratory experiments are characterized by their strong control over experimental design and treatment specification, but often lack the ability to use representative subjects or simulate convincingly real-world settings. Field experiments combine the randomized assignment of treatment with actual real-world programs and participants, but are often more constrained in their design choices, face stronger exogenous pressures to succeed and are resourceintensive.

# **_Meta-studies_**

(Jenkins, Mitra, et al. 1998) provided the first meta-analysis of the psychological literature on the impact of financial incentives on performance and concluded, based on analysis of 47 studies that these incentives resulted in a 12% improvement in performance quantity and a negligible effect on performance quality. The study however had several limitations as it did not distinguish between types of incentive programs, complexity of tasks, and differences in organizational settings.

Building on the Jenkins study, (Condley, Clark, et al. 2003) conducted a meta-analysis of 64 field and laboratory experiments, as well as observational studies, on the impact of monetary and non-monetary individual and group incentives on performance. The criteria for inclusion of studies in the analysis was that studies had to (a) have a use a control group or a pre-treatment measure of average performance; (b) involve the use of incentives to enhance performance; and (c) and report some statistical data. The studies included private sector settings, public sector, as well as laboratory experiments with college students. Within the public sector, however the vast majority were of schools, with only one study in a core government setting, thereby greatly limiting the generalizability of the findings to the core civil service. Importantly, the analysis also distinguished between studies that looked at cognitive (38 studies) versus mechanical manual tasks (26 studies), and the measurability of the tasks, looking at both quantitative and qualitative performance targets. The results of the meta-analysis showed that employees and other research participants who received performance incentives achieved an average 22% increase in work performance. The findings were irrespective of the settings, although again it should be noted that there was only one study of performance incentive in a government agency. Monetary incentives were found to be more effective than non-monetary gifts, and group-based incentives were significantly more effective (48% increase in performance) than individual incentives (19% increase). While the incentives worked for both cognitive and physical tasks, the gains were higher for the latter (30% increase compared to 20% increase). No significant difference was found based on the measurability of the tasks.

(Weibel, Rost, et al. 2009) conduct a meta analysis of 46 high quality empirical studies published in the fields of economics and psychology and covering both simple and complex tasks, and with both quantitative and qualitative outcome measurements. Overall the study finds a statistically significant and positive effect of pay for performance on performance; however, and in contrast to the above, the findings were significantly positive for simple tasks and smaller, but still statistically significant, negative effect for complex tasks. The authors argue that this negative effect is due to the reduction in intrinsic motivation brought about by the incentive scheme.

21
# **_Laboratory experiments_**

Building on these psychological studies, a number of behavioral economists have done laboratory experiments to explore the different aspects of performance pay, including the functional relationship between bonus sizes and performance, the incentive and sorting affects of performance pay, the impact of different types of bonuses, and the possible tradeoffs between extrinsic and intrinsic motivation.

(Ariely, Gneezy, et al. 2009), explore the effect of bonus size on performance in laboratory experiments using subjects in the US and India, with 24 and 87 participants respectively. Participants had to solve cognitive tasks under time pressure and were incentivized with bonuses that varied from small to large relative to their normal pay. They found evidence for an ―inverse-U‖ relationship between bonus size and performance, with the ―choking-under-pressure'' effect where bonuses at very high levels lead to a worsening of performance compared to bonuses at low and moderate levels.

An experiment with 115 Australian students that tried to distinguish the potential incentive and sorting effect of performance pay found supportive evidence for both hypotheses (Cadsby, Song, et al. 2007). In addition they found not only that low productivity subjects were less likely to sort into pay-forperformance jobs, but also that subjects with higher levels of risk-aversion avoided pay-for-performance, suggesting important unintended side effects. The experimental comparison of piece-rate, team rewards and relative performance schemes found that piece-rate systems and team rewards overall induce similar effort levels (free-riding in team rewards is compensated by higher effort contributions). Effort was higher, but also more variable in tournament-based reward systems. The experiment also revealed that subjects' attitudes about the varying reward systems differed widely (van Dijk, Sonnemans, et al. 2001). (Straberg 2010) showed in an empirical study concerning the perceived impact of performance-related pay in Sweden, that men were much more likely to see the arrangement as fair and reasonable.

Tackling the problem of multi-dimensionality of many tasks, (Fehr and Schmidt 2004) conduct an experiment with university students to understand the effects of varying bonus schemes on effort provision on two distinct tasks, only one of which is contractible. They find that simple piece-rate contracts lead to a focusing on the contractible task, while bonus arrangements designed to be more encompassing and to explicitly address the multi-tasking problem also induce participants to spend time on the second task.

The issue of extrinsic versus intrinsic motivation and how performance pay can change the perception of salary arrangement is the subject of a laboratory experiment by (Gneezy and Rustichini 2000). They use high school and university students in Israel and offer them different size of bonuses for specific tasks. The results suggest that subjects showed higher levels of productivity when offered large rewards, but small awards led to worse performance than offering no monetary reward at all. This suggests the importance of framing of performance pay — if bonuses adequately communicate the importance of performing assigned tasks well compared to the overall goals of the organization, they can work, but if bonuses trigger a change in evaluation of the worker relationship, crowding-out of intrinsic motivation can worsen productivity.

An interesting laboratory experiment recruits future teachers in India to assess the possibility of gaming effects under performance pay. The experiment assesses teacher efforts when rewards are a function of average student test scores. In a situation with strong social heterogeneity and prejudice, teachers might focus on assisting high-status students, while neglecting lower caste pupils. The experiments reveals that poorly designed incentive plans lead to such a misallocation of teacher effort, which produces unequal 22
distribution of effort across student groups, but properly designed incentives can mitigate such behavior (Jain and Narayan 2011).

# **_Public sector craft jobs_**

The evidence from experimental studies for public sector craft jobs is basically similar to that from observational studies – there is generally a positive impact but the limitations are also similar, in that the evidence is largely from OECD countries with the significant exception of teaching and health care where several significant experiments have been undertaken in developing countries.

# **Tax administration, job placement**

(Burgess, Propper, et al. 2010) use a randomized controlled trial (RCT) to examine the impact of a pilot team-based incentive scheme introduced in 2002 in Her Majesty‘s Customs and Excise (HMCE), the indirect tax assessment and collection agency of the UK government. Each team consisted of a small number of tax offices, and ranged from 150 to 280 workers; there were two treatment teams that received two different types of incentive payments — one a bonus that was a fixed percentage of the officer‘s salary and the other a flat rate bonus — and a control team. The incentive scheme for both the treatment teams consisted of meeting a set of targets on revenue collection and conduct of audits, with an average bonus size of approximately 3% of annual salary. If the target was met than all staff in the team received the bonus. The authors‘ use detailed data from the HMCE‘s performance management system and personnel records to show that the tax yield increased for both the treatment teams relative to the control group, and that these increases were due to more time spent auditing that resulted in the recovery of greater tax revenue. The study also found that the strategies of the two incentivized teams were different, with the managers in team 2 allocating more incentive tasks to efficient workers than managers in team 1. Whether the flat rate bonus structure contributed to this task allocation was not examined.

(Burgess, Propper, et al. 2011) also did another RCT to examine the impact of a team-based incentive scheme introduced in a large UK public agency, Jobcentre Plus, which is tasked with placing the unemployed into jobs and administering welfare benefits. Each team consisted of a district, which had several offices, ranging from 250 to 1500 staff. There were 17 treatment districts and 73 control districts. The incentives were based on achieving both quantitative and qualitative targets — number of individuals placed in jobs, customer and employer service, and reducing benefit calculation error and fraud. The job placement target was weighted by the types of clients who found employment (e.g. highest weights to an unemployed single parent) with extra points if the person retained employment. While individuals worked in offices, the targets were set at the district level, which consisted of many offices that operated independently of each other, a potential flaw in the schemes design. The study was explicitly designed to assess the impact of incentives given multi-dimensionality of tasks, as well as possible free-riding given the nature of the team incentive.

The study‘s findings were that while overall there was little difference between the treatment group and the control group on job placement, in smaller teams (fewer offices per district, and smaller offices) the incentives resulted in 10% greater job placements than in the control group, with the effect declining for larger districts such that there was a significant negative impact on productivity for the largest quartile of districts. This suggests considerable free-riding behavior in larger teams. The authors also found that none of the quality measures were significant in the treatment group, irrespective of team size, suggesting that measurement problems were important.

23
# **Teaching**

A number of field experiments have evaluated the impact of performance pay for teachers on reducing absenteeism and improving learning outcomes. The findings are generally mixed. In a study, (Duflo, Hanna, et al. 2010) show that random assignment for monitoring and financial incentives for teachers in rural India led to a strong reduction of teacher absenteeism and increased students' test scores by approximately 0.17 standard deviations. Units of observation for the study were single-teacher schools run by an NGO. The NGO selected 120 schools for testing the monitoring and incentive program, 60 which were randomly assigned to the treatment group. In the treatment groups, teachers had to use tamper-proof cameras to document their classroom presence at the start and end of each school day. Teacher salary was then made a function of the days in attendance, ranging from 50% to 130% of the pay in the fixed wage control group.

(Kremer and Chen 2001) by contrast show that subjective monitoring arrangements by an individual in the institutional hierarchy (like the headmaster of a school) may not work in developing country settings because the monitor might shirk, try to avoid confrontation, or collude with the workers. In Kenya, the Early Childhood Education Project offered substantial material incentives to teachers (bicycles) with good attendance as reported by their supervisor. Yet, the study found no effect of this program on absences as there was considerable cheating. In every school, the headmaster reported sufficient attendance for the teacher to receive the prize; however, when the research team independently verified absence through unannounced visits in both treatment and comparison schools, they found that the absence rate was actually exactly at the same high level in treatment and in comparison schools. This and Duflo‘s study suggest that impersonal, external monitoring by a camera coupled with a clear, credible, and automatic threat of punishment and promise of reward was the key design feature for program success.

Studies of performance pay linked to student outcomes are also similarly mixed. A field experiment in 50 Kenyan schools linking teacher salaries to student test scores failed to find lasting effects (Glewwe, Ilias, et al. 2010). Teacher attendance did not improve; teachers did not adjust their teaching methods or conduct more preparation sessions. Students in treated schools did perform better during the program duration, but these gains did not carry beyond the study period.

A field experiment conducted in NYC public schools also failed to find statistically significant effects of team incentives for teachers on student outcomes (Fryer 2011). In 2007 New York City launched a pilot program of financial incentives to teachers in 400 low-performing schools with the goal of improving student outcomes. There were explicit eligibility criteria for schools to be part of the pilot program; about half were randomly selected to receive treatment, which consisted of a school bonus of $3,000 per staff member if certain standards are reached. The study did not find any effect of the financial incentives on teacher or student behavior. The surprising null finding is seen to be potentially produced by a lack of strong individual incentives, driven by the small size of the average bonus (4% of annual teacher salary), free-riding of teachers within schools and the overall complexity of the incentive scheme. A related study that also assesses the effects of the NYC group incentive program on classroom activities and teacher turnover and qualification, apart from test scores and teacher effort, similarly finds no effects (Goodman and Turner 2010). A three-year experimental evaluation of the Project on Incentives in Teaching (POINT) in Metropolitan Nashville schools also found no significant effects of bonus incentives on student test scores (Springer, Ballou, et al. 2010).

On the other hand a large-scale field experiment in a representative sample of 300 government-run rural primary schools in India found that bonus pay linked to the mean improvement of student test scores in an independent learning assessment led to a statistically significant and substantively meaningful 24
improvement of student outcomes (Muralidharan and Sundararaman 2009). In the treatment group scores were higher by 0.28 standard deviations in math tests and by 0.16 standard deviations in language tests, across ―conceptual‖ and ―mechanical‖ parts of the test. They also find positive spill-over effects to subjects not part of the official student assessment.

# **Health sector**

A number of randomized-controlled trials have been implemented to determine the role of performance pay on health worker productivity, patient treatment and outcomes. Similar to studies on healthcare relying on observational data, the majority of studies assess these questions in the context of OECD health care systems. (Prentice, Burgess, et al. 2007) point out that improving quality rather than quantity of output is the primary focus for many, if not all, performance pay schemes implemented in the healthcare sector.

One of the first studies to employ an experimental design was work on performance incentives for nursing homes (Norton 1992). He finds that nursing homes assigned to the treatment group show better resident health outcomes and shorter stays.

(Kouides, Bennett, et al. 1998) implement a randomized-controlled trial, offering a randomly selected set of primary care physicians financial incentives based on influenza immunization rates of the elderly, as part of a Medicare demonstration project. Doctors in the treatment group were eligible to receive a $0.80 payment per shot for an immunization rate of 70% and $1.60 for each shot, if an immunization rate of 85% was attained. The experiment finds a difference of 7% in the immunization rate between treatment and control groups.

On the other hand, (Hillman, Ripley, et al. 1998; Hillman, Ripley, et al. 1999) use two RCT designs to incentivize cancer screenings for women of age 50 and above and pediatric immunizations, respectively. In both studies the authors document no significant difference between treatment and control groups. Similarly, a RCT implemented by (Grady, Lemkau, et al. 1997) finds no clear effects of financial incentives on mammography referrals by primary care physicians.

Contrastingly, a set of studies (Fairbrother, Hanson, et al. 1999; Fairbrother, Siegel, et al. 2001), also focusing on pediatric immunizations, finds that performance incentives increased immunizations rates by several percentage points compared to the control group. A randomized field trial at the clinic-level found that financial incentives improved treatment of smoking cessation outcomes (Roski et al. 2003). Work on performance pay for cognitive services interventions by pharmacists also finds positive effects (Christensen et al. 2000).

To our knowledge, the only two available randomized-controlled trials on performance pay in health care in a low income country are a study by Basinga et al. (2010) in Rwanda and a study by Singh (2010) in India. Basinga et al. use an RCT design to evaluate performance pay in Rwandan primary health care centers. The authors took advantage of a sequenced roll-out of the scheme across Rwandan health care facilities, collecting data on child preventive care and prenatal delivery. To isolate the performance-pay effect from a general increase in resources, comparison facilities received an equivalent increase in their budgets. The study uses information from 166 facilities and 2158 households. They find large effects on all central outcome measures, but with particularly striking effects for services with the highest payoffs and smallest necessary staff effort.

25
(Singh 2010) treated three groups of mothers and staff providing child care and nutritional advice to them in Chandigarh, India: in one group the workers received with performance pay; in a second group the workers had no performance pay but the women that they worked with were separately given factual information about nutrition; and the third group received both treatments. The study found that children‘s weights improved only in the third group compared to the control group.

It is noteworthy that nearly all studies on the health care sector so far focus on fairly narrow types of performance pay and specific, single outcome measures in preventative care, not necessarily overall multidimensional patient treatments and outcomes.

## **_Private sector: Craft or coping jobs_**

The evidence from experimental studies for private sector craft or coping jobs is largely from OECD countries. In a field experiment from the private sector (Bandiera, Barankay, et al. 2006), some managers were treated with the introduction of a performance-pay system and productivity of lower-tier workers was used as an outcome measure. The study finds evidence of both an incentive and sorting effect, i.e. managers support their high-productivity workers and fire the least qualified employees. Evidence from a quasi-experiment in a private sector company found that the introduction of a new appraisal system that feeds into performance pay can improve trust in top-level management (Mayer and Davis 1999). An experimental treatment of monitoring efforts by management in a call center found that employees largely behave according to a rational-cheater model of human behavior, highlighting the importance of performance measurement, but at the same time a substantial portion of employees remains unaffected by monitoring attempts (Nagin, Rebitzer, et al. 2002).

# **5. Assessing the evidence**

To assess the overall evidence, the 152 studies that were reviewed in this paper (see the Appendix for a list) were grouped into three categories: _positive_ if their findings provide positive evidence for the effectiveness of incentive schemes;<sup>8</sup> _neutral_ if the study is largely descriptive or finds contradicting evidence; and _failed_ if the evidence indicates no effect or negative effect of performance pay. Figure 1 shows the overall frequency of results. A majority of studies (93 out of the 152) presents supportive evidence for some form of effect of performance pay schemes, with experimental studies showing more positive findings than observational ones.

In drawing lessons however, it is important to distinguish the findings more systematically by the research quality of the concerned study. Study quality was ranked in two different ways. First, each study was assessed for its ̳internal validity‘, or strength of the causal arguments being made, using a five-point ranking (from weak to strong) as follows:

1. no empirical study or faulty research design

2. descriptive; small sample size 3. secondary data analysis and/or descriptive data analysis; small sample size; some statistical analysis 4. quasi-experimental design; reasonable sample size; conclusions based on statistical analysis

> 8 Inevitably, there is some subjectivity in the classification of studies. Studies were rated as _positive_ if there was general evidence on the basic functionality of incentive schemes, even if additional results qualify the effect, e.g. studies on crowding-out of intrinsic motivation generally still find positive effects of explicit incentives.

26
5. laboratory experiments; randomized controlled trial; large sample size; strong statistical analysis; strong conclusions Second, studies were also evaluated on the dimension of ―external validity‖, or to what extent the causal connections drawn in the specific context of the study would remain valid if replicated in other contexts. So for example, lab experiments and RCTs offer very strong evidence about causality (high internal validity), but in a specific context — they tell us the average impact of a particular intervention in a particular location with a particular sample at a particular point in time. They are often accused of being low on external validity as the study subjects (usually college students in the case of laboratory experiments) are not representative of the general population, or in this case the population of interest (civil servants) and the requirements of the experiment imply very particular conditions that may not approximate real world settings.

**Figure 1: Aggregate findings on performance-related pay**

<!-- Start of picture text --> Aggregate findings (number of studies) Findings by study type<br>100 93 80<br>68<br>90 70 Failed Neutral<br>80 60<br>70 50<br>60 40 35<br>50<br>30<br>37<br>40 20 16 17<br>30 23 10 7 8<br>20 2 0 0<br>0<br>10 Observational Field experiment Lab experiment<br>0<br>Failed Neutral Positive<br>Number of studies<br><!-- End of picture text --> Figure 2 (left panel) shows the overall results by the measure of internal validity. The majority of high quality studies (score of 4 and above) show a positive finding, while a majority of the low quality study (score of 1 or 2) show a negative or neutral finding. When assessing the findings according to the external validity of studies (Figure 2 right panel) again more positive results are found in more externally valid studies.

**Figure 2: Findings by internal and external validity**

<!-- Start of picture text --> Findings by measures of internal validity Findings by measures of external validity<br>50 70<br>45<br>60<br>40 Failed Neutral Positive<br>35 Failed Neutral Positive 50<br>30<br>40<br>25<br>20 30<br>15<br>20<br>10<br>5 10<br>0<br>0<br>1 2 3 4 5<br>Low High<br>Ranking: 1 = lowest quality; 5 = highest quality<br>External validity<br>Number of studies Number of studies<br><!-- End of picture text --> 27
Parsing the evidence by job type, a majority of studies of craft and production jobs show positive results, while a majority of studies of coping jobs find negative or neutral results ( Figure 3 left panel). These limitations of PRP to the demanding contextual environments of coping jobs are in line with the theoretical arguments, though the total number of studies of coping jobs (16 in total in this review) are too few to allow generalizations with any degree of confidence. Analyzing studies of relevance to the public sector — i.e. craft or coping jobs only, or 110 studies in the review — by country context reveals that, counter-intuitively, the weight of the evidence is somewhat stronger for developing countries ( Figure 3 right panel). However, it must be emphasized that the number of studies for developing countries is low.

<!-- Start of picture text --> Figure 3: Findings by job type<br>Findings by job type Relevant studies (craft or coping job) by country<br>70 context<br>60<br>60 Failed Neutral Positive 50 46 Failed Neutral Positive<br>45<br>50 40<br>35<br>40<br>30<br>24<br>30 24 25 19<br>19 20 17<br>20 15 15<br>10 1 3 3 8 5 105 1 3<br>0 0<br>Prod. Job Craft Job Coping Job OECD Developing Country<br>Number of studies<br><!-- End of picture text --> When the quality of study (internal validity) is considered separately for studies of craft and coping jobs, again more rigorous research shows on average more positive findings.

Figure 4 illustrates the breakdown of results by study quality for relevant studies. The left panel shows that a majority of high quality studies (ranked 4 and 5 in the measure of internal validity) of craft and coping jobs show positive findings (46 out of 68) while a majority of low quality studies show negative or neutral results. However, the dearth of the evidence on coping jobs is even more apparent for high quality studies as the bulk of the literature has focused on craft jobs, in particular teaching and healthcare (right panel). The findings for craft jobs is generally more positive in these high quality studies in developing country contexts than in OECD contexts, although the number of such studies is very small and largely limited to health and education (Table 3).

# **6. Summary**

Overall the body of evidence paints a supportive picture of performance pay in craft jobs within the public sector, but less so for coping jobs. So, as incentive theory would predict, PRP seemingly has a greater role to play in jobs where the outputs are more readily observable, such as teaching and health care jobs, than it does in more general administration. That these are jobs where the day to day actions of staff are unobservable does not seem to be an obstacle — apparently confounding, at least in the short term, the behavioral economics concern about crowding out intrinsic incentives. It is also in relation to 28
craft jobs that there have been more observational and experimental studies in developing countries — and generally the evidence from those settings is more positive than in OECD settings.

Among the observational studies, work analyzing the introduction of performance pay in the private sector finds nearly uniform support for the effectiveness of explicit incentive schemes. Observational studies of PRP for craft jobs in the public sector highlight that measurability of effort and output is more difficult in that environment and present slightly more mixed findings, but it seems the better output measurement and performance assessment are, the better performance pay works. The study of Brazilian

**Figure 4: Findings for craft and coping tasks by research quality and country context**

<!-- Start of picture text --> Relevant studies by quality (total 109) Findings for high quality (4 and 5) relevent<br>40 studies only<br>34 50<br>35 Failed Neutral Positive 44 Failed Neutral Positive<br>45<br>30 40<br>25 35<br>30<br>20<br>25<br>15 12 12 20<br>10 9 7 9 7 15 12 10<br>5 5 5 10<br>5 1 2 0 1 1 5 0 0 2<br>0 0<br>1 2 3 4 5 Craft Job Coping Job<br>1= lowest quality; 5 = highest quality<br><!-- End of picture text -->

**Table 3: Findings of high quality craft and coping studies by sector and country context**

|||**Craft**|**jobs**||**Copin**|**g jobs**|
|---|---|---|---|---|---|---|
||_Education_|_Health_|_Tax_|_Other_|_Public_|_Private_|
|**_OECD_**|**19**|**27**|**2**|**7**|**1**|**1**|
|_Positive_|12|16|1|6|1|1|
|_Negative or neutral_|7|11|1|1|0|0|
|**_Developing country_**|**6**|**4**|**1**|0|0|0|
|_Positive_|4|4|1||||
|_Negative or neutral_|2|0|0||||
|**Total**|**25**|**31**|**3**|**7**|**1**|**1**|

tax collectors is a good example of public sector work in which performance pay can be successfully implemented. Equally, Lavy's study of Israeli teacher competition (Lavy 2009) shows the ability of a well-designed performance scheme to function properly. At the same time, several observational studies identify problems with unintended consequences, generically subsumed under ―gaming‖ the incentive scheme, that can run counter to the original intentions of the reforms. With the current state of evidence though it remains unclear whether incidents of gaming have a net negative effect in the presence of increased productivity. Furthermore, while explicit incentive schemes certainly increase the opportunity for gaming, standard civil service arrangements have their own unintended incentive effects, i.e. employees will engage in behavior that increases the chances of easy work assignments or promotions and it is simply unknown whether existing forms of gaming are worse than similar behavior under performance pay.

Moving to studies that attempt to fulfill the gold standard of experimental design, the evidence overall again speaks in favor of the potential utility of performance pay for craft jobs. Comparing various laboratory experiments, the results suggest that indeed explicit performance incentives can work, but the studies employ easily measurable performance indicators and use fairly unrepresentative subject pools.

29
Both concerns should caution policy makers against accepting the results independently of other research. On the other hand, similar results have been found across a varied set of experimental settings, test locations and subject pools and overall findings do resonate with the observational literature, improving overall credibility and external validity.

The strongest form of evidence comes from field experimental studies for craft jobs that neatly address concerns of internal and external validity. Here, evidence is somewhat more mixed. Several studies of teacher incentive programs found no or transient effects of bonus pay systems in the context of US schools but in the developing world, evidence has been more positive. The discrepancy between teacher incentives in the developed and developing world could on the one hand stem from the relative magnitude of incentives compared to normal salary or on the other hand come from higher marginal effects in the education production function in developing countries. Many factors enter the production of education, all of which are likely lacking in many developing country schools. Improving one input aspect, e.g. teacher presence and effort, could have conceivably larger marginal effects than the same input improvement in a developed country school.

However, three limitations of these research findings loom large. First the number of high quality studies overall is limited, and the number for developing countries is particularly low. This is particularly the case for coping jobs reflective of the work of the core public administration for which there is at this stage simply not enough evidence on whether or not PRP can work. This small pool of research studies makes it hard to disagree with the observation that: ―much of the early evidence is not very robust. Consequently, a meta-analysis in which it is concluded that the ―bulk‖ of the evidence shows this or that cannot be provided as — despite the strong positions taken by the proponents or opposers of attempts to incentivise the public sector — there is no bulk‖ (Prentice, Burgess, et al. 2007, pp.12-13).

Second, the studies inevitably tend to look at the impact of PRP in the short term. Is it possible that the positive results are the result of Hawthorne Effects or new incentives which degrade over time as attitudes towards work change? It is generally accepted that ―(p)erformance related pay requires a long list of supportive local conditions before it stands a good chance of working as intended‖ (Pollitt and Dan 2011b, p.46) and it seems possible that the supportiveness of those conditions may only emerge over the longer term. In particular, to fully explore the persistent concern about crowding-out of intrinsic motivation, long term studies are necessary to assess how the design of incentive schemes and how they are communicated to employees minimizes the risk of employees changing attitudes toward their task.

Third, existing research exclusively focuses on the mechanical aspects of performance pay reforms, i.e. how to measure performance, how to design the incentive scheme and how large the bonus should be. No study explicitly considers contextual factors that can affect the operation of even the most well-designed incentive schemes. Most glaringly, the role of politicized bureaucracies has not been addressed properly. One, we need a better understanding of how politics can subvert flexible pay reforms and create important unintended side-effects and it is worth investigating how performance pay can potentially be used as a tool in de-politicizing the civil service and increase transparency and meritocracy. Interestingly, the move away from individualized pay in various area of the civil service, to uniform salary scales, was partly driven by the prevalence of patronage politics and clientelistic linkages that permeated the civil service in 19th century Western countries. Research could draw on this rich historical data and contemporary experience of civil service reform in developing countries to inform our understanding of performance pay. Apart from the influence of political parties or influential patrons, the role of civil service unions needs more attention. While most unions oppose in principle performance pay as a threat to union power, a better understanding of unions in the reform initiation and implementation could vastly improve future reform attempts. Countries vary dramatically in their wage bargaining structures and the strength of 30
unions, introducing performance pay in one scenario is likely to have vastly different effects than in others.

Further studies will need to broaden the coverage of developing countries and review the evidence over a longer time scale. If they continue to show a broadly positive association between PRP and performance in craft jobs, they will also need to shed light on a number of important design questions concerning individual versus group rewards, size of bonus, and types of performance measures used. They will also need to unpack the political process underlying reform initiation, <mark>implementation and sustained support for reforms — are the ideological or other pressures for PRP at odds with the long term fine-tuning which is undoubtedly necessary to maintain any positive results?</mark> <mark>Pending that further research, the guidance for practitioners seems to be that PRP for craft jobs is a feasible possibility for performance improvement and has some evidence behind it — but that careful design and piloting, accompanied by a technical willingness and a political ability to change direction is key. For coping jobs however, there is at this stage simply not enough robust evidence to draw any conclusions either way.</mark> This review has examined almost exclusively the effects of PRP on effort as that has been the focus of much of the theoretical and empirical literature. However, there is a small policy literature that also points to potential agency level and broader public sector wide effects that merit further more rigorous empirical examination. PRP‘s contribution to agency level productivity through the sorting channel — attracting higher quality workers who are likely to do better in this scheme — has already been discussed, and there is some fairly robust evidence of this in developed country contexts (Booth and Frank 1999; Bandiera, Barankay, et al. 2006; Cadsby, Song, et al. 2007). Individuals sorting into jobs with performance pay are on average higher educated, more qualified, less risk-averse and male. In a comprehensive laboratory experiment that features the self-selection of participants into performance schemes and also compares fixed wage, piece-rate and tournament-based rewards, researchers found clear incentive effects for variable payment schemes, driven by sorting of individuals. Individual-level characteristics like selfassessment, risk preferences, gender and social preferences systematically predict sorting decisions (Dohmen and Falk 2007).

In consequence, it is possible that PRP would have a clearer effect on individual incentives if the sorting effect had longer to work its way through the system. To evaluate this effect however, future studies need a longer time-frame to assess changes in recruitment and work-place behavior.

It is important to note that in OECD countries, performance-related pay has been introduced at a time of increasing ―work intensification‖ through control measures such as the use of ICT to track individual production and working habits (Green 2001; Beynon, Grimshaw, et al. 2002). These are somewhat hard drivers of changes in what Marsden refers to as the ―effort bargain‖ (Marsden 2004) — the implicit understanding between staff and management about ―how hard we work around here‖. PRP can offer an additional and less coercive point of entry into renegotiating this bargain. (Marsden 2004) cites two examples of changes in working practices that were sought in hospital management and in the tax service, both in the UK. In both cases, the task of implementing the changes fell to managers who were under pressure from their staff to be lenient with work assignments and generous with pay increases. In both cases, he argues, that individual incentives were only a modest part of the function of performance-related pay. Its real contribution was ―to enable management to redefine the established performance norms in their organization, and then to obtain effective compliance with those norms, with the explicit or tacit agreement of as many employees as possible‖ (Marsden 2004, p.351). In sum, by incrementally ratcheting up the performance expectations through the many thousands of performance appraisal discussions, the informal agency working culture was changed.

31
PRP can also be considered as an element in overall cost containment as it provides the employer with the additional tactical option of proposing that pay increases should only be provided in the form of (less costly) enhanced performance bonuses (Marsden and French 1998). (French 2005) identifies the significant additional control that performance-related pay gave managers within the UK Inland Revenue in the early 1990s, allowing them to cease automatic cost-of-living allowances, largely restrict additional pay increases to the performance component of pay, combining the latter with a de facto forced curve for allocating performance-based rewards. (French 2005) also highlights how individual negotiations within a performance-related pay scheme in the UK allowed managers to sever the previous linkages between the level of complexity of the tasks required of staff and their grade. Under the performance-related pay scheme, targets were negotiated individually and not by grade.

Finally, at least in OECD countries, PRP has usually been accompanied by other changes in pay policy, of which the two noteworthy ones are differentiation - creating pay differences within and across government agencies for civil servants doing the same job, based on the need to attract and retain qualified staff for these jobs; and delegation of pay-setting authority away from a central civil service agency to ministries, agencies, or departments. The linkages between PRP and pay differentiation and delegation remain underexplored. For example, it is possible that delegation is reported to lead to PRP since managers have an incentive to look for mechanisms that maximize the results can be achieved at minimum cost.

32
# **Appendix A: List of empirical PRP studies reviewed**

||**Authors**|**Year**|**Type**|**Quality**|**Effect**|**Country conte**|**xt Type of Job**|
|---|---|---|---|---|---|---|---|
|1|Aboody et al|2010|observational|4|positive|OECD|Coping|
|2|Amundson et al|2003|observational|3|positive|OECD|Craft|
|3|Ariely et al.|2008|lab experiment|5|positive|Developing|Production|
|4|Asch|1990|observational|4|positive|OECD|Craft|
|5|Atkinson et al.|2004|observational|4|positive|OECD|Craft|
|6|Ballou|2001|observational|3|neutral|OECD|Craft|
|7|Bandiera, Barankay and Ras|2005|observational|4|positive|OECD|Production|
|8|Bandiera, Barankay and Ras|2006|field experiment|5|positive|OECD|Production|
|9|Basinga et al|2010|field experiment|5|positive|Developing|Craft|
|10|Beaulieu and Horrigan|2005|observational|4|positive|OECD|Craft|
|11|Beer and Cannon|2004|observational|2|failed|OECD|Production|
|12|Beer and Katz|2003|observational|3|positive|OECD|Production|
|13|Belfield and Marsden|2003|observational|4|positive|OECD|Production|
|14|Bender and Elliott|2003|observational|4|positive|OECD|Unclassified|
|15|Bertelli|2006|observational|4|neutral|OECD|Craft|
|16|Blasi et al|2008|observational|4|positive|OECD|Craft|
|17|Booth and Frank|1999|observational|4|positive|OECD|Production|
|18|Brudney and Condrey|1993|observational|3|positive|OECD|Coping|
|19|Burgess et al|2010|field experiment|5|positive|OECD|Craft|
|20|Burgess et al|2004|observational|4|positive|OECD|Craft|
|21|Burks, Carpenter and Goett|2009|field experiment|5|positive|OECD|Production|
|22|Cadsby, Song and Tapon|2007|lab experiment|5|positive|OECD|Production|
|23|Camerer et al.|1997|observational|4|neutral|OECD|Production|
|24|Campbell et al|2007|observational|4|positive|OECD|Craft|
|25|Campbell et al|2005|observational|3|positive|OECD|Craft|
|26|Cardona|2007|observational|1|failed|OECD|Unclassified|
|27|Casalino et al|2003|observational|4|positive|OECD|Craft|
|28|Chalkley et al|2010|observational|4|positive|OECD|Craft|
|29|Christensen et al|2000|field experiment|5|positive|OECD|Craft|
|30|Chung et al|2007|observational|3|failed|OECD|Craft|
|31|Clarketal|1995|observational|4|neutral|OECD|Craft|
|32|Clotfelter et al.|2004|observational|4|failed|OECD|Craft|
|33|Clotfelter et al.|2008|observational|4|positive|OECD|Craft|
|34|Clotfelter, Ladd and Vigdor|2007|observational|4|positive|OECD|Craft|
|35|Cohn|1996|observational|2|neutral|OECD|Craft|
|36|Coleman et al.|2007|observational|4|positive|OECD|Craft|
|37|Condrey and Brudney|1992|observational|3|neutral|OECD|Coping|
|38|Cooper and Cohn|1997|observational|4|positive|OECD|Craft|
|39|Courty and Marschke|2003|observational|4|positive|OECD|Craft|
|40|Daley|1993|observational|3|positive|OECD|Coping|
|41|Dee and Keys|2004|observational|4|positive|OECD|Craft|
|42|Dohmen and Falk|2007|lab experiment|5|positive|OECD|Production|
|43|Doran et al.|2006|<br>observational|1|positive|OECD|Craft|
|44|Dowling and Richardson|1997|observational|4|positive|OECD|Coping|
|45|Duflo, Hanna and Ryan|2007|field experiment|5|positive|Developing|Craft|
|46|Eberts, Hollenbeck and Ston|2002|observational|4|failed|OECD|Craft|
|47|Eichler and Levine|2009|observational|4|positive|Developing|Craft|

33
||**Authors**|**Year**|**Type**|**Quality**|**Effect**|**Country conte**|**xt Type of Job**|
|---|---|---|---|---|---|---|---|
|48|Eichler, Auxila and Pollock|2001|observational|2|positive|Developing|Craft|
|49|Eriksson and Villeval|2004|lab experiment|5|positive|OECD|Production|
|50|Fairbrother et al|1999|field experiment|5|positive|OECD|Craft|
|51|Fairbrother et al|2001|field experiment|5|positive|OECD|Craft|
|52|Fehr and Goette|2007|field experiment|5|positive|OECD|Production|
|53|Fehr and Schmidt|2004|lab experiment|5|positive|OECD|Production|
|54|Felt-Lisk et al|2007|observational|4|neutral|OECD|Craft|
|55|Figlio and Kenny|2007|observational|4|positive|OECD|Craft|
|56|Figlio and Winicki|2005|observational|4|neutral|OECD|Craft|
|57|Fryer|2011|field experiment|5|failed|OECD|Craft|
|58|Gavaghan et al|2010|<br>observational|4|failed|OECD|Craft|
|59|<br>Gerhart and Milkovich|1990|observational|4|positive|OECD|Production|
|60|Glewwe, Ilias and Kremer|2010|field experiment|5|neutral|Developing|Craft|
|61|Glickman et al|2007|observational|4|neutral|OECD|Craft|
|62|Gneezy and Rustichini|2000|lab experiment|5|positive|OECD|Production|
|63|Goldhaber and Brewer|2000|observational|4|positive|OECD|Craft|
|64|Goodman and Turner|2010|field experiment|5|failed|OECD|Craft|
|65|Grady et al|1997|field experiment|5|failed|OECD|Craft|
|66|Gratz|2009|observational|2|neutral|OECD|Craft|
|67|Grimshaw|1998|observational|1|failed|OECD|Unclassified|
|68|Hamilton, Nickerson and Ow|2003|observational|4|positive|OECD|Craft|
|69|Heckman, Heinrich and Smit|1997|observational|4|neutral|OECD|Production|
|70|Heneman and Milanowski|1999|observational|3|neutral|OECD|Craft|
|71|Hillman et al|1998|field experiment|5|failed|OECD|Craft|
|72|Hillman et al|1999|field experiment|5|failed|OECD|Craft|
|73|Hillman et al|1991|observational|3|neutral|OECD|Craft|
|74|Hochberg and Lindsey|2010|observational|4|positive|OECD|Craft|
|75|Ingraham|1993|observational|1|failed|OECD|Unclassified|
|76|Jack|2003|observational|2|positive|Developing|Craft|
|77|Jacob|2005|observational|4|positive|OECD|Craft|
|78|Jacob and Levitt|2003|observational|4|neutral|OECD|Craft|
|79|Jain and Narayan|2011|lab experiment|5|positive|Developing|Craft|
|80|Kahn, Silva and Ziliak|2001|observational|4|positive|Developing|Craft|
|81|Kelley|1999|observational|2|neutral|OECD|Craft|
|82|Kellough and Nigro|2002|observational|3|failed|OECD|Unclassified|
|83|Kellough and Selden|1997|observational|3|neutral|OECD|Unclassified|
|84|Kerr|1975|observational|1|failed|OECD|Unclassified|
|85|Ketelaar, Manning and Turk|2006|observational|3|neutral|OECD|Unclassified|
|86|Kim|2002|observational|2|neutral|OECD|Unclassified|
|87|Kingdon and Teal|2008|observational|4|positive|Developing|Craft|
|88|Kiragu and Mukandala|2003|observational|2|neutral|Developing|Unclassified|
|89|Knez and Simester|2001|observational|4|positive|OECD|Production|
|90|Koretz|2002|observational|1|failed|OECD|Craft|
|91|Kremer and Chen|2001|field experiment|5|failed|Developing|Craft|
|92|Kouides et al|1998|field experiment|5|positive|OECD|Craft|

34
||**Authors**|**Year**|**Type**|**Quality**|**Effect**|**Country conte**|**xt Type of Job**|
|---|---|---|---|---|---|---|---|
|93|Ladd|1999|observational|4|positive|OECD|Craft|
|94|Lavy|2008|observational|4|positive|OECD|Craft|
|95|Lavy|2009|observational|4|positive|OECD|Craft|
|96|Lazear|2000|observational|4|positive|OECD|Production|
|97|Lazear|2003|observational|1|positive|OECD|Production|
|98|Le Grand|2007|observational|2|positive|OECD|Unclassified|
|99|Leford, Lawler and Mohrma|1995|observational|1|neutral|OECD|Production|
|100|Levin-Scherz et al.|2006|observational|4|positive|OECD|Craft|
|101|Li et al|2011|observational|4|neutral|OECD|Craft|
|102|Lindenauer et al|2007|observational|4|positive|OECD|Craft|
|103|Mandel and Kortagal|2007|observational|3|positive|OECD|Craft|
|104|Marsden|2004|observational|3|neutral|OECD|Unclassified|
|105|Marsden|2009|observational|1|neutral|OECD|Unclassified|
|106|Marsden and French|1998|observational|3|neutral|OECD|Unclassified|
|107|Mayer and Davis|1999|field experiment|5|positive|OECD|Production|
|108|McMenamin et al|2003|observational|4|positive|OECD|Craft|
|109|McNamara|2005|observational|2|neutral|Developing|Craft|
|110|Meessen et al|2006|observational|4|positive|Developing|Craft|
|111|Meessen, Kashala and Musa|2008|observational|3|positive|Developing|Craft|
|112|Milkovich and Wigdor|1991|observational|2|positive|OECD|Production|
|113|Muralidharan and Sundarar|2009|field experiment|5|positive|Developing|Craft|
|114|Murnane and Cohen|1986|observational|1|failed|OECD|Craft|
|115|Nagin et al.|2002|field experiment|5|neutral|OECD|Production|
|116|Norton|1992|field experiment|5|positive|OECD|Craft|
|117|Odden and Kelley|2002|observational|2|positive|OECD|Craft|
|118|OECD|1996|observational|2|neutral|OECD|Unclassified|
|119|OECD|1997|observational|2|neutral|OECD|Unclassified|
|120|OECD|2005|observational|2|neutral|OECD|Unclassified|
|121|OECD|2008|observational|2|neutral|OECD|Unclassified|
|122|Paarsch and Shearer|1999|observational|4|positive|OECD|Unclassified|
|123|Pearson et al|2008|observational|4|failed|OECD|Craft|
|124|Pires|2007|observational|2|failed|Developing|Unclassified|
|125|Pourat et al|2005|observational|4|positive|OECD|Craft|
|126|Reid|1992|observational|2|positive|Developing|Unclassified|
|127|Rexed et al.|2007|observational|2|neutral|OECD|Unclassified|
|128|Rogers and Vegas|2009|observational|1|positive|Developing|Craft|
|129|Rosenthal et al|2005|observational|4|neutral|OECD|Craft|
|130|Roski et al|2003|field experiment|5|positive|OECD|Craft|
|131|Safran et al|2000|observational|4|positive|OECD|Craft|
|132|Schick|1998|observational|1|failed|OECD|Unclassified|
|133|Shaw, Gupta and Delery|2002|observational|4|positive|OECD|Production|
|134|Shearer|2004|field experiment|5|positive|OECD|Production|
|135|Shen|2003|observational|4|failed|OECD|Craft|
|136|Singh|2010|field experiment|5|positive|Developing|Craft|
|137|Soeters and Griffiths|2003|observational|2|positive|Developing|Craft|
|138|Soeters et al|2006|observational|3|positive|Developing|Craft|

35
||**Authors**|**Year**|**Type**|**Quality**|**Effect**|**Country context **|**Type of Job**|
|---|---|---|---|---|---|---|---|
|139|Springer et al.|2010|field experiment|5|failed|OECD|Craft|
|140|Stajkovic and Luthans|2001|field experiment|5|positive|OECD|Production|
|141|Stazyk|2010|observational|4|positive|OECD|Unclassified|
|142|Steel et al|2007|observational|3|positive|OECD|Craft|
|143|Straberg|2010|observational|3|neutral|OECD|Craft|
|144|Streib and Nigro|1993|observational|3|neutral|OECD|Unclassified|
|145|Vaghela et al.|2009|observational|3|positive|OECD|Craft|
|146|van Dijk, Sonnemans and va|2001|lab experiment|5|positive|OECD|Production|
|147|Vegas|2005|observational|3|positive|Developing|Craft|
|148|Vegas and Umansky|2005|observational|3|positive|Developing|Craft|
|149|Willems, Janvier and Hende|2006|observational|2|neutral|OECD|Unclassified|
|150|Witter et al|2011|observational|3|neutral|Developing|Craft|
|151|Woessmann|2010|observational|4|positive|OECD|Craft|
|152|World Bank|2001|observational|3|positive|Developing|Craft|
|153|Younget al|2007|observational|4|positive|OECD|Craft|

36
# **Appendix B: List of High Quality Studies of Craft and Coping Jobs**

## **EDUCATION**

|_Co_<br>_Typ_|_untry Context_: OCED<br>_e of study_: Observational<br>|**Findings**<br>||
|---|---|---|---|
||**_Positive_**|**_Neutral_**|**_Negative_**|
|1.|Atkinson et al.(2004)|1. Figlio & Winicki(2005)|1. Clotfelter et al.(2004)|
|2.|Clotfelter et al.(2008)|2. Jacob & Levitt(2003)|2. Eberts et al.(2002)|
|3.|Clotfelter, Ladd,& Vigdor<br>(2007)|||
|4.|Cooper & Cohn(1997)|||
|5.|Dee & Keys(2004)|||
|6.|Figlio & Kenny (2007)|||
|7.|Goldhaber & Brewer(2000)|||
|8.|Jacob(2005)|||
|9.|Ladd(1999)|||
|10.|Lavy (2008)|||
|11.|Lavy (2009)|||
|12.|Woessman(2010)|||

|_Country Context_: OCED<br>_Type of study_: Experimental|||
|---|---|---|
||**Findings**||
|**_Positive_**|**_Neutral_**|**_Negative_**|
|No studies|No studies|1. Fryer(2011)|
|||2. Goodman and Turner(2010)|
|||3. Springer et al(2010)|
|_Country Context_: Developing<br>_Type of study_: Observational|||
||**Findings**||
|**_Positive_**|**_Neutral_**|**_Negative_**|
|1. Kingdon & Teal(2008)|No studies|No studies|

|_Country Context_: Developing<br>_Type of study_: Experimental|||
|---|---|---|
||**Findings**||
|**_Positive_**|**_Neutral_**|**_Negative_**|
|1. Duflo et al(2007)|1. Glewwe et al(2010)|1. Kremer & Chen(2001)|
|2. Muralidharan & Sundararaman<br>(2009)|||
|3. Jain & Narayan(2011)|||

37
# **HEALTH**

_Country Context_ : OCED _Type of study_ : Observational

|||**Findings**<br>||
|---|---|---|---|
||**_Positive_**|**_Neutral_**|**_Negative_**|
|1.|Beaulieu & Horrigan(2005)|1. Clark et al(1995)|1. Gavaghan et al.(2010)|
|2.|Campbell et al.(2007)|2. Felt-Lisk et al(2007)|2. Pearson et al.(2008)|
|3.|Casalino et al(2003)|3. Glickman et al(2007)|3. Shen(2003)|
|4.|Chalkleyet al(2010)|4. Li et al(2011)||
|5.|Coleman et al(2007)|5. Rosenthal et al(2005)||
|6.|Levin-Scherz et al(2006)|||
|7.|Lindenauer et al(2007)|||
|8.|McMenamin et al(2003)|||
|9.|Pourat et al(2005)|||
|10.|Safran et al(2000)|||
|11.|Younget al(2007)|||

|_Co_<br>_Typ_|_untry Context_: OCED<br>_e of study_: Experimental<br>|**Findings**<br>||
|---|---|---|---|
||**_Positive_**|**_Neutral_**|**_Negative_**|
|1.|Fairbrother et al(2001)|No studies|1. Gradyet al(1997)|
|2.|Fairbrother et al(1999)||2. Hillman et al(1998)|
|3.|Kouides et al(1998)||3. Hillman et al(1999)|
|4.|Norton(1992)|||
|5.|Roski et al(2003)|||

|_Country Context_: Developing<br>_Type of study_: Observational<br>**_Positive_**|**Findings**<br>**_Neutral_**|**_Negative_**|
|---|---|---|
|6. Eichler & Levine(2009)|No studies|No studies|
|7. Meesen et al(2006)|||

_Country Context_ : Developing _Type of study_ : Experimental

|||**Findings**||
|---|---|---|---|
||**_Positive_**|**_Neutral_**|**_Negative_**|
|8.|Basinga et al(2010)|No studies|No studies|
|9.|Singh(2010)|||

38
# **REVENUE ADMINISTRATION**

|_Country Context_: OECD<br>_Type of study_: Experimental or obs<br>|ervational<br>**Findings**<br>||
|---|---|---|
|**_Positive_**|**_Neutral_**|**_Negative_**|
|1. Burgess et al(2010)|1. Bertelli(2006)|No studies|
|_Country Context_: Developing<br>_Type of study_: Experimental or obs|ervational||
||**Findings**||
|**_Positive_**|**_Neutral_**|**_Negative_**|
|1. Kahn et al(2001)|No studies|No studies|

# **OTHER (Job placement, recruitment, private sector)**

_Country Context_ : OECD _Type of study_ : Experimental or observational

||**_Positive_**|**Findings**<br>**_Neutral_**|**_Negative_**|
|---|---|---|---|
|1.|Asch(1990)|1. Heckman et al(1997)|No studies|
|2.|Burgess et al(2004)|2.||
|3.|Courty& Marschke(2003)|3.||
|4.|Christensen et al(2000)|4.||
|5.|Blasi et al(2008)|5.||
|6.|Hochberg& Lindsey (2010)|6.||

# **COPING JOBS (Health managers, private sector)**

|_Country Context_: OECD<br>_Type of study_: Experimental or observ<br>**_Positive_**|ational<br>**Findings**<br>**_Neutral_**|**_Negative_**|
|---|---|---|
|1. Aboodyet al(2010)|No studies|No studies|
|2. Dowling& Richardson(1997)|||

39