approached questions of performance pay slightly more systematically, comparing several reform cases with each other and using convenience samples of employees and senior management, subject to performance pay, to collect data on self-assessed motivation and satisfaction with newly introduced performance pay. While attempting to provide a systematic evaluation of the successes and failures of performance pay reform, to a large extent these studies rely on weak research designs. Despite their descriptive value, inferences based on the single or comparative evaluation of reform cases, without the use of counterfactuals, cannot establish the existence or absence of potential program effects. Furthermore, utilizing convenience samples and using survey instruments that only measure self-assessed motivation suffer from selection and perception bias problems, while ignoring the information on actual outcomes.

Although the tenor of this initial wave of research was rather discouraging about the results on the effectiveness and popularity of performance pay among staff, incentive schemes for the public sector did not lose their appeal to policy makers (Marsden 2009). An intense empirical debate on teacher incentive was sparked in the context of local American school reforms and a national debate on test-score based school accountability. A series of papers by education economists used the opportunity of local reform attempts to collect data on samples of schools and students subject to new performance pay arrangements and comparable control populations. By measuring detailed public service outcomes, student's grades and test scores, drop-out rates and attendance records, plus detailed information on bonus programs, the quantitative evaluation of this new data allowed a more stringent test of the competing theoretical arguments. The advantage of these studies compared to prior work is their ability to disentangle the effect of teacher effort from other factors that also determine student outcomes.

While an improvement, many of these quantitative observational studies and similar work on private companies and the public service still fall short of an ideal research design for causal inference on program effects. The gold standard for program evaluation is the use of randomized-controlled trials (RCT), in which treatment assignment to subjects is randomized and unrelated to other observable and unobservable characteristics. This randomization allows the estimation of the treatment effect by a comparison of treated and control units. Observational studies on the other hand rely on treatment and control groups created not by controlled random assignment, but produced by social processes, often related to the research question at hand. Issues of selection bias and confounding factors can undermine the internal validity of such studies.

Utilizing the power of a randomization framework, several behavioral economists have used laboratory experiments to test hypotheses with regard to incentive schemes. Laboratory experiments offer at least two distinct advantages over observational studies: the researcher can use randomization to ensure unbiased and consistent estimation of treatment effects; and researchers can design their experiment to directly relate to theoretical questions at hand. The review of the theoretical literature has identified the importance of design details and the plethora of possibilities when it comes to performance pay arrangements. Observational studies have to rely on bonus pay schemes that have been implemented in real life, which do not necessarily cover all interesting variations and often mix different elements that conflate theoretical questions of interest. Being able to design a clean and tailored experiment to trace the effect of intrinsic versus extrinsic motivation in a linear bonus pay scheme for example is a huge advantage which laboratory experiments offer. A wave of articles in the late 1990s and 2000s explores various issues of performance pay. While offering certain advantages over observational studies, laboratory experiments often use notoriously small samples and student subjects that share few characteristics with actual workers or public servants. Furthermore, laboratory experiments can hardly ever replicate real work place settings or offer bonus schemes that remotely approach bonus sizes common even in only moderately incentivized performance pay schemes. This raises concerns from a sampling perspective and the representativeness of the subject pool, as well as the comparability of 12