the correct signal of the importance of the outcome while not putting the service providers at too much risk financially. If the RBF were to be 10 percent of the total payment it would likely provide enough incentive while not presenting too much of a risk financially if they failed to achieve the targeted outcome. A more rigorous analysis would need to be determined based on specific training programs before coming to a final number.

# **(iv) Acceptance of teaching practices as a measure**

The RBF could be based on different measures as discussed in the next area of consideration below. Any measure would need to be considered valid and reliable by the service providers since their payment would be based on the measure. They would also need to believe that they have sufficient control over affecting the measure through their service delivery, including the content they develop and the implementation of the training.

Teaching practices are an outcome that the service providers have only partial control over. They could conduct training that is of sufficient quality to instill the necessary knowledge and skills for the teacher, but it is ultimately up to the teacher whether she or he actually implements in the classroom what is learned.

There is also an issue of whether the teacher implements what is learned on the day or days of the observation. If a single lesson is observed, then there is less likelihood of detecting the change than if multiple lessons are observed. But there are costs involved in the observation. In the pilot, two full lessons were observed for each teacher and each lesson was made up of two segments.

In addition to the number of lessons observed for individual teachers, the sample size would also need to be sufficiently large to ensure an accurate and statistically significant result. This could be the Achilles heel of using lesson observation in an RBF scheme. For large-scale and prominent professional development activities, it may be worth the effort and cost to implement a lesson observation measure, but for smaller programs it would be very difficult to justify lesson observation to detect changes in the teaching practices. The cost considerations are explored in more detail later.

- **(v) Cost-benefit considerations of using RBF in professional development**

Implementation of an RBF scheme to service providers for professional development activities will entail additional costs. The extent of these costs will vary depending on the type of professional development and the extent and rigor of measurement, as well as the RBF incentive itself. The cost of implementing an RBF scheme would be dependent on several factors, including:

- **RBF as a top-up incentive vs. reducing the base payment:** Is the RBF bonus incentive a top-up to what would otherwise be offered or is the base payment reduced so that eventual overall payment (made up of base payment + bonus incentive) is equal to what would otherwise have been offered in the non-RBF scheme?

- **Cost of measures:** Because the RBF requires a measure or measures upon which the bonus would be provided, there would need to be additional activities for data collection. It could be argued that outcomes of training programs should be measured either with or without an RBF to ensure quality of professional development programs. This is in part what the Guangdong project’s QAME component would do. In this case, the cost would not be additional (or at the most it would be the additional cost required to have the extra level of rigor required upon which the bonus could be based).

- **Different costs depending on the type of measure:** Each type of measure will have its own costs. A lesson observation measure is among the most expensive, particularly if it involves capturing video as opposed to real-time observation. The use of tests to check

42