metadata: authors: 0: author: Hong,Seo Yeon
1: author: Cao,Xiaonan
2: author: Mupuwaliywa,Mupuwaliywa
count: Zambia
docna: 0: docna: Impact of Financial Incentives and the Role
            of Information and Communication in Last-Mile Delivery of
            Textbooks in Zambia
docty: Policy Research Working Paper
subtopic: Educational Populations,Education For All,Educational Policy and Planning - Textbook,Education for Development (superceded),Educational Sciences,Transport Services,Effective Schools and Teachers,Educational Institutions & Facilities,Health Economics & Finance
teratopic: Transport,Health, Nutrition and Population,Education
abstracts: cdata!: This study investigates the impact of
            financial incentives and role of information and
            communication in textbook availability (especially those in
            local languages) in Zambia. It uses the
            difference-in-difference estimation method to identify the
            causal links among the factors. The data used for the study
            were collected in 2017 and 2019 for baseline and end-line
            information. The study shows that providing financial
            incentives to schools increases the likelihood of a school
            receiving textbooks by 0.126. This improvement is mainly
            driven by raising the likelihood of a school collecting the
            books from zone-center schools and District Education Board
            Secretaries offices, by 0.356 and 0.158, respectively.
            Providing financial incentives to District Education Board
            Secretaries does not have any impact on the likelihood of a
            school receiving textbooks.  This is partly because the
            current textbook delivery practice in districts and
            communities relies heavily on the action of schools and less
            on District Education Board Secretaries. Thus, providing
            incentives directly to schools seems to intensify the
            current last-mile textbook distribution practice and has
            better results. In addition, providing information to
            schools on the availability of textbooks at District
            Education Board Secretaries offices improves the outcomes
            significantly with minimum cost.
geo_reg_key: 517195,517191
repnme: repnme: Impact of Financial Incentives and the Role
            of Information and Communication in Last-Mile Delivery of
            Textbooks in Zambia
display_title: Impact of Financial Incentives and
            the Role of Information and Communication in Last-Mile
            Delivery of Textbooks in Zambia
listing_relative_url: /research/2020/06/32191998/impact-financial-incentives-role-information-communication-last-mile-delivery-textbooks-zambia
topicv3: Teachers Management m1326119 1523,Results-Based Financing for Health
            m1326324 1298,Transport m1326661 135,Textbooks m1326802 2129,Education m1327048 470
publishtoextweb_dt: 2020-06-29T00:00:00Z
historic_topic: Transport,Health, Nutrition and Population,Education
origu: Education AFR 3 (HAWE3)
no_of_pages: 34
author_keys: 0: author_key: 1080882
1: author_key: 126311
2: author_key: 1277179
model_extractions: Policy Research Working Paper 9305 Impact of Financial Incentives and the Role of Information and Communication in Last-Mile Delivery of Textbooks in Zambia _Seo Yeon Hong Xiaonan Cao Mupuwaliywa Mupuwaliywa_ Education Global Practice June 2020
Policy Research Working Paper 9305

# **Abstract**

This study investigates the impact of financial incentives and role of information and communication in textbook availability (especially those in local languages) in Zambia. It uses the difference-in-difference estimation method to identify the causal links among the factors. The data used for the study were collected in 2017 and 2019 for baseline and end-line information. The study shows that providing financial incentives to schools increases the likelihood of a school receiving textbooks by 0.126. This improvement is mainly driven by raising the likelihood of a school collecting the books from zone-center schools and District Education Board Secretaries offices, by 0.356 and 0.158, respectively.

Providing financial incentives to District Education Board Secretaries does not have any impact on the likelihood of a school receiving textbooks. This is partly because the current textbook delivery practice in districts and communities relies heavily on the action of schools and less on District Education Board Secretaries. Thus, providing incentives directly to schools seems to intensify the current last-mile textbook distribution practice and has better results. In addition, providing information to schools on the availability of textbooks at District Education Board Secretaries offices improves the outcomes significantly with minimum cost.

This paper is a product of the Education Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at xcao@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Impact of Financial Incentives and the Role of Information and Communication in Last-Mile Delivery of Textbooks in Zambia**

Seo Yeon Hong, Xiaonan Cao, and Mupuwaliywa Mupuwaliywa

**JEL Code:** I20, I21, I22, I28

**Keywords** : Impact evaluation, financial incentive, results-based financing, textbook distribution, textbook availability, information and communication
# **Acknowledgments**

The authors would like to thank the Ministry of General Education in Zambia, especially those who were involved in this evaluation: Mr. Louis Mwansa, Director of the Department of Planning and Information; Mrs. Cecilia Sakala, Director of the Department of Standards and Curriculum Development; Mr. James Chilufya, Chief Curriculum Specialist at the Curriculum Development Centre; Mr. Kenneth Mapani, Head of the Procurement and Supplies Unit; and Mr. Jacob Masatunya, Procurement Specialist for the Zambia Education Enhancement Project.

We are grateful for their support and for their successful implementation of the financial incentive schemes as the designed interventions of this evaluation. Appreciation also goes to Dr. Chirspin Mphuka and Dr. Dale Mudenda as well as their team at University of Zambia for contributing to finalizing the design of this evaluation and for conducting baseline and end-line surveys.

The authors are grateful for the advice received from Education Practice Managers Halil Dundar and Safaa El Tayeb El-Kogali in the Africa Region of the World Bank; the guidance received from Samer Al-Samarrai and Hadi Khan of the REACH Trust Fund Administration; and general support from Elif Yonca Yukseker, in conducting this study.

ii
# **Announcement and Funding**

This study is sponsored by the Results in Education for All Children (REACH) Trust Fund. The goal of the study is to support Zambia’s Ministry of General Education (MoGE) in its effort in improving education quality and student learning in the country. In particular, this work aimed to help MoGE find out how to improve its textbook distribution and make textbooks more available to schools and students. All findings, interpretations, and conclusions expressed in this study are entirely those of the authors, and do not represent the views of the World Bank, its Executive Directors, or the countries they represent.

iii
# **Table of Contents**

|Acknowledgments ......................................................................................................................................... ii|
|---|
|Announcement and Funding ........................................................................................................................ iii|
|1.<br>Introduction ........................................................................................................................................... 1<br>|
|2.<br>Literature Review and Research Questions .......................................................................................... 2|
|3.<br>Textbook Procurement and Distribution in Zambia .............................................................................. 4|
|Procurement of Textbooks ........................................................................................................... 4|
|Distribution of Textbooks.............................................................................................................. 4|
|4.<br>Methodology: Intervention, Data, and Empirical Analysis Strategy .................................................... 6|
|Intervention .................................................................................................................................. 6|
|Data and Measurement ................................................................................................................ 7|
|Empirical Analysis Strategy ........................................................................................................... 9|
|5.<br>Findings............................................................................................................................................... 11|
|5.1. What is the current status of textbook distribution in Zambia and what are the factors related to<br>textbook availability? ...................................................................................................................... 11|
|Textbook availability in DEBS ...................................................................................................... 11|
|Textbook availability in schools .................................................................................................. 11|
|Factors related to the availability of textbooks at baseline ........................................................ 12|
|Cost estimates for delivery or collection of textbooks ............................................................... 13|
|5.2. Do the financial incentives have an impact on textbook distribution and availability? What kind of<br>financing scheme is more effective? ............................................................................................... 14|
|5.3. Does information and communication on textbook availability improve textbook availability in<br>schools? ........................................................................................................................................... 17|
|6.<br>Summary of Results ............................................................................................................................ 19|
|7.<br>Discussion and Recommendations ...................................................................................................... 20|
|References ................................................................................................................................................... 22|
|Appendix ..................................................................................................................................................... 23|
|**List of Tables**|

|Table 1. Phase-based textbook procurement in Zambia ............................................................................... 4|
|---|
|Table 2. Description of Treatments ............................................................................................................... 7<br>|
|Table 3. Description of outcome variables ................................................................................................... 9<br>|
|Table 4. Baseline descriptive by treatment group ....................................................................................... 10|
|Table 5. Textbook availability and distribution—school response ............................................................. 12|
|Table 6. Factors related to textbook availability at baseline—Grade 2 textbooks ...................................... 13|
|Table 7. Cost estimates for the delivery of textbooks to school ................................................................. 14|
|Table A.1. Textbook availability and distribution—DEBS’s response ...................................................... 23|
|Table A.2. Factors related to the distribution procedure from DEBS to school ......................................... 23|
|Table A.3. Impact of incentives on textbook availability (difference-in-difference, school fixed effect) .. 24|
|Table A.4. Impact of incentives on textbook availability (difference-in-difference, OLS) ........................ 25|
|Table A.5. Impact of incentives on textbook distribution procedure from DEBS to school ...................... 26|
|Table A.6. Impact of incentives on textbook distribution procedure from DEBS to school ...................... 27|

iv
# **List of Figures**

|Figure 1. Impact of financial incentives on textbook availability in schools compared to the control group<br>.................................................................................................................................................................... 15|
|---|
|Figure 2. Impact of financial incentives on textbook distribution mechanism compared to the control<br>group............................................................................................................................................. 16|
|Figure 3. Comparison of the impacts of the school incentive scheme and the DEBS incentive—difference<br>in coefficients of interaction terms ............................................................................................. 17|
|Figure 4. Role of information and communication on textbook availability .............................................. 18|

v
# **1. Introduction**

Many children in developing countries leave school without the skills of simple calculation and reading that are necessary to have a productive life. The learning crisis starts in the early grades and gets magnified over the schooling years. The majority of Grade 2 students cannot read a single word of a short text and more than 60 percent of primary school children in developing countries do not attain a minimum proficiency in learning (World Bank 2019).

Interventions such as the provision of well-designed learning materials, teachers’ guides to help to use those materials, and the training of teachers are proven to be effective in improving students’ learning (Piper et al. 2018). Among these, the importance of learning materials seems apparent: children can copy short texts from a blackboard, but this learning is limited without access to rich reading materials. Many teachers in developing countries use textbooks as basic pedagogical tools (Lee and Zuilkowski 2015; Kim et al. 2017). Much of the existing literature repeatedly confirms the positive association between textbook availability and increased learning (Fehrler, Michaelowa, and Wechtler 2009; Yara and Otieno 2010). Some research finds that providing textbooks without additional support to teachers and students does not result in a significant increase in learning (Glewwe, Kremer, and Moulin 2009). However, access to textbooks is a fundamental component for other programs (such as teacher training or teachers’ guides) to be effective.

In many Sub-Saharan African (SSA) countries, however, textbooks are in seriously short supply. On average in 22 SSA countries in 2012, 1.4 students shared one textbook for reading and math and the majority of the countries had a textbook-pupil ratio of 3 in 22 (UNESCO Institute for Statistics 2012). Furthermore, there is a big discrepancy between textbook availability in classrooms and official estimates of demand at the planning and procurement stage (Fredriksen et al. 2015). This is partly due to a poor distribution system and low quality of textbooks, resulting in a shorter life-cycle for texts than expected. In addition, as schools are located further away from the distribution center, the textbook scarcity worsens.

The situation in Zambia is not an exception. Zambia has invested heavily in education with about 20 percent of the government budget. However, such investment has not translated into student learning outcomes in primary education. One chief contributing factor to poor outcomes is the severe shortage of learning materials, particularly textbooks in both English and local languages. A public expenditure review shows that 91 percent of schools in Zambia lack textbooks and on average 5–6 pupils share less than one textbook, including textbooks in local languages (World Bank 2015).

The objective of this study is to help the Ministry of General Education (MoGE) by investigating the factors related to low textbook availability in schools and evaluating a set of financial incentive mechanisms for improving the last-mile delivery of textbooks in Zambia.

The study is organized as follows: Section 2 presents a brief review of existing literature on the topic and outlines the research questions; Section 3 provides a background on textbook procurement and distribution in Zambia; Section 4 describes the methodology used for the study, including the description of interventions, data, and empirical analysis strategy; Section 5 presents findings of each research question; Section 6 summarizes the findings; and Section 7 concludes with policy recommendations. Tables that capture the results of regression analysis are included in the Appendix at the end of the study.

1
# **2. Literature Review and Research Questions**

There are multiple factors related to textbook scarcity in developing countries. These factors include poor planning, management, and monitoring of textbook procurement; lack of information on textbook availability and use; and poor storage and distribution systems (Fredriksen, Brar, and Trucano 2015). In order to solve the low textbook availability problem, all stages of the textbook supply chain need to be addressed. Examples of good practices include providing cost-effective packages of teaching and learning materials with a predictable and sustainable financing strategy for procurement and distribution, monitoring the demand for and availability of textbooks, and so forth.

This study focuses on the distribution part of textbook supply, in particular the last-mile delivery of textbooks from regional education offices to schools in Zambia. Specifically, the study evaluates resultsbased financing mechanisms for the delivery of textbooks. Results-based financing mechanisms—in other words, financial incentives—are an active area of research. For a long time, these mechanisms have been more widely implemented in the private sector than in the public sector. However, interest has grown over the past few decades in using financial incentives to improve public sector performance and service delivery. Financial incentives take a variety of forms, such as pay-for-performance, pay for activities, bonuses, performance-based fee schedules, financial awards, and so forth (Custers et al. 2008).

Among the financial incentive schemes, pay-for-performance (P4P) has been studied the most. However, findings on its effectiveness are mixed. A recent review on the effect of P4P in the health sector in 14 OECD countries concluded that the impact of P4P is unclear. Some positive program effects can be attributed to improved monitoring and data recording rather than the P4P scheme itself (Milstein and Schreyoegg 2016). Several studies examine the effects of team-based performance pay. Studies of lowerlevel public sector workers in the United Kingdom found that team-based P4P had positive impacts on small team productivity and negative impacts on large teams (Burgess et al. 2010, 2017). Furthermore, P4P can cause some negative behavioral changes. An example is “cream-skimming” or “parking,” which allocates a team’s resources and services to low-risk clients with high probability of success rather than to high-risk clients who may need the services most. “Parking” behavior was observed during the implementation of performance-based contracting in social welfare services in the Netherlands (Koning and Heinrich 2013).

This study evaluates a financial incentive scheme that combines “partial pay for activities” and “pay for performance” in Zambia. The incentives were given for completing a specific task—delivering textbooks to school. The predetermined compensation in the form of reimbursement as a bonus was largely to incentivize agents who are already in place and performing the tasks. In addition, the incentive schemes were designed to prevent the cream-skimming behavior of the regional education offices by providing further financial incentives for serving hard-to-reach clients (schools in remote locations). Given that the pay for activities has the potential of freezing existing service activities due to financial constraint, a small bottom-line grant was given to all regional education offices for starting and organizing the basic service of textbook delivery. This study also evaluates which service agents can be better incentivized to deliver textbooks—schools or regional education offices. In Zambia, although the official responsibility of textbook delivery goes to the regional education offices, the actual practice in most districts is that the regional education offices ask schools to come to the offices and collect their books.

In addition to evaluating the designed financial incentive schemes, this study examines the role of information and communication in improving textbook availability in schools. Due to unpredictable procurement and distribution planning, limited resources, and challenges of geographical accessibility, information on textbook availability is often not shared with schools in Zambia. As a part of implementing the above-mentioned incentive schemes, information sharing was implemented in all schools in the school 2
districts involved in the study. The role of information sharing in the availability of textbooks in schools was then investigated.

Using the above background and the review of relevant literature, this study investigated the following questions:

- 1) What is the current status of textbook distribution in Zambia and what are the factors related to textbook availability?

- 2) Do financial incentives have an impact on textbook distribution and availability? What kind of financing scheme is effective for the last-mile delivery of textbooks?

- 3) Does sharing information and strengthening communication on textbook obtainability at points of distribution improve textbook availability in schools?

3
# **3. Textbook Procurement and Distribution in Zambia**

The textbook distribution system in Zambia relies on centralized procurement undertaken at the national level by the MoGE, followed by the transfer of the books to the Provincial Education Offices (PEO), then to the District Education Board Secretaries (DEBS) offices, and then final delivery by DEBS to individual schools. An independent review identified a breakdown in transfer from DEBS to schools as the foremost weakness in the textbook supply chain in the country.

## **Procurement of Textbooks**

In 2014, the MoGE revised the curriculum of basic education, including teaching in local languages in early grades (Grades 1–4). In line with the revised curriculum, the MoGE planned to procure textbooks in phases over four years, from 2015 to 2018. In each phase, the contracts for publishing textbooks were awarded to multiple publishers through an international bidding process, except for the language literacy subject in seven Zambian local languages and English. Private publishers were less interested in supplying certain textbook titles, especially those in local languages where the printing volumes for each subject were small. The Curriculum Development Center (CDC) together with the Zambian Education Publishing House (ZEPH) printed such titles.

Table 1 describes the MoGE’s phase-based procurement plan and its implementation status. In the first and second phases, the textbooks were procured according to initial plans; textbooks for Grades 1, 5, 8, and 10 were procured and distributed in 2015; and the textbooks for Grades 2, 6, 9, and 11 were procured and distributed in 2016–17. The third procurement was delayed but completed in 2018, while the last phase could not be undertaken due to unavailability of funds from the government budget. For Grade 4 in phase 4, however, the textbooks in Zambian local languages were procured and published by ZEPH but could not be distributed on time. Each subject for each grade has three books for three terms and the books are procured with the assumption of 2–3 students sharing one book.

**Table 1. Phase-based textbook procurement in Zambia**

|**Phase**|**Original Procurement Plan**|**Changes**|
|---|---|---|
|1<sup>st</sup>phase in 2015<br>|Grades 1, 5, 8, and 10 (done)|Procured as planned|
|2<sup>nd</sup>phase in 2016|Grades 2, 6, 9, and 11 (procured and<br>delivered to DEBS four months after the<br>procurement)|Procured as planned|
|3<sup>rd</sup>phase in 2017|Grades 3, 7, and 12 (procurement<br>completed by August 2017)|Procurement was delayed but<br>completed. The distribution of<br>textbooks is on-going as of 2019.|
|4<sup>th</sup>phase in 2018|Grade 4|Procurement is not complete due to<br>limited funding. Only literacy course<br>books for seven Zambian languages<br>and English textbooks are procured<br>in 2018 and distributedin 2019.|

## **Distribution of Textbooks**

In Zambia, textbooks are usually distributed from MoGE’s central warehouse in Lusaka, first to PEOs, then to DEBS, and finally to schools. When publishers deliver the procured textbooks to MoGE’s warehouse in Lusaka, the PEOs are informed about the availability of textbooks. Textbook distribution 4
responsibilities, including information sharing and communication with schools on textbook availability, stays with the regional authorities (PEOs and DEBS), especially for the last-mile delivery to remote schools.

The distribution practice from Lusaka to PEOs varies from year to year, depending on whether the MoGE’s procurement and the distribution budget is secured from the Ministry of Finance (MoF) each year. In instances where the funds have been secured, the MoGE headquarters team has distributed the textbooks to PEOs and DEBS using the Ministry’s own transport and private hire. This makes it hard for each PEO and its schools to predict the availability and distribution of textbooks. In 2015, 70 percent of primary schools requested their textbooks to be delivered, but only 26 percent of the requests received books.

Therefore, the informal practice of textbook distribution is that resourceful PEOs collect the textbooks early and distribute the books to DEBS, while resource-constrained PEOs postpone the collection of textbooks from Lusaka until the distribution budget is secured. The situation is similar from PEOs to DEBs. Even for the last-mile delivery of textbooks from DEBS to schools, in most cases schools organize collection of textbooks from DEBS with their own resources, or from zone-center schools to which DEBS sometimes deliver the books, then distributing to nearby schools. DEBS are also responsible for informing schools on textbook availability, but the communication practice varies across the DEBS, from announcing during unrelated workshops or meetings to sending out group messages via phone to head teachers.

To incentivize the last-mile delivery of textbooks to schools, DEBS are usually provided by MoGE with a small bottom-line grant to cover transportation costs when funds are available. Because these funds are provided upfront without clear guidelines and monitoring of performance and are too small in amount to complete the activities, books end up stored for a long time (sometimes indefinitely) at DEBS offices. For those DEBS that manage to deliver the transferred textbooks, most of them deduct the cost of textbook delivery from the annual budget of their schools. So often, textbooks do not get delivered or at least not on time.

5
# **4. Methodology: Intervention, Data, and Empirical Analysis Strategy**

## **Intervention**

In order to identify an effective financing mechanism for the last-mile delivery of textbooks, the research team together with the MoGE team designed two treatments to provide financial incentives for the delivery of textbooks: “partial pay for activities” and “pay for performance.” The incentives are given for completing specific tasks – delivering textbooks to schools. A predetermined amount of compensation (largely for reimbursing the delivery costs) is given as a bonus (for better performance) to incentivize agents already tasked by the MoGE to deliver the textbooks. The incentive scheme is also designed to prevent cream-skimming behavior by regional education offices by providing greater financial incentives to agents that serve hard-to-reach clients (schools in remote locations). The pay-for-activities method has the potential of freezing existing service activities due to financial constraints; therefore, a small bottom-line grant was given to all DEBS to help them organize and start up basic textbook delivery services.<sup>1</sup> The two incentive treatments differ by the targeted agents in order to evaluate which agent can be better incentivized to perform the task (delivering textbooks) – schools or DEBS. In Zambia, the official responsibility of textbook delivery goes to the regional education offices. However, the actual practice in most districts is that regional education offices ask their schools to come to their offices and collect the books.

In addition to the financial incentive schemes, a centrally managed information and communication strategy was implemented in all districts, including those districts in the control group. Instead of leaving communication about textbook availability to PEOs and DEBS, the head office at the MoGE managed communication to the PEOs and DEBS directly and continued follow-ups with the DEBS to confirm that textbook availability information had reached schools. The information and communication along with the monitoring was a precondition of successful implementation of the financial incentive schemes. All schools, including those in the control group that did not receive any financial incentives, were informed about the textbook availability through multiple communication channels, which included phone calls to schools and group-messages to school head teachers using SMS. Prior to this intervention, systematic information and communication on textbook availability did not exist; it was left to the discretion of DEBS who made announcements through phone calls, at unrelated meetings, or even just posted a notice on a DBES office information board. Because of lack of systematic and uncoordinated communication, many schools were unaware of the textbook availability. Hence, the books remained uncollected or undistributed at the DEBS warehouse or even at the zone-center schools for a longer period.

Two separate treatments were implemented in 18 districts (9 districts per each treatment arm) and another 9 districts were selected as a control group that only received information about textbook availability from their respective DEBS. The description of each group is shown in detail in Table 2.

> 1 The designed intervention combines two concepts – “partial pay for activities” and “pay for performance”. The incentive scheme was actually “partial pay for activities” because its disbursement is only made based on the estimated cost of delivery. But it was provided as a form of bonus (pay for performance) because it was not budgeted and in the previous years, DEBS did not receive funding for carrying out delivery activities.

6
# **Table 2. Description of Treatments**

|_Tr_|_eatment 1-School incentive: Incentivizing remote schools for collecting textbooks in nine districts_|
|---|---|
|-<br>-<br>-|It is announced to DEBS and the remote schools that (i) incentive amounts of K 700 (US$50) shall<br>be disbursed to each remote school when staff from the school collect textbooks within four weeks<br>of receipt of an official notice to collect textbooks from a DEBS office; (ii) if remote school staff<br>collect the textbooks after four weeks but before eight weeks after receipt of an official notice of<br>textbook collection from the DEBS, K 400 (US$28) shall be disbursed to that remote school. The<br>list of remote schools in each treatment district was shared by the MoGE.<br>All DEBS received an up-front grant of K 7,800<sup>2</sup>for the distribution of textbooks.<br>In addition, the MoGE head office managed the information and communication on textbook<br>availability to individual schools through phone calls, group text messages with WhatsApp, and<br>official emails and letters.|
|_Tr_|_eatment 2- DEBS incentive: Incentivizing DEBS to deliver textbooks to schools in nine districts_|
|-<br>-<br>-|It was announced to DEBS that K 325 (US$23) per remote school and K 180 (US$13) per non-<br>remote school shall be disbursed to the office if DEBS delivered textbooks to the assigned schools<br>within 1 month of receipt of textbooks from MoGE.<br>All DEBS received an up-front grant of K 7,800 for the distribution of textbooks.<br>In addition, the MoGE head office managed the information and communication on textbook<br>availability to individual schools through phone calls, group text messages with WhatsApp, and<br>official emails and letters.|
|_Co_|_ntrol: Information and communication to all schools in nine districts_|
|-<br>-|All DEBS received up-front grants of K 7,800 for distributing their textbooks.<br>In addition, the MoGE head office managed the information and communication on textbook<br>availability to individual schools through phone calls, group text messages with WhatsApp, and<br>officialemails andletters.|

Average cost was K 513 per school for the school incentive group and K 280 per school for the DEBS incentive group. This excludes the up-front grants for all districts. In addition, a total of K 4,000 (K 2.8 per school) was spent on the calls/text messages and group WhatsApp messages to schools to announce the availability of textbooks and the interventions for selected districts.

# **Data and Measurement**

# _Survey sample_

Originally, a baseline survey was conducted in a nationally representative sample of schools and their assigned DEBS with a two-stage stratified cluster design. During the survey in November and December 2017, information on distribution and availability of Grade 2 textbooks procured in 2016 was collected. Prior to the baseline survey, calls were made to DEBS offices to enquire about the distribution status of Grade 2 textbooks. Based on the information collected, the sample size of 225 schools was determined with a minimum detectable effect of 35–40 percent increase in probability of receiving the textbooks within one month upon being made available. The school samples were randomly drawn from 45 randomly selected districts in Zambia.

However, due to the unexpected changes in textbook procurement and distribution schedule described earlier in Section 3, the study had to shift its focus to the last 27 districts that had not received Grade 4

> 2 This is to reflect the inflation and to evaluate results-based finance schemes compared to the status quo of underfunded up-front grants from the MoGe.

7
textbooks until July 2019 due to the severe resource constraints they experienced. Even though the sample districts originally selected for the baseline study were changed, the baseline information, including availability and distribution status of Grade 2 textbooks, was collected during the end-line survey. This data collection was made possible by MoGE’s phased procurement and distribution approach of textbooks for each grade. For example, no additional procurement or distribution of Grade 2 textbooks took place after 2016 until October/November 2019, when the end-line survey of this study was executed. This enabled the study to collect baseline information (availability of Grade 2 textbooks procured in 2016 and its distribution timeline) during the end-line survey.

Lastly, the 27 selected districts were randomly assigned to three groups—two treatment groups and one control group. Nine schools were randomly drawn from each district in those districts for the survey, amounting to a total of 243 schools. During the end-line survey, two-period outcome information (pre and post treatment) was collected at once: the outcome measure for the pre-treatment was Grade 2 textbook distribution and that for the post-treatment was Grade 4 textbook distribution. The primary survey respondents were teachers who were in charge of textbooks. Their answers were cross-checked with the documents and information from the MoGE head offices, such as DEBS’s surveys and textbook records/storage kept in schools.<sup>3</sup>

# _Measurement_

Following the textbook procurement schedule for early grades, this study focuses on the availability of Grade 2 (pre-treatment) and Grade 4 (post-treatment) textbooks procured in 2016 and 2018, respectively. This sample included both English and local language textbooks. Two different constructs were measured—one for the availability of textbooks and one for the textbook delivery mechanism. Outcome variables for textbook availability were measured by constructing (i) a dummy variable of whether the school received the textbooks from DEBS, (ii) the textbook-pupil ratio in the school, and (iii) a dummy variable of whether the school received the textbooks within one month after they were made available in DEBS. The textbook delivery mechanism construct captures how the textbooks reach schools—whether schools collect the books from DEBS offices or from zone-center schools or whether DEBS transport the books to schools.

> 3The surveyor was able to check records for the Grade 2 textbook for 56 percent of the schools and storage records for 70 percent of schools that kept the books on the premises.

8
**Table 3. Description of outcome variables**

|**Construct**|**Outcome variables**|**Measure**|
|---|---|---|
|Textbook<br>Availability|Received textbooks*|Dummy variable: 1 - if school received or collected<br>textbooks for new curriculum; 0-otherwise|
||Textbook-pupil ratio**|Number of textbooks available (for literacy course per<br>term) in the school divided by number of relevant grade<br>students enrolledinthe school|
||Receiving textbooks in 1<br>month|Dummy variable: 1 - if school received/collected<br>textbooks within 1 month after the books were available in<br>DEBS officeforpick-up; 0-otherwise|
|Tbk|Textbook transported to<br>school by DEBS|Dummy variable: 1 - if DEBS transported the textbooks to<br>school; 0-otherwise|
|extoo<br>delivery<br>mechanism|Collection of the books<br>from DEBS|Dummy variable: 1 - if school collected the books from<br>DEBS on their own; 0-otherwise|
||Collection of the books<br>from zone-center school|Dummy variable: 1 if school collected the books from<br>zone school on their own; 0 otherwise|

_Note:_ *For the pre-treatment (baseline), the study measured the availability of Grade 2 textbooks for a literacy course under the new curriculum that schools were supposed to receive after June 2016 when books became available in MoGE's warehouse. For post-treatment (end-line), the study measured the availability of Grade 4 textbooks for a literacy course under the new curriculum that schools were supposed to receive after October 2018 when books became available in MOGE's central warehouse. When schools reported receiving books for a specific grade before these dates and before DEBS’s reported receiving date, the books were considered as old curriculum books that were delivered late; this event was coded as not receiving new curriculum books. **For the textbook-pupil ratio, the study measured both old and new curriculum books delivered since 2015. This approach was chosen because schools did not keep separate records for old and newly published books.

# **Empirical Analysis Strategy**

In order to answer the first research question, descriptive statistics were analyzed. These include (i) textbook delivery and availability status based on the school and DEBS surveys in the baseline and the end-line; and (ii) regressing outcome variables, at baseline, on possible contributing factors, including measures for information and communication on textbook availability, school-level financial resources (PTA funds and education budget), and location (remoteness) of schools.

The second and third research questions were studied by conducting econometric analysis employing difference-in-difference estimation. Difference-in-difference captures changes in outcomes of the treatment and control groups before and after the treatment using the following equation: i 0 1T T 1 2T T 2 3P 4T T 1 ∙P 5T T 2 ∙P i i YY denotes the outcome variable for each individual school; “= ββ + ββ TT + ββ TT + ββ PPTT+ ββ _i_ TT” for two different periods and “PPTT+ ββ TT PPTT+ εε _t_ ” (Grade 2 textbooks for pre-treatment and Grade 4 textbooks for post-treatment). T T 1 and T T 2 are dummy variables of whether the school belongs to the first treatment group (school incentive group) or to the second YY treatment group (DEBS incentive group). P is post-treatment dummy variable and TT i is an idiosyncratic TT error term. PPTT εε The coefficients of the interaction terms, T T 1 ∙P and T T 2 ∙P ( 4 and 5), capture the impact of the school incentive scheme and of the DEBS incentive scheme on textbooks availability. For the estimation of difference-in-difference, the school-level fixed effect is used with TT PPTT TT PPTTP ββ and interaction terms ββ and other time-varying variables, including dummy variables for the information and communication on PPTT 9
textbook availability and school budget for education and PTA funds. This estimation controls for any timeunvarying variables that might affect the textbooks availability, such as remoteness of schools or other school- or district-level characteristics. For the binary outcome variable, we used a linear probability model for the convenient interpretation of interaction terms (Ai and Norton 2003).

A balance check for randomness shows some statistically significant differences at baseline across the treatment groups (Table 4). For example, the school incentive group has a significantly lower proportion of schools receiving textbooks compared to the control group; the DEBS incentive group has the highest proportion of schools receiving textbooks, conditional on textbooks being available at the DEBS office. More schools in the control group received communication about the availability of textbooks compared to the control group at baseline. The difference-in-difference method resolves the bias caused by the possibility of one group having a better outcome than others at baseline by comparing the changes in performance between periods across the treatment groups.

**Table 4. Baseline descriptive by treatment group**

||(Group 1)<br>School Incentive|(Group 2)<br>DEBS Incentive|(Group 3)<br>Control|
|---|---|---|---|
|Pr(Received textbooks) - new curriculum|0.12*** (0.04)|0.40 (0.05)|0.32 (0.05)|
|Textbook-pupil ratio|0.20 (0.04)|0.14 (0.03)|0.13 (0.04)|
|Pr(Received textbooks within a month of<br>textbook available at DEBS)|0.16 (0.05)|0.54*** (0.07)|0.09 (0.04)|
|Textbook available at DEBS|0.56 (0.056)|0.67 (0.05)|0.67 (0.05)|
|Received communication about textbooks|0.58* (0.05)|0.59* (0.05)|0.72 (0.05)|
|Remote schools (more than 45 km away)|0.52 (0.06)|0.42* (0.06)|0.56 (0.06)|
|School budget, per student|38.40 (6.01)|44.44 (8.06)|67.11 (16.01)|
|PTA fund, per student|15.27 (3.48)|17.95 (5.20)|13.92 (3.33)|

_Note:_ Statistical test for two treatment groups relative to control group is shown at 1% significance level (***), 5% significance level (**), and 10% significance level (*). Standard errors are in parentheses.

10
# **5. Findings**

- 5.1. What is the current status of textbook distribution in Zambia and what are the factors related to textbook availability?

## **Textbook availability in DEBS**

Before the intervention, textbooks were distributed across the districts unsystematically and unpredictably. According to the DEBS survey, 85 percent of DEBS reported receiving Grade 2 textbooks. However, after the curriculum revision in 2014, the old curriculum books were still circulated and only 63 percent of DEBS received new curriculum books that were procured in 2016 (Appendix Table A.1). The average time for books to reach DEBS offices from MoGE’s central warehouse is 13 months, but times vary significantly across DEBS. A quarter of DEBS reported receiving books within two months and another quarter of DEBS reported receiving the books after 19 months. For the case of Grade 4 textbooks that were targeted for the intervention of last-mile delivery of textbooks, distribution from the central warehouse to DEBS was organized in a way that all the textbooks were delivered to 27 selected DEBS in July/August in 2019. Based on the number of books distributed and the number of Grade 4 students in the sample districts, about 2–3 students are supposed to share the new curriculum textbooks (Ministry of General Education in Zambia 2019). In terms of the distribution practice from DEBS to schools, DEBS reported that on average 44 percent of schools had collected the Grade 2 books from DEBS on their own, compared to 55 percent for the Grade 4 textbooks.

## **Textbook availability in schools**

According to the school survey, 49 percent of schools reported receiving Grade 2 textbooks, including both new and old curriculum books delivered after the curriculum revision in 2015. Yet, only 28 percent schools received the Grade 2 books for the new curriculum. This low number is partly due to the fact that many DEBS did not receive the new curriculum books. However, even after taking into account textbook availability at DEBS, a large percentage of schools reported not receiving textbooks. Only 44 percent of schools reported receiving the Grade 2 textbooks. In terms of textbook-pupil ratio, 4–5 students shared one Grade 2 textbook in one of the local Zambian languages,<sup>4</sup> while every student had one Grade 4 textbook in one of the Zambian local languages.

At the baseline, 27 percent of schools reported receiving Grade 2 textbooks within one month after they were delivered to DEBS offices from MoGE’s central warehouse. After one month, however, the textbook distribution did not happen as quickly as expected; even after six months, only 31 percent of schools reported receiving the books. Post treatment, distribution time was shortened dramatically; 90 percent of schools received Grade 4 textbooks within two months.

Looking at how books reach schools from DEBS, the majority of schools collected the books from DEBS offices at their own expense, both in the baseline and end-line surveys. Among the schools that received the new curriculum textbooks, 69 percent collected the Grade 2 books from DEBS at baseline and 74 percent of schools collected the Grade 4 books from DEBS at end-line. The DEBS delivered books to only 14–15 percent of schools. This finding contrasts to the DEBS’s responses, which understated the proportion of schools collecting the books from DEBS—about 44 percent of the schools at baseline and 55 percent at end-line.

> 4 This combines both new and old curriculum textbooks.

11
At baseline, 69 percent of the schools, conditional on the books being available at DEBS offices, reported receiving any communication about textbook availability. Most communication was to inform schools that the books were available for collection. Many schools reported receiving the information via a phone call or a text message from DEBS. However, few schools reported learning that Grade 2 books were available via a DEBS office information board when visiting a DEBS office, an announcement during a meeting at DEBS, or through zone-center schools. At end-line, due to the fact that the information and communication was a precondition for successful implementation of financial incentive treatments, communication to inform schools of textbook availability was made through multiple channels, including phone calls to individual schools, group WhatsApp text messages to head teachers in each district, and official messages to schools from DEBS. All these communications were organized and managed by the officials of the MoGE head office. As a result, 94 percent of schools reported receiving the communication at end-line.

**Table 5. Textbook availability and distribution—school response**

||Pre-treatm|ent (Grade 2|textbooks)<br>|Post-treatm|ent (Grade 4|textbooks)<br>|
|---|---|---|---|---|---|---|
||N|Mean|Std|N|Mean|Std|
|_To what degree are textbooks av_|_ailable in sch_|_ools?_|||||
|Received textbooks—new<br>curriculum|243|28%|45%|243|97%|18%|
|Received textbooks—new<br>curriculum, conditional on<br>available in DEBS|153|44%|50%|243|97%|18%|
|Textbook-pupil ratio|243|0.2|0.3|243|1.1|1.1|
|Textbook-pupil ratio|||||||
|conditional on receiving books|119|0.3|0.4|236|1.2|1.1|

|_How long does it take to deliver bo_<br>_(new curriculum books only)_<br>|_oks to scho_<br>|_ols from DE_<br>|_BS, condition_<br>|_al on textbo_<br>|_oks available_<br>|_in DEBS?_<br>|
|---|---|---|---|---|---|---|
|Receiving books in 1 month|153|27%|44%|243|81%|39%|
|Receiving books in 2 months|153|27%|45%|243|90%|30%|
|Receiving books in 6 months|153|31%|46%||N/A||
|_How are textbooks distributed from_<br>|_DEBS to s_<br>|_chools? (con_<br>|_ditional on s_<br>|_chools recei_<br>|_ving textboo_<br>|_ks)_<br>|
|DEBS transported to school|68|15%|36%|235|14%|34%|
|School collected from Zone|68|12%|32%|235|10%|30%|
|School collected from DEBS|68|69%|47%|235|74%|44%|
|_Information and communication ab_<br>|_out textboo_|_ks availabili_|_ty_||||
|School received communication<br>about textbook availability|153|69%|46%|243|94%|23%|

# **Factors related to the availability of textbooks at baseline**

A set of potential factors related to textbook availability was investigated by the study, such as information and communication on textbook availability, school resources, and location of schools. Unexpectedly, the location of schools, measured in distance from a DEBS office, does not correlate with the probability of receiving textbooks, textbook-pupil ratio, or the time for distribution of textbooks. The most important variable that is positively and significantly correlated with textbook availability is whether the school received communication on the availability of textbooks. Having communication on textbook availability increases the probability of receiving textbooks by 0.284, the textbook-pupil ratio by 0.19, and 12
the probability of receiving the books in one month by 0.198. School financial resources measured by PTA funds per student was correlated negatively with the probability of receiving textbooks. One possible explanation could be a reverse causality: schools that had not received the books tried to raise more PTA funds for collecting the books.

**Table 6. Factors related to textbook availability at baseline—Grade 2 textbooks**

||Pr(Receiving<br>textbooks)|Textbook-pupil<br>ratio|Pr(Receiving in a<br>month)|
|---|---|---|---|
|Communication on availability of<br>textbook, dummy|0.284***|0.190***|0.198***|
||(0.051)|(0.046)|(0.048)|
|School budget per student, log(amount)|-0.015|0.024|-0.028|
||(0.020)|(0.018)|(0.019)|
|PTA fund per student, log(amount)|-0.050***|-0.005|-0.006|
||(0.017)|(0.015)|(0.016)|
|Located 21–44 km away, dummy|0.040|0.084|-0.012|
||(0.066)|(0.060)|(0.062)|
|Located 45–80 km away, dummy|0.057|0.011|0.041|
||(0.065)|(0.059)|(0.061)|
|Located more than 80 km away, dummy|0.074|-0.002|0.005|
||(0.065)|(0.059)|(0.062)|
|Constant|-0.065|0.003|-0.026|
||(0.086)|(0.078)|(0.081)|
|N|243|243|243|
|R-squared|0.370|0.103|0.200|

_Note:_ Regression is estimated using ordinary least square (OLS). Availability of textbooks in DEBS and the dummy variables for missing observations in school resources are controlled in the regression but not shown. The coefficient is significant at the1% level (***), the 5% level (**), and the 10% level (*). Standard errors are in parentheses.

# **Cost estimates for delivery or collection of textbooks**

The survey asked the sampled schools how much they spent or would spend if they collected the books from DEBS. The school would, on average, spend about K 308 (about US$22). The estimated cost is different depending on where the school is located: the average estimated cost is K 142 (about US$10) for schools located within 20 km from DEBS; K 202 (about US$14) for schools located 21–44 km away; K 323 (about US$23) for schools located 45–80 km away; and K 573 (about US$40) for schools located more than 80 km away. The source of funds for textbook collection varies: some schools raise funds through the PTA or use the education budget received from the MoGE; others use school staff’s personal money. Thirtyfour percent of schools responded that funds were raised through the PTA, 19 percent using education budget, and the majority of the rest mentioned using their staff’s personal money to collect the books.

DEBS, on average, spent K 347 (about US$24) per school for the distribution of textbooks at baseline, and the cost estimate per school is K 506 (about US$35) if DEBS were to deliver the books to all schools in their districts. This is slightly lower than the per-school cost for the DEBS incentive (Treatment 2), which is K 455 including financial incentives (K 280) and the up-front grant (K 174).

13
**Table 7. Cost estimates for the delivery of textbooks to school**

|School response:<br>Estimated cost if schools collect the books from<br>DEBS|DEBS respo|nse|
|---|---|---|
|Distance of 20 km<br>K 142|Actual cost|K 347|
|21–44 km away<br>K 202|Eid if||
|45–80 km away<br>K 323|stmate cost<br>DEBS delivered books|K 506|
|More than 80 km away<br>K 573|<br>to schools||
|All schools(average)<br>K308|||

_Note:_ The exchange rate from Zambian kwacha (K) to U.S. dollars was approximately 0.08 in November 2019.

- 5.2. Do the financial incentives have an impact on textbook distribution and availability? What kind of financing scheme is more effective?

The impact of financial incentive schemes on textbook availability is shown in Figure 1. Each bar indicates the coefficient estimate of the interaction term described in the empirical analysis strategy, which captures the effect of each treatment. It can be interpreted as: the incentive scheme increases or decreases the set of outcome variables compared to the control group. The left-hand column pair shows the impact of both the school incentive and the DEBS incentive. The school incentive increases the probability of receiving textbooks by 0.126 compared to the control group. However, the DEBS incentive does not show any statistically significant impact. The middle column pair shows that neither incentive scheme significantly increased the textbook-pupil ratio. This may be because planning of the distribution by the MoGE is not done according to the needs of schools or districts based on the current number of students in the district. Instead, the MoGE distributes a predetermined number of books to each district, which results in the DEBS distributing books to schools on an ad hoc basis. This is supported by a large variation in textbook-pupil ratios across districts and across schools within districts. The district-level textbook-pupil ratio ranges from 0.02 to 1.13 for the 27 sample districts and the textbook-pupil ratio for the middle 50 percent of schools ranges from 0.54 (25<sup>th</sup> percentile) to 1.63 (75<sup>th</sup> percentile). Lastly, the right-hand column pair shows the impacts of the incentive schemes on the probability of schools receiving textbooks in one month. Interestingly, the figure shows that the DEBS incentive scheme actually lowered the probability of receiving textbooks within a month by 0.324.

14
**Figure 1. Impact of financial incentives on textbook availability in schools compared to the control group**

_Note:_ A solid bar indicates that the coefficient statistically significant at least at the 10% level and a striped bar indicates the coefficient is not statistically significant. Coefficients are shown as a label on the top/bottom of each bar at 1 % (***), 5% (**), and 10% (*) significance levels. The regression models were estimated using difference-in-difference method with school fixed effect and the full results are shown in Appendix Table A.3.

Figure 2 shows how the incentive schemes change the textbook distribution mechanism from DEBS to school. There are three possible distribution procedures: the DEBS transport the textbooks to schools, schools collect the textbooks from nearby zone schools, or schools collect the books from DEBS offices at their own expense. The interpretation of these coefficients is the incentive scheme increases or decreases the probability of each procedure, as opposed to not delivering textbooks to schools at all. The left-hand column pair shows not statistically significant coefficients, meaning that both the school incentive and the DEBS incentive scheme do not increase or decrease the probability of DEBS transporting the textbooks to schools as opposed to schools not receiving textbooks at all. The middle column pair indicates the school incentive has a statistically significant impact, as it increases the probability of schools collecting books from zone-center schools by 0.356. Lastly, right-hand column pair shows the incentive impact on the probability of schools collecting books from DEBS offices using their own expenses, as an alternative to not receiving the books at all. As described earlier, more than two-thirds of schools collect books from DEBS offices, conditional on receiving the books. The school incentive scheme increases this probability by 0.158 at the 5 percent significance level.

15
**Figure 2. Impact of financial incentives on textbook distribution mechanism compared to the control group**

<!-- Start of picture text --> 0.221<br>0.173 0.158**<br>0.001<br><!-- End of picture text --> _Note:_ A solid bar indicates that the coefficient is statistically significant at least at the 10% level and striped bar indicates that the coefficient is not statistically significant. Coefficients are shown as label on the top/bottom of each bar with significance level at 1 % (***), 5% (**), and 10% (*). Full regression results are shown in Appendix Table A.5.

Finally, the impacts of the two incentive schemes on the set of outcome variables were compared. Figure 3 compares the impacts of the school incentive scheme with the DEBS incentive, and finds they are statistically significant in several outcomes. The school incentive increased the probability of receiving textbooks by 0.209, the probability of receiving the books within a month by 0.225, and the probability of schools collecting the books from DEBS by 0.278.

16
**Figure 3. Comparison of the impacts of the school incentive scheme and the DEBS incentive— difference in coefficients of interaction terms**

<!-- Start of picture text --> 0.500<br>0.278***<br>0.225***<br>0.250 0.209***<br>0.183<br>0.131<br>0.000<br>-0.250<br>-0.220<br>-0.500<br>Pr(Receiving Textbook-pupil Pr(Receiving in Pr(DEBS Pr(Collect from Pr(Collect from<br>textbooks) ratio a month) transport) zone) DEBS)<br>Coefficient<br><!-- End of picture text --> _Note:_ A solid bar indicates that the coefficient is statistically significant at least at the 10% level and the striped bar indicates the coefficient is not statistically significant. The difference between coefficients of � two interaction terms ( 4 5 ) is shown as a data label at the top/bottom of columns with significance level at 1% (***), 5% (**), and 10% (*). ββ�−ββ

- 5.3. Does information and communication on textbook availability improve textbook availability in schools?

Figure 4 shows how information and communication on textbook availability played a role in various outcomes. The figure columns denote coefficients of the dummy variable, indicating the school received any communication about the textbook availability.<sup>5</sup> Having received communication had positive effects on almost all outcome variables. Information and communication increased the probability of receiving textbooks by 0.334, the textbook-pupil ratio by 0.226, and the probability of receiving textbooks one month after availability by 0.202 at the 1 percent significance level. The impact is larger than for the school incentive. Information and communication increase the probability of collecting books both from zonecenter schools and DEBS, but does not affect the probability of DEBS transporting books to schools. This may indicate that “not knowing textbook availability” in part explains irregular textbook procurement and distribution schedules, and is a key barrier for school access to textbooks. Removing this barrier could encourage motivated schools to collect the books from wherever they are, either zone-center schools or DEBS.

> 5 The results should be accepted with caution due to data limitations. Difficulties in collecting Grade 2 textbook information may have caused measurement errors that positively correlate to a positive outcome variable. In other words, schools that received Grade 2 textbooks may be more likely to recall receiving communication about the textbooks than otherwise.

17
**Figure 4. Role of information and communication on textbook availability**

<!-- Start of picture text --> 0.503***<br>0.5<br>0.334***<br>0.274***<br>0.226**<br>0.25 0.202*** 0.181<br>0<br>-0.25<br>-0.5<br>Coefficient<br><!-- End of picture text --> _Note:_ A solid bar indicates that the coefficient is statistically significant at least at the 10% level and a striped bar indicates that the coefficient is not statistically significant. Results are from the school-level fixed effect model controlling for other time varying variables. Significance levels are shown at 1% (***), 5% (**), and 10%. Full regression results are shown in Appendix Table A.3. and Table 5.

18
# **6. Summary of Results**

During the pre-treatment period (2016–18), textbook distribution to schools was not well coordinated from the central ministry, making it hardly predictable from the perspective of DEBS and schools. Although DEBS were supposed to receive Grade 2 textbooks for local-language literacy courses and English courses procured in 2016, only 63 percent of the sampled DEBS reported receiving the textbooks in 27 sample districts. The textbook availability in schools was worse: only 28 percent of schools received Grade 2 new curriculum-based textbooks and most of these schools reported collecting the books from DEBS using their own resources rather than DEBS transporting the books to them. This low textbook availability at school is a multifaceted problem. It includes factors such as textbooks not reaching DEBS, lack of information and communication on textbook availability from DEBS to schools, and limited financial resources. In 2019, after financial incentives were provided for delivery and collection of textbooks with communication extended to all schools about textbook availability at DEBS, the number of schools receiving books greatly increased: 97 percent of schools reported receiving Grade 4 new curriculum-based textbooks.

Looking at the specific impact of the incentive schemes, this study finds that the incentives to schools increases the probability of schools receiving textbooks by 0.126, although this result was not accompanied by a statistically significant improvement in the textbook-pupil ratio. This improvement is mainly driven by an increased probability of schools collecting books from zone-center schools (by 0.356) and of schools collecting books from DEBS (by 0.158). On the contrary, the incentives to DEBS do not increase the probability of schools receiving textbooks. In fact, these incentives decrease the probability of receiving textbooks within a month after made available at the DEBS office. Moreover, compared to the no-incentive group, incentives to DEBS hardly raised the probability of DEBS transporting the books to schools or the probability of schools collecting the books from DEBS.

There was remarkably improved performance in the last-mile delivery of textbooks in 2019 in the 27 districts in the study. This is not only a result of the financial incentive schemes but also an outcome of other components that are necessary preconditions for successful implementation of these incentive schemes. First, information and communication on textbook availability to all schools are important. Using a school-level fixed effect regression model, this study identified a sizable positive impact of information and communication on almost all aspects of textbook availability. Providing information on textbook availability to all schools increases the probability of schools receiving textbooks by 0.334, the textbookpupil ratio by 0.226, and the probability of receiving textbooks within a month by 0.202. This impact is mainly achieved through increasing the probability of schools collecting textbooks from DEBS or zonecenter schools. This result may imply that lack of transparency in textbook delivery is a key barrier and removing it would encourage motivated schools to collect the books from wherever they are—either zonecenter schools or DEBS. Second, in 2019 the MoGE organized book distribution and transportation, thus ensuring that all textbooks at least reached the designated DEBS offices. Organized distribution and delivery is another necessary precondition for any incentive scheme to work properly.

The school incentive scheme would, on average, cost K 513 per school, provided that all remote schools collected books within one month of availability. This cost is slightly greater than what DEBS estimated (K 506) for them to transport textbooks to all schools in their districts. However, the DEBS incentives hardly changed the probability of DEBS transporting books to schools. Communication to all schools in multiple channels cost about K 2.8 per school, which is highly cost effective in comparison with the school incentive scheme. However, relying only on better information and communication without providing necessary resources for textbook delivery is not a good long-term solution. Schools could be discouraged from collecting books without proper incentives, and lack of resources for book delivery would also generate inequality across the schools.

19
# **7. Discussion and Recommendations**

In conclusion, providing financial incentives to encourage schools to collect textbooks from DEBS could be an effective results-based finance mechanism for the MoGE to consider as a policy option for improving textbook availability in schools in Zambia. The current practice of providing financial incentives to DEBS does not seem to be effective in this regard. This is partly due to the fact that although DEBS play a part in the distribution of books, there are several factors that tend to limit their effectiveness in delivering textbooks to schools on time. Instead, the current textbook delivery practice in districts and communities relies heavily on the action of schools. As a result, providing incentives directly to DEBS does not improve much the textbook availability to schools. However, providing incentives to schools intensifies the current delivery mechanisms and has better results. In addition, providing information to schools on the availability of textbooks in DEBS significantly improves outcomes with minimum cost.

In order to successfully design and implement a financial scheme for the last-mile delivery of textbooks, an intervention that this study indicates is suitable for a resource constrained economy like Zambia. There are several binding constraints that the MoGE should consider, however. One is to ensure that textbooks can reach DEBS at least from the central government in a predictable and organized manner. The other is to ensure that the transparency of textbook distribution is exercised and that all schools are informed on the availability of textbooks from their local offices. In addition, monitoring could improve the effectiveness of the financial incentive scheme. Unfortunately, how monitoring would change behaviors could not be investigated in this study. It remains a topic for future research.

Fine-tuning the amounts of financial incentives should also be considered, given that the cost of collecting the textbooks is highly dependent upon geographic locations of schools in Zambia. When deciding the incentive amounts, however, long-term financial sustainability should be carefully considered, along with possible negative consequences such as discouraging intrinsic motivations of schools.

Note that current textbook distribution practices might help explain why the textbook-pupil ratio is not affected by any of the suggested financial incentives. The current practice is not based on the needs of schools but is ad hoc, based on the funding MoGE receives from treasury, which makes the textbook-pupil ratio at a school loosely correlated with the probability that the school will receive textbooks. There are several reasons for this practice. Although the procurement of textbooks is based on the total number of enrolled students in a procurement year, it does not predict or accommodate future changes in textbook demand resulting from the fluctuation of student enrollment due to, for example, population growth and new schools. This, along with the prolonged procurement process and the poor budget execution of the central government, causes a distortion in supply and demand of textbooks at regional and sub-regional levels. The distortions of current practice result in almost equal distribution of textbooks across regions and schools without calculating more accurately the actual needs. Furthermore, at present the MoGE does not have clear and consistent guidelines for textbook distribution, because it is trying to find the best distribution methods from the previous years.

At the macro level, many of the above-mentioned problems related to textbook distribution could be resolved by proper procurement planning based on future demand, not on the current demand, along with improved budget execution for textbook procurement. These reforms, however, largely depend on various factors and the country’s economic and political situation. Therefore, a more practical solution is to provide very specific but clear and easy-to-follow guidelines to all relevant stakeholders in the textbook supply chain, including publishers, MoGE headquarters, PEOs, DEBS, and schools. Such guidelines should include the requirement of using current enrollment figures, a systematic information and communication 20
channel on textbook availability, an accountability mechanism for textbook delivery, and the provision of financial incentives to ensure that textbooks can reach all schools and students for learning improvement.

21