metadata: authors: 0: author: Andreottola,Michele
1: author: Basenya,Olivier
2: author: Orozco Olvera,Victor Hugo
3: author: Reichert,Arndt Rudiger
4: author: De Souza Leao Spinola,Paula
count: Burundi
docna: 0: docna: Community Health Workers as Key Providers of
            Easy-to-Use Contraceptive Injectables : Experimental
            Evidence from Rural Burundi
docty: Policy Research Working Paper
theme: Health Systems and Policies,Human Development and Gender,Reproductive and Maternal Health,Health Finance
abstracts: cdata!: This study employs a cluster randomized
            controlled trial and administrative health center data to
            investigate the effects of authorizing community health
            workers to deliver a new generation of contraceptive
            injections directly to women during routine home visits
            following comprehensive training. The paper observes a
            significant increase of approximately 70 percent in the
            administered quantity of these injections, which provide
            average protection for three months. However, the results
            suggest that the intervention does not produce a
            statistically significant change in contraceptive coverage
            because of significant substitution effects away from
            long-acting contraceptive implants and intrauterine devices
            that women might otherwise have adopted.
display_title: Community Health Workers as Key
            Providers of Easy-to-Use Contraceptive Injectables :
            Experimental Evidence from Rural Burundi
listing_relative_url: /research/2025/02/34459367/community-health-workers-key-providers-easy-to-use-contraceptive-injectables-experimental-evidence-rural-burundi
topicv3: Health Worker Training and Education
            m1326147 1337,Contraceptives m1327726 1453,Community-Based Health Workers m1504315 3587,Health Behavior Change Uptake m1504354 3659,Autonomy of Health Facilities m1504663 3592
publishtoextweb_dt: 2025-02-24T00:00:00Z
dois: 10.1596/1813-9450-11074
origu: Off of Sr VP Dev Econ/Chief Econ (DECVP)
no_of_pages: 30
author_keys: 0: author_key: 1895443
1: author_key: 1223885
2: author_key: 810348
3: author_key: 1303688
4: author_key: 1663190
model_extractions: Policy Research Working Paper 11074 Community Health Workers as Key Providers of Easy-to-Use Contraceptive Injectables Experimental Evidence from Rural Burundi _Michele Andreottola Olivier Basenya Victor Orozco-Olvera Arndt Reichert Paula Spinola_ Development Economics Development Impact Group February 2025 A verified reproducibility package for this paper is available at http://reproducibility.worldbank.org, click **here** for direct access.
Policy Research Working Paper 11074

# **Abstract**

This study employs a cluster randomized controlled trial and administrative health center data to investigate the effects of authorizing community health workers to deliver a new generation of contraceptive injections directly to women during routine home visits following comprehensive training. The paper observes a significant increase of approximately 70 percent in the administered quantity of these injections, which provide average protection for three months. However, the results suggest that the intervention does not produce a statistically significant change in contraceptive coverage because of significant substitution effects away from long-acting contraceptive implants and intrauterine devices that women might otherwise have adopted.

This paper is a product of the Development Impact Group, Development Economics. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at mandreottola@worldbank.org, obasenya@worldbank.org, vorozco@worldbank.org, arndt.reichert@ihe.uni-hannover.de, or paulade@iadb.org. A verified reproducibility package for this paper is available at http://reproducibility.worldbank.org, click **here** for direct access.

<!-- Start of picture text --> TRANSPARENT<br>ANALYSIS<br>G<br>R E<br>N<br>S<br>I P<br>Y E<br>K A<br>C A<br>R P<br>LI R<br>O E<br>O C<br>W R<br>H<br>S<br>P<br><!-- End of picture text --> _The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
Community Health Workers as Key Providers of Easy-to-Use Contraceptive Injectables: Experimental Evidence from Rural Burundi<sup>1</sup> Michele Andreottola<sup>3</sup> Olivier Basenya<sup>4</sup> Victor Orozco-Olvera<sup>3</sup> Arndt Reichert<sup>5</sup> Paula Spinola<sup>2</sup><sup>_,_6</sup>

# **Key words:** Family planning, fertility preferences, contraceptive injections, community health

1 Computational replicability was verified by the World Bank Development Impact (DIME) Analytics team. We thank the World Bank’s Strategic Impact Evaluation Fund for providing financial support for this research. We are grateful to the Burundi’s Ministry of Health, particularly Cyrille Sindahabaye, Stanislas Nyabenda, Olivier Gahungere, and Anaclet Nahayo, for promptly sharing relevant information on the evaluated intervention. We appreciate the detailed feedback provided by Ann-Kristin Reitmann, Christoph Strupat, and Jan Priebe. We also thank the participants of the 2024 German Health Economics Association conference and the University of Hannover internal seminars. This study was registered with the stated objective of assessing the privacy benefits of Sayana Press at the American Economic Association’s registry for randomized controlled trials (https://www.socialscienceregistry.org/trials/3947). This research received ethical clearance from Solutions IRB under the protocol 0373).

- 2Corresponding author: paulade@iadb.org

- 3DIME, The World Bank.

> 4Burundi Country Office, The World Bank.

- 5Institute of Health Economics, University of Hannover, Germany.

> 6Inter-American Development Bank - IDB.
workers, randomized field experiment, administrative data, health centers.

**JEL Codes:** I10, I15, I18, J13.

2
# **1 Introduction**

In Sub-Saharan Africa and many other parts of the world, there persists a notable discrepancy between high fertility rates and the desire of many women for fewer children (United Nations, 2022; Dupas, 2011). This underscores the significant challenge of low utilization rates of (free-of-charge) modern contraceptive services. Governments and development organizations are increasingly focused on enhancing contraception coverage not only to support women in aligning reproductive choices with their preferences and reducing the risks associated with early childbearing (Patton et al., 2009), but also to capitalize on the substantial economic advantages such measures can provide including for instance, professional career investments of women (Ashraf et al., 2014; Goldin and Katz, 2002).<sup>1</sup> Accessing contraceptives typically requires women to visit health centers, as current methods, such as the most popular intramuscular contraceptive injections in Sub-Saharan Africa (Tsui et al., 2017), require administration by trained and highly-skilled health care professionals (PATH, 2018). Barriers such as travel distance, transportation costs, and logistical challenges related to scheduling appointments and arranging child care likely deter utilization. Long wait times and the risk of not finding qualified staff available may further discourage visiting health centers. Additionally, entrenched social norms and significant stigma surrounding contraception potentially constrain women from seeking birth control at health centers because of confidentiality risks (Bassi and Rasul, 2017; Munshi and Myaux, 2006). These risks arise from, for example, being seen entering health centers or the lack of private spaces within them, which can disclose a woman’s contraceptive intentions. While reducing the number of health center visits, home delivery by _external_ health care officials (i.e., professionals from outside the community) can also inadvertently disclose contraceptive use to community members who typically associate these professionals with delivering family planning services (Ndayizigiye et al., 2017).

In this randomized field experiment conducted in a rural setting, we estimate the effect of authorizing _internal_ (i.e., local community members), low-skilled community health workers (CHWs) to administer contraceptive injectables within villages following comprehensive train-

> 1 Improved contraceptive coverage is known to spur economic growth by increasing the proportion of working-age population and optimizing family investments in education and health (Bloom et al., 2009; Miller, 2010; Angrist et al., 2010).

3
ing. A large network of CHWs typically performs routine home visits in their area of residence for various services unrelated to contraception, thus avoiding the need for extra appointments and reducing the ability of the social environment to infer about the provision of injectable shots.<sup>2</sup> We take advantage of a unique setting in Burundi, where Sayana Press, a new generation of non-intramuscular contraceptive injections, had been deployed across the country in recent years. Their operational advancement consists of simplified injections by using a pre-filled syringe for subcutaneous administration, making it easier for lower-skilled professionals to administer while offering the same three-month protection as the traditional injectable method (Keith et al., 2014; Askew and Wells, 2018). However, their key attribute for rural areas—safe administration by CHWs (Burke et al., 2014)—was not yet leveraged. In coordination with health authorities willing to allow administration upon proper theoretical and practical training more broadly and with a large World Bank operation providing financing, we implemented an intervention at the health-center level to make CHWs eligible for administering non-intramuscular injections.<sup>3</sup> It comprised a theoretical training that would help CHWs encourage rural women to visit health centers for medical assessment and receive their initial non-intramuscular injection. After completing a practical phase, CHWs became eligible to administer injection renewals during their routine home visits in the communities. Rather than the overall effect of introducing the new generation of injectables, the randomized intervention allows us to evaluate their novel feature of a substantively easier but still equally safe administration, which makes it feasible to entrust CHWs with the deployment of contraceptive injections in remote areas.

Using detailed administrative data from health centers, we find that the average number of non-intramuscular injections administered was approximately 13 units higher per month within health centers’ catchment areas due to the intervention, representing roughly a 70% increase over the mean of the control group. This statistically significant increase was observed during both the theoretical and practical phases of the CHW training. After certification, there was also a strong and statistically significant increase in renewal injections administered by CHWs

> 2 There is also the possibility that the intervention improves bodily autonomy vis-a-vis the partner (Ashraf et al., 2014) and other household members, as appointments can be scheduled at short notice at a convenient time when privacy is ensured.

> 3 As explained in Section 2, health centers manage services provided both in their facilities and within communities in their designated catchment area.

4
during home visits.

However, the total quantity of health centers’ dispensed contraceptives was not significantly affected by the intervention. Consistent with this finding, we observe a statistically significant shift away from long-acting contraceptive methods which would have been introduced in the absence of the intervention, particularly implants and intrauterine devices (IUDs). The estimated monthly decline in the use of these devices amounts to 2.7 units per health center, which greatly diminished the protection benefits of Sayana Press. Accounting for the different protection horizons of the available contraceptive methods, explorative estimates suggest that the intervention, in net terms, did not result in added contraceptive coverage throughout the analysis period.

Our paper contributes to the literature on the effects of public efforts to enhance contraceptive protection. Most existing studies focus on improving or facilitating access to health facilities and family planning centers (Dupas et al., 2024; Miller, 2010; Ferre et al., 2023; Cronin et al., 2018; Karra et al., 2022) and promoting awareness through education interventions (Strupat, 2017; Carr and Packham, 2017). Community outreach programs have also been studied (Sinha, 2005; Phillips et al., 2012; Joshi and Schultz, 2013; Desai and Tarozzi, 2011). However, none of these studies examines a family planning strategy that fully leverages the advantages of an established rural health service delivery approach centered around CHW routine home visits for the provision of modern contraceptives. Our study therefore fills an important evidence gap. It is also the first to elucidate the potential of the substantively simplified administration of the new generation of injectables to enhance contraceptive coverage rates, whose relevance is arguably further heightened when women eventually are entrusted with self-administration (Burke et al., 2018; Aderoba et al., 2023). Additionally, given the added privacy arguably associated with the examined intervention, our paper relates to previous literature on interventions aimed at enhancing the privacy of health services (Ashraf et al., 2014; Church et al., 2013; Li et al., 2013).

The remainder of this paper is structured as follows. Sections 2 and 3 present the background and the intervention under evaluation, respectively. Section 4 details the study design, including the randomization procedure, the data used, the study sample, as well as the empirical strategy adopted. Results are reported in Section 5 and further discussed in Section 6.

5
Finally, Section 7 concludes.

# **2 Background**

## **Contraceptive Coverage and Provision**

Burundi has one of the lowest contraceptive coverage rates globally (United Nations, 2022). In 2017, only 21.5% of women aged 15 to 49 used reversible modern contraceptive methods such as pills, condoms, injectables, implants, and IUDs, while 6.3% used traditional methods (Burundi Institute of Statistics and Economic Studies, 2017).<sup>4</sup> Even though birth control tools are free of charge, there is a significant unmet need for family planning in the country, with three out of every ten women of childbearing age not using any contraceptive method despite wanting to space their next birth or stop childbearing altogether. The average number of children per woman in a union is 5.5, which exceeds their desired number of children by 1.9 (Burundi Institute of Statistics and Economic Studies, 2017).

In Burundi, contraceptive services are primarily provided by the government, especially for modern methods. Approximately 85% of modern contraceptives dispensed in the country between 2016 and 2017 were supplied by the public sector (Burundi Institute of Statistics and Economic Studies, 2017).<sup>5</sup> This percentage is expected to be significantly higher in rural areas, where private facilities are scarce. Within this framework, health centers play a crucial role by delivering health planning services to defined catchment areas. Each health center is staffed with skilled personnel, supported by affiliated CHWs ( _Agents de Santé Communautaires_ ) and, occasionally, an associated health promotion officer (HPO, _Technicien de Promotion de la Santé_ ). Although less common, HPOs are trained healthcare professionals who possess more advanced skills compared to CHWs. HPOs and CHWs primarily operate within the community, with their affiliated health centers monitoring their activities and supplying them with the necessary resources for their home visits during monthly coordination meetings. CHWs receive performance incentives based on the delivery of a comprehensive package of community activities, with a key focus on family planning. Their tasks typically include distributing

> 4 These rates decrease to 21.1% and 6.1% in rural areas, where approximately 90% of the population reside.

> 5 Pharmacies are not common providers of modern contraceptive methods; for instance, only 0.2% of administered injectables are dispensed by pharmacies (Burundi Institute of Statistics and Economic Studies, 2017).

6
condoms and pills, as well as referring women to health centers if they express interest in the adoption of more modern family planning alternatives.

## **Sayana Press Introduction**

In early 2019, Sayana Press (SP) was added into the pool of modern contraceptive methods available in Burundi. SP represents a significant technological and operational advancement, enabling lower-skilled professionals to safely administer contraceptive injections (Burke et al., 2014). Unlike the incumbent injectable solution, DMPA-IM, SP uses a relatively short syringe pre-filled with the contraceptive drug (subcutaneous depo medroxyprogesterone acetate). It simplifies the administration process because the syringe is injected into the fat underneath the skin rather than into the muscle and is longer filled from a separate vial (PATH, 2018). SP has been evaluated as equally effective as the DMPA-IM, providing protection for a duration of three months (Keith et al., 2014; Askew and Wells, 2018).

An initial visit to the health center, with or without a referral from community workers, is required for eligibility assessment and administration of the initial SP injection. Renewals, needed every three months for continuous protection, were available at health centers or, depending on availability, by HPOs at the women’s homes upon request. Despite being eligible to administer the injectable, the limited availability of HPOs hindered their involvement in providing SP renewals. Health center personnel and affiliated HPOs, already skilled in delivering DMPA-IM injections, required minimal training to start administering SP due to its simpler application. CHWs were not eligible to administer SP prior to the intervention.

# **3 The Intervention**

The intervention being evaluated involved training CHWs to become certified and administer SP injection renewals during routine home visits within communities. This training was part of the Reproductive Health National Program under the Ministry of Health’s supervision. It was financed by the World Bank through the Investing in Early Years and Fertility in Burundi project, which targeted six of the country’s 18 provinces, namely Bubanza, Cankuzo, Cibitoke, Kirundo, Makamba, and Muyinga.<sup>6</sup>

> 6 The selection of these provinces was based on several criteria, including high children stunting and low percentage of women currently using a modern contraceptive method.

7
The training comprised three stages: a theoretical component, a practical internship, and a final stage where CHWs administered SP renewals within communities. The theoretical sessions, conducted at the provincial level, lasted three days and included approximately 40 CHWs from geographically connected health centers. They aimed to equip CHWs with comprehensive knowledge about Sayana Press, covering product characteristics, potential side effects, counseling skills, and general instructions on how to administer the injectable. The practical internship involved a five-day training at their affiliated health centers, where CHWs gained hands-on experience administering SP injections under the supervision of health staff. CHWs needed to apply five SP injections to complete their internship and be certified for its administration.

Figure 1 outlines the timeline for the introduction of the SP program and the CHW training sessions. Upon our engagement with the ministry, health centers started recording SP injections in January 2020. In April 2022, CHWs in the treatment arm commenced the theoretical stage of training, followed by the practical internship in October 2022. By November 2022, all health centers in the treatment group had completed the practical internship phase and were authorized to offer the injectable within their communities. Since CHW training was extended to the control group in December 2023, our analysis focuses on the period before this month.

8
Figure 1: Intervention timeline

### Introduction of Sayana Press

- first injection delivered in health centers

<!-- Start of picture text --> • renewals delivered in health centers (or during home visits by HPO, with very limited<br>CHW practical<br>training<br>CHW CHW eligible<br>Qty of delivered SP injections theoretical to deliver<br>started to be recorded training SP renewals<br>Treatment<br>Control<br>Jan 2019 Jan 2020 Apr 2022 Oct 2022 Dec 2023<br>Jan 2020 - Nov 2023: Start of<br>analysis period CHW training<br>(control group)<br><!-- End of picture text -->

- renewals delivered in health centers (or during home visits by HPO, with very limited capacity)

# **4 Study Design**

## **Randomization**

Our study sample includes all health centers within the six provinces targeted by the World Bank operation, where SP was already available but CHWs had not yet been trained to administer the injectable. This cluster randomized controlled trial assigned half of the 138 health centers to either a treatment or a control group. Nearly all the health centers in our study sample are classified by the government as rural (136 of 138). To improve comparability between the treatment and control groups and achieve efficiency gains (Bruhn and McKenzie, 2009), the randomization was stratified based on province, population density, total number of contraceptive products, and the proportion of alternative contraceptive injections (DMPA-IM) delivered in 2021. The group assignments for health centers were disclosed to the government on February 17, 2022.

## **Data and Outcomes**

9
The data were sourced from the National Health Information System (NHIS). This system provides monthly data at the health center level on the quantity of each contraceptive method distributed within their designated catchment area by the Burundian government as well as the number of CHW referrals to health center facilities. An important limitation of the NHIS data is the difficulty of distinguishing between true zeros and missing values when a health center reports no data for a contraceptive method in a given month. To partly address this issue, we classify observations as missing if all contraceptive methods show zero quantities for a given health center and month. This classification applies to approximately 5% of our sample.

The primary outcome of our study is the quantity of SP injections reported by health centers in the NHIS data. As secondary outcomes, we employ the quantities of relevant alternative contraceptive methods to investigate potential substitution effects.

# **Study Sample**

Table 1 summarizes key information on the 138 study health centers, revealing the limited capacity of health centers and low contraceptive utilization (column 1). Panel A shows that only 38 health centers (28%) had HPO availability. Among these health centers, each had only one HPO, indicating very limited in-home administration capacity before the CHW intervention rollout. On average, there were 16 CHWs serving a population of 15,775 within the health centers’ catchment areas. Of this population, an estimated 3,739 are women of childbearing age, translating to roughly 230 women per CHW.

Panel B presents the baseline number of SP injections administered per month within the catchment area of health centers as well the overall quantity of all contraceptives dispensed, including injectables (SP and DMPA-IM), condoms, birth control pills, emergency contraceptive pills, implants, and intra-uterine devices. It is worth noting that this measure should be interpreted with caution given the varying protection duration across contraceptive methods. A detailed breakdown by each contraceptive method is provided in Table A1. Given the estimated average of 3,739 women of childbearing age residing in the catchment area served by each health center, these figures suggest low contraceptive coverage in our study areas. According to Burundi Institute of Statistics and Economic Studies (2017), the weighted average contraceptive 10
coverage across the six provinces for the methods listed in Table A1 was 20.8% during the years of 2016 and 2017, just below the national average of 21.5% (Burundi Institute of Statistics and Economic Studies, 2017).<sup>7</sup> Table 1: Characteristics and contraceptive <u>provision</u> by health centers

||(1)<br>All|(2)<br>Treatment|(3)<br>Control|(4)<br>Treatment –<br>Control|
|---|---|---|---|---|
|No. health centers|138|69|69|-|
|_Panel A: Cha_|_racteristi_|_cs of health c_<br>_Mean/(SD)_|_enters_|_Mean diference_|
|% of health centers with HPO availability|0.28<br>(0.45)|0.33<br>(0.48)|0.22<br>(0.42)|0.12|
|No. HPO (conditional on >0)|1.0<br>(0.00)|1.0<br>(0.00)|1.0<br>(0.00)|-|
|No. CHW|16.12|15.29|16.94|-1.65|
||(9.6)|(9.34)|(9.86)||
|Distance (in km) to local health authority|27.48|30.11|24.85|5.26|
||(20.14)|(21.8)|(18.1)||
|Distance (in km) to regional health authority|14.95<br>(10.09)|15.15<br>(9.34)|14.76<br>(10.86)|0.39|
|Population|15775|15387|16163|-776|
||(8564)|(7465)|(9577)||
|Childbearing-aged women|3739|3647|3831|-184|
||(2030)|(1769)|(2270)||

_Panel B: Monthly number of contraceptives administered by health centers between Jan 2020 and Mar 2022_

|||_Mean/(SD)_||_Mean diference_|
|---|---|---|---|---|
|Sayana Press|8.17|8.47|7.87|0.6|
||(13.63)|(15.2)|(11.96)||
|Any contraceptive dispensed|447.4|445.5|449.3|-3.7|
||(328.5)|(305.1)|(352.6)||

Notes: Mean (SD) represents the mean and standard deviation at the health center level, while the last column shows the mean difference between the treatment and control groups. Significance: ***=.01, **=.05, *=.1 Panel A reports characteristics of health centers, based on data provided by the Burundian Ministry of Health in the first trimester of 2024. The count of women of childbearing age (15-49 years old) at each health center is estimated by multiplying the local population within the health center’s catchment area by the national proportion of this demographic (23.7%). The local health authority refers to the nearest administrative office within the community, while the regional health authority corresponds to the official health district office where the health center is located. Panel B presents the monthly quantity of contraceptives administered during the baseline period, according to the National Health Information System (NHIS). Data for the period between January 2020 and March 2022 (i.e., before training was implemented in the treatment arm) were collapsed at the health center level.

# All statistics presented in the table are statistically similar between the treatment and

# control groups (columns 2 and 3). We observe statistically significance differences neither in the capacity levels of health centers nor contraceptive delivery between the two groups (column

> 7 The weighted average of contraceptive coverage takes into account the proportion of each province in our sample. The province-level coverage rates for 15-49 year old women in unions at the time of the DHS survey were reported as follows: 16.2% for Bubanza, 20.3% for Cankuzo, 12.8% for Cibitoke, 27.4% for Kirundo, 8.6% for Makamba, and 27.8% for Muyinga (Burundi Institute of Statistics and Economic Studies, 2017). These rates encompass reversible modern contraceptive methods such as injectables, monthly pills, morning-after pills, condoms, implants, and IUDs. When considering sterilization, natural, and traditional methods, coverage rises to 26.8% in our sample and 28.5% nationally.

11
<!-- Start of picture text --> Ke) Pratical training<br>Hn i Theoretical a<br>4==i=°oC7) 2© training otmtA44 Eligibility period<br>-£mLo~ia'ot1' _<br>ooa t1! '11 ,>~* \ \<br>.- a / an<br>”goc oO - wm e -x’\,/,-_-<br>S ‘ eS fain > was ~s . —<br>oO<br>xo]<br>7)<br>77)<br>ar)<br>a oO<br>ocodn © ty ‘ mt"\a<br>® r . 5 aa i \ vf 1 .<br>—=> S v\\ Jtf ‘iv ~ ~ , ‘\so > ‘4 “1 \\ >7 an 11 \A,v/ “4 vaMu 1 !14\ ‘ve Ud? . A ‘ ‘ ;<br>rT) + vw \, ve vl Xv ‘ \ . Ny ‘\ ’<br>%<br>pe<br>-5 --- Treatment group<br>2 S| — Control group<br>8 “N<br>Ee Q So = = A n nw wy ©<br>ro)N SN rot]N N i)N NOo NSo ~No No agoO Qo ag<br>s 3 °s 3 S$= «€£- 5F2 8O08 S$€&€ §&£ 57 86<br><!-- End of picture text -->
were observed between the treatment and control groups after the rollout of the intervention.

## **Empirical Strategy**

We implement a differences-in-differences strategy to assess the extent to which the intervention contributed to the delivery of SP injections. While positive trends are anticipated for both groups considering its ongoing SP expansion, we hypothesize that the increase will be significantly higher among health centers in the treatment arm.<sup>10</sup> The empirical specification of the model is presented below.

where the left-hand side variable represents the outcome in health center _i_ at month _t_ . _Treati_ is a dummy variable that takes the value one if health center _i_ is part of the treatment group, and zero otherwise. _Postt_ equals one if month _t_ falls after or during April 2022, and zero otherwise. Strata fixed effects, _Stratai_ , are also added to the regression model.<sup>11</sup> We estimate it employing ordinary least squares. The error term is denoted by _εit_ . To account for serial correlation within health centers’ catchment area, we cluster standard errors at the health center level.

The interaction term coefficient, _β_ , captures the average treatment effect. Random differences between treated and control groups during the baseline period are captured by _λ_ . Finally, _θ_ captures the changes experienced by the control group during the post-intervention period compared to the baseline period, representing the counterfactual scenario of how the treatment group would have evolved had the intervention not been implemented. Results are also estimated for alternative contraceptive methods as well as for the overall quantity of all contraceptives dispensed within the health centers’ catchment areas.

# **5 Results**

## **Effects on Sayana Press Injections and Overall Contraceptive Delivery**

> 10 As previously explained, the quantity of contraceptives reported by health centers includes provisions made both in their facilities and within the local communities of their designated catchment area.

> 11 The stratification process during randomization distributed the 138 health centers in our sample into 21 different strata (for details on stratifying variables, see the previous section).

13
Column 1 of Table 2 shows results for the effects of the intervention on the total number of SP shots. We find an additional 12.72 units in the treatment group per month and health center. Given a post-intervention quantity in the control group of 18.47 (10.22 + 8.25), our point estimate indicates an increase in SP injections of roughly 70% (12.72/18.47). Results are robust across Poisson and Negative Binomial regression specifications.

<u>Table 2: Efect on the quantity of Sayana Press and overall contraceptives dispensed</u>

||(1)|(2)<br>Sayana P|(3)<br>ress|(4)|(5)<br>Any|
|---|---|---|---|---|---|
||Total|Health Center|Home|visit|contraceptive|
||||By HPO|By CHW|dispensed|
|**_Treat x Post_**|**12.72*****|**9.38*****|**0.18**|**3.16*****|**8.42**|
||**(2.78)**|**(2.53)**|**(0.19)**|**(0.59)**|**(36.27)**|
|_Post_|10.22**|9.85***|0.18**|0.19*|-4.30|
||(1.25)|(1.26)|(0.07)|(0.11)|(28.29)|
|_Treat_|0.62|0.58|-0.06|0.10|-7.02|
||(2.16)|(2.13)|(0.09)|(0.11)|(41.96)|
|Observations|6,139|6,139|6,139|6,139|6,139|
|Baseline mean|8.25|8.16|0.08|0.00|450.68|

Notes: The table presents results based on the estimation of Equation 1, where the outcome concerns quantities of Sayana Press injections as described in the column titles. Standard errors, presented in parentheses, are clustered at the health center level. The stars next to the estimated coefficients follow the usual convention (*** p<0.01, ** p<0.05, * p<0.1).

Columns 2 to 4 of Table 2 break down the total quantity of SP injections into shots delivered by the health center and (renewal) shots by HPO and CHW, respectively. We observe that the effects are driven in large part by shots in health centers and to some extent by CHWs. Shots provided by HPO, on the contrary, were not affected by the intervention. While the initial shots were always required to be provided by the health center, we note that about half of the shots added as a result of the intervention were still administered by the centers during the eligibility phase (Appendix Table A2).<sup>12</sup> The last column of Table 2 displays the estimated impact of the intervention on the total number of contraceptives dispensed, which is statistically insignificant. The remaining part of this section explores the possibility that the intervention may have caused substitution away from other contraceptive methods.

> 12 We attribute the increase in SP injections at health centers in the later phases of the intervention to several possible factors. First, shots at health centers remained a feasible alternative for renewal administration after CHWs were granted eligibility. Second, the intervention’s training increased health center staff’s awareness and knowledge of SP. Third, CHWs administered several shots under supervision at health centers. We describe plausible mechanisms in more detail in the mechanism section of the Appendix.

14
# **Substitution Effects**

Given the focus of the intervention on the integration of CHWs into the delivery of contraceptive injectables, we might expect possible substitution effects to be concentrated among alternative contraceptive methods that are routinely administered by them and therefore potentially crowded by the new injectable method. However, as shown in the previous section, Sayana Press dispensing in health centers was also strongly influenced by the intervention, which makes it important to examine its effects on the broad spectrum of contraceptive methods.

Table 3 shows the intervention’s effects on the use of alternative reversible modern contraceptive methods. To address multiple hypothesis testing concerns, we include adjusted p-values in brackets, based on Romano and Wolf (2005), following the procedure outlined by Clarke (2021). We observe negative coefficients for DMPA-IM, monthly pills, and implants/IUD, suggesting a switch from these alternative methods to Sayana Press. While the negative coefficients for DMPA-IM injections and monthly pills are not statistically significant,<sup>13</sup> we observe a statistically significant decrease of 2.7 per month in the introduction of new IUDs/implants. This indicates that the additional SP injections due to the intervention at least partly replaced the adoption of long-acting devices that would have occurred in the absence of the intervention. The last column of the table indicates that the intervention did not induce the removal of previously inserted devices.<sup>14</sup>

> 13 Lower adjusted p-values may occur when accounting for the dependence structure in the data during bootstrap resampling. This phenomenon was observed for monthly pills and device removal, where RomanoWolf p-values pointed to statistical significance at a 10% level. Our reference for conventional levels is set at 5%.

> 14 The positive coefficients observed for condoms and morning-after pills, which provide protection for individual instances of sexual intercourse, may indicate their use for one-time protection to avoid unintended pregnancies during transitions between longer-term birth control methods. The coefficients are highly statistically insignificant, possibly due to varying degrees in which this approach is used among the limited group of women switching between contraceptive methods.

15
Table 3: Effect on the <u>quantity</u> of alternative contraceptives delivered, by type

||(1)|(2)|(3)|(4)|(5)|
|---|---|---|---|---|---|
||Injectable|Short-|acting|Long-|acting|
||DMPA-IM|Monthly Pill|Condom &<br>Morning Pill|Implants & IUD<br>Introduced|Implants & IUD<br>Removed|
|**_Treat x Post_**|**-5.06**|**-8.67**|**12.15**|**-2.70****|**-0.33**|
||**(5.90)**|**(6.51)**|**(33.50)**|**(1.05)**|**(0.28)**|
||**[0.159]**|**[0.052]**|**[0.494]**|**[0.004]**|**[0.064]**|
|Post|-3.48|2.94|-17.98|4.01***|0.16|
||(4.65)|(4.23)|(26.92)|(0.80)|(0.19)|
|Treat|11.91*|7.88|-27.99|0.55|0.28|
||(6.85)|(5.49)|(38.08)|(1.03)|(0.37)|
|Observations|6,139|6,139|6,139|6,139|6,486|
|Baseline mean|95.37|43.03|293.91|10.11|2.19|

Notes: The table presents results based on the estimation of Equation 1, where the outcome concerns quantities of contraceptives described in the column titles. Standard errors, presented in parentheses, are clustered at the health center level. The stars next to the estimated coefficients follow the usual convention (*** p<0.01, ** p<0.05, * p<0.1). To interpret the statistical significance of the interaction term coefficient, we correct for multiple hypothesis testing following the procedure outlined by Clarke (2021). More specifically, we use the Stata command -rwolf2with 250 bootstrap replications to obtain Romano-Wolf p-values which are presented in brackets.

# **6 Discussion**

Our estimates should be considered lower bounds for the effects of the intervention on SP injections and delivery of alternative methods in the presence of potential spillover effects, such as information spreading from treated health centers to control health centers. While the observed (lower) positive trend in SP adoption for the control group (as indicated by the coefficient of the binary post-intervention dummy variable in Table 2) was anticipated due to the ongoing expansion of the introduced innovation, it is possible that such spillover effects also played a role in the positive trend of SP injections in the control group. The risk of information spillover becomes more pertinent towards the final period of our analysis, as control units were closer in time to receiving the intervention themselves.

The study’s main limitation is related to data constraints and the restricted time horizon available for the analysis. First, the inability to distinguish zero values from missing information in our data implies that our analyses suffer from measurement error in the dependent variable, which yields inflated standard errors. Second, we are unable to ascertain whether women initiating SP receive timely renewals, i.e., maintaining uninterrupted contraceptive protection, since we cannot track them over time with aggregated health facility data. Third, we do not have information on women’s demographics in our data. Consequently, we cannot 16
determine whether women who would have chosen long-acting devices belong to the same demographic group as those opting for the new product. Contraceptive coverage for younger women is particularly beneficial, given the disruptive impact of unintended pregnancies on this demographic, especially teenagers.<sup>15</sup> Finally, we are unable to assess the longer-term effects of the intervention.

# **7 Conclusion**

This paper assesses the effects of an intervention designed to incorporate community health workers as key participants in the distribution of a new generation of injectable contraceptives in a rural setting. CHWs in the treatment group received theoretical and hands-on-training followed by eligibility to administer these non-intramuscular injections directly to women at their homes. In contrast, in the control group, the distribution of the new contraceptive was exclusively handled by more skilled workers primarily within health centers.

Our experimental findings indicate a notable change in injectable contraceptive delivery by health centers. On average, we observe roughly 12.72 additional Sayana Press injections as a result of the intervention. This implies an increase of 70 percent over the control group mean. Assuming uniform coverage rates across the six provinces in our sample, this point estimate corresponds to roughly 8% of the overall coverage rate in 2017, estimated to range around 20.8%.<sup>16</sup> However, despite this strong surge in the number of administered injections, our experimental results concerning the overall number of dispensed contraceptives, albeit positive, are statistically insignificant. We explain this finding by substitution effects away from long-acting contraceptive devices. Specifically, we observe a monthly decrease of 2.70 implants and IUDs due to the intervention. In an exploratory analysis that accounts for differences in the protection horizon offered for each of the contraceptive methods dispensed by health centers in our sample, in net terms, we find no evidence that Sayana Press significantly affected the overall contraceptive coverage. If at all, our analysis points to a statistically insignificant net reduction

> 15 Past surveys suggested that injectable users tend to be slightly younger than women adopting long-acting devices (Burundi Institute of Statistics and Economic Studies, 2017).

> 16 This corresponds to a total number of 65.2 new users throughout the evaluated intervention (3 _._ 26 _∗_ 20), which represents approximately 1.7% of the local target population (65 _._ 2 _/_ 3 _,_ 739).

17
# (see the Appendix for details).

In the overall assessment of the novel intervention examined in our paper, we aim to take a quite nuanced perspective. On the one hand, our results show that extending contraceptive provision to community health workers has the potential to effectively increase adoption of the latest generation of injectables in rural villages. By educating and training community health workers to administer injections locally, the scale-up of the new family planning services can be expedited. These results underscore the promising potential of the self-administration attribute of the injectable. On the other hand, the intervention did not significantly increase overall contraceptive coverage, meaning it failed to better match contraceptive options with the preferences of women in the study. While the rise in new contraceptive injections suggests an improved fit of the menu of contraceptive services for women in the area, the decline in the use of longer-acting methods is a concern for public health and economic development, especially when follow-up injections are missed.<sup>17</sup> Our findings highlight the need for further research. First, individual-level data will be required to identify which sub-populations are gaining protection coverage and which are losing it as a result of the intervention. Second, a more comprehensive intervention should be tested that exploits the potential associated with the self-administration attribute of the new injectable method while ensuring that the adoption of longer-acting contraceptive methods is not penalized. This may involve a better integration of service delivery models and associated health worker incentives, i.e., actively offering the entire contraceptive choice set and measuring health center performance based on women-months of coverage. A promising component is educational campaigns that highlight the benefits of the various contraceptive methods with an emphasis of the importance of maintaining continuous protection. Third, future research should aim at separately exploring the relevance of the various barriers to contraceptive utilization. This knowledge will likely stimulate development of tailored contraceptive technologies and research on effective service delivery strategies.

> 17 This becomes even more critical considering evidence that injectables’ discontinuation rates are roughly double those of implants. According to Burundi Institute of Statistics and Economic Studies (2017), implants have a discontinuation rate of 21%, contrasting with injectables at 47%. This discrepancy likely stems from the requirement to physically remove the implant for discontinuation, whereas discontinuation of injectables often occurs when women fail to receive timely renewals when the efficacy of the last injection wanes.

18