metadata: authors: 0: author: Carranza,Eliana
1: author: Meeks,Robyn
count: Kyrgyz Republic
docna: 0: docna: Shedding light : understanding energy
            efficiency and electricity reliability
docty: Policy Research Working Paper
subtopic: Disease Control & Prevention,Public Health Promotion,Reproductive Health,Early Child and Children's Health,Nutrition,School Health,Educational Populations,Energy Policies & Economics,Hydrology,Education for Development (superceded),Education For All,Energy Demand,Energy Finance,Energy and Environment,Energy and Mining
teratopic: Environment,Energy
abstracts: cdata!: Overloaded electrical systems are a
            major source of unreliable power (outages) in developing
            countries. Using a randomized saturation design, we estimate
            the impact of energy efficient lightbulbs on household
            electricity consumption and local electricity reliability in
            the Kyrgyz Republic. Receiving compact fluorescent lamps
            (CFLs) significantly reduced household electricity
            consumption. Estimates not controlling for spillovers in
            take-up underestimate the impacts of the CFLs, as control
            households near the treated are likely to take-up CFLs
            themselves. Greater saturation of CFLs within a transformer
            leads to aggregate reliability impacts of two fewer days per
            month without electricity due to unplanned outages relative
            to pure controls. Increased electricity reliability permits
            households to consume more electricity services, suggesting
            that CFL treatment results in technological externalities.
            The spillovers in take-up and technological externalities
            may provide an additional explanation for the gap between
            empirical and engineering estimates of the impacts of energy
            efficient technologies.
geo_reg_key: 517208,517191,82689,545134,517198
repnme: repnme: Shedding light : understanding energy
            efficiency and electricity reliability
display_title: Shedding light : understanding
            energy efficiency and electricity reliability
listing_relative_url: /research/2016/11/26939234/shedding-light-understanding-energy-efficiency-electricity-reliability
topicv3: Water m1326522 137,Energy and Extractives m1326595 507,Deworming m1326778 1541,Power and Electricity Sector m1327094 515,Household Energy Expenditures m1503721 2620
publishtoextweb_dt: 2016-11-15T00:00:00Z
historic_topic: Environment,Energy
no_of_pages: 56
author_keys: 0: author_key: 746344
model_extractions: WPS7891 Policy Research Working Paper 7891 Shedding Light Understanding Energy Efficiency and Electricity Reliability _Eliana Carranza Robyn Meeks_ Poverty and Equity Global Practice Group November 2016
Policy Research Working Paper 7891

# **Abstract**

Overloaded electrical systems are a major source of unreliable power (outages) in developing countries. Using a randomized saturation design, we estimate the impact of energy efficient lightbulbs on household electricity consumption and local electricity reliability in the Kyrgyz Republic. Receiving compact fluorescent lamps (CFLs) significantly reduced household electricity consumption. Estimates not controlling for spillovers in take-up underestimate the impacts of the CFLs, as control households near the treated are likely to take-up CFLs themselves. Greater saturation of CFLs within a transformer leads to aggregate reliability impacts of two fewer days per month without electricity due to unplanned outages relative to pure controls. Increased electricity reliability permits households to consume more electricity services, suggesting that CFL treatment results in technological externalities. The spillovers in take-up and technological externalities may provide an additional explanation for the gap between empirical and engineering estimates of the impacts of energy efficient technologies.

This paper is a product of the Poverty and Equity Global Practice Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at ecarranza@worldbank.org and meeks@umich.edu.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# Shedding Light: Understanding Energy Efficiency and Electricity Reliability<sup>_∗_</sup>

Eliana Carranza<sup>_†_</sup> Robyn Meeks<sup>_‡_</sup> Keywords: Energy Efficiency, Externalities, Outages JEL: D01, D62, O13

> _∗_ We thank Arstan Omuraliev and Ruslan Isaev for invaluable input. We are grateful to Sendhil Mullainathan, Rohini Pande, Rema Hanna, Claudia Goldin, Kelsey Jack, Sebastian Bauhoff, Jeremiah Johnson, Hunt Allcott, Raymond Guiteras, Shaun McRae, Mushfiq Mobarak, Ach Adhvaryu, Michael Moore, Catie Hausman, Judd Boomhower, Paul Ferraro, as well as many seminar participants for helpful discussions and comments. Cholpon Chargynova, Saikal Nogoibaeva, Merim Japarova, Wincy Poon, Ryoko Sato, and Daniel Velez-Lopez provided excellent research assistance. Generous funding was provided by the Harvard Sustainability Science Program, Harvard Graduate Student Grant, the University of Michigan, the Weiser Center, and the National Science Foundation (Award #1061989). All errors are our own.

> _†_ World Bank. Email: ecarranza@worldbank.org

> _‡_ University of Michigan. Email: meeks@umich.edu (corresponding author)
# **1 Introduction**

Overloaded systems are a substantial source of unreliable power in developing countries. Overloads occur when the grid is asked to deliver more power than its capacity allows, causing components of the grid to break. Such breakage results in electricity outages, a major problem for electricity service provision in developing countries. This problem is likely to worsen in the future for at least two reasons. First, residential electricity demand is expected to increase with pro-poor development, as households buy their first appliances (Wolfram et al., 2012), putting further pressure on the existing electrical grid. Additionally, in certain settings, subsidized electricity prices provide little incentive for service providers to upgrade infrastructure, resulting in persistent low quality infrastructure (McRae, 2015).

With this in mind, programs distributing energy efficient technologies in developing countries<sup>1</sup> potentially provide an additional benefit beyond reducing emissions and decreasing the cost of energy services to technology adopters. CFLs are frequently deployed en masse in developing countries via programs with the specific goals of reducing electricity outages and increasing reliability of electricity services.<sup>2</sup> Compact fluorescent lightbulbs (CFLs), which consume 25% of the electricity used by traditional incandescent bulbs (per lumin), can reduce overall household electricity consumption without requiring a reduction in the hours of lighting services consumed (DOE, 2009). They may also provide important aggregate impacts in the form of reduced peak electricity load and improved reliability, permitting households to consume more electricity services (World Bank, 2006).<sup>3</sup> Such technological externalities, through which the returns to an individual adopting CFLs are increasing in the fraction of the population utilizing the technology, can be a substantial benefit from adoption of energy efficient technologies.<sup>4</sup>

> 1There are many such programs. Just between 1990 and the mid-2000s (World Bank, 2006), the World Bank alone committed more than US$11 billion to energy efficiency in developing countries.

> 2For example, 600,000 CFLs were distributed in Uganda to reduce peak load by 25 MW. In Rwanda, 400,000 CFLs were distributed to reduce peak load by 16 MW and offset the need for diesel-based power generation. In Ethiopia, 200,000 CFLs were provided to reduce peak load by 6.8 MW to increase reliability.

3At an aggregate level, energy efficient lightbulbs can help address electric power shortages, permit utilities to reach a greater number of customers with existing supplies, reduce need for investment in capacity and distribution, accommodate growth in economic activity, and reduce environmental impact.

> 4We follow the definition of technological externality employed by Foster and Rosenzweig (2010). For a particular technology, adoption of the technology by others is said to generate a positive (negative) technological externality if it increases (decreases) an individual’s returns to the adoption of such technology. Miguel and Kremer (2007) discuss such effects in the context of de-worming medication. By reducing the exposure to individuals infected with worms, the net benefit of the deworming pill to an individual is small (as long as the fraction of treated individuals is reasonably high). Similarly, Dupas (2014) cites medical

1
The potential gains from distribution of energy efficient technologies in developing countries are highly promising; however, there is relatively little evidence to date on the empirical impact of energy efficiency on individual household electricity consumption.<sup>5</sup> Perhaps more notably, there is a dearth of evidence on the technological externalities that may result from the aggregate consumption impacts of energy efficiency. This is likely due to several reasons. First, spillovers in the adoption of energy efficient technologies confound the estimated impact on individual household energy consumption. Second, data on outcomes related to infrastructure quality (such as actual outages) tend to be difficult to acquire for developing countries (Klytchnikova and Lokshin, 2009). And third, levels of energy efficiency take-up and reliability of electricity services within a community are mutually endogenous<sup>6</sup> and jointly explained by community characteristics<sup>7</sup> , making the causal aggregate impact of energy efficiency challenging to estimate.

To overcome these empirical challenges and assess the impacts of energy efficient technologies, we implemented an experimental distribution of CFLs employing a randomized saturation design, which allows us to account for spillovers when estimating the impact of treatment on individual household electricity consumption.<sup>8</sup> In a novel application of the randomized saturation design, we experimentally vary the treatment saturation at a technologicallymeaningful and policy-relevant scale. Specifically, we randomize at the level of the typical infrastructure failure, the distribution transformer.<sup>9</sup> While the direct aggregate impacts of CFL treatment, in the form of reduced transformer-level electricity loads, may be proportional to CFL saturation, the externality benefits are likely to vary non-linearly with CFL technology saturation. By randomizing saturation intensity at the technologically-relevant scale, we are able to test for evidence of technological externalities in the form of increased evidence suggesting that the population of infected mosquitoes decreases as the fraction of households using a high quality bednet increases, thus reducing the individual benefits from bednet use.

> 5The notable exception is a quasi-experimental evaluation of a Mexican appliance (air conditioners and refrigerators) replacement program (Davis, Fuchs and Gertler, 2014).

> 6Greater consumption of CFLs within a community increases electricity availability. Greater electricity availability increases the gains from adoption of CFLs, potentially leading households to consume more CFLs. These effects reinforce each other.

> 7For instance, CFL consumption and electricity availability may be greater in communities with higher education levels, higher incomes, etc.

> 8Baird et al. (2014) demonstrate the use of a randomized saturation design to account for spillovers.

> 9Distribution transformers, which convert higher-voltage electricity from the distribution system to lowvoltage electricity for household use, are a crucial part of the electrical grid and their failure is a common source of electricity outages (Glover, Sarma and Overbye, 2011).

2
electricity service reliability (reduced blackouts) and increased household electricity consumption in higher saturation transformers.

We implement this experiment in the Kyrgyz Republic, a lower-middle-income developing country that suffers from frequent electricity outages and therefore offers suitable conditions for research on electricity reliability. Although CFLs were available in some stores at the project onset, they were not commonly used outside of Bishkek, the capital city. Implemented in a district adjacent to Bishkek, the randomized saturation design employs an electricity utility’s data on its residential customers and the infrastructure through which these consumers are served.

Treatment status was randomly assigned in two stages. First, we randomize electricity transformers to different treatment saturations (i.e. different proportions of households are assigned to treatment within a transformer). Second, we randomize households individually to treatment and control groups, according to the transformer saturation intensity assigned in the first stage. After completing a baseline survey, the treated households receive up to four CFLs at a highly subsidized price. We follow household monthly electricity consumption via the electricity utility’s records for 18 months post-intervention to estimate the impacts on electricity consumption. To measure CFL use and spillovers in adoption, we return for a follow-up survey one year after CFL distribution.

We measure the impacts of CFLs on residential electricity consumption, both with and without controls for any potential spillovers in CFL take-up. The randomized saturation design in the initial CFL distribution allows us to test for such spillovers. In our sample, control households close to treated households are more likely to have CFLs at follow-up than pure control households in located control transformers. Not accounting for spillovers in take-up among the control group leads to estimates of electricity consumption reductions that are downward biased (in other words, they understate the savings from energy efficiency). When accounting for these spillovers, we find that CFLs lead to a significant reduction in monthly electricity consumption that falls within the expected range for the technology based on engineering performance estimates.

Using the transformer-level randomization of treatment intensity, we then test for externality effects resulting from the energy efficiency, in the form of improved electricity reliability.

3
We find that households in transformers with a higher intensity of treatment (i.e., a higher saturation of households within a transformer receive CFLs) have fewer days reported without electricity due to unplanned outages.<sup>10</sup> This result is robust to including a number of controls, such as the baseline number of reported outages, and is supported by calculations of expected peak load reduction. We also find evidence suggesting that households in transformers with improved reliability consume more hours of electricity services due to fewer outages, and that the household energy savings are larger after accounting for this technological externality. Lastly, using these results we perform a simple cost-benefit analysis, which illustrates the importance of accounting for these spillovers and technological externalities when estimating the welfare impacts of such a program.

Our experiment contributes to the literature on energy efficiency in several ways, and offers a novel experimental design for identifying externalities at various treatment levels.

First, it contributes to the debate regarding the impacts of energy efficient technologies and their potential to reduce energy demand. Energy efficiency depends not only on the development of relevant technologies, but also on the interaction of end-user behavior with such technologies (Allcott and Mullainathan, 2010). Prior to ours, no experimental or quasiexperimental studies existed measuring the impacts of energy efficient lighting. The one previous study of a large-scale CFL distribution program found that approximately 20% of initial electricity savings dissipated over time (Costolanski et al., 2013), thereby contributing to skepticism regarding the gains from efficient lighting technologies and from energy efficiency more generally. Our experimental finding that CFL treatment results in a significant electricity consumption reduction very close in size to the expected technologically feasible reductions contributes to balance the evidence from such existing misidentified studies. Our finding is not only relevant for developing countries, but is also consistent with results from the United States, where lighting is a relatively smaller share of energy consumption.<sup>11</sup> This new evidence suggests that energy efficient lighting, and perhaps other energy efficient technologies as well, should remain on the menu of potential policy options.

Second, to our knowledge, our experiment is the first to use a randomized saturation design

> 10We collect data on reported days without electricity due to unplanned outages during both the baseline and follow-up surveys.

> 11Burlig et al. (2016) are performing a machine-learning analysis of energy efficiency in California schools. Their preliminary results indicate substantial savings from the energy efficient lighting interventions.

4
to provide evidence of technological externalities at multiple policy-relevant levels. Given the scale at which technological externalities typically occur, designing an experiment that can identify such externalities is challenging; as a result, there is relatively little causal empirical evidence on them.<sup>12</sup> This study designs such experiment by taking into account the constraint within the electricity distribution system that most frequently causes electricity outages and randomly varies the saturation intensity of CFLs at that level. By doing so, we are able to provide evidence of technological externality effects on both electricity reliability at the transformer level and on household electricity consumption. We gauge how the externality varies with the number of treated households within a transformer to decompose the technology’s overall effect on household energy consumption into its direct and externality components. Our results suggest that energy efficiency programs can improve electricity reliability, not only in the Kyrgyz Republic but also in other contexts, if they reach sufficient levels of saturation. More generally, they speak to the extent to which energy efficiency programs can accomplish various, and sometimes opposing, goals using a technology-based solution.<sup>13</sup> Third, our experiment contributes to the discussion regarding the energy efficiency gap. In spite of their promise as potentially welfare-improving, many believe households are not using energy efficient technologies when they should. Building upon Hausman (1979), cumulative research suggests an energy efficiency gap due to individuals not maximizing the net present value of their energy spending when making energy purchase decisions.<sup>14</sup> There is a growing body of work investigating the factors impacting purchase decisions of energy efficient technologies, thereby causing these investment inefficiencies (Jaffe and Stavins, 1994; Allcott and Greenstone, 2012; Gillingham and Palmer, 2014). Most existing economic research on the take-up of energy efficient technologies largely focuses on private adoption decisions and

> 12A notable exception is Miguel and Kremer (2004), who provide experimental evidence of positive crossschool externalities from deworming medicines in Kenya, but have to rely on more tentative non-experimental methods to decompose the overall effect on treated schools into a direct effect and within-school externality effect. As acknowledged by them: “When local treatment externalities are expected, field experiments can be purposefully designed to estimate externalities by randomizing treatment at various levels. [...] However, this multi-level design may not be practical in all contexts: for example, in our context it was not possible to randomize treatment within schools. Randomization at the level of clusters of schools also dramatically increases the sample size needed for adequate statistical power, raising project cost.”

> 13For example, energy efficiency programs may be designed to reduce overall consumption and pollution or to improve service reliability in a way that could result in increases in electricity consumption.

> 14Energy efficient products often require a larger upfront cost than the standard products, but exhibit lower operating costs. Consumers’ decision to invest in energy-saving devices relies on this trade-off between initial investment and operating costs.

5
the returns to individual adopters.<sup>15</sup> In contrast, very little attention has been paid to the role of spillovers in adoption decisions and technological externalities. In their presence, the level of private investment in some energy efficient technologies may be even less optimal than previously thought.

Finally, it connects with the literature on infrastructure and development. Research indicates that electrification is important for development (Dinkelman, 2011; Lipscomb, Mobarak and Barnham, 2013; Rud, 2012; Van de Walle et al., 2013), and residential access to modern energy and lighting can improve living standards and productivity (World Bank, 2006). And in many developing countries, lighting is a major component of residential electricity consumption. Yet access to electricity infrastructure does not guarantee reliable service (Klytchnikova and Lokshin, 2009). Electricity outages can impact both households (Chakravorty, Pelli and Marchand, 2013) and firms (Allcott, Collard-Wexler and O’Connell, 2015; Alam, 2013), yet low-quality electricity infrastructure can be persistent (McRae, 2015).

The remainder of the paper is as follows: Section 2 provides information on electricity use in the Kyrgyz Republic and the potential impacts of energy efficiency; Section 3 details the sampling process and randomized design; Section 4 addresses the data collected and results of the randomization and compliance checks; Section 5 estimates impacts of CFLs at the household level, both with and without spillovers; Section 6 estimates the impacts of energy efficient technology at an aggregate level; Section 7 provides additional supporting evidence of a technological externality; and Section 8 concludes.

# **2 Institutional setting and energy efficiency**

## **2.1 Electricity services and infrastructure in the Kyrgyz Republic**

The Kyrgyz Republic provides a suitable context in which to study energy efficiency and electricity reliability in a developing country setting.<sup>16</sup> Due to its history as part of the former Soviet Union, the country is highly electrified, with nearly 100 percent of households covered by formal electricity connections (Gassmann, 2014). Residential electricity demand

> 15 Interventions to understand or increase uptake of energy efficient technologies have focused on energy labeling (Newell and Siikamaki, 2015), social norms (Herberich, List, and Price, 2011), information on energy costs (Allcott and Taubinsky, 2015), and subsidies (Allcott and Sweeney, 2015).

> 16The country ranks 147th out of 187 countries for GDP (PPP) per capita (IMF, 2012).

6
has increased since the country’s independence in 1992. Over the past two decades, the proportion of total electricity consumption comprised by the residential sector steadily increased, with 63% of the country’s current electricity supply consumed by the residential sector (Obozov et al., 2013).

Similar to many other developing countries, the electricity infrastructure is insufficient to meet current and growing electricity demand. In the Kyrgyz Republic, much of the existing electricity infrastructure dates back to the Soviet Union including all 16 of its power plants (Zozulinsky, 2007). Ninety percent of electricity generation within the country is hydroelectric, so supply fluctuates with annual variability in reservoir water levels.<sup>17</sup> Technically, the capacity of both generation and transmission infrastructure could constrain household electricity services and result in unreliable electricity services (frequent electricity outages); however, during the study period, distribution constraints are the primary source of unreliable service.

Due to very low residential electricity prices ($0.02 per kWh throughout this study), many households heat with electricity in winter. In spite of low prices, concerns regarding electricity bills and household energy expenditures are common. Household energy expenditures comprise an estimated 7.1 percent of total household expenditures in the Kyrgyz Republic (Gassmann, 2014), much of which is due to winter heating. Electric heating leads to large seasonal variations in electricity consumption, with average winter consumption approximately three times that of summer. The country’s utilities face growing electricity consumption while constrained by a distribution system designed for substantially lower demand. As a result, the country is plagued by frequent unplanned outages particularly in winter.<sup>18</sup> Transmission and distribution systems are consistently overloaded, acting as a binding constraint and source of unplanned outages. To put this in perspective, most of the transformers have a load factor of 0.9 - 1.2, with 0.7 being the optimal load (Amankulova, 2006).<sup>19</sup> These constraints are a concern for both consumers and the electricity utility. Unplanned outages typically occur when local distribution systems experience an overload. The distribution

> 17In years of low water availability, the country has instituted planned, rolling blackouts during the winter. This did not occur during our study period, according to the electricity utility.

> 18For example, in 2010 the country had 12,578 unplanned power outages (approximately 34 outages per day), which is considered unreliable service by international standards (USAID, 2011).

> 19This is for the 35/220 kV transformers, which is the last step in delivering electricity to homes.

7
network was constructed for peak electricity consumption associated with households owning only a few electricity-using durables. However, peak household electricity demand has increased as households have bought more appliances, which is consistent with predictions made for pro-poor economic growth in developing countries, more broadly (Wolfram, Gertler and Shelef, 2012).

Transformers are a critical part of the electricity distribution infrastructure. A distribution transformer on the electrical grid converts high-voltage electricity to usable, low-voltage electricity for household consumption (Glover, Sarma and Overbye, 2011). There is a maximum electricity load that a transformer can transfer at any given time and exceeding that may lead to unplanned outages. During the study, transformer overloads were the primary source of unplanned outages. Quality of electricity services is correlated within a transformer, as households thus are exposed to the same electricity-related shocks. A transformer-level outage affects all households sharing the transformer.<sup>20</sup>

# **2.2 Energy efficient lighting**

Energy efficient technologies have the potential to decrease individual residential electricity consumption and therefore costs. Using approximately 75% less electricity than incandescent lightbulbs, CFLs are one tool for meeting lighting needs while reducing electricity consumption. If take-up rates are sufficiently high, energy efficient technologies could impact electricity load at an aggregate level. For example, CFLs could decrease aggregate electricity demand within a transformer reducing the probability of a transformer overload. By alleviating the stress on the infrastructure, energy efficiency could reduce unplanned electricity outages and increase service reliability.

Prior to this study, CFLs were available for purchase in large home repair stores and markets located in the capital city, but not in villages. CFLs cost between 100 and 170 Kyrgyz soms, depending on the quality.<sup>21</sup> In contrast, incandescent lightbulbs were available to purchase in both rural and urban markets for approximately 15 to 20 Kyrgyz soms. Even with low electricity prices, the payback period for the CFLs was between 1 and 2 years.<sup>22</sup> This study

> 20In our setting, 54 households on average receive their electricity via a single transformer.

> 21In March 2013, the exchange rate was approximately 1 USD = 46 Kyrgyz soms.

> 22Payback period calculations (not shown) were based on typical lightbulb use in our sample and electricity and CFL prices within the region. We show cost-benefit calculations for the first year post-distribution in Appendix Calculation 4.

8
was implemented in a district adjacent to the capital, Bishkek. Given residents of this district frequently travel to the city, they could have purchased CFLs pre-intervention; however, very few households outside of the capital used CFLs at baseline.

# **2.3 Potential impacts of energy efficiency**

Based on constraints in the electricity distribution system and the potential for energy efficient technologies to decrease demand both at the household and aggregate-level, we might expect the following impacts on electricity consumption following installation of CFLs:

1. _Impacts on household electricity consumption:_ The effect of energy efficient lightbulbs within one’s home depends on the net sum of both engineering and human factors. Based on engineering calculations, the technologically feasible benefit would be a reduction in electricity consumption, and thus in the electricity bill.<sup>23</sup> There are, however, multiple reasons a household’s electricity consumption might not change, might decrease by less than expected, or might even increase following the installation of CFLs. These vary from CFLs not meeting their technological promise (poor quality, failure), human interactions with the technology (incorrect use), human behavioral responses (rebound effect)<sup>24</sup> or some combination of these.<sup>25</sup>

2. _Spillovers:_ Installation of energy efficient technologies could influence others’ adoption decisions via channels such as learning, imitation, etc. These spillovers in take-up could be experienced by households with close linkages to the original adopters, and could be positive, negative, or zero. If present, spillovers in take-up can bias empirical analyses that do not account for them. For instance, when estimating the impacts of CFLs on household electricity consumption, if the comparison group finds alternative means by which to access the technology, they could also experience reductions in electricity

> 23 With some basic information (for example, information on the wattage of both the original bulb and the energy efficient bulb to which one is switching, hours of lightbulb use, etc.), one can perform simple calculations to predict reductions in electricity consumption. We perform such calculations and discuss them in Section 5.

> 24By cutting down the energy requirements and monetary costs of lighting, CFLs may promote additional electricity consumption or generate a “rebound” that offsets the technologically feasible savings. As a result, the kWh consumed by a household may decrease, remain unchanged or even increase. For theoretical discussion of the rebound effect, see Borenstein (2015) and Chan and Gillingham (2015). Although the existing literature appears to agree on the existence of rebound effects, there is much debate over the magnitude of such an effect (Gillingham, Rapson and Wagner, forthcoming).

> 25For example, by being more efficient, CFLs will result in less waste heat lost than incandescent bulbs. This may result in households compensating through other practices, such as winter heating.

9
consumption, leading to underestimation of true impacts. A reduction in electricity consumption may occur, but it may not be detected (or muted) due to contamination of the comparison group.

3. _Aggregate effect:_ Energy efficient technologies could have an effect on aggregate electricity consumption within the local distribution system. The sign and magnitude of this aggregate effect will depend on the proportion of households that use CFLs, as well as on the sign and magnitude of the impacts they individually experience. If a large enough proportion of households within a transformer adopt CFLs and their individual impacts combine into aggregate electricity savings, the probability of an outage occurring due to a transformer overload would be reduced, thus increasing reliability of electricity services.

4. _Technological externality:_ The distribution of CFLs could generate a positive (negative) technological externality if the returns to a household from adopting such technology increase (decrease) in the fraction of the population utilizing the energy efficient technology. In the scenario of a positive technological externality, when consumers install energy efficient devices at a high saturation and aggregate demand decreases such that reliability improves, households are able to consume more hours of electricity services. This effect is experienced by all households within a transformer, regardless of the individual household’s own adoption. The returns from CFL adoption increase because the potential for electricity consumption increases and therefore the potential for energy savings through CFL use does as well.<sup>26</sup> This field experiment employs a randomized saturation design to address these potential impacts of energy efficiency.

# **3 Randomized experiment with energy efficiency**

## **3.1 Sampling process**

In order to identify transformers and households within the transformers, we used data from electricity utility records on over 40,000 residential customers in a district adjacent to

> 26If electricity is limited, the benefits from CFL use are positive but small–zero if there is no electricity. As electricity becomes more available the benefits increase, possibly in a non-linear way, at a decreasing rate and/or discontinuously.

10
Bishkek, the country’s capital.<sup>27</sup> These records contain crucial information for the design of our sampling procedure. The data identify the address of each of the households in the district as well as the transformer through which each of them is served.

Within the district for which we have utility data, seven villages (comprising 248 eligible transformers)<sup>28</sup> were chosen due to their accessibility from Bishkek during the winter months.<sup>29</sup> Given that the area surrounding the capital tends to be better off than those in the country’s other regions, the district in which the study occurs is not representative of the entire country. In order to improve external validity, the study sample was restricted to the 124 eligible transformers with below median monthly household electricity consumption in the year prior to the intervention. Such households and transformers are more representative of the country as a whole. To complete the sampling process, 20 percent of the households from each transformer were randomly selected to participate in the study.<sup>30</sup> Because the number of households per transformer is heterogeneous, the exact number sampled in each transformer also varies.

# **3.2 Experimental saturation design**

The randomized saturation design varies household exposure to the CFL technology to test for evidence of a household energy efficiency effect, spillovers in take-up, and a technological externality due to aggregate impacts. Transformers are first randomized to differing treatment saturations and then households within those transformers are randomized either to receive CFLs or to control status, according to the saturation previously assigned.

Figure 1 depicts how treatment status was randomly assigned in two stages. In the first stage, the 124 sampled eligible transformers were randomized into three groups: control, lower treatment saturation, and higher treatment saturation. Due to funding constraints, households in 14 control transformers were not surveyed. This resulted in only 110 eligible transformers being finally included in the study, with 25, 45, and 40 transformers in control,

> 27The data were provided by the electricity utility through a Data Use Agreement. The utility, however, was not informed as to which transformers or households were being treated through the experiment.

> 28Transformers providing at least 5 households with electricity were eligible. According to the utility, transformers serving fewer households likely also serve industrial consumers.

> 29This was to ensure that survey enumerators could reach households in March, when weather conditions can make transportation challenging.

> 30Due to funding constraints, households in only 25 of the 39 control transformers were surveyed, resulting in households in 110 transformers being surveyed.

11
lower and higher saturation groups respectively. In control transformers, no study households were treated. In lower-saturation transformers, 60% of study households were treated. In higher-saturation transformers, 80% of study households were treated. As mentioned in the sampling discussion above, 20 percent of the households in each transformer were sampled into the study. Thus, the first stage randomization results in approximately 12% to 16% of all households within treated transformers being assigned to treatment.

In the second stage, a total of 1,000 study households within these 110 transformers were randomized into either control or treatment status. This resulted in 457 and 543 households in each group respectively. By design, as depicted in Figure 2, treated households are found only in treated transformers, in the proportions set by each transformer’s treatment intensity. Control households, however, can be in either control or treated transformers. The two stages of randomization also induced spatial heterogeneity in the location of the treated households, leading to variation in the proximity to and number of treated neighbors.

# **3.3 Intervention and household survey**

In the spring of 2013, all study households were visited and invited to participate in a baseline survey regarding electricity use. Their treatment status assignment was not disclosed at the time of the baseline survey. All households were given 150 Kyrgyz soms (approximately 3.26 USD) to compensate them for their time, after finishing the survey.<sup>31</sup> After the baseline survey, baseline interaction with the control households was complete. In contrast, households randomly assigned to the treatment group were offered the opportunity to purchase up to 4 CFLs at a highly subsidized, randomly-drawn price, via a willingness to pay experiment.<sup>32</sup> The set of possible prices was _{_ 0, 5, 10, 15, 20 _}_ . The market price for CFLs was a minimum of 100 KGS, so treated households were paying a maximum of 20% of market price. The market price for incandescent lightbulbs was between 15 and 20 KGS. On average, treated households received 3.2 CFLs from the study intervention. Take-up of CFL treatment was not perfect, as some treated households opted to stop after the survey.

> 31As of 2011, the average monthly nominal employee wage was 9,352 KGS per month (or an estimated 467 KGS per day of work) (National Statistical Committee of the Kyrgyz Republic, 2012).

> 32We provided up to four CFLs as pilot surveys indicated that, on average, households had five to six lightbulbs at baseline. We sought to replace most of their incandescent bulbs with CFLs. The experiment, which utilizes the Becker-de Groot-Marschak methodology to measure demand for CFLs, is explained in Meeks (2016).

12
The rate of non-compliance that resulted was 12%. In those cases, the household received zero CFLs.<sup>33</sup> All study households were visited again in the spring of 2014, one year following the intervention, for a follow-up survey. Of the original 1,000 study households, 101 addresses were identified as having new tenants in the year since the intervention. We interviewed all households currently living at the original addresses, as it was difficult to know exactly when residents moved or if the CFLs had moved with them.<sup>34</sup> After completion of the follow-up survey, survey respondents were again offered 150 KGS to compensate for their time.

# **4 Study sample**

## **4.1 Data**

We use several datasets, including electricity utility data, baseline and follow-up household survey data, and GIS spatial data, which are matched at the household-level. All of these data are then matched to heating degree day data from a nearby weather station. A description is provided in the Appendix.

**Electricity utility data:** We utilize electricity utility records identifying both transformers and households served by them, as well as the monthly household electricity bills. These data start in October 2010 and continue through September 2014. This provides observations 30 months prior to and 18 months following the intervention, held in March 2013. One important feature of this time period is that electricity prices remained constant at 0.02 USD per kWh.<sup>35</sup> The utility, however, was not collecting transformer-specific outage data at the time of the intervention.<sup>36</sup>

**Household survey data:** As indicated in the previous section, we conducted two rounds of household surveys, one immediately prior to the intervention (baseline) and one a year

> 33For intent-to-treat estimates, these households are considered treated. Compliance is discussed below.

> 34Survey enumerators made at least four attempts to survey the household. If enumerators were informed that the previous respondents had moved, then the new residents were surveyed. A total of 835 original respondents were re-interviewed for the follow-up survey.

> 35A tariff reform was introduced in late 2014 (see McRae and Meeks (2016) for details of reform) and therefore we end our analysis in September to avoid conflating the CFL intervention with the tariff change.

> 36Because electricity consumption data are monthly, we cannot use them to calculate timing of outages.

13
following the intervention (follow-up). The baseline survey collected information on appliance ownership and use, lightbulb ownership (type, wattage, etc) and use (room of use, hours used in a typical day, etc.), electricity-related behaviors, and various household demographics. Importantly, we asked households to report the number of days in the past month that they did not have electricity due to outages.<sup>37</sup> Implementation of both survey rounds began in the month of March, specifically so that, when asking about the outages in the month prior, we were capturing winter months.<sup>38</sup> The number of CFLs distributed to each treatment household was tracked. The follow-up survey repeated questions from the baseline and also asked new questions on perception and understanding of CFLs.

**Spatial data:** GPS data were collected on the location of each participating house during the baseline survey. These data points permit various calculations relevant to potential spillovers in adoption, including the distance to nearest treated household and the number of treated households within certain radii. In addition, we use GIS data from OpenStreetMap on locations of all buildings in the study villages. These data permit calculations of the total number of households located within various radii (e.g., 100 meters, 200 meters, and so on) of each participating household.<sup>39</sup>

# **4.2 Household characteristics**

Households in the study sample are small, with an average of just under 4 members. Household monthly income per capita is on average 76 USD per month (2.45 USD per person per day) and household heads are educated, with 84 percent having finished secondary school. Most houses (91 percent) are owner-occupied, the majority of which were constructed during the period of the Soviet Union. They comprise an average of 4.3 rooms, and are typically constructed of brick (54 percent) or a hay/adobe mix (38 percent). Houses are served by formal connections to the electrical grid and metered individually (i.e., houses do not share meters).

Households receive a monthly electricity bill based on the meter readings of their consumption. At baseline, they use an average of 232 kWh per month in the summer (June to

> 37These questions were piloted extensively prior to the baseline survey. We asked about the number of days due to heterogeneity in outage length. The transformer repair required and availability of replacement parts determines the outage length after a transformer overload. According to the utility, transformer outages last between a few hours and a few days.

> 38Winter is when electricity demand is the greatest, stressing the transformers. For this reason, outages are most prevalent in the winter months.

> 39The calculations of the total number of buildings are further described in Appendix Calculation 1.

14
– – September) and 633 kWh per month more than double in the winter (November to February). On average, households have 8 electricity-using durables<sup>40</sup> and many households (39 percent) report heating with electricity.<sup>41</sup> A small proportion have an electric hot water heater (14 percent), and almost none (2 percent) have air conditioners.

Households largely indicate that they both frequently worry about saving electricity (95 percent) and take measures to save electricity (86 percent). More than half the households report knowing about energy efficient lightbulbs (56 percent). However, CFLs use was low. Only a few households had them, and in small numbers (resulting in 0.17 CFLs per household, on average). The majority of households did not know or believe that CFLs consume less electricity (70 percent), did not expect savings in their electricity bill from replace incandescent bulbs with CFLs (72 percent), and did not believe the electricity savings would pay back the upfront costs of the CFLs (69 percent).<sup>42</sup>

# **4.3 Randomization balance**

Baseline characteristics were compared both at the transformer and household levels to understand the outcome of randomization. Comparisons use both household survey data and data on transformer characteristics, provided by the electricity utility.

Transformer-level balance test results are shown in Appendix Table 1. There are no statistically significant differences between the transformers treated with a low saturation and the control transformers, nor are there any statistically significant differences between the high saturation transformers and the control transformers. The high and low saturation transformers do have one significant difference from each other. They differ in the number of households within the transformers. Following Bruhn and McKenzie (2008), we chose not to re-randomize and instead control for this characteristic in related regressions and perform additional robustness checks.

Results from the household-level balance tests are shown in Appendix Table 2. The two

> 40Almost all homes have a television and refrigerator. Approximately three-quarters of households have electric stoves, an iron, and a clothes washing machine.

> 41Households conserve on heating. Most households report heating their houses at least sometimes with coal (80%), and on average they heat 3/4 of all their rooms during winter.

> 42Households were also unaware of differences in quality (88%) and in potential electricity savings (80%) between different types and brands of CFL.

15
household-level treatment groups are statistically identical along most dimensions. Importantly, most households in all groups have their own individual electricity meters. There are, however, two slight differences. Control households are slightly more likely than treatment households to have a household head that has completed secondary school education, and to be in homes that are single family buildings (in comparison to the multi-unit apartment buildings). This difference is about what could be expected by chance. If anything, these would downward bias our results.

A graph of pre-intervention electricity consumption over time, in Appendix Figure 1, highlights important heterogeneities in electricity consumption over a year and shows the same seasonal variation for both treatment and control households. The large spike in winter electricity consumption is due in large part to electric heating and is the reason that electricity outages are most frequent in that season. Longer hours of lighting due to shorter days play a lesser role. The winter peak is somewhat greater among the treated households than the control in both pre-intervention winters (winter 2011 and winter 2012). To account for these patterns, our analysis will include household fixed effects and month-by-year fixed effects.

# **4.4 Compliance and attrition**

There are two aspects of the intervention take-up and survey data collection, described in Section 3.3, with which we might be concerned: (i) whether or not treated households took the CFLs that we distributed as the treatment, and (ii) whether study households were still inhabiting the home at time of follow-up survey. We address these issues in the following ways. First, in our intent-to-treat estimates, treated households that did not comply with the treatment are considered treated. Second, we obtain estimates including all households (movers and non-movers) and excluding those houses with new tenants (just non-movers). Results are consistent across these analyses.

16
# **5 The impacts of energy efficient lighting on household electricity consumption**

## **5.1 Bench-marking and previewing impacts**

Prior to estimating the intervention’s impacts, we first calculate the technologically feasible expected electricity reductions for the winter, spring/fall, and summer seasons.<sup>43</sup> Back-ofthe-envelope calculations for a typical household in our sample, in Appendix Calculation 2 and Appendix Table 3, suggest that replacing 3.2 incandescent lightbulbs of 100 watts each by CFLs of 21 watts each could potentially decrease electricity consumption by between 26 and 42 kWh per month.<sup>44</sup> These technologically feasible savings do not account for human interaction or behavioral responses to the technology. Estimates of the effect of CFLs in actual use conditions could be smaller or larger. They can differ from the engineering calculations for reasons such as a rebound in electricity consumption, a reduction in electricity outages permitting more electricity services, or spillovers in technology take-up that bias the results. The randomized saturation design helps us understand the impacts of energy efficient lighting and untangle the channels through which they operate.

To motivate the regression analysis, we begin with an event study-style graph of electricity consumption month-by-month (Figure 3). We plot our calculations of the technologically feasible electricity reductions alongside the estimated impacts of CFL treatment (the result of comparing treatment relative to control households in control transformers) monthby-month.<sup>45</sup> A few patterns are noticeable. First, the figure shows a downwards shift in electricity consumption for treatment households post-intervention (with CFL distribution starting in late March 2013). Second, the impacts are quite noisy, particularly in the winter months.<sup>46</sup> Although they indeed have reductions in electricity consumption, the noisiness

> 43Through discussions with the electricity utility in fall 2012, we learned that winter (also known as the “heating season”) is the time of highest electricity demand and greatest strain on transformers.

> 44During early survey piloting, we learned that households typically used 100 watt bulbs in their homes. We provided 21 watt CFLs, because they were advertised to be equivalent to 100 watt incandescent bulbs. These and other underlying assumptions are further described in the Appendix Calculation 2.

> 45Because in this figure we calculate the treatment impact separately for each month, we cannot include household

> 46This noisiness is likely due to some households heating with electricity in the winter (which would cause a large spike in their electricity consumption), while others heat with coal.

17
might make them less salient for treated households. Third, there is much heterogeneity in the impacts across seasons. Electricity savings start in the months following the distribution of CFLs in the spring of 2013, and continue over the summer and fall. In the winter, impacts diverge greatly from the engineering predictions, but in the spring of 2014 the electricity savings return and are similar to those predicted. The regression analysis that follows looks more closely into these impacts of the CFL treatment.

# **5.2 Basic estimates**

To assess the impacts on household electricity consumption, we first estimate a simple difference-in-differences model using the electricity utility records: where _qit_ is household _i_ ’s electricity consumption (kWh) in month _t_ , _Treati_ is an indicator of treatment status, and _Xit_ is a vector of household-level variables.<sup>47</sup> Month-by-year fixed effects, _γt_ , and household fixed effects, _λi_ , address the large seasonal variations in consumption and potential heterogeneities across households in baseline electricity consumption, as identified in the event-study graph.

The _Treati ∗ Postt_ interaction denotes the household’s assignment to treatment and if that treatment occurred in that month or prior months. The coefficient on this interaction term, _τ_ , is therefore an estimate of the average change in household monthly electricity consumption (in kWh) that resulted from random assignment to treatment.

Table 1 reports intent-to-treat estimates of _τ_ . In Table 1, Column 1, the estimated impact is identified according to the above basic regression, from variation within households over time, controlling for month-by-year shocks. This basic estimate indicates that the CFL treatment reduces household electricity consumption by 16 kWh per month. The magnitude of this impact is only half the expected size based on engineering calculations. However, in Column 1, the omitted group is comprised of all control households, regardless of whether 47These include controls for number of days in monthly billing period and whether the household uses electricity for heating. We also control for heating degree days; however, we only have variation in temperature over time. The 7 villages in the study sample are all covered by one weather station and data are reported at that level. Nevertheless, we do not expect for there to be much spatial variation in temperatures across villages included in the study given their size and proximity.

18
they are in a treated or control transformer. If there are any spillovers in adoption among control households close to treatment households, or any technological externality affecting all households within a treated transformer, the omitted group will be contaminated and the basic impact may be mis-estimated.

# **5.3 Accounting for contamination**

We re-estimate the impacts of CFL treatment on household electricity consumption, accounting for potential contamination within the transformer. We employ the following differencein-differences specification:<sup>48</sup> _qigt_ = _τTreatig ∗ Postt_ + _βPostt_ + _δTreatig_ + _θCig ∗ Postt_ + _φCig_ + _αXig_ + _γt_ + _λig_ + _εigt_ (2) where _qigt_ is our outcome of interest for household _i_ located in transformer group _g_ ; _Treatig_ indicates the treatment status of household _i_ in transformer group _g_ ; _Cig_ indicates if the household _i_ is a control household in a treated transformer group; and _Xig_ is a vector of variables, additionally including the number of houses served by a particular transformer.

As before, _τ_ is an estimate of the average impact of the CFL treatment on household monthly electricity consumption; and the new coefficient _θ_ estimates the average impact on control households in treated transformers. Importantly, the omitted group in these regressions is now a “pure control”, in that it only includes control households that are located in control transformers.

The results presented in Table 1, Columns 2 and 3, indicate that the CFL treatment led to a reduction in household electricity consumption of 30 kWh per month. The coefficients are significant regardless of the level at which standard errors are clustered. Accounting for potential contamination within transformer, the estimated impacts are statistically and substantially larger than basic estimates in Column 1, and comparable to the expected engineering reductions.

We replicate the previous analysis, but this time separately for each season.<sup>49</sup> Appendix

> 48We account for the potential within-transformer contamination in a fashion similar to Gine and Mansuri (2011) and Banerjee et al. (2014).

> 49We divide the months according to the electricity utility’s seasonal definitions, with the winter heating months including November, December, January, and February.

19
Table 4 shows that the only season in which the impacts of treatment are of the magnitude expected and statistically significant is in the spring/fall. Impacts in the winter and summer seasons are far smaller than expected and insignificant.<sup>50</sup> These findings suggest a technological externality, by which households in higher intensity treatment transformers consume more electricity due to fewer electricity outages in the winter. Winter is when unplanned outages most frequently occur, as peak electricity demand is quite high. CFLs have the potential to reduce electricity consumption, but they also have the potential to reduce outages in winter (more than any other season). If households have more reliable electricity service (fewer outages) in the winter, then they will be able to consume more electricity services and we would not see the expected reduction in electricity consumption. Potential technological externalities could indeed account for heterogeneity in impacts across seasons.

In summary, contamination of control households may be biasing downwards the basic impact estimates. That we see a significant reduction in electricity consumption among control households in treated transformers suggests that spillovers in CFL take-up may be biasing downwards the basic impact estimates. Potential technological externalities due to an aggregate effect on electricity reliability at the transformer level may also contribute to the miss-estimation of impacts. We look more into these and other possible mechanisms in Section 7. Next, we analyze the impacts of CFL treatment at the transformer level.

# **6 The impacts of energy efficient lighting on aggregate electricity loads**

## **6.1 Bench-marking and previewing aggregate impacts**

As in many developing countries, in the Kyrgyz Republic outages are most frequently the result of overloads within the distribution system. Overloads typically occur during peak times and are more common in the winter, particularly in the winter evenings, when household energy demand is greatest.

We perform a back-of-the-envelope calculation of the peak load reduction induced by the CFL treatment. Switching 4 incandescent lightbulbs to CFLs could save 60 kWh per month,

> 50Based on our ex-ante calculations of the expected electricity savings, we believed that electricity consumption reductions would be greatest in the winter (followed by fall/spring and then summer).

20
which is 10% of a 600 kWh household monthly bill. However, lighting is disproportionately used “on peak”. Therefore, the resulting reduction is actually 21% of the household peak load demand.<sup>51</sup> For a transformer with about 20% of all households treated, this would equal approximately a 4% reduction in transformer peak load.<sup>52</sup> Using household survey data on the number of days without electricity (due to outage) during the month prior, we provide suggestive evidence of an aggregate impact of transformer-level treatment on unplanned outages.<sup>53</sup> In Figure 4, we plot the reported outages at follow-up by treatment group. As shown, the distribution of reported outages among households in treated transformers is shifted leftward (towards zero outages) in comparison to the graphed responses of households in the control transformers. This figure motivates the regression analysis that follows.

# **6.2 Estimating aggregate impacts**

We use the transformer-level randomization to estimate the impact on reported unplanned electricity outages using the following equation: where _Oig_ is the number of days without electricity due to outages in the month prior to the follow-up survey as reported by household _i_ in transformer _g_ ; _Hig_ indicates if household _i_ is in a higher-saturation transformer, with between 15 to 18 percent of all households assigned to treatment; _Lig_ indicates whether household _i_ is in a lower-saturation transformer, with between 10 to 14 percent of households assigned to treatment;<sup>54</sup> _Treatig_ remains the household’s own treatment status; and _Xg_ is a vector of transformer-level controls. Standard

> 51According to the utility, times of peak demand are between 6 to 9 am and 6 to 10 pm. The calculation assumes a reduction of peak demand of 1.5 kW peak, and switching from 100 W incandescent bulbs to 21 W CFLs (Appendix Calculation 3).

> 52We “ground-truth” these results by discussing them with our utility collaborators. Engineers indicated that this would be a substantial reduction in peak demand and would very likely reduce transformer outages.

> 53The follow-up survey occurred in March and April 2014, so the months in which we are measuring days without electricity due to outage include February and March.

> 54In higher (lower) saturation transformers, 80% (60%) of study households were assigned to treatment. There is heterogeneity in the number of households across transformers, and we sampled 20% of all households within each transformer to include them in the study. Therefore, the randomized study saturation proportions result in between 15 to 18 (10 to 14) percent of all households in high (low) saturation transformers actually receiving the CFL treatment

21
errors are clustered at the transformer level.<sup>55</sup> We also perform the standard error correction for multiple hypothesis testing developed by List, Shaikh and Xu (2016).

Results in Table 2 indicate that the energy efficient technology led to an aggregate impact in the form of improved reliability. Table 2, Column 1, shows the basic results at the transformer level, given by _π_ and _ρ_ in the equation above. We see between one and two fewer days without electricity in both the low and high saturation transformers, respectively. However, the reduction in days without electricity due to outages is only statistically significant for households in the high saturation treatment transformers. They report approximately half as many days of outages (two in comparison to four) as households in control transformers. The estimates are robust to including a suite of different transformer-level controls, most important of which are the number of households within the transformer and the baseline number of outages reported. The difference in outage-days reductions between households in high and low saturation transformers (two versus one) is also statistically significant.

Table 2, Column 2, controls for the reporting household’s own treatment status. High saturation transformers have more treatment households (by definition), so we may worry that those households have an incentive to report fewer outages. Accounting for individual household treatment status shows that such possibility is not a concern. The estimates remain the same as the basic impacts, suggesting that differences in reporting between treated and control households are not driving results.

# **6.3 Evidence of technological externality**

We re-estimate the impacts of CFL treatment on household electricity consumption, this time breaking up the analysis by whether the household is in a higher or lower treatment saturation transformer. Importantly, these estimations still use variation induced by the randomized saturation process. Results shown in Table 3 indicate a difference between the transformers with lower and higher intensities of treatment in the extent to which a technological externality is created. These results are consistent with our working definition of a technological externality, in that the more individuals that take up the technology, the

> 55 We can and have also run these regressions collapsing to the transformer level and using the average reported outages per transformer. In doing so, we get similar results; however, in using the transformer level average, we lose our ability to control for the respondent household’s own treatment status.

22
greater are the returns to the technology.<sup>56</sup> The intuition is simple. In higher saturation transformers, the households that take up CFLs generate a reduction in aggregate demand load substantial enough to significantly decrease the number of days without electricity due to outages. Therefore, treated households attain smaller reductions in electricity consumption. Although they use fewer Kw per hour of electricity consumed (due to the greater energy efficiency of CFL technologies), they can consume more hours of electricity (due to the greater electricity reliability induced by the take-up of CFLs). In contrast, in the lower saturation transformers, treated households attain much larger overall electricity savings. The households’ take-up of the CFLs reduces to some extent the aggregate load, but its impact in the number of outage-days is insignificant.

In summary, when a large enough proportion of households take up the CFLs, an aggregate effect is possible and, if substantial enough, it may induce a technological externality in the form of a reduction in electricity outages. This increased electricity reliability is experienced by all households served by the impacted transformer, regardless of whether they themselves are treated or control households. When controlling for this potential externality, the CFLs exert a significant reduction in household electricity consumption that is of a similar magnitude to that which is technologically feasible. This is the case of the coefficient on the treated households in low-saturation transformers, which do not experience changes in outage-days.

# **7 Additional checks and supporting evidence**

## **7.1 Evidence of spillovers in CFL take-up**

As discussed in Section 5.3, control households located close to a treated household were likely to experience spillovers, and this contamination may have biased the measurement of the CFL treatment impacts on electricity consumption.<sup>57</sup> To test whether there are indeed spillovers in CFL take-up following the intervention, we estimate the impact of our intervention on

> 56We think of this as an analogy to children within a school being vaccinated against a disease, such as measles. Vaccinated children provide a positive externality to non-vaccinated children. With each additional child vaccinated, the probability of a measles outbreak within the school is reduced; however, the proportion of children within the school that are vaccinated must surpass some threshold in order for the school community to achieve herd immunity.

> 57Our definition of closeness between households is a distance measure of geographical proximity. As discussed in Breza (2015), a number of studies have used geographical proximity to measure spillovers, including Dupas (2014), Godlonton and Thornton (2012), and Cohen, Dupas and Schaner (2015).

23
the number of CFLs in a house at the follow-up survey, controlling for the number of CFLs distributed to the households at baseline. We utilize the spatial variation in exposure of control households to close treated neighbors induced by the two-stage randomization to account for potential peer effects in technology adoption as follows: where the new term _C_ is an indicator for whether the household _i_ is a control household _ig_ with at least one treated household within 100 meters from the house; _Nd_ is the total number of households within 100 meters from the house;<sup>58</sup> and _Xig_ is a vector of controls, including the number of CFLs the household received from our project at baseline. Standard errors are clustered at the transformer level.

Table 4 reports the estimated impacts on CFL adoption at the follow-up. Column 1 uses the basic specification from Eq. 1. As before, these results are expected to be biased because the omitted group comprises all control households and some of them are potentially contaminated by spillovers. Column 2 shows results from Eq. 4, which accounts for spillovers by controlling for whether a control household is within 100 meters from any treated household; and Column 3 mirrors the alternative specification given by Eq. 2, which corrects for contamination by including a dummy variable for control households in treated transformers. Their corresponding omitted groups are detailed at the bottom of each column.

Results from Columns 2 and 3 are similar, as they should be, given that households within the same transformer are in close proximity spatially.<sup>59</sup> Both treated and control households have an average of half an additional CFL at follow-up, controlling for the CFLs distributed by our intervention. This result contrasts with the basic specification’s much smaller and insignificant estimated impact, making clear that spillovers also bias the basic estimates of the effect on later take-up. Interestingly, the correct estimated impacts suggest that treatment households likely purchased CFLs themselves between baseline and follow-up, in addition to those they had received from our project. The correct estimates also indicate that, indeed, there is a positive and statistically significant relationship between being close

> 58We consider a control household to be “close” if the entrance is within 100 meters of a treated household. We choose 100 meters as the cutoff, as some prior analyses of peer effects on a number of outcomes variables had suggested that this distance was an important cutoff, past which the effect dissipates.

> 59This reassures that the impacts on household electricity consumption reported in Table 1, Columns 2 and 3, are well estimated.

24
to a treated household and the number of CFLs that household has at follow-up.

# **7.2 Checks of aggregate reliability effects**

To ensure that in Section 6.2 we are indeed measuring a local aggregate reliability effect, we perform two checks.

First, we calculate the intra-cluster correlation in the number of reported outages at followup. We have argued that intense saturation of treatment households within a transformer is causing a reduction in outages at the transformer-level. Then, household reported outages should be correlated with other household responses within the transformer. Our calculation indicates a intra-class correlation of 0.56, which means that responses within transformers are indeed highly correlated.<sup>60</sup> Second, we consider the estimated reduction in outages in conjunction with the difference between the expected and the estimated reductions in household electricity consumption to confirm that our series of results tell a consistent story. We make this comparison for the winter months, as that is the season for which we collected outage data.<sup>61</sup> In Section 5, we expected a reduction in households’ winter electricity consumption of approximately 42 kWh per month (Appendix Table 3). However, what we actually estimated was a winter reduction in electricity consumption of approximately 26 kWh per month (Appendix Table 4). This is a difference of 16 kWh per month, which is equivalent to approximately 14.4 hours of additional electricity consumption. In Section 6, we found the aggregate impacts of these electricity savings to be two fewer days without electricity per month due to unplanned outages. In conjunction with the previous result, this would mean two outages of approximately seven hours in duration each. This is very reasonable and assures us of the consistency across results.<sup>62</sup>

> 60Responses won’t be perfectly correlated with one another, as households in a transformer were surveyed on different dates and therefore the reference point of “past month” differs across households within a transformer.

> 61Winter is when outages are most prevalent and therefore when we would expect to see an aggregate reliability impact, if one occurs.

> 62Anecdotal evidence from both consumers and the utility indicate that outages can last between a few hours to a few days, depending on the repair required following the transformer overload.

25
# **7.3 Evidence on alternative mechanisms**

Although we cannot perform a direct test to rule out the possibility of a rebound, a number of analyses are inconsistent with a direct or indirect rebound in electricity consumption. For instance, in Figure 3 we provided an event-study graph of the impacts on electricity consumption by season. We found that following a winter spike, savings in electricity consumption returned to the expected amount in the spring.<sup>63</sup> If there were a rebound, a persistent change in behavior would be expected, and a reversal in the spring would have been unlikely. Furthermore, using our detailed baseline and follow-up survey data, we can test for impacts of treatment on appliance ownership and average monthly and daily use (results not shown).<sup>64</sup> We found no effects of treatment on lightbulb use (which would be the measure of a direct rebound) and only one significant effect of treatment on use of any household appliances (which would be a measure of an indirect rebound). In fact, treatment households were significantly less likely to report using electric heaters at follow-up. If anything, this finding is evidence counter to a rebound

# **7.4 External validity**

We implemented a randomized saturation design to study the impact of energy efficient lightbulbs on household electricity consumption and local electricity reliability in a lowermiddle-income developing country. A policy-relevant question that guides our thinking about external validity is to what extent our results are location-specific or technology-specific.

A feature of this study’s setting, the Kyrgyz Republic, is that essentially all households are electrified, formally connected to the network and metered individually. We believe that research in this context does not harm the external validity of our study, rather it just helps with the internal validity. It has the additional benefit of ensuring that we are not just studying the responses of the richest households. The Kyrgyz Republic also has an extensive electricity generation and distribution infrastructure. However, similar to other developing countries, the Kyrgyz Republic has a problem of insufficient infrastructure capacity relative

> 63This is also supported by the regression results by season in Appendix Table 4.

> 64In both survey rounds we ask a number questions regarding the household use of appliances, including (1) “On average, how many days per month do you use [the appliance]?” (2) “On average, how many hours per day does your HH use [the appliance]?” Important to note, these specific survey questions are asking about average use in the a month and in a day. In contrast, our survey question on outages asks about “the past month.” For this reason, that we find no impacts on the use of lightbulbs is not inconsistent with our aggregate reliability results.

26
to demand. Constraints in the distribution system, more frequently than other factors limiting generation capacity, are a source of unplanned outages in developing countries. Thus, electricity reliability is largely driven by congestion within the electricity distribution network. Aggregate electricity demand relative to infrastructure capacity, not infrastructure alone, determines the probability of system overload, which results in unplanned outages. As such, we believe this work is relevant to many developing countries in which demand for electricity is either currently pushing the capacity of the current infrastructure or is rapidly increasing such that we expect the constraint to be binding in the near future.

Our study setting is also unique in that it allows us to generate agregate reliability effects and observe technological externalities with relatively low treatment saturation. However, the point we are making is not that at some particular saturation of energy efficiency, technological externalities can be induced. Rather, we illustrate that these externalities are possible to generate and that they are crucial in understanding the welfare implications of such energy efficiency programs. Although similarly low saturation levels in other settings may not induce such effects, their existence should not be ruled out. Instead, we should ask how high the saturation ought to be to generate such aggregate effects and externalities. The answer will depend on many factors, including the expected technologically feasible impact of the technology distributed, the type of aggregate impact a program should induce, the capacity of the electricity infrastructure, the number of consumers the infrastructure serves, etc. Given a variety of factors, a program might need a higher or lower saturation to achieve agregate effects and technological externalities. They may not be limited to a reduction in outages, but may take other forms such as reductions in prices due to lower peak loads.

Although deployment of CFL technologies in higher income countries may not have similar impacts, we believe our findings are not just specific to energy efficient lighting or developing countries. In developing countries, lighting comprises a substantial portion of electricity consumption for many households. For this reason, an energy efficient lighting intervention can reduce a household’s electricity bill by a substantial proportion. In wealthier settings, such as the United States and other developed countries, where households own a greater number of electricity-using durables, introducing energy efficient lighting would likely result in a smaller reduction in the electricity bill proportional to the total. In such settings, a program wishing to induce an aggregate impact may need to focus on a technology that accounts for a larger proportion of the electricity bill.

27
# **8 Conclusions**

Through an experiment with a randomized saturation design, we provide several substantial contributions to the literatures on energy efficiency, electrification, and electricity reliability. We show that energy efficient lightbulbs can indeed lead to significant reductions in electricity consumption. We find that controlling for spillovers is critical, suggesting that some estimates of energy efficiency that do not account for such spillovers may be downward biased.

In addition, we show that the energy efficient technology, when taken up at a high enough saturation levels, can have a local aggregate reliability effect, in the form of fewer days without electricity due to outages at the transformer level. By improving electricity service reliability, the energy efficient technology becomes more valuable; households can use the CFL for more hours when the electricity service is more reliable (i.e. providing electricity for more hours per month at a lower cost than traditional lightbulbs). This is a classic example of a technological externality, through which the returns to a particular technology are increasing with the number of other adopters. The results in both Table 1 and those in Appendix Table 4 highlight ways in which estimates of the effects of energy efficiency could be biased, if we do not account for spillovers in take-up, potential externalities, and heterogeneity in impacts across seasons.

Our study also highlights that an increase in consumption following the introduction of CFLs is a welfare improvement, not a sign of ineffective technology. Other technologies inducing positive externalities may create incentives for households to free-ride on the adoption by others. We show that in this case, in which the aggregate effect increases the returns to the technology, the externality may ameliorate (or even offset) the incentive to free-ride. Thinking about this interaction between technological externalities and incentivizing adoption is important for both policy design and the development of new technologies.

As a result of this analysis, we can perform several variations of cost-benefit analyses both with and without accounting for the aggregate benefits of improved electricity service reliability. These calculations, shown in Appendix Calculation 4, demonstrate that accounting for externalities in the welfare calculations is crucial. Benefit calculations that include both reductions in electricity consumption and increased electricity services are more than double the estimated benefits from electricity savings alone in the first year post-adoption (approximately $15 in estimated benefits instead of $7). The benefits in year 1 of the program are 28
substantially larger than the upfront cost of purchasing and distributing the CFLs (approximately $9 per household). These simple calculations provide a lower bound estimate of benefits from such energy efficiency distribution programs, given they do not account for other benefits, such as pollution reductions. An energy efficiency distribution program, such as ours, looks much more favorable after making this correction.

Finally, the paper provides a novel application of a randomized saturation design at a policyrelevant and technologically meaningful scale. In doing so, we demonstrate the usefulness of such a design to inform our understanding of aggregate impacts and technological externalities from various interventions. This methodology can be applied to study other topics for which decomposing private returns and technological externalities is important in measuring the impacts of technology adoption and choosing between various policy options or program subsidy levels.

29