metadata: authors: 0: author: Cruces,Guillermo A.
1: author: Tortarolo,Dario
2: author: Vazquez Bare,Gonzalo
count: World
docna: 0: docna: Design of Partial Population Experiments
            with an Application to Spillovers in Tax Compliance
docty: Policy Research Working Paper
abstracts: cdata!: This paper develops a framework to
            analyze partial population experiments, a generalization of
            the cluster experimental design where clusters are assigned
            to different treatment intensities. The framework allows for
            heterogeneity in cluster sizes and outcome distributions.
            The paper studies the large-sample behavior of OLS
            estimators and cluster-robust variance estimators and shows
            that (i) ignoring cluster heterogeneity may result in
            severely underpowered experiments and (ii) the
            cluster-robust variance estimator may be upward-biased when
            clusters are heterogeneous. The paper derives formulas for
            power, minimum detectable effects, and optimal cluster
            assignment probabilities. All the results apply to cluster
            experiments, a particular case of the framework. The paper
            sets up a potential outcomes framework to interpret the OLS
            estimands as causal effects. It implements the methods in a
            large-scale experiment to estimate the direct and spillover
            effects of a communication campaign on property tax
            compliance. The analysis reveals an increase in tax
            compliance among individuals directly targeted with the
            mailing, as well as compliance spillovers on untreated
            individuals in clusters with a high proportion of treated taxpayers.
display_title: Design of Partial Population
            Experiments with an Application to Spillovers in Tax Compliance
listing_relative_url: /research/2025/02/34453743/design-partial-population-experiments-application-spillovers-tax-compliance
topicv3: Taxation m1327615 1121
publishtoextweb_dt: 2025-02-06T00:00:00Z
dois: 10.1596/1813-9450-11059
origu: Off of Sr VP Dev Econ/Chief Econ (DECVP)
no_of_pages: 81
author_keys: 0: author_key: 647404
1: author_key: 1547216
2: author_key: 1163700
model_extractions: Policy Research Working Paper 11059 Design of Partial Population Experiments with an Application to Spillovers in Tax Compliance _Guillermo Cruces Dario Tortarolo Gonzalo Vazquez-Bare_ Development Economics Development Research Group February 2025 A verified reproducibility package for this paper is available at http://reproducibility.worldbank.org, click **here** for direct access.
Policy Research Working Paper 11059

# **Abstract**

This paper develops a framework to analyze partial population experiments, a generalization of the cluster experimental design where clusters are assigned to different treatment intensities. The framework allows for heterogeneity in cluster sizes and outcome distributions. The paper studies the large-sample behavior of OLS estimators and cluster-robust variance estimators and shows that (i) ignoring cluster heterogeneity may result in severely underpowered experiments and (ii) the cluster-robust variance estimator may be upward-biased when clusters are heterogeneous. The paper derives formulas for power, minimum detectable effects, and optimal cluster assignment probabilities. All the results apply to cluster experiments, a particular case of the framework. The paper sets up a potential outcomes framework to interpret the OLS estimands as causal effects. It implements the methods in a large-scale experiment to estimate the direct and spillover effects of a communication campaign on property tax compliance. The analysis reveals an increase in tax compliance among individuals directly targeted with the mailing, as well as compliance spillovers on untreated individuals in clusters with a high proportion of treated taxpayers.

This paper is a product of the Development Research Group, Development Economic. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at dtortarolo@worldbank.org. A verified reproducibility package for this paper is available at http:// reproducibility.worldbank.org, click **here** for direct access.

<!-- Start of picture text --> TRANSPARENT<br>ANALYSIS<br>G<br>R E<br>N<br>S<br>I P<br>Y E<br>K A<br>C A<br>R P<br>LI R<br>O E<br>O C<br>W R<br>H<br>S<br>P<br><!-- End of picture text --> _The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# **Design of Partial Population Experiments**

**with an Application to Spillovers in Tax Compliance**<sup>*</sup>

Guillermo Cruces, _U. of Nottingham & CONICET-CEDLAS-UNLP_ Dario Tortarolo, _World Bank DECRG_ Gonzalo Vazquez-Bare, _UC Santa Barbara_ JEL CODES: C01, C93, H71 , H26 , H21 , O23.

KEYWORDS: partial population experiments, spillovers, randomized controlled trials, cluster experiments, two-stage designs, property tax, tax compliance.

> *We thank Yuehao Bai, Youssef Benzarti, Augustin Bergeron, Javier Birchenall, Matias Cattaneo, Max Farrell, Kelsey Jack, Heather Royer, Doug Steigerwald and Alisa Tazhitdinova for valuable discussions and suggestions, and seminar participants at the 2021 National Tax Association conference, IFS, CEDLAS-UNLP, and the 2022 Advances with Field Experiments conference. We thank Julian Amendolaggine and Juan Luis Schiavoni for their invaluable support throughout the project. We thank Bruno Cr ́epon and Roland Rathelot for their help in obtaining their data. The views expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Corresponding author: Dario Tortarolo, E-mail: dtortarolo@worldbank.org. This project was reviewed and approved in advance by the Institutional Review Board at the University of Nottingham. The design for this experiment was preregistered in the AEA RCT Registry (RCT ID: **AEARCTR-0006569** ). All remaining errors are our own.
# **1 Introduction**

Randomized controlled trials (RCTs) are extensively used in economics. A large fraction of these experiments are based on the assumption that the treatment assignment of one unit or subject does not influence the outcomes of others. The assumption of no interference, however, may be violated in many settings. In such cases, identifying and measuring spillovers between units is crucial for understanding the nature and magnitude of interactions between subjects, as well as for accurately assessing the direct impact of the treatment.

While the early experimental literature considered the impact on untreated units in an ex-post manner (e.g. Miguel and Kremer, 2004), field experiments incorporating spillover effects into their design have gained traction in applied research. In settings where units are grouped into independent clusters, such as schools, villages, or firms, a common design is the _partial population design_ . Partial population designs are a generalization of the clustered design wherein clusters assigned to different treatment intensities or _saturations_ are compared to pure control clusters with no treated units (Moffit, 2001; Duflo and Saez, 2003; Hudgens and Halloran, 2008; Hirano and Hahn, 2010; Baird et al., 2018). The variation in treatment intensity allows researchers to disentangle the direct and indirect effects of a treatment. In this paper, we provide a framework to analyze this type of experiment when clusters are heterogeneous.

We consider two dimensions of cluster heterogeneity that have important practical implications: heterogeneity in cluster sizes and heterogeneity in outcome distributions across clusters ( _distributional heterogeneity_ )<sup>1</sup> . When analyzing an experiment with heterogeneous clusters, correctly accounting for this heterogeneity is crucial for several reasons. On the one hand, variance formulas have to be adjusted accordingly, and failing to do so may result in severely underpowered experiments. On the other hand, cluster heterogeneity can affect the accuracy of the large sample normal approximation, and inference based on this approximation can be misleading when clusters are very heterogeneous (Carter, Schnepel and Steigerwald, 2017; Djogbenou, MacKinnon and Ørregaard Nielsen, 2019; Hansen and Lee, 2019; Sasaki and Wang, 2022; Chiang, Sasaki and Wang, 2023).

With these challenges in mind, our paper provides five contributions. First, in Theorem 1, we derive an asymptotic distributional approximation for OLS regression estimators in a setting with between-cluster heterogeneity. We consider a double-array asymptotic setting where cluster sizes are allowed, but not required, to grow with the sample size. We provide conditions under which OLS estimators are consistent for cluster-size-weighted averages of within-cluster differences in means, and are asymptotically normal. We also show that, in the presence of distributional heterogeneity, the usual cluster-robust variance estimator is generally upward-biased, and hence inference based on this estimator is conservative (Proposition 1). While similar results have been obtained in design-based settings with non-random potential outcomes (see e.g. Hudgens and Halloran, 2008; Basse and Feller, 2018; Abadie et al., 2022; Jiang, Imai and Malani,

> 1We note that our framework allows for general forms of between-cluster heterogeneity, but assumes that outcomes are identically distributed within each cluster. The generalization of our results to the case where outcome distributions are heterogeneous within a cluster is left for future research.

2
2023), to our knowledge we are the first to show this result in a superpopulation setting under distributional heterogeneity.

Our second contribution is to derive explicit, closed-form formulas to conduct power and minimum detectable effect (MDE) calculations under the two aforementioned sources of cluster heterogeneity. We then consider an intermediate setting where clusters differ in size but not in their outcome distributions, which simplifies power and minimum detectable effects calculations and can be applied more easily when baseline outcome data is not available. We show how our formulas generalize those available in the existing methodological literature on experimental design (Duflo, Glennerster and Kremer, 2007; Hirano and Hahn, 2010; Baird et al., 2018) by allowing for multiple treatment intensities, cluster heterogeneity, heteroskedasticity and general forms of intracluster correlation in outcomes and treatments.

Our third contribution is to derive optimal assignment probabilities determining the proportion of clusters to be assigned to each treatment saturation (Theorem 2). We provide a tractable, closed-form solution to the optimal choice problem of minimizing a weighted average of estimators’ variances. We also discuss how alternative optimality criteria may be used in combination with our variance formulas using numerical methods.

Our fourth contribution is to set up a potential outcomes framework with within-cluster spillovers, heterogeneous treatment effects, and heterogeneous clusters. We use this framework to provide sufficient conditions for OLS estimands to recover causal direct and spillover effects.

Fifth, based on our framework, we designed and conducted a large-scale field experiment to estimate direct and spillover effects of a randomized communication campaign on property tax compliance in Argentina. Our experiment sent personalized letters to randomly selected dwellings with reminders about taxes due, information about the status of the account, due dates, past due debt, and payment methods. While there is ample evidence on the effect of tax reminders on compliance and collection (Antinyan and Asatryan, 2024), our goal was to find evidence on relatively elusive spillover effects from information campaigns on tax collection. We designed the experiment based on our methodological results to capture spillover effects of our mailings on neighbors who live in the same street blocks of treated individuals but who did not receive a letter. Our results reveal higher payment rates for treated individuals, but also for their untreated neighbors in the same street block, compared to accounts in pure control blocks where no one received the letter. Spillover effects are lower in magnitude but still substantial and precisely estimated in high-saturation street blocks, especially when accounting for expected (pre-registered) heterogeneity in past compliance: payment rates of untreated accounts in high saturation blocks with above median past compliance increased by 2.6 percentage points, compared to direct effects of about 5.1 percentage points.

**Comparison with current literature.** Our paper contributes to a growing literature on experimental design (Duflo, Glennerster and Kremer, 2007; Bruhn and McKenzie, 2009; Bugni, Canay and Shaikh, 2018, 2019; Bai, 2022) and in particular to the literature on design and analysis of experiments under

3
spillovers or interference (Hirano and Hahn, 2010; Athey, Eckles and Imbens, 2018; Baird et al., 2018; Basse, Feller and Toulis, 2019; Jiang, Imai and Malani, 2023; Puelz et al., 2022; Viviano, 2024; Leung, 2022; Liu, 2023). More specifically, our results generalize those of Hirano and Hahn (2010), Hudgens and Halloran (2008) and Baird et al. (2018) by allowing for cluster heterogeneity, heteroskedasticity, general treatment assignment mechanisms and within-group correlation structures and alternative criteria for optimal treatment assignment.

In related work, Athey, Eckles and Imbens (2018), Basse, Feller and Toulis (2019) and Puelz et al. (2022) derive randomization inference tests for a general class of null hypotheses under interference. A closely related study is Jiang, Imai and Malani (2023), who analyze two-stage completely randomized experiments and provide randomization-based variance estimators and sample size formulas. Our results complement this literature by considering different estimands, different assignment mechanisms and by conducting super-population-based large-sample (instead of design-based) inference in a double array asymptotic framework. Our approach allows us to determine the role of cluster heterogeneity in the asymptotic behavior of the treatment effect estimators.

Our paper is also related to the literature on inference in clustered experiments, which are a particular case of partial population experiments with only two saturations and no within-cluster treatment variation. Bugni et al. (2023) study inference in clustered experiments with non-ignorable cluster sizes and derive variance estimators and valid inference procedures in a setup with random cluster sizes. We further discuss the relationship between our results and that paper in Section 3.5.

We also contribute to a large empirical literature on property taxes and a small but growing empirical literature on spillover effects in tax compliance. On property taxes, recent contributions include Brockmeyer et al. (2020) study of Mexico City, Bergeron, Tourek and Weigel (2024) and Weigel (2020) for the Democratic Republic of Congo, and Krause (2020) for Haiti, among others. The latter two are randomized controlled trials, and in both cases, the authors address the presence of spillovers, but in ex-post analysis rather than in the experimental designs. The effect of social interactions in tax compliance interventions has remained a relatively elusive issue in the broader experimental compliance literature. Some notable exceptions are Pomeranz (2015), who detects enforcement spillovers up the VAT chain in Chilean firms, Drago, Mengel and Traxler (2020) who study enforcement spillovers of TV licensing inspections on untreated households in Austria, and Boning et al. (2020) who analyze direct and network effects from in-person visits by revenue officers on visited and non-visited firms in the United States (see the review in Pomeranz and Vila-Belda, 2019, for more studies covering spillover effects). In Argentina, a recent study by Carrillo, Castro and Scartascini (2021) finds neighborhood spillover effects from a program that randomly awarded 400 taxpayers with the repair of a sidewalk. Whereas these papers find spillover effects in tax compliance, their original experiments were not designed to capture these effects. We build on these pioneering works with an intervention designed with the purpose of capturing spillovers.

The paper is organized as follows. Section 2 illustrates the practical importance of cluster heterogeneity when conducting power calculations. In Section 3, we set up our framework and derive the main results. In 4
Section 4, we implement our methods in a large-scale randomized communication campaign, we describe the administrative data used in the analysis, the empirical strategy, and evidence of direct and spillover effects. Section 5 provides some practical recommendations for designing and analyzing partial population experiments. Section 6 concludes.

# **2 Why is Cluster Heterogeneity Important?**

We consider a population where units are grouped into mutually exclusive and independent clusters. Common examples of this type of clustering are students in schools (Miguel and Kremer, 2004; Beuermann et al., 2015), family members in households (Barrera-Osorio et al., 2011; Foos and de Rooij, 2017), job seekers in local labor markets (Cr ́epon et al., 2013), employees in firms or organizations (Duflo and Saez, 2003), or households in neighborhoods, villages or other geographic administrative units (Angelucci and De Giorgi, 2009; Ichino and Sch ̈undeln, 2012; Haushofer and Shapiro, 2016; Gin ́e and Mansuri, 2018). In our application, a local property tax reminder information campaign, the population of interest consists of taxpayers in residential blocks. Within this population, we study an experimental design where treatment assignments can vary both between and within clusters.

Figure 1 shows the distribution of cluster sizes in six partial population experiments, including our analysis sample and five published papers (Cr ́epon et al., 2013; Gin ́e and Mansuri, 2018; Haushofer and Shapiro, 2016; Ichino and Sch ̈undeln, 2012; Imai, Jiang and Malani, 2021). The figure reveals substantial variation in cluster sizes. When cluster sizes are heterogeneous, it is likely that the distribution of outcomes will vary across clusters as well. For instance, one may expect the mean and the variance of the outcome to be different in large clusters compared to small clusters. We refer to the variation in outcome distributions across clusters as _distributional heterogeneity_ .

Intuitively, with heterogeneous clusters, the variance of an estimator of interest _β_<sup>ˆ</sup> , such as a difference in means between units in treated and untreated clusters (we define the estimators of interest precisely in the next section), can be decomposed into four parts:

|+ clustering with equally-sized clusters|(2)|
|---|---|
|+ cluster size heterogeneity|(3)|
|+ cluster distributional heterogeneity|(4)|

The first term is the variance that would be obtained if observations were uncorrelated within clusters. The second term is an adjustment factor that accounts for the within-cluster correlation, often known as the “design effect” or the “Moulton factor” (after Moulton, 1986) that depends on the average cluster size. The term in the third line represents the additional variation due to the heterogeneity in cluster sizes, 5
which intuitively accounts for the variance of cluster sizes (Moulton, 1986, also derives this adjustment for a random effects model). Finally, the last component accounts for the between-cluster heterogeneity in outcome distributions. While the need to account for within-cluster correlations (lines (1) and (2)) is well-understood for designing and analyzing clustered experiments, the adjustment terms that account for cluster heterogeneity are typically assumed away by the literature on experimental design (e.g. Bloom, 2005; Duflo, Glennerster and Kremer, 2007; Hirano and Hahn, 2010; Baird et al., 2018).

To numerically illustrate the importance of appropriately accounting for cluster heterogeneity in this design, we consider the simple setting of a cluster RCT (which is a particular case of a partial population experiment) where “a few” clusters are “large”. Specifically, we consider a sample of 200 clusters, indexed by _g_ = 1 _, . . . ,_ 200, each having size _ng_ . The first 10 clusters contain 100 units, _ng_ = 100, and the remaining 190 clusters contain 25 units each, _ng_ = 25 (these values are chosen to match the median values in the literature in Figure 1). We assume the treatment has no effect, and the outcome of unit _iid i_ = 1 _, . . . , ng_ in cluster _g_ is given by a random effects model: _Yig_ = _αg_ + _νg_ + _ωig_ , _νg ∼N_ (0 _,_ 1 _/_ 2), _iid ωig ∼N_ (0 _,_ 1 _/_ 2) with _νg_ independent of _ωig_ and where _αg_ is a (non-random) intercept with _αg_ = 0 if _ng_ = 25 and _αg_ = 1 if _ng_ = 100. This model implies that the average outcome is E[ _Yig_ ] = 1 in large clusters and E[ _Yig_ ] = 0 in small clusters. In addition, V[ _Yig_ ] = 1 and the within-cluster correlation between outcomes is _cor_ ( _Yig, Yjg_ ) = 0 _._ 5.

Figure 2 plots three power functions for the difference in means between treated and untreated clusters that a researcher may consider when designing this experiment. The short-dashed curve represents the power function that is obtained when ignoring both sources of heterogeneity, that is, considering only the terms in lines (1) and (2) of the variance formula. Using this formula, the MDE at 80% power, given this sample size, is 0.29 standard deviations. However, when accounting for the variation in cluster sizes, the corresponding power function is represented by the long-dashed curve. According to this curve, the power to detect an effect of 0.29 is not 80% but 69%, so the experiment is underpowered. Furthermore, the true power function that accounts for both sources of heterogeneity (sizes and outcome distributions) is represented by the solid curve. This curve shows that the true power to detect an effect of 0.29 in this setting with heterogeneous clusters is 48%, significantly below the desired power of 80%. This numerical exercise shows how ignoring heterogeneity may result in severely underpowered experiments. We provide further examples of the importance of accounting for heterogeneity in Section 4.

# **3 Analysis of Partial Population Experiments**

## **3.1 Setup**

We consider a sample of observations (units) that are divided into mutually independent clusters _g_ = 1 _, . . . , G_ , where each cluster _g_ contains _ng_ observations _i_ = 1 _, . . . , ng_ and the total sample size is _n_ = 6
� _Gg_ =1<sup>_ng_.Weviewclustersizesasnon-random(seeBugnietal.,2023;SasakiandWang,2022,for</sup> an alternative sampling approach where cluster sizes are random). In a partial population experiment, clusters are randomly divided into categories or _saturations_ denoted by _Tg ∈{_ 0 _,_ 1 _,_ 2 _, . . . , M }_ , where by convention _Tg_ = 0 denotes a pure control cluster (i.e. a cluster where no unit is treated). Let P[ _Tg_ = _t_ ] = _qt ∈_ (0 _,_ 1) denote the probability that cluster _g_ is assigned to saturation _t_ . Within each cluster, a binary treatment _Dig_ is assigned to units with probability P[ _Dig_ = 1 _|Tg_ = _t_ ] where P[ _Dig_ = 0 _|Tg_ = 0] = 1.<sup>2</sup> We let **D** _g_ = ( _D_ 1 _g, D_ 2 _g, . . . , Dngg_ )<sup>_′_</sup> be the vector of unit-level treatment assignments in cluster _g_ , **D** = ( **D**<sup>_′_</sup> 1<sup>_, . . . ,_</sup><sup>**D**</sup><sup>_′_</sup> _G_<sup>)</sup><sup>_′_and</sup><sup>**T**=(</sup><sup>_T_1</sup><sup>_, . . . , TG_)</sup><sup>_′_.FigureA.3providesanexampleofapartialpopulationdesign</sup> with four saturations. Notice that both standard RCTs with independent observations and cluster RCTs are particular cases of partial population experiments, as we further illustrate in Section 3.5.

The observed outcome of interest for unit _i_ in cluster _g_ is denoted by _Yig_ and we let **Y** _g_ = ( _Y_ 1 _g, . . . , Yngg_ )<sup>_′_</sup> be the vector of observed outcomes in cluster _g_ . In partial population experiments, the estimands of interest are typically comparisons of average outcomes between treated or untreated units in treated clusters to pure control units, E[ _Yig|Dig_ = _d, Tg_ = _t_ ] _−_ E[ _Yig|Tg_ = 0], pooled across clusters. In the first part of the paper, we take these estimands as given since they are the most commonly analyzed estimands in the empirical literature. In Section 3.6, we set up a potential outcomes framework to rigorously justify the causal interpretation of these estimands. Let _μg_ ( _d, t_ ) = E[ _Yig|Dig_ = _d, Tg_ = _t_ ] be the conditional expectation of the outcome in cluster _g_ given assignment ( _d, t_ ). We consider the following sample means estimators:

where 1<sup>_t_</sup> _g_<sup>=1(</sup><sup>_Tg_=</sup><sup>_t_),</sup><sup>_N d_</sup> _g_<sup>=�</sup> _i_<sup>1(</sup><sup>_D_</sup> _ig_<sup>=</sup><sup>_d_)and</sup><sup>_Y_ ̄</sup> _g_<sup>_d_=�</sup> _i_<sup>_Y_</sup> _ig_<sup>1(</sup><sup>_D_</sup> _ig_<sup>=</sup><sup>_d_)</sup><sup>_/N_</sup> _g_<sup>_d_,definedwhenever</sup> _Ng_<sup>_d>_0.TheseestimatorsarecommonlycomputedbyrunninganOLSregressionoftheoutcomeon</sup> a full set of indicators ( 1 ( _Dig_ = _d, Tg_ = _t_ ))( _d,t_ ), without an intercept. Thus, in what follows, we refer to these estimators as OLS estimators. Our parameter of interest is the vector of cluster-size-weighted average of cluster-specific differences in means: We note that our framework can easily accommodate other parameters with different weighting schemes, such as the simple average across clusters<sup>�</sup><sup>_G_</sup> _g_ =1<sup>(</sup><sup>_μg_(</sup><sup>_d, t_)</sup><sup>_−μg_(0</sup><sup>_,_0))</sup><sup>_/G_.</sup>

> 2In practice, some desired saturations may not coincide with the observed proportion of treated units for some cluster sizes. For instance, if P[ _Dig_ = 1 _|Tg_ = _t_ ]=0.5 but _ng_ is odd, the observed proportion of treated cannot be exactly 0.5. Appendix A.4 proposes an assignment mechanism that ensures that the expected proportion of treated coincides with P[ _Dig_ = 1 _|Tg_ = _t_ ].

7
# **3.2 Asymptotic Behavior of OLS Estimators**

We now study the asymptotic distribution of the OLS estimators defined in Equation (5) and functions thereof. We consider a double-array asymptotic setting where the cluster sizes are allowed, but not required, to grow with the sample size. This type of approximation is more appropriate than the bounded cluster size approach when groups can be large and heterogeneous in size, but we note that the settings with bounded cluster sizes and/or equally-sized clusters are nested as particular cases of our analysis.<sup>3</sup> We consider the following sampling scheme.

## **Assumption 1 (Sampling)**

- _(i)_ ( **Y** _g_<sup>_′,_</sup><sup>**D**</sup><sup>_′_</sup> _g_<sup>_, Tg_)</sup><sup>_G_</sup> _g_ =1<sup>_are mutually independent across g._</sup>

- _(ii) For each g and for all i_ = 1 _, . . . , ng,_ E[ _Yig_<sup>_l|Dig_=</sup><sup>_d, Tg_=</sup><sup>_t_]=</sup><sup>_μl_</sup> _g_<sup>(</sup><sup>_d, t_)</sup><sup>_forall_(</sup><sup>_d, t_)</sup><sup>_andforalll_</sup> _such that_ E[ _|Yig|_<sup>_l_</sup> _|Dig_ = _d, Tg_ = _t_ ] _< ∞._

- _(iii) For each g and for all i_ = 1 _, . . . , ng,_ P[ _Dig_ = _d|Tg_ = _t_ ] = _pg_ ( _d|t_ ) _and_ P[ _Dig_ = _d, Djg_ = _d_<sup>_′_</sup> _|Tg_ = _t_ ] = _pg_ ( _d, d_<sup>_′_</sup> _|t_ ) _for all d, d_<sup>_′_</sup> _and t._

Part (i) states that clusters are mutually independent, a standard assumption in the clustering literature. Notice that we do not require clusters to be identically distributed, so outcome distributions can be heterogeneous across clusters. Part (ii) states that average conditional outcomes are the same for all units in the same cluster. In what follows we define _μ_<sup>_l_</sup> _g_<sup>(</sup><sup>_d, t_)=</sup><sup>_μg_(</sup><sup>_d, t_)for</sup><sup>_l_=1toreducenotation.Part</sup> (iii) states that the unit-level treatment probabilities are the same within a cluster. Note that within-cluster assignments may be correlated.

Next, let **D** ( _i_ ) _g_ = ( _Djg_ ) _j_ = _i_ denote the vector of treatments excluding unit _i_ and **D** ( _ij_ ) _g_ = ( _Dkg_ ) _k_ =( _i,j_ ) denote the vector of treatments excluding units _i_ and _j_ . We introduce the following restriction on the conditional moments of the outcome.

## **Assumption 2 (Exchangeability)** _For all i, j and g,_

This assumption is a high-level condition stating that, conditional on own treatment assignment and the cluster-level assignment _Tg_ , the first and second moments of _Yig_ do not vary with the peers’ treatment indicators.<sup>4</sup> As shown in Theorem 1 below, these conditions guarantee that the OLS estimator is consistent

> 3The number of parameters remains fixed in our setup. See Vazquez-Bare (2023) for an alternative approach in which the number of parameters is allowed to grow with the sample size.

> 4We refer to unit _i_ ’s “peers” as all the units other than _i_ in the same cluster.

8
for a weighted average of cluster-specific conditional means and that the outcome variance only depends on ( _d, t_ ), the variation in treatment assignment that is controlled by the experimental design. Assumption 2 can be interpreted as a requirement that the assignment ( _Dig, Tg_ ) contains all the relevant variation in the outcome moments, so that the spillovers model is “correctly specified”. To further justify this assumption, in Section 3.6 we show that this condition is guaranteed when peers are assumed to be exchangeable, so that potential outcomes only depend on the proportion of treated peers and not on their identities. This exchangeability assumption is very common in the spillovers literature. This requirement may be violated, for example, in networks where units have different degrees of network centrality, and thus both the proportion of and the identities of the treated units matter.

Finally, we restrict cluster heterogeneity in the following way.

# **Assumption 3 (Cluster heterogeneity and bounded moments)**

2 _/r (i) For some_ 2 _≤ r < ∞, as n →∞,_ max _g n_<sup>2</sup> _g_<sup>_/n →_0</sup><sup>_and_��</sup> _g_<sup>_nr_</sup> _g_ � _/n ≤ C < ∞. (ii) For some l> r,_ sup _i,g,d,t_ E[ _|Yig|_<sup>_l_</sup> _|Dig_ = _d, Tg_ = _t_ ] _≤ C_<sup> ̃</sup> _< ∞._

Condition (i) is taken from Hansen and Lee (2019). The first part ensures that the largest cluster is small relative to the total sample size, so no cluster dominates the sample. The second part of condition (i) is a regularity condition that rules out unbounded _r_ -th moments in the distribution of cluster sizes. As an example, setting _r_ = 4 restricts the fourth moment of the cluster size distribution, which rules out heavy tails.<sup>5</sup> Condition (ii) is a standard regularity condition that ensures that the _l_ -th conditional moment of the outcome is bounded.

In what follows, we use “ _→_ P” to denote convergence in probability “plim _n→∞_ ” to denote probability limits, “ _→D_ ” to denote convergence in distribution and _∥·∥_ to denote the Euclidean norm. We define any generic (2 _M_ + 1)-dimensional vector **v** as: Consider the vector of estimators **_μ_ ˆ** _n_ = (ˆ _μ_ ( _d, t_ ))<sup>_′_</sup> ( _d,t_ )<sup>from (5) and define the vector:</sup> Define the (2 _M_ + 1) _×_ (2 _M_ + 1) covariance matrix Ω _n_ with elements: 5Notice that condition (i) holds automatically when group sizes are seen as fixed or bounded in the asymptotic analysis.

9
where _p_ ̄ _n_ ( _d|t_ ) :=<sup>�</sup> _g_<sup>_n_</sup> _g_<sup>_p_</sup> _g_<sup>(</sup><sup>_d|t_)</sup><sup>_/n_.InwhatfollowsweuseΩ</sup> _n_<sup>(</sup><sup>_d, t_)torefertothediagonalelementsof</sup> Ω _n_ . We introduce the following technical conditions to guarantee invertibility of the covariance matrix and to ensure the denominators of the estimators are bounded below.

## **Assumption 4 (Invertibility conditions)**

- _(i) The minimum eigenvalue of_ Ω _n is bounded away from 0._

The following theorem characterizes the asymptotic distribution and variance of the OLS estimators in (5). Let _I_ 2 _M_ +1 is a (2 _M_ + 1)-dimensional identity matrix.

All the proofs can be found in Appendix D. Because the estimator **_μ_ ˆ** can be obtained through a saturated OLS regression including one regressor per distinct treatment assignment, Theorem 1 can be thought of as generalizing the results in Hansen and Lee (2019) to a specific type of nonparametric regression where coefficients are heterogeneous across clusters.

# **3.3 Estimation and Inference for Differences in Means**

Theorem 1 has two main implications. First, each _μ_ ˆ( _d, t_ ) estimates a weighted average of cluster-specific means _μg_ ( _d, t_ ), where the weights depend on the cluster size _ng_ and the within-cluster probability of treatment _pg_ ( _d|t_ ). Second, the distribution of **_μ_ ˆ** _n_ can be approximated as **_μ_ ˆ** _n ∼Na_ ( **_μ_** _pn_<sup>_,_Ω</sup><sup>_n/n_) where the</sup> variance matrix Ω _n_ allows for heterogeneity in cluster sizes and outcomes distributions, heteroskedasticity, different treatment assignment probabilities across clusters and intracluster correlation in both outcomes and unit-level treatment assignments. This result can be applied to obtain an asymptotic distributional approximation and variance formulas for functions of **_μ_ ˆ** _n_ , such as subvectors, linear combinations (like the pooled and slope effects proposed by Baird et al., 2018) or nonlinear functions thereof, applying the delta method when needed.

ˆ ˆ Theorem 1 implies that the difference-in-means estimators _β_<sup>ˆ</sup> ( _d, t_ ) = _μ_ ( _d, t_ ) _− μ_ (0 _,_ 0) consistently estimate: which is different from our parameter of interest (6) because treatment probabilities may differ across clusters. When the treatment probabilities are equal across clusters, _pg_ ( _d|t_ ) = _p_ ( _d|t_ ) for all _g_ , _βn_<sup>_p_(</sup><sup>_d, t_)=</sup> _βn_ ( _d, t_ ) so the parameter of interest can be consistently estimated by OLS. Thus, in settings with heterogeneous clusters, the experimenter may prefer designs in which the within-cluster treatment probabilities

10
do not vary across clusters with the same assignment _Tg_ = _t_ , or to reweight the estimators by the inverse of _pg_ ( _d|t_ ).

When conducting inference and hypothesis testing, the variance of the estimators of interest is commonly estimated using a cluster-robust variance estimator. In this setting, and ignoring finite-sample degrees-of-freedom adjustments, the cluster-robust variance estimator of Ω _n_ is: where 1 _g_ = ( 1<sup>_′_</sup> 1 _g_<sup>_, . . . ,_1</sup> _n_<sup>_′_</sup> _gg_<sup>)</sup><sup>_′_isan</sup><sup>_ng×_(2</sup><sup>_M_+ 1)matrixand1</sup><sup>_ig_=(1(</sup><sup>_Dig_=</sup><sup>_d, Tg_=</sup><sup>_t_))(</sup><sup>_d,t_)isan</sup> _ng_ -dimensional column vector. Based on this matrix estimator, the cluster-robust variance estimator for the difference in means _β_<sup>ˆ</sup> ( _d, t_ ) is _V_<sup>ˆ</sup> `cr` ( _d, t_ ) = Ω<sup>ˆ</sup> `cr` ( _d, t_ ) + Ω<sup>ˆ</sup> `cr` (0 _,_ 0) using that Ω<sup>ˆ</sup> `cr` (( _d, t_ ) _,_ ( _d_<sup>_′_</sup> _, t_<sup>_′_</sup> )) = 0 for _t_ = _t_<sup>_′_</sup> . The following result shows that, in a setting with distributional heterogeneity, the cluster-robust variance estimator for the difference in means can be conservative.

**Proposition 1** _Let Vn_ ( _d, t_ ) = Ω _n_ ( _d, t_ )+Ω _n_ (0 _,_ 0) _−_ 2Ω _n_ (( _d, t_ ) _,_ (0 _,_ 0)) _denote the true asymptotic variance of β_<sup>ˆ</sup> ( _d, t_ ) _. Under Assumptions 1 to 4,_ plim _n→∞_ Ä _V_ ˆ `cr` ( _d, t_ ) _/Vn_ ( _d, t_ )ä _≥_ 1 _._

The reason why the cluster-robust variance estimator can be conservative is that the true asymptotic variance can be approximated as: The first two lines in Equation (8) represent the average within-cluster variation in outcomes for the units in treated and pure control clusters, respectively. The third and fourth lines represent the between-cluster variation in average outcomes for treated and control clusters, respectively. Finally, the fifth line can be interpreted as a the between-cluster variance of the difference in means, weighted by the relative ̄ probabilities of treatment in each cluster. Note that when _pg_ ( _d|t_ ) = _pn_ ( _d|t_ ), this last term becomes � _g_<sup>_n_2</sup> _g_<sup>(</sup><sup>_βg_(</sup><sup>_d, t_)</sup><sup>_−βn_(</sup><sup>_d, t_))2</sup><sup>_/n_.The last three lines in this formula equal zero when outcome distributions</sup> are homogeneous between clusters, as we discuss below.

The last term in Equation (8) is not estimable because it depends on the within-cluster difference in 11
means between assignments, which is never observed. The cluster-robust variance estimator is, asymptotically: which is Equation (8) without the last term. Thus, _V_<sup>ˆ</sup> `cr` ( _d, t_ ) can be asymptotically upward-biased, and thus inference based on this variance estimator can be conservative. Similar results have also been obtained in design-based causal inference settings with non-random potential outcomes, see for example Hudgens and Halloran (2008); Basse and Feller (2018); Abadie et al. (2022) and Jiang, Imai and Malani (2023). Proposition 1 shows that an analogous result holds in a superpopulation setting when clusters exhibit distributional heterogeneity. In particular, when outcome distributions are homogeneous across clusters, this additional term disappears and inference based on the cluster-robust variance estimator is asymptotically exact, as we discuss further in Section 3.5.

# **3.4 Power Calculations and Optimal Design**

By Theorem 1, the power of a two-sided hypothesis test of _βn_<sup>_p_(</sup><sup>_d, t_) = 0, can be approximated by:</sup> for some appropriately chosen asymptotic variance _V_ , where _z_ 1 _−α/_ 2 is the (1 _− α/_ 2)-quantile from the standard normal distribution. To use the true variance formula, the researcher may replace _V_ by Equation (8). This variance depends on the within-cluster variances, intra-cluster correlation and the betweencluster variation in outcomes, which can be imputed using baseline data, the cluster size distribution, which is observable, and the cluster- and unit-level assignment probabilities which are chosen by the researcher. One issue with this choice of variance formula is that, as shown in Proposition 1, the variance estimator that is actually used when conducting inference may be upward biased, which may result in an underpowered study. To avoid this issue, the researcher may instead conduct power calculations using the variance formula in Equation (9).

The number of saturations _M_ and the within-cluster treatment probabilities _pg_ ( _d|t_ ) and _pg_ ( _d, d|t_ ) play a crucial role in identification, as they determine the type of comparisons that can be made between treated and control units. The choice of these parameters can be guided by previous knowledge or assumptions on how the conditional average outcome varies as a function of the treatment saturation. For instance, if this function is assumed to be linear or close to linear, two saturations would be enough to identify the shape of this function, whereas if the function can be approximated by a quadratic function, one would need three saturations, and so on. In turn, the choice of within-cluster treatment probabilities _pg_ ( _d|t_ ) depends on the slope of the conditional average outcome as a function of the treatment saturation. For instance,

12
̄ with three saturations _M_ = _{_ 0 _,_ 1 _,_ 2 _}_ , if average outcomes are expected to jump around some value _p_ but to be relatively flat below or above _p_ ̄, the researcher can choose _pg_ (1 _|_ 1) _< p_ ̄ and _pg_ (1 _|_ 2) _> p_ ̄ to increase the chance of detecting these changes. Without knowledge of how this function is expected to change, the researcher may spread these probabilities approximately uniformly, choosing some “low”, “intermediate” and “high” treatment probabilities. While we do not provide formal guidance on choosing _M_ and the within-cluster treatment probabilities, our results in Theorem 1 and power function (10) can be used to compare the power and MDEs of competing designs.

We now propose a method to optimally choose the cluster-level assignment probabilities _{qt}_<sup>_M_</sup> _t_ =0<sup>.</sup> Given _M_ and the within-group treatment probabilities, optimally choosing _{qt}_<sup>_M_</sup> _t_ =0<sup>requiresdefiningan</sup> optimality criterion that determines how the variances of all the estimators of interest are aggregated. The literature on optimal design of experiments has proposed several criteria (see e.g. Silvey, 1980; Melas, 2006; Berger and Wong, 2009). We consider _A-optimality_ , which minimizes the trace of the variancecovariance matrix of the difference in means estimators ( _β_<sup>ˆ</sup> ( _d, t_ ))( _d,t>_ 0) (or equivalently, the average of the asymptotic variances).<sup>6</sup> The justification of this criterion is that the trace of the variance-covariance matrix can be seen as a measure of the size of the confidence ellipsoid (i.e. the multidimensional confidence interval) for the vector of parameters of interest. One advantage of A-optimality is its tractability, as the optimal choice has a simple closed-form solution in this setting. In the theorem below, we consider a generalized version of A-optimality that allows the researcher to assign different weights to different variances.

**Theorem 2** _Let_ **_ω_** = ( _ωdt_ )<sup>_′_</sup> ( _d,t>_ 0)<sup>_be a known vector of weights with ωdt≥_0</sup><sup>_, ω_1</sup><sup>_t_+</sup><sup>_ω_0</sup><sup>_t>_0</sup><sup>_,_�</sup> _t>_ 0<sup>(</sup><sup>_ω_</sup> 0 _t_<sup>+</sup> _ω_ 1 _t_ ) = 1 _. Consider the optimal design problem:_

_with qt >_ 0 _,_<sup>�</sup><sup>_M_</sup> _t_ =0<sup>_qt_=1</sup><sup>_usingthevarianceformulainEquation_(8)</sup><sup>_or_(9)</sup><sup>_.Theoptimalassignment_</sup> _probabilities are given by:_ _where_

> 6Notice that this criterion is different from the one in Baird et al. (2018), who minimize the average standard error. We propose this alternative method as it is in line with the theoretical literature on optimal design, while also allowing for a simple, closed-form solution to the optimal design problem.

13
_and for t >_ 0 _,_ Theorem 2 provides the formula for the optimal cluster assignment probabilities that minimize a weighted average of estimators variances. By choosing the vector ( _ωdt_ )<sup>_′_</sup> ( _d,t>_ 0)<sup>,theresearchercanassignlower(or</sup> zero) weights to some parameters that are not of interest, and larger weights to parameters that are deemed more important. For instance, to focus on comparisons between untreated units in treated clusters and pure controls, the researcher can set _ω_ 1 _t_ = 0 for all _t_ .

While A-optimality has the advantage of a simple closed form solution, there are other optimality criteria that may be desirable in different settings. Optimization problems based on these alternative criteria do not have closed form solutions in general, but can be solved numerically using our variance formulas. See Silvey (1980), Melas (2006) and Berger and Wong (2009) for further details and discussions.

It should be noted that researchers may often need to incorporate different sets of constraints (such as logistical, budgetary, political or administrative constraints) when choosing assignment probabilities. These restrictions can be incorporated when choosing _qt_ , either directly into the optimization problem in Theorem 2 or on a case-specific basis. For example, in the experiment we describe in the next section, the total number of treated units was set by the government agency. We set up a system of equations incorporating this restriction to control the variance of the smallest treatment cells (i.e. the noisiest estimators). See Section 4.3 for details.

Finally, we note that our optimality criterion does not incorporate baseline covariates. A strand of the literature on experiments has considered alternative designs that include observed covariates in the treatment assignment mechanism as a way to improve balance and increase precision. A common design in these settings is the matched pairs design (Imai, King and Nall, 2009; Bai, 2022; Liu, 2023), where each cluster is paired with another one with similar covariates and then treatment is randomized within each pair. Our results provide a different and complementary approach that does not require covariates, and instead allows the researcher to assign different weights to the different variances of the estimators of interest.

14
# **3.5 Power Calculations under Distributional Homogeneity**

The formulas for Ω _n_ from Theorem 1 and Equations (8) and (9) can be difficult to implement when the researcher does not have access to baseline outcome data. We now introduce an additional assumption that simplifies the variance formulas and makes them easier to implement in the absence of this information. Specifically, the following assumption rules out between-cluster heterogeneity in conditional outcome moments.

**Assumption 5 (Between-Cluster Moment Homogeneity)** E[ _Yig_<sup>_l|Dig_=</sup><sup>_d, Tg_=</sup><sup>_t_] =</sup><sup>_μl_(</sup><sup>_d, t_)</sup><sup>_and_E[</sup><sup>_Y_</sup> _ig_<sup>_lY_</sup> _jg_<sup>_l|Dig_=</sup> _d, Djg_ = _d_<sup>_′_</sup> _, Tg_ = _t_ ] = ̃ _c_<sup>_l_</sup> ( _d, d_<sup>_′_</sup> _, t_ ) _for all g,_ ( _d, d_<sup>_′_</sup> _, t_ ) _and for any l for which the moments exist._

As before, we write _μ_<sup>1</sup> ( _d, t_ ) = _μ_ ( _d, t_ ) to reduce notation. Under this additional assumption we obtain the following result.

**Corollary 1** _Suppose Assumptions 1 to 5 hold. Then μ_<sup>_p_</sup> _n_<sup>(</sup><sup>_d, t_) =</sup><sup>_μ_(</sup><sup>_d, t_)</sup><sup>_for all_(</sup><sup>_d, t_)</sup><sup>_, Theorem 1 holds and_</sup> _the variance_ Ω _n takes the following form:_

_and where σ_<sup>2</sup> ( _d, t_ ) = V[ _Yig|Dig_ = _d, Tg_ = _t_ ] _, ρ_ ( _d, t_ ) = cor( _Yig, Yig|Dig_ = _d, Djg_ = _d, Tg_ = _t_ ) _, pg_ ( _d, d_<sup>_′_</sup> _|t_ ) = P[ _Dig_ = _d, Djg_ = 1 _|Tg_ = _t_ ] _, and ρ_ (0 _,_ 1 _, t_ ) = Cov( _Yig, Yig|Dig_ = 0 _, Djg_ = 1 _, Tg_ = _t_ ) _. In_ ˆ _addition,_ plim _n→∞_ Ä _V_ `cr` ( _d, t_ ) _/Vn_ ( _d, t_ )ä = 1 _._

Corollary 1 has three main implications. First, under between-cluster homogeneity, the difference-in- _−_ means estimators are consistent for the population differences in means _β_ ( _d, t_ ) = _μ_ ( _d, t_ ) _μ_ (0 _,_ 0). Second, it shows that under this additional assumption, the cluster-robust variance estimator is consistent and thus inference based on this estimator is asymptotically exact. Third, it provides a simplified variance formula that allows for heterogeneity in cluster sizes and within-cluster probabilities, conditional heteroskedasticity and intracluster correlation in outcomes and treatments, but does not depend on cluster-specific average outcomes like the variance formula in Theorem 1. This simplified formula can be readily used to conduct 15
power and MDE calculations for the parameters of interest. Specifically, which only depends on the variance and conditional intracluster correlation in outcomes (as in any standard power calculation), the assignment probabilities, which are chosen by the experimenter, and the sample distribution of cluster sizes, which is observable. This variance can be fed into the power formula (10) to calculate power and MDEs. We discuss practical implementation issues in more detail in Sections 4 and 5.

As a word of caution, we note that, just like ignoring cluster size heterogeneity, incorrectly imposing Assumption 5 when conducting power calculations can result in variances and MDEs that are too small because they ignore between-cluster variability in outcomes. While this assumption may be strong in some settings, most of the formulas for experimental design available in the literature rely on it. To illustrate this point, the following examples show how our general formulas simplify to the ones proposed in the literature under further assumptions.

**Example 1 (Standard RCT with a binary treatment)** _Suppose that each cluster has one unit (ng_ = 1 _), and there are two saturations so that each (single-unit) cluster is assigned to treatment or control with probability q and_ 1 _− q respectively. In this case, qt_ = _q, q_ 0 = 1 _− q,_<sup>�</sup> _g_<sup>_n_</sup> _g_<sup>_p_</sup> _g_<sup>(1</sup><sup>_|_1)=</sup><sup>_nandunder_</sup> _Assumptions 1 to 5,_ V[ _β_<sup>ˆ</sup> (1 _,_ 1)] _≈ σ_<sup>2</sup> (1 _,_ 1) _/nq_ + _σ_<sup>2</sup> (0 _,_ 0) _/n_ (1 _− q_ ) _. In addition, under the homoskedasticity assumption σ_<sup>2</sup> (1 _,_ 1) = _σ_<sup>2</sup> (0 _,_ 0) = _σ_<sup>2</sup> _,_ V[ _β_<sup>ˆ</sup> (1 _,_ 1)] _≈ σ_<sup>2</sup> _/nq_ (1 _− q_ ) _which is Equation (6) in Duflo, Glennerster and Kremer (2007)._

**Example 2 (Cluster RCT)** _Suppose that clusters are assigned to two saturations Tg ∈{_ 0 _,_ 1 _} and that all units within the same cluster receive the same treatment. In this case, pg_ (1 _|_ 1) = _pg_ (1 _,_ 1 _|_ 1) = 1 _and under Assumptions 1 to 4,_

_where μg_ (1) = _μg_ (1 _,_ 1) _, μg_ (0) = _μg_ (0 _,_ 0) _and similarly for the remaining terms. This formula is analogous to the one derived by Bugni et al. (2023) for what they call the size-weighted cluster-level average treatment effect, up to a term in their formula that accounts for the stratification procedure._<sup>7</sup> _Furthermore, if Assumption 5 holds,_ 7See also Liu (2023) for an analysis of stratification in two-stage experiments.

16
̄ _Finally, suppose clusters are equally-sized, ng_ = _n, and assume a random effects structure so that σ_<sup>2</sup> (1 _,_ 1) = _σ_<sup>2</sup> (0 _,_ 0) = _σ_<sup>2</sup> + _τ_<sup>2</sup> _and ρ_ (1 _,_ 1) = _ρ_ (0 _,_ 0) = _τ_<sup>2</sup> _/_ ( _σ_<sup>2</sup> + _τ_<sup>2</sup> ) _. In this case,_ V[ _β_<sup>ˆ</sup> (1 _,_ 1)] _≈_ ( ̄ _nτ_<sup>2</sup> + _σ_<sup>2</sup> ) _/_ [ _q_ (1 _− q_ ) _Gn_ ̄] _which is Equation (9) in Duflo, Glennerster and Kremer (2007)._

**Example 3 (Homoskedastic case with two treatment saturations)** _Suppose there are only two saturations, so that M ∈{_ 0 _,_ 1 _}, as in Duflo and Saez (2003). Let q_ = P[ _Tg_ = 1] _and p_ = P[ _Dig_ = 1 _|Tg_ = 1] _. Assume that σ_<sup>2</sup> ( _d, t_ ) = 1 _and ρ_ ( _d, t_ ) = 0 _for all_ ( _d, t_ ) _. In this case, for assignment_ ( _d, t_ ) = (0 _,_ 1) _, under Assumptions 1 to 5,_ V[ _β_<sup>ˆ</sup> (0 _,_ 1)] _≈_ (1 _− pq_ ) _/_ [(1 _− p_ ) _q_ (1 _− q_ )] _which corresponds to the variance formula in Hirano and Hahn (2010)._

**Example 4 (Random effects structure with equally-sized clusters)** _Suppose that all clusters are equally_ ̄ _sized, ng_ = _n for all g, and consider a random effects covariance structure so that σ_<sup>2</sup> ( _d, t_ ) = _σ_<sup>2</sup> + _τ_<sup>2</sup> _, ρ_ ( _d, t_ ) = _τ_<sup>2</sup> _for all_ ( _d, t_ ) _. In addition, suppose that the within-cluster assignment given Tg_ = _t sets_ ̄ _a fixed number of treated units npt in each cluster, which implies that_ P[ _Dig_ = 1 _, Djg_ = 1 _|Tg_ = _t_ ] = _pt_ ( ̄ _npt −_ 1) _/_ ( ̄ _n −_ 1) _. In this case, for assignment_ (1 _, t_ ) _, under Assumptions 1 to 5,_ V[ _β_<sup>ˆ</sup> (1 _, t_ )] _≈_ ( _σ_<sup>2</sup> + _τ_<sup>2</sup> ) ¶ _nρ_ ̄ Ä _q_ <u>1</u> _t_<sup>+</sup> _q_<sup><u>1</u></sup> 0 ä + (1 _− ρ_ ) Ä _pt_ <u>1</u> _qt_<sup>+</sup> _q_<sup><u>1</u></sup> 0 ä© _/n which corresponds to Equation (3) in Baird et al. (2018)._

# **3.6 A Potential Outcomes Framework**

In this section we introduce a potential outcomes framework to study the causal interpretation of the OLS estimands discussed in the previous sections. Let _Yig_ ( _d,_ **d** _g, t_ ) denote unit _i_ ’s (random) potential outcomes where _d_ denotes own treatment, **d** _g ∈{_ 0 _,_ 1 _}_<sup>_ng−_1</sup> is a vector denoting unit _i_ ’s peers’ treatments and _t_ denotes the cluster-level assignment. To be able to compare outcomes across clusters, our first assumption is an exclusion restriction stating that the cluster level assignment _t_ does not directly affect potential outcomes.

## **Assumption 6 (Exclusion restriction)** _Yig_ ( _d,_ **d** _g, t_ ) = _Yig_ ( _d,_ **d** _g_ ) _for all_ ( _d,_ **d** _g, t_ ) _._

While this assumption is required to identify treatment effects using variation across clusters, to our knowledge we are the first to make it explicit. This potential outcome structure allows for within-cluster spillovers, an assumption often known as _stratified interference_ (Hudgens and Halloran, 2008). Specifically, _Yig_ (1 _,_ **d** _g_ ) _−Yig_ (0 _,_ **d** _g_ ) is the direct effect of the treatment on unit _i_ in cluster _g_ , _Yig_ (0 _,_ **d** _g_ ) _−Yig_ (0 _,_ **d**<sup>** ̃**</sup> _g_ ) is the spillover effect on an untreated unit and _Yig_ (1 _,_ **d** _g_ ) _− Yig_ (1 _,_ **d**<sup>** ̃**</sup> _g_ ) is the spillover effect on a treated unit. The observed outcome of interest for unit _i_ in cluster _g_ is denoted by _Yig_ =<sup>�</sup> _d,_ **d** _g_<sup>_Y_</sup> _ig_<sup>(</sup><sup>_d,_</sup><sup>**d**</sup> _g_<sup>)1(</sup><sup>**D**</sup> _g_<sup>=</sup> ( _d,_ **d** _g_ )).

Next, we assume that the vector of treatment assignments ( **D** _g, Tg_ ) is independent of the vector of potential outcomes, which is guaranteed by random assignment of the treatment.

## **Assumption 7 (Independence)** ( _Yig_ ( _d,_ **d** _g_ ))( _d,_ **d** _g_ ) _⊥⊥_ ( **D** _g, Tg_ ) _._

17
Finally, we assume that peers are exchangeable. Under this assumption, potential outcomes can depend flexibly on the proportion of treated peers, as long as they do not depend on the peers’ identities. This assumption reduces the dimensionality of potential outcomes and is ubiquitous when analyzing spillovers (see Vazquez-Bare, 2023, and references therein for further discussion). In what follows let **1** _g_ be an ( _ng −_ 1)-dimensional column vector of ones.

**Assumption 8 (Exchangeability)** _For all_ **d** _g, Yig_ ( _d,_ **d** _g_ ) = _Yig_ ( _d, πg_ ) _where πg_ = **1**<sup>_′_</sup> _g_<sup>**d**</sup><sup>_g/_(</sup><sup>_ng−_1)</sup><sup>_isthe_</sup> _proportion of unit i’s treated peers._

The following result links moments of observed outcomes to average potential outcomes.

**Proposition 2** _Under Assumptions 6 to 8, letting Sig_ =<sup>�</sup>

_for any l such that the expectations are well defined._ Proposition 2 implies that the conditional mean E[ _Yig|Dig_ = _d, Tg_ = _t_ ] in cluster _g_ equals an average of average potential outcomes over the proportions of treated peers that are consistent with the assignment mechanism. In particular, if the treatment assignment mechanism exactly determines the proportion of treated units, so that P[ _Sig_ = _sg|Dig_ = _d, Tg_ = _t_ ] = 1 for some _sg_ , each observed conditional mean point-identifies the average potential outcome.

**Theorem 3** _Let Ng_<sup>1=�</sup><sup>_n_</sup> _i_ =1<sup>_gDigbethetotalnumberoftreatedunitsinclusterganddefine_</sup><sup>**y**</sup><sup>_g_=</sup> ( _Yig_ ( _d,_ **d** _g_ )) _i,d,_ **d** _g . Suppose that:_

- _(i) Assumptions 6 to 8 hold._

- _(ii)_ ( **y** _g_<sup>_′,_</sup><sup>**D**</sup><sup>_′_</sup> _g_<sup>_, Tg_)</sup><sup>_G_</sup> _g_ =1<sup>_are mutually independent across g; for each gand for all i,_E[</sup><sup>_Y_</sup> _ig_<sup>_l_(</sup><sup>_d, π_)]=</sup><sup>_μ_ ̃</sup><sup>_l_</sup> _g_<sup>(</sup><sup>_d, π_)</sup> _for all_ ( _d, π_ ) _and for all l such that_ E[ _Yig_<sup>_l_(</sup><sup>_d, π_)]</sup><sup>_< ∞; Assumption 1(iii) holds._</sup>

- _(iii) Assumption 3(i) holds and for some l> r,_ max _i,g,d,π_ E[ _|Yig_<sup>_l_(</sup><sup>_d, π_)</sup><sup>_|_]</sup><sup>_≤C_ ̃</sup><sup>_< ∞._</sup>

- _(iv) Ng_<sup>1</sup><sup>_is nonrandom conditional on Tg,with_P[</sup><sup>_N_1</sup> _g_<sup>=</sup><sup>_ngpg_(1</sup><sup>_|t_)</sup><sup>_|Tg_=</sup><sup>_t_]=1</sup><sup>_and pg_(</sup><sup>_d|t_)=</sup><sup>_p_(</sup><sup>_d|t_)</sup><sup>_for_</sup> _all g._

_Then, Theorem 1 holds and_ 18
Theorem 3 provides conditions on the potential outcomes and experimental design to guarantee that Theorem 1 holds. By Proposition 2 and Condition (iii) of Theorem 3, moments of observed outcomes for ( _d, t_ ) can be replaced by moments of potential outcomes for ( _d,_ ( _ngp_ (1 _|t_ ) _− d_ ) _/_ ( _ng −_ 1)) so the formulas in Theorem 1 can be readily applied, as long as the variance matrix is invertible. In addition, Theorem 3 ensures that differences in means have a causal interpretation: each _βn_ ( _d, t_ ) equals a cluster-size-weighted average of average differences in potential outcomes. By Theorem 1, these parameters can be consistently estimated by OLS.

When cluster sizes vary, average potential outcomes may vary across clusters, even within the set of clusters with the same assignment _Tg_ = _t_ and when the observed proportion of treated units is fixed. To see this, consider the following example. Suppose there are two cluster sizes, _ng_ = 16 and _ng_ = 20, and consider clusters with _pg_ (1 _|t_ ) = 0 _._ 5 so that half the units are assigned to treatment. In clusters with _ng_ = 16, the total number of treated units will be 8 and thus the proportion of treated peers for each unit is 8 _/_ 15 _≈_ 0 _._ 535 for untreated units and 7 _/_ 16 = 0 _._ 438 for treated units. On the other hand, in clusters with _ng_ = 20 there will be 10 treated units and thus the proportion of treated peers is 10 _/_ 19 _≈_ 0 _._ 526 for untreated units and 9 _/_ 19 _≈_ 0 _._ 474 for treated units. Hence, an untreated unit in a cluster with treatment intensity _pg_ (1 _|t_ ) = 0 _._ 5 will have a proportion of 0.533 treated peers if the cluster size is 16, and a proportion of 0.526 treated peers if the cluster size is 20, so the proportions are slightly different even though the treatment assignment is the same. As a result, to be able to use the simplified formula in Corollary 1, the outcome homogeneity assumption needs to be strengthened to ensure that average potential outcomes are invariant to small perturbations in the proportion of treated units, as shown below.

# **Theorem 4** _Suppose that the conditions for Theorem 3 hold, and that:_

- _(i)_ E[ _Yig_<sup>_l_(</sup><sup>_d, π_)] =</sup><sup>_μ_ ̃</sup><sup>_l_(</sup><sup>_d, π_)</sup><sup>_and_E[</sup><sup>_Y_</sup> _ig_<sup>_l_(</sup><sup>_d, π_)</sup><sup>_Y_</sup> _jg_<sup>_l_(</sup><sup>_d′, π′_)] =  ̃</sup><sup>_cl_(</sup><sup>_d, d′, π, π′_)</sup><sup>_for all g and_(</sup><sup>_d, d′, π, π′_)</sup><sup>_._</sup>

- _(ii) For each_ ( _d, d_<sup>_′_</sup> _, t_ ) _there exists a π_ ( _d|t_ ) _such that for πg_ ( _d|t_ ) = ( _ngpg_ (1 _|t_ ) _−d_ ) _/_ ( _ng−_ 1) _,_ max _g_ ��� _μ_ ̃ _l_ ( _d, πg_ ( _d|t_ )) _− μ_ ̃ _l_ 0 _and_

̃ _l ′ ′_ ̃ _l ′ ′_ max _g_ ��� _c_ ( _d, d , πg_ ( _d|t_ ) _, πg_ ( _d |t_ )) _− c_ ( _d, d , π_ ( _d|t_ ) _, π_ ( _d |t_ ))��� = 0 _._

_Then Corollary 1 holds and β_ ( _d, t_ ) := _μ_ ( _d, t_ ) _− μ_ (0 _,_ 0) = E [ _Yig_ ( _d, π_ ( _d|t_ )) _− Yig_ (0 _,_ 0)] _._

Condition (i) above states that, for a given ( _d, π_ ), potential outcome moments do not vary across clusters, whereas condition (ii) formalizes the requirement that potential outcome moments are invariant to perturbations in the proportion of treated peers generated by the variation in cluster sizes, that is, the function is locally flat. Intuitively, in the example from the previous paragraph, this condition implies for instance that E[ _Yig_ (0 _,_ 0 _._ 533)] = E[ _Yig_ (0 _,_ 0 _._ 526)]. While this second condition may be unlikely to hold exactly in practice, it can be a reasonable approximation when _πg_ ( _d|t_ ) shows little variation across _g_ for each ( _d, t_ ) (which happens for example when clusters are not very small) and/or the function _μ_ ̃<sup>_l_</sup> ( _d, π_ ) is relatively flat around relevant values of _π_ . Under these conditions, Corollary 1 can be applied to estimate direct and spillover effects by OLS and to conduct power calculations.

19
# **4 Estimating Spillovers in Tax Compliance**

## **4.1 Background**

There is a large literature on nudges and tax compliance (Antinyan and Asatryan, 2024), but there is relatively scant evidence on the social interaction effects behind these interventions. We designed and implemented an intervention based on the framework presented in the previous sections to illustrate its potential to capture social interaction effects in tax compliance.

The intervention took place in a large municipality of Argentina where dwellings are billed and required to pay a municipal property tax on a monthly basis (the _Tasa por Servicios Generales_ ). The treatment consisted of a one-page personalized letter with information on the current billing period, past due debt, and how to pay online or in person.<sup>8</sup> The randomized treatment assignment was conducted in two stages—first at the street block level (clusters), and then at the taxpayer account/dwelling level (units). In the first stage, we randomly divided blocks into four categories with different intensity of treatment, as depicted in Figures A.2 and A.3: (1) pure control blocks where no accounts were treated, (2) blocks with 20% of the accounts treated, (3) blocks with 50% of the accounts treated, and (4) blocks with 80% of the accounts treated. These different treatment intensities were designed to assess whether spillovers depend on the saturation of our information campaign at the block level (namely, low, medium, and high saturation levels).<sup>9</sup> In the second stage, we randomly selected accounts within the latter three groups of blocks according to their treatment saturation to receive the letter. The experiment was run on residential dwellings present in the municipality in 2019. The timeline of the intervention is displayed in Figure A.4. The letters were delivered between September 28th and October 7th, 2020, corresponding to payments due on October 9th, 2020, as well as past due debt (if any).

## **4.2 Administrative Data**

We use a combination of administrative databases provided by the revenue agency of the municipality where the experiment took place. The main database is constructed from the monthly bills issued to account holders between January 2018 and December 2020. The unit of observation is an account ( _cuenta_ ), which coincides with a dwelling unit. The data contain the following billing details and demographic characteristics of the account holder ( _titular_ ): account number (unique ID), address, block number, name of locality (neighborhood), year and month of the bill (12 bills per year), monthly fee (in pesos), paid fee

> 8Figure A.1 in the appendix provides an anonymized example of the intervention letter. Our simple design emphasized actionrelevant information, in accordance with De Neve et al. (2021) who show that simplified tax letters are an effective way to increase tax compliance.

> 9The choice _p_ 1 = 20%, _p_ 2 = 50% and _p_ 3 = 80% attempts to balance parsimony with the flexibility to detect nonlinearities in direct and spillover effects without having to estimate too many parameters. See Section 3.4 for further discussion.

20
(amount in pesos), due date, date of payment, days overdue, means of payment (cash or electronic), type of account (residential, retail store, factory), gender of the account holder, age of the account holder, linear front meters of the lot/property, assessed value of the property.

The municipality authorities required us to target blocks with eight to fifty accounts, neither very sparse nor very dense, which was the target for their mailing campaign. Figure 1 shows the distribution of accounts per block. Table A2 shows some descriptive statistics for the year 2019. Our sample size consists of 68,808 accounts distributed in 3,982 blocks. The frequency of payments is highly polarized. About 45 percent of the accounts paid the twelve 2019 monthly bills, and about 35 percent did not pay any bill at all.<sup>10</sup> We call these two core groups _always payers_ and _never payers_ , respectively. The proportion of always payers is relatively low (45 percent) and, therefore, leaves room for potential behavioral responses from non-compliant and partially-compliant neighbors, and this was compounded by the context of the pandemic, during which lockdown measures reduced payments even from highly compliant individuals.

For the randomization, power calculations, and simulations, we use baseline data from the year 2019. We rely on three different pre-treatment outcomes: (i) an indicator equal to 1 if the account paid the twelve monthly bills of 2019, (ii) an indicator equal to 1 if the account paid at least one bill in 2019, and (iii) an indicator equal to 1 if the account paid six bills or more in 2019.

# **4.3 Experimental Design and MDEs**

Following the notation in Section 3, the block-level treatment indicator is denoted by _Tg ∈{_ 0 _,_ 1 _,_ 2 _,_ 3 _}_ with distribution P[ _Tg_ = _t_ ] = _qt_ for _t_ = 0 _,_ 1 _,_ 2 _,_ 3 where _Tg_ = 0 indicates the pure control blocks, _Tg_ = 1 indicates the blocks with 20% treated, _Tg_ = 2 indicates blocks with 50% treated, and _Tg_ = 3 indicates blocks with 80% treated. The account-level treatment indicator is _Dig ∈{_ 0 _,_ 1 _}_ .

We use an independent within-cluster treatment assignment and constant within-cluster treatment probabilities. In the absence of data from a pilot experiment, we assume equal moments across assignments _σ_<sup>2=</sup><sup>_σ_2</sup><sup>_μg_(</sup><sup>_d, t_)</sup><sup>_−μn_(</sup><sup>_d, t_)=</sup><sup>_μg_(0</sup><sup>_,_0)</sup><sup>_−μn_(0</sup><sup>_,_0)and</sup><sup>_ρg_(</sup><sup>_d, t_)=</sup><sup>_ρg_(0</sup><sup>_,_0)forall</sup><sup>_g, d, t_.We</sup> _g_<sup>(</sup><sup>_d, t_)</sup> _g_<sup>(0</sup><sup>_,_0),</sup> further assume that the intracluster correlation is constant across clusters. We then impute all these magnitudes based on our baseline data. The parameters of interest are the difference in means between treated or untreated units in each treated group and the pure control units, _βn_ ( _d, t_ ) for _d_ = 0 _,_ 1 and _t_ = 1 _,_ 2 _,_ 3.

The municipality authorities requested that the total number of letters sent be set to _L_ = 25 _,_ 061. To incorporate this constraint into the choice of the saturation probabilities _qt_ , we set up a system of equations as follows. The expected number of treated units is _n_ 1 = _n_ (0 _._ 2 _q_ 1 + 0 _._ 5 _q_ 2 + 0 _._ 8 _q_ 3). Since the assignments _Tg_ = 1 and _Tg_ = 3 can be seen as symmetric, we set _q_ 1 = _q_ 3. Finally, we add an equation that ensures that the variance of the effect at 50% saturation is equal to the variance for the “small” cells (treated units in 20% clusters and untreated units in 80% clusters), so that V[ _β_<sup>ˆ</sup> ( _d,_ 2)] = V[ _β_<sup>ˆ</sup> (0 _,_ 3)] = V[ _β_<sup>ˆ</sup> (1 _,_ 1)]. This 10 For the full distribution, see Figure B.15.

21
gives a third equation of the form _q_ 2 = _Rq_ 3 where _R_ is a constant obtained from our variance formulas. Our system of equations consists of four equations: (i) _L_ = _n_ (0 _._ 2 _q_ 1 + 0 _._ 5 _q_ 2 + 0 _._ 8 _q_ 3), (ii) _q_ 1 = _q_ 3, (iii) _q_ 2 = _Rq_ 3 and (iv). We use the results in Theorem 1 to approximate the variances and calculate the ratio _R_ .

For our MDE calculations, and to illustrate our methods, we consider three scenarios: one with “substantial” heterogeneity (scenario 1), one with “moderate” heterogeneity (scenario 2), and one with “limited” heterogeneity (scenario 3). The first one uses our raw data set that contains all clusters with eight households or more. This raw data contains one cluster with a very large number of units. This is the scenario where cluster heterogeneity is most substantial. The second scenario considers an intermediate case where we drop all clusters with more than 500 units. This second data set still exhibits substantial heterogeneity but eliminates one extreme outlier. Finally, the third scenario considers clusters of size between eight and 50, which is the sample we use in our experiment. While scenarios 1 and 2 are not used in our empirical analysis, we use them to illustrate how ignoring cluster heterogeneity can result in severely underpowered experiments.

These three scenarios are described in Table 1. In scenario 1, the average cluster size is around 21 households, but the data set contains one very large outlier with 2,754 units. This large cluster makes the standard deviation of cluster sizes very large, indeed larger than the average size, so our theoretical results indicate that the adjustment for heterogeneity will make a substantial difference when calculating power and MDEs. In scenario 2, we remove this outlier from the data and this reduces the standard deviation of cluster size, slightly below but very close to the average cluster size. Finally, in scenario 3, while cluster sizes are still heterogeneous and range from eight to 50, the standard deviation is about half the average size. Thus, we may expect the power and MDE adjustment for cluster heterogeneity to be less sizeable in this case. In each scenario, we consider the results obtained with our general formula from Theorem 1 (“Het”), the formulas that rule out between-cluster moment heterogeneity from Corollary 1 (“Homog”) and the formulas that assume homogeneous, equally sized clusters (“Equal”). We emphasize that this last case imposes an incorrect assumption in all three scenarios, as cluster sizes are not homogeneous in our sample.

Table 1 shows the cluster assignment probabilities and MDEs for the binary outcomes of interest. We refer to the corresponding MDEs for the parameters _βn_ (0 _,_ 1), _βn_ (0 _,_ 2) and _βn_ (0 _,_ 3) as _MDE_ 1, _MDE_ 2 and _MDE_ 3, respectively (the MDEs for _βn_ (1 _, t_ ) are symmetric and therefore not reported). Our calculations reveal that in scenario 1, the MDEs vary dramatically and range from 0.02 to almost 0.15 depending on the assumptions one is willing to make about cluster heterogeneity. In scenario 2, the difference in MDEs is less pronounced but still substantial: the MDEs under full heterogeneity are about twice as large as the case with equally-sized and homogeneous clusters. Reassuringly, in scenario 3, the one that we use in our experiment, the MDEs are much more robust to the different assumptions about cluster heterogeneity, although ignoring heterogeneity may still result in MDEs that are about 30% smaller than the ones that account for it.

22
For comparison, we repeat these MDE calculations using the optimal cluster assignment probabilities from Theorem 2 to assess the extent to which the constraint in the number of treated units affects the power of our experiment. The results are shown in the bottom panels of Table 1. These calculations reveal that our results are very robust: using the optimal cluster assignment probabilities instead of the constrained probabilities would give different proportions of clusters in each saturation, but similar MDEs. The final sample sizes for our experiment are shown in Table A1. We assign 1,102 cluster to pure control, 1,100 clusters to the 20% and 80% saturation and 680 clusters to the 50% saturation. Note that the 20% and 80% saturations are “oversampled” relative to the 50% saturation because they contain small cells (20% treated and 20% untreated, respectively).

# **4.4 Empirical Results**

## **4.4.1 Direct and Spillover Effects on the Treated Tax Bill**

We begin the empirical analysis by estimating direct and spillover effects on payments of the October 2020 property tax bill. The due date was October 9th, and the letters were delivered between September 28th and October 7th. We show graphical evidence of the causal effect of the intervention in Figure 3 and summarize the point estimates in Table 2.<sup>11</sup> Figure 3 presents the coefficients and 95% confidence intervals from a saturated regression that estimates the day by day difference in payment rates between treated and untreated units relative to accounts in pure control blocks.<sup>12</sup> The figure focuses on high-saturation blocks with 80% treated units. The complete analysis with the three saturation groups is presented in Figure B.8 and Table 2.

The top left panel of Figure 3 reveals a clear positive effect of the intervention on tax compliance of treated accounts. The payment rate of treated units started to diverge from the pure control group as soon as the intervention began. This treatment effect reached a magnitude of about 4.5 percentage points exactly by the due date of the current billing period, and it stayed relatively constant thereafter.<sup>13</sup> The top right panel of Figure 3 shows a clear spillover effect of the intervention on untreated accounts in high-saturation blocks. It is smaller in size than the main direct effect but still substantial. The payment rates increase by about 1.1 percentage points, and the effect is statistically significant in the early days of the intervention, losing significance from the due date onward for the full sample.

Table 2 presents the direct and spillover effect coefficients for the three saturation groups. Panels A, B, and C display the effects in blocks where 80%, 50%, and 20% of accounts were treated, respectively.

> 11Appendix Section A.3 confirms that our groups are balanced and comparable. Appendix Section B.2 further shows that our tax communication campaign also increased the subscriptions to receive an electronic bill by e-mail and induced taxpayers to pay past-due tax bills.

> 12The top panels of Figures B.6 and B.7 display the corresponding payment rates in levels, i.e., the cumulative share of individuals paying the October 2020 bill over time for treated, untreated, and pure control units.

> 13Appendix Section B.3 shows that pure control blocks are not (indirectly) affected by adjacent treated blocks and thus provide a valid counterfactual for our analysis.

23
The omitted category comprises pure control blocks with untreated accounts only. Columns (1) and (4) show the coefficients and block-clustered standard errors for October 2020 bill payments on two different dates: October 3 (early payments) and October 31 (includes overdue payments). As a benchmark for our treatment effects, the last row reports the average payment rate in pure control blocks at each of these dates (i.e., the constant of each regression).

The results in Table 2 indicate that in the early stage of the intervention high-saturation blocks with 80% treated accounts present statistically significant direct and spillover effects of about 1 percentage point. This is relatively large, considering that only 5.2% of neighbors in pure control blocks had paid their October 2020 bill by this date. Naturally, as time passes and more individuals pay their bills (reaching 34.4% in pure control blocks by the end of the month), small effects become harder to detect. Thus, while the spillover effect on untreated units remains unchanged in size, it statistical significance diminishes over time. Conversely, the direct effect on treated units rises to 4.5 percentage points, representing 13.2% of the payment rate in pure control blocks.<sup>14</sup>

# **4.4.2 Heterogeneous Effects**

The results from the full experimental sample presented in Section 4.4.1 revealed modest spillover effects in the high saturation group, primarily in the early days of the intervention. However, as outlined in our pre-analysis plan, treatment effects are likely to vary along a fundamental dimension, namely pretreatment tax compliance behavior. In this section, we study heterogeneous effects along this dimension by dividing the sample into blocks that exhibited average past compliance (i.e., payments) in 2019 above and below the block median.<sup>15</sup> Columns (2)-(3) and (5)-(6) of Table 2 break down the results from columns (1) and (4), respectively, into blocks with below and above median 2019 (i.e., pre-intervention) compliance. The direct effects at the end of the month are generally larger but not substantially different: for blocks with 80%, 50%, and 20% saturation, direct effects are about 5.1, 5.7, and 4.4 percentage points for street blocks above the median average compliance in 2019, compared to about 4.1, 4.8 and 5.4 for those below the median. The differences are relatively small for early payments.

The division of the sample into these two groups reveals a much starker contrast for spillover effects. As in the main analysis in column (1) of Table 2, there is a spillover effect in early payments for the 80% saturation group but only for blocks above median compliance in 2019. The middle and bottom panels of Figure 3 make this pattern all the more apparent. This effect is relatively large: 1.58 percentage points, larger in fact than the direct effect of 1.06. The end-of-month spillover effect is much larger: 2.56 percentage points, about half of the direct effect in the high-saturation group (5.09 percentage points).

> 14Figure B.9 and Table A4 validate our analysis by showing experimental balance (i.e., no effects) on the pre-intervention bill of September 2020.

> 15See Appendix Section B.4 for more details and the intuition behind this exercise.

24
# **5 Recommendations for Practice**

In this section, we outline the steps for designing partial population experiments and refer to an example of spillovers on student test scores of an education intervention, the distribution of One Laptop per Child (as in Beuermann et al. 2015) to illustrate these steps.

**Step 1.** First, the researcher needs to select the number of saturations _M_ (i.e., categories with different intensity of treatment) and the within-cluster treatment probabilities _{pg_ ( _d|t_ ) _}_ ( _g,d,t_ ) (i.e., the proportion of within-clusters treated units), as discussed in Section 3.4. The choice of _M_ could be guided by previous knowledge or by assumptions on how conditional average outcomes vary as a function of the treatment saturation. Consider, for example, the case of One Laptop per Child (OLPC)-type experiment in which each cluster is a school. Assuming spillovers are linear as a function of saturations, a pure control group of schools with untreated pupils, and two groups of schools with different degrees of intensity of treatment or saturation (low and high) would suffice. To test for non-linear spillovers as a function of saturation, the researcher should specify at least three saturation levels (low, medium, and high). Our framework does not provide specific guidance on these choices, but highlights the trade-off between the level of detail in which this function can be traced and the availability of units in each treatment assignment - i.e., there might not be enough schools or laptops to distribute to test many different saturation levels. Our power formulas quantify these trade-offs in terms of the statistical power for different designs.

**Step 2.** Use baseline data to assess the degree of cluster size heterogeneity. In the OLPC case, cluster size consists of each school’s enrollment which may vary substantially, especially across districts or geographical areas (for instance, if there are large urban and smaller rural schools in the population). In some cases, the researcher may consider excluding clear outliers to satisfy the “no cluster too large” requirement, Assumption 3(i). In the OLPC context, it may be necessary to exclude one particularly large school from the experiment. It should be kept in mind that excluding outliers generally changes the population of interest and thus affects the external validity of the estimates. In addition to accounting for variation in cluster sizes, the researcher may need to account for distributional heterogeneity, i.e., variation in outcome distributions across clusters (see Theorem 1). The variation in outcome distributions across clusters may be assessed using baseline outcome data and possibly some distributional assumptions on outcomes.

**Step 3.** Select the variance formula for the power and MDE calculations. This step may be based on the general formula in Equation (9) or on the simplified formula in Equation (11) under distributional homogeneity. These formulas may be further simplified under additional assumptions on the data-generating process or the experimental design. For instance, one may assume equal within-cluster probabilities across _g_ . Another possible simplifying assumption is homoskedasticity, _σ_<sup>2</sup> ( _d, t_ ) = _σ_<sup>2</sup> and _ρ_ ( _d, t_ ) = _ρ_ for all ( _d, t_ ), which means, for example, that the variance and intra-school correlation in student test scores are the same across treatment assignments. This assumption may be reasonable when the effects of the treatment (e.g., the effects of OLPC on test scores) are approximately constant across units.

**Step 4.** Choose the cluster-level assignment probabilities _{qt}_<sup>_M_</sup> _t_ =0<sup>—thatis,whatproportionofclusters</sup>

25
to assign to each of the saturation levels defined in point 1 above. These probabilities can be chosen using Theorem 2 when the goal is to minimize a weighted average of estimators variances, incorporating ad-hoc constraints as in Section 4.3, or based on another optimization criteria (and possibly numerical methods) as discussed in Section 3.4. One common ad-hoc constraint is having a fixed number of treated units. In this case, researchers may rely on a system of equations as in Section 4.3. For example, in the OLPC RCT example, the government may have mandated the distribution of exactly 10,000 laptops for the experiment. In that case, if we have two saturation groups of 25% and 75% treated pupils within a school (i.e., clusters), and the saturation probabilities are _q_ 1 and _q_ 2, respectively, researchers should use an equation that represents the treatment units as 10 _,_ 000 = _n_ (0 _._ 25 _q_ 1 + 0 _._ 75 _q_ 2), where _n_ is the total number of pupils. Another condition may, for example, equalize the variance of the estimators in the “small” ˆ ˆ cells (i.e., treated units in 25% and untreated units in 75% class), that is: V î _β_ (1 _,_ 1)ó = V î _β_ (0 _,_ 2)ó. See Section 4.3.

**Step 5.** Use the power formula in Equation (10) together with the variance formula chosen in step 3 and the cluster probabilities in step 4 to calculate power and/or MDEs.

It should be noted that our framework encompasses several other common settings that are particular cases of partial population experiments. For example, our formulas can be used for designing clustered RCTs, such as an intervention where all students in treated schools receive an OLPC laptop. Finally, we note that our results allow for general between-cluster heterogeneity but assume that outcome distributions are homogeneous within each cluster. We leave the generalization of our framework to within-cluster heterogeneity for future work.

# **6 Conclusion**

We provide a general framework to analyze and design partial population experiments with heterogeneous clusters. We derive an asymptotic approximation and variance formulas for general clustered experimental designs, allowing for multiple treatment intensities, general forms of intracluster correlation, and two sources of cluster heterogeneity: heterogeneity in cluster sizes and distributional heterogeneity. We then apply our results to analyze inference and to conduct power and MDE calculations in partial population experiments, and derive formulas for optimal group-level assignment probabilities. Our formulas are easy to adapt to other experimental designs.

We estimate total and neighborhood spillover effects of a randomized communication campaign on property tax compliance in a large municipality of Argentina where neighbors must pay a monthly bill on their real estate. We estimate direct effects on monthly payments and analyze whether the campaign creates spillover effects on neighbors who live nearby within a treated block but who do not receive a letter. We find evidence of direct and spillover effects on property tax payment rates. Our results reveal higher payment rates of treated and untreated accounts relative to neighbors in pure control blocks where nobody 26
received the communication letter. We find that spillover effects are stronger in blocks that exhibited a higher degree of tax compliance in the pre-treatment period. This application showcases the usefulness of our methodological framework for designing partial population experiments.

# **References**

- **Abadie, Alberto, Susan Athey, Guido W Imbens, and Jeffrey M Wooldridge.** 2022. “When Should You Adjust Standard Errors for Clustering?” _Quarterly Journal of Economics_ , 138(1): 1–35.

- **Angelucci, Manuela, and Giacomo De Giorgi.** 2009. “Indirect Effects of an Aid Program: How Do Cash Transfers Affect Ineligibles’ Consumption?” _American Economic Review_ , 99(1): 486–508.

- **Antinyan, Armenak, and Zareh Asatryan.** 2024. “Nudging for Tax Compliance: a Meta-Analysis*.” _The Economic Journal_ , ueae088.

- **Athey, Susan, Dean Eckles, and Guido W. Imbens.** 2018. “Exact P-values for Network Interference.” _Journal of the American Statistical Association_ , 113(521): 230–240.

- **Baird, Sarah, Aislinn Bohren, Craig McIntosh, and Berk Ozler.**<sup>** ̈**</sup> 2018. “Optimal Design of Experiments in the Presence of Interference.” _The Review of Economics and Statistics_ , 100(5): 844–860.

- **Bai, Yuehao.** 2022. “Optimality of Matched-Pair Designs in Randomized Controlled Trials.” _American Economic Review_ , 112(12): 3911–3940.

- **Barrera-Osorio, Felipe, Marianne Bertrand, Leigh L. Linden, and Francisco Perez-Calle.** 2011. “Improving the Design of Conditional Transfer Programs: Evidence from a Randomized Education Experiment in Colombia.” _American Economic Journal: Applied Economics_ , 3(2): 167–195.

- **Basse, Guillaume, and Avi Feller.** 2018. “Analyzing two-stage experiments in the presence of interference.” _Journal of the American Statistical Association_ , 113(521): 41–55.

- **Basse, G W, A Feller, and P Toulis.** 2019. “Randomization tests of causal effects under interference.” _Biometrika_ , 106(2): 487–494.

- **Berger, Martijn P.F., and Weng-Kee Wong.** 2009. _An Introduction to Optimal Designs for Social and Biomedical Research._ Wiley.

- **Bergeron, Augustin, Gabriel Tourek, and Jonathan Weigel.** 2024. “The State Capacity Ceiling on Tax Rates: Evidence from Randomized Tax Abatements in the DRC.” _NBER working paper_ .

## **Beuermann, Diether W., Julian Cristia, Santiago Cueto, Ofer Malamud, and Yyannu Cruz-Aguayo.**

2015. “One Laptop per Child at Home: Short-Term Impacts from a Randomized Experiment in Peru.” _American Economic Journal: Applied Economics_ , 7(2): 53–80.

27
**Bloom, Howard S.** 2005. “Randomizing Groups to Evaluate Place-Based Programs.” _Learning More from Social Experiments: Evolving Analytic Approaches_ , 115–172. Russell Sage Foundation.

**Boning, William C., John Guyton, Ronald Hodge, and Joel Slemrod.** 2020. “Heard it through the grapevine: The direct and network effects of a tax enforcement field experiment on firms.” _Journal of Public Economics_ , 190(C).

- **Brockmeyer, A, A Estefan, K Ramirez Arras, and J.C. Suarez Serrato.** 2020. “Taxing Property in Developing Countries: Theory and Evidence from Mexico.” _IFS Working Paper_ .

**Bruhn, Miriam, and David McKenzie.** 2009. “In Pursuit of Balance: Randomization in Practice in Development Field Experiments.” _American Economic Journal: Applied Economics_ , 1(4): 200–232.

- **Bugni, Federico A., Ivan A. Canay, and Azeem M. Shaikh.** 2018. “Inference under Covariate-Adaptive Randomization.” _Journal of the American Statistical Association_ , 113(524): 1784–1796.

- **Bugni, Federico A, Ivan A. Canay, and Azeem M. Shaikh.** 2019. “Inference under Covariate-Adaptive Randomization with Multiple Treatments.” _Quantitative Economics_ , 10(4): 1747–1785.

- **Bugni, Federico A., Ivan A. Canay, Azeem M. Shaikh, and Max Tabord-Meehan.** 2023. “Inference for Cluster Randomized Experiments with Non-ignorable Cluster Sizes.” _arXiv:2204.08356_ .

- **Carrillo, Paul E., Edgar Castro, and Carlos Scartascini.** 2021. “Public good provision and property tax compliance: Evidence from a natural experiment.” _Journal of Public Economics_ , 198: 104422.

- **Carter, Andrew V., Kevin T. Schnepel, and Douglas G. Steigerwald.** 2017. “Asymptotic Behavior of a t-Test Robust to Cluster Heterogeneity.” _Review of Economics and Statistics_ , 99(4): 698–709.

- **Chiang, Harold, Yuya Sasaki, and Yulong Wang.** 2023. “On the Inconsistency of Cluster-Robust Inference and How Subsampling Can Fix It.” _arXiv:2308.10138_ .

- **Cr ́epon, Bruno, Esther Duflo, Marc Gurgand, Roland Rathelot, and Philippe Zamora.** 2013. “Do Labor Market Policies have Displacement Effects? Evidence from a Clustered Randomized Experiment.” _Quarterly Journal of Economics_ , 128(2): 531–580.

# **De Neve, Jan-Emmanuel, Cl ́ement Imbert, Johannes Spinnewijn, Teodora Tsankova, and Maarten**

- **Luts.** 2021. “How to Improve Tax Compliance? Evidence from Population-Wide Experiments in Belgium.” _Journal of Political Economy_ , 129(5): 1425–1463.

- **Djogbenou, Antoine A., James G. MacKinnon, and Morten Ørregaard Nielsen.** 2019. “Asymptotic theory and wild bootstrap inference with clustered errors.” _Journal of Econometrics_ , 212(2): 393–412.

- **Drago, Francesco, Friederike Mengel, and Christian Traxler.** 2020. “Compliance Behavior in Networks: Evidence from a Field Experiment.” _American Economic Journal: Applied Economics_ , 12(2): 96–133.

28
- **Duflo, Esther, and Emmanuel Saez.** 2003. “The Role of Information and Social Interactions in Retirement Plan Decisions: Evidence from a Randomized Experiment.” _Quarterly Journal of Economics_ , 118(3): 815–842.

- **Duflo, Esther, Rachel Glennerster, and Michael Kremer.** 2007. “Using Randomization in Development Economics Research: A Toolkit.” In _Handbook of Development Economics_ . Vol. 4 of _Handbook of Development Economics_ , , ed. T. Paul Schultz and John A. Strauss, 3895–3962. Elsevier.

- **Foos, Florian, and Eline A. de Rooij.** 2017. “All in the Family: Partisan Disagreement and Electoral Mobilization in Intimate Networks—A Spillover Experiment.” _American Journal of Political Science_ , 61(2): 289–304.

- **Gin ́e, Xavier, and Ghazala Mansuri.** 2018. “Together We Will: Experimental Evidence on Female Voting Behavior in Pakistan.” _American Economic Journal: Applied Economics_ , 10(1): 207–235.

- **Hansen, Bruce E., and Seojeong Lee.** 2019. “Asymptotic theory for clustered samples.” _Journal of Econometrics_ , 210(2): 268–290.

- **Haushofer, Johannes, and Jeremy Shapiro.** 2016. “The Short-term Impact of Unconditional Cash Transfers to the Poor: Experimental Evidence from Kenya.” _Quarterly Journal of Economics_ , 131(4): 1973– 2042.

- **Hirano, Keisuke, and Jinyong Hahn.** 2010. “Design of Randomized Experiments to Measure Social Interaction Effects.” _Economics Letters_ , 106(1): 51–53.

- **Hudgens, Michael G., and M. Elizabeth Halloran.** 2008. “Toward Causal Inference with Interference.” _Journal of the American Statistical Association_ , 103(482): 832–842.

- **Ichino, Nahomi, and Matthias Sch ̈undeln.** 2012. “Deterring or Displacing Electoral Irregularities? Spillover Effects of Observers in a Randomized Field Experiment in Ghana.” _Journal of Politics_ , 74(1): 292–307.

- **Imai, Kosuke, Gary King, and Clayton Nall.** 2009. “The Essential Role of Pair Matching in ClusterRandomized Experiments, with Application to the Mexican Universal Health Insurance Evaluation.” _Statistical science_ , 24(1): 29–53.

- **Imai, Kosuke, Zhichao Jiang, and Anup Malani.** 2021. “Causal Inference With Interference and Noncompliance in Two-Stage Randomized Experiments.” _Journal of the American Statistical Association_ , 116(534): 632–644.

- **Jiang, Zhichao, Kosuke Imai, and Anup Malani.** 2023. “Statistical Inference and Power Analysis for Direct and Spillover Effects in Two-Stage Randomized Experiments.” _Biometrics_ , 79(3): 2370–2381.

29
**Krause, Benjamin.** 2020. “Balancing Purse and Peace: Tax Collection, Public Goods and Protests.” _Mimeo_ .

- **Leung, Michael P.** 2022. “Rate-optimal cluster-randomized designs for spatial interference.” _The Annals of Statistics_ , 50(5): 3064 – 3087.

- **Liu, Jizhou.** 2023. “Inference for Two-stage Experiments under Covariate-Adaptive Randomization.” _arXiv:2301.09016_ .

**Melas, Viatcheslav B.** 2006. _Functional Approach to Optimal Experimental Design._ Springer New York.

- **Miguel, Edward, and Michael Kremer.** 2004. “Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities.” _Econometrica_ , 72(1): 159–217.

**Moffit, Robert.** 2001. “Policy Interventions, Low-level Equilibria and Social Interactions.” In _Social Dynamics_ . , ed. Stephen N. Durlauf and Peyton Young, 45–82. MIT Press.

- **Moulton, Brent R.** 1986. “Random group effects and the precision of regression estimates.” _Journal of Econometrics_ , 32(3): 385–397.

- **Pomeranz, Dina.** 2015. “No Taxation without Information : Deterrence and Self-Enforcement in the Value Added Tax.” _American Economic Review_ , 105(8): 2539–2569.

- **Pomeranz, Dina, and Jos ́e Vila-Belda.** 2019. “Taking State-Capacity Research to the Field: Insights from Collaborations with Tax Authorities.” _Annual Review of Economics_ , 11(1): 755–781.

- **Puelz, David, Guillaume Basse, Avi Feller, and Panos Toulis.** 2022. “A graph-theoretic approach to randomization tests of causal effects under general interference.” _Journal of the Royal Statistical Society: Series B_ , 84(1): 174–204.

- **Sasaki, Yuya, and Yulong Wang.** 2022. “Non-Robustness of the Cluster-Robust Inference: with a Proposal of a New Robust Method.” _arXiv:2210.16991_ .

- **Silvey, Samuel D.** 1980. _Optimal Design: An Introduction to the Theory for Parameter Estimation._ Springer Netherlands.

- **Vazquez-Bare, Gonzalo.** 2023. “Identification and Estimation of Spillover Effects in Randomized Experiments.” _Journal of Econometrics_ , 237(1): 105237.

**Viviano, Davide.** 2024. “Policy design in experiments with unknown interference.” _working paper_ .

- **Weigel, Jonathan L.** 2020. “The Participation Dividend of Taxation: How Citizens in Congo Engage More with the State When it Tries to Tax Them.” _Quarterly Journal of Economics_ , 135(4): 1849–1903.

30
# **Figures and Tables**

Figure 1: Distribution of cluster sizes in six partial population experiments <!-- Start of picture text --> 300<br>200<br>100<br>0<br>10 20 30 40 50<br>Cluster (block) size<br>Frequency<br><!-- End of picture text --> <!-- Start of picture text --> (a) This paper; n = 68 , 808; G = 3 , 982; n/G =<br>17 . 3; sd ( ng ) = 8 . 25.<br><!-- End of picture text --> <!-- Start of picture text --> 30<br>20<br>10<br>0<br>0 20 40 60 80<br>Cluster (village) size<br>Frequency<br><!-- End of picture text --> (c) Haushofer and Shapiro (2016); _n_ = 2 _,_ 880; _G_ = 123; _n/G_ = 23 _._ 4; _sd_ ( _ng_ ) = 14 _._ 8.

<!-- Start of picture text --> 50<br>40<br>30<br>20<br>10<br>0<br>0 100 200 300<br>Cluster (local agency) size<br>Frequency<br><!-- End of picture text -->

- (b) Cr ́epon et al. (2013); _n_ = 21 _,_ 431; _G_ = 235; _n/G_ = 91 _._ 2; _sd_ ( _ng_ ) = 42 _._ 2.

<!-- Start of picture text --> 6<br>4<br>2<br>0<br>0 20 40 60 80<br>Cluster (neighborhood) size<br>Frequency<br><!-- End of picture text -->

- (d) Gin ́e and Mansuri (2018); _n_ = 2 _,_ 637; _G_ = 67; _n/G_ = 39 _._ 4; _sd_ ( _ng_ ) = 16 _._ 7.

<!-- Start of picture text --> 4<br>3<br>2<br>1<br>0<br>0 10 20 30 40<br>Cluster (constituency) size<br>Frequency<br><!-- End of picture text --> (e) Ichino and Sch ̈undeln (2012); _n_ = 868; _G_ = 39; _n/G_ = 22 _._ 3; _sd_ ( _ng_ ) = 9 _._ 6.

<!-- Start of picture text --> 80<br>60<br>40<br>20<br>0<br>0 20 40 60 80 100<br>Cluster (village) size<br>Frequency<br><!-- End of picture text --> (f) Imai, Jiang and Malani (2021); _n_ = 11 _,_ 089; _G_ = 437; _n/G_ = 25 _._ 4; _sd_ ( _ng_ ) = 16 _._ 7.

_Notes_ : This figure shows the distribution of cluster sizes in six partial population experiments; _n_ denotes the total sample size; _G_ denotes the number of clusters; _n/G_ denotes the average cluster size; _sd_ ( _ng_ ) denotes the standard deviation of cluster sizes. Average values across studies are _n_ = 7 _,_ 781; _G_ = 180; _n/G_ = 40 _._ 4; _sd_ ( _ng_ ) = 20. Median values across studies are _n_ = 2 _,_ 880; _G_ = 123; _n/G_ = 25 _._ 6; _sd_ ( _ng_ ) = 16 _._ 7. The data source for Cr ́epon et al. (2013) is: DARES (2010) “Enquˆete aupr`es des jeunes ́eligibles `a la prestation d’insertion jeunes diplˆom ́es,” Progedo-Adisp. doi:10.13144/lil-1596.

31
Figure 2: Power functions - numerical illustration <!-- Start of picture text --> power = 0.80<br>power = 0.69<br>power = 0.48<br>MDE = 0.29<br>−0.4 −0.2 0.0 0.2 0.4<br>Difference in means<br>1.0<br>0.8<br>0.6<br>Power<br>0.4<br>0.2<br>0.0<br><!-- End of picture text --> _Notes_ : This figure illustrates how ignoring heterogeneity can result in severely underpowered experiments. We consider the simple setting of a cluster RCT with a few “large” clusters and variation in the distribution of outcomes across clusters. We assume 200 clusters, with 10 clusters containing 100 units each and the remaining 190 clusters containing 25 units each. The figure plots three power functions corresponding to different variance formulas: the short-dashed curve depicts the power function for the variance formula that accounts for clustering assuming equally-sized clusters. The long-dashed curve depicts the power function using a variance formula that accounts for variation in cluster sizes. The solid curve depicts the power function using a variance formula that accounts for heterogeneity in both cluster sizes and outcome distributions. Given this sample size, the MDE at 80% power, ignoring cluster heterogeneity, is 0.29. Accounting for cluster size heterogeneity decreases the power to detect an effect of 0.29 from 80% to 69%. Accounting for both sources of heterogeneity decreases the power further to 48%.

32
Figure 3: Direct and spillover effects on property tax payments in high-saturation blocks

**Direct Effects Spillover Effects Treated vs. Pure Control Untreated vs. Pure Control**

<!-- Start of picture text --> Intervention Intervention<br>7 begins Due date of 7 begins Due date of<br>6 Oct'20 bill 6 Oct'20 bill<br>5<br>5<br>4<br>4<br>3<br>3 2<br>2 1<br>1 0<br>0<br>25sep2020 02oct2020 09oct2020 16oct2020 23oct2020 30oct2020 25sep2020 02oct2020 09oct2020 16oct2020 23oct2020 30oct2020<br>Calendar date Calendar date<br>Above Median Compliance<br>Intervention<br>7 Intervention 7 begins Due date of<br>begins Due date of Oct'20 bill<br>Oct'20 bill 6<br>6<br>5<br>5<br>4<br>4<br>3<br>3<br>2<br>2<br>1<br>1<br>0<br>0<br>25sep2020 02oct2020 09oct2020 16oct2020 23oct2020 30oct2020 25sep2020 02oct2020 09oct2020 16oct2020 23oct2020 30oct2020<br>Calendar date Calendar date<br>Below Median Compliance<br>Intervention Intervention<br>7 begins Due date of 7 begins Due date of<br>Oct'20 bill 6 Oct'20 bill<br>6<br>5<br>5 4<br>3<br>4<br>2<br>3 1<br>0<br>2<br>1<br>0<br>25sep2020 02oct2020 09oct2020 16oct2020 23oct2020 30oct2020 25sep2020 02oct2020 09oct2020 16oct2020 23oct2020 30oct2020<br>Calendar date Calendar date<br>Treatment effect (p.p.) Treatment effect (p.p.)<br>Treatment effect (p.p.) Treatment effect (p.p.)<br>Treatment effect (p.p.) Treatment effect (p.p.)<br><!-- End of picture text --> _Notes_ : These figures show the coefficients and 95% confidence intervals from a saturated regression that computes, at each calendar day, the payment rate difference between treated and untreated groups relative to the pure control group (i.e., blocks where no accounts were treated). We focus the attention on blocks where 80% of the units were treated. The three left figures exhibit the direct effects on treated accounts, and the three right figures present the spillover effects on untreated accounts. The top panel includes all the observations in high-saturation blocks, the middle panel focuses on blocks with baseline compliance above the median, and the bottom panel focuses on blocks with baseline compliance below the median. We define compliance as the share of bills paid by block in 2019. The median compliance is 0.56 (see Figure B.15). Standard errors are clustered by block. The first vertical bar shows the due date for the September 2020 bill. This corresponds to a bill issued and due for payment before our intervention began, thus serving as a placebo. The second vertical bar indicates the start of the intervention. The letters were delivered between September 28th and October 7th.

33
Table 1: MDEs with constrained and optimal choice of cluster probabilities

|||**Scenario**|**1**||**Scenario**|**2**||**Scenario**|**3**|
|---|---|---|---|---|---|---|---|---|---|
||Het|Homog|Equal|Het|Homog|Equal|Het|Homog|Equal|
|**Restricted**_qt_||||||||||
|_q_0|0_._408|0_._408|0_._408|0_._388|0_._388|0_._388|0_._272|0_._272|0_._272|
|_q_1|0_._199|0_._209|0_._230|0_._219|0_._231|0_._239|0_._273|0_._283|0_._286|
|_q_2|0_._194|0_._173|0_._131|0_._173|0_._149|0_._135|0_._182|0_._162|0_._157|
|_q_3|0_._199|0_._209|0_._230|0_._219|0_._231|0_._239|0_._273|0_._283|0_._286|
|**MDEs**||||||||||
|_MDE_1|0_._145|0_._047|0_._022|0_._042|0_._027|0_._022|0_._033|0_._025|0_._023|
|_MDE_2|0_._146|0_._051|0_._029|0_._047|0_._033|0_._029|0_._039|0_._032|0_._030|
|_MDE_3|0_._146|0_._051|0_._030|0_._047|0_._034|0_._030|0_._040|0_._033|0_._032|
|**Optimal**_qt_||||||||||
|_q_0|0_._364|0_._352|0_._314|0_._348|0_._328|0_._313|0_._332|0_._315|0_._309|
|_q_1|0_._212|0_._219|0_._238|0_._221|0_._231|0_._238|0_._229|0_._237|0_._240|
|_q_2|0_._211|0_._211|0_._210|0_._210|0_._210|0_._210|0_._210|0_._210|0_._210|
|_q_3|0_._212|0_._219|0_._238|0_._221|0_._231|0_._238|0_._229|0_._237|0_._240|
|**MDEs**||||||||||
|_MDE_1|0_._145|0_._047|0_._023|0_._043|0_._028|0_._023|0_._033|0_._026|0_._024|
|_MDE_2|0_._145|0_._049|0_._025|0_._045|0_._030|0_._025|0_._036|0_._028|0_._027|
|_MDE_3|0_._146|0_._051|0_._030|0_._047|0_._034|0_._030|0_._040|0_._033|0_._032|
|**Scenario Statistics**||||||||||
|_n_||84,175|||81,961|||68,808||
|_G_||4,139|||4,138|||3,982||
|min_ng_||8|||8|||8||
|max_ng_||2,754|||376|||50||
|mean(_ng_)||20.5|||19.8|||17.3||
|_sd_(_ng_)||45.9|||17.5|||8.3||

_Notes_ : This table shows the cluster assignment probabilities and MDEs for the binary outcomes of interest. The parameters of interest are the difference in means between untreated units in each treated group and the pure control units, _βn_ (0 _, t_ ), with _t_ = 1 _,_ 2 _,_ 3 indicating the groups with 20%, 50%, and 80% treated units, respectively. We refer to the corresponding MDEs for the parameters _βn_ (0 _,_ 1), _βn_ (0 _,_ 2) and _βn_ (0 _,_ 3) as _MDE_ 1, _MDE_ 2 and _MDE_ 3, respectively. Scenario 1 exhibits “substantial” heterogeneity, scenario 2 has “moderate” heterogeneity, and scenario 3 presents “limited” heterogeneity. _n_ denotes the sample size; _G_ denotes the number of clusters; min _ng_ and max _ng_ show the smallest and largest cluster; _mean_ ( _ng_ ) is the average cluster size; _sd_ ( _ng_ ) is the standard deviation of cluster sizes. In each scenario, we consider the results obtained with our general formula from Theorem 1 (“Het”), the formulas that rule out between-cluster moment heterogeneity from Corollary 1 (“Homog”) and the formulas that assume homogeneous, equally-sized clusters (“Equal”). Panels 1 and 2 show the constrained and optimal cluster assignment probabilities and their corresponding MDEs. Panels 3 and 4 show the optimal cluster assignment probabilities and their corresponding MDEs.

34
Table 2: Direct and spillover effects on property tax payments

||**E**|**arly Paymen**|**ts**|**On-tim**|**e & Late Pa**|**yments**|
|---|---|---|---|---|---|---|
||All|Below<br>Median|Above<br>Median|All|Below<br>Median|Above<br>Median|
||(1)|(2)|(3)|(4)|(5)|(6)|
|**_A. Blocks with 80% treated_**|||||||
|Treated|0.96***|0.86**|1.06**|4.55***|4.12***|5.09***|
||(0.28)|(0.34)|(0.42)|(0.74)|(0.79)|(0.81)|
|Untreated|1.10**|0.55|1.58**|0.79|-1.25|2.56**|
||(0.43)|(0.50)|(0.67)|(1.01)|(1.16)|(1.27)|
|**_B. Blocks with 50% treated_**|||||||
|Treated|1.07***|1.24**|1.02|4.87***|4.81***|5.67***|
||(0.41)|(0.50)|(0.62)|(0.93)|(1.07)|(1.08)|
|Untreated|-0.02|0.10|-0.03|-0.10|1.34|-0.76|
||(0.34)|(0.43)|(0.50)|(0.91)|(1.00)|(1.14)|
|**_C. Blocks with 20% treated_**|||||||
|Treated|0.69*|0.85*|0.52|4.97***|5.41***|4.40***|
||(0.42)|(0.52)|(0.63)|(0.99)|(1.21)|(1.27)|
|Untreated|0.11|0.68**|-0.42|-0.18|0.61|-1.09|
||(0.26)|(0.33)|(0.38)|(0.72)|(0.77)|(0.82)|
|Payment rate of pure control|5.15|3.63|6.49|34.37|23.53|43.91|
|Observations|68,806|32,361|36,445|68,806|32,361|36,445|
|Number of clusters (blocks)|3,981|2,013|1,968|3,981|2,013|1,968|

_Notes_ : This table shows the results from saturated OLS regressions. The dependent variable in columns (1)-(3) is an indicator for paying the October 2020 bill by October 3rd (early payments); while in columns (4)-(6) we use an indicator for paying the October 2020 bill by October 31st (includes early, on time, and overdue payments). Columns (2) and (3) break the main result from column (1) into blocks below and above median compliance in 2019. We define compliance as the share of bills paid by block in 2019 with median value of 0.56 (see Figure B.15). Each column corresponds to a separate regression. The omitted category corresponds to blocks where no accounts were treated (pure control). Panel A shows the results for blocks where 80% were treated, panel B for blocks with 50% treated, and panel C for blocks with 20% treated. The letters were delivered between September 28th and October 7th. The due date for the October 2020 bill was October 9th. The row _Payment rate of pure control_ displays the constant of each regression, corresponding to the average payment rate in blocks with no treated units. Standard errors clustered by blocks are reported in parentheses. * p _<_ 0.10, ** p _<_ 0.05, *** p _<_ 0.01 35
# Supplementary Materials for:

“Design of Partial Population Experiments with an Application to Spillovers in Tax Compliance” A-1
<!-- Start of picture text --> éCOMO PAGAR?<br>Ingresando a tasas.tresdefebrero.gov.ar completa los datos:<br>DESCARGATASAServicios OIMPUESTO generaesA BACAR0 PAGA TU BOLETA . 1)) PodésSpasmercadopagarPago ONLINE —p conEnnuestra web.el momento desde<br>pago electrénico para<br>pagar desde la plataforma<br>de tu banco 0 cajero<br>automatico.<br>© _RECIBIR LA BOLETA POR MAIL<br>2) Podés pagaren EFECTIVO en<br>CLICKEApavey recibilaESTE todosBOTON ese los & rapipago —P DESCARGALADESCARGALA olevatuDIE)<br>También podés entrar a<br>miboleta.tresdefebrero.gov.ar<br><!-- End of picture text -->
<!-- Start of picture text --> 0%<br>——— 20%<br>50% ie<br>80% 7, faPitta tone [S,ee<br>4 hie ave)<br>——— Not assigned orDogmeiS im me AS#)<br>Paps pee Ape a ied a eat OROOY _ctep ae DS<br>eG AA et rest ah = Lary Mette? or COS IEEE apr jis was.<br>heal = ackias Be teFit te yn ao a4 SOE CES ir ites EPS rS<br>borisoes PTT TELGeecee iaHIaEoy rereRHyi”= BOSpak, BASEaesTS EyTE toPRED,S23<br>asTAD Ty 4 ay A Ss Fe ENEEe ASSoe %% TH eeet, FX Ll ILE<br>FpNia gsack DesMDVaasa gralLOSOSEBSRSISLESOEIIISSON O MGhd pimyYs MOTLeery LEG odoORerosNTT LLTDeee<br>VEE<br>oF KEELEYLI GEG<br>Fo)<br><!-- End of picture text -->
Figure A.3: A Partial Population Design <!-- Start of picture text --> Sample of clusters<br>q 0 q 1 q 2 q 3<br>Pure control 20% treated 50% treated 80% treated<br>0.8 0.2 0.5 0.5 0.2 0.8<br>C C T C T C T<br><!-- End of picture text --> _Notes_ : In a partial population design, clusters are first randomly assigned to different treatment intensities or saturations. Within each cluster, units are randomly assigned to treatment with a probability equal to their cluster saturation. The figure above shows an example of a partial population design with four saturations, including pure control clusters with no treated units.

Figure A.4: Timeline of the randomized communication campaign 25,000 letters delivered

|September 28|October 7|October 9|Timeline|
|---|---|---|---|
|**First day**|**Last day**|**October 2020**|2020|
|**of campaign**|**of campaign**|**bill is due**||

# **A.2 Sample Sizes and Descriptive Statistics**

Table A1: Sample sizes

|||Blocks|Control Obs|Treated Obs|
|---|---|---|---|---|
|_Tg_ =|0<br>Pure control|1_,_102|19_,_103|0|
|_Tg_ =|1<br>20% treated|1_,_100|15_,_060|3_,_853|
|_Tg_ =|2<br>50% treated|680|5_,_905|5_,_897|
|_Tg_ =|3<br>80% treated|1_,_100|3_,_677|15_,_311|
|Total||3_,_982|43_,_745|25_,_061|

_Notes_ : This table shows the final sample sizes used in our experiment. We limit the analysis to clusters of size ranging between 8 and 50 property tax accounts per street-block.

A-4
Table A2: Descriptive statistics in 2019 (baseline year)

||Blocks|Obs|Mean|SD|ICC|
|---|---|---|---|---|---|
|Paid the twelve bills in 2019|3_,_981|68_,_808|0_._449|0_._497|0_._062|
|Paid at least one bill in 2019|3_,_981|68_,_808|0_._650|0_._477|0_._071|
|Paid six bills or more in 2019|3_,_981|68_,_808|0_._572|0_._495|0_._073|

_Notes_ : This table shows descriptive statistics about the frequency of payments in 2019. This is the baseline year we used for the randomization, power calculations, and simulations. The data set is restricted to blocks with size between 8 and 50 accounts. Figure 1 shows the distribution of accounts per block. Our sample size consists of 68,808 accounts distributed in 3,982 blocks. The frequency of payments is very polarized. About 45 percent of the accounts paid the twelve bills and about 35 percent did not pay any bill. We call these two core groups _always payers_ and _never payers_ , respectively. The perfect compliance rate of 45 percent is presumably low and, therefore, leaves room for potential behavioral responses from non-compliant and partially-compliant neighbors.

A-5
Figure A.5: Distribution of payment date for treated, untreated, and pure control (October 2020 billing period) (a) Treated vs. Pure Control <!-- Start of picture text --> .15<br>Treated<br>Due date of<br>October bill Pure Control<br>.1<br>.05<br>0<br>30sep2020 14oct2020 28oct2020 11nov2020 25nov2020 09dec2020 23dec2020<br>(b) Untreated vs. Pure Control<br>.15<br>Untreated<br>Due date of<br>October bill Pure Control<br>.1<br>.05<br>0<br>30sep2020 14oct2020 28oct2020 11nov2020 25nov2020 09dec2020 23dec2020<br>Fraction<br>Fraction<br><!-- End of picture text --> _Notes_ : These figures show the fraction of individuals paying the October 2020 bill before and after the due date (October 9th, 2020). Panel (a) shows the distribution of payments for treated units (in blue) relative to pure control units (in red). We pool together treated units from _Tg_ = 1 _,_ 2 _,_ 3. Panel (b) shows the distribution of payments for untreated units (in blue) relative to pure control units (in red). We pool together untreated units from _Tg_ = 1 _,_ 2 _,_ 3. The area of each histogram integrates to one. A larger bar on a particular date means that the payment frequency of the corresponding group is higher than the other group.

A-6
# **A.3 Balance Checks**

We ran balance test checks to verify the comparability of the treated, untreated, and pure control groups in terms of demographic and account-related characteristics in 2019. We jointly estimate the parameters of interest through the following saturated OLS regression: where _Xig_ is one of the account holder or dwelling characteristics contained in our baseline data. We allow _εig_ to be correlated within blocks and use a cluster-robust variance estimator. In this regression, _θt_ captures the average difference of _Xig_ of untreated units in groups with _Tg_ = _t_ relative to the pure control group, and _τt_ captures the average difference of _Xig_ of treated units in groups with _Tg_ = _t_ relative to the pure control group. The results are reported in Table A3 and reassuringly confirm that our groups are highly balanced. The null effect on timely payments (i.e., excluding past-due payments) of the September 2020 bill—the bill prior to our intervention— sheds further light on the balance between groups (see Figure B.9).

A-7
Table A3: Balance test saturated regressions

||Property<br>Value|Front<br>Metres|House<br>type|Tenant<br>Male|Tenant<br>Age|Bill<br>amount|N Bills<br>paid 2019|Digital<br>payment|
|---|---|---|---|---|---|---|---|---|
||(1)|(2)|(3)|(4)|(5)|(6)|(7)|(8)|
|**_A. Blocks with 80%_**|**_treated:_**||||||||
|Treated|0.01|_−_8.27|_−_0.00|_−_0.00|_−_0.14|2.81|0.05|_−_0.00|
||(0.02)|(17.77)|(0.00)|(0.01)|(0.40)|(7.81)|(0.09)|(0.01)|
|Untreated|0.00|_−_1.76|0.00|0.00|_−_0.53|6.27|_−_0.06|_−_0.00|
||(0.02)|(20.70)|(0.01)|(0.01)|(0.53)|(12.95)|(0.12)|(0.01)|
|**_B. Blocks with 50%_**|**_treated:_**||||||||
|Treated|0.01|12.65|_−_0.00|_−_0.00|_−_0.47|1.16|0.03|0.00|
||(0.02)|(20.38)|(0.01)|(0.01)|(0.50)|(9.21)|(0.11)|(0.01)|
|Untreated|0.01|25.30|_−_0.00|_−_0.00|_−_0.42|1.88|0.02|0.01|
||(0.02)|(20.66)|(0.01)|(0.01)|(0.48)|(9.66)|(0.11)|(0.01)|
|**_C. Blocks with 20%_**|**_treated:_**||||||||
|Treated|0.02|32.57*|_−_0.01|0.01|0.10|5.94|0.07|_−_0.01|
||(0.02)|(16.79)|(0.01)|(0.01)|(0.54)|(9.55)|(0.12)|(0.01)|
|Untreated|0.02|19.14|_−_0.01|_−_0.01|0.12|1.32|0.00|0.00|
||(0.02)|(14.05)|(0.00)|(0.01)|(0.40)|(7.77)|(0.09)|(0.01)|
|Mean Pure Control|13.64|841.50|0.91|0.62|19.15|368.66|6.71|0.35|
|Observations|64,932|68,808|68,808|46,419|52,714|68,808|68,808|38,112|
|Number of clusters|3,979|3,981|3,981|3,973|3,976|3,981|3,981|3,968|

Notes: This table shows balance test regressions to formally test for differences in observable characteristics between the treatment and control groups. Each column corresponds to a separate regression (equation (12) in the text). The dependent variables in each column are: (1) the log of assessed property value; (2) the front metres of the property; (3) an indicator for the property being a house versus a house with a store; (4) whether the tenant is male; (5) a proxy for the tenant’s age (first two digits of the ID); (6) the amount paid in the bill corresponding to December 2019 (including zeroes); (7) the number of bills paid in 2019 (the maximum is 12); (8) for those who paid, whether they did so digitally. The row _Mean Pure Control_ displays the constant of each regression, corresponding to the average of the dependent variable for accounts in blocks with no treated units ( _Tg_ = 0). Missing/non-missing indicators for the dependent variables with missing observations (columns 1, 4, 5 and 8) are also balanced between groups (results not reported). Standard errors clustered by blocks are reported in parentheses. * p _<_ 0.10, ** p _<_ 0.05, *** p _<_ 0.01

# **A.4 Further Details on Within-Group Assignment Mechanisms**

**Fixed Margins.** The within-group treatment is often assigned by choosing a fixed (i.e. nonrandom) number of treated units within each group. Given _Tg_ = _t_ , suppose the researcher wants to assign a

A-8
proportion _pt_ of, or a total of _ngpt_ , units to treatment. Assigning exactly _ngpt_ units to treatment is not possible when _ngpt_ is not an integer. We propose the following procedure to deal with this issue. Define an independent binary random variable _ξg_ and let the number of treated units in cluster _g_ be: so that _ξg_ plays the role of an adjusting factor that randomly rounds the number of treated up or down. Given _Tg_ = _t_ , set the probability that _ξg_ = 1 to: This implies that, given _Tg_ = _t_ , the expected number of treated units in group _g_ is _ngpt_ and that P _g_ [ _Dig_ = 1 _|Tg_ = _t_ ] = _pt_ . Then, given _Tg_ = _t_ , the expected number of treated units in group _g_ is _ngpt_ and that P _g_ [ _Dig_ = 1 _|Tg_ = _t_ ] = _pt_ . More precisely,

using that _⌊ngpt⌋_ = _ngpt_ when _ngpt ∈_ N. It follows that: which doesn’t vary across groups conditional on _Tg_ = _t_ . On the other hand, defining _Ng_<sup>0=</sup><sup>_ng−N_1</sup> _g_<sup>, we</sup> have that: Next, for this assignment mechanism, A-9
where Similarly, where Notice that even if P[ _Dig_ = _d|Tg_ = _t_ ] does not change across _g_ , the joint probabilities do. Nevertheless, these terms can be calculated for any sample using the chosen probabilities _pt_ and the cluster sizes _{ng}_<sup>_G_</sup> _g_ =1<sup>.</sup>

**Bernoulli Trials.** Alternatively, the within-cluster treatment may be assigned to each unit independently as a “coin flip” with probability _pt_ . Under this mechanism, independence between treatment indicators implies that:

which do not vary over _g_ . It follows that: Then the variances are approximated by: and B-10
# **B Additional Empirical Results**

## **B.1 Further Details and Figures for Main Results**

B-11
Figure B.6: Payment rates: Treated groups vs Pure control blocks

# (a) Payment rates in levels

<!-- Start of picture text --> Due date of<br>40 Intervention October 2020 Treated groups<br>begins<br>billing period<br>30 Pure Control<br>20<br>10<br>0<br>25sep2020 02oct2020 09oct2020 16oct2020 23oct2020 30oct2020<br>Calendar date<br>(b) Difference relative to pure control group<br>Due date of T50<br>Intervention<br>5 October 2020<br>begins<br>billing period T80<br>T20<br>4<br>3<br>2<br>1<br>0<br>25sep2020 02oct2020 09oct2020 16oct2020 23oct2020 30oct2020<br>Calendar date<br>Cumulative payment rate (%)<br>Treatment effect (p.p.)<br><!-- End of picture text --> _Notes_ : These figures show the effect of the intervention on payments of the October 2020 bill for treated groups. Panel (a) shows the cumulative share of individuals paying the October 2020 bill over time. The brown dashed line shows the payment rate for pure control units. The blue dashed line corresponds to treated units in group _Tg_ = 1 (blocks with 20% treated). The black dashed line corresponds to treated units in group _Tg_ = 2 (blocks with 50% treated). The red solid line corresponds to treated units in group _Tg_ = 3 (blocks with 80% treated). Panel (b) shows, for each calendar date, the difference between each treated group and the pure control group (treatment effect coefficients). The letters were delivered between September 28th and October 7th. The first vertical bar denotes the start of the intervention. The due date was October 9th and is indicated with another vertical bar.

B-12
Figure B.7: Payment rates: Untreated groups vs Pure control blocks

# (a) Payment rates in levels

<!-- Start of picture text --> Due date of<br>Intervention<br>40 October 2020<br>begins<br>billing period<br>Untreated groups<br>30<br>Pure Control<br>20<br>10<br>0<br>25sep2020 02oct2020 09oct2020 16oct2020 23oct2020 30oct2020<br>Calendar date<br>(b) Difference relative to pure control group<br>Due date of<br>Intervention<br>5 October 2020<br>begins<br>billing period<br>Treated<br>(pooled)<br>4<br>3<br>2<br>1 T80<br>T50<br>0<br>T20<br>25sep2020 02oct2020 09oct2020 16oct2020 23oct2020 30oct2020<br>Calendar date<br>Cumulative payment rate (%)<br>Treatment effect (p.p.)<br><!-- End of picture text --> _Notes_ : These figures show the effect of the intervention on payments of the October 2020 bill for untreated groups. Panel (a) shows the cumulative share of individuals paying the October 2020 bill over time. The brown dashed line shows the payment rate for pure control units. The blue dashed line corresponds to untreated units in group _Tg_ = 1 (blocks with 20% treated). The black dashed line corresponds to untreated units in group _Tg_ = 2 (blocks with 50% treated). The red solid line corresponds to untreated units in group _Tg_ = 3 (blocks with 80% treated). Panel (b) shows, for each calendar date, the difference between each untreated group and the pure control group (treatment effect coefficients). For comparison, the gray solid line shows the treatment effects for treated units (pooled from _Tg_ = 1 _,_ 2 _,_ 3). The letters were delivered between September 28th and October 7th. The first vertical bar denotes the start of the intervention. The due date was October 9th and is indicated with another vertical bar.

B-13
Figure B.8: Direct effects on treated accounts and spillover effects on untreated accounts <!-- Start of picture text --> Direct Effects Spillover Effects<br>Treated vs. pure controls Untreated vs. pure controls<br>Intervention Intervention<br>7 begins Due date of 7 begins Due date of<br>6 Oct'20 bill 6 Oct'20 bill<br>5<br>5<br>4<br>4<br>3<br>3 2<br>2 1<br>1 0<br>0<br>25sep2020 02oct2020 09oct2020 16oct2020 23oct2020 30oct2020 25sep2020 02oct2020 09oct2020 16oct2020 23oct2020 30oct2020<br>Calendar date Calendar date<br>Blocks with 50% treated Blocks with 50% treated<br>Intervention Intervention<br>7 begins Due date of 7 begins Due date of<br>6 Oct'20 bill 6 Oct'20 bill<br>5<br>5<br>4<br>4<br>3<br>3 2<br>2 1<br>0<br>1<br>0<br>25sep2020 02oct2020 09oct2020 16oct2020 23oct2020 30oct2020 25sep2020 02oct2020 09oct2020 16oct2020 23oct2020 30oct2020<br>Calendar date Calendar date<br>Blocks with 20% treated Blocks with 20% treated<br>Intervention Intervention<br>7 begins Due date of 7 begins Due date of<br>6 Oct'20 bill 6 Oct'20 bill<br>5<br>5<br>4<br>4<br>3<br>3<br>2<br>2 1<br>1 0<br>0<br>25sep2020 02oct2020 09oct2020 16oct2020 23oct2020 30oct2020 25sep2020 02oct2020 09oct2020 16oct2020 23oct2020 30oct2020<br>Calendar date Calendar date<br>Treatment effect (p.p.) Treatment effect (p.p.)<br>Treatment effect (p.p.) Treatment effect (p.p.)<br>Treatment effect (p.p.) Treatment effect (p.p.)<br><!-- End of picture text --> _Notes_ : These figures show the coefficients and 95% confidence intervals from a saturated regression that computes, at each calendar day, the payment rate difference between each treated and untreated group relative to the pure control group (i.e., blocks where no accounts were treated). The top panel shows the effect on treated (left) and untreated (right) units in blocks with 80% treated ( _Tg_ = 3). The middle panel shows the effect on treated (left) and untreated (right) units in blocks with 50% treated ( _Tg_ = 2). The bottom panel shows the effect on treated (left) and untreated (right) units in blocks with 20% treated ( _Tg_ = 3). These point estimates coincide with those reported in panel (b) of Figures B.6 and B.7. Standard errors are clustered by block. The first vertical bar denotes the start of the intervention. The due date for the October 2020 bill was October 9th and is indicated with another vertical bar. The letters were delivered between September 28th and October 7th.

B-14
Figure B.9: Placebo. Direct and spillover effects for the pre-intervention Sep’20 bill <!-- Start of picture text --> Treated groups Untreated groups<br>vs. pure controls vs. pure controls<br>Blocks with 80% treated Blocks with 80% treated<br>7 Due date of Interventionbegins 7 Due date of Interventionbegins<br>6 Sept 2020 bill 6 Sept 2020 bill<br>5 5<br>4<br>4<br>3<br>3<br>2<br>2<br>1<br>1 0<br>0<br>01sep2020 08sep2020 15sep2020 22sep2020 29sep202 01sep2020 08sep2020 15sep2020 22sep2020 29sep202<br>Calendar date Calendar date<br>Blocks with 50% treated Blocks with 50% treated<br>7 Due date of Interventionbegins 7 Due date of Interventionbegins<br>6 Sept 2020 bill 6 Sept 2020 bill<br>5 5<br>4 4<br>3 3<br>2 2<br>1<br>1<br>0<br>0<br>01sep2020 08sep2020 15sep2020 22sep2020 29sep202 01sep2020 08sep2020 15sep2020 22sep2020 29sep202<br>Calendar date Calendar date<br>Blocks with 20% treated Blocks with 20% treated<br>7 Due date of Interventionbegins 7 Due date of Interventionbegins<br>6 Sept 2020 bill 6 Sept 2020 bill<br>5 5<br>4 4<br>3 3<br>2 2<br>1 1<br>0 0<br>01sep2020 08sep2020 15sep2020 22sep2020 29sep202 01sep2020 08sep2020 15sep2020 22sep2020 29sep202<br>Calendar date Calendar date<br>Treatment effect (p.p.) Treatment effect (p.p.)<br>Treatment effect (p.p.) Treatment effect (p.p.)<br>Treatment effect (p.p.) Treatment effect (p.p.)<br><!-- End of picture text --> _Notes_ : These figures show the coefficients and 95% confidence intervals from a saturated regression that computes, at each calendar day, the payment rate difference between each treated and untreated group relative to the pure control group (i.e., blocks where no accounts were treated). The top panel shows the effect on treated (left) and untreated (right) units in blocks with 80% treated ( _Tg_ = 3). The middle panel shows the effect on treated (left) and untreated (right) units in blocks with 50% treated ( _Tg_ = 2). The bottom panel shows the effect on treated (left) and untreated (right) units in blocks with 20% treated ( _Tg_ = 3). Standard errors are clustered by block. The first vertical bar shows the due date for the September 2020 bill. This corresponds to a bill issued and due for payment before our intervention began, thus serving as a placebo. The second vertical bar indicates the start of the intervention. The letters were delivered between September 28th and October 7th.

B-15
Table A4: Placebo. Direct and spillover effects for the pre-intervention September 2020 bill

|**Dependent variable:**||**Placebo bill (September 202**|**0)**|
|---|---|---|---|
|Pr(pay the bill)|All|Below Median|Above Median|
||(1)|(2)|(3)|
|**_A. Blocks with 80% treated_**||||
|Treated|0.12|0.10|0.28|
||(0.69)|(0.73)|(0.81)|
|Untreated|-0.30|-1.55|0.78|
||(0.95)|(1.09)|(1.24)|
|**_B. Blocks with 50% treated_**||||
|Treated|0.76|1.54|0.69|
||(0.88)|(0.99)|(1.12)|
|Untreated|0.26|0.81|0.36|
||(0.88)|(0.94)|(1.15)|
|**_C. Blocks with 20% treated_**||||
|Treated|0.85|1.32|0.27|
||(0.93)|(1.11)|(1.24)|
|Untreated|0.07|0.27|-0.32|
||(0.68)|(0.72)|(0.80)|
|Payment rate of pure control|29.70|20.05|38.19|
|Observations|68,806|32,361|36,445|
|Number of clusters (blocks)|3,981|2,013|1,968|

_Notes_ : Notes: This table shows the results from saturated OLS regressions using as dependent variable an indicator for paying the September 2020 bill by September 15th (pre-intervention). Each column corresponds to a separate regression. The omitted category corresponds to blocks where no accounts were treated (pure control). Panel A shows the results for blocks where 80% were treated, panel B for blocks with 50% treated, and panel C for blocks with 20% treated. Columns (2) and (3) split the sample from column (1) into blocks below and above median compliance in 2019, respectively. We define compliance as the share of bills paid by block in 2019 with median value of 0.56 (see Figure B.15). The estimates reported in column (1) correspond exactly to the numbers shown in Figure (B.9). The row _Payment rate of pure control_ displays the constant of each regression, corresponding to the average payment rate in blocks with no treated units. Standard errors clustered by blocks are reported in parentheses. * p _<_ 0.10, ** p _<_ 0.05, *** p _<_ 0.01 B-16
# **B.2 Other Margins**

**Subscriptions to electronic billing.** We find evidence that our tax communication campaign also increases the subscriptions to receive an electronic bill by e-mail.<sup>1</sup> These effects are greater in highsaturation blocks, albeit small in absolute value. Appendix Section B.2.1 presents graphical evidence of direct and spillover effects (Figure B.10), which are then summarized in Table A5, although spillover effects in this outcome are much more tenuous.

**Backward and forward payments.** We also find that the effects of our letters are not solely concentrated on the October 2020 billing period (the bill targeted by our intervention). Section B.2.2 presents graphical evidence that the letters also increased the payment rates in subsequent billing periods. Perhaps more strikingly, we also show that some neighbors made backward payments to cancel past-due debt from previous billing periods. This is especially prominent after April 2020 when the COVID-19 lockdown measures were established in Argentina (See Figure B.11).

## **B.2.1 Effects on Subscriptions to Electronic Billing**

The communication campaign also included information about how to sign up for electronic billing, a system introduced in June 2020. We briefly analyze the effect of our mailing on subscription to this service.

We rely on a database that contains the individuals who signed up for the electronic billing option. This database goes through December 2020 and contains the account number, date of subscription, and email address. This source is linked with the main data through the unique account identifier.

We analyze the intervention’s effect on subscriptions to electronic billing and present convincing graphical evidence that the tax communication campaign increased subscriptions to receive an electronic bill by e-mail. These effects are greater in high-saturation blocks, albeit small in absolute value.

The results are summarized in Figure B.10, which follows a similar structure as Figure B.8 but for e-bill subscriptions. We run dynamic difference-in-differences comparing subscription rates between each treated and each untreated group relative to pure control blocks, day by day (fixing September 27, 2020, as the baseline date).

Four important points are worth highlighting: (1) trends are generally parallel, as we estimate no significant differences between the treatment and control groups prior to the intervention; (2) the difference in subscription rates between treated accounts and pure control blocks experiences a noticeable break at the time we started sending letters, which is reassuring and implies that the effects we estimate are indeed caused by our experiment; (3) direct effects are greater in high-saturation blocks with 50% and 80% treated units relative to low-saturation blocks where only 20% received the letter. As happened with payment

> 1Note that nudging individuals to sign up for e-billing was an explicit content of the letter (see Figure A.1).

B-17
rates, this could be interpreted as a spillover effect, whereby the intervention creates interference between treated units strengthening the effect of the letter; and (4) although less clear than the left-hand-side panels for treated units, the right-hand-side panels of Figure B.10 also suggest the presence of spillover effects in subscriptions to e-billing for untreated accounts in high-saturation blocks. As was the case with payment rates, these effects are harder to detect. They are precisely estimated but only significant at the 5% level at the beginning of the intervention.

Lastly, Table A5 summarizes the corresponding diff-in-diffs estimates reported in Figures B.10, with the same structure as Table 2.<sup>2</sup> To benchmark our estimates, in the last row we report the share of e-bill subscribers in pure control blocks on September 27 (our baseline date). For treated accounts, the table shows an immediate effect in the three saturation groups that increases over time. This effect is higher in blocks with 80% treated units, consistent with interference that strengthens the effect. In such blocks, the total effect reaches 0.86 percentage points by the end of October. Although this represents about 20% of the baseline 4.25% share of e-bill subscribers, we find it striking that so few individuals switched to the digital bill. In the case of untreated accounts, spillover effects on subscription rates are smaller and, therefore, much harder to detect than in the analysis of payment rates. The clearest effect arises in blocks with 50% treated accounts with a spillover effect of 0.25 percentage points, significant at the 10% level. The somewhat absence of spillovers in this case can be explained by the fact that the outcome of analysis (subscription rate) has very low take up, making it harder for interference between neighbors to emerge.

In sum, we find that our tax communication campaign also generates direct effects and spillover effects among neighbors in subscriptions to electronic billing. These effects are greater in high-saturation blocks, albeit small in absolute value.

> 2Column (1) validates the experiment by showing a placebo saturated regression that compares subscription rates between each group and the pure control group on September 17, before the intervention began. None of the coefficients are statistically significant or large in magnitude.

B-18
Figure B.10: Direct effects on treated accounts and spillover effects on untreated accounts (subscriptions to e-billing). Difference in differences <!-- Start of picture text --> Treated groups Untreated groups<br>vs. pure controls vs. pure controls<br>Blocks with 80% treated Blocks with 80% treated<br>1 Intervention<br>1 Interventionbegins Due date ofOct'20 bill begins Due date ofOct'20 bill<br>.5<br>.5<br>0 0<br>-.5 -.5<br>16sep2020 30sep2020 14oct2020 28oct2020 16sep2020 30sep2020 14oct2020 28oct2020<br>Calendar date Calendar date<br>Blocks with 50% treated Blocks with 50% treated<br>1 Intervention<br>1 Intervention begins Due date ofOct'20 bill<br>Due date of<br>begins Oct'20 bill<br>.5<br>.5<br>0<br>0<br>-.5 -.5<br>16sep2020 30sep2020 14oct2020 28oct2020 16sep2020 30sep2020 14oct2020 28oct2020<br>Calendar date Calendar date<br>Blocks with 20% treated Blocks with 20% treated<br>1 Intervention 1 Intervention<br>begins Due date ofOct'20 bill begins Due date ofOct'20 bill<br>.5 .5<br>0 0<br>-.5 -.5<br>16sep2020 30sep2020 14oct2020 28oct2020 16sep2020 30sep2020 14oct2020 28oct2020<br>Calendar date Calendar date<br>Treatment effect (p.p.) Treatment effect (p.p.)<br>Treatment effect (p.p.) Treatment effect (p.p.)<br>Treatment effect (p.p.) Treatment effect (p.p.)<br><!-- End of picture text --> _Notes_ : These figures show the coefficients and 95% confidence intervals from dynamic difference-in-differences regressions where the outcome of interest is a dummy equal to one if the account is subscribed to an electronic bill. All the coefficients are estimated with respect to September 27th, 2020 (baseline date) and relative to the pure control group (i.e., blocks where no accounts were treated). The top panel shows the effect on treated (left) and untreated (right) units in blocks with 80% treated ( _Tg_ = 3). The middle panel shows the effect on treated (left) and untreated (right) units in blocks with 50% treated ( _Tg_ = 2). The bottom panel shows the effect on treated (left) and untreated (right) units in blocks with 20% treated ( _Tg_ = 1). Standard errors are clustered by block. The first vertical bar denotes the start of the intervention. The due date for the October 2020 bill was October 9th and is indicated with another vertical bar. The letters were delivered between September 28th and October 7th.

B-19
Table A5: Total effects and spillover effects for subscriptions to e-billing

|**Dependent variable:**|**Placebo:**|**Interv**|**ention:**|
|---|---|---|---|
|Pr(subscribe to e-bill)|By Sep 20|Early|By Oct 31|
||(1)|(2)|(3)|
|**_A. Blocks with 80% treated_**||||
|Treated|-0.02|0.31***|0.86***|
||(0.04)|(0.06)|(0.12)|
|Untreated|0.04|0.11|0.08|
||(0.03)|(0.08)|(0.15)|
|**_B. Blocks with 50% treated_**||||
|Treated|0.03|0.18**|0.81***|
||(0.03)|(0.08)|(0.18)|
|Untreated|-0.07|0.10|0.25*|
||(0.05)|(0.06)|(0.13)|
|**_C. Blocks with 20% treated_**||||
|Treated|-0.04|0.15*|0.57***|
||(0.05)|(0.08)|(0.19)|
|Untreated|-0.01|0.05|-0.09|
||(0.03)|(0.04)|(0.09)|
|Mean of Pure Control at baseline|4.25|4.25|4.25|
|Observations|137,612|137,612|137,612|
|Number of clusters (blocks)|3,981|3,981|3,981|

Notes: This table shows the results from a saturated dynamic difference-in-differences regression where the dependent variable is an indicator for subscribing to electronic billing. The regression computes the outcome difference between each of the treated and untreated groups relative to the pure control group for each calendar date relative to September 27th, 2020 (baseline date). The estimates correspond exactly to the numbers shown in Figure (B.10). Column (1) shows the results for e-bill subscriptions made before the letters were delivered (placebo); Column (2) shows the results for early subscriptions right after the letters started to be delivered (by October 3); Column (3) shows the results for subscriptions made up to the end of October 2020. The letters were delivered between September 28 and October 7. The due date for the October 2020 bill was October 9th. The row _Mean of Pure Control_ displays the constant of the regression, corresponding to the average susbcription rate for units in blocks with no treated units on September 27, 2020. Standard errors clustered by blocks are reported in parentheses. * p _<_ 0.10, ** p _<_ 0.05, *** p _<_ 0.01 B-20
# **B.2.2 Timing of Payments and Due Bills**

For completeness, we analyze the effects of the intervention on backward and forward payments corresponding to billing periods before and after month 10, the month of our intervention. These results are summarized in Figure B.11.

Intuitively, neighbors can pay their property tax bill at any time before or after the due date, and hence, payments from previous billing periods can also be affected by our intervention.<sup>3</sup> To illustrate this, the left panels of Figure B.11 only consider timely payments, defined as bills paid before the 27th of the corresponding month. We set any payment made after the 27th as unpaid in our data. Hence, preintervention bills mechanically exclude any past-due payment triggered by our intervention. In contrast, the right panels of Figure B.11 consider timely as well as past-due payments made until December 2020 and, thus, capture backward payments triggered by our intervention (e.g., individuals that decide to pay the October 2020 bill as well as previous unpaid bills after receiving the letter).

The top figures show payment rates in levels for treated units (black line) and pure control units (gray line), for 24 consecutive monthly bills between January 2019 and December 2020. Treated units are pooled from groups _Tg_ = 1 _,_ 2 _,_ 3. The bottom figures report total treatment effects—i.e., the difference between treated and pure control units—and 95% confidence intervals for the 24 billing periods. The first vertical bar denotes the start of the COVID-19 pandemic in Argentina, and the second vertical bar flags the October’20 bill targeted by our intervention.

Four important points are worth noting: (1) Overall, payment rate levels are low. The top left panel shows that about 48% of households pay their bill before the 27th of each month. This share is relatively constant until March 2020, when the COVID-19 pandemic hit Argentina and payment rates decreased sharply to 23%; (2) a similar pattern emerges when we consider timely and past-due payments. The reason why levels are higher and decrease over time is that as time goes by, it is more likely that individuals cancel unpaid bills; (3) placebo direct effects (red line), based on payment rates constructed with timely payments only, are precisely estimated and not different from zero for the 21 pre-intervention bills. For the October 2020 bill, however, timely payments are 4.4 p.p. higher in treated units relative to control blocks. This is reassuring and implies that our sample is balanced and that the effects we estimate are indeed caused by our experiment; and (4) when we account for past-due payments, the blue line shows that our intervention nudged some individuals to catch up with unpaid bills. The difference in payment rates between treated and pure control accounts experiences a noticeable increase in the pandemic billing periods from April 2020 onward. Although the October bill, when the intervention took place, presents the highest effect (4.2 p.p.), the letters also had some residual positive effects in November and December.

> 3The treatment letter included past due balances and could therefore induce neighbors to make backward payments to cancel debt.

B-21
# Figure B.11: Direct effects on pre- and post-intervention bills

<!-- Start of picture text --> Timely payments Timely and past-due<br>only payments<br>(a) Payment rates in levels<br>% paid the bill % paid the bill<br>COVID-19 Intervention COVID-19 Intervention<br>60 60<br>billing periods bill billing periods bill<br>55 55<br>50 50<br>45 45<br>Treated (pooled) Treated (pooled)<br>40 40<br>Pure Control Pure Control<br>35 35<br>30 30<br>25 25<br>20 20<br>2019m1 2019m4 2019m7 2019m10 2020m1 2020m4 2020m7 2020m10 2021m1 2019m1 2019m4 2019m7 2019m10 2020m1 2020m4 2020m7 2020m10 2021m1<br>Billing Period Billing Period<br>(b) Difference relative to pure control group<br>Treatment Treatment<br>effect (p.p.) effect (p.p.)<br>COVID-19 COVID-19<br>6 6<br>billing periods billing periods<br>5 5<br>Oct 2020: Oct 2020:<br>4 4.4 p.p. 4 4.2 p.p.<br>3 3<br>2 2<br>1<br>1<br>0<br>0<br>-1<br>-1<br>2019m1 2019m4 2019m7 2019m10 2020m1 2020m4 2020m7 2020m10 2021m1 2019m1 2019m4 2019m7 2019m10 2020m1 2020m4 2020m7 2020m10 2021m1<br>Billing Period Billing Period<br><!-- End of picture text --> _Notes_ : These figures show the effect of the communication campaign on payment rates of pre- and post-intervention bills. The left panels only consider timely payments, defined as bills paid before the 27th of the corresponding month (i.e., any payment made after the 27th is considered unpaid). Hence, pre-intervention bills mechanically exclude any past-due payment triggered by our intervention. The right panels consider timely as well as past-due payments made until December 2020 and, thus, capture backward payments triggered by our intervention (e.g., individuals who, after receiving the letter, pay the October 2020 bill as well as previous unpaid bills). The top figures show payment rates in levels for treated units (black line) and pure control units (gray line), for 24 consecutive monthly bills between January 2019 and December 2020. Treated units are pooled from groups _Tg_ = 1 _,_ 2 _,_ 3. The bottom figures report total treatment effects—i.e., the difference between treated and pure control units—and 95% confidence intervals for the 24 billing periods. The letters were delivered between September 28th and October 7th. The vertical bar denotes the start of the COVID-19 pandemic in Argentina. Each coefficient is estimated in separate regressions. Standard errors are clustered at the block level. The red line shows no difference on timely payments for pre-intervention bills. In contrast, when we account for past-due payments, the blue line shows that our intervention nudged some individuals to catch up with unpaid bills from April 2020 onwards.

B-22
# **B.3 Are Untreated Blocks Affected by the Intervention?**

A crucial aspect of partial population experiments is the unit within which the experimenter will test the presence of spillovers. In some settings, these are relatively straightforward to establish: electoral precincts for political outcomes, towns for regional policies, and schools or school districts for educational interventions. In our application, we aim to measure information spillovers among taxpayers. Discussions with municipal tax authorities and with taxpayers, as well as the context of our intervention, led us to select city street blocks as the relevant clusters for potential information spillovers about tax reminders and deadlines and their effects on tax compliance. Specifically, the campaign was motivated by the sharp drop in compliance in April 2020 induced by the severe lockdown imposed in the Greater Buenos Aires area in Argentina during the COVID pandemic in a context where most payments were made in person (see Figure B.11). The lockdown was strongly enforced, and as a result, citizens’ mobility was severely limited, which justifies the choice of the city street block—a relatively small cluster—as the relevant unit for information spillover since it reflects the limited physical interactions generated by the lockdown. A further justification is the city’s street layout, which consists mainly of relatively homogeneous straight streets with orthogonal intersections in square/rectangular city blocks (see Figure A.2).

A potential concern with this setup is that the city street block may not be the relevant unit to capture information spillovers. The random assignment process and the city’s physical layout imply that taxpayers in pure control street blocks (i.e., blocks where no one received a tax reminder) were still adjacent and/or surrounded by blocks with treated taxpayers, as shown by inspection of the map in Figure A.2. Interference between adjacent blocks is possible, and this would induce a downward bias in our results, since individuals in pure control (untreated) blocks would be affected by the information campaign via spillovers from adjacent (treated) blocks. Our empirical setup allows for an auxiliary test to rule out this concern and establish that units in pure control blocks indeed provide a valid counterfactual in our analysis.<sup>4</sup> To test the robustness of untreated blocks as pure controls, we leverage our experimental assignment process, which implies that the “intensity” of treatment in the surrounding blocks is random by definition. Pure control units are by chance surrounded by blocks with varying degrees of treatment intensity (0%, 20%, 50%, or 80%), and thus by a random number of treated taxpayers. If there is interference between treated and untreated blocks, we should observe that pure control payment rates increase with the exposure of untreated blocks to treated blocks.

We construct our measure of the potential exposure of a street-block to the intervention in two steps. First, we use GIS software to calculate a buffer of 100, 200, and 300 meters around the centroid of each street-block (see the three figures in the top panel of Figure B.12), given the typical street block length of 100 meters. Second, for each street-block and radius, we calculate the share of properties receiving a letter (treated) relative to all the properties in the buffer zone. The three figures in the middle panel of Figure

> 4This is an auxiliary analysis in the sense that while it exploits features of our experimental design, it does not correspond to our pre-registered analysis and only represents an ex-post robustness check.

B-23
# B.12 display the distribution of the share of treated units around pure control blocks.

With this exposure measure at hand, we test whether payment rates of the October 2020 bill in pure control blocks increase with the exposure to the proportion of treated units in surrounding blocks. The figures in the bottom panel of Figure B.12 present parametric and non-parametric evidence of this relationship. Each panel shows a binned scatterplot of payment rates of the October 2020 bill (y-axis) by equally-sized bins of exposure to treated units within the buffer zone (x-axis). Reassuringly, the relationship is flat, and it is robust to increasing the size of the buffer zone to 200 and 300 meters. This is confirmed by the small linear regression coefficients and large p-values reported in these figures.

Our main results indicated that we only found spillover effects in our main research design for high saturation blocks with high previous compliance, as illustrated by the results in Figure 3 and Table 2. We conduct a similar analysis with the exposure measure for the 100-meter buffer in Figure B.13. The parametric and non-parametric results presented there confirm a flat gradient for untreated blocks with both high and low compliance in 2019, further confirming that untreated blocks were not affected by the intervention even when considering this heterogeneity.

Finally, for completeness, we also study the relationship between payment rates and exposure to adjacent treated blocks in blocks where 80% of the units were treated, again for the 100-meter buffer. The results of this exercise are reported in Figure B.14. The left panel corresponds to the October 2020 bill affected by our intervention, whereas the middle and right panels correspond to pre-intervention bills of July and August 2020. In all these cases, the relationship between exposure and payment rates is flat and statistically not significant for both the pure control blocks (with blue dots and blue linear fit) and the 80% saturation blocks (with red triangles and a red linear fit). Interestingly, the vertical distance between the red and blue linear fit in the left panel captures the treatment effect of our experiment, which is clearly uniform in the exposure measure.

Taken together, the results from the exercise in this section indicate that pure control blocks were not affected by adjacent treated blocks, and thus provide a valid counterfactual for the analysis. In more general terms, information spillovers do not seem to have happened at a higher degree of aggregation than the city street block. When combined with the presence of information spillovers documented in the main body of the paper, the city street block seems to have been the relevant level of information dissemination for this campaign.

B-24
<!-- Start of picture text --> 300 6.1731<br>850 NS 900 61730 yj; wy,<br>g00 800 6.1779 Uh<br>350 6.1728<br>Jo y, 700 y 6.1727<br>650 600 y 6.1726<br>«00 s00 6.1725<br>550 lA 6.1774<br>400 Va 6.1723 -<br>400 500 600 700 300 400 #500 600 700 800 200 400 B00 800<br>.<br>° ° °<br>*. ee? * Je e . ~ °. ee , e e ee<br>> 9 ee o— og es<br>o° 0? oof oe e<br>°° oo e<br><!-- End of picture text -->
Figure B.13: Payment rates and exposure of untreated blocks above and below median 2019 compliance, 100 meters buffer <!-- Start of picture text --> Above Median Below Median<br>Payment All Payment Payment<br>rate rate Compliance rate Compliance<br>b=-0.018<br>(p=0.687)<br>b=.000<br>(p=.979)<br>b=-0.013<br>(p=0.745)<br>.05 .15 .25 .35 .45 .55 .05 .15 .25 .35 .45 .55 .05 .15 .25 .35 .45 .55<br>.55 .55 .55<br>.5 .5 .5<br>.45 .45 .45<br>.4 .4 .4<br>.35 .35 .35<br>.3 .3 .3<br>.25 .25 .25<br>.2 .2 .2<br>.15 .15 .15<br><!-- End of picture text --> _Notes_ : This figure shows binned scatterplots of payment rates (y-axis) in pure control blocks by equally-sized bins of exposure to treated units within a buffer zone of 100 meters (x-axis). The left panel replicates the bottom left panel of Figure B.12. The middle and right panels split pure control blocks into blocks with above- and below-median compliance defined in 2019, respectively. The regressions flexibly control for a cubic polynomial of the number of properties in the buffer zone. This variable is highly correlated with payment rates, and its omission leads to omitted variable bias.

Figure B.14: Payment rates and exposure of untreated blocks and blocks with 80% treated units, 100 meters buffer <!-- Start of picture text --> Paymentrate October 2020 Paymentrate July 2020 Paymentrate August 2020<br>Pre intervention Pre intervention<br>Blocks with<br>80% treated<br>Untreated<br>blocks<br>.05 .15 .25 .35 .45 .55 .05 .15 .25 .35 .45 .55 .05 .15 .25 .35 .45 .55<br>.5 .5 .5<br>.45 .45 .45<br>.4 .4 .4<br>.35 .35 .35<br>.3 .3 .3<br>.25 .25 .25<br><!-- End of picture text --> _Notes_ : This figure shows binned scatterplots of payment rates (y-axis) by equally-sized bins of exposure to treated units within a buffer zone of 100 meters (x-axis). The left panel shows the gradient in both untreated blocks (blue dots) and blocks with 80% treated units (red triangles) for the October 2020 bill (the one affected by the intervention). The middle and right panels correspond to the pre-intervention bills of July and August 2020, respectively. The regressions flexibly control for a cubic polynomial of the number of properties in the buffer zone. This variable is highly correlated with payment rates and its omission leads to omitted variable bias.

B-26
# **B.4 Heterogeneity and Pre-treatment Tax Compliance**

Section 4.4.2 analyzes heterogeneous effects based on pre-treatment tax compliance behavior in the year 2019. The distribution of the 68,806 accounts by the number of bills paid in 2019 is bi-modal, with a core group of neighbors not paying any bills (35%) and another group paying all of them (45%). Panel (a) of Figure B.15 shows the individual-level distribution. We define past compliance by computing the average number of payments of the twelve monthly bills for 2019 in each block. Panel (b) of Figure B.15 shows the block-level distribution. We use this measure to divide our sample into two groups – those above and those below the median block average payment rate.

The logic of this exercise goes as follows. A large fraction of neighbors who typically paid their bills stopped doing so during the pandemic in the first few months of 2020. This decrease in compliance was stronger in blocks that had higher compliance in 2019. Hence, we argue that such a core group of “good compliers” is more likely to be nudged to pay by our intervention, and where spillover effects are more likely to manifest.

Figure B.16 suggests that 2018 and 2019 are comparable in terms of compliance, but compliance decreased substantially in 2020 because of the pandemic—the sharp fall corresponds to the lockdown measures put in place. Figure B.17 shows that payment rates in 2020 decreased more in blocks with higher compliance in 2019. In contrast, 2018 and 2019 show similar levels of compliance. This set of figures thus helps us rationalize the reasoning behind the heterogeneity analysis presented in Section 4.4.2.

Table 2 confirms that spillover effects are driven by blocks with baseline compliance above the median in high saturation blocks (80% treated). Spillover effects are more muted and insignificant in medium (50% treated) and low (20% treated) saturation blocks, however. Reassuringly, the first two columns also show no effects for the pre-intervention bill of September 2020, either above or below the median.

B-27
# Figure B.15: Distribution of bill payments in 2019 for individuals and blocks

## (a) **Number of monthly bills paid in 2019 (by individuals)**

<!-- Start of picture text --> Share of<br>accounts<br>.5<br>.4 45%<br>.3 35%<br>.2<br>.1<br>0<br>0 1 2 3 4 5 6 7 8 9 10 11 12<br>Number of bills paid in 2019<br><!-- End of picture text -->

- (b) **Share of bills paid in 2019 (by blocks)**

<!-- Start of picture text --> N blocks<br>300<br>Mean = 0.59<br>p50 = 0.56<br>N = 3,981<br>200<br>100<br>0<br>0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1<br>Share of bills paid in 2019 - by block<br><!-- End of picture text --> _Notes_ : Panel (a) shows the distribution of the 68,806 accounts by the number of bills paid in 2019. The distribution is bi-modal with a core group of neighbors not paying any bills (35%) and another group paying all of them (45%). Panel (b) uses the information from panel (a) to compute the share of total bills paid in 2019 for each block. We use this measure of block-level compliance for the heterogeneity analysis, to split our sample into blocks below and above the median of 0.56 (see Table 2). These two figures and values look very similar for the year 2018.

B-28
# Figure B.16: Compliance in the first nine months of 2018, 2019, and 2020

<!-- Start of picture text --> (a) 2018 vs 2019<br>N blocks<br>2018<br>300<br>2019<br>200<br>100<br>0<br>0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1<br>Share of bills paid - by block<br>(b) 2019 vs 2020<br>N blocks<br>2020<br>300<br>2019<br>200<br>100<br>0<br>0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1<br>Share of bills paid - by block<br><!-- End of picture text --> _Notes_ : These figures show compliance in the first 9 billing periods of the year. For each block, we compute the share of total bills paid out of 9. Panel (a) compares 2018 and 2019, and panel (b) compares 2019 and 2020. We restrict the analysis to the first 9 bills because our intervention takes place in October. To make it comparable, the numerator excludes overdue payments (i.e., payments made after the due date of each month). The figure suggests that 2018 and 2019 are comparable in terms of compliance and that compliance decreased substantially in 2020 because of the pandemic.

B-29
Figure B.17: Payment rates in 2020 decreased more in blocks with higher compliance in 2019 <!-- Start of picture text --> 1 45° line<br>.9<br>.8<br>2018<br>.7<br>.6 2020<br>.5<br>.4<br>.3<br>.2<br>.1<br>0<br>0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1<br>Share of bills paid in 2019<br>2018 or 2020<br>Share of bills paid in<br><!-- End of picture text --> _Notes_ : This figure compares compliance in 2018 or 2020 (vertical axis) relative to 2019 (horizontal axis) at the block level. To that end, we split the sample of blocks into ten evenly-spaced groups using the share of payments in 2019 (horizontal axis). For each bin, we then compute the average share of payments in 2018, 2019, and 2020. The red triangles compare 2018 to 2019, and the blue circles compare 2020 to 2019. The 45<sup>_◦_</sup> line corresponds to the situation where compliance remains unchanged over time. The figure suggests that the drop in compliance in 2020 highlighted in Figure B.16 is more prominent for higher levels of baseline compliance. That is, blocks that had high compliance in 2019 are those where the payment rate decreased the most in the first nine months of 2020. In contrast, 2018 and 2019 display similar levels of compliance. This stylized fact suggests that blocks with high compliance in 2019 (and low compliance in 2020) are more likely to be nudged by our intervention and, thus, where spillovers are more likely to manifest.

B-30
# **C Additional Numerical Illustration**

Figure 1 summarizes the distribution of cluster sizes in five published studies employing partial population designs: Cr ́epon et al. (2013), Gin ́e and Mansuri (2018), Haushofer and Shapiro (2016), Ichino and Sch ̈undeln (2012) and Imai, Jiang and Malani (2021).

For this numerical illustration, we calculate the estimators standard errors and minimum detectable effects based on our formulas from Section 3 using the cluster size distribution of these four studies. We refer to these magnitudes as “adjusted” standard errors and MDEs, since they are adjusted for cluster size variation. For comparison, we also calculate the “unadjusted” standard errors and MDEs using average cluster size and assuming that the variance of group size is equal to zero, that is, ignoring cluster size heterogeneity. To make the results comparable, we use as a benchmark the design in our application to tax compliance, which has four saturations: _p_ 0 = 0, _p_ 1 = 0 _._ 2, _p_ 2 = 0 _._ 5, _p_ 3 = 0 _._ 8. We compute the optimal probabilities _{q_ 0 _, q_ 1 _, q_ 2 _, q_ 3 _}_ using Theorem 2. We assume for simplicity that outcomes are homoskedastic with _σ_<sup>2</sup> ( _dt, dt_ ) = 1 for all _d, t_ so that effects are measured in standard deviations, and consider four values for the intraclass correlation, _ρ ∈{_ 0 _._ 1 _,_ 0 _._ 2 _,_ 0 _._ 5 _,_ 0 _._ 8 _}_ . The parameter of interest is the spillover effect on untreated units in groups with 80% treated.

Table A6: Numerical results

||**Sta**|**ndard er**|**ror**||**MDE**||
|---|---|---|---|---|---|---|
||Adj.|Unadj.|Ratio|Adj.|Unadj.|Ratio|
|_ρ_= 0_._1|||||||
|GM|0_._1262|0_._1181|1_._0687|0_._3536|0_._3308|1_._0689|
|HS|0_._1053|0_._0932|1_._1307|0_._2951|0_._2610|1_._1307|
|IS|0_._1768|0_._1667|1_._0608|0_._4954|0_._4670|1_._0608|
|IJM|0_._0569|0_._0497|1_._1453|0_._1595|0_._1393|1_._1450|
|_ρ_= 0_._5|||||||
|GM|0_._2593|0_._2393|1_._0835|0_._7265|0_._6705|1_._0835|
|HS|0_._2098|0_._1783|1_._1761|0_._5877|0_._4997|1_._1761|
|IS|0_._3437|0_._3171|1_._0840|0_._9630|0_._8884|1_._0840|
|IJM|0_._1136|0_._0950|1_._1961|0_._3183|0_._2661|1_._1962|
|_ρ_= 0_._8|||||||
|GM|0_._3252|0_._2997|1_._0851|0_._9112|0_._8397|1_._0851|
|HS|0_._2622|0_._2218|1_._1818|0_._7345|0_._6215|1_._1818|
|IS|0_._4284|0_._3941|1_._0869|1_._2002|1_._1042|1_._0869|
|IJM|0_._1420|0_._1181|1_._2024|0_._3979|0_._3309|1_._2025|

The numerical results are shown in Table A6. When the intraclass correlation is low ( _ρ_ = 0 _._ 1), accounting for cluster size heterogeneity increases standard errors and MDEs between 6.8% and 14.5%. The problem worsens for larger intraclass correlations. When _ρ_ = 0 _._ 5, adjusted standard errors and MDEs are between 8.3% and 19.6% larger, and between 8 _._ 5% and 20 _._ 2% larger when _ρ_ = 0 _._ 8.

D-31
» » » (x jx y » ~~8~~ (e) ~~f~~ s ] ~~-~~ x ~~-~~ > ~~> >> >~~ ) ~~ae~~
<!-- Start of picture text --> F- [| ~—— |<br>e le f e | ——<br>ee f e<br>is<br>> | ><br>—{ r ry<br><!-- End of picture text -->
<!-- Start of picture text --> f s] - y - Y<br>Hi f e<br>o s<br>F— | - =<br>f p<br>> HH<br>ne )<br>—— HH — }<br>-~HU— J<br><!-- End of picture text -->
<!-- Start of picture text --> FA<br>Ht —<br>' J - ¢<br>E e<br>“ (x ) |<br>- (5 '<br>ce) ( ef)<br><!-- End of picture text -->
» <!-- Start of picture text -->
|— |<br>»<br>a<br>x [= y |<br>ee<br>—_-y<br><!-- End of picture text -->
>

~~—— ——~~ |

~~——~~ | ) ~~<u>a</u> ———~~ 5° ~~— ———~~ 55 ~~———_ ———~~ So ~~— ———~~ 5° ~~— a~~ | ~~ae~~

~~--~~ r | ~~-~~ x
<!-- Start of picture text --> = e 7* a<br>= e »<br>c=Je)<br>ee<br>© eS )<br>o e »<br>EEnT<br>> r y<br>= ><br>: = > = ><br><!-- End of picture text -->
<u>5</u> <!-- Start of picture text -->
><br><!-- End of picture text -->

<!-- Start of picture text --> 5<br><!-- End of picture text --> <!-- Start of picture text --> /<br>——<br>— {- E<br>- x<br>><br>( =)<br>- y > > > > ss<br>- y - y<br>- y<br>an<br>- ¥<br>- ¥<br><!-- End of picture text -->
<!-- Start of picture text -->
- » - »<br>ye ee<br>- » - » - »<br>= pr t ot<br>y s zr<br>><br>- oa<br>= > ( =) x<br>(=<br>os<br>- ¥<br>[-<br><!-- End of picture text -->
<!-- Start of picture text --> — [ z |<br>yl<br>ee ee |<br>> [<br>pil<br>yl<br>S o _<br>a So — ce<br>S o— _<br>_ So — _ _<br><!-- End of picture text -->
<!-- Start of picture text --> — 5° ——— ee<br>- y oI |<br>S o— ee<br>5° — __<br>><br>> ee<br>S> _<br>> y— —(5 )<br><!-- End of picture text -->
using that _Bt_ ( **_ω_** ) _>_ 0 for all _t_ so the Hessian is positive definite as required. □

# **D.6 Proof of Corollary 1**

This proof follows from Theorem 1 and Proposition 1 setting _μg_ ( _d, t_ ) = _μ_<sup>_p_</sup> _n_<sup>(</sup><sup>_d, t_) =</sup><sup>_μ_(</sup><sup>_d, t_) throughout.□</sup>

# **D.7 Proof of Proposition 2**

Let _Dg_ ( _d, t_ ) denote the set of possible values for **D** ( _i_ ) _g_ given _Dig_ = _d_ and _Tg_ = _t_ . Then, where the first equality follows by the law of iterated expectations, the second equality plugs in the potential outcomes under the exclusion restriction (Assumption 6), the third equality uses independence (Assumption 7), and the fourth equality uses exchangeability (Assumption 8).

# **D.8 Proof of Theorem 3**

We verify the conditions for Theorem 1. First, condition (i) implies that Proposition 2 holds. Second, condition (ii) and Proposition 2 imply Assumption 1. Next, condition (iii) implies that: D-43
and thus by Proposition 2, E[ _Yig_<sup>_l|Dig_=</sup><sup>_d, Tg_=</sup><sup>_t_] = E</sup> î _Yig_<sup>_l_</sup> Ä _d,_<sup>_ngp_</sup> _n_<sup>_<u>g</u>_</sup> _g_<sup><u>(1</u></sup> _−_<sup>_<u>|t</u>_</sup> 1<sup><u>)</u></sup><sup>_−d_</sup> äó. This fact also implies that

and an analogous argument gives the result for the joint moments, so Assumption 2 holds. Next, condition (iii) implies that Assumption 3 holds, so all the requirements for Theorem 1 are satisfied. Finally, by Proposition 2 and condition (iv), which completes the proof. □

# **D.9 Proof of Theorem 4**

This result follows from the fact that conditions (i) and (ii) imply Assumption 5 and thus under the conditions for Theorem 3, Corollary 1 holds. □ D-44