metadata: authors: 0: author: Fujii,Tomoki
1: author: Van Der Weide,Roy
count: World
docna: 0: docna: Is predicted data a viable alternative to
            real data ?
docty: Policy Research Working Paper
subtopic: Health Care Services Industry,Labor Markets,Nutrition,Poverty Impact Evaluation,Poverty Monitoring & Analysis,Inequality,Poverty Assessment,Poverty Diagnostics,Poverty Lines,Small Area Estimation Poverty Mapping
teratopic: Poverty Reduction
abstracts: cdata!: It is costly to collect the household-
            and individual-level data that underlies official estimates
            of poverty and health. For this reason, developing countries
            often do not have the budget to update their estimates of
            poverty and health regularly, even though these estimates
            are most needed there. One way to reduce the financial
            burden is to substitute some of the real data with predicted
            data. An approach referred to as double sampling collects
            the expensive outcome variable for a sub-sample only while
            collecting the covariates used for prediction for the full
            sample. The objective of this study is to determine if this
            would indeed allow for realizing meaningful reductions in
            financial costs while preserving statistical precision. The
            study does this using analytical calculations that allow for
            considering a wide range of parameter values that are
            plausible to real applications. The benefits of using double
            sampling are found to be modest. There are circumstances for
            which the gains can be more substantial, but the study
            conjectures that these denote the exceptions rather than the
            rule. The recommendation is to rely on real data whenever
            there is a need for new data, and use the prediction
            estimator to leverage existing data.
repnme: repnme: Is predicted data a viable alternative to
            real data ?
display_title: Is predicted data a viable
            alternative to real data
listing_relative_url: /research/2016/09/26822026/predicted-data-viable-alternative-real-data
topicv3: Poverty m1326687 695,Poverty Measurement and Analysis m1326941 707,Labor Force Surveys m1504075 2765,Health m1504661 3480,Malnutrition m1627088 4737
publishtoextweb_dt: 2016-09-28T00:00:00Z
historic_topic: Poverty Reduction
no_of_pages: 45
author_keys: 0: author_key: 605729
1: author_key: 677092
model_extractions: WPS7841 Policy Research Working Paper 7841 Is Predicted Data a Viable Alternative to Real Data?

_Tomoki Fujii Roy van der Weide_ Development Research Group Poverty and Inequality Team September 2016
Policy Research Working Paper 7841

# **Abstract**

It is costly to collect the household- and individual-level data that underlies official estimates of poverty and health. For this reason, developing countries often do not have the budget to update their estimates of poverty and health regularly, even though these estimates are most needed there. One way to reduce the financial burden is to substitute some of the real data with predicted data. An approach referred to as double sampling collects the expensive outcome variable for a sub-sample only while collecting the covariates used for prediction for the full sample. The objective of this study is to determine if this would indeed allow for realizing meaningful reductions in financial costs while preserving statistical precision. The study does this using analytical calculations that allow for considering a wide range of parameter values that are plausible to real applications. The benefits of using double sampling are found to be modest. There are circumstances for which the gains can be more substantial, but the study conjectures that these denote the exceptions rather than the rule. The recommendation is to rely on real data whenever there is a need for new data, and use the prediction estimator to leverage existing data.

This paper is a product of the Poverty and Inequality Team, Development Research Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at rvanderweide@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# Is predicted data a viable alternative to real data?<sup>_∗_</sup>

Tomoki Fujii<sup>_†_</sup> Roy van der Weide<sup>_‡_</sup>

**JEL classification c odes:** C20, C53, I32.

**Keywords:** Prediction; Double sampling; Survey costs; Poverty.

> _∗_ This study has been funded by the World Bank’s Knowledge for Change Program for which the authors are grateful.

> _†_ Singapore Management University. Email: tfujii@smu.edu.sg

> _‡_ World Bank. Email: rvanderweide@worldbank.org
# **1 Introduction**

In economics, health sciences, and other disciplines, data on the outcome variable of interest is often costly to collect. The measurement of poverty for instance relies on household consumption expenditure data. Collecting this data involves long questionnaires administered over an extended period of time which substantially adds to the data collection costs. Individual-level data collection that involves physical examinations also tends to be costly. This has implications for the sample size and the frequency with which the data is collected. In many developing countries for example, particularly in Sub-Saharan Africa, estimates of poverty, malnutrition and health are obtained highly irregularly and as a result are often outdated.<sup>1</sup> The demand for bigger and more frequent data has motivated researchers to explore ways to substitute predicted data for real data. Consider the application to poverty measurement by Douidich et al. (2015) where household consumption poverty is predicted into annual labor force surveys in order to increase the frequency of poverty estimates from once every 7 years to every year.<sup>2</sup> Similar applications can be found in Stifel and Christiaensen (2007) and Christiaensen et al. (2012). Prediction methods have also been used to dramatically expand the sample size of poverty and health data. A prominent example of this is the small area estimation of welfare where the outcome variable of interest (i.e. consumption poverty) is predicted into a population census which covers (almost) all members of the population unlike typical household surveys. This allows researchers to obtain estimates of poverty at a highly disaggregated level such as districts, communities, and towns. These small-area estimates are often plotted in the form of a map known as a poverty map. The small-area estimation, which provides the methodological foundations for poverty mapping, was pioneered by Elbers et al. (2003) and extended by e.g. Tarozzi and Deaton (2009), Tarozzi (2011) and Elbers and van der Weide (2014). Fujii (2010) has modified the approach to obtain small areas estimates of the prevalence of

> 1For example, across the 26 low-income countries in Sub-Saharan Africa over the period between 1993 and 2012, the national poverty rate and prevalence of stunting for children under five are on average reported only once every five years and once every ten years in the World Development Indicators.

> 2This approach could of course be expanded to other outcome variables of interest (such as health outcomes), while surveys other than labor force surveys could be considered to further extend the frequency of estimates. The same approach could also be adopted to construct a measure of consumption poverty that alleviate concerns of comparability over time when the original consumption data is deemed incomparable due to changes in the questionnaire. See for example the debate on the comparability of poverty estimates in India documented in e.g. Deaton (2003), Deaton and Dr`eze (2002), Kijima and Lanjouw (2005), Tarozzi (2007), and Deaton (2005).

2
stunting and underweight of children using predicted data.

When new data is needed, it may be tempting to purposefully only collect the covariates _x_ , which are used to predict the outcome variable of interest _y_ , and to scale down the collection of _y_ itself, particularly if this yields a significant financial cost reduction. It is not uncommon that predictors of household welfare such as demographic and dwelling characteristics, education, employment, and asset ownership, can indeed be collected relatively inexpensively. In health sciences, simple oral questions and anthropometric data taken by a simple non-invasive device too may serve as a predictor of the outcome that is expensive to measure. Collecting _y_ for a sub-sample only, and the covariates _x_ for all, is referred to as “double-sampling”,<sup>3</sup> see e.g. Hidiroglou (2001).<sup>4</sup> The advantage of this approach is that the prediction model can be estimated with data for the relevant population and time period (the same population for which the covariate data is available). The alternative where the model is estimated to data from an entirely different dataset that may describe a different population at a different point in time is referred to as “non-nested double sampling”, in which case one will have to make assumptions about how the model has evolved between the two different datasets (or assume that the model is invariant), see e.g. Kim and Rao (2012) as well as Douidich et al. (2015) for an empirical application to poverty measurement. We will refer to any estimator that works with predicted data (i.e. imputations of _y_ ) as a prediction estimator.

There is a considerable practical interest in adopting a double-sampling approach and relying on predicted data in the hope of reducing financial costs while preserving a reasonable degree of statistical precision. Consider for example the recent initiative by the World Bank under the name of SWIFT (Survey of Well-being via Instant and Frequent Tracking) which “does not collect direct income or consumption data which can be both time-consuming and vulnerable to error without the right know-how and resources; instead, it collects poverty correlates, such as household size, ownership of assets or education levels, and then converts them

> 3It is also known as “two-phase sampling”.

> 4The literature on double sampling dates back to Neyman (1938) and Bose (1943). Many of the existing studies, including Hidiroglou (2001), Kim et al. (2006), Palmgren (1987), Rao and Sitter (1995), and Sitter (1997), provide analytical or simulation results on the properties of the estimators based on double sampling. While there are some explicit empirical applications of double sampling such as Tamhane (1978), Hansen and Tepping (1990), and Armstrong et al. (1993), the use of double sampling appears to be comparatively limited. Even fewer studies take into account the costs of data collection (Cochran (1977), Davidov and Haitovsky (2000), S ̈arndal et al. (2003), and Fujii and van der Weide (2013)), despite the fact that one major attraction of double sampling is the reduction in data collection cost.

3
to poverty statistics using estimation models.” (Yoshida et al. (2015)). It employs Computer Assisted Personal Interviewing technology to collect the data and carefully builds and estimates the model that is used for prediction. Both nested and non-nested double sampling approaches are considered.<sup>5</sup> While SWIFT is still relatively young, at the time of writing, it has already been applied 34 times in 27 countries. Assessments of the cost-precision trade-offs however are still limited. Ahmed et al. (2014) denotes an exception which provides a SWIFT-like application of double sampling to poverty estimation in Bangladesh, where a variety of different data collection scenarios are being considered. Their simulation results indicate that substantial cost savings could be achieved with a moderate loss in precision. It is unclear however how much of this may be attributed to the use of double sampling as, in addition to relying on predicted data, they also reduce the number of primary sampling units.<sup>6</sup> Pape and Mistiaen (2015) apply a double sampling approach to poverty measurement in Mogadishu. They extend the set of predictors by including a sub-set of consumption items. Ligon and Sohnesen (2016) explore a similar strategy and include an empicical illustration using data from Uganda, Tanzania and Rwanda. Both these studies build on the idea that was put forward by Lanjouw and Lanjouw (2001). This approach strengthens the correlation with total household consumption but also adds to the costs, although Pape and Mistiaen (2015) in their application to Mogadishu are able to keep the face-to-face interview time below 60 minutes.<sup>7</sup> Other popular examples of prediction-based poverty estimation include the “Simple Poverty Scorecard” (SPS) project and the “Poverty Assessment Tool” (PAT) developed by the IRIS Center for USAID, see Schreiner (2014a) for a comparison.<sup>8</sup> These approaches predict a household’s poverty status on the basis of a small number of questions, mostly relying on nonnested double sampling, and then aggregate these predictions to obtain estimates of poverty at the national level (and possibly other administrative levels). SPS and PAT have been applied to

> 5In its version of the non-nested double sampling approach infrequent conventional surveys, which collect both _y_ and _x_ for all households, are alternated with more frequent low cost surveys that only collect _x_ . The infrequent full surveys are used to estimate (and update) the prediction model which is then used to predict poverty into the subsequent low cost surveys that are conducted for the years in between the full surveys.

> 6Furthermore, they do not provide a full break-down of the costs involved; only selected cost components are reported which excludes transport costs for example.

> 7Security concerns denote an important motivation for reducing the time of data collection by means of faceto-face interviews.

> 8The poverty scorecard takes a more pragmatic approach to building its prediction model as it emphasizes simplicity and ease of implementation. See Schreiner (2014b) for further details on the Simple Poverty Scorecard.

4
around 60 and 40 countries, respectively. A recent evaluation can be found in Diamond et al. (2015). To the best of our knowledge, no cost-precision assessments are available for any of these initiatives.

The objective of this study is to determine if and when double sampling may reasonably be expected to yield meaningful reductions in the cost of data collection while preserving statistical precision. We do this using analytical calculations that rely on an approximation to the financial cost function and on the asymptotic variance as the measure of statistical precision. This allows us to consider an inclusive set of parameter values. We subsequently make an attempt to calibrate the parameters involved to a variety of real data. Specifically, we solve a cost minimization problem subject to a statistical precision constraint and its dual problem of variance minimization problem subject to a budget constraint. This helps us identify the conditions under which the gains from double sampling are relatively large (and small). To the best of our knowledge, this is the first study to attempt an analytical assessment of how much precision prediction estimators trade for financial cost savings. We treat the sample direct estimator and the prediction estimator under single- and double-sampling in a unified framework and derive the analytic results for cost or variance reduction. The assumed model allows for clustering, which plays an important role in empirical applications but which is often ignored in theoretical work.

We find that the financial gains from double sampling over optimal single sampling tend to be modest for many of the parameter values considered, which are calibrated to real data. The magnitude of the potential discounts are mostly below 20 percent and can be as low as zero percent. There are circumstances in which the gains can be more substantial, but we conjecture that these denote the exceptions rather than the rule. Double sampling is most advantageous when: (a) the marginal cost of collecting _y_ is particularly large, (b) _x_ is highly correlated with _y_ , (c) travel costs between clusters are modest, (d) the sampling error is large relative to the model error, and (e) the spatial correlation in the data is modest. Unfortunately, these conditions are rarely jointly satisfied. When the covariates _x_ are particularly low-cost (as is the case with SPS and PAT), then the correlation between _x_ and _y_ tends to be weak. When the covariates offer exceptionally good predictors (as may be the case in Pape and Mistiaen (2015) and Ligon and Sohnesen (2016)), then the marginal cost of collecting _y_ tends to be low. SWIFT arguably lies 5
somewhere in between.

We have assumed away any error that may stem from model misspecification, i.e. all results hold under the standard assumption that the prediction model is correctly specified. This means that the real gains realized by the prediction estimators could be more modest yet. Model misspecification would introduce an entirely new source of error which is not accounted for in estimates of statistical precision.<sup>9</sup> Consequently, applied users should always bear in mind that prediction estimators are arguably less precise than is suggested by conventional standard errors.

The financial savings are larger for non-nested double sampling estimators. However these are also based on stronger assumptions. The added assumption is that the model parameters did not change between the dataset used for estimation and the dataset used for prediction which can be multiple years apart.<sup>10</sup> This adds to the risk of model misspecification error.

Our recommendation is to rely on real data on the outcome variable of interest _y_ whenever there is a need for new data. There is an argument for scaling back the collection of _y_ , and collecting covariates of _y_ instead, if there is insufficient budget to accommodate an adequate sample of observations with real data on _y_ (and _x_ ). Note that this does not in any way rule out the use of prediction estimators, such as the approaches employed in Douidich et al. (2015) and Elbers et al. (2003). This variety of non-nested double sampling estimators provides researchers with a means of leveraging existing data, which adds to the value of these data.

The remainder of this paper is organized as follows. Section 2 presents the prediction estimators under the single- and double-sampling setup. Then, they are applied to cost-effective sampling in Section 3. We study the conditions under which double sampling is most useful and explore the magnitude of potential gains from double sampling under practical conditions in Section 4. Finally, Section 5 provides some discussion.

> 9As any given model can be misspecified in infinitely different ways, there are currently no methods available that would account for model misspecification error.

> 10One does not necessarily need to assume that the model parameters are time-invariant. Alternatively, one could make assumptions about how the model has evolved exactly between the two datasets, but this assumption is just as strong.

6
# **2 Prediction estimator**

## **2.1 Preliminaries**

Consider the following data generating process: where _c_ and _h_ are the indexes of clusters and households, respectively. The continuous state variable _ych_ , which may or may not be observable, is related to the observable outcome variable of interest _Ych_ = _m_ ( _ych_ ) by some function _m_ . The _L_ -vectors of (observable) covariates and coefficients are denoted by _xch_ and _β_ , respectively. The idiosyncratic error term _uch_ (= _ηc_ + _ech_ ), which consists of the cluster-specific error _ηc_ and the household-specific error _ech_ , is unobservable. This error structure allows for some degree of spatial correlation in the errors. We denote the variances of error terms by _σe_<sup>2</sup><sup>_≡_var[</sup><sup>_ech_],</sup><sup>_σ_</sup> _η_<sup>2</sup><sup>_≡_var[</sup><sup>_ηc_], and</sup><sup>_σ_</sup> _u_<sup>2</sup><sup>_≡_var[</sup><sup>_uch_](=</sup><sup>_σ_</sup> _e_<sup>2+</sup><sup>_σ_</sup> _η_<sup>2).</sup> Each cluster is assumed to consist of _K_ sampled households. We make the following assumptions about _xch_ , _ηc_ , and _ech_ :

**Assumption 1** _The triple_ ( _{xch}_<sup>_K_</sup> _h_ =1<sup>_, {ech}K_</sup> _h_ =1<sup>_, ηc_)</sup><sup>_is iid across c._</sup>

**Assumption 2** _xch, ech, and ηc are independent of each other for all c. Furthermore, ech is independent across h for all c._

It is convenient to denote the stacked error terms for households in cluster _c_ by _Uc ≡_ ( _uc_ 1 _, uc_ 2 _, . . . , ucK_ )<sup>_T_</sup> and all households by _U ≡_ ( _U_ 1<sup>_T, U_</sup> 2<sup>_T, . . . , U T_</sup> _J_<sup>),where</sup><sup>_J_isthenumber</sup> of clusters in the sample. Similarly, we denote all _x_ ’s stacked together in cluster _c_ by _Xc_ , and all _Xc_ stacked together by _X_ . We define Ω _≡ E_ [ _UU_<sup>_T_</sup> ] and Ω _c ≡ E_ [ _UcUc_<sup>_T_].</sup> Let us define the expected value of _Ych_ conditional on _xch_ by _g_ ( _xch, θ_ ) _≡ Eu_ [ _Ych|xch_ ], where _θ_ is a _κ_ -vector of identifiable model parameters and _g_ is assumed differentiable with respect to _θ_ . We also define _εch ≡ Ych − g_ ( _xch, θ_ ).

The parameter of interest in this study is _μ ≡ E_ [ _Ych_ ] = _Ex_ [ _g_ ( _xch, θ_ )] and not _θ_ . The standard estimator for _μ_ is the sample mean _Y_<sup> ̄</sup> = _n_<sup>_−_1 �</sup> _c_ � _h_<sup>_Ych_, where</sup><sup>_n_=</sup><sup>_JK_denotes the</sup> sample size. This estimator will be referred to as the sample direct estimator.

7
# **2.2 Binary outcome variable**

For ease of exposition, we specialize in a case where the outcome variable is binary. This is an important special case, because binary outcomes are routinely encountered in empirical applications. Examples include the poverty rate (proportion of the individuals under the poverty line), prevalence of undernourished children, and the share of underemployed people among the employed individuals. We will subsequently use the poverty rate for the purpose of illustration, but it is straightforward to apply our theory to other contexts.

In the context of binary outcome, we maintain the following assumption:

**Assumption 3** _The variables Ych and ych are related by Ych_ = Ind( _ych < z_ ) _, where z is a constant. Furthermore, ηc and ech are normally distributed._

The normality of _ηc_ and _ech_ is not essential but it helps us to simplify our presentation as _uch_ is also a normal random variable in this case. When _μ_ represents the poverty rate, the constant _z_ corresponds to the poverty line. We denote the probability density function and cumulative distribution function for a standard normal random variable by _φ_ and Φ, respectively. In this case it follows that: _g_ ( _xch, θ_ ) = Φ �( _z − x_<sup>_T_</sup> _ch_<sup>_β_)</sup><sup>_/σu_</sup> �.

# **2.3 Single-sample prediction estimator**

The prediction estimator for _μ_ relies on the assumption that the functional form of _g_ is known.<sup>11</sup> This is true whether we use the single-sample or the double-sample estimator. Consider first the single-sample prediction estimator: where _θ_<sup>ˆ</sup> denotes the estimator for _θ_ . All the model uncertainty is captured by the estimator ˆ _θ_ of the unknown model parameter _θ_ . Rather than taking the sample average over the actual realizations of _Ych_ , it estimates the average of the conditional mean _g_ given _xch_ . For the binary outcome variable the prediction estimator is seen to solve:<sup>12</sup>

> 11We avoid using the term “regression estimator” in this study because it typically refers to the prediction estimator under the assumption of linearity (and often a single covariate).

> 12Notice that the predicted values do not incorporate estimates of the cluster-specific effects _ηc_ conditional on the available data in eq. (2). In other words, it is not an Empirical Bayes estimator (See Elbers and van der

8
where: _Bch ≡_ ( _z − x_<sup>_T_</sup> _ch_<sup>_β_)</sup><sup>_/σu_.</sup> The sample direct and prediction estimators are subject to different sources of error. The former is purely a function of the sampling error. The latter trades some of the sampling error for the model error. More precisely, it averages out the error terms _ηc_ and _ech_ and reduces the sampling error component. However, it introduces the model error instead because the prediction estimator uses an estimate of _θ_ rather than the true parameter value. Put differently, the contributions of _η_ and _e_ to the error of the estimate of _μ_ are “re-packaged” from sampling error to model error in the prediction estimator.

To study the properties of prediction estimators, we need to make some assumptions about ˆ ˆ _θ_ . Let us begin conservatively by merely assuming that _θ_ is a consistent and asymptotically normal estimator for _θ_ . Note that this accommodates practically all commonly used estimators.

**Assumption 4** _The estimator θ_<sup>ˆ</sup> _of the model parameters θ satisfies the following properties:_

_where Vθ is a symmetric positive-definite κ × κ asymptotic covariance matrix of θ_<sup>ˆ</sup> _._

**Remark 5** _Because K is fixed, there is a bijective correspondence between n and J. Therefore, it is also possible to write Assumption 4 as follows:_

Hereafter, we assume that suitable regularity conditions always hold. In particular, we assume the almost-sure existence and non-singularity of relevant moments, which typically pose no problem in empirical applications. With these assumptions, we have the following theorem (all proofs are in the Appendix):

> Weide (2014) for an application of Empirical Bayes estimator in a similar context). While the prediction estimator in eq. (2) is generally inefficient when _ych_ is observed, we chose to use this form for two reasons. First, the exposition is much simpler when eq. (2) is used. Second, the subsequent discussion is applicable without much modifications even when _ych_ and _z_ are unobservable.

9
**Theorem 6** _Let Mg ≡ E_ [ _∂g_ ( _xch, θ_ ) _/∂θ_ ](= 0 _κ_ ) _, where_ 0 _κ is a column κ-vector of zeros. Then, under Assumptions 1, 2, and 4, we have:_

The factor _K_<sup>_−_1</sup> in the definition of _Vg_ is included to normalize the variance at the cluster level. Notice that _Vg_ = _K_<sup>_−_1</sup> var[ _g_ ( _xch, θ_ )] holds when _xch_ is independent across all _c_ and _h_ . It should also be noted that the asymptotic variance of _μ_ ˆ( _θ_<sup>ˆ</sup> ) can be consistently estimated by replacing _Vg_ , _Mg_ and _Vθ_ with their consistent estimators. For ease of presentation, we hereafter ˆ simply write _μ_ dropping the argument _θ_<sup>ˆ</sup> .

ˆ As discussed earlier, the prediction estimator _μ_ may be more precise than the sample direct estimator _Y_<sup> ̄</sup> . The improvement in precision achieved by the prediction estimator is not necessarily the main attraction of the prediction estimator; its advantage is that it allows for a double sampling strategy, which potentially leads to the reduction in financial costs for collecting survey data without compromising the statistical precision. We pursue this idea in Section 3.

# **2.4 Double-sample prediction estimator**

The prediction estimator given in eq. (2) uses a single sample. However, it is clear that we only need the observations of _xch_ once the estimates of _β_ , _σe_ , and _ση_ are obtained. This, in turn, means that it is not necessary to observe _Y_ (or _y_ ) for all sample households; a sub-sample will do. We may still use the full sample to evaluate the mean prediction estimator if _x_ is collected for all. That is, even when data on _Y_ (or _y_ ) is collected only for a sub-sample, provided that _x_ is collected for the full sample of households, predicted values of _Y_ may still be evaluated for the full sample. This approach is referred to as “double sampling”.

If observing _Y_ is much more expensive than observing _x_ , then double sampling may be preferred to the standard single sampling approach—where both _x_ and _Y_ are observed for all households in the sample; double sampling has the potential to realize a reduction in the financial costs associated with collecting the necessary survey data while maintaining the desired statistical precision.

10
To formally introduce double sampling, we make a few assumptions.

**Assumption 7** _The covariates x are observed for all sample households, while y is observed for the first k ≤ K households in all J clusters._<sup>13</sup>

Let _sch_ denote the indicator variable that equals one if the household is in the full sample and zero otherwise. We further denote by _s_<sup>_I_</sup> _ch_<sup>[</sup><sup>_sII_</sup> _ch_<sup>(=</sup><sup>_sch−sI_</sup> _ch_<sup>)]theindicatorvariablethatthe</sup> household is in the sub-sample containing data on both _y_ and _x_ [ _x_ only]. Using this notation, the number of households included in the former and latter sub-samples equals, respectively, _rn_ and (1 _− r_ ) _n_ , where _n_ is the sample size of the full sample and _r_ (= _k/K_ ) is the ratio of sample households with an observation of _y_ .

**Assumption 8** _The distribution of_ ( _xch, ηc, ech_ ) _is independent of_ ( _s_<sup>_I_</sup> _ch_<sup>_, sII_</sup> _ch_<sup>)</sup><sup>_._</sup>

This requires that the selection into either sample carries no information about _xch_ , _ηc_ , or _ech_ . This is a reasonable assumption in our setup because the researcher chooses whether to observe _ych_ .

We use the households with the observations of _y_ and _x_ to compute the estimator _θ_<sup>ˆ</sup><sup>_I_</sup> of _θ_ , where _θ_<sup>ˆ</sup><sup>_I_</sup> satisfies the following assumption, which is a double-sample analogue of Assumption 4:

**Assumption 9** _The estimator θ_<sup>ˆ</sup><sup>_I_</sup> _of the model parameters θ satisfies the following properties:_

# _where Vθ_<sup>_Iis a symmetric positive-definite κ × κ asymptotic covariance matrix ofθ_ˆ</sup><sup>_I._</sup>

13 Since data on _x_ is already being collected for all households in all clusters of the sample, the marginal cost of collecting data on _y_ from all clusters is identical to the cost of collecting _y_ for the same number of households from half the number of clusters, say. Given that any two households from different clusters carry more information than a pair of households from the same cluster, due to spatial correlation, it is optimal to collect _y_ for a sub-sample of households from all clusters. The added advantage of collecting household expenditure data from all clusters is that it allows the user to construct unit value prices for each cluster which are needed to convert nominal household expenditure data into real terms. Alternatively, the survey could collect its price data by visiting local markets, i.e. by including a community price module. In this study we will abstract away from spatial (and temporal) price adjustments, and assume that this is taken care off. Note however that this is not a trivial matter, see e.g. van Veelen and van der Weide (2008).

11
Using _θ_<sup>ˆ</sup><sup>_I_</sup> , we can predict _μ_ by _g_ ( _xch, θ_<sup>ˆ</sup><sup>_I_</sup> ) for all observations in _S_ . Therefore, the prediction estimator for _μ_ under Assumption 7 is given by: The following is a direct extension of Theorem 6 to the double-sample estimator:

**Theorem 10** _Suppose that Assumptions 1, 2, 7, 8, and 9 hold. Then, μ_ ˆ<sup>_DS_</sup> _satisfies the following properties as J →∞:_

Note that Theorem 6 is a special case of Theorem 10 when _r_ = 1 (i.e., _k_ = _K_ ).

In the binary context, we have the following results:

**Theorem 11** _Suppose that Assumptions 1, 2, 3, 7, and 8 hold. Further, θ_<sup>ˆ</sup><sup>_I_</sup> = ( _β_<sup>ˆ</sup><sup>_T_</sup> _,_ ˆ _ση_<sup>2</sup><sup>_,_ˆ</sup><sup>_σ_</sup> _e_<sup>2)</sup><sup>_Tis a_</sup> _maximum-likelihood estimator. Then, μ_ ˆ<sup>_DS_</sup> _satisfies the following as J →∞:_

_where_ Σ _φx and_ Σ _φB are defined as follows:_ _Further,_ Ω<sup>_−_</sup> _c_<sup>1</sup> _can be written as:_ _where Ik and_ **1** _k are the K × K-identity matrix and K-vector of ones, respectively._ Let us further develop the expression for the asymptotic variance by making some modest simplifying assumptions that will ease the exposition.

12
<!-- Start of picture text --> »<br>»<br>| [ (———JI l L t) 4<br>| [ t——)] ) |b)<br><!-- End of picture text -->
When eq. (9) holds, _E_<sup>_−_1</sup> [ _Xc_<sup>_T_Ω</sup><sup>_−_</sup> _c_<sup>1</sup><sup>_Xc_] can be approximated as follows:</sup> where we have taken a first-order approximation with respect to _α_ in the second line using the formula of the differentiation of an matrix inverse (e.g., p.151 of Magnus and Neudecker (2007)). Plugging eq. (10) in eq. (6) and using Lemma 13, the variance of _μ_ ˆ<sup>_DS_</sup> can be approximated as follows: where _Vi_ , _Vh_ , and _Vc_ are the variance components due to household-specific sampling errors, model errors, and cluster-specific errors, respectively, and have the following definitions: The formula for _Vc_ shows that the variance component due to cluster-specific errors has two important parts: the first term ( _Vg_<sup>1) represents the sampling errors at the cluster level (i.e., errors</sup> due to the variations of _x_<sup>0</sup> _c_<sup>) whereas the second and third terms represent the idiosyncratic errors</sup> at the cluster level (i.e., errors due to the variations of _ηc_ ).

One important observation to make here is that the total variance _V_ consists of three components, each inversely proportionate to the full sample size (i.e., _n_ ), the size of sub-sample with the outcome variable (i.e., _nr_ ), and the number of clusters (i.e., _J_ ). Note also that eq. (11) 14
P ~~it) (II~~ ( ~~—), I]~~ | [ ~~-——)]~~ | ~~(——)|~~

»
Douidich et al. (2015) estimate the relationship between consumption poverty and household characteristics using a household consumption survey and then use this model to predict consumption poverty into a series of annual labor force surveys. This approach enables users to leverage existing data sources. If one decides to collect new data for the estimation of poverty, one has the option of only collecting the covariates _x_ , since an estimator for _θ_ can be obtained from the secondary data source. Put differently, the covariate-only sample is typically part of data collection planning when _μ_ ˆ<sup>_DS_</sup> is used but this is not necessarily the case when _μ_ ˆ<sup>_NN,DS_</sup> is used.

ˆ We will not formally derive the precision of _μ_<sup>_NN,DS_</sup> here as this would require us to make assumptions about the dynamics of the model parameters. When a previous survey is used as in Douidich et al. (2015) for example, one would have to make an assumption about how the model has evolved over time (or assume that it is time-invariant), which is not required for _μ_ ˆ<sup>_DS_</sup> . In Section 4.1 we will however provide a brief discussion on the financial costs savings that may be expected when using the non-nested double sampling estimator (under some simplifying assumptions).

# **3 Cost efficient sampling**

To see whether a meaningful reduction in costs is achievable, we examine the trade-offs between financial costs and statistical precision analytically. A stylized yet informative financial cost function is used. For a measure of statistical precision we appeal to the analytic expression for the asymptotic variance. The advantage of studying this trade-off analytically is that it allows us to work out the conditions under which double-sampling may be expected to be most beneficial. And, similarly, under what conditions the benefits will be marginal.

Specifically, we consider the problem of minimizing financial costs given a statistical precision constraint and its dual problem of maximizing statistical precision under a given budget constraint. Formally, we make the following assumption:

**Assumption 15** _The cost of collecting only x for any additional household in a given cluster equals τ ∈_ (0 _,_ 1) _. The travel cost to visit an additional cluster is equal to c._

16
Here, we normalize the cost of collecting ( _xch, ych_ ) to be equal to one. As a result, it is reasonable to require _τ ∈_ (0 _,_ 1), because it costs something to observe the covariates but not as much as it would if both the covariates and the outcome variable are to be observed. Under Assumption 15, the total variable cost of data collection is given by: We ignore the fixed cost of data collection as it does not affect the optimal sampling design. If the fixed cost differs between the single and double sampling, the difference has to be taken into account in the choice of optimal design.

We now consider the optimal sampling under single and double sampling. In the former case, we fix _r_ = 1 and choose _n_ and _J_ to minimize the financial cost for a given variance _V_<sup> ̄</sup> or minimize the variance for a given total cost _C_<sup> ̄</sup> (i.e. budget for data collection). In the case of the latter, we also allow _r_ to vary.

# **3.1 Optimal single sampling**

To assess how much one stands to gain by adopting double sampling over single sampling, we derive the level of statistical precision and the cost for the optimal single sample case. To provide a competitive benchmark we will consider the optimal single-sampling prediction estimator, which may be a sample direct estimator or a prediction estimator.

Suppose that one wants to minimize the cost of data collection subject to a required accuracy. This formulation is relevant, for example, when the researchers or policy-makers know how accurate the estimate ˆ _μ_ should be. Therefore, when the variance has the form of eq. (11) and the cost function is given in eq. (16), the cost minimization problem can be formulated as follows: Ignoring the integer constraints for _n_ and _J_ for simplicity of presentation, we can obtain the 17
following minimizing arguments ( _n_<sup>_∗_</sup> 1<sup>_, J_</sup> 1<sup>_∗_) and minimized cost</sup><sup>_C_</sup> 1<sup>_∗_:</sup> where _H_ 1 _≡_<sup>_√_</sup> _Vi_ + _Vh_ +<sup>_√_</sup> _Vcc_ .

When the budget for data collection is exogenously given, the following dual problem would be more relevant.

Solving this yields: The solution above shows that the optimal total sample size _n_ increases with _Vi_ (as a larger _n_ will be needed to curb the sampling error). Similarly, the optimal number of clusters _J_ increases with _Vc_ (as a larger _J_ in this case is needed to curb the cluster level error component) and decreases with _c_ (i.e. the price tag associated with adding to the number of clusters).

# **3.2 Optimal double sampling**

We now turn to the optimization problem for double sampling, in which _r_ ( _≤_ 1) is also a choice variable. In this case, the cost minimization corresponding to eq. (17) is as follows: When we have an interior solution, solving the first order conditions yields: where _H_ 2 _≡_<sup>_√_</sup> _τVi_ + ~~�~~ (1 _− τ_ ) _Vh_ +<sup>_√_</sup> _Vcc_ and _wh ≡ Vh/Vi_ . As with the case of single sampling, we can also consider the dual problem of eq. (18). In this case, the total variance is minimized 18
under a fixed budget _C_<sup> ̄</sup> for data collection: Solving this, we obtain: The interpretation of the solutions for _n_ and _J_ is intuitive and similar to the case of single sampling. The optimal solution for _r_ (i.e., the share of observations for which data on both _y_ and _x_ will be collected) is found to be an increasing function of _wh_ and of the cost parameter _τ_ . The positive relationship with _wh_ conveys the fact that it takes data on _y_ to reduce the model error component: If the model error is important relative to the sampling error, then it is optimal to collect more data on _y_ (i.e. increase _rn_ ). If the sampling error is relatively more important, then it is optimal to expand the total sample size (i.e. _n_ ) at the expense of limiting the number of households for which data on _y_ is collected. The positive relationship between _r_<sup>_∗_</sup> (or _r_<sup>+</sup> ) and _τ_ conveys the fact that collecting data on _y_ is relatively lighter on the budget when _τ_ is larger.

Note that the solution above does not necessarily satisfy _r ≤_ 1. For this to hold, the following condition for an interior solution needs to be satisfied:

# **4 Evaluating the potential gains from double sampling**

A general comparison between sample direct estimators and prediction estimators is complicated by the fact that the latter relies on a prediction model which can be estimated using different methods. Prediction estimators may or may not outperform and the sample direct estimator (see Matloff (1981) and Fujii and van der Weide (2013)). In Section 4.1, we first consider the comparison between the optimal single- and double-sample estimators. This choice has at least two advantages. First, because both use prediction estimators, all the error components (i.e., _Vi_ , _Vh_ , and _Vc_ ) are the same. Their differences come only from the differences in sampling (i.e., 19
the choices of _n_ , _r_ , and _J_ ). This in turn means that the difference can be taken as the pure effect of choosing double sampling. If we compare the optimal double-sampling estimator with the sample direct estimator, then part of the difference must be attributed to the fact that the optimal double-sampling estimator is a prediction estimator (while the direct estimator is not). Second, because the variance components are the same between optimal single- and double-sampling estimators, the comparison provides a clear prediction about the circumstances under which double sampling is most useful.

In Section 4.2, we compute the potential gains from the optimal double sampling estimator not only in comparison with the optimal single sampling estimator but also with the sample direct estimator using empirically-relevant parameter values. This analysis provides plausible ballpark estimates of the gains from optimal double sampling for the estimation of poverty rates.

# **4.1 Optimal single sampling vs optimal double sampling**

One intuitive measure of comparative performance between the single- and double-sampling estimator is the ratio of their respective variances given the same budget. Another candidate measure is the ratio of financial cost between the optimal double- and single-sampling estimators given the same statistical precision. Under our assumptions, it conveniently follows that these two measures coincide and can be expressed as follows: where _wc ≡ Vc/Vi_ . Note that the relative performance measure satisfies _ρ ∈_ (0 _,_ 1] as long as eq. (21) is satisfied, where lower values of _ρ_ indicate larger gains from double sampling. It can be verified that _ρ_ = 1 holds if and only if eq. (21) is satisfied with equality. This is the threshold case where optimal double sampling reduces to optimal single sampling. When eq. (21) is violated, the double-sampling optimization problems in eqs. (18) and (20) have a corner solution, which is the solution for the comparable single-sampling optimization problems. In this case, double-sampling offers no advantage over single sampling, a situation that occurs when _wh_ and _τ_ are sufficiently high.

The following lemma, which follows directly from eq. (22), shows under what variance 20
and cost parameters one stands to gain the most from adopting the optimal double sampling estimator:

**Lemma 16** _Suppose that eq. (21) holds. Then, ρ satisfies the following conditions:_

These results are intuitive. First, consider the impact of _c_ on _ρ_ . When the travel costs make up a larger share of total costs, _ρ_ goes up. This makes the optimal double sampling estimator less attractive relative to the optimal single sampling estimator. Second, the impact of _wc_ on _ρ_ is also positive. When cluster-specific variations make up a larger share of the total variance of the prediction estimator, the gains from double sampling will be smaller as double sampling helps to reduce neither the travel cost nor cluster-specific variations. These two points can also be readily seen from the facts that eq. (22) is an increasing function of _wcc_ and that _ρ_ tends to 1 in the limit where _wcc_ tends to infinity. In this case, the number of clusters to be visited is the only thing that matters asymptotically and thus there are no gains from double sampling.

Third, _ρ_ tends to go up when _wh_ is higher. This essentially means that double sampling is beneficial when the household-specific sampling error is important relative to the model error. This result is also intuitive as the use of double sampling does not reduce the model error but it helps to reduce the household-specific sampling error. Finally, _ρ_ also tends to go up when _τ_ is higher. Hence, the double sampling strategy is most useful when we have covariates that can be collected cheaply. It can be verified that _τ_ functions as a lower bound for _ρ_ .

**Lemma 17** _Suppose that eq. (21) holds. Then, ρ ≥ τ ._

To further understand the difference between the optimal single- and double-sampling schemes, it is useful to consider the following ratio of cluster sizes for the main and dual problems: It is clear that _rJ_<sup>_∗<_1 whereas</sup><sup>_r_</sup> _J_<sup>+</sup><sup>_>_1 when eq. (21) holds with a strict inequality.Therefore,</sup> in the main [dual] problem where the cost [variance] is to be minimized, the number of clusters in the optimal double sampling is smaller [larger] from its counterpart in the optimal single 21
sampling to save the travel cost [cancel out the cluster-specific errors]. Further, because _rJ_<sup>_∗_[</sup><sup>_r_</sup> _J_<sup>+]</sup> is an increasing [a decreasing] monotonic transformation of _ρ_ , the signs of its partial derivatives with respect to _c_ , _wc_ , _τ_ , and _wh_ are the same as [the opposite of] those in Lemma 16.

It is also interesting to note that the number of households to be sampled in each cluster is the same between the main and dual problems as the following equation shows: Both equations show that the cluster size tends to get smaller when the errors due to the cluster-level variations become more pronounced. On the other hand, the cluster size tends to get larger (and the number of clusters to be visited get smaller) when the travel cost is larger. The ratio _K_ 2 _/K_ 1 = 1 _/_ ~~�~~ (1 + _wh_ ) _τ_ , which is no less than one when eq. (21) is satisfied, represents the change in the cluster size when one switches from optimal single sampling to optimal double sampling. When the parameters are unfavorable to double sampling (i.e., when _wh_ and _τ_ are high), the ratio of cluster sizes tends to be smaller.

# **4.2 Realistic estimates of gains from double sampling**

Let us now evaluate the potential benefits of double sampling using a realistic set of parameter values ( _c, τ, wc, wh_ ) taken from existing surveys. While the parameter values are clearly context-dependent and cannot be readily extrapolated to other contexts, this exercise is still useful as it gives practitioners a sense of how much they could reasonably expect to gain from double sampling. It also facilitates comparisons between the sample direct estimator and the optimal single sampling estimator.

Ideally, all the parameter values should come from a single survey. However, we are unable to do so due to the lack of data. In particular, the information on survey costs is typically unavailable to the public. Therefore, we collect empirical values of _c_ , _τ_ , , _wc_ , and _wh_ from various sources. For each of these parameters, we specify low, mid, and high values. The low and high values are close to the minimum and maximum observed in our data sources. The mid value is either the arithmetic or geometric mean of minimum and maximum. The parameter values are presented in Table 1. The details of the data sources and assumptions are provided in 22
Table 1: Set of parameter values used in this study.

|Value|Low||Mid||High||
|---|---|---|---|---|---|---|
|_c_|_c_<sup>_L_ </sup>=|4|_c_<sup>_M_ </sup>=|16|_c_<sup>_H_ </sup>=|64|
|_τ_|_τ _<sup>_L_ </sup>=|0_._06|_τ _<sup>_M_ </sup>=|0_._36|_τ _<sup>_H_ </sup>=|0_._66|
|_wc_|_w_<sup>_L_</sup><br>_c_|<sup>= 0</sup><sup>_._6</sup>|_w_<sup>_M_</sup><br>_c_ <sup>=</sup>|<sup>1</sup><sup>_._2</sup>|_w_<sup>_H_</sup><br>_c_|<sup>= 1</sup><sup>_._8</sup>|
|_wh_|_w_<sup>_L_</sup><br>_h_|<sup>= 0</sup><sup>_._4</sup>|_w_<sup>_M_</sup><br>_h_ <sup>=</sup>|<sup>1</sup><sup>_._2</sup>|_w_<sup>_H_</sup><br>_h_|<sup>= 3</sup><sup>_._6</sup>|

Appendix B.

While the parameter values reported in Table 1 are based on real data and existing studies, they do not necessarily represent the full range of values one might encounter in practice. Furthermore, because the parameter values are taken from different sources, we do not know the correlational structure of these parameters. To make the most of the available data, we choose to calculate _ρ_ for all the 81(= 3<sup>4</sup> ) combinations.

Table 2 provides the values of _ρ_ under different combinations of parameter values. A few points are worth mentioning here. First, Table 2 shows that eq. (21) is not satisfied for some combinations of parameter values. This occurs when _τ_ and _wh_ are relatively high. In this case, it does not save much to omit _y_ from observations and the model error is relatively high. Thus, optimal double sampling has no advantage over optimal single sampling. In fact, from a practical perspective, both _τ_ and _wh_ have to be low for optimal double sampling to have a meaningful advantage over optimal single sampling.

Second, in comparison with _τ_ and _wh_ , the values of _c_ and _wc_ have limited impact on _ρ_ for the range of values we consider. Third, the gains from optimal double sampling relative to optimal single sampling appear to be reasonably modest, even when eq. (21) is satisfied. The lowest number reported in Table 2 is 0.776. This means that the cost saving from optimal double sampling is at best 22.4 percent of the cost for optimal single sampling within the set of parameters we considered.

Figure 1 gives an idea of what values _ρ_ might attain by plotting _ρ_ as a function of _τ_ for four different choices of _wcc_ and _wh_ . The top (bottom) row corresponds to a relatively low (high) value for _wh_ , while the left (right) column refers to relatively low (high) values for _wcc_ (see Table 1). The diagonal line denotes the lower bound for _ρ_ from Lemma 17. Note that _ρ_ is only evaluated for _τ < τmax_ = 1 _/_ (1 + _wh_ ) (see eq. (21)). Judging by these figures, _ρ_ mostly 23
||(_wH_<br>_c , wH_<br>_h_)|0.975|0.984|0.991|—|—|—|—|—|—|
|---|---|---|---|---|---|---|---|---|---|---|
||_wH_<br>_c , wM_<br>_h_ )<br>|0.917|0.949|0.971|0.997|0.998|0.999|—|—|—|
|(_τ, c, wc, wh_).|(_wH_<br>_c , wL_<br>_h_)<br>(|0.839|0.903|0.946|0.960|0.977|0.987|0.999|0.999|1.000|
|parameters|(_wM_<br>_c , wH_<br>_h_)|0.972|0.982|0.989|—|—|—|—|—|—|
|binations of|(_wM_<br>_c , wM_<br>_h_ )|0.906|0.941|0.966|0.996|0.998|0.999|—|—|—|
|different com|(_wM_<br>_c , wL_<br>_h_)|0.817|0.887|0.936|0.955|0.972|0.985|0.999|0.999|1.000|
|ues of_ρ_for|(_wL_<br>_c , wH_<br>_h_)|0.968|0.977|0.986|—|—|—|—|—|—|
|Table 2: Val|(_wL_<br>_c , wM_<br>_h_ )|0.887|0.925|0.955|0.995|0.997|0.998|—|—|—|
||(_wL_<br>_c , wL_<br>_h_)|0.776|0.854|0.914|0.944|0.964|0.979|0.999|0.999|0.999|
|||(_τ L, cL_)|(_τ L, cM_)|(_τ L, cH_)|(_τ M, cL_)|(_τ M, cM_)|(_τ M, cH_)|<br>(_τ H, cL_)|(_τ H, cM_)|<br>(_τ H, cH_)|

24
ranges between 0.8 and 1, and it takes incredibly low values of _τ_ to realize reductions in costs (or variance) of 10 percent or more (i.e. values of _ρ_ below 0.9). Assuming that the parameter values for _wcc_ and _wh_ are indeed reasonable, the gains from double sampling are expected to be modest unless conditions are particularly favorable.

<!-- Start of picture text --> 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0<br>tau tau<br>0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0<br>tau tau<br>1.0 1.0<br>0.8 0.8<br>0.6 0.6<br>rho rho<br>0.4 0.4<br>0.2 0.2<br>0.0 0.0<br>1.0 1.0<br>0.8 0.8<br>0.6 0.6<br>rho rho<br>0.4 0.4<br>0.2 0.2<br>0.0 0.0<br><!-- End of picture text --> Figure 1: Plots of _ρ_ versus _τ_ for different choices of _wcc_ and _wh_ : (a) _wcc_ = 1 and _wh_ = 0 _._ 1 (top-left), (b) _wcc_ = 50 and _wh_ = 0 _._ 1 (top-right), (c) _wcc_ = 1 and _wh_ = 2 (bottom-left), and (d) _wcc_ = 50 and _wh_ = 2 (bottom-right) Recall that _ρ_ is the ratio of data collection costs between optimal single and double sampling strategies. One may also be interested in the gains relative to the sample direct estimator. Let _ω_ denote the ratio of data collection costs between the sample direct estimator and the double sample estimator given the required accuracy (variance) constraint.

25
To this end, we first derive the ratio _ψ_ of data collection costs between the sample direct estimator and the single-sampling prediction estimator (given the required accuracy constraint), which solves: This quantity can be computed when all the relevant inputs (i.e., _Vi_ , _Vh_ , _Vc_ , _V_<sup> ̃</sup> _i_ , _V_<sup> ̃</sup> _c_ , and _c_ ) can be obtained from a single data source. This is indeed the case for Malawi ( _ψ_ = 0 _._ 945) and Niger ( _ψ_ = 0 _._ 977). The gain of using the optimal single-sample estimator instead of the sample-direct estimator is thus found to be very small, at least for these two countries.

If we assume that the choice of _K_ in a given survey is optimized for the sample direct estimator, we are able to derive _ψ_ without data on _c_ . To see this point, notice first that the derivation of eq. (24) does not depend on the nature of the variance components _Vi_ , _Vh_ , and _Vc_ . Therefore, we can apply the variance components of the sample direct estimator defined in Theorem 14 to eq. (24), where we have _Vh_ = 0 because there is no model error for the sample direct estimator. Solving for _c_ , we obtain: where _K_ is the cluster size in the sample. The resulting ̃ _c_ can be interpreted as the implied travel cost that justifies the design of the sample, because the observed choice of _K_ is consistent with the optimal design for the sample direct estimator. We can then substitute _c_ ̃ in place of _c_ in eq. (25). We denote the left hand side of eq. (25) derived in this way by _ψ_<sup> ̃</sup> to distinguish it from _ψ_ .

The values of _c_ ̃ tend to be higher than the values of _c_ observed in Malawi and Niger (see Tables 3 and 4). However, the resulting values of _ψ_<sup> ̃</sup> are not very different from _ψ_ ( _ψ_<sup> ̃</sup> = 0 _._ 948 in Malawi and _ψ_<sup> ̃</sup> = 0 _._ 983 in Niger). Therefore, the alternative derivation of _ψ_ appears to provide a reasonable indication of the gains associated with the optimal single-sample estimator relative to the sample direct estimator.

We also derived _ψ_<sup> ̃</sup> for Tanzania ( _ψ_<sup> ̃</sup> = 0 _._ 961) as well as the following three strata of Cambodia: Phnom Penh ( _ψ_<sup> ̃</sup> = 0 _._ 945), Other Urban ( _ψ_<sup> ̃</sup> = 0 _._ 977), and Rural ( _ψ_<sup> ̃</sup> = 0 _._ 965). All these 26
estimates are close to unity. Hence, even if we take the lowest value of _ρ_ and _ψ_ among our estimates, the resulting value of the ratio _ω ≡ ψρ_ of minimized variances [costs] under the optimal double sampling and optimal sample direct estimators for an exogenously given cost [variance] is 0 _._ 733( _≃_ 0 _._ 776 _×_ 0 _._ 945). This suggests that the gain from optimal double sampling relative to the sample direct estimator is only about 27 percent even in the most optimistic case.

Let us also briefly comment on the financial cost savings that may be expected when the poverty rate is estimated using the non-nested double sampling estimator from eq. (15). We expect the reduction in costs to be more substantial in this case. For ease of exposition, suppose that the primary sample (the newly collected data) and the secondary sample (a previous survey, say) would offer similar estimates of the model parameters _θ_ , in terms of precision, if the primary sample indeed includes the observations of _y_ for all households. This ensures that the non-nested double sampling estimator (which predicts poverty into the primary sample using a model that is estimated from the secondary sample) will match the precision of the single sampling estimator (which uses the primary sample only), allowing for a fair comparison of costs. The non-nested double sampling estimator is able to achieve this level of precision without using any of the data on _y_ from the primary sample. By not collecting any _y_ but only the covariates _x_ the data for the primary sample come at a cost of _τ_ rather than unit cost. Note that this coincides with the lower bound value for _ρ_ as derived in Lemma 17. The difference between _ρ_ and _τ_ tends to be substantial, as can be seen in Figure 1, which suggests that the gains from non-nested double sampling are indeed expected to be substantially larger compared to those obtained from nested double sampling. It should be noted however that it is by no means guaranteed that a secondary sample (which may denote a noticeably older survey) can compete with an up-to-date sample as far as the estimation of _θ_ is concerned. A more detailed study of the cost savings that can be realized with non-nested double sampling, under different assumptions of model stability, is recommended but is beyond the scope of this paper.

# **4.3 When is double sampling promising?**

A relatively high value of _ω_ under the most optimistic case indicates that gains from double sampling are rather limited. However, this finding should not be overextrapolated because the 27
parameter values depend on the application. Different survey designs and data collection techniques may lead to different values of _ω_ . Let us therefore also consider the possibility where _ω_ may be substantially lower than 0 _._ 733.

In some contexts, the cost of observing outcomes may be very expensive. For example, if the data collection involves physically invasive techniques (e.g., blood testing), observing _y_ may be costly and thus the value of _τ_ may be substantially lower than 0 _._ 06. If we use _τ_<sup>_L_</sup> = 0 _._ 001 instead of _τ_<sup>_L_</sup> = 0 _._ 06, the lowest estimate for _ρ_ in Table 2 would be _ρ_ = 0 _._ 656 instead of _ρ_ = 0 _._ 776. The use of new data collection technology may also allow for significantly lower data collection costs. The marginal cost of collecting data online for example is typically much lower than that through traditional face-to-face interviews.

It is conceivable that the values we use for _wh_ and _wc_ too may be lower in other contexts, particularly when an obvious and strong proxy for the outcome of interest is available. Consider an application where consumption data derived from a short-form questionnaire serves as a proxy for the complete consumption aggregate that is based on a long-form questionnaire. The predictive power of the model is likely to be strong in that case such that _wc_ and _wh_ may be much lower. Just for the purpose of illustration, if we use ( _wc_<sup>_L, w_</sup> _h_<sup>_L_)=(0</sup><sup>_._15</sup><sup>_,_0</sup><sup>_._1)insteadof</sup> (0 _._ 6 _,_ 0 _._ 4) while keeping ( _τc_<sup>_L, cL_) = (0</sup><sup>_._06</sup><sup>_,_4), then</sup><sup>_ρ_can go as low as 0</sup><sup>_._529.Therefore, one can</sup> think of circumstances where the gains from double sampling can be substantial, but these may be the exceptions rather than the rule.

To further elucidate this point, consider a study by Ahmed et al. (2014), which implies considerable gains from using a prediction estimator in an application to poverty measurement in Bangladesh. Unfortunately, their approach does not fit perfectly into our analytical framework such that we are unable to compute the values for ( _wc, wh_ ) that would apply to their data. Specifically, their prediction estimator works with different data (compared to the sample direct estimator), the number of clusters is not necessarily chosen optimally, and neither the level of statistical precision nor the financial costs are fixed. This hampers a fair comparison between the two estimators.

However, the standard errors for the national poverty rate derived from a small sub-sample of 640 households are 2 _._ 9 and 2 _._ 4 percentage points (see Table 5 of Ahmed et al. (2014)), respectively, for the sample direct and prediction estimators, which corresponds to a 32 percent 28
( _≈_ 1 _−_ (2 _._ 4 _/_ 2 _._ 9)<sup>2</sup> ) reduction in variance. Without further information, however, it is hard to determine how much of this cost-precision trade-off can be attributed to the fact that their double sampling estimator substituted predicted data for real data.<sup>15</sup> It is unlikely that the gains are due to a favorably low level of _τ_ as _τ_ is estimated to be around 0 _._ 6 in their case, which is in the mid range. If the gains can indeed be attributed to the use of predicted data, then it is more likely that this is due to favorable values of _wh_ and _wc_ .

# **5 Discussion**

The primary motivation for the use of prediction in economics, health sciences, and other disciplines has been to deal with various forms of missing data problems. One could also make a case for adopting prediction estimators to obtain more cost-efficient estimates of the population mean when it is expensive to observe the outcome of interest in comparison with its covariates. For example, consider the estimation of poverty and malnutrition rates. The conventional sample direct estimators in this case require household- and individual-level data on expenditures and health outcomes. Collecting this data is generally costly. It is not uncommon that in developing countries, where poverty and poor health outcomes are most pressing, statistical agencies do not have the budget that is needed to collect these data frequently. As a result, official estimates of poverty and malnutrition are often outdated. This then makes it difficult to monitor progress (or lack thereof) in times when circumstances might be subject to considerable change, such as shocks to international staple prices, domestic climate shocks, etc. Using predicted data as a substitute for real data may then offer a valuable alternative.

In recent years, a number of studies have explored the option of predicting household expenditure data into existing secondary surveys in an effort to supplement existing poverty estimates and increase their frequency (Stifel and Christiaensen, 2007; Douidich et al., 2015). Douidich et al. (2015) for example considers the Labour Force Survey as their secondary survey, which is often available at a higher frequency than household expenditure surveys.

There is also a large literature that predicts household expenditure data into the population census, see for example Elbers et al. (2003) and the references therein. The objective here is to

> 15Also, the study does not report the total cost of data collection, but selected cost components instead.

29
obtain estimates of poverty at a high level of disaggregation, or at the level of small area such as a district. It would be unpractical to use the sample direct estimator because the data must contain an extraordinarily large number of households to obtain a reliable estimate for each small area, which would be financially infeasible.

It is then a small step to purposefully collect data on covariates that are ideally suited for the prediction of household- or individual-level outcomes of interest. If real data on the variable of interest is collected for a sub-sample of households, then this sub-sample can be used to estimate the model parameters that are used for prediction. The advantage of this double sampling approach is that the prediction model will apply to the population of interest by construction. There is certainly a considerable interest in adopting such an approach in practice in the hope that this will enable a meaningful reduction in financial costs while preserving a reasonable level of statistical precision.

The objective of our study is to investigate the potential gains that might be derived from a double sampling approach under a set of fairly general conditions. We achieve this by analytically deriving the asymptotic variances of the single- and double-sample prediction estimators and by considering approximations to a financial cost function. This allows us to maximize statistical precision [minimize financial costs] under a budget constraint [statistical precision constraint] for a wide set of parameter values. Even though we are working with analytic approximations, we expect that the broad findings coming out of this analysis may carry over to real applications.

When we calibrate the parameters from the variance structure and the financial cost function to real data in the context of the estimation of poverty, we find that the reductions in costs rarely exceed 25 percent and are often below 10 percent. Furthermore, we find that the magnitude of the gains derived from double sampling are primarily determined by the following factors: (a) relative size of the travel costs, (b) degree of spatial correlation between residuals, (c) financial discount obtained by not collecting the outcome variable of interest, and (d) the share of total error that may be attributed to model error (versus sampling error). Double sampling is most effective when a reasonably large geographic coverage can be obtained without having to spend a disproportionate share of the budget on travel, and when the spatial correlation between the residuals is smaller rather than larger. The financial discount obtained by not collecting the 30
expensive outcome variable of interest is most notable when a larger share of the total error is due to the sampling error (rather than the model error).

It is conceivable that larger gains can be obtained under certain conditions, for example, when the cost of collecting the outcome variable of interest is extraordinarily high, when the cheaply available predictors exhibit an exceptionally high correlation with the outcome variable of interest, and when the data exhibits very little spatial correlation. We conjecture, however, that these circumstances represent the exception rather than the rule. Moreover, we have currently abstracted away from model misspecification error. Ignoring this component of error obviously favors the prediction estimator. Accounting for misspecification error is not obvious; it is hard to quantify since the true model is inherently unknown and any given estimate of the model can be misspecified in infinitely many ways.

Given these observations, when new data is to be collected, we recommend that the outcome variable of interest should be included so that one does not have to rely on predicted data. This does not mean that there is no role for prediction estimators. Under the right circumstances we believe they could be of great value. For one, prediction estimators provide the means of leveraging already existing data (think of non-nested double sampling estimators, see for example Kim and Rao (2012) and Douidich et al. (2015)). Furthermore, if no previous data exists and the budget is particularly constrained such that one may be left with the choice between predicted data or no data, then the former may be preferred over the latter. In such a data-poor environment, which is not unheard of in developing countries, double sampling estimators may continue to provide a valuable option.

# **References**

- Ahmed, F., C. Dorji, S. Takamatsu, and N. Yoshida (2014) ‘Hybrid survey to improve the reliability of poverty statistics in a cost-effective manner.’ World Bank Policy Research Working Paper 6909, The World Bank

- Aliaga, A., and R. Ren (2006) ‘Optimal sample sizes for two-stage cluster sampling in demographic and health surveys.’ DHS Working Papers 2006 No.30, ORC Macro

31
- Armstrong, J., C. Block, and K.P. Srinath (1993) ‘Two-phase sampling of tax records for business surveys.’ _Journal of Business & Economic Statistics_ 11(4), 407–419

- Beegle, K., J. De Weerdt, J. Friedman, and J. Gibson (2012) ‘Methods of household consumption measurement through surveys: Experimental results from tanzania.’ _Journal of Development Economics_ 98(1), 3–18

- Bose, C. (1943) ‘Note on the sampling error in the method of double sampling.’ _Sankhy ̄a_ 6, 329– 330

- Christiaensen, L., P. Lanjouw, J. Luoto, and D. Stifel (2012) ‘Small area estimation-based prediction methods to track poverty: validation and applications.’ _Journal of Economic Inequality_ 10(2), 267–297

- Cochran, W.G. (1977) _Sampling Techniques,_ 3rd edition ed. (John Wiley & Sons)

- Davidov, O., and Y. Haitovsky (2000) ‘Optimal design for double sampling with continuous outcomes.’ _Journal of Statistical Planning and Inference_ 86, 253–263

- Deaton, A. (2003) ‘Adjusted Indian poverty estimates for 1999-2000.’ _Economic and Political Weekly_ 38(4), 322–326

- (2005) ‘Data and dogma: The great indian poverty debate.’ _World Bank Research Observer_ 20(2), 177–199

- Deaton, A., and J.P. Dr`eze (2002) ‘Poverty and inequality in India: A reexamination.’ _Economic and Political Weekly_ 37(36), 3729–3748

- Diamond, A., M. Gill, M. Dellepiane, E. Skoufias, K. Vinha, and Y. Xu (2015) ‘Estimating poverty rates in target populations: An assessment of the simple poverty scorecard and alternative approaches.’ _mimeo_

- Douidich, M., A. Ezzrari, R. van der Weide, and P. Verme (2015) ‘Estimating quarterly poverty rates using labor force surveys: A primer.’ _World Bank Economic Review._ Advance Access pulished 2015

32
- Elbers, C., and R. van der Weide (2014) ‘Estimation of normal mixtures in a nested error model with an application to small area estimation of poverty and inequality.’ World Bank Policy Research Working Paper 6962, The World Bank

- Elbers, C., J. Lanjouw, and P. Lanjouw (2003) ‘Micro-level estimation of poverty and inequality.’ _Econometrica_ 71, 355–364

- (2005) ‘Imputed welfare estimates in regression analysis.’ _Journal of Economic Geography_ 5, 101–118

- Fujii, T (2006) ‘Community-level estimation of poverty measures and its application in cambodia.’ In ‘Spatial Disparities in Human Development’ (United Nations University Press) pp. 289–314

- Fujii, T. (2010) ‘Micro-level estimation of child undernutrition indicators in Cambodia.’ _World Bank Economic Review_ 24(3), 520–553

- Fujii, T., and R. van der Weide (2013) ‘Cost-effective estimation of the population mean using prediction estimators.’ World Bank Policy Research Working Paper 6509, The World Bank

- Grosh, M.E., and Mu ̃noz (1996) ‘A manual for planning and implementing the living standards measurement study survey.’ Living Standards Measurement Study Working Paper 126, The World Bank

- Hansen, M.H., and B.J. Tepping (1990) ‘Regression estimates in federal welfare quality control programs.’ _Journal of the American Statistical Association_ 85(411), 856–864

- Hidiroglou, M. (2001) ‘Double sampling.’ _Survey Methodology_ 27(2), 143–154

- Humphreys, C.P. (1979) ‘The cost of sample survey designs.’ In ‘Proceedings of the Survey Research Methods Section’ American Statistical Association pp. 395–400

- Kijima, Y., and P. Lanjouw (2005) ‘Economic diversification and poverty in rural India.’ _Indian Journal of Labour Economics_ 48(2), 349–374

- Kim, J., A. Navarro, and W. Fuller (2006) ‘Replication variance estimation for two-phase stratified sampling.’ _Journal of the American Statistical Association_ 101(473), 312–320

33
- Kim, J., and J. Rao (2012) ‘Combining data from two independent surveys: A model-assisted approach.’ _Biometrika_ 99, 85–100

- Lanjouw, J., and P. Lanjouw (2001) ‘How to compare apples and oranges? poverty measurment based on different definitions of consumption.’ _Review of Income and Wealth_ 47(1), 25–42

- Ligon, E., and T. Sohnesen (2016) ‘Using reduced consumption aggregates to track and analyze poverty.’ _mimeo_

- Magnus, J., and H. Neudecker (2007) _Matrix Differential Calculus with Applications in Statistics and Econometrics: Revised Edition_ (John Wiley & Sons)

- Matloff, N. (1981) ‘Use of regression functions for improved estimation of means.’ _Biometrika_ 68, 685–689

- Ministry of Planning, and United Nations World Food Programme (2002) ‘Estimation of poverty rates at commune-level in cambodia: Using the small area estimation technique to obtain reliable estimates.’ , Ministry of Planning, Royal Government of Cambodia and United Nations World Food Programme, Phnom Penh, Cambodia

- Neyman, J. (1938) ‘Contribution to the theory of sampling human populations.’ _Journal of the American Statistical Association_ 33, 101–116

- Palmgren, J. (1987) ‘Precision of double sampling estimators for comparing two probabilities.’ _Biometrika_ 74(4), 687–694

- Pape, U., and J. Mistiaen (2015) ‘Measuring household consumption and poverty in 60 minutes: The Mogadishu high frequency survey.’ mimeo, The World Bank

- Pettersson, H., and B. Sisouphanthong (2005) ‘Cost model for an income and expenditure survey.’ In ‘Household Sample Surveys in Developing and Transition Countries,’ vol. ST/ESA/STAT/SER.F/96 of _Series F_ (Department of Economic and Social Affairs, United Nations Statistics Division) studies in methods 13, pp. 267–277

- Rao, J., and R. Sitter (1995) ‘Variance estimation under two-phase sampling with application to imputation for missing data.’ _Biometrika_ 82(2), 453–460

34
- S ̈arndal, C.-E., B. Swensson, and J. Wretman (2003) _Model Assisted Survey Sampling_ (Springer)

- Schreiner, M. (2014a) ‘How do the poverty scorecard and the PAT differ?’ _mimeo_

- (2014b) ‘The process of poverty-scoring analysis.’ _mimeo_

- Sitter, R. (1997) ‘Variance estimation for the regression estimator in two-phase sampling.’ _Journal of the American Statistical Association_ 92, 780–787

- Stifel, D., and L. Christiaensen (2007) ‘Tracking poverty over time in the absence of comparable consumption data.’ _World Bank Economic Review_ 21(2), 317–341

- Tamhane, A.C. (1978) ‘Inference based on regression estimator in double sampling.’ _Biometrika_ 65(2), 419–427

- Tarozzi, A. (2007) ‘Calculating comparable statistics from incomparable surveys, with an application to poverty in India.’ _Journal of Business & Economic Statistics_ 25(3), 314–336

- (2011) ‘Can census data alone signal heterogeneity in the estimation of poverty maps?’ _Journal of Development Economics_ 95(2), 170–185

- Tarozzi, A., and A. Deaton (2009) ‘Using census and survey data to estimate poverty and inequality for small areas.’ _Review of Economics and Statistics_ 91(4), 773–792

- van Veelen, Matthijs, and Roy van der Weide (2008) ‘A note on different approaches to index number theory.’ _American Economic Review_ 98(4), 1722–1730

- Yoshida, N., R. Munoz, A. Skinner, C. Kyung-eun Lee, M. Brataj, W. Durbin, and D. Sharma (2015) ‘Swift data collection guidelines version 2.’ mimeo, The World Bank

# **A Proofs**

**Proof of Theorem 6** Letting _k_ = _K_ (or _r_ = 1) and _θ_<sup>ˆ</sup><sup>_I_</sup> = _θ_<sup>ˆ</sup> in the proof of Theorem 10, we obtain the proof of Theorem 6. □

35
<!-- Start of picture text --> y y -yE<br>; =y [ - r | -y-E—— ><br>J - ( j e (——) -<br>-- ¥ —| r | [- ]<br>[—|<br>[—<br><!-- End of picture text -->
where **0** _L_ is an _L_ -vector of zeros.

By the definition of _Mg_ , we have: Applying eqs. (28) and (29) to Theorem 10, we obtain eq. (6).

**Proof of Theorem 14** By the definition of _Y_<sup> ̄</sup> , the following holds:

By Assumptions 1 and 2, Lemma 13, and the Law of Iterated Variance, we have:

**Proof of Lemma 17** The results follows directly from the fact that _ρ_ = _τ_ when _wcc_ = 0 and _wh_ = 0 and the fact that _ρ_ is an increasing function of _wcc_ and _wh_ for all _τ_ . The latter is established in Lemma 16. □

# **B Details of the variance and cost parameter estimates**

To compute _ρ_ in eq. (22), we need a realistic set of values for the following parameters: _τ_ , _c_ , _wc_ , and _wh_ . All these values are context-dependent. Further, _wc_ , and _wh_ are also dependent on the choice of covariates and poverty line. However, to see the potential benefits of double sampling, it is useful to have a reasonable range of values that the designers of surveys may encounter. Therefore, we compiled the estimates of these parameter values from various sources to obtain 37
a plausible range for each of these parameters.

# **Some empirical values of** _τ_

Beegle et al. (2012) consider the cost implications for various types of questionnaire in Tanzania. For example, Beegle et al. (2012, Table 10) report that four households can be interviewed by an interviewer per day. Assuming that each interviewer interviews for eight hours a day, it takes 8 _·_ 60 _/_ 4 = 120 minutes to complete a survey with recall consumption model.

The same study also reports that it takes on average 41 minutes to complete a short consumption module with 17 most important items (Beegle et al., 2012, Figure 1). Therefore, the proportion of time that is spent to collect consumption data is 41 _/_ 120 _≈_ 0 _._ 34. If we assume that the data entry cost is roughly proportionate to the time needed for data collection and that there is no other cost items, we have _τ ≈_ 1 _−_ 0 _._ 34 = 0 _._ 66 in this case.

This estimate may be close to the upper bound of _τ_ . The average time to complete a recall consumption module is longer when a long questionnaire (with 58 items) is used and when a longer recollection period is adopted, even though the difference is modest. When the personal diary format is used, the cost of collecting consumption data can be much higher because it involves frequent visits to the household, As reported in Beegle et al. (2012, Table 10), the number of interviews that can be performed per day may be as low as 0.35. Assuming that the time required to complete the non-consumption component remains the same, the value of _τ_ when personal diary format is used is _τ ≈_ 0 _._ 66 _·_ 0 _._ 35 _/_ 4 = 0 _._ 06.

It should be noted that Beegle et al. (2012) only consider a relatively short questionnaire with a complete consumption module and not a large multi-topic survey. Therefore, when we consider a typical Living Standard Measurement Survey, the value of _τ_ may be higher as the weights of non-consumption modules become more important. On the other hand, if we are only concerned with getting an accurate estimate of poverty, _τ_ could be lower because we typically need a small fraction of non-consumption modules to predict consumption.

In Bangladesh, Ahmed et al. (2014) adopt the assumption that five enumerators have to stay for two weeks to complete the survey in one primary sampling unit (PSU) to estimate the cost of conducting Household Income Expenditure Survey in Bangladesh. They further assume that 38
two persons engage in collecting consumption data and three persons collect non-consumption data for two weeks in one PSU. Under these assumptions, we have _τ_ = 3 _/_ 5 = 0 _._ 6. Given these examples, we use _τ_ = _{_ 0 _._ 06 _,_ 0 _._ 36 _,_ 0 _._ 66 _}_ as a plausible set of values to consider.

# **Some empirical values of** _c_

Compared with _τ_ , there are more studies that provide us with some empirical values of _c_ . For Demographic and Health Surveys, Aliaga and Ren (2006, Table 3.1) present an estimate of _c_ based on past eight surveys. The value varies from 10 in Cambodia and Uganda to 52 in Togo. For consumption surveys, we are not aware of a study that provides cost comparisons from multiple surveys. Therefore, we derive the estimates of _c_ from existing studies.

Let us start with Pettersson and Sisouphanthong (2005, p.275), who provide the cost ratio, or the ratio of the cost of adding a PSU to the cost of adding a household, for the third Lao Expenditure and Consumption Survey (LECS-3) conducted in 2002-2003. This ratio is nothing but _c_ in our notation. They report _c_ = 3 _._ 9 in urban areas and _c_ = 6 _._ 1 for rural areas. The lower cost of _c_ in urban areas reflects the lower cost of travel to move between primary sampling units in the urban areas.

A few cautions are in order here. First, travel costs and field allowances are included in the calculation of _c_ but the permanent staff salaries are excluded. As Pettersson and Sisouphanthong (2005) claim, the cost ratio may be affected only slightly by the omission of salaries as the omission will have rather similar effects on both the denominator and numerator of the ratio.

Second, as noted by Pettersson and Sisouphanthong (2005), the cost ratios in their study are rather low. This reflects the fact that the survey required considerable time for interview and follow-up per household over the month when the interviewer-supported diary method was used. Therefore, when a simpler consumption module is used, _c_ is likely to be higher.

Next, we also calculated _c_ based on the survey costs reported in Humphreys (1979, p.396) for the Ada Baseline Survey (ABS) conducted in rural Ethiopia. Their first stage sample consists of 87 administrative localities and costs $5,547, whereas 632 observations were in the second stage costs at the cost of $5,899.<sup>16</sup> Therefore, we have an estimate of _c_ as follows: 16In addition, there were fixed costs of $3,589. Humphreys (1979) use a slightly different cost function for their analysis, but their cost model reduces to ours when we ignore the terms involving square roots.

39
_c_ = (5547 _/_ 87) _/_ (5899 _/_ 632) _≈_ 7.

The numbers obtained above are calculated from a highly aggregated budgetary figures. Therefore, we consider the generic, all-inclusive budget for a one year, 3,200-household living standards survey presented in Grosh and Mu ̃noz (1996, Table 8.2). While the numbers are hypothetical, they are meant to be used to create a prototype budget and may well serve our purpose to create a ballpark figure. We assume that each cluster has 16 households as with a majority of Living Standards Measurement Study (LSMS) surveys reviewed in Table 4.1 of Grosh and Mu ̃noz (1996), which implies that there are 200 clusters in this hypothetical survey.

There remain a challenge in deriving an estimate of _c_ because we need to assign each cost components to (i) variable costs proportionate to the number of clusters, (ii) variable costs proportionate to the number of households, and (iii) the fixed costs. We decided to assign all costs relating to “travel allowance” and vehicles, fuel, and car maintenance to (i) variable costs proportionate to the number of clusters, which amounts to $345,880 for 200 clusters. For (ii) variable costs proportionate to the number of households, we include all the costs relating to “base salaries” and printing of questionnaire as well as “materials” other than vehicles, fuel, and car maintenance, which in total amount to $475,150 for 3,200 households. The remaining costs including “consultancy and travel” and “other” are taken as (iii) fixed costs. Based on these classifications, an estimate of _c_ can be obtained as follows: _c_ = (345880 _/_ 200) _/_ (475150 _/_ 3200) _≈_ 12.

We have also collected similar budgetary information for several household surveys through personal communications. We then computed _c_ in a similar manner. Table 3 summarizes all the estimates of _c_ we have obtained using the budgetary information we obtained from personal communications. Based on this table, we choose to use the following values of the cost ratio: _c ∈{_ 4 _,_ 16 _,_ 64 _}_ .

Note that the cost calculations are based on average cost. Because the actual cost function for a survey depends on the logistical arrangement and is in general not exactly equal to eq. (16), the marginal and average costs are likely to differ. For example, we would not need to incur additional training cost to just add one more household to the sample. However, if we scale up the survey significantly, it is likely that we need to increase most of the cost components, such as the costs of personnel, travel, and materials. Therefore, the calculations of _c_ provided in Table 3 should be taken as approximations.

40
|Source|Aliaga and Ren (2006)<br>Aliaga and Ren (2006)<br>Aliaga and Ren (2006)<br>Aliaga and Ren (2006)<br>Aliaga and Ren (2006)<br>Aliaga and Ren (2006)<br>Aliaga and Ren (2006)<br>Aliaga and Ren (2006)<br>Pettersson and Sisouphanthong (2005)|Pettersson and Sisouphanthong (2005)<br>Humphreys (1979)|Grosh and Mu ̃noz (1996)|Personal communication|Personal communication<br>Personal communication|Personal communication|
|---|---|---|---|---|---|---|
|Survey|Demographic and Health Survey<br>Demographic and Health Survey<br>Demographic and Health Survey<br>Demographic and Health Survey<br>Demographic and Health Survey<br>Demographic and Health Survey<br>Demographic and Health Survey<br>Demographic and Health Survey<br>Third Lao Expenditure and Consumption Survey|Third Lao Expenditure and Consumption Survey<br>Ada Baseline Survey|Generic LSMS|Malawi Third Integrated Household Survey 2010/11|Nigeria General Household Survey 2012/13<br>2011 National Survey on Household Living Conditions and Agriculture|Household Budget Survey 2012|
|_c_|10<br>10<br>12<br>12<br>15<br>27<br>48<br>52<br>3.9|6.1<br>7|12|39|24<br>31|10|
|ntry/Area|bodia<br>nda<br>an<br>opia<br>i<br>key<br>kina Faso<br>o<br>s-Urban|s-Rural<br>opia|_−_|awi|eria<br>er|nia|
|u|m<br>a<br>d<br>hi<br>it<br>r<br>r<br>g<br>o|o<br>hi|_−_|al|g<br>g|to|
|Co|Ca<br>Ug<br>Jor<br>Et<br>Ha<br>Tu<br>Bu<br>To<br>La|La<br>Et|_−_|M|Ni<br>Ni|Es|

41
# **Some empirical values of** _wc_ **and** _wh_

To consider the values of _wc_ and _wh_ in a realistic setup, we take several prediction models and poverty lines used in small-area estimation or survey-to-survey imputation in several countries. For Cambodia, we adopt the consumption model and poverty line used in a small-area estimation project detailed in Ministry of Planning and United Nations World Food Programme (2002) and Fujii (2006). This project uses the Cambodia Socioeconomic Survey 1997. In each of the three strata (Phnom Penh, Other Urban and Rural), a separate consumption model and a separate poverty line are used. Our point estimates of _β_ (unreported) are slightly different from those used in this project because the heteroskedasticity of _ech_ is allowed for in the former and the estimation method is slightly different as a result. However, the difference is not important for our purpose because we are only interested in the plausible range of _wc_ and _wh_ .

For Malawi, Niger, and Tanzania, we used a prediction model and national poverty line from an independent survey-to-survey imputation project.<sup>17</sup> The datasets used are Malawi Third Integrated Household Survey 2010-2011 for Malawi, l’Enquˆete Nationale sur les Conditions de Vie des M ́enages et l’Agriculture 2011-12 for Niger, and Tanzania National Panel Survey 201011 for Tanzania. All these surveys are a part of the Living Standard Measurement Surveys by the World Bank.

Table 4 provides a summary of the estimates of _wc_ , _wh_ , _c_ ̃, _ψ_<sup> ̃</sup> , and _α_ for various surveys. The first two columns report the basic information about the survey such as the number of observations ( _n_ ) and number of clusters ( _K_ ). The next two columns show that the values of _wc_ and _wh_ vary substantially across countries. Based on this, we use _wc ∈{_ 0 _._ 6 _,_ 1 _._ 2 _,_ 1 _._ 8 _}_ and _wh ∈{_ 0 _._ 4 _,_ 1 _._ 2 _,_ 3 _._ 6 _}_ . The next two columns report the implied travel cost _tildec_ under the assumption of optimal sampling for the sample direct estimator and the ratio _ψ_<sup> ̃</sup> of data collection costs between the sample direct and single-sample prediction estimators with the implied travel cost. As shown in the last column, the ratio _α_ of variances between the cluster- and householdspecific error terms satisfies eq. (9), which is the condition for the approximation we use.

> 17Specific details, including the model specifications used, are available upon request.

42
Table 4: Summary of the estimates of _wc_ <u>,</u> _wh_ <u>, ̃</u> _c_ <u>,</u> _<u>ψ</u>_<sup> ̃</sup> <u>, and</u> _α_ .

|Country|_n_|_K_|_wc_|_wh_| ̃_c_<sup>_†_</sup>| ̃_ψ_|_α_|
|---|---|---|---|---|---|---|---|
|Cambodia (Phnom Penh)|1200|120|1.141|3.600|15|0.945|0.073|
|Cambodia (Other Urban)|1000|100|0.961|1.357|26|0.977|0.013|
|Cambodia (Rural)|3810|254|1.809|1.999|81|0.965|0.199|
|Malawi|12239|768|0.612|0.667|67|0.948|0.268|
|Niger|3961|270|1.415|0.761|117|0.983|0.201|
|Tanzania|3272|349|0.708|0.990|22|0.961|0.123|

_†_ In Malawi, Niger, and Tanzania, there are some variations in the cluster size. Therefore, we use the average cluster size ( _K_<sup> ̄</sup> _≡ n/J_ ) in eq. (26) to derive ̃ _c_ . In Cambodia, all the cluster have exactly the same number of households.

43