metadata: authors: 0: author: Ahmed,Faizuddin
1: author: Dorji,Cheku
2: author: Takamatsu,Shinya
3: author: Yoshida,Nobuo
count: Bangladesh
docna: 0: docna: Hybrid survey to improve the reliability of
            poverty statistics in a cost-effective manner
docty: Policy Research Working Paper
subtopic: Labor & Employment Law,State Owned Enterprise Reform,Public Sector Administrative & Civil
            Service Reform,Urban Poverty,Urban Partnerships & Poverty,Inflation,Economic Services to the Urban Poor,Inequality,Peri-Urban Communities,Democratic Government,Economics and Finance of Public
            Institution Development,Public Sector Administrative and Civil
            Service Reform,De Facto Governments
teratopic: Poverty Reduction,Macroeconomics and Economic Growth,Private Sector Development
abstracts: cdata!: This paper studies the benefits, in
            terms of reliability and frequency of poverty statistics, of
            conducting a hybrid survey that collects non-consumption
            data from all surveyed households and consumption data from
            only a small subsample. Collecting detailed consumption or
            income data for the purpose of estimating poverty is costly
            and many low-income countries cannot afford to carry out
            such surveys on a regular basis. One option is to collect
            only non-consumption data and use consumption models
            developed from a previous round of household survey data to
            project poverty data. Although this approach is
            cost-effective because collection of non-consumption data is
            much cheaper than collection of consumption data, it is
            vulnerable to a structural change between the current and
            previous household surveys and might produce poverty
            estimates that are not comparable with the previous ones.
            Instead, the hybrid approach creates consumption models from
            a subsample of the current survey and applies them to the
            entire survey to project consumption data for all households
            in the sample. This paper examines the hybrid approach with
            data from the Bangladesh Household Income Expenditure
            Surveys of 2000 and 2005. Improvements in accuracy are
            achieved even with subsamples of just 320 or 640 households.
            Budget simulations confirm that the additional cost of
            collecting consumption data for such small subsamples is minimal.
geo_reg_key: 517198,517212,517191
repnme: repnme: Hybrid survey to improve the reliability of
            poverty statistics in a cost-effective manner
display_title: Hybrid survey to improve the
            reliability of poverty statistics in a cost-effective manner
listing_relative_url: /research/2014/06/19641283/hybrid-survey-improve-reliability-poverty-statistics-cost-effective-manner
topicv3: State Owned Enterprises m1326323 721,Poverty m1326687 695,Inflation m1326709 0870F4727ACED1CF8525815C00471347,Urban Poverty m1327411 710,Household Surveys m1504071 2761
publishtoextweb_dt: 2014-06-01T00:00:00Z
historic_topic: Poverty Reduction,Macroeconomics and Economic Growth,Private Sector Development
no_of_pages: 40
author_keys: 0: author_key: 956314
1: author_key: 1108915
2: author_key: 669778
3: author_key: 642421
model_extractions: WPS6909 Policy Research Working Paper 6909 Hybrid Survey to Improve the Reliability of Poverty Statistics in a Cost-Effective Manner _Faizuddin Ahmed Cheku Dorji Shinya Takamatsu Nobuo Yoshida_ The World Bank Poverty Reduction and Economic Management Network Poverty Reduction and Equity Unit June 2014
Policy Research Working Paper 6909

# **Abstract**

This paper studies the benefits, in terms of reliability and frequency of poverty statistics, of conducting a hybrid survey that collects non-consumption data from all surveyed households and consumption data from only a small subsample. Collecting detailed consumption or income data for the purpose of estimating poverty is costly and many low-income countries cannot afford to carry out such surveys on a regular basis. One option is to collect only non-consumption data and use consumption models developed from a previous round of household survey data to project poverty data. Although this approach is cost-effective because collection of nonconsumption data is much cheaper than collection of consumption data, it is vulnerable to a structural change between the current and previous household surveys and might produce poverty estimates that are not comparable with the previous ones. Instead, the hybrid approach creates consumption models from a subsample of the current survey and applies them to the entire survey to project consumption data for all households in the sample. This paper examines the hybrid approach with data from the Bangladesh Household Income Expenditure Surveys of 2000 and 2005. Improvements in accuracy are achieved even with subsamples of just 320 or 640 households. Budget simulations confirm that the additional cost of collecting consumption data for such small subsamples is minimal.

This paper is a product of the Poverty Reduction and Equity Unit, Poverty Reduction and Economic Management Network. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http:// econ.worldbank.org. The author may be contacted at nyoshida@worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# Hybrid Survey to Improve the Reliability of Poverty Statistics in a Cost-Effective Manner

Faizuddin Ahmed, World Bank Cheku Dorji, National Statistics Bureau, Bhutan Shinya Takamatsu, World Bank Nobuo Yoshida, World Bank JEL: I32.

Keywards: Survey to survey imputation, subsample, poverty measurement

## **Acknowledgments**

The authors thank Peter Lanjouw for his thorough review of the previous draft and useful suggestions, which improved the quality of analyses in this paper; Kinnon Scott for her support in designing the pilot survey to test price data collection and the effect of switching from diary- to recall-based approaches in collecting consumption data; and the Bangladesh Bureau of Statistics for their help in designing the sampling of the pilot survey and for providing details of survey logistics to help estimate survey implementation costs. Finally, the authors are grateful for their discussions with Roy van der Weide and Kathleen Beegle, and insightful comments from Salman Zaidi.
# **I. Introduction and Literature Review**

Following the recent economic crisis, which affected both developed and developing countries, policy makers and other stakeholders are demanding more frequent and timely poverty data. During the crisis, food and fuel prices rose sharply and employment opportunities deteriorated substantially. These impacts aggravated the situation of poor and vulnerable groups and likely increased the overall prevalence of poverty in many developing countries. Policy makers and stakeholders need to know to what extent poverty has increased, whether existing poverty alleviation programs are working, and if not, how they can be improved. However, for many developing countries, it is not yet (and may never be) known how much the prevalence of poverty increased simply because poverty statistics were not available during and in the immediate aftermath of the crisis.

Why do so many developing countries fail to publish poverty data regularly and frequently? Producing good quality poverty data requires collecting detailed household expenditure (or income) information at the household level. This is a resource-intensive and time-consuming process. To overcome these constraints, conducting rapid monitoring surveys with highly simplified survey questionnaires and/or shortened survey periods is sometimes suggested. However, many analysts are concerned about the quality of poverty data from such rapid monitoring surveys in terms of their comparability to official poverty data from large-scale full-fledged household surveys. If the data are not comparable, tracking poverty becomes extremely tricky and policy makers and other stakeholders cannot be sure how much (and sometimes even in what direction) poverty has changed.

Imputation procedures offer a potentially cost-effective solution for improving the frequency and comparability of poverty data.<sup>1</sup> The idea is simple: suppose consumption or income data are not collected in a particular year, but nonconsumption data from other surveys are available, then consumption or income data can be imputed into the nonconsumption/income data set using imputation models calibrated on consumption data collected previously. Because we are using data already available, there is no data collection cost (see, for example, Stifel and Christiaensen [2007] or Douidich et al. [2013]). Even if a survey is necessary to collect the nonconsumption/income data, its cost is likely to be low compared to that of a full-fledged consumption survey. Ravallion (1996), one of the early advocates of the imputation approach, experimented with how well household consumption expenditure can be estimated from information on housing conditions and subjective well-being, which can be collected quickly and cheaply.

For the imputation approach to work, it is important to assess whether imputation models can accurately predict household consumption/income and produce comparable poverty data. The imputation models are calibrated in a year when both household consumption data and nonconsumption data are available, and then applied to impute household consumption data for years when only the nonconsumption data are available.<sup>1</sup> A potentially serious problem is that there is no guarantee that imputation models will accurately reflect consumption patterns in those periods when only nonconsumption data are available. For example, ownership of cell phones is a good predictor of household wealth, but its relationship with total household expenditure has been changing rapidly in many Sub-Saharan African countries as cell phone coverage increases and cell phone fees decline. Ten years ago, ownership of cell phones was a good predictor of extreme wealth; today such phones are prevalent among the lower-middle-income classes and even among the poor. Using imputation models calibrated 10 years ago with cell phones as covariates would likely underestimate present-day poverty.

Because of these considerations, there is ongoing research and debate among analysts on methods to ensure that imputed poverty data are reliable and comparable. Stifel and Christiaensen (2007) provide

> 1 The imputation method can be used for many other purposes as well. It is used for poverty mapping (e.g., Elbers, Lanjouw and Lanjouw 2003), restoring comparability of consumption data over time (e.g., Deaton and Dreze 2002, Kijima and Lanjouw 2003, Allwine et al. 2013, Revilla et al. 2010), and creating synthetic panel data for vulnerability analysis (Dang and Lanjouw 2013).

2
theoretical guidance regarding variables to be included in imputation models to ensure comparability and reliability of imputed poverty data. They recommend including covariates that change over time, but call for excluding variables whose rates of return are likely to change markedly in the face of evolving economic conditions. This argument makes sense in theory, but it is difficult to identify which variables would satisfy these conditions. Stifel and Christiaensen (2007) included several household durables in their imputation models, but Harttgen, Klasen and Vollmer (2012) criticized this decision because of the so-called “asset drift” effect—where the pace of improvement in asset ownership is much faster than the pace of income growth. As a result, the relationship between asset ownership and income or consumption could well be changing over time. In the end, which variables satisfy the recommendations of Stifel and Christiaensen (2007) is intrinsically an empirical question. At the same time, ascertaining empirically how best to proceed is difficult because actual household consumption data are needed. Remember, imputation methods are being employed precisely because actual household consumption data are not available.

Christiaensen et al. (2012) examined some of the arguments raised by Stifel and Christiaensen (2007) through an experiment. They collected multiple rounds of cross-sectional data including both household consumption and nonconsumption variables from four developing countries, namely rural China, Kenya, the Russian Federation, and Vietnam. They created imputation models from one round and imputed household consumption expenditure into other rounds that also had actual consumption data. They illustrated with their four case studies that imputation models can reproduce poverty statistics generated from directly collected consumption data. The results support the notion that imputation models constructed from past surveys can predict household consumption data of future rounds. Such results are encouraging and give us a certain level of confidence in the stability of imputation model coefficients over time. However, this exercise is still experimental. Validation is possible only in those settings where imputed estimates can be compared to actual estimates. In practice, the imputation approach needs to be used in those circumstances where actual consumption data are not available, and therefore, analysts must be willing to assume that the imputation model is accurately predicting household expenditure and poverty rates.

This paper explores a hybrid approach—implementation of a survey that collects a large sample of nonconsumption data, which can be collected easily and cheaply, but also collects a small subsample of both consumption and nonconsumption data. Collection of household consumption data in the subsample provides several advantages. First, the data can be used to estimate poverty rates directly. Because of the small size of the subsample, this direct estimation will not be precise: sampling errors can be significant. However, direct estimation of the poverty rate from the subsample might still provide a good ball park figure. Second, using the subsample, which includes both consumption and nonconsumption data, we can create imputation models and apply them to the full (and larger) sample to impute consumption for all sample households. This hybrid approach will result in an overall poverty estimate with the sampling error associated with the large household survey and a modeling error associated with our having imputed consumption into the large survey. Encouragingly, literature on small area estimation (for example, Elbers, Lanjouw, and Lanjouw [2003], hereafter ELL) suggests that model errors need not be excessive if proper care is taken in specifying and estimating the imputation model.

This hybrid approach can address the problems faced by both Stifel and Christiaensen (2007) and Christiaensen et al. (2012). Both papers create imputation models from past data and apply them to the latest household survey data. As discussed above, we cannot test whether the imputation models are still valid or reflect the relationship among household consumption and other correlates. In the hybrid approach, the imputation models are created from the subsample of the latest household survey, and so, by construction, the models reflect the latest relationship among household consumption and other correlates.

Despite these advantages, the hybrid approach is not perfect. First, collecting consumption data is costly, even if it is collected only for a subsample. We know that the larger the subsample size, the more accurate 3
the poverty estimates will be. But, we also know that as the subsample size increases, the cost of implementing the hybrid survey also rises. So, a key policy question is how much will a reduction in the size of the subsample increase the standard errors (SEs) of the poverty estimates. Furthermore, consumption data need to be comparable over time.

Second, the hybrid approach calls for regular re-estimation of the underlying imputation models. Developing reliable imputation models has become simpler since the World Bank Development Research Group created the PovMap 2 software for this purpose, but it remains a not insignificant task and a formidable challenge in those settings where statisticians have limited experience with these methods.

Third, the hybrid approach will produce the current value of imputed expenditure; therefore, some adjustments for intertemporal and spatial price differences need to be made. Such price adjustments often use unit values from household surveys, but the reliability of unit values will be limited since the size of subsample is small. In contrast, the approach proposed by Christiaensen et al. (2012) is attractive because (i) imputation models will be estimated only for the year in which a big household survey with a full sample of consumption data was carried out; (ii) imputed expenditure/income is already adjusted to prices of the big household survey year and, as a result, we do not have to make any additional price adjustments for the following years; and (iii) except for the big household survey year, we do not have to collect consumption data at all.

The objective of this paper is to compare the following three different poverty estimation approaches: (i) estimating poverty rates from consumption data in a subsample (“Subsample Direct Estimation”); (ii) estimating poverty rates using household expenditure imputed from models created from a subsample (“Subsample Imputation”); and (iii) estimating poverty rates using imputed household expenditure based on models created from a previous round of a large household survey (“Full Sample Imputation,” used by Christiaensen et al. [2012]).

This paper conducts the above analysis using the 2000 and 2005 Bangladesh Household Income and Expenditure Survey (HIES) data. Bangladesh was chosen because Bangladesh Bureau of Statistics expressed interest on the subsample imputation approach and also offered help to carry out additional surveys to see the effects of slight modification of survey modules on survey implementation costs. The 2000 and 2005 HIES data were chosen although a more recent round of HIES data (the 2010 HIES) is available because the poverty lines are slightly modified for the 2010 datasets. The modification of poverty lines in addition to price adjustments usually causes additional challenge to “Full Sample Imputation.” Therefore, the 2000 and 2010 HIES were selected. However, its robustness against changes in methodology is also can be seen as an advantage of the subsample approach.<sup>2</sup> This paper also studies to what extent the limited sample size affects price adjustments, poverty lines, and poverty estimates. For situations where unit values from a subsample are too noisy to produce reliable poverty estimates, we also evaluated another way of conducting price adjustments using price data from an independent price survey. An issue here is that if unit values and prices are too different, poverty lines updated by price data would be very different and produce poverty estimates that are not comparable to the previous official poverty rates, which were estimated with price adjustments based on unit values. This paper examines whether differences between prices and unit values, both collected in a pilot survey, are statistically significant.

Finally, this paper examines the cost implications of collecting consumption data for a subsample, as well as the reliability of price data collected from price surveys. Collection of consumption expenditure data for a subsample might be useful to improve and ensure the statistical accuracy of imputed expenditure data; however, if it is costly, annual collection of such data might not be a realistic option for developing countries. We therefore examine the costs of collecting consumption data for a subsample after reviewing the current data collection process adopted by the Bangladesh Bureau of Statistics (BBS).

> 2 This insight was drawn from discussions with Salman Zaidi.

4
This paper is not the first paper that evaluates the statistical precision and cost-effectiveness of the hybrid approach. A Fujii and van der Weide (2013) study examined the hybrid approach in a general environment. Their paper focused on the statistical properties of indicators derived from the hybrid approach and, after considering the implementation cost of the hybrid approach, derived the optimal sample sizes of both a large survey and its subsample given a total implementation cost. There are several differences between this paper and Fujii and van der Weide (2013). First, this paper not only estimates the statistical property of the hybrid approach, but also compares it with a simpler survey-to-survey imputation method, while Fujii and van der Weide (2013) focus on the statistical property of the hybrid approach. Second, this paper evaluates the accuracy of price adjustments—as mentioned, this can be a big obstacle in obtaining precise and reliable poverty statistics using the hybrid approach. Third, survey cost estimation requires detailed information on survey logistics, which can differ from country to country. By focusing on one country, this paper produces more precise estimates of survey implementation costs.

Finally, this paper focuses on The following section presents the three proposed approaches to increase the frequency of poverty data in Bangladesh and the methods used to evaluate these approaches. Section III presents the key results of the performance of each approach. Section IV discusses the effects of updating inflation rates and poverty lines using a subsample or an independent price survey. Cost is an important factor in survey implementation, and section V assesses cost implications each approach. Section VI summarizes the study findings and their validation of the usefulness of the hybrid survey approach.

# **II. Three Approaches for High-Frequency Poverty Measurement in Bangladesh**

This paper examines three low-cost approaches to increase the frequency of poverty data using Bangladesh 2000 and 2005 Household Income Expenditure Surveys (HIESs) data: Subsample Imputation, Subsample Direct Estimation, and Full Sample Imputation.

The Subsample Imputation approach estimates consumption models from a subsample of a hybrid survey, where both consumption and nonconsumption data are collected. It then uses the models to impute household expenditures for all households in the sample, even though there are consumption data for only the households in the subsample. Poverty rates are estimated from the imputed expenditures.

The advantage of this approach is that the consumption models are estimated from current year data. So, if a crisis hits a country and changes consumption patterns, the consumption models are also updated. The SE of this estimation includes both imputation errors and sampling errors, but the latter can be reduced significantly by ensuring a sufficiently large sample size of the overall hybrid survey. Because collection of the nonconsumption data is significantly cheaper, the cost of implementing such a survey is far lower than implementing a HIES. The imputation errors are expected to be non-negligible, although the literature on poverty mapping suggests that with proper care, imputation errors can be small, even when consumption models are estimated with a very small sample size.

The Subsample Direct Estimation approach simply estimates poverty statistics from the consumption data in the subsample of a hybrid survey. This approach is easier than the Subsample Imputation approach because it does not estimate imputation models and simulate poverty rates. However, there is a challenge, because the poverty estimates will likely come with higher SEs than the official poverty estimates due to the small sample size.

The Full Sample Imputation estimates consumption models using the previous round of household survey, like the HIES, where both consumption and nonconsumption data are collected, and imputes household expenditure in the future round where only nonconsumption data are collected. Like the Subsample Imputation, this approach will also face two types of errors—imputation and sampling errors.

5
The sampling errors can be reduced by increasing the size of the sample collected in the future round. The imputation errors can also be low since the models are estimated using a full sample. However, this approach requires the assumption that underlying parameter estimates in the imputation models remain stable over time. This could become controversial if a large structural change or economic shock occurs between a household survey with consumption data and a survey without consumption data. In the face of such shocks, the consumption models estimated from the household survey with consumption data might not reflect current consumption patterns accurately.

# **Data and “Full Sample Direct Estimation” Approach as a Benchmark**

Bangladesh has been conducting a nationally representative multitopic household survey every five years since 1995; poverty statistics are available every five years. Both the 2000 and 2005 HIESs have a very similar questionnaire and follow the same sampling design. Food consumption data were collected using daily diary forms for 200 items and weekly for 30 items. Nonfood consumption data were collected using monthly and yearly recall forms for about 50 and 150 nonfood items, respectively. Enumerators stayed in primary sampling units (PSUs) for around two weeks, and visited each household every two days to collect information on daily food consumption for two days. Also, enumerators visited the same households for two more days to collect data on nondaily food and nonfood consumption. The 2000 HIES has 14 strata and includes 7,440 households, while the 2005 HIES has 16 strata and includes 10,800 households. The number of PSUs ranges across strata. Each PSU had 10 or 20 households in 2000, and exactly 20 households in 2005. In total, the 2000 and 2005 HIESs had 442 and 504 PSUs, respectively.<sup>2</sup> Both the 2000 and 2005 HIESs selected PSUs following the probability-proportional-to-size (PPS) sampling while selecting households using the simple random sampling (SRS) from the selected PSUs.

The hybrid survey is constructed from the 2005 HIES data. Subsamples are drawn from the 2005 HIES data and include both consumption and nonconsumption data—procedures for selection of subsamples are explained in the next section. The full sample is the same as the 2005 HIES data, and except for households selected for the subsample, has only nonconsumption data. For the Full Imputation approach, consumption models are developed from the full sample of the 2000 HIES data. The models are applied to impute household expenditures into the entire 2005 HIES sample using nonconsumption data.

All three approaches produce poverty headcount rates at the national level and for urban and rural areas in 2005. A benchmark for all three approaches is the poverty statistics estimated from actual consumption data of the 2005 HIES data. This benchmark is the Full Sample Direct Estimation approach. As shown in table 1, the poverty headcount rate is 40.0 percent for the entire country, and 43.8 and 28.4 percent in rural and urban areas, respectively. The SEs are 1.1 percent for the entire country, and 1.3 and 1.9 percent, respectively, in the rural and urban areas.

**Table 1: Poverty Headcount Rates Using Full Sample of 2005 HIES**

|Full sample|Povert|yrates|95%|CI|Range of CI|
|---|---|---|---|---|---|
|National|40.0|(1.1)|37.8|42.2|4.5|
|Rural|43.8|(1.3)|41.3|46.3|4.9|
|Urban|28.4|(1.9)|24.6|32.1|7.5|

_Source_ : Authors’ estimations using 2005 HIES data. _Note_ : Parentheses are SEs. CI = confidence interval.

# **Subsample Selection in a Hybrid Survey**

To preserve the sampling design in the 2005 HIES, we followed its sampling as much as possible. For example, like the 2005 HIES, PSUs of a subsample were selected following PPS sampling (see Lohr 6
[1999, 108] for more details). For selection of households in select PSUs, we used the SRS and selected 10 households per PSU. We reduced the 20 households in the 2005 HIES to 10 to reduce the sampling error. To see the effects of sample size, this analysis used two different sizes for the subsample: 320 and 640. Note that increasing the size of the subsample increases the cost of data collection, but likely improves the precision of poverty estimates as well.

|**Table 2: Sampling Schemes and Biases in Poverty Estim**<br>|**ates**||
|---|---|---|
|Sampling scheme|Pover|tyrate|
|1. PPS to select 4 PSUs from each stratum and PPS to<br>select 10 households each|42.2|(2.4)|
|2. PPS to select 4 PSUs from each stratum and SRS to<br>select 10 households each|40.4|(3.7)|
|3. PPS to select PSUs of different size from each stratum<br>(64 PSUs in total)and SRS to select 10 households|40.2|(2.9)|
|_Source_: Authors’ estimations from 640 households from the 2005 HIES.<br>_Note_: SEs are inparentheses.|||

The process of selecting subsamples impacts the results. If we use actual consumption data from the 2005 HIES, the national poverty headcount rate is 40 percent (table 1). If we select the same number of PSUs (or four PSUs) from each of all 16 strata and select 10 households by PPS, then the poverty rate from the subsample is biased upward (table 2). If we keep the same number of PSUs from each stratum, but select households by SRS, then the national poverty rate is close to the true rate. Finally, if we use a different number of PSUs from each stratum based on population, then the national poverty estimate is very close to the true estimate and the SE also declines. This analysis used the third approach in table 2 to select a subsample from the 2005 HIES.

# **Imputation Process for Subsample**

The Subsample Imputation and Full Sample Imputation both involve imputation of household expenditures from an ancillary survey. The imputation process follows methodology developed in ELL (2003), which has two stages: first, a model of log per capita consumption expenditure of household _h_ (lnyh) is estimated in a subsample of the hybrid survey (in the case of Subsample Imputation) or the 2000 HIES data (in case of Full Sample Imputation). The model can be defined as where α is an intercept, Xch′ is the vector of explanatory variables for household _h_ and location _c_ , and β is the vector of regression coefficients, Z<sup>′</sup> is the vector of location specific variables, γ is the vector of coefficients, and uch is a residual. This residual is decomposed into two independent components: uch = ηc + εch with a cluster-specific effect, ηc, and a household-specific effect, εch . This structure allows both a location effect—common to all households in the same area—and heteroskedasticity in the household-specific errors. The location variables can be at any level— _zila, upazila_ , union, _mauza_ , and village (in the case of Bangladesh)—and can be drawn from any data sources that include all locations in the country. All parameters regarding the regression coefficients ( α, β, γ ) and distributions of the disturbance terms are estimated by feasible generalized least squares. In the second stage, poverty estimates and their SEs are computed. There are two sources of errors involved in the estimation process: errors in the estimated regression coefficients (α�, β̂, γ�) and in the estimated residuals, both of which affect poverty estimates and the level of their accuracy. ELL (2003) propose a way to properly calculate poverty estimates as well as their SEs while taking into account these sources of bias. A simulated value of expenditure for each census household is calculated with predicted log expenditure α�+ Xh′ β̂ + Z′γ� and 7
random draws from the estimated distributions of the residuals, ηc and εh. For each subsample, these simulations are repeated 100 times. The mean across the 100 simulations of a poverty statistic provides a point estimate of the statistic for the subsample. To respond to apparent differences in consumption patterns between urban and rural areas, separate consumption models for urban and rural areas are estimated.

# **Estimation of Poverty Headcount Rates and Their Standard Errors: Subsample Imputation and Subsample Direct Estimation**

With either the Subsample Imputation or Subsample Direct Estimation, subsample selection can have a significant impact on poverty estimates. To see the statistical performance, we randomly drew subsamples and repeatedly estimated poverty rates with the subsamples. For the Subsample Direct Estimation approach, we estimated poverty numbers from each subsample and used the mean of poverty rates as the estimate of poverty rate and the standard deviation as the SE.

For the Subsample Imputation approach, we estimated consumption models from each subsample and estimated poverty rates using household expenditure imputed from the models. For the first 30 subsamples, we estimated the consumption models using PovMap2 software, and so were able to follow ELL’s method exactly. However, although most results become stable after the exercise is repeated 30 times, there remains a concern on the robustness of the findings.

To see the robustness of the results from the 30 observations, we simplified the imputation method slightly and repeated it 1,000 times. Like ELL, we assumed the error structure consists of the cluster (ηc) and household errors (εch). But, although ELL and PovMap2 allow for a flexible error structure, we assumed both cluster and household level errors are normally distributed with zero means.

The variances of the errors were estimated as follows: first, residuals from the ordinary least squares (OLS) estimation, ech, were calculated. Secondly, the empirical cluster effects, ec, were calculated using the residuals within each cluster. The SE of ηc is equal to the standard deviations of the empirical cluster effects (ec). Third, the empirical household effect was defined by the difference between the residuals and the empirical cluster errors, such that εch = ech −ec. Using the obtained two SEs of ηc and εch, the two error terms were randomly produced from two normal distributions and the imputed consumption was calculated using this formula: Such simplifications will increase the SEs of coefficients and model errors.

Finally, although the first 30 simulations trimmed extreme values, the following 1,000 simulations did not control them. Such trimming likely has a small effect on the estimation of poverty headcount rates, although it can have a significant impact on inequality measures, such as Gini coefficients. Because this paper focuses on poverty headcount rates, ignoring trimming is of less concern.

All in all, the 1,000 simulations following the first 30 simulations likely increased the SEs of poverty statistics estimated by the Subsample Imputation approach, but can be used to check whether the results of the first 30 simulations are vulnerable to the small sample errors.

SEs estimated by ELL’s method need to be modified to incorporate sampling errors because the ELL method was originally used to impute consumption for all households in census data. As a result, estimated SEs consist only of modeling errors and do not include sampling errors. However, in both Subsample Imputation and Full Sample Imputation, we imputed consumption for all households in a household survey, and this means that the estimated SEs should include sampling errors that are not originally incorporated into the ELL method. To incorporate sampling errors, we used the SEs from the Full Sample Direct Estimation approach, which included only sampling errors, and added them to the SEs obtained from the ELL method. We assumed zero correlation between the modeling and sampling errors since we could not predict the sign or size of the correlation.

8
# **III. Results**

This section presents the results of the Subsample Direct Estimation, Subsample Imputation, and Full Sample Imputation.

**Table 3: Results from the First 30 Replications, Using Subsamples of 640 Households**

|Levels/Areas|Pover|ty rate|95%|CI|Range<br>of CI|
|---|---|---|---|---|---|
|_Subsample Imput_|_ation_|||||
|National|39.4|(2.0)|35.4|43.4|7.9|
|Rural|43.1|(2.5)|38.3|47.9|9.7|
|Urban|28.2|(3.1)|22.2|34.2|12.0|
|_Subsample Direct_|_Estimation_|||||
|National|40.2|(2.7)|35.0|45.5|10.5|
|Rural|43.8|(3.2)|37.5|50.1|12.6|
|Urban|29.0|(4.5)|20.2|37.8|17.6|

_Source:_ Authors’ estimations using 2005 HIES data. _Note_ : Cumulative means of poverty estimates and SEs (standard deviation of estimates) with 30 replications are shown. Estimates for 30 replications are shown in appendix table A2.2. Numbers in parentheses refer to SEs.<sup>3</sup>

## **Results Using Subsample Imputation and Subsample Direct Estimation**

Table 3 shows the cumulative means of the poverty estimates after 30 replications with a subsample of 640 households, out of which 410 households are rural and 230 are urban. The cumulative means of the 30 replications are presented in the poverty rate column and the standard deviations of poverty rates in 30 simulations are reported as SEs in parentheses.

All cumulative means of both the Subsample Imputation and the Subsample Direct Estimation are close to the true or Full Sample Direct Estimation of poverty headcount rates at the national level as well as for urban and rural areas. For example, the cumulative means of the Subsample Imputation are 39.4 percent at the national level and 43.1 and 28.2 percent for rural and urban areas, respectively. The corresponding full sample direct estimates are 40.0 percent (national), 43.8 percent (rural), and 28.4 percent (urban) (table 1). The cumulative means of the Subsample Direct Estimation are 40.2 percent at the national level, and 43.8 and 29.0 percent for rural and urban areas, respectively. Furthermore, all cumulative means from the Subsample Imputation and the Subsample Direct Estimation are in the 95 percent confidence interval (CI) of the corresponding full sample direct estimates (table 1).

In terms of precision, Subsample Imputation outperforms Subsample Direct Estimation. For example, the SEs of poverty rates at all three levels/areas using Subsample Imputation are 2.0 percent (national), 2.5 percent (rural) and 3.1 percent (urban), which are substantially smaller than those of the Subsample Direct Estimation, which are 2.7 percent (national), 3.2 percent (rural), and 4.5 percent (urban).

Figure 1 plots the cumulative means of poverty estimates using the 640 households in the subsample (410 rural and 230 urban) for each replication. This indicates that both poverty estimates from the Subsample Imputation and Direct Estimation approaches very quickly converge to the “true” poverty rates, or

> 3 All standard errors of poverty estimates except for Full Sample Direct Estimation do not include the sampling error of poverty estimates in the full sample. For more details, see Fujii and van der Weide (2013) and Matloff (1981).

9
poverty rates based on the Full Sample Direct Estimation. But, estimates from Subsample Imputation appear to outperform those of Subsample Direct Estimation, particularly for urban areas.

**Figure 1: First 30 Simulations for Subsample Imputation and Subsample Direct Estimation**

<!-- Start of picture text --> National<br>50 50 Rural (410 households)<br>45 45<br>40 40<br>35 35<br>30 30<br>25 25<br>20 20<br>1 4 7 10 13 16 19 22 25 28 1 4 7 10131619222528<br>Replication Replication<br>Imputation Imputation<br>Direct Direct<br>Full Sample (mean) Full Sample (mean)<br>Full Sample (95% CI) Full Sample (95% CI)<br>50 Urban (230 households)<br>45<br>40<br>35<br>30<br>25<br>20<br>1 4 7 10 13 16 19 22 25 28<br>Replication<br>Imputation<br>Direct<br>Full Sample (mean)<br>Full Sample (95% CI)<br>Cumulative mean headcount ratio, %<br><!-- End of picture text --> _Source_ : Authors’ estimations using 2005 HIES data.

10
# **Reducing Subsample Size**

Reducing the size of the subsample is expected to worsen the precision of the poverty estimates of both the Subsample Imputation and the Subsample Direct Estimation. Reducing the subsample size increases the sampling errors of poverty estimates from the Subsample Direct Estimation. Also, reducing the subsample size likely increases the SEs of poverty estimates from the Subsample Imputation, because the sample size that can be used to estimate consumption models is reduced. As a result, consumption models are more vulnerable to small sample bias, or the so-called “overfitting” problem. Restricting the number of variables is a typical way of minimizing the risk of small sample bias, but such a strategy tends to worsen the model’s fit.

However, reducing the size of the subsample is a cost-effective strategy for survey implementation. As discussed above, collection of consumption data is costly, particularly so in Bangladesh, where enumerators need to visit households seven times in two weeks during their stay in one village. Halving the size of the subsample with detailed consumption data will reduce survey implementation costs dramatically. This point is discussed more later in this section.

Table 4 shows that cumulative means from both the Subsample Imputation and the Subsample Direct Estimation remain close to those of the Full Sample Direct Estimation. It appears that the Subsample Direct Estimation slightly outperforms the Subsample Imputation in terms of the cumulative means as the size of the subsample declines. The SEs of the Subsample Direct Estimation deteriorate rapidly as the size of the subsample declines while SEs of the Subsample Imputation did not. For poverty estimates at the national level, the estimate from the Subsample Imputation remains reasonably accurate and its SE hardly increases. In this sense, if the government is interested in poverty rates at the national level only, reducing the subsample size to 320 households using the Subsample Imputation approach might be an option.

**Table 4: Comparing Results from First 30 Replications**

|Approach|Level/Area|320 Hou|seholds|640 Ho|useholds|
|---|---|---|---|---|---|
||National|39.1|(2.1)|39.4|(2.0)|
|Subsample Imputation|Rural|42.1|(3.1)|43.1|(2.5)|
||Urban|30.0|(3.6)|28.2|(3.1)|
||National|40.3|(3.5)|40.2|(2.7)|
|Subsample Direct Estimation|Rural|44.0|(4.3)|43.8|(3.2)|
||Urban|29.2|(5.9)|29.0|(4.5)|
||National||40.0|(1.1)||
|Full Sample Direct Estimation<sup>a</sup>|Rural||43.8|(1.3)||
||Urban||28.4|(1.9)||

_Source_ : Authors’ estimations using 2005 HIES data.

a. Full Sample Direct Estimation reports poverty rates by using household consumption per capita data from all households included in the 2005 HIES. So, the subsample size does not apply to the numbers. Estimates for 30 replications are available in appendix tables A2.2 and A2.3. Numbers in parentheses are SEs.

11
**Figure 2: Convergence in Poverty Estimates Based on Subsample Imputation and Subsample Direct Estimation**

<!-- Start of picture text --> National Rural (200 households) Urban (120 households)<br>50 50 50<br>45 45 45<br>40 40 40<br>35 35 35<br>30 30 30<br>25 25 25<br>20 20 20<br>1 4 7 10 13 16 19 22 25 28 1 4 7 10131619222528 1 4 7 10 13 16 19 22 25 28<br>Replication Replication Replication<br>Imputation Imputation Imputation<br>Direct Direct Direct<br>Full Sample (mean) Full Sample (mean) Full Sample (mean)<br>Full Sample (95% CI) Full Sample (95% CI) Full Sample (95% CI)<br> headcount ratio, %<br>mean<br>Cumulative<br><!-- End of picture text --> _Source_ : Authors’ estimations using 2005 HIES data.

Figure 2 shows the convergence of the cumulative means of poverty rates as the number of replications increases. The cumulative means of both the Subsample Imputation and the Subsample Direct Estimation appear to converge even with only 30 replications. This gives us a certain level of confidence about the results.

# **Robustness Check for Subsample Imputation and Subsample Direct Estimation for 1,000 Replications**

In the above analysis, even with only 30 replications, the cumulative means appear to converge. This is encouraging. Nevertheless, there is no full guarantee that the above results would hold even if the number of replications increased. As a robustness check, we examined whether results would change dramatically after increasing the number of replications.

However, as mentioned before, increasing the number of replications is not a simple task. Implementing ELL’s methodology is simplified dramatically thanks to development of PovMap 2 software. But, each replication needs to be implemented manually. For example, users of PovMap2 need to select models and run simulations for each replication separately. It is not possible to repeat the selection of subsamples and estimation of poverty headcount rates 1,000 times. Therefore, we simplified the estimation and simulation process significantly so that we can repeat the process 1,000 times (section II provided an exact description of the simplified process).

12
**Table 5: Results of 1,000 Replications—Cumulative Means and Standard Errors**

|Approaches|Areas|First<br>replica|30<br>tions|Following<br>replicat|1,000<br>ions|95% CI from<br>Direct Es|Full Sample<br>timation<sup>a</sup>|
|---|---|---|---|---|---|---|---|
||Subsa|mple size =|640 house|holds||Lower|Upper|
||National|39.4|(2.0)|38.4|(2.4)|37.8|42.2|
|Subsample<br>Imputation|Rural|43.1|(2.5)|41.8|(2.9)|41.3|46.3|
||Urban|28.2|(3.1)|27.9|(3.2)|24.6|32.1|
||National|40.2|(2.7)|40.4|(2.9)|||
|Subsample Direct<br>Estimation|Rural|43.8|(3.2)|44.3|(3.5)|||
||Urban|29.0|(4.5)|28.4|(4.6)|||
||Subsa<br>|mple size =<br>|320 house<br>|holds<br>||||
||National|39.1|(2.1)|38.5|(3.5)|||
|Subsample<br>Imputation|Rural|42.1|(3.1)|42.6|(4.2)|||
||Urban|30.0|(3.6)|29.0|(4.3)|||
||National|40.3|(3.5)|40.6|(4.1)|||
|Subsample Direct<br>Estimation|Rural|44.0|(4.3)|44.1|(5.1)|||
||Urban|29.2|(5.9)|29.8|(6.9)|||

_Source_ : Authors’ estimations using 2005 HIES data. _Notes_ : Numbers in parentheses are SEs. For each replication, a model is estimated using OLS and it keeps variables with significance levels higher than 0.001 and 0.0001 in rural and urban areas, respectively. More details are available in appendix tables A2.4–A2.6. a. The 95% CIs are presented in the first three rows only because these numbers remain the same irrespective of approaches or subsample size.

# **_For subsamples of 640 households_**

The cumulative means of poverty rates and SEs after 1,000 replications are both close to those based on the first 30 replications for both approaches. As shown in table 5, the cumulative means of the Subsample Imputation are 38.4, 41.8, and 27.9 percent for the national, rural and urban levels/areas, respectively. These are slightly underestimated compared with the poverty rates of the first 30 replications and the Full Sample Direct Estimation. However, all estimates from the Subsample Imputation are in the 95 percent CIs of the Full Sample Direct Estimation. The poverty estimates of the Subsample Direct Estimation are 40.4, 44.3, and 28.4 percent for the national, rural and urban levels/areas, respectively. They are very close to those of the Full Sample Direct Estimation. Like the results from the first 30 replications, the SEs of poverty estimates from the Subsample Direct Estimation are higher than those of the Subsample Imputation, and particularly higher for urban areas.

# **_For subsamples of 320 households_**

The results on the cumulative means of poverty rates are similar to those of the subsample with 640 households. Also, they do not change much from the results of the first 30 replications. The SEs of poverty rates increased significantly by reducing the size of subsamples to 320 households. But, the simpler and less accurate estimation process used for the 1,000 replications resulted in much bigger SEs of the poverty rates when the sample size is reduced. For example, when the Subsample Imputation approach is used, the SE of the national poverty rate increases from 2.2 percent to 3.5 percent after switching from a more accurate estimation method used in the first 30 replications to a less accurate and 13
simpler estimation method used for the following 1,000 replications. This is the case for SEs of poverty estimates in both urban and rural areas.

**Table 6: Comparison of Full Sample Direct Estimation between Only Households Included in both the 2000 and 2005  HIESs and All** **<u>Households in the 2005 HIES</u>**

|Area/Level|Poverty estimates from<br>“new” full sample|Poverty estimates from all<br>households in the 2005<br>HIES|
|---|---|---|
|National|39.8<br>(1.0)|40.0<br>(1.1)|
|Rural|43.6<br>(1.2)|43.8<br>(1.3)|
|Urban|28.4<br>(1.6)|28.4<br>(1.9)|

_Source_ : Authors’ estimations using data from the 2000 and 2005 HIESs. _Note_ : Numbers in parentheses are SEs. Because of the mismatch of location codes between 2000 and 2005, 860 samples, 60 (8.1 percent of samples in 2005) were dropped for the “new” full sample estimates. Cluster-robust SEs were used.

# **Full Sample Imputation**

Using the ELL method as in Christiaensen et al. (2012), the parameters for consumption models and errors were estimated from the 2000 HIES data and then applied to the 2005 HIES to impute household expenditure, from which poverty headcount rates for 2005 were estimated. To reflect differences in consumption patterns between urban and rural areas, we prepared separate consumption models for urban and rural areas. As discussed above, a critical assumption of the Full Sample Imputation is that the estimated consumption models are stable between 2000 and 2005.

Because of changes in subdistrict boundaries between the two years, a small portion (8.1 percent) of samples in 2005 cannot be used, because we could not match some subdistricts in 2000 to those in 2005. The poverty rates and SEs in 2005 were obtained using the remaining 91.9 percent of the 2005 sample. However, as shown in table 6, the poverty rates in the “new” 2005 full sample are very close to those estimated from all households in the 2005 HIES.

Models with and without ownership of household durables were estimated to test “asset drift.” As explained earlier, the relationship between asset ownership and income or consumption likely changes over time, so Full Sample Imputation with household durables could be affected or biased if an asset drift exists. Household durables are important correlates of income or consumption, so not including these variables could reduce the model’s fit and thus increase the SEs of poverty estimates. To see the potential impact of the asset drift, we compared poverty estimates with and without household durables in consumption models.

**Table 7: Poverty Estimates from Full Sample Imputation (%)**

|Area/Level|With d|urables|Without|durables|Full sample direct<br>estimation<sup>a</sup>|
|---|---|---|---|---|---|
|National<sup>b</sup>|38.2|(4.0)<sup>c</sup>|38.9|(3.7)|39.8|
|Rural|41.8|(5.0)|43.1|(4.6)|43.6|
|Urban|26.0|(3.1)|24.3|(3.6)|28.4|

_Source_ : Authors’ estimations using data from the 2000 and 2005 HIESs.

- _Notes_ : Numbers in parentheses are SEs. Povmap2 Software is used for the small area estimation. a. Full sample refers to the number of households in the 2005 HIES that belong to subdistricts in both the 2000 and 2005  HIES data.

- b. Weighted average of rural and urban estimators.

- c. SEs are calculated assuming poverty rates of urban and rural areas are not correlated.

Table 7 summarizes the results of the Full Sample Imputation. Inclusion of ownership of durable goods has less impact on the national poverty rate, but it has a noticeable impact on urban and rural poverty rates. Compared with the true poverty rates (or Full Sample Direct Estimation), inclusion of durable 14
goods tends to underestimate poverty rates in rural areas, while estimating a more accurate poverty rate for urban areas. Dropping ownership of durable goods improves rural poverty rates, but underestimates urban poverty rates.

Finally, we compared the poverty estimates of all three approaches: Subsample Imputation, Subsample Direct Estimation, and Full Sample Imputation. Although poverty rates from the Full Sample Imputation were estimated from a slightly different set of households in the 2005 HIES, we directly compared the poverty rates of all three approaches; table 8 contains the results. For this comparison, the subsample size was 640 households, and the Full Sample Imputation approach excluded ownership of durable goods from both urban and rural consumption models. All three approaches estimated rural poverty rates very well, while the subsample approaches appeared to show better estimations of urban poverty rates.

The Full Sample Imputation approach was worst, underestimating the urban poverty rate by more than 4 percentage points. As a result, the urban poverty rate estimated by the Full Sample Imputation approach was outside the 95 percent CI of that of the Full Sample Direct Estimation. Finally, the Subsample Direct Estimation and Full Sample Imputation both suffered large SEs compared with those from the Subsample Imputation. In all levels, the SEs from Subsample Imputation were smaller than those from the Subsample Direct Estimation and Full Sample Imputation. In particular, the urban poverty estimate in Subsample Direct Estimation and the national and rural poverty estimates in Full Sample Imputation were large. All in all, the best performer was the Subsample Imputation approach using a subsample size of 640 households.

**Table 8: Comparison of Three Approaches with the Full Sample Direct Estimation**

|Approach|Area|Poverty<br>rate|SEs|
|---|---|---|---|
|Sbl|National|39.4|(2.0)|
|usampe<br>Imputation<sup>a</sup>|Rural|43.1|(2.5)|
||Urban|28.2|(3.1)|
|Subsample|National|40.2|(2.7)|
|Direct|Rural|43.8|(3.2)|
|Estimation<sup>a</sup>|Urban|29.0|(4.5)|
|Fll Sml|National|38.9|(3.7)<sup>d</sup>|
|u ape<br>Imputation<sup>b</sup>|Rural|43.1|(4.6)|
||Urban|24.3|(3.6)|
|Full Sample|National|40.0|(1.1)|
|Direct|Rural|43.8|(1.3)|
|Estimation<sup>c</sup>|Urban|28.4|(1.9)|

_Source_ : Authors’ estimations using data from the 2005 HIES. a. The size of subsample is 640 households.

b. Consumption models without durables are shown here. c. Full sample includes all households included in the 2005 HIES data.

d. This SE is estimated assuming there is no correlation between urban and rural poverty rates.

# **IV. Price Data Analysis**

Although the Subsample Imputation approach performs well, there is one more challenge to overcome to implement it in Bangladesh, where poverty lines are estimated and updated for 16 strata separately. When updating the poverty lines, the government calculates an average of food price inflation and nonfood price inflation weighted by averages of the budget shares between two survey years. An important point here is 15
that food price inflation is estimated with unit values calculated from consumption data in the HIES. Unit values are used because there is a concern that price data collected for constructing the Consumer Price Index (CPI) have limited sample sizes at the stratum level, resulting in potentially large sampling errors. Unit values can fill the data gaps because they can be calculated from consumption data in the HIES, which has a large enough sample for each stratum. Two issues in the use of unit values are: (i) unit values are not available for most nonfood items because households do not have to report quantities for nonfood consumption, and (ii) unit values are known to be quite noisy. As a compromise, the Bangladesh Bureau of Statistics (BBS) uses an urban-rural breakdown of nonfood CPI data to estimate nonfood inflation, and estimates food inflation using unit values to construct a Tornqvist price index while carefully controlling for outliers.<sup>3</sup> Price adjustment for food items is a challenge when using the Subsample Imputation approach. The hybrid survey does not collect consumption data, except for the subsample; as a result, unit values can be calculated only from food consumption data in the subsample. Since the size of the subsample is restricted to just 640 households, or less, the number of unit values for each stratum can become very small, resulting in a potentially large sampling error.

This section tests two approaches for conducting price adjustments. First, we followed the BBS’s approach using the subsample, and quantified the extent of the potential bias. Second, we collected price data for food items and compared them with unit values. Compared to unit values, price data are often argued to be more reliable. For this experiment, we collected price data and consumption data in a few villages and compared them with unit values.

**Using Unit Values in Subsamples**

We tested the effect of updating poverty lines using subsamples of poverty estimates and SEs obtained by bootstrapping 100 subsamples. For each subsample, we calculated food inflation rates for 16 strata based on unit values available in the subsample. We then updated 16 poverty lines along with nonfood CPI, and implemented the Subsample Imputation. We compared the resulting poverty rates with those estimated from the Subsample Imputation using the official poverty lines of 2005 based on the full sample of 2005 HIES data. To determine the impact of sample size, we conducted this test with two subsample sizes: 640 and 320 households.

The food price inflation rates were calculated in four steps: first, we defined 13 food groups and calculated average budget shares of these groups for all 16 strata and both the 2000 and 2005 HIES j j rounds, say w0k and w5k. Second, we calculated unit values of a representative item (k) in each of 13 j food groups and selected the median value across all 16 strata (j) and for both rounds (0 or 5), say p0k and j p5k. Third, the stratum-specific food price inflation rates were calculated following Tornqvist: Finally, stratum-specific inflation rates were calculated by taking averages of the food price inflation rates and the nonfood inflation rates (estimated from nonfood CPI) weighted with average budget shares of food and nonfood items.

In the simulations to follow, the 2005 budget shares and median unit values were estimated from a subsample, although those of 2000 were estimated from its full sample. For the official update of the poverty lines, the 2005 budget shares and median unit values were estimated from the full sample of the 2005 HIES data.

16
Limited observations of unit values might inflate the SEs of budget shares and median prices of 2005 substantially, or even worse, we might not able to calculate inflation rates for some strata because no unit value is available for some food groups in some rounds of the simulation. If this happened, we excluded the groups, and reweighted the expenditure shares of the rest of food groups so that the revised weights totaled 1.

Table A3.1 shows the numbers of simulations without any unit value for each food group and each stratum. We can see unit values for cooking oil and soft drinks were missing for several rounds of simulation, but the impact would be negligible since the budget expenditure shares for these items are very small (table A3.2). On the other hand, it is worrisome that unit values of food grains were frequently missing for Rajshahi (SMA) and Sylhet (Municipality) because the budget shares of food grains are large.

Table 9 shows the effects of the limited observations of unit values on the inflation rate and poverty line estimates. The first column shows the inflation rates constructed using the full 2000 and 2005 HIES samples, while the second column shows the inflation rates constructed using the full sample of 2000 and a subsample from 2005. The differences are very limited. For most strata, the differences in inflation rates between the first and the second columns are minimal. As expected, Rajshahi (SMA) and Sylhet (Municipality) exhibit relatively large differences, but they are still below 10 percentage points. For rural and urban areas, the differences are less than 1 percentage point.

|**Table 9: Estimates of Price In**|**dex and 2005**<br>Price<br>Full|**Poverty Lin**<br>index based<br>Subsampl<br>HHs, 1<br>repetiti|**es from su**<br>on:<br>e (640<br>00<br>ons)|**bsamples**<br>2005 pove<br>Full|rty line base<br>Subsampl<br>HHs, 1<br>repetitio|d on:<br>e (640<br>00<br>ns)|
|---|---|---|---|---|---|---|
|Stratum|sample|Estimate|SE|sample|Estimate|SE|
|Barisal (Rural)|1.283|1.277|0.014|915.6|911.2|9.8|
|Barisal (Municipality)|1.229|1.224|0.046|938.5|935|35.4|
|Chittagong (Rural)|1.206|1.208|0.018|884|885.3|13|
|Chittagong (Municipality)|1.153|1.154|0.021|954.8|955.2|17.3|
|Chittagong (SMA)|1.183|1.177|0.019|1,157.9|1151.5|18.6|
|Dhaka (Rural)|1.278|1.28|0.011|831.7|832.9|7|
|Dhaka (Municipality)|1.183|1.183|0.013|877.6|877.5|9.8|
|Dhaka (SMA)|1.173|1.17|0.023|1,003.9|1001.2|19.8|
|Khulna (Rural)|1.263|1.263|0.011|735.3|735.4|6.4|
|Khulna (Municipality)|1.177|1.172|0.016|812.8|809.5|11.1|
|Khulna (SMA)|1.193|1.197|0.029|922.2|925|22.7|
|Rajshahi (Rural)|1.270|1.273|0.018|758.8|760.8|10.7|
|Rajshahi (Municipality)|1.194|1.198|0.019|844.5|847.5|13.4|
|Rajshahi (SMA)|1.235|1.138|0.05|843|776.7|34.3|
|Sylhet (Rural)|1.237|1.213|0.04|817|801|26.5|
|Sylhet (Municipality)|1.201|1.182|0.046|1,012.3|996.4|39|
|Rural (population weighted)|1.257|1.257|0.016|815.2|814.8|10.7|
|Urban (population weighted)|1.182|1.178|0.023|965.2|961.6|18.9|

_Source:_ Authors’ estimations using 2000 and 2005 HIES data. _Note:_ HH = household.

17
Another important observation is that SEs at stratum-level inflation rates are quite low. Despite extremely limited observations at the stratum level, SEs of inflation rates are all below 5 percentage points. SEs of rural and urban average inflation rates are just 1.6 percent and 2.3 percent, respectively.

As a result, stratum-specific poverty lines were also quite accurately estimated using subsample data from the 2005 HIES. The fourth column of table 9 shows poverty lines estimated using the full 2000 and 2005 HIES samples, while the fifth column shows those estimated using the full sample of the 2000 HIES and a subsample from the 2010 HIES. Most exhibit less than a 1 percent difference between the fourth and fifth columns. Like above, Rajshahi (SMA) and Sylhet (Municipality) show relatively large differences, but still they are less than 10 percent. If all poverty lines are aggregated for rural and urban areas, the differences become less than 1 percent. Furthermore, the SEs of poverty lines based on the subsamples of 2005 are also, in general, small. All of these observations suggest that poverty lines and inflation rates estimated from the subsamples of 2005 are robust and reliable.

|**Table 10: Estimated Pover**<br>Updating povertylines<br>Estimation method|**ty Rates**<br>Using<br>Subsa<br>Imput|subsam<br>dat<br>mple<br>ation<sup>a</sup>|ples of<br>a<br>Subs<br>Dir|2005<br>ample<br>ect|Subs<br>Impu|Usingth<br>ample<br>tation<sup>a</sup>|e full sam<br>Subsa<br>Dir|ple of 2<br>mple<br>ect|005 dat<br>Full S<br>Di<br>Estim|a<br>ample<br>rect<br>ation|
|---|---|---|---|---|---|---|---|---|---|---|
|National, 640 households|38.0|(2.5)|38.8|(2.7)|38.9|(2.5)|39.8|(2.7)|40.0|(1.1)|
|Rural, 410 households|41.2|(3.1)|42.9|(3.4)|42.1|(3.1)|43.9|(3.6)|43.8|(1.3)|
|Urban, 230 households|28.3|(3.6)|26.4|(4.2)|29.2|(3.6)|27.4|(4.2)|28.4|(1.9)|
|National, 320 households|38.2|(3.4)|39.6|(4.5)|39.2|(3.4)|40.9|(4.5)|40.0|(1.1)|
|Rural, 200 households|41.0|(4.3)|42.8|(5.5)|41.9|(4.3)|44.1|(5.5)|43.8|(1.3)|
|Urban, 120 households|29.7|(5.3)|29.7|(6.8)|31.0|(5.3)|31.2|(7.3)|28.4|(1.9)|

_Source:_ Authors’ calculations using 2000 and 2005 HIES data. a. A simplified method, which is described in detail in section II, is used.

Finally, we examined whether updating poverty lines using a subsample of the 2005 HIES data has a large impact on poverty estimates derived from the Subsample Imputation. To determine this impact, we compared poverty estimates from poverty lines and inflation rates estimated from subsamples of 2005 HIES data with those estimated from the full sample of 2005 HIES data.

In the Subsample Imputation described in section III, inflation rates and poverty lines are estimated using the full 2000 and 2005 HIES samples with only the consumption models developed using a subsample from the 2005 HIES. Here inflation rates, poverty lines, and consumption models are all estimated using a subsample from the 2005 HIES. To reduce the computational burden of this analysis, we adopted the simplified projection model (described in detail in section II) rather than a full-fledged ELL method when creating consumption models. Also, to see the effect of sample size, we conducted this analysis with subsamples of 320 and 640 households.

Table 10 summarizes estimated poverty rates. The first column shows poverty estimates from Subsample Imputation when a subsample of 2005 HIES data is used to estimate poverty lines and inflation rates, 18
while the fifth column shows poverty estimates using the full sample of 2005 HIES data to estimate poverty lines and inflation rates. Like before, the simplified projection method slightly underestimates the national and rural poverty rates, and SEs are slightly higher than when the full-fledged ELL is applied. The difference in poverty rates between the first and the fifth columns is minimal, no more than 1 percentage point for national, rural, and urban poverty rates. This is the case for both sample sizes, 320 and 640. This makes sense, because both inflation rates and poverty lines estimated from a subsample of 2005 HIES data are very similar to those from the full 2005 HIES sample. Also, it is interesting to see that SEs of all poverty rates do not change when a subsample of 2005 HIES data are used to estimate inflation rates and poverty lines. The results of Direct Estimation of poverty rates are also similar.

This analysis confirms that using data from the 2000 and 2005 HIESs, even if only a subsample of 2005 HIES data is used, does not change the inflation rates and poverty lines much. As a result, the results of the Subsample Imputation do not change much. This is very encouraging for the imputation approach.

# **Using Price Survey Data**

Even though estimating inflation rates and poverty lines using a subsample of 2005 HIES data worked well, unit values are known to be noisy and vulnerable to outliers. Price data are often argued to be more reliable than unit values. One of the issues with using price data, which are currently collected for the CPI, is that these data do not have enough observations to create stratum-specific inflation rates. Therefore, the BBS is currently using unit values from HIES data to estimate stratum-specific food price inflation rates. However, the lack of observations can be overcome with relatively limited additional costs. If price data are collected in all PSUs of a hybrid survey, there will be enough observations to create inflation rates that are representative at the stratum level. Furthermore, collecting price data in this manner saves transportation and lodging costs because enumerators are in PSUs anyway to collect other data and collecting price data is not time consuming.

Even though price data can be collected at minimal additional cost, switching to price data can be problematic. Switching to price data can make new poverty estimates incomparable with past estimates if the price data are not comparable to unit values.

To determine the quality and comparability of price data, as well as the cost of collecting price data, the BBS and a World Bank team carried out a pilot survey, which collected price data and consumption data from 60 PSUs and 720 households (12 households in each PSU) in four strata: Dhaka urban, Dhaka rural, Khulna rural, and Rajshahi rural. The pilot was conducted in June and July 2011 with trained enumerators.

|**Table 11: Dif**|**ferenc**|**e betw**<br>D|**een Me**<br>haka rura<br>|**dian U**<br>l<br>|**nit Val**<br>K|**ues and **<br>hulna rur<br>|**Prices**<br>al<br>|**for Sel**<br>Raj|**ect Ite**<br>shahi rur<br>|**ms**<br>al<br>|D|haka urb<br>|an<br>|
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|Items|Unit|Price|Unit<br>value|%<br>diff|Price|Unit<br>value|%<br>diff|Price|Unit<br>value|%<br>diff|Price|Unit<br>value|%<br>diff|
|Rice, medium|Kg|35|35|0|35|33|6|32|31|**2**|36|36|1|
|Rice, coarse|Kg|32|34|**-6**|31|31|0|29|30|-3|32|35|**-8**|
|Beef meat|Kg|250|250|0|250|250|0|250|250|0|250|259|**-4**|
|Chicken meat|Kg|145|150|**-3**|131|141|**-7**|140|143|-2|145|147|-1|
|Milk, liquid|L|40|40|0|30|30|0|30|30|0|50|50|-1|
|Soybean oil|L|118|120|-2|120|120|0|120|120|0|120|120|0|

_Source_ : Authors’ estimations using pilot survey data. _Note_ : Bold numbers indicate the medians are statistically different between survey prices and unit values with p-values less than 5 percent. Shares of differences between survey and unit values = (survey price - unit value) / mean (survey price, unit value). All prices are reported in Bangladesh taka.

19
The consumption module was identical to that of the 2000 and 2005 HIESs, while the consumption data were collected via two week recall, as well as by diary. Price data were collected using a questionnaire created for this pilot by the World Bank team and a survey company, DATA. Price data include 32 major food and nonfood items.

Table 11 shows the median prices from the price survey and unit values from the consumption module for the six most important food items by four strata (Dhaka rural, Khulna rural, Rajshahi rural, and Dhaka urban). The histograms of these numbers are shown in the appendix (figures A2.1–A2.4). Table 11 shows the differences between the median price and unit value as proportions of their average so that the size of relative difference can be easily discerned. If the relative difference is statistically significant, percentage difference is in bold with p-values less than 5 percent. Positive (negative) differences indicate that prices (unit values) are larger than unit values (prices).

According to table 11, medians of prices and unit values are usually very similar, but reflecting small movements in both values, SEs are small, and even small differences in median prices and unit values become significant. More specifically, in Dhaka rural, the prices and unit values for coarse rice and chicken meat are statistically significantly different, but the relative differences of the two items are only 6 and 3 percent, respectively. In Khulna rural, the noticeable differences in medians between unit values and prices are found in medium size rice and chicken meat, although only the difference for chicken meat is statistically significant. In Rajshahi rural, medium rice is statistically significantly different, but the relative difference is only 2 percent. In Dhaka urban, median prices of coarse rice and beef meat are statistically significantly different, and the relative differences are 8 and 4 percent, respectively.

Overall, medians of unit values can be similar to those of prices if the price surveys were collected from each stratum. This implies that unit values from the subsample might be quite reliable even though the sample size is limited. Also, although we need to see whether the results hold when expanding the sample to the whole country, this analysis suggests that switching to price data might have minimal impacts on price adjustments and poverty estimates.

Bangladesh poverty measurement uses unit values from consumption data of the HIES to estimate inflation rates and poverty lines. Therefore, if consumption data are collected from only a subsample of the HIES data, estimation of inflation rates and poverty lines, and thus the resulting poverty estimates, can be noisy and unreliable. However, the above analysis shows, at least in the case of Bangladesh’s 2000 and 2005 data, that using only a subsample of 2005 data does not increase noise in estimated inflation rates, poverty lines, or poverty headcount rates.

Price data collection is simple and less costly, but raises the possibility of incomparability between price data and unit values, which would make future poverty estimates incomparable with past poverty estimates. However, analysis based on the pilot survey reviewed here suggests that price data are comparable to unit values from consumption data. These results suggest that inflation rates and poverty lines can be accurately updated with a subsample of HIES data or additional price surveys. This is certainly encouraging for the Subsample Imputation approach, and even for the Subsample Direct Estimation.

# **V. Cost Analysis of Three Proposed Approaches and Effects of Switching from Diary- to Recall-Based Data Collection**

Results here show that even if the Bangladesh economy did not experience a large economic crisis between 2000 and 2005, the subsample approaches provide better results for 2005 than the Full Sample Imputation that builds consumption models from the previous round of the HIES, that is, from the 2000 HIES data. But, as discussed earlier, collecting consumption data is costly, even for a subsample. Also, 20
the Subsample Imputation approach needs a large sample of nonconsumption data. The government of Bangladesh recently carried out a simpler version of the HIES to collect nonconsumption data, the Welfare Monitoring Survey (WMS). It simplified the HIES questionnaire significantly and does not include consumption and income data. According to the BBS, WMS implementation costs are only around one-tenth of the HIES.

How do the implementation costs of the HIES compare with those of the WMS and a collection of a subsample of consumption data? Although this cost estimation is admittedly rough, it was carried out in close collaboration with BBS survey experts to incorporate several aspects of the survey data collection process. In terms of the WMS sample size, we tentatively assumed that 12,240 households formed 612 PSUs, following the sampling frame of the latest round of the HIES (2010). Instead of estimating survey implementation costs in local currency, we estimated three key aspects of survey implementation, namely staff weeks, number of moves and days of lodging, which are all closely related to survey implementation costs. For example, the number of moves should be closely related to transportation costs, the days of lodging should be proportional to the total lodging cost, and the staff weeks should be proportional to the total staffing costs.

This cost comparison also included an experiment: switching the mode of consumption data collection from a diary-based approach to a recall-based approach. Consumption data in the HIESs are collected by a diary-based approach, or more precisely, seven consecutive visits by enumerators to a household to collect the last two day’s consumption data. In this sense, the approach undertaken in the HIES could also be considered a recall-based approach, but since consumption data are collected in an almost continuous manner for two weeks, we call it a diary-based approach.

Obviously, the diary approach is very costly since a group of enumerators needs to stay in a village for 14 days to collect consumption data. If consumption data were collected in one visit with two weeks of recall, then the cost of implementation would be much cheaper. But this can cause comparability problems between the consumption data and data from previous rounds. Beegle et al. (2011) reviewed previous studies on this method of consumption measurement and noted that “well-implemented diary surveys might be expected to yield higher (and presumably closer to actual) levels of consumption, experimental evidence for this from developing countries is fragmentary.” The following analysis finds cost savings from switching from the diary approach to the two-week recall approach, and shows the impact of switching on comparability, based on analysis from pilot survey data.

# **Costs of WMS as Baseline and Assumptions for Estimation**

WMS costs were used as a baseline. In the WMS, consumption data are not collected, and without he consumption module the survey is shorter than the HIES. Therefore, after consultation with BBS survey experts, we assumed that one enumerator could handle the survey in two PSUs for one week. So, the total staff weeks for 612 PSUs was equal to 306 (612 PSUs × 0.5 week). In addition, the number of movements is assumed to be proportional to PSUs covered by the number of enumerators, and the days of lodging are assumed to be (the number of PSUs) × (the number of weeks) × 7. So, the number of movements is 612, and the days of lodging are 2,142 (= 612 PSUs × 0.5 week × 7) days.<sup>4</sup>

# **HIES Costs Compared to WMS Costs**

Compared to the WMS, the HIES questionnaire is much more detailed and comprehensive. For example, even if the questionnaires of both the WMS and HIES include an education section, the HIES ask many more questions in the section than the WMS. Also, the HIES questionnaire includes questions regarding economic activities and agricultural and nonagricultural enterprises, which are not included in the WMS. Collection of consumption data for the HIES requires seven visits to each household in two weeks. After consultation with BBS survey experts, we assumed as reasonable the following regarding staff weeks, number of moves, and days of lodging: 21
- Five enumerators have to stay for two weeks to complete the survey in one PSU. Among the five enumerators, two persons engage in collecting (diary-based) consumption data, and three persons collect nonconsumption data for two weeks.

- Since the HIES has 612 PSUs, the total number of staff weeks in the HIES is equal to 6,120 staff weeks (3,672 for nonconsumption + 2,448 for consumption parts).

- For the number of movements and days of lodging, we assumed that the enumerators for consumption and nonconsumption parts move together and can share transportation, but not lodging. So, the number of movements is independent of the number of staff in a PSU, but the days of lodging depend on staff weeks in which all staff stay in a PSU. As a result, the number of movements is 612, and the days of lodging are 42,840 (= 5 enumerators × 2 weeks × 612 PSUs × 7 days).

Table 12 confirms that the HIES costs much more than the WMS. HIES data are attractive because the consumption data are available for all households, but the required staff work weeks and days of lodging for the HIES are 20 times more than those of the WMS.

# **Costs of Collecting Consumption Data from WMS Subsamples**

Collecting consumption data from just small subsets of WMS households instead of from all households in the survey is obviously a cost-saving option. Depending on the size of subsamples, 64 and 32 PSUs were selected. Unlike the sampling of the HIES, 10 (not 20) households were chosen from each PSU. Another important assumption was that collection of consumption data was carried out separately from data collection of other modules. The first WMS data collection was conducted over one month, since most questions were not affected by seasonality. However, consumption data are likely to fluctuate in a year, so collection of the current WMS data was separated from the collection of consumption data from a WMS subsample.

**<u>Table 12: Comparison of Survey Implementation Costs</u>**

|Options|Numb<br>Total|er of PSUs<br>With<br>subsample|Staff<br>weeks|Cost<br>No. of<br>moves|Days of<br>lodging|Compar<br>Staff<br>weeks<br>(%)|ison to HIE<br>No. of<br>moves<br>(%)|S 2010<br>Days of<br>lodging<br>(%)|
|---|---|---|---|---|---|---|---|---|
|Full 2010 HIES|612|612|6,120|612|42,840|100|100|100|
|WMS only|612|NA|306|612|2,142|5|100|5|
|WMS with a<br>subsample of<br>640 households|612|64|434|676|3,038|7|110|7|
|WMS with a<br>subsample of<br>320 households|612|32|370|644|2,590|6|105|6|
|WMS with a<br>subsample of<br>640 households<br>(recall)|612|64|370|676|2,590|6|110|6|

_Source_ : Authors’ projections after consultations with BBS.

Since the number of sample households in a PSU was reduced from 20 to 10, we assumed only half the staff weeks would be needed to collect consumption data from each PSU, which implies a total of 128 staff weeks needed for collecting consumption data from 640 sample households. If the size of the subsample was set at 320, then we would need a total of 64 staff weeks. As for the number of moves, enumerators needed to move among PSUs 64 times if the size of the subsample size is 640, or just 32 times for a subsample of 320. As for the days of lodging, 896 days were needed for the subsample of 640 households, or 448 days for the subsample of 320 households.

22
Even if the additional requirements in staff weeks, number of moves, and days of lodging are added, the cost savings from the WMS with a subsample of consumption data seem significant. For example, if the subsample size is set at 640, the combination needs only 7 percent of staff weeks and days of lodging required for HIES data collection, although the number of moves needed for the combination is projected to be slightly higher, 110 percent of the HIES (table 12). The latter is due to separate data collections for consumption data and nonconsumption data. If the size of the subsample is set at 320, the WMS with a subsample of consumption data needs even fewer staff weeks, moves, and days of lodging.

# **Diary-Based Versus Recall-Based Consumption Data Collection**

As discussed above, there is potentially a large cost-saving effect by switching the current diary-based data collection to a recall-based collection. Currently, two enumerators need to visit each household seven times during their two-week stay in a PSU. This implies that if a two-week, recall-based collection is selected, in theory, an enumerator would need only two days to cover 20 households in a PSU. Since the number of households for a subsample of the WMS is reduced to 10 households, theoretically, one enumerator could collect consumption data from all 10 households in one day. In the cost projection above, we made a more conservative assumption—that one staff week would be needed to collect consumption data from all households in each PSU. As a result, if the subsample size is set at 640, then the WMS with a subsample of consumption data needs a total of 370 staff weeks, 676 moves among PSUs, and 2,590 days of lodging, which correspond to 6 percent, 110 percent, and 6 percent, respectively, of the HIES.

As mentioned earlier, how consumption data are collected affects how households respond. In general, we do not know whether a switch from a diary-based collection to a recall-based collection increases reporting of consumption expenditures (Beegle et al. 2011). Such a switch often reduces consumption aggregates simply because households remember their expenses better when entering them regularly in a diary.

**Figure 3: Comparison of Distributions of Food Expenditures and Nonfood Expenditures between Diary-  and Recall-Based Data Collections**

<!-- Start of picture text --> Food<br>Recall<br>Outliers =1 if logY > 2.5 * sd(logY))+mean(logY) in each division.<br>Food<br>Diary<br>0 2000 4000 6000<br>Outliers =1 if logY > 2.5 * sd(logY))+mean(logY) in each division.<br>Taka, p.c.<br>Density Kernel density<br>Graphs by diary<br>.001<br>5.0e-04<br>0<br>.001<br>5.0e-04<br>0<br><!-- End of picture text --> <!-- Start of picture text --> Nonfood<br>Recall<br><!-- End of picture text --> <!-- Start of picture text --> .001<br>5.0e-04<br><!-- End of picture text --> <!-- Start of picture text --> Outliers =1 if logY > 2.5 * sd(logY))+mean(logY) in each division.<br>Nonfood<br><!-- End of picture text --> <!-- Start of picture text --> Nonfood<br>.001 Diary<br>5.0e-04<br>0<br><!-- End of picture text --> <!-- Start of picture text --> 15000<br><!-- End of picture text --> <!-- Start of picture text --> 10000<br><!-- End of picture text --> <!-- Start of picture text --> Outliers =1 if logY > 2.5 * sd(logY))+mean(logY) in each division.<br>Taka, p.c.<br>Density<br><!-- End of picture text --> <!-- Start of picture text --> Kernel density<br><!-- End of picture text --> <!-- Start of picture text --> Graphs by diary<br><!-- End of picture text --> _Sources_ : Authors’ estimations using pilot data.

To examine the effects of switching the method of collecting consumption data, the BBS and the World Bank conducted a pilot survey. To investigate the comparability of consumption aggregates when diarybased consumption data collection is replaced by recall-based collection, the two collection methods were randomly assigned to half of the households in each PSU. In four strata: Dhaka urban, Dhaka rural, Khulna rural and Rajshahi rural, 720 households from 60 PSUs (12 households from each PSU) in the 23
2010 HIES were surveyed. Within each PSU, six households were randomly allocated to each the diary and recall collections.

The experiment shows that per capita food consumption collected with the diary survey is higher than that of the recall survey. Figure 3 shows the histograms and kernel distribution of per capita food consumption for all households. The natural logarithm was taken and outliers removed. The mode of the distribution for the diary approach is to the right of that of the recall approach.

Table 13 contains the medians of per capita total, food, and nonfood consumption that were collected by the diary and recall methods for all four areas. Medians were used because they are less vulnerable to outliers. For all the areas combined, the median per capita food consumption from the diary collection was 10.1 percent larger from the recall collection. This difference is statistically significant. In all four areas, median per capita food consumption expenditure collected by diary was larger than that collected by recall although the difference is not statistically significant except for the Khulna rural stratum, possibly because of limited sample sizes. Results from the full sample suggest that collecting consumption data via the recall would result in the underestimation of per capita food consumption.

**Table 13: Comparison between the Diary and Recall Based Collections for Food and Nonfood Expenditure per Capita (medians)**

|||Per cap<br>me<br>|ita, taka,<br>dian<br>|Differenc<br>diary an<br>|e between<br>d recall<br>|
|---|---|---|---|---|---|
|Areas|Category|(D)iary|(R)ecall|Difference|% of D - R|
||Food|1,557.0|1,406.8|150.3|10.1|
|All|Nonfood|1,061.7|908.0|153.7|15.6|
||Total|2,662.0|2,414.6|247.4|9.7|
||Food|1,570.3|1,455.7|114.7|7.6|
|Dhaka rural|Nonfood|1,051.4|833.8|217.7|23.1|
||Total|2,650.7|2,326.1|324.6|13.0|
||Food|1,373.4|1,208.1|165.3|12.8|
|Khulna rural|Nonfood|950.9|750.1|200.8|23.6|
||Total|2,448.9|2,097.6|351.2|15.4|
||Food|1,454.0|1,369.5|84.5|6.0|
|Rajshahi rural|Nonfood|875.0|909.5|-34.5|-3.9|
||Total|2,466.1|2,394.8|71.3|2.9|
||Food|2,115.2|1,913.5|201.7|10.0|
|Dhaka urban|Nonfood|2,125.7|1,597.7|528.0|28.4|
||Total|4,127.6|3,499.1|628.5|16.5|

_Source_ : Authors’ calculations based on pilot survey. _Note_ : Outliers are defined as = 1 if logy > 2.5 × sd (logY) + mean (logY) in each division. P-values are calculated based on bootstrap resampling in the median regression. Percentage of D - R = (D - R) / mean (D, R).

An interesting observation is that even though nonfood consumption data were collected by the recall, irrespective of how food consumption data were collected, median per capita nonfood consumption expenditure was larger when food consumption was collected via the diary approach. This is puzzling, but a similar result in the nontreated part (nonfood) was also found in Beegle et al. (2011).

Switching the data collection methods from diary to recall likely underestimates household consumption data, overestimates poverty rates, and thus makes poverty estimates incomparable with those of previous 24
rounds in which consumption data were collected by diary. Therefore, even if the recall approach is cost saving, it is not recommended for the calculation of poverty estimates.

# **VI. Conclusion**

Despite suggestive empirical evidence provided by Christiaensen et al. (2012), imputing household expenditure or income using models estimated from past data is risky because there is no guarantee that consumption models will not change over time. For example, even Christiaensen et al. (2012) show the consumption models estimated from past data could not predict future poverty data in the Russian Federation. They argued that the divergence in poverty rates between direct estimation and imputation can be attributed to questionable inflation adjustments, but it is difficult to prove that the inflation adjustment is the only source of the divergence.

The biggest challenge for the existing imputation approach is that if consumption or income data are not collected, there is no way to test whether the imputed poverty rate is correct until new consumption data are collected in the future. Christiaensen et al. (2012) could test the accuracy of their imputation model because they could compare the imputed consumption data with the actual consumption data. But, if we truly move to the imputation approach, except for big survey years, only nonconsumption data will be collected and poverty rates will be estimated using imputed consumption data. The accuracy of the poverty rates calculated using imputed consumption data cannot be tested in years when only nonconsumption data are available.

This paper provides empirical evidence that a hybrid approach—collecting consumption data for a subsample only—can be extremely useful in improving the statistical reliability of poverty data. For example, collecting consumption data from a subsample enables us to produce consistent but likely noisy poverty estimates by directly estimating them from the subsample. These estimates can be used to examine whether imputed poverty rates are accurate. Furthermore, this analysis shows that if consumption models are developed from the subsample, poverty rates can be estimated reliably by applying the consumption models to a large sample of the nonconsumption data. The estimates based on the Subsample Imputation approach are shown to be as precise as, or even better than, those based on models estimated from past data, even if the size of a subsample is as small as 320, at least as far as poverty estimates at the national level are concerned. Another advantage of the Subsample Imputation approach is that consumption models reflect the contemporaneous relationship between consumption data and nonconsumption data, which is not the case for the approach proposed and tested by Christiaensen et al. (2012).

Subsample Imputation, or even Subsample Direct Estimation, is very attractive, but there is one challenge to overcome, particularly in the case of Bangladesh, where poverty lines are updated with inflation rates estimated for each of the 16 strata separately. The official method constructs food inflation rates based on unit values calculated from HIES consumption data. If consumption data are collected only from a subsample, the number of unit values will be very limited, and thus, the reliability of unit values will be questionable. This is particularly true because unit values are known to be much noisier than price data.

However, evidence from the pilot survey suggests that unit values for key consumption items are not so noisy and quite similar to price data. Also, this analysis shows that even if unit values from a subsample are used to estimate inflation rates and poverty lines, the estimates are quite accurate. As a result, poverty rates estimated via either the Subsample Imputation or Subsample Direct Estimation are quite precise, even when the consumption models as well as the price adjustments are made using consumption data from a small subsample.

When determining which approach to use, implementation cost is also an important consideration. Subsample Imputation needs a large sample with nonconsumption data, and a small subsample with both consumption and nonconsumption data. If the subsample size is 640, then this combined survey needs 25
only 7 percent of the staff weeks and days of lodging required for typical HIES data collection although it requires a slightly higher transportation cost—110 percent of the transportation costs required for HIES data collection. The transportation cost is slightly higher than that of HIES because consumption and nonconsumption data need to be collected by different teams. Although the total cost saving depends on costs of lodging, staff time, and transportation, it is likely that the cost of data collection for Subsample Imputation is likely to be much lower than for HIES data collection.

These results are encouraging for the Subsample Imputation approach. But, it is also worth noting that all the analyses were conducted for a specific time period, for a specific country, that is, Bangladesh in 2000 and 2005. In a different time period, or for a different country, the superiority of the Subsample Imputation approach might disappear. However, this paper provides an analytical framework to determine the magnitude of cost savings and statistical advantages of Subsample Imputation over other approaches. Furthermore, to implement the Subsample Imputation approach requires a modeling exercise every year, the burden of which cannot be underestimated in some developing countries where technical and analytical capacity of statistics bureau is often severely limited.

Finally, this is not the first evaluation of the statistical precision and cost-effectiveness of the hybrid approach. Fujii and van der Weide (2013) studied the hybrid approach in a quite general environment. For the most part, these two papers are mainly complementary and reinforce the evidence of the hybrid approach’s usefulness. However, there are differences: first, this paper compares the existing survey-tosurvey imputation approach, which is identical to Full Sample Imputation, with the new hybrid approach. Fujii and van der Weide (2013) focus on derivation of properties of statistics estimated from the hybrid approach. Second, one potential shortcoming of the hybrid approach is that it needs to conduct price adjustments and update poverty lines using a limited number of unit values available in a subsample. Practically, this is an important challenge since it might expand the margin of error in poverty estimates. This paper studied the effect of limited sample size, while Fujii and van der Weide (2013) did not. Third, this paper conducts all analyses using data from Bangladesh 2000 and 2005, while the results from Fujii and van der Weide (2013) are general. All in all, these two papers fill each other’s gaps and together build a strong case for a hybrid survey approach.

26