# 5. CONCLUSION

Our key finding is that observationally equivalent as well as same households answer the same questions differently when interviewed with a short questionnaire vs. the longer counterpart that, in a prior survey round, would have informed the prediction model for a proxy-based poverty measurement exercise. We pick up statistically significant differences in reporting across all topics and types of questions, particularly those related to consumption of non-food and food consumption items, experience of household shocks, subjective welfare and housing. Relying on prediction models based on the national household survey data collected with the standard questionnaire in 2010, we find that the differences in reporting are sufficient to give predicted poverty rates and Gini coefficients that are significantly different from each other. While the difference in predicted poverty estimates ranges from approximately 3 to 7 percentage points depending on the model specification, restricting the proxies to those that are determined prior to the variation in questionnaire design predicts the same poverty rates in both samples.

Although the poverty proxy comparisons are made across different samples without the luxury of the truth, this point matters less in our case precisely because of the focus on proxy-based poverty measurement. The analyst, who would employ the method in the interim years of a complex household survey, also does not know the truth, and would work under the assumption that the available short household survey data would be consistent with the data that would have been collected through the same complex household survey that had generated the poverty prediction model. The short household survey instrument tested in our experiment is one variant out of many that would have been deemed, prior to implementation, sensible and feasible by the research community focused on proxy-based poverty measurement. Abstracting away from possible interview mode effects, our findings should also be of interest to those thinking of using new technologies, such as mobile phones, for collecting consumption or poverty proxy data through succinct interviews.

It is clear that the explanation of the differences is undoubtedly more complex than what is implied by Table 7, and we cannot convincingly map out the mechanisms. The magnitudes of survey treatment effects on questions appearing in the later modules of the standard household questionnaire, such as shocks and subjective well-being, are not larger than those observed earlier as food and non-food consumption related questions. This implies that interview length alone cannot explain the discrepancies. Further, the binary variables are subject to the largest survey treatment effects, and the experiment versions of their respective modules were also the ones where the change in the immediate context of the question was the largest. For instance, in the standard questionnaire, the food consumption module is set up such that a yes/no question is asked for all items to determine consumption in the last 7 days. Once all yes/no questions have been answered they receive follow up questions on quantity consumed, quantity purchased, value of purchases, quantity received as gifts and quantity originating from own-production for items 13