metadata: authors: 0: author: Beegle,Kathleen G.
1: author: Kastelic,Kristen Himelein
2: author: Ravallion,Martin
count: World
docna: 0: docna: Frame-of-reference bias in subjective
            welfare regressions
docty: Policy Research Working Paper
theme: Rural non-farm income generation,Gender,Vulnerability assessment and monitoring,Poverty strategy, analysis and monitoring
majtheme: Rural development,Social dev/gender/inclusion,Social protection and risk management
subtopic: Health Care Services Industry,Labor Markets,Educational Sciences,Food Security,Inequality
teratopic: Agriculture,Macroeconomics and Economic Growth,Poverty Reduction,Communities and Human Settlements
abstracts: cdata!: Past research has found that subjective
            questions about an individuals' economic status do not
            correspond closely to measures of economic welfare based on
            household income or consumption. Survey respondents
            undoubtedly hold diverse ideas about what it means to be
            "poor" or "rich." Further, this
            heterogeneity may be correlated with other characteristics,
            including welfare, leading to frame-of-reference bias. To
            test for this bias, vignettes were added to a nationally
            representative survey of Tajikistan, in which survey
            respondents rank the economic status of the theoretical
            vignette households, as well as their own. The vignette
            rankings are used to reveal the respondent's own scale.
            The findings indicate that respondents hold diverse scales
            in assessing their welfare, but that there is little bias in
            either the economic gradient of subjective welfare or most
            other coefficients on covariates of interest. These results
            provide a firmer foundation for standard survey methods and
            regression specifications for subjective welfare data.
repnme: repnme: Frame-of-reference bias in subjective
            welfare regressions
display_title: Frame-of-reference bias in
            subjective welfare regressions
listing_relative_url: /research/2009/04/11140696/frame-of-reference-bias-subjective-welfare-regressions
topicv3: Agriculture and Food Security m1326205 435,Poverty m1326687 695,Education m1327048 470,Jobs m1503931 2670,Health m1504661 3480
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Agriculture,Macroeconomics and Economic Growth,Poverty Reduction,Communities and Human Settlements
no_of_pages: 39
author_keys: 0: author_key: 435247
1: author_key: 903202
2: author_key: 99002
model_extractions: WPS4904 Policy Research Working Paper 4904 Frame-of-Reference Bias in Subjective Welfare Regressions _Kathleen Beegle Kristen Himelein Martin Ravallion_ The World Bank Development Research Group Poverty Team April 2009
Policy Research Working Paper 4904

# **Abstract**

Past research has found that subjective questions about an individuals’ economic status do not correspond closely to measures of economic welfare based on household income or consumption. Survey respondents undoubtedly hold diverse ideas about what it means to be “poor” or “rich.” Further, this heterogeneity may be correlated with other characteristics, including welfare, leading to frame-of-reference bias. To test for this bias, vignettes were added to a nationally representative survey of Tajikistan, in which survey respondents rank the economic status of the theoretical vignette households, as well as their own. The vignette rankings are used to reveal the respondent’s own scale. The findings indicate that respondents hold diverse scales in assessing their welfare, but that there is little bias in either the economic gradient of subjective welfare or most other coefficients on covariates of interest. These results provide a firmer foundation for standard survey methods and regression specifications for subjective welfare data.

This paper—a product of the Poverty Team, Development Research Group—is part of a larger effort in the department to test new methods of measuring well-being. Policy Research Working Papers are also posted on the Web at http://econ. worldbank.org. The authors may be contacted at kbeegle@worldbank.org, khimelein@worldbank.org and mravallion@ worldbank.org.

_The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent._ Produced by the Research Support Team
# Frame-of-Reference Bias in Subjective Welfare Regressions

## Kathleen Beegle, Kristen Himelein and Martin Ravallion<sup>1</sup>

_Development Research Group, World Bank 1818 H Street NW, Washington DC, 20433, USA_

> 1 This paper would not have been possible without the work of the _Tajikistan_ Living Standards Survey Team, including Diane Steele, Sasun Tsirunyan, Vladimir Kolchin, Farhod Khamidov, Oleksiy Ivaschenko and staff of the National Statistics Committee of Tajikistan ( _Goskomstat_ ). We thank Gero Carletto and seminar participants at IFPRI and the AEA annual meetings for very useful comments. All views are those of the authors and do not reflect the views of the World Bank or its member countries.
# **1. Introduction**

Subjective measures of welfare are widely used in psychological and social sciences, including economics.<sup>2</sup> Typically a survey respondent is asked to rate their “economic welfare,” “satisfaction with life” or “happiness” on an ordinal scale, sometimes referred to as a Cantril ladder following Cantril (1965).<sup>3</sup> A large literature has studied the covariates of answers to such subjective welfare questions.<sup>4</sup> The most common method is a regression (typically an ordered probit) of the survey responses on individual and household characteristics, including age, gender, income, education, employment status and household demographics. Such regressions have been used to assess the welfare effects of, _inter alia,_ own income (“does money buy you happiness?”), employment (“does unemployment lower welfare at given income?”) and relative position (“do people care about relative deprivation?”).

Measurement problems clearly confound interpersonal comparisons of welfare using subjective data (as they do with objective data). A long standing concern about subjective welfare (and health) questions is that different people may use different criteria for scaling their welfare—that they have different ideas about what it means to be “rich” or “poor,” or what it means to be “satisfied” or not with one’s life.<sup>5</sup> Such latent heterogeneity in scales has often been seen as invalidating the use of subjective welfare questions for inferring utility.<sup>6</sup> This recognizes explicitly that latent heterogeneity in factors that are essentially irrelevant to welfare but influence responses to subjective welfare questions casts doubt on the implied interpersonal comparisons of welfare from a subjective measure. In these circumstances, heterogeneity in

> 2 A cross-country compendium of the questions asked and a summary of the answers can be found in Veenhoven et al. (1993). The literature in economics is reviewed by Di Tella and MacCulloch (2006). The psychological literature on subjective welfare is reviewed in Diener et al. (1999) and Furnham and Argyle (1998). Since 2000, a scholarly journal has been devoted to the scientific study of subjective welfare, namely the _Journal of Happiness Studies_ .

> 3 An alternative approach is to ask what level of income is needed to attain a given position on a Cantril ladder, such as not being “poor.” This is the “Leyden method” devised by van Praag (1968). While we do not use this type of data here, the same concerns about bias arise in the Leyden method.

> 4 Examples include van Praag (1968), van de Stadt et al. (1985), Clark and Oswald (1994, 1996), Kapteyn et al. (1998), Easterlin (1995), Oswald (1997), Winkelmann and Winkelmann (1998), Pradhan and Ravallion (2000), McBride (2001), Ravallion and Lokshin (2001, 2002, 2009) Senik (2004),  Luttmer (2005), Ferrer-i-Carbonell (2005) and Fafchamps and Shilpi (2009). 5

While this paper focuses on heterogeneity in scales, there are other concerns. For example, Conti and Pudney (2008) find that minor re-designs in questions on satisfaction of life/work led to large changes in answers, particularly for women. Moreover, they conclude that these distortions in survey responses influence research findings with respect to understanding women’s job satisfaction. 6 For a recent discussion, see Di Tella and MacCulloch (2006).

2
scales will translate into corresponding differences in subjective welfare at any given level of objective welfare or other relevant covariates. This will, of course, reduce the explanatory power of the regression models for subjective welfare. However, if such heterogeneity was purely random then it would not invalidate inferences from such regressions (at least for linear models). Thus it has been argued that, while inter-personal welfare comparisons are invalidated by heterogeneous scales, the regressions are likely to be robust to such heterogeneity.<sup>7</sup> However, that claim is questionable. People will answer subjective questions in surveys relative to their personal frame-of-reference, which depends on the respondent’s own knowledge and experience, and therefore is likely to vary systematically with the characteristics of that person, including objective measures of economic welfare. For example, it can be conjectured that people living in poor areas of a developing country tend to have a more limited knowledge of the full range of levels of living found in the society as a whole. Someone living in a poor, remote area who has only infrequently left the village and gone no further than the district town, may rate her welfare higher than someone with the same real income who lives in a city and sees far greater affluence around her. Similarly, it can be conjectured that relatively well-off people are often unaware of how poor some people are, and may thus rate their own welfare lower on a Cantril scale.

When this effect is present and it impacts on the coefficients of interest, we shall say that there is a frame-of-reference bias (FORB). The potential for FORB raises concerns about the (enumerable) regression models found in past literature. Consider, for example, the many papers that have used subjective welfare regressions to test for reference-group effects, such as whether higher neighbors’ income makes one feel worse off through perceptions of relative deprivation. It seems likely that the same reference group also influences the respondents’ interpretation of scales used in subjective questions. The reference group acts as both the comparator in assessing relative position and a key element of the information set used by respondents when interpreting the Cantril scales.

To give a sharp illustration of the problem in the present context, suppose that three people are asked to rate their own welfare on a scale of 1,2 and 3, with “1” the poorest. For the sake of the argument, let us also suppose that “wealth” is the only parameter for defining

> 7 For example, Frey and Stutzer (2002, p.406) note the possibility of heterogeneity in the scales used in self-reported welfare questions but claim that this does not invalidate regression models for such data.

3
“welfare.” Wealth is normalized to be in the [0, 1] interval. The first person is relatively poor, the second has the overall modal wealth, denoted _M_ , and the third is relatively rich. The frame-ofreference effect implies that the poor person is only aware of levels of wealth in the interval [0, _M_ ] while the rich person is only aware of those in the interval [ _M_ , 1]. Furthermore, the poor person has a wealth somewhere near the middle of the [0, _M_ ] interval while the rich person has wealth near the middle of [ _M_ , 1]. This suggests a potentially large downward bias in the regression coefficient of subjective welfare on wealth. Indeed, all three responses to the subjective welfare question may well be “2”. This type of bias may also be present when using other concepts of well-being such as life satisfaction or happiness.

There are antecedents to the idea of frame-of-reference effects in the literature. It is a well-established idea that people assess their welfare relative to some “comparison group” such as neighbors or co-workers.<sup>8</sup> This argument has emphasized relativist welfare comparisons. It has also been argued that reference groups play an important role in expectations formation.<sup>9</sup> It is a small step from these ideas to the proposition that survey respondents answer questions with reference to their immediate experiences and that this may well be highly localized in some relevant social or geographic dimensions. In the specific context of subjective welfare measurement, Seidl (2004) argues that van Praag’s (1968) method of calibrating a utility function to subjective data confounds the underlying utility function with a “welfare evaluation function” whereby (for example) “respondents belonging to the middle income strata can evaluate the welfare of the middle income range more precisely” (Seidl, 2004, p.1653).<sup>10</sup> The frame-of-reference effect can also be interpreted as a special case of what is termed “differential item functioning” (DIF) in the literature on educational testing. In this literature, DIF exists if students with equal latent ability have different probabilities of giving a correct answer.<sup>11</sup> This paper explores the role of the frame-of-reference effect in influencing self-reported economic status and offers various tests for FORB, to see whether the standard regressions for subjective welfare found in the literature are contaminated by this effect and to assess the consequences of corrections for this bias to the types of results reported in the literature. Toward

> 8 See, for example, Frank (1997). Frey and Stutzer (2002) provide a useful overview of evidence related to comparison-group effects.

> 9 This was argued by Hirshman (1973). For more recent discussions and evidence, see Ravallion and Lokshin (2000) and Senik (2004).

> 10 See the response by Van Praag and Kapteyn (1994) to this critique.

> 11

> For an overview of the history and methods of addressing DIF, see Angoff (1993).

4
these aims, we adapt the vignette methodology that has been used in a number of recent studies of subjective data on ( _inter alia_ ) health status, political efficacy and job satisfaction. King et al. (2004) and King and Wand (2007) designed vignettes to establish common points on the heterogeneous reference scales regarding political efficacy in China and Mexico. Kristensen and Johansson (2008) used vignettes in anchoring subjective scales for job satisfaction. Bago d’Uva et al. (2008) used them for correcting self-assessed health data for reporting bias. Vignettes have also been used in testing the competence of doctors (Das et al., 2008). To our knowledge, this is the first work to use vignettes for anchoring self-reported economic status, although Kapteyn et al. (2008) use vignettes to compare life satisfaction between respondents in the U.S. and the Netherlands.

The paper provides three tests for FORB where our subjective welfare measure is selfreported economic status. In the first we simply test whether vignette responses are correlated with the regressors typically used in subjective welfare analysis, including objective welfare measures. If everyone has essentially the same idea of what it means to be “poor” or “rich” then we would not expect to find significant correlations between the vignette responses and the covariates used to explain subjective welfare. Under certain forms of FORB, we may find a negative economic gradient in the vignette responses. Consider again the three people in the example above and now suppose that these three people are the vignettes. A plausible set of responses to the vignette questions is given in Table 1, indicating that the wealthier the respondent, the lower (or at least not higher) the rating of each vignette.

So Test 1 provides a very direct test for FORB in subjective welfare regressions, under which we would look for a negative wealth gradient in each set of vignette responses as well as correlation with other household characteristics.

We then provide two further tests that can help quantify the extent of any bias due to heterogeneity in scales by providing a method of correcting subjective welfare regressions for the presence of individual-specific scales or standards. In Test 2, the vignette responses enter as control variables in standard regressions, to purge the error term of the heterogeneity in scales used, under the assumption that differences in vignette responses are solely attributable to differences in the personal scales used. In the third test, an alternative to Test 2, we use the rescaling method proposed by King et al. (2004) in which subjective welfare measures are recalibrated for consistently across respondents based on the vignette responses.

5
We use data from Tajikistan. For the purpose of this paper, we specially designed and included a set of vignettes in the 2007 national household survey for Tajikistan. Respondents were asked to place themselves on a subjective welfare ladder with six rungs. Later in the questionnaire they were asked to place four vignette households on this ladder, and finally to (again) place their own household on the same ladder after ranking the vignettes. Unlike past research using vignettes, we have assured that the subjective welfare questions were asked both before and after the vignettes. The second (post-vignette) subjective reporting enables the rescaling of subjective responses, since we need respondents to place their household in reference to the vignettes themselves.

We begin with a description of our data in section 2. Section 3 presents our results on FORB while section 4 concludes.

# **2. Data**

Tajikistan is one of the poorest and most isolated of the countries in the former Soviet Union, with a per capita income of 430 USD in 2007. Its mountainous location and deteriorating physical infrastructure make transportation difficult and leave certain parts of the country completely isolated during winter months. About one-third of households are located in the capital Dushanbe or other urban areas. Overall, 47 percent of the population lived below the country’s poverty line,<sup>12</sup> and 21 percent in 2004 lived below the World Bank’s international poverty line of $1.25 a day at 2005 purchasing power parity (estimated using PovcalNet at iresearch.worldbank.org/PovcalNet/jsp/index.jsp).

# _Survey data and instrument design_

The 2007 Tajikistan Living Standards Measurement Survey (TLSMS) surveyed a random sample of 4,860 households in September and November 2007. The sample is designed to be representative at the national level, urban and rural levels, and at the _oblast_ (administrative region) level. Data were collected in two visits, with the subjective welfare modules being asked in the first visit. Summary statistics for the variables used can be found in Appendix 1.

> 12 See World Bank (2008). The poverty line is based on the cost of 2,250 calories per day and includes an additional non-food component calculated from the share of total consumption going to nonfood purchases with the reference population being households with food spending just above the food poverty line. All values are regionally deflated.

6
In addition to the standard questions on household characteristics and expenditures (including imputed values of self-produced food) common to multi-topic household questionnaires, subjective welfare data were collected at two different points in the questionnaire. In the Subjective Poverty and Food Security Module, respondents were asked: _“Imagine a 6-step ladder where on the bottom, the first step, stand the poorest people, and the highest step, the sixth, stand the rich. On which step are you today?”_ In a later section of the questionnaire, respondents were asked to place four vignettes of hypothetical households on a six-step ladder and then to place themselves on the same scale. The latter is the same question asked in the earlier portion of the questionnaire. The actual vignettes from the questionnaire (translated into English) are given in Appendix 2.

We developed the vignettes for this experiment in consultation with local counterparts. The vignettes were designed to capture representative snapshots of various levels of welfare in Tajikistan. Characteristics incorporated in the vignettes include land holdings, education, diet, clothing, and the ability to heat the home during the winter. The vignettes were developed in a clear, expected hierarchy with respect to welfare, with all aspects of socio-economic status increasing monotonically. This structure was used to minimize the effects of multidimensionality. Multi-dimensionality can lead to the perverse sequencing of the vignettes with respect to the intended ordering if respondents place different values on various characteristics contained in the vignettes with respect to overall welfare (King et al., 2004). This does not seem to be a major concern for our vignettes: of the 4,860 households in the sample, only 89 have any instances of an “incorrect” ordering. The most common characteristic of respondents who perversely order the vignettes is a low level of education of the household head; see Appendix 3 for results from a probit estimation of the correlates of perverse ordering. These 89 households are excluded from our analysis.

# _Pre and post vignettes_

If responses are influenced by heterogeneous scales, it might be expected that subjective welfare responses will be affected by familiarity with the vignettes. By asking the vignette questions, the survey may focus the respondent to think about, and possibly revise, their own scale used to report their self-assessment welfare.

In Table 2 we compare the pre- and post-vignette responses to the household’s selfassessment welfare. On average, respondents place themselves between steps 2 and 3 in both the 7
pre-vignette and post-vignette question, though the mean was slightly higher post-vignette (2.75 vs. 2.80). Most respondents place themselves in the same position in the pre- and post-vignette self-reporting, although fully 25 percent change their position. Of those who change their position, 57 percent adjust upwards and 43 percent downwards. Generally respondents give similar responses; 82 percent of those changing their position move only one step up or down. Among the 4,771 households, only 39 respondents switch dramatically from the “rich” category (steps 4-6) when asked before the vignettes to the “poor” (steps 1-2) category when asked after the vignettes. In the opposite direction, 72 report themselves in the “poor” categories prevignette and in the “rich” category post-vignette.

We explore whether specific types of households are more inclined to change pre- and post-vignette reports, but we do not observe any striking differences in such changes across different categories of households (Table 3). For example, comparing those classified as poor using objective measures (per capita households expenditure below the poverty line), the poor report a lower subjective welfare score than the non-poor. Both groups adjust their scores up slightly following the vignettes, but neither experiences a marked transformation. We observe similar patterns for other groups, namely urban/rural populations, households in which the head is employed or unemployed, male and female headed households, and households headed by persons older or younger than 65.

_Subjective vs. objective economic welfare_ To compare the two subjective measures against our objective measure based on expenditure per person, we define the sample sizes of six categories of expenditure per capita, based on the distribution of households in the subjective categories. If there are _N_ respondents who place themselves on the lowest subjective step, the lowest _N_ households in terms of expenditure per capita will make up the lowest category in the objective measure. Table 4 presents the results for the pre-vignette subjective rankings.

If the subjective measures are perfectly explained by the objective measure, all observations in the matrix would be along the shaded diagonal. Though the subjective measures are highly correlated with the objective measure, the matching is imperfect. Nearly half (43 percent) of those in the lowest objective classification place themselves on steps 3 or higher in the subjective measure. Of the richest decile of the population according to objective measures, 19 percent place themselves on the lowest two rungs of the subjective ladders, and less than half 8
position themselves on the top three rungs. Of those households classified as extreme poor, living below the food poverty line (roughly 15 percent of the population), only 55 percent place themselves on the lowest two subjective rungs. Similarly, among poor households, only 45 percent place themselves on the lowest two subjective rungs.

The post-vignette rankings show a similar relationship to objective measures as for the pre-vignette rankings (Table 5). Twenty percent of those in the richest decile position themselves on the lowest two rungs. Among extreme poor households and poor households, 53 and 44 percent respectively place themselves on the bottom two rungs. It is also interesting to note that the pre-vignette rankings seem more consistent with our objective measure than the post-vignette rankings. The Cramer’s V statistic is higher in the pre-vignette question, indicating a stronger association between the rows and columns, and a better overall fit to the objective measure.

Comparing the correlation between the pre- and post-vignette rankings and the objective per capita expenditure measure, the correlations are higher for pre-vignette reports (Table 6, row 1). This is also true universally across various populations of interest. It is interesting to note that within some household categories we find lower average subjective welfare reports among the sub-group with higher per capita expenditure. This is true, for example, among female and male headed households (where female-headed households are on average richer by objective measures). Across all household categories the post-vignette scores are higher than those reported prior to the vignettes.

# _Placement of vignettes_

The vignettes were designed such that the first vignette presented a scene of extreme poverty, the second vignette of improved conditions though still poor, the third of middle class and the fourth of relative affluence.

If the vignettes are an effective method of imposing a uniform scale, we would expect the placement of particularly the first vignette to be a good indicator of objective welfare. As the first vignette presents a picture of extreme poverty, most respondents would be expected to place this vignette on step 1. Households which position the first vignette on step 2 might be expected, on average, to be poorer than those who placed it on step 1, as they are able to conceive of a household situation that was even poorer than the first vignette. Similarly households which place the first vignette on step 3 we might assume them to be poorer than those who place the vignette on steps 1 or 2 as they could imagine even worse living conditions compared to that 9
depicted in the first vignette. Following a similar logic, we would also expect the average subjective welfare score to be higher for those that locate the first vignette on step 1 as opposed to those placing it on steps 2 or 3.

The data, however, did not bear out this expectation (Table 7). Those households that place the first vignette on step 1 are, on average, poorer than those who place the first vignette on steps 2 or higher. Though there is no statistical significance in terms of objective measures between households that place the first vignette on step 1 and those who place it on steps 2 or higher, there is a statistically significant difference in terms of subjective welfare measures. This would indicate that those households who position the first vignette on steps 2 or higher perceive themselves as better off than their objective circumstances would indicate.

# **3. Results**

<u>Test 1: We begin by asking whether vignette responses are correlated with widely-used</u> covariates from the literature, including objective measures of economic welfare. We assume an ordered probit specification, which has become standard in the literature. The specification for the underlying continuous variable (generating the ordinal categorical responses) is as follows: _VWik_   _k_ ln _PCEi_   _k X i_   _ki_ ( _k_ =1,4; _i_ =1,.., _N_ ) (1) where _VWik_ is a latent continuous variable for respondent _i_ ’s assessment of vignette _k_ ’s welfare, which generates a discrete response on the scale from 1-6, _PCE_ is per capita expenditure, _X_ is a vector of other household-level variables.

Table 8 summarizes the results while Appendix 4 presents the complete ordered probit regressions estimating the parameters of (1) for the four vignettes. The pseudo R<sup>2</sup> ’s are low, at approximately 0.02. In general, vignette rankings are not consistently or significantly correlated with household characteristics. Geographic characteristics are more likely to be significant for the vignettes higher on the consumption scale (vignettes 3 and 4).

For vignettes 3 and 4 (but not 1 and 2), we find a positive and statistically significant relationship between log _PCE_ and the vignette rankings. Richer households are more likely to give a high welfare ranking to the better-off households described by vignettes 3 and 4. This is not what one would expect with the frame-of-reference effect described in the introduction, which would suggest that poor people would tend to rank the relatively rich vignette higher than 10
rich people. More suggestive of this FORB is our finding that smallholders (in terms of land) tend to rate the poorest vignette higher than do other households.

So the results of Test 1 show that there are only a few significant correlates of vignette responses amongst the types of regressors commonly used in subjective welfare regressions. But the evidence is mixed on FORB: How much do these effects bias the standard subjective welfare regressions found in the literature?

<u>Test 2: In our second test we examine a standard subjective welfare regression,</u> employing widely-used covariates from the literature, with the difference that we also estimate specifications augmented with the vignettes. The augmented specification is as follows: _SWi_   ln _PCEi_   _X i_  _Vi_   _i_ ( _i_ =1,.., _N_ ) (2) where _SWi_ is a latent continuous variable for the subjective welfare of respondent _i_ , which also generates a discrete response on the scale from 1-6 and _V_ is a vector representing the vignette responses. The vector _V_ translates the vignette responses into a series of dummy variables. This eliminates FORB under the assumption that inter-personal differences in vignette responses stem solely from differences in how the scales are interpreted. We refer to the estimated  as the economic gradient in subjective welfare.

With six possible steps for each of the four vignettes, there would theoretically be twenty dummy variables with one step omitted for each vignette. In practice, however, some steps are omitted due to an insufficient number of responses, leaving a total of fourteen vignette dummy variables to capture the complete set of responses observed in the data. We examine the two measures of _SW_ : _SW_ reported before the vignettes and the post-vignette _SW_ .

Table 9 presents the results of the ordered probit based on equation (2) using the prevignette self-reported welfare. We estimate four alternative specifications. In column 1 we find that logged per capita expenditure is significantly positively associated with _SW_ . When vignettes are introduced in the second specification (column 2), the coefficient on _PCE_ is basically unchanged. We do, however, find that the set of vignette dummy variables are jointly significant. Column 2 suggests that there is a frame-of-reference effect on _SW_ , although comparing columns 1 and 2 the vignette effects are not sufficiently correlated with the household’s _PCE_ to generate more than negligible bias in the unconditional economic gradient in subjective welfare. That is, there is negligible FORB.

11
In column 3 of Table 9 we include non-income household characteristics and omit the vignettes. A number of these characteristics have significant effects on subjective welfare controlling for _PCE_ . Female-headed households have lower _SW_ . Households where the head has completed higher education, those where the head has a professional job (such as sales, service and public administration) and larger households generally have higher _SW_ . We do not find consistent urban/rural patterns across areas. Households in urban Gbao report higher _SW_ compared to the reference group in Dushanbe. In rural Khatlon and RRP as well as urban Khatlon, households have lower _SW_ compared to households in Dushanbe. One might expect that households with migrants would have lower subjective welfare scores as they have a wider scope of knowledge and are therefore less likely to overstate their position. The coefficient is positive though not statistically significant.

After adding controls for other household characteristics, we find an increase in the economic gradient (the increase in the coefficient on PCE in columns 1 and 3 of Table 9). This is a statistically significant difference at the 10 percent level. The household characteristics in column 3 may capture several things, including frame-of-reference effects or other effects such as perceived vulnerability, permanent income, or security.

Introducing both the vignettes and the set of non-income household characteristics in column 4, we find that the economic gradient is basically unchanged from column 3. The set of vignette dummy variables remain statistically significant. Thus, while it appears that there is a frame-of-reference effect being picked up by the vignettes, it is still not influencing the economic gradient in subjective welfare. In addition, including the vignettes does not alter the coefficients on other household characteristics.

Table 10 gives our results using the post-vignette subjective welfare measures. The magnitude of the coefficients tends to be lower. In columns 1 and 2, there is still no statistical significant difference between the economic gradient in subjective wellbeing. As before, we observe an increase in the economic gradient when we control for non-income household characteristics; the coefficient on PCE is statistically different at the 10% level between columns 1 and 3. And, again, there is no significant change in this coefficient when we further include the vignettes (columns 3 and 4). Likewise, the coefficients on other covariates remain largely unchanged. Our finding that FORB is negligible remains valid.

12
<u>Test 3: A concern with Test 2 is that the vignette responses are strictly endogenous, given</u> that they come from the same respondent at the same time and so could be jointly influenced by some latent characteristic. One response to this concern is to instead use the vignette responses to re-calibrate the subjective welfare responses. Our rescaling method follows that developed by King et al. (2004) and King and Wand (2007) in which only the relative position of the selfreported score in relation to the vignette rankings matters to the analysis. For example, all respondents who ranked themselves below vignette 1 would have a score of 1 in the re-scaled rankings, regardless of the actual values given to either the self-reported score or the ranking of the first vignette. Similarly, all those respondents who placed themselves at the same level as the first vignette would have a re-scaled ranking of 2, those between vignettes 1 and 2 would have a 3, and so on. Rescaling therefore gives nine possible values to the dependent variable.

Table 11 repeats the analysis presented in Tables 9 and 10 with a rescaled post-vignette ranking as the dependent variable. (For completeness we also estimate regressions for the rescaled responses with controls for the vignettes.) The rescaled regressions show similar relationships with the household characteristics, including significant correlations with PCE, higher education, professional jobs, larger households and the geographic variables. The vignettes, in general, lose their significance with the notable exception of vignette 3.

Tables 9-11 assess the impact of the vignettes on the economic gradient in subjective welfare under the assumption that this impact is constant across the income distribution. To allow for more flexibility in the specification we estimate non-parametric regression functions with linear controls (“partial linear regressions”).<sup>13</sup> By comparing the non-parametric results with and without vignettes (as linear controls), we can test for FORB across the income distribution. We define two alternative binary outcome variables: _SW_ being poor (steps 1-2) and _SW_ being rich (steps 4-6). Since we have both pre- and post-vignette _SW_ , we have 4 outcome variables (pre/post, poor/rich). Further, we assess the FORB from estimates with and without the nonincome traits. Thus, we have a total of eight pairs of partial linear regressions (Figures 1-4).

Figure 1 shows the results for reporting oneself as poor and as rich, with and without vignettes as controls. The “poor” curve is downward sloping: households are less likely to place themselves on the lowest two steps as _PCE_ increases. The opposite is true for those who place themselves on the upper rungs (“rich”). The economic gradient in subjective wellbeing is

> 13 We used the PLREG program for STATA written by Lokshin (2006).

13
unchanged if we control for the household’s scoring of vignettes. That is, the FORB appears to be minimal across the whole income distribution. If there was FORB, we would expect the curves with the vignettes to be steeper than those without, assuming that the poor would overstate and the rich understate their subjective welfare, with attenuation towards the mean.

We find the same results when we include non-income controls (Figure 2) and when we define the dependent variables using the post-vignette _SW_ (Figures 3 and 4).

As noted above, the economic gradient was lower for the post-vignette _SW_ compared to the pre-vignette score (Tables 9 and 10, column 2). This difference in the economic gradient may also vary across the income distribution. Partial linear regressions of poor _SW_ and rich _SW_ are presented in Figure 5, which compares the results based on pre- and post-vignette _SW_ . Figure 5 shows that households at all income levels are more likely to report being rich after scoring the vignettes compared to their prior response; that is, the “rich” curve is shifted up after the hearing the vignettes. There is little change in the probability of reporting being poor for the pre- versus post-vignette _SW_ .

Non-income household characteristics have a modest impact the estimates of the economic gradient of subjective welfare. The economic gradient with respect to self-reporting as poor is slightly steeper when additional control variables are included (Figure 6 “poor”). There is only a slight shift in the gradient with respect to self-reporting as rich after adding controls (Figure 6 “rich”). These results are the same for the post-vignette _SW_ (Figure 7).

# **4. Conclusions**

A cloud of doubt has hung over subjective welfare regressions, arising from concerns about likely heterogeneity in the interpretation of the Cantril scales widely used to measure subjective welfare. This heterogeneity undoubtedly reduces the power of standard covariates in explaining perceived welfare. More worrying, however, is the possibility that the heterogeneity in scales is leading to biased inferences from studies of subjective welfare, including biases in identifying its economic gradient, as well as the effects of other variables such as education, employment status and relative economic position. Bias arises if these variables are correlated with the latent heterogeneity in scales. It can be conjectured that poorer people tend to have more limited horizons in life, due to more limited experiences with the extent of the disparities in levels of living in society as a whole; a poor person’s idea of what it means to be “rich” may then 14
be very different to that of a middle or upper income person with a very different frame of reference. This may be correlated with certain attributes of the household, such as if rural or more isolated households may overstate their welfare given that they have a limited experience or exposure to higher living standards with which to judge their own economic standing.

We have provided various tests for bias due to such heterogeneity in individual scales. The tests entailed adding vignettes of hypothetical households to a national household survey for Tajikistan in 2007. Respondents placed these vignettes on the same six-step ladder used to report their own subjective economic welfare. In our first test for this bias, we do find some significant covariates for vignette responses amongst the regressors commonly used to explain subjective welfare. However, the effects are neither very strong nor consistent across different vignettes.

In the second and third tests, we explore the extent of bias due to the frame of reference effect by embedding vignette rankings by households into standard regressions for subjective welfare. We do this in two ways: we include vignettes among the covariates and, to address endogeneity concerns with this approach, we also recalibrate the self-assessments of welfare to accord with the heterogeneity in scales revealed by the vignette responses. The striking finding of these further tests is that the bias is negligible with respect to the “income effect” on subjective welfare as well as other covariates. Based on this study, the concerns that past uses of subjective economic welfare data are compromised by systematic differences in the meaning given to the scales used appear to be unwarranted.

15
**Table 1: Plausible responses to vignette questions when there is a frame-of-reference bias**

|||Res|pondent’s wealth|
|---|---|---|---|
|||Poor|Middle<br>Rich|
||Poor|2|1<br>1|
|Vignette|Middle|3|2<br>1|
||Rich|3|3<br>2|

**Table 2: Pre-vignette and post-vignette subjective welfare rankings**

|Pre-vignette|Post-vi<br>1<br>poores|gnette<br>t<br>2|3|4|5|6<br>richest|Total|
|---|---|---|---|---|---|---|---|
|1 poorest|247|64|23|3|4|0|341|
|2|72|933|232|53|7|5|1,302|
|3|34|242|1,735|223|37|4|2,275|
|4|4|33|112|592|34|1|776|
|5|2|0|2|11|51|1|67|
|6 richest|0|0|1|1|4|4|10|
|Total|359|1,272|2,105|883|137|15|4,771|

**Table 3: Summary statistics pre and post-vignettes**

||Pre-vignette|Post-vignette|
|---|---|---|
|Overall mean|2.75|2.8|
||(0.87)|(0.92)|
|_Household characteristics_|||
|Poor|2.53|2.59|
|Non-poor|2.94|2.99|
|Rural|2.67|2.73|
|Urban|2.91|2.93|
|Unemployed head|2.61|2.66|
|Employed head|2.82|2.88|
|Female head|2.58|2.62|
|Male head|2.79|2.84|
|Pensioner head|2.78|2.63|
|Non-pensioner|2.59|2.83|

Note: Standard deviation in parentheses.

16
**Table 4: Comparison of pre-vignette subjective welfare with objective measure**

|Subjective<br>welfare<br>rank|1poor|est<br>2|Expendit<br>3|ure per capit<br>4|a rank<br>5|6<br>richest|Total|
|---|---|---|---|---|---|---|---|
|1 poorest|84|120|108|26|3|0|341|
|2|109|461|586|137|8|1|1,302|
|3|130|582|1,184|345|29|5|2,275|
|4|18|135|369|227|23|4|776|
|5|0|3|25|36|3|0|67|
|6 richest|0|1|3|5|1|0|10|
|Total|341|1,302|2,275|776|67|10|4,771|

Notes: Cramer's V = 0.1484; Chi-square=525 (significant at 1%)

**Table 5: Comparison of post-vignette subjective welfare with objective measure**

|Subjective||E|xpenditure p|er capita|rank|||
|---|---|---|---|---|---|---|---|
|welfare rank|1poore|st 2|3|4|5|6 rich|est Total|
|1 poorest|78|128|98|47|7|1|359|
|2|126|422|549|154|19|2|1,272|
|3|126|560|998|362|53|6|2,105|
|4|23|150|402|260|43|5|883|
|5|6|10|53|54|13|1|137|
|6 richest|0|2|5|6|2|0|15|
|Total|359|1,272|2,105|883|137|15|4,771|

Notes: Cramer's V = 0.1355; Chi-square= 438 (significant at p <0.0005) 17
**Table 6: Correlations with objective measures**

||Expenditure per<br>capita<br>(Tajik somoni)|Correlation with<br>pre-vignette<br>ranking|Correlation with post-<br>vignette ranking|
|---|---|---|---|
|Overall mean|176<br>(162)|0.203|0.178|
|_Household character_|_istics_|||
|Poor|100|0.188|0.176|
|Non-poor|245|0.135|0.106|
|Rural|157|0.206|0.187|
|Urban|218|0.193|0.169|
|Unemployed head|161|0.176|0.176|
|Employed head|185|0.209|0.173|
|Female head|192|0.167|0.135|
|Male head|173|0.225|0.202|
|Pensioner head|157|0.151|0.157|
|Non-pensioner|180|0.209|0.179|

Note: Standard deviation in parentheses.

**Table 7: Placement of vignettes**

|First Vignette ranked|Expenditure per|Subjective||
|---|---|---|---|
|on Step:|capita|position|N|
|1|173|2.64|3,276|
|2|181|2.95|1,352|
|3|195|3.12|140|

Note: Three households which ranked the first vignette above step 4 are excluded.

18
**Table 8: Significant predictors of how households rank the four vignettes (See Appendix 4 for complete regression results)**

|Vignette|1 (poorest)|2|3|4 (richest)|
|---|---|---|---|---|
|Significant<br>covariates at the|Household size (+)<br>Basic education (-)|Special secondary<br>schooling (+)|PCE (+)<br>Uzbek (+)|PCE (+)<br>Uzbeck (+)|
|10% level or<br>better|Services sector<br>occupation (+)<br>Small holding (+)<br>Khatlon urban (-)<br>Gbao rural (-)|Number of<br>employed (+)<br>Small holding (-)<br>Sogd (+)<br>Khatlon (+)<br>Gbao urban (-)<br>Gbao rural (+)|Primary schooling<br>(+)<br>General secondary<br>(+)<br>Number of<br>employed (+)<br>Small holding (-)<br>Sogd rural (-)<br>Khatlon urban (+)<br>RRP rural (-)<br>Gbao (+)|Number of<br>employed (+)<br>Agriculture sector<br>(-)<br>Small and medium<br>holding (-)<br>Sogd (-)<br>Khatlon urban (+)<br>Khatlon rural (-)<br>RRP (-)<br>Gbao rural (+)|
|PseudoR<sup>2</sup>|0.022|0.018|0.016|0.019|

19
**Table 9: Pre-vignette self-assessed subjective welfare positions**

|(1)|(2)|(3)||(4)||
|---|---|---|---|---|---|
|PCE|PCE +<br>vignettes|PCE +<br>household<br>controls||PCE +<br>household<br>controls +<br>vignettes||
|Coef.<br>se|Coef.|se<br>Coef.|se|<br>Coef.|se|
|Log expenditure per capita<br>0.630***<br>(0.037)|0.631***|(0.037)<br>0.727***|(0.043)|0.736***|(0.043)|
|_Household Demographics_||||||
|Female headed household||-0.151***|(0.053)|-0.148***|(0.053)|
|Age of household head||-0.014|<br>(0.010)|-0.017*|(0.010)|
|<br>Age of household head squared||0.000|(0.000)|0.000*|(0.000)|
|<br>Household size||0.076***|<br>(0.013)|0.082***|(0.013)|
|Number of children||-0.007|(0.018)|-0.011|(0.018)|
|Number of older adults||-0.030|(0.047)|-0.037|(0.048)|
|Number of migrants||0.049|(0.043)|0.034|(0.044)|
|_Ethnicity (Reference: Tajik)_||||||
|Uzbek||0.020|(0.046)|0.019|(0.046)|
|Russian||-0.337**|(0.157)|-0.350**|(0.165)|
|Other||-0.693***|(0.159)|-0.718***|(0.150)|
|_Education (Reference: No Education)_||||||
|Primary||0.066|(0.116)|0.001|(0.118)|
|Basic||-0.118|(0.109)|-0.129|(0.111)|
|General Secondary||0.028|(0.107)|0.007|(0.109)|
|Special Secondary||0.143|(0.114)|0.111|(0.115)|
|Technical Secondary||0.103|(0.117)|0.068|(0.119)|
|<br>Higher Education||0.304***|<br>(0.115)|0.285**|<br>(0.117)|
|Graduate School||0.545|(0.368)|0.617**|(0.305)|
|_Employment Characteristics of Household, Head’s O_|_ccupation (R_|_eference for Occupation:_|_Unemploy_|_ed)_||
|Number of employed||0.035*|(0.018)|0.032*|(0.018)|
|Agriculture / Fishing / Forestry||0.009|(0.064)|-0.003|(0.065)|
|<br>Manufacture / Mining||-0.023|<br>(0.094)|-0.009|<br>(0.096)|
|Services||0.363***|(0.132)|0.307**|(0.131)|
|Construction||-0.063|(0.080)|-0.093|(0.080)|
|Public Administration / Education / Health||0.186***|(0.070)|0.190***|(0.071)|
|Sales and Services||0.310***|(0.064)|0.314***|(0.065)|
|Other||-0.158|(0.107)|-0.156|(0.105)|
|_Agriculture (Reference for holdings: No Land)_||||||
|Small Holding||-0.063|(0.066)|-0.037|(0.067)|
|Medium Holding||-0.103|(0.079)|-0.087|(0.080)|
|<br>Large Holding||0.020|(0.081)|0.051|(0.082)|
|Anylivestock, poultry, etc||-0.007|(0.058)|-0.008|(0.058)|
|_Geography (Reference: Dushanbe)_||||||
|<br>Sogd Urban||-0.011|(0.075)|-0.067|(0.076)|
|<br>Sogd Rural||-0.106|<br>(0.079)|-0.159**|<br>(0.080)|
|<br>Khatlon Urban||-0.184*|(0.095)|-0.344***|(0.097)|
|Khatlon Rural||-0.509***|<br>(0.074)|-0.570***|<br>(0.075)|
|RRP Urban||-0.158|(0.108)|-0.200*|(0.110)|
|RRP Rural||-0.263***|(0.075)|-0.319***|(0.077)|

20
|(1)<br>PCE<br>Coef.|se|(2)<br>PCE +<br>vignettes<br>Coef.|se|(3)<br>PCE +<br>household<br>controls<br>Coef.|se|(4)<br>PCE +<br>household<br>controls +<br>vignettes<br>Coef.|se|
|---|---|---|---|---|---|---|---|
|Gbao Urban||||0.343***|(0.107)|0.409***|(0.113)|
|Gbao Rural||||-0.062|<br>(0.080)|-0.076|<br>(0.081)|
|_Vignette 1 (positions 3 to 6 omitted)_||||||||
|<br>vign1==1||-0.258*|(0.140)|||-0.281*|(0.144)|
|<br>vign1==2||-0.022|<br>(0.133)|||-0.067|<br>(0.138)|
|<br>_Vignette 2 (position 1 omitted)_||||||||
|<br>vign2==2||1.106***|(0.223)|||1.168***|(0.227)|
|<br>vign2==3||1.337***|<br>(0.230)|||1.439***|<br>(0.233)|
|<br>vign2==4||1.515***|<br>(0.261)|||1.668***|<br>(0.265)|
|<br>vign2==5||1.724***|<br>(0.574)|||1.670***|<br>(0.468)|
|<br>_Vignette 3 (positions 1 and 2 omitted)_<br>vign3==3||-0.777***|<br>(0.257)|||-0.735**|<br>(0.290)|
|<br>vign3==4||-0.852***|<br>(0.263)|||-0.754**|<br>(0.295)|
|<br>vign3==5||-0.844***|<br>(0.270)|||-0.742**|<br>(0.301)|
|<br>vign3==6||-1.505***|(0.401)|||-1.631***|(0.457)|
|<br>_Vignette 4 (positions 1 to 3 omitted)_||||||||
|<br>vign4==4||0.735***|(0.245)|||0.630**|(0.281)|
|<br>vign4==5||0.794***|<br>(0.247)|||0.594**|<br>(0.282)|
|<br>vign4==6||0.793***|<br>(0.251)|||0.582**|<br>(0.285)|
|<br>Number of observations<br>4,771||4,77|<br>1|4,771||4,771|<br>|
|Pseudo R<sup>2</sup><br>0.037||0.05|5|0.080||0.099||

Note: *** p<0.01, ** p<0.05, * p<0.1 21
# **<u>Table 10: Post-vignette self-assessed subjective welfare positions</u>**

|(1)<br>PCE|(2)<br>PCE +<br>vignettes||(3)<br>PCE +<br>household<br>controls||(4)<br>PCE +<br>household<br>controls +<br>vignettes||
|---|---|---|---|---|---|---|
|Coef.<br>se|Coef.|se|Coef.|se|<br>Coef.|se|
|Log expenditure per capita<br>0.560*** (0.037)|0.573***|(0.037)|0.652***|(0.041)|0.673***|0.042|
|_Household Demographics_|||||||
|Female headed household|||-0.178***|(0.052)|-0.173***|(0.053)|
|Age of household head|||-0.005|(0.009)|-0.008|(0.009)|
|Age of household head squared|||0.000|(0.000)|0.000|(0.000)|
|Household size|||0.060***|(0.013)|0.068***|(0.013)|
|Number of children|||0.008|(0.017)|0.002|(0.018)|
|Number of older adults|||-0.039|(0.046)|-0.053|(0.046)|
|Number of migrants|||0.070|(0.043)|0.054|(0.043)|
|_Ethnicity (Reference: Tajik)_|||||||
|Uzbek|||0.064|(0.046)|0.070|(0.046)|
|Russian|||-0.412**|(0.161)|-0.435***|(0.166)|
|Other|||-0.650***|(0.159)|-0.664***|(0.149)|
|_Education (Reference: No Education)_|||||||
|<br>Primary|||-0.013|(0.113)|-0.112|(0.115)|
|<br>Basic|||-0.184*|<br>(0.109)|-0.220**|(0.111)|
|General Secondary|||-0.086|(0.105)|-0.129|(0.107)|
|<br>Special Secondary|||-0.021|<br>(0.113)|-0.072|(0.114)|
|<br>Technical Secondary|||-0.067|<br>(0.115)|-0.123|(0.116)|
|<br>Higher Education|||0.189*|<br>(0.112)|0.157|(0.114)|
|<br>Graduate School|||0.511|<br>(0.399)|0.612*|(0.368)|
|_Employment Characteristics of Household, Head’s_<br>|_Occupation (_|_Reference_|_for Occupation_<br>|_: Unemplo_<br>|_yed)_<br>||
|Number of employed|||0.013|(0.018)|0.007|(0.019)|
|Agriculture / Fishing / Forestry|||0.063|(0.063)|0.056|(0.064)|
|<br>Manufacture / Mining|||0.054|(0.092)|0.082|(0.095)|
|<br>Services|||0.447***|(0.120)|0.391***|(0.118)|
|Construction|||-0.032|(0.077)|-0.071|(0.077)|
|Public Administration / Education / Health|||0.094|<br>(0.067)|0.100|(0.067)|
|Sales and Services|||0.236***|(0.063)|0.241***|(0.063)|
|Other|||-0.142|(0.108)|-0.141|(0.107)|
|_Agriculture (Reference for holdings: No Land)_|||||||
|Small Holding|||-0.039|(0.064)|0.006|(0.066)|
|<br>Medium Holding|||0.055|<br>(0.078)|0.089|(0.080)|
|Large Holding|||0.088|(0.077|0.138*|(0.078)|
|<br>Any livestock, poultry, etc|||-0.017|<br>(0.056)|-0.021|(0.057)|
|<br>_Geography (Reference: Dushanbe)_|||||||
|<br>Sogd Urban|||-0.016|(0.072)|-0.087|(0.072)|
|<br>Sogd Rural|||-0.090|(0.078)|-0.170**|(0.079)|
|<br>Khatlon Urban|||-0.092|<br>(0.089)|-0.334***|(0.088)|
|Khatlon Rural|||-0.471***|<br>(0.073)|-0.560***|(0.074)|
|RRP Urban|||-0.286***|<br>(0.107)|-0.343***|(0.108)|
|RRP Rural|||-0.243***|(0.076)|-0.306***|(0.077)|
|Gbao Urban|||0.571***|(0.115)|0.657***|(0.120)|

22
|(1)<br>PCE<br>Coef.|se|(2)<br>PCE +<br>vignettes<br>Coef.|se|(3)<br>PCE +<br>household<br>controls<br>Coef.|se|(4)<br>PCE +<br>household<br>controls +<br>vignettes<br>Coef.|se|
|---|---|---|---|---|---|---|---|
|Gbao Rural||||0.036|(0.079)|0.006|(0.080)|
|_Vignette 1 (positions 3 to 6 omitted)_||||||||
|vign1==1||-0.385***|(0.126)|||-0.404***|(0.135)|
|<br>vign1==2||-0.184|(0.120)|||-0.231*|(0.129)|
|_Vignette 2 (position 1 omitted)_||||||||
|vign2==2||0.940***|(0.193)|||0.991***|(0.203)|
|vign2==3||1.345***|(0.199)|||1.449***|(0.209)|
|<br>vign2==4||1.525***|(0.230)|||1.693***|(0.240)|
|<br>vign2==5||2.842***|(0.363)|||2.882***|(0.314)|
|_Vignette 3 (positions 1 and 2 omitted)_||||||||
|vign3==3||-0.115|(0.243)|||-0.045|(0.281)|
|<br>vign3==4||-0.265|(0.249)|||-0.157|(0.287)|
|<br>vign3==5||-0.160|(0.257)|||-0.056|(0.294)|
|vign3==6||-0.636|(0.431)|||-0.698*|(0.423)|
|_Vignette 4 (positions 1 to 3 omitted)_<br>vign4==4||0.666**|(0.265)|||0.584**|(0.287)|
|<br>vign4==5||0.667**|(0.268)|||0.510*|(0.290)|
|<br>vign4==6||0.652**|(0.271)|||0.492*|(0.293)|
|Number of observations<br>4,771||4,77|1|4,771||4,77|1|
|Pseudo R<sup>2</sup><br>0.029||0.05|7|0.061||0.09|2|

note: *** p<0.01, ** p<0.05, * p<0.1 23
# **Table 11: Rescaled SW responses using Post Vignettes**

|(1)<br>PCE|(2)<br>PCE +<br>vignettes|(3)<br>PCE +<br>household<br>controls||(4)<br>PCE +<br>household<br>controls +<br>vignettes||
|---|---|---|---|---|---|
|Coef.<br>se|Coef.<br>se|Coef.|se|<br>Coef.|se|
|Log expenditure per capita<br>0.466***<br>0.034|0.559***<br>0.035|0.553***|0.039|0.646***|0.040|
|_Household Demographics_||||||
|Female headed household||-0.157***|0.052|-0.163***|0.052|
|Age of household head||-0.011|0.009|-0.009|0.009|
|Age of household head squared||0.000|0.000|0.000|0.000|
|<br>Household size||0.062***|0.013|0.063***|0.013|
|Number of children||-0.003|0.017|0.004|0.017|
|Number of older adults||-0.047|0.045|-0.034|0.046|
|Number of migrants||0.051*|0.028|0.067**|0.028|
|<br>_Ethnicity (Reference: Tajik)_||||||
|<br>Uzbek||0.032|0.045|0.067|0.046|
|Russian||-0.422***|0.161|-0.431***|0.156|
|Other||-0.488***|0.124|-0.613***|0.145|
|_Education (Reference: No Education)_||||||
|Primary||-0.208*|0.113|-0.086|0.113|
|Basic||-0.218**|0.110|-0.184*|0.109|
|General Secondary||-0.155|0.105|-0.112|0.105|
|Special Secondary||-0.147|0.112|-0.083|0.113|
|<br>Technical Secondary||-0.154|0.116|-0.089|0.116|
|<br>Higher Education||0.091|0.113|0.162|0.113|
|<br>Graduate School||0.602**|0.257|0.468*|0.253|
|_Employment Characteristics of Household, Head’s_|_Occupation (Refere_|_nce for Occu_|_ation: U_|_nemployed)_||
|<br>Number of employed||<br>-0.012|<br>0.018|<br>0.007|0.018|
|<br>Agriculture / Fishing / Forestry||0.057|0.065|0.041|0.066|
|<br>Manufacture / Mining||0.034|0.088|0.034|0.090|
|Services||0.333**|0.133|0.404***|0.124|
|Construction||-0.095|0.076|-0.070|0.078|
|Public Administration / Education / Health||0.097|0.067|0.108|0.067|
|Sales and Services||0.201***|0.062|0.229***|0.063|
|Other||-0.084|0.103|-0.142|0.114|
|_Agriculture (Reference for holdings: No Land)_||||||
|<br>Small Holding||0.055|0.064|-0.006|0.066|
|Medium Holding||0.113|0.080|0.071|0.080|
|<br>Large Holding||0.135*|0.077|0.133*|0.079|
|<br>Any livestock, poultry, etc||-0.004|0.055|-0.005|0.058|
|<br>_Geography (Reference: Dushanbe)_||||||
|<br>Sogd Urban||-0.099|0.067|-0.086|0.070|
|<br>Sogd Rural||-0.150*|0.077|-0.200**|0.078|
|<br>Khatlon Urban||-0.497***|0.085|-0.287***|0.089|
|Khatlon Rural||-0.525***|0.072|-0.527***|0.072|
|RRP Urban||-0.242**|0.108|-0.316***|0.109|

24
|(1)<br>PCE<br>Coef.|se|(2)<br>PCE +<br>vignettes<br>Coef.|se|(3)<br>PCE +<br>household<br>controls<br>Coef.|<br>se|(4)<br>PCE +<br>household<br>controls +<br>vignettes<br>Coef.|se|
|---|---|---|---|---|---|---|---|
|RRP Rural||||-0.210***|0.075|-0.263***|0.076|
|Gbao Urban||||0.370***|0.098|0.439***|0.097|
|Gbao Rural||||-0.079|0.076|0.003|0.079|
|_Vignette 1 (positions 3 to 6 omitted)_||||||||
|vign1==1||0.139|0.134|||0.161|0.145|
|vign1==2||0.168|0.132|||0.152|0.144|
|_Vignette 2 (position 1 omitted)_<br>vign2==2||0.497***|0.179|||0.498**|0.198|
|<br>Vign2==3||0.207|0.182|||0.226|0.201|
|<br>Vign2==4||-0.353*|0.205|||-0.311|0.225|
|<br>Vign2==5||-0.075|0.433|||-0.195|0.416|
|<br>_Vignette 3 (positions 1 and 2 omitted)_||||||||
|<br>Vign3==3||-1.143***|0.290|||-1.137***|0.314|
|<br>Vign3==4||-1.739***|0.293|||-1.722***|0.318|
|Vign3==5||-1.778***|0.297|||-1.772***|0.321|
|<br>Vign3==6||-2.110***|0.437|||-2.258***|0.446|
|<br>_Vignette 4 (positions 1 to 3 omitted)_||||||||
|<br>Vign4==4||-0.112|0.388|||-0.212|0.399|
|<br>Vign4==5||-0.137|0.390|||-0.309|0.402|
|<br>Vign4==6||-0.172|0.392|||-0.351|0.403|
|Number of observations<br>4,771||4,771||4,771||4,771||
|Pseudo R<sup>2</sup><br>0.017||0.075||0.042||0.101||

Note: *** p<0.01, ** p<0.05, * p<0.1 25
<!-- Start of picture text --> Pre-Vignette SV¥ Question Pre-Vignette SV Question<br>c Without Controls = With Controls<br>o * om aaa<br>a a & ny<br>o “Ay wo Aha,<br>ow a o<br>mi ee<br>z =<br>=a _ =a<br>5 i<br>ceer Eo.<br>= ‘ EEE = “Seamed<br>= 3<br>1G _ a0 re<br>20 a og wllF 3 um<br>2 “ im<br>a 3 4 i) 6 ri<br>3 4 rn) . 6 ? log per capita real consumption<br>log per capita real consumption<br>** Poor (withoutvigneties) —* Poor (with vignettes) ‘PFFish twtthoutithout vingnettesytt '#Prich ethith vionattesyvignett<br>Rich without vignettes) «= Rich éwith vignettes)<br>Post-Vignette SVV Question Post-Vignette SV¥ Question<br>= Without Controls c With Controls<br>E wg, BS My,<br>oa ty ao ‘a<br>Be TM Bo TM<br>z zr<br>= sae eevee a or oie RE<br>=o =<br>5" En<br>oa a<br>iraao irsao<br>‘am eT Bey<br>= eee = , eee<br>3 =<br>oc pout ees rid vy apes<br>So ae aoe<br>im o<br>3 4 8 ; 6 f 3 4 5 6 fi<br>log per capita real consumption log per capita real consumption<br>' Poor without vignettes) ° Poor Qwith vignettes) + Poor (without vignettes) «+ Poor (with vignettes)<br>* Rich fwithoutvignettes) «= Rich (with vignettes) * Rich gvithout vignettes) * Rich éwith vignettes)<br><!-- End of picture text -->
<!-- Start of picture text --> Pre vs. Post<br>a :<br>EE oo ar<br>o1<br>awa ae<br>xTo!<br>5) ee<br>in<br>ao<br>Qa<br>oa<br>o<br>am<br>= ST<br>=<br>o oom<br>ei.<br>3 4 4 6<br>log per capita real consumption<br>* Poor (pre-vignette} * Poor (pastvignette)<br>» Rich (pre-vignette} » Rich (postyignette)<br><!-- End of picture text --> <!-- Start of picture text --> Control vs No Control f Pre }<br>co<br>F; met<br>onToo me<br>se<br>=<br>= orie TE<br>et<br>oa<br>a<br>am<br>ra<br>am<br>=]= SSRimimmis<br>27x - ee<br>ie wom<br>3 4 4 6<br>log per capita real consumption<br>* Poor (no contrals) + Poor tno cantrols)<br>| Rich (controls) * Rich tcontrals)<br><!-- End of picture text -->

<!-- Start of picture text --> Control vs No Control f Post }<br>S ‘,<br>feo no,<br>5 mS,<br>a wt<br>= ‘.<br>ee<br>=<br>=a pe .<br>B=<br>par<br>ao<br>irs<br>3S<br>adLud _oos!<br>a i<br>oo<br>3 4 5 6<br>log per capita real consumption<br>* Poor (no contrals) + Poor tno cantrols)<br>| Rich (controls) * Rich tcontrals)<br><!-- End of picture text -->
# **Appendix 1: Summary statistics (weighted)**

|**Variable**|**mean**|**sd**|
|---|---|---|
|Log expenditure per capita (Tajik somoni)|5.0|0.54|
|_Household Demographics_|||
|Female-headed household|0.19||
|Age of household head<br>|50.6<br>|13.74<br>|
|Household size|6.3|2.80|
|Number of children (<15)|2.2|1.69|
|Number of older adults (65+)|0.3|0.57|
|Number of migrants|0.3|0.68|
|_Ethnicity_|||
|Tajik|0.79||
|Uzbek|0.18||
|Russian<br>|0.01<br>||
|Other|0.02||
|_Education of household head_|||
|No education|0.04||
|Primary (grades 1-4)|0.07||
|Basic (grades 1-8)|0.12||
|Secondary general (grades 9-10)|0.34||
|Secondary special|0.13||
|Secondary technical|0.11||
|Higher education|0.19||
|Graduate school/_aspirantura_|0.002||
|_Employment characteristics of Household, Head’s_|_occupation_||
|Number of employed|1.76|1.33|
|Not employed|0.37||
|Agriculture, fishing and forestry|0.18||
|Manufacture and mining|0.04||
|Services (electricity, gas, hot water, etc.)|0.02||
|Construction|0.07||
|Public administration, education, health|0.13||
|Sales and services|0.16||
|Other|0.03||
|_Agriculture_|||
|No land holdings|0.37||
|<br>Small holding (1-10 acres)|0.29||
|Medium holding (11-20 acres)|0.15||
|Large holding (21+ acres)|0.20||
|Anylivestock, poultry, beehives,fishetc.|0.55||

28
# **Appendix 2: The vignettes** in the TLSMS

<u>Vignette 1:</u> Family A can only afford to eat meat on very special occasions. During the winter months, they are able to partially heat only one room of their home. They cannot afford for children to complete their secondary education because the children must work to help support the family. When the children are able to attend school, they must go in old clothing and worn shoes. There is not enough warm clothing for the family during cold months. The family does not own any farmland, only their household vegetable plot.

<u>Vignette 2:</u> Family B can afford to eat meat only once or twice a week. During winter months, they can heat several rooms, but not the entire house. They cannot afford for all their children to complete secondary education. Their clothing is sufficiently warm, but they own only simple garments. In addition to their household vegetable plot, they own a small plot of poor quality farmland that is distant from their home.

<u>Vignette 3:</u> Family C can afford to eat meat everyday. During the winter months, generally they are able to keep their home warm. They can afford for all their children to complete secondary education. They have sufficient clothing to keep warm in the winter. Their everyday clothing is simple, but they also have some fancy items for special occasions. In addition to their household vegetable plot, they have a larger plot of good quality farmland, not too distant from their home.

<u>Vignette 4:</u> Family D can afford to eat whichever foods they would like, including sweets and imported food. During the winter months, they have no problems with heating and are able to keep their entire house warm. They can afford for all of their children to complete their education, and then to continue at a local university. They are able to afford a variety of fancy traditional clothes and also imported brand clothing. The family owns property, including a good car. The family also has a large farm and acts as landlord to others in their area.

29
# **Appendix 3: Probit for perversely-ordered vignettes**

|**Coeff.**|**s.e.**|
|---|---|
|Log expenditure per capita<br>0.045|0.092|
|_Household Head Demographics_||
|Female<br>0.190|0.141|
|Age<br>-0.028|0.025|
|Age squared<br>0.000|0.000|
|_Ethnicity (Reference: Tajik)_||
|Uzbek<br>-0.227|0.155|
|Russian<br>0.184|0.439|
|Other<br>0.629**|0.289|
|_Education (Reference: No Education)_||
|Primary<br>-0.368|0.244|
|Basic<br>-0.536**|0.266|
|General Secondary<br>-0.561**|0.235|
|Special Secondary<br>-0.771**|0.309|
|Technical Secondary<br>-0.356|0.265|
|<br>Higher Education<br>-0.800***|0.291|
|_Household Characteristics_||
|Household size<br>-0.017|0.052|
|Number of children (<15)<br>-0.028|0.062|
|Number of elderly (65+)<br>-0.278**|0.133|
|Number of migrants<br>-0.071|0.086|
|_Head’s Employment (Reference for Occupation: Unemployed)_||
|Number of employed<br>0.086|0.060|
|Agriculture / Fishing / Forestry<br>0.065|0.187|
|Manufacture / Mining<br>-0.509|0.363|
|<br>Services<br>0.314|0.306|
|Construction<br>0.467**|0.219|
|Public Administration / Education / Health<br>0.078|0.200|
|Sales and Services<br>0.265|0.190|
|Other<br>0.055|0.323|
|_Ownership of Land / Livestock_||
|Small Holding<br>0.174|0.191|
|Medium Holding<br>0.153|0.243|
|<br>Large Holding<br>0.449**|0.214|
|Any livestock, poultry, etc.<br>0.214|0.166|
|_Geography_||
|Sogd Urban<br>0.219|0.225|
|Sogd Rural<br>-0.389|0.262|
|Khatlon Urban<br>-0.393|0.317|
|Khatlon Rural<br>-0.348|0.271|
|RRP Urban<br>0.507**|0.248|
|RRP Rural<br>-0.396|0.256|
|Gbao Urban<br>-0.030|0.450|
|Gbao Rural<br>0.348|0.252|
|_Relative Subjective Welfare_||
|<br>Neighbor's Step<br>-0.247***|0.074|
|Number of observations<br>4,852||
|Pseudo R2<br>0.133||

30
**Appendix 4: Vignette Regressions (Ordered Probit)** **<u>Vignette 1</u>**

|(1)|(2)<br>PCE +<br>household||
|---|---|---|
|PCE|characteristics||
|Coef.<br>se|Coef.|se|
|Log expenditure per capita<br>0.008<br>0.038|0.044|0.043|
|_Household Demographics_|||
|Female headed household|0.047|0.059|
|Age of household head|-0.009|0.011|
|<br>Age of household head squared|0.000|0.000|
|Household size|0.030**|0.014|
|Number of children|-0.017|0.020|
|Number of older adults|-0.073|0.054|
|Number of migrants|-0.006|0.032|
|<br>_Ethnicity (Reference: Tajik)_|||
|<br>Uzbek|-0.074|0.052|
|Russian|0.041|0.166|
|Other|0.146|0.151|
|_Education (Reference: No Education)_|||
|Primary|-0.072|0.121|
|Basic|-0.207*|0.112|
|General Secondary|-0.023|0.109|
|Special Secondary|0.018|0.119|
|Technical Secondary|0.037|0.123|
|<br>Higher Education|-0.020|0.118|
|Graduate School|-0.098|0.458|
|_Employment Characteristics of Household, Head’s Occu_|_pation_||
|_(Reference for Occupation: Unemployed)_|||
|Number of employed|-0.020|0.021|
|Agriculture / Fishing / Forestry|0.074|0.074|
|Manufacture / Mining|0.101|0.103|
|Services|0.341**|0.173|
|Construction|0.046|0.095|
|Public Administration / Education / Health|0.021|0.082|
|Sales and Services|-0.009|0.072|
|Other|0.077|0.116|
|_Agriculture (Reference for holdings: No Land)_|||
|Small Holding|0.168**|0.072|
|Medium Holding|0.086|0.092|
|<br>Large Holding|0.057|0.089|
|Anylivestock, poultry, etc|0.039|0.062|
|_Geography (Reference: Dushanbe)_|||
|<br>Sogd Urban|0.095|0.082|
|<br>Sogd Rural|0.009|0.087|
|<br>Khatlon Urban|-0.670***|0.133|
|Khatlon Rural|-0.115|0.083|
|RRP Urban|0.101|0.120|
|RRP Rural|0.132|0.080|
|Gbao Urban|-0.121|0.123|
|Gbao Rural|-0.281***|0.088|
|Number of observations<br>4,771|4,771||
|Pseudo R2<br>0.000|0.022||

31
**<u>Vignette 2</u>**

|(1)|(2)<br>PCE +<br>household||
|---|---|---|
|PCE|characteristics||
|Coef.<br>se|Coef.|se|
|Log expenditure per capita<br>0.057<br>0.036|0.047|0.059|
|_Household Demographics_|||
|Female headed household|-0.024|0.056|
|Age of household head<br>|0.009<br>|0.011<br>|
|Age of household head squared|-0.000|0.000|
|Household size|-0.018|0.014|
|Number of children|0.023|0.019|
|Number of older adults|0.042|0.053|
|Number of migrants|0.037|0.031|
|_Ethnicity (Reference: Tajik)_|||
|Uzbek|0.006|0.050|
|Russian|0.063|0.165|
|Other|-0.072|0.147|
|_Education (Reference: No Education)_<br>|||
|Primary|0.093|0.113|
|Basic|0.154|0.108|
|General Secondary|0.182|0.116|
|Special Secondary|0.216*|0.119|
|Technical Secondary|0.152|0.116|
|Higher Education|-0.330|0.596|
|Graduate School|-0.024|0.056|
|_Employment Characteristics of Household, Head’s Occu_|_pation_||
|<br>_(Reference for Occupation: Unemployed)_|||
|Number of employed|0.041**|0.020|
|Agriculture / Fishing / Forestry|-0.032|0.070|
|Manufacture / Mining|-0.051|0.094|
|Services|0.223|0.139|
|Construction|0.110|0.087|
|Public Administration / Education / Health|0.021|-0.017|
|Sales and Services|0.000|0.067|
|Other|-0.113|0.111|
|_Agriculture (Reference for holdings: No Land)_|||
|<br>Small Holding|-0.164**|0.066|
|Medium Holding|-0.137|0.084|
|Large Holding|-0.124|0.084|
|Anylivestock, poultry, etc|0.013|0.059|
|_Geography (Reference: Dushanbe)_|||
|<br>Sogd Urban|0.232***|0.077|
|<br>Sogd Rural|0.164**|0.083|
|<br>Khatlon Urban|0.743***|0.110|
|Khatlon Rural|0.263***|0.081|
|RRP Urban|0.070|0.105|
|RRP Rural|0.067|0.077|
|Gbao Urban|-0.269*|0.139|
|Gbao Rural|0.175**|0.085|
|Number of observations<br>4,771|4,771||
|Pseudo R2<br>0.000|0.018||

32
**<u>Vignette 3</u>**

|(1)|(2)<br>PCE +<br>household||
|---|---|---|
|PCE|characteristics||
|Coef.<br>se|Coef.|se|
|Log expenditure per capita<br>0.092***<br>0.035|0.070*|0.039|
|_Household Demographics_|||
|Female headed household<br>|0.043<br>|0.053<br>|
|Age of household head<br>|0.012<br>|0.010<br>|
|Age of household head squared|-0.000|0.000|
|Household size|-0.017|0.013|
|Number of children|0.029|0.018|
|Number of older adults|0.048|0.052|
|Number of migrants|0.021|0.029|
|_Ethnicity (Reference: Tajik)_|||
|Uzbek|0.089*|0.047|
|Russian|0.035|0.148|
|Other|-0.177|0.143|
|_Education (Reference: No Education)_<br>|||
|Primary|0.183*|0.110|
|Basic|0.126|0.107|
|General Secondary|0.191*|0.114|
|Special Secondary|0.165|0.118|
|Technical Secondary|0.128|0.114|
|Higher Education|-0.429|0.682|
|Graduate School|0.043|0.053|
|_Employment Characteristics of Household, Head’s Occupa_|_tion_||
|<br>_(Reference for Occupation: Unemployed)_|||
|Number of employed|0.036*|0.019|
|Agriculture / Fishing / Forestry|-0.068|0.066|
|Manufacture / Mining|-0.028|0.089|
|Services|0.108|0.149|
|Construction|0.055|0.084|
|Public Administration / Education / Health|0.021|-0.025|
|Sales and Services|-0.019|0.062|
|Other|-0.021|0.119|
|_Agriculture (Reference for holdings: No Land)_|||
|<br>Small Holding|-0.114*|0.066|
|Medium Holding|-0.091|0.080|
|Large Holding|0.035|0.081|
|Anylivestock, poultry, etc|0.005|0.056|
|_Geography (Reference: Dushanbe)_|||
|<br>Sogd Urban|-0.050|0.072|
|<br>Sogd Rural|-0.173**|0.082|
|Khatlon Urban|0.466***|0.098|
|<br>Khatlon Rural|0.006|<br>0.078|
|RRP Urban|-0.127|0.111|
|RRP Rural|-0.166**|0.075|
|Gbao Urban|0.372***|0.138|
|Gbao Rural|0.220***|0.083|
|Number of observations<br>4,771|4,771||
|Pseudo R2<br>0.001|0.016||

33
**<u>Vignette 4</u>**

|(1)|(2)<br>PCE +<br>household||
|---|---|---|
|PCE|characteristics||
|Coef.<br>se|Coef.|se|
|Log expenditure per capita<br>0.119***<br>0.033|0.083**|0.037|
|_Household Demographics_|||
|Female headed household<br>|0.043<br>|0.053<br>|
|Age of household head<br>|0.002<br>|0.011<br>|
|Age of household head squared|-0.000|0.000|
|Household size|-0.016|0.013|
|Number of children|0.027|0.018|
|Number of older adults|0.064|0.052|
|Number of migrants|0.012|0.028|
|_Ethnicity (Reference: Tajik)_|||
|Uzbek|0.078*|0.047|
|Russian|0.099|0.154|
|Other|-0.162|0.162|
|_Education (Reference: No Education)_<br>|||
|Primary|0.089|0.122|
|Basic|0.163|0.119|
|General Secondary|0.194|0.126|
|Special Secondary|0.200|0.129|
|Technical Secondary|0.123|0.125|
|Higher Education|-0.325|0.452|
|Graduate School|0.043|0.053|
|_Employment Characteristics of Household, Head’s Occupa_|_tion_||
|<br>_(Reference for Occupation: Unemployed)_|||
|Number of employed|0.079***|0.020|
|Agriculture / Fishing / Forestry|-0.118*|0.066|
|Manufacture / Mining|0.032|0.087|
|Services|-0.191|0.152|
|Construction|-0.006|0.082|
|Public Administration / Education / Health|0.021|0.002|
|Sales and Services|-0.047|0.062|
|Other|-0.002|0.117|
|_Agriculture (Reference for holdings: No Land)_|||
|<br>Small Holding|-0.156**|0.067|
|Medium Holding|-0.165**|0.082|
|Large Holding|-0.031|0.082|
|Anylivestock, poultry, etc|0.030|0.059|
|_Geography (Reference: Dushanbe)_|||
|<br>Sogd Urban|-0.301***|0.068|
|<br>Sogd Rural|-0.384***|0.079|
|<br>Khatlon Urban|0.203*|0.104|
|Khatlon Rural|-0.219***|0.077|
|RRP Urban|-0.247**|0.112|
|RRP Rural|-0.340***|0.072|
|Gbao Urban|0.140|0.131|
|Gbao Rural|0.140*|0.084|
|Number of observations<br>4,771|4,771||
|Pseudo R2<br>0.002|0.019||

34