Expanding Upon Issues in

Measuring What We Spend:

Toward a New Consumer Expenditure Survey

Melvin Stephens Jr.

University of Michigan and NBER

October 16, 2012

Stephens (UM) Issues in Measuring What We Spend
Measuring What We Spend

The report contains the results of tremendous efforts by the panel to

synthesize a wealth of information (prior research, workshops, etc.)

identify potential improvements in the collection of expenditure data

produce prototype surveys that meet the CE requirements while improving

the overall quality of the CE data

do so within a relatively short period of time

Stephens (UM) Issues in Measuring What We Spend
Measuring What We Spend

The report contains the results of tremendous efforts by the panel to

synthesize a wealth of information (prior research, workshops, etc.)

identify potential improvements in the collection of expenditure data

produce prototype surveys that meet the CE requirements while improving

the overall quality of the CE data

do so within a relatively short period of time

While agreeing with the vast majority of the report, three of us felt that

there were some important areas that may be overlooked and/or were

underdeveloped.

Our discussion is labelled as a “dissent” in the report, but not really.

This presentation highlights these issues.

Stephens (UM) Issues in Measuring What We Spend
Current Interview and Diary Surveys

While the goal is to develop a new methodology, there are many lessons to

be learned from existing surveys.

The report suggests expanding diary-style data collection

Also states that existing interview and diary survey data are not comparable

Stephens (UM) Issues in Measuring What We Spend
Current Interview and Diary Surveys

While the goal is to develop a new methodology, there are many lessons to

be learned from existing surveys.

The report suggests expanding diary-style data collection

Also states that existing interview and diary survey data are not comparable

Recent bias and precision estimates compare these two methods

(Bee et al 2012).

Most interview survey expenditure categories are closer to national income

account totals than are diary survey categories.

Nearly all diary survey categories have much higher dispersion than the

corresponding interview categories.

Stephens (UM) Issues in Measuring What We Spend
Current Interview and Diary Surveys

While the goal is to develop a new methodology, there are many lessons to

be learned from existing surveys.

The report suggests expanding diary-style data collection

Also states that existing interview and diary survey data are not comparable

Recent bias and precision estimates compare these two methods

(Bee et al 2012).

Most interview survey expenditure categories are closer to national income

account totals than are diary survey categories.

Nearly all diary survey categories have much higher dispersion than the

corresponding interview categories.

Recent evidence from Canadian survey redesign

Switch from interview to diary led to 9-14 percent more under-reporting on

average (Debreuil et al. 2011).

Stephens (UM) Issues in Measuring What We Spend
Current Interview and Diary Surveys

While the goal is to develop a new methodology, there are many lessons to

be learned from existing surveys.

The report suggests expanding diary-style data collection

Also states that existing interview and diary survey data are not comparable

Recent bias and precision estimates compare these two methods

(Bee et al 2012).

Most interview survey expenditure categories are closer to national income

account totals than are diary survey categories.

Nearly all diary survey categories have much higher dispersion than the

corresponding interview categories.

Recent evidence from Canadian survey redesign

Switch from interview to diary led to 9-14 percent more under-reporting on

average (Debreuil et al. 2011).

Diary survey which includes use of incentives and multiple respondents

with tablet technology are primary features of report

Stephens (UM) Issues in Measuring What We Spend
Potential Limitations in Redesigns

While we are very positive about the redesigns, there are elements which

would be helpful to address

Stephens (UM) Issues in Measuring What We Spend
Potential Limitations in Redesigns

While we are very positive about the redesigns, there are elements which

would be helpful to address

Panel data is needed for microdata research that investigates changes in

consumption

Short waves (e.g., two weeks) provide incomplete picture of spending

Small number of waves reduces effective sample size for any given change

Inability to examine changes at lower frequencies throughout the designs

except Design B intensive subsample

Stephens (UM) Issues in Measuring What We Spend
Potential Limitations in Redesigns

While we are very positive about the redesigns, there are elements which

would be helpful to address

Panel data is needed for microdata research that investigates changes in

consumption

Short waves (e.g., two weeks) provide incomplete picture of spending

Small number of waves reduces effective sample size for any given change

Inability to examine changes at lower frequencies throughout the designs

except Design B intensive subsample

Modeling of expenditures reduces respondent burden

Complexity of decision requires including individual specific prices in model

Thwarts use of data to uncover these behavioral relationships

Potentially problematic for distributional analysis

Modeling based on current income problematic for numerous economic uses

Collection of spending over differing time periods in same survey raise a

different set of modeling issues.

Stephens (UM) Issues in Measuring What We Spend
Quite Helpful But Nearly Invisible Recommendations

We felt the next set of issues deserved greater attention than given, i.e.,

recommendations

Immediately changing language of intro letter to allow for linkages as in

American Community Survey (ACS)

Survey of Program Participation (SIPP)

Current Population Survey (CPS)

Stephens (UM) Issues in Measuring What We Spend
Quite Helpful But Nearly Invisible Recommendations

We felt the next set of issues deserved greater attention than given, i.e.,

recommendations

Immediately changing language of intro letter to allow for linkages as in

American Community Survey (ACS)

Survey of Program Participation (SIPP)

Current Population Survey (CPS)

Development of a gold standard research panel

Advocated in design B: the development of an intensive gold standard

survey that could be linked to a subset of the sample

Stephens (UM) Issues in Measuring What We Spend
Quite Helpful But Nearly Invisible Recommendations

We felt the next set of issues deserved greater attention than given, i.e.,

recommendations

Immediately changing language of intro letter to allow for linkages as in

American Community Survey (ACS)

Survey of Program Participation (SIPP)

Current Population Survey (CPS)

Development of a gold standard research panel

Advocated in design B: the development of an intensive gold standard

survey that could be linked to a subset of the sample

Having ability to balance budget during sample period

Requires spending and income collected over same interval

Provides additional check (“balance edit”) to improve data quality

Stephens (UM) Issues in Measuring What We Spend
Improving the Accuracy of the CPI

Using information on underreporting to improve survey design

Emphasis on gathering detail could be linked to item-specific underreporting

Stephens (UM) Issues in Measuring What We Spend
Improving the Accuracy of the CPI

Using information on underreporting to improve survey design

Emphasis on gathering detail could be linked to item-specific underreporting

Addressing underreporting by high income households

Recent study documents lower participation of high income households in

CE (Sabelhaus et al. 2011).

Underreporting of certain items (e.g., jewelry) is linked to higher income

households

Stephens (UM) Issues in Measuring What We Spend
Improving the Accuracy of the CPI

Using information on underreporting to improve survey design

Emphasis on gathering detail could be linked to item-specific underreporting

Addressing underreporting by high income households

Recent study documents lower participation of high income households in

CE (Sabelhaus et al. 2011).

Underreporting of certain items (e.g., jewelry) is linked to higher income

households

Role of complementary data sources

Much as the PCE combines expenditures from various sources

Helpful to define where might the CPI benefit from such data

E.g., Alcohol is underreported by 75% in CE

Ability to match such data (pulled from a variety of sources) to households

may be challenge that is as yet unexplored

Stephens (UM) Issues in Measuring What We Spend