Comments on Improving Consumption Measurement

Chris Carroll

Johns Hopkins University

Remarks At CRIW-NBER Conference On Improving Consumption Measurement

Cambridge, MA

July 13, 2009
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Why Are Some Surveys/Countries Better Than Others?

My Best Guess

We Have A Serious Problem

1980 1985 1990 1995 2000 2005

7.5

7.6

7.7

7.8

7.9

8

8.1

8.2

8.3

Food Consumption Per−capita

Year

Log − 2003 $

1980 1985 1990 1995 2000 2005

8.8

8.9

9

9.1

9.2

9.3

9.4

9.5

9.6

Nondurable Consumption Per−capita

Year

Log − 2003 $

1980 1985 1990 1995 2000 2005

7.1

7.2

7.3

7.4

7.5

7.6

7.7

7.8

7.9

Durable Consumption Per−capita

Year

Log − 2003 $

1980 1985 1990 1995 2000 2005

7.4

7.5

7.6

7.7

7.8

7.9

8

8.1

8.2

Housing Services Per−capita

Year

Log − 2003 $

NIPA

CEX

PSID

NIPA

CEX

NIPA

CEX

NIPA

CEX

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Why Are Some Surveys/Countries Better Than Others?

My Best Guess

CE vs NIPA for Canada

Figure 2: Mean Nominal Non-Durable Consumption

0 5000 10000 15000

Consumption

1975 1980 1985 1990 1995 2000 2005

Year

SNA FAMEX/SHS

the SCF had unusually small samples in 1977, 1979, 1981 and 1984, which explains

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Why Are Some Surveys/Countries Better Than Others?

My Best Guess

CE vs NIPA for Germany

1985 1990 1995 2000 2005

1

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8 x 104 Per Capita Disposable Income from NIPA, EVS & GSOEP

Year

Per Capita Income in Euros, Prices of 2000

EVS

GSOEP

NIPA

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Why Are Some Surveys/Countries Better Than Others?

My Best Guess

CE vs NIPA for Italy

1 1.2 1.4 1.6 1.8

1980 1985 1990 1995 2000 2005

National Accounts SHIW

Figure 1: Average per-capita total consumption in the SHIW 

and in the National Accounts 

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Why Are Some Surveys/Countries Better Than Others?

My Best Guess

CE vs NIPA for Russia

son of RLMS with Official Statistics 

2004

IPA

1000 2000 3000 4000 5000

1994 1996 1998 2000 2002 2004

RLMS NIPA

B.Consumption per capita

 member

00

D.Non-food consumption per household member

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Why Are Some Surveys/Countries Better Than Others?

My Best Guess

CE vs NIPA for Sweden

         

1984 1986 1988 1990 1992 1994 1996 1998 2000

90

95

100

105

110

115

120

125

130

NIPA

HUT

SEK 1000

              

         

       

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Why Are Some Surveys/Countries Better Than Others?

My Best Guess

CE vs NIPA for UK

Figure 2.6: Income and Consumption Coverage 

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

19781980198219841986198819901992199419961998200020022004

Year

Coverag

Income Coverage

Consumption Coverage

In order to try to understand what may be behind the declining performance of 

the expenditure data, it is worth looking at some of the components behind the 

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Why Are Some Surveys/Countries Better Than Others?

My Best Guess

Commonalities

In Many Countries, Coverage Has Declined Seriously

In Some, Deterioration Is Accelerating

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Why Are Some Surveys/Countries Better Than Others?

My Best Guess

Commonalities

In Many Countries, Coverage Has Declined Seriously

In Some, Deterioration Is Accelerating

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Why Are Some Surveys/Countries Better Than Others?

My Best Guess

Differences:

In Person Vs Phone Interviews

Diary Vs Recall

Incentives To Participate

Quality of Interviewers

Quality of Survey (Focus Groups!)

Use Of Budget Constraint

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Why Are Some Surveys/Countries Better Than Others?

My Best Guess

Differences:

In Person Vs Phone Interviews

Diary Vs Recall

Incentives To Participate

Quality of Interviewers

Quality of Survey (Focus Groups!)

Use Of Budget Constraint

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Why Are Some Surveys/Countries Better Than Others?

My Best Guess

Differences:

In Person Vs Phone Interviews

Diary Vs Recall

Incentives To Participate

Quality of Interviewers

Quality of Survey (Focus Groups!)

Use Of Budget Constraint

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Why Are Some Surveys/Countries Better Than Others?

My Best Guess

Differences:

In Person Vs Phone Interviews

Diary Vs Recall

Incentives To Participate

Quality of Interviewers

Quality of Survey (Focus Groups!)

Use Of Budget Constraint

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Why Are Some Surveys/Countries Better Than Others?

My Best Guess

Differences:

In Person Vs Phone Interviews

Diary Vs Recall

Incentives To Participate

Quality of Interviewers

Quality of Survey (Focus Groups!)

Use Of Budget Constraint

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Why Are Some Surveys/Countries Better Than Others?

My Best Guess

Differences:

In Person Vs Phone Interviews

Diary Vs Recall

Incentives To Participate

Quality of Interviewers

Quality of Survey (Focus Groups!)

Use Of Budget Constraint

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Why Are Some Surveys/Countries Better Than Others?

My Best Guess

Differences:

In Person Vs Phone Interviews

Diary Vs Recall

Incentives To Participate

Quality of Interviewers

Quality of Survey (Focus Groups!)

Use Of Budget Constraint

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Why Are Some Surveys/Countries Better Than Others?

My Best Guess

Biggest Problem: Increased Reluctance to Participate

Differential Sample Selection: Richer ⇒ Less Willing

Second Biggest: Changes In Technology (Automated Payment, Etc)

If You Asked Me These Questions, I’d Have No Idea!

I’m Concerned You’re Drilling A Drying Up Hole

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Why Are Some Surveys/Countries Better Than Others?

My Best Guess

Biggest Problem: Increased Reluctance to Participate

Differential Sample Selection: Richer ⇒ Less Willing

Second Biggest: Changes In Technology (Automated Payment, Etc)

If You Asked Me These Questions, I’d Have No Idea!

I’m Concerned You’re Drilling A Drying Up Hole

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Why Are Some Surveys/Countries Better Than Others?

My Best Guess

Biggest Problem: Increased Reluctance to Participate

Differential Sample Selection: Richer ⇒ Less Willing

Second Biggest: Changes In Technology (Automated Payment, Etc)

If You Asked Me These Questions, I’d Have No Idea!

I’m Concerned You’re Drilling A Drying Up Hole

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Why Are Some Surveys/Countries Better Than Others?

My Best Guess

Biggest Problem: Increased Reluctance to Participate

Differential Sample Selection: Richer ⇒ Less Willing

Second Biggest: Changes In Technology (Automated Payment, Etc)

If You Asked Me These Questions, I’d Have No Idea!

I’m Concerned You’re Drilling A Drying Up Hole

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Why Are Some Surveys/Countries Better Than Others?

My Best Guess

Biggest Problem: Increased Reluctance to Participate

Differential Sample Selection: Richer ⇒ Less Willing

Second Biggest: Changes In Technology (Automated Payment, Etc)

If You Asked Me These Questions, I’d Have No Idea!

I’m Concerned You’re Drilling A Drying Up Hole

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Nielsen Data

Credit Card Data

Personal Finance Online

QuickenOnline

iBank

http://www.mint.com/product/videos/mint-demo/

Big Advantage: People Get Something Valuable At the End

50 percent chance that current structure should be ditched altogether

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Nielsen Data

Credit Card Data

Personal Finance Online

QuickenOnline

iBank

http://www.mint.com/product/videos/mint-demo/

Big Advantage: People Get Something Valuable At the End

50 percent chance that current structure should be ditched altogether

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Nielsen Data

Credit Card Data

Personal Finance Online

QuickenOnline

iBank

http://www.mint.com/product/videos/mint-demo/

Big Advantage: People Get Something Valuable At the End

50 percent chance that current structure should be ditched altogether

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Nielsen Data

Credit Card Data

Personal Finance Online

QuickenOnline

iBank

http://www.mint.com/product/videos/mint-demo/

Big Advantage: People Get Something Valuable At the End

50 percent chance that current structure should be ditched altogether

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Nielsen Data

Credit Card Data

Personal Finance Online

QuickenOnline

iBank

http://www.mint.com/product/videos/mint-demo/

Big Advantage: People Get Something Valuable At the End

50 percent chance that current structure should be ditched altogether

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Nielsen Data

Credit Card Data

Personal Finance Online

QuickenOnline

iBank

http://www.mint.com/product/videos/mint-demo/

Big Advantage: People Get Something Valuable At the End

50 percent chance that current structure should be ditched altogether

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Nielsen Data

Credit Card Data

Personal Finance Online

QuickenOnline

iBank

http://www.mint.com/product/videos/mint-demo/

Big Advantage: People Get Something Valuable At the End

50 percent chance that current structure should be ditched altogether

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Nielsen Data

Credit Card Data

Personal Finance Online

QuickenOnline

iBank

http://www.mint.com/product/videos/mint-demo/

Big Advantage: People Get Something Valuable At the End

50 percent chance that current structure should be ditched altogether

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Nielsen Data

Credit Card Data

Personal Finance Online

QuickenOnline

iBank

http://www.mint.com/product/videos/mint-demo/

Big Advantage: People Get Something Valuable At the End

50 percent chance that current structure should be ditched altogether

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Nielsen Data

Credit Card Data

Personal Finance Online

QuickenOnline

iBank

http://www.mint.com/product/videos/mint-demo/

Big Advantage: People Get Something Valuable At the End

50 percent chance that current structure should be ditched altogether

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Comparison of CEX Data With Other Sources Should Be Routinized

Every CE Report Should Show Comparisons With Other Sources, Say:

NIPA

Nielsen

CPS

Biannual Review Report Should Contain Broader Comparisons

SCF, Labor Market, SOI Tax Income, and “special topics”

Explain What It Should/Should Not Be Used For

Occasional External Reviews, Not Just Internal

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Comparison of CEX Data With Other Sources Should Be Routinized

Every CE Report Should Show Comparisons With Other Sources, Say:

NIPA

Nielsen

CPS

Biannual Review Report Should Contain Broader Comparisons

SCF, Labor Market, SOI Tax Income, and “special topics”

Explain What It Should/Should Not Be Used For

Occasional External Reviews, Not Just Internal

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Comparison of CEX Data With Other Sources Should Be Routinized

Every CE Report Should Show Comparisons With Other Sources, Say:

NIPA

Nielsen

CPS

Biannual Review Report Should Contain Broader Comparisons

SCF, Labor Market, SOI Tax Income, and “special topics”

Explain What It Should/Should Not Be Used For

Occasional External Reviews, Not Just Internal

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Comparison of CEX Data With Other Sources Should Be Routinized

Every CE Report Should Show Comparisons With Other Sources, Say:

NIPA

Nielsen

CPS

Biannual Review Report Should Contain Broader Comparisons

SCF, Labor Market, SOI Tax Income, and “special topics”

Explain What It Should/Should Not Be Used For

Occasional External Reviews, Not Just Internal

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Comparison of CEX Data With Other Sources Should Be Routinized

Every CE Report Should Show Comparisons With Other Sources, Say:

NIPA

Nielsen

CPS

Biannual Review Report Should Contain Broader Comparisons

SCF, Labor Market, SOI Tax Income, and “special topics”

Explain What It Should/Should Not Be Used For

Occasional External Reviews, Not Just Internal

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Comparison of CEX Data With Other Sources Should Be Routinized

Every CE Report Should Show Comparisons With Other Sources, Say:

NIPA

Nielsen

CPS

Biannual Review Report Should Contain Broader Comparisons

SCF, Labor Market, SOI Tax Income, and “special topics”

Explain What It Should/Should Not Be Used For

Occasional External Reviews, Not Just Internal

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Comparison of CEX Data With Other Sources Should Be Routinized

Every CE Report Should Show Comparisons With Other Sources, Say:

NIPA

Nielsen

CPS

Biannual Review Report Should Contain Broader Comparisons

SCF, Labor Market, SOI Tax Income, and “special topics”

Explain What It Should/Should Not Be Used For

Occasional External Reviews, Not Just Internal

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Comparison of CEX Data With Other Sources Should Be Routinized

Every CE Report Should Show Comparisons With Other Sources, Say:

NIPA

Nielsen

CPS

Biannual Review Report Should Contain Broader Comparisons

SCF, Labor Market, SOI Tax Income, and “special topics”

Explain What It Should/Should Not Be Used For

Occasional External Reviews, Not Just Internal

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Comparison of CEX Data With Other Sources Should Be Routinized

Every CE Report Should Show Comparisons With Other Sources, Say:

NIPA

Nielsen

CPS

Biannual Review Report Should Contain Broader Comparisons

SCF, Labor Market, SOI Tax Income, and “special topics”

Explain What It Should/Should Not Be Used For

Occasional External Reviews, Not Just Internal

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Comparison of CEX Data With Other Sources Should Be Routinized

Every CE Report Should Show Comparisons With Other Sources, Say:

NIPA

Nielsen

CPS

Biannual Review Report Should Contain Broader Comparisons

SCF, Labor Market, SOI Tax Income, and “special topics”

Explain What It Should/Should Not Be Used For

Occasional External Reviews, Not Just Internal

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

Comparison of CEX Data With Other Sources Should Be Routinized

Every CE Report Should Show Comparisons With Other Sources, Say:

NIPA

Nielsen

CPS

Biannual Review Report Should Contain Broader Comparisons

SCF, Labor Market, SOI Tax Income, and “special topics”

Explain What It Should/Should Not Be Used For

Occasional External Reviews, Not Just Internal

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

And Collaboration

Openness, Transparency, Accountability

How Do These Buzzwords Apply?

Unfolding Brackets of Complexity Of Data Use

Very Easy To Get Top Level Data

Clear Path to “Drill Down” to Deeper and Deeper Levels

A Lot Of Researchers Have Been Defeated By Complexity

Experiment, e.g., With HRS or PSID Instrument On CE Sample

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

And Collaboration

Openness, Transparency, Accountability

How Do These Buzzwords Apply?

Unfolding Brackets of Complexity Of Data Use

Very Easy To Get Top Level Data

Clear Path to “Drill Down” to Deeper and Deeper Levels

A Lot Of Researchers Have Been Defeated By Complexity

Experiment, e.g., With HRS or PSID Instrument On CE Sample

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

And Collaboration

Openness, Transparency, Accountability

How Do These Buzzwords Apply?

Unfolding Brackets of Complexity Of Data Use

Very Easy To Get Top Level Data

Clear Path to “Drill Down” to Deeper and Deeper Levels

A Lot Of Researchers Have Been Defeated By Complexity

Experiment, e.g., With HRS or PSID Instrument On CE Sample

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

And Collaboration

Openness, Transparency, Accountability

How Do These Buzzwords Apply?

Unfolding Brackets of Complexity Of Data Use

Very Easy To Get Top Level Data

Clear Path to “Drill Down” to Deeper and Deeper Levels

A Lot Of Researchers Have Been Defeated By Complexity

Experiment, e.g., With HRS or PSID Instrument On CE Sample

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

And Collaboration

Openness, Transparency, Accountability

How Do These Buzzwords Apply?

Unfolding Brackets of Complexity Of Data Use

Very Easy To Get Top Level Data

Clear Path to “Drill Down” to Deeper and Deeper Levels

A Lot Of Researchers Have Been Defeated By Complexity

Experiment, e.g., With HRS or PSID Instrument On CE Sample

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

And Collaboration

Openness, Transparency, Accountability

How Do These Buzzwords Apply?

Unfolding Brackets of Complexity Of Data Use

Very Easy To Get Top Level Data

Clear Path to “Drill Down” to Deeper and Deeper Levels

A Lot Of Researchers Have Been Defeated By Complexity

Experiment, e.g., With HRS or PSID Instrument On CE Sample

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

And Collaboration

Encourage Researchers With Setup Like Census Data Centers

Carroll Improving the CEX
It’s A Global Problem

New Technologies

Checks and Balances

Openness, Transparency, Accountability

Conclusion

A Golden Age!

CE Should Be A Part Of It, Not A Drowning Polar Bear

Carroll Improving the CEX