DOI: 10.7910/dvn/snuugo
title: 2023 Consumer Spending by US Census Block Group

description: &lt;h1 id="blockgroupspending"&gt;blockgroupspending&lt;/h1&gt;
&lt;hr&gt;
&lt;h2 id="opportunity"&gt;Opportunity&lt;/h2&gt;
&lt;hr&gt;
&lt;p&gt;US Consumers express their behavior in a number of ways, but critically in their spending decisions. The US Bureau of Labor Statistics is charged with publishing spending activity and provides its Consumer Expenditure Survey (CEX) annually with US totals, with selected states (40) and cities (23). &lt;/p&gt;
&lt;p&gt;Limited to aggregates, the survey only needs 10s of thousands of observations in the original collection. While this is sufficient for macroeconomic use, the volume gives a weak basis for estimating lower levels of geography. In addition, the CEX includes demographic measurements that are similar, but not directly related, to Census variables. So, the CEX does not integtate well with the American Commuity Survey or other Census publications.&lt;/p&gt;
&lt;p&gt;This &lt;strong&gt;&lt;em&gt;blockgroupspending&lt;/em&gt;&lt;/strong&gt; publication by Open Environments attempts to address this problem by using the BLS&#39; Public Microdata (PUMD) sample to allocate CEX spending categories across 220,000 US Census block group geographies. For each block group, the effort applies two models to estimate:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;total consumer spending (regression)&lt;/li&gt;
&lt;li&gt;distribution of spending across spending categories (penetration) including Food, Transportation, Housing and Health costs.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Ultimately, these project spending on block groups that can be joined to US Census publications for additional demographics.&lt;/p&gt;
&lt;p&gt;Understanding the results requires awareness of the BLS&#39; CEX data structures. This is available in the markdown file named oe_bls_cex_EDA.md&lt;/p&gt;
&lt;p&gt;The publication is made together with the source python code and notebooks used for repeatability. The materials are maintained under version control at &lt;a href="https://github.com/OpenEnvironments/blockgroupspending"&gt;https://github.com/OpenEnvironments/blockgroupspending&lt;/a&gt;. All feedback and development requests are welcome.&lt;/p&gt;
&lt;h2 id="model-details"&gt;Model details&lt;/h2&gt;
&lt;p&gt;--
The CEX publication includes many files reflecting &lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;detailed &#39;diary&#39; surveys capturing spend on thousands of items every two weeks&lt;/li&gt;
&lt;li&gt;family &#39;interviews&#39; collecting household spending over the previous 3 months&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The models are trained upon the latter, &#39;FMLI&#39; files. The regression model uses extreme gradient boosting, or XGBoost methods that apply many decision trees to iteratively correct prediction error. The subcategory models also use tree based methods, trained upon a the family interview details. The spending variables are named, following the BLS&#39; CEX convention:
|Variable|Definition|2023|pct| 
|---|---|---|---|
|TOTEXP|Average annual expenditures|77280||
|FOOD|Food|9985|0.129|
|ALCBEV|Alcoholic beverages|637|0.008|
|HOUS|Housing|25436|0.329|
|APPAR|Apparel and services|2041|0.026|
|TRANS|Transportation|13174|0.17|
|HEALTH|Healthcare|6159|0.08|
|ENTERT|Entertainment|3635|0.047|
|PERSCA|Personal care products and services|950|0.012|
|READ|Reading|117|0.002|
|EDUCA|Education|1656|0.021|
|TOBACC|Tobacco products and smoking supplies|370|0.005|
|MISC|Miscellaneous|1184|0.015|
|CASHCO|Cash contributions|2378|0.031|
|RETPEN|Personal insurance and pensions|9556|0.124|&lt;/p&gt;
&lt;p&gt;During the exploratory phase of this effort, ensemble modelling was evaluated finding that different groupings of income did not appreciably change model estimates while racial and ethnic categories did. As a result, the models are case for major races (White, African American, Asian, Other) and Hispanic.&lt;/p&gt;
&lt;p&gt;The ACS is collected by API at the block group level. Block group geographies are the lowest level of Census ACS detail and consolidate into Census tracts which in turn consolidate into counties.&lt;/p&gt;
&lt;p&gt;The FMLI responses are recorded in nominal dollars throughout the year, while total expenditure and ACS data represent year end states. As a result, the models&#39; prediction for total expenditure is cast up using monthly inflation, weighted by monthly expenditure.&lt;/p&gt;
&lt;h2 id="additional-caveats"&gt;Additional Caveats&lt;/h2&gt;
&lt;p&gt;It is import to note, analytically, that the results are a stretch for credibility.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;CEX Consumer Units (people sharing financial decisions) are not exactly Census households (people in a housing unit)&lt;/li&gt;
&lt;li&gt;CEX demographics are not exactly Census demographics, with the CEX imputing incomes differenly than the Census medians.&lt;/li&gt;
&lt;li&gt;The CEX applies population weightings to the microdata while the Census primarily aggregates from respondents.&lt;/li&gt;
&lt;li&gt;The CEX observations are from 1 household (race is a 0/1 indicator) while Census demographics are many households (races are proportions)&lt;/li&gt;
&lt;li&gt;Models are trained upon repeated measures from a Consumer unit but not revised for ANOVA.&lt;/li&gt;
&lt;li&gt;Several of the CEX subcategories are very small, as spending has changed over the years. Reading, Alcohol and Tobacco use are still top level subcategories, for example as those have declined significantly since the CEX was first designed. So, this model is limited to the major subcategories of food, housing, transportation, health and retirement spending.*&lt;/li&gt;
&lt;li&gt;The model apply machine learning to large datasets so significance is not a consideration. However, in practice, those very small subcategories should be avoided.&lt;/li&gt;
&lt;li&gt;Difference in spending across racial categories also have different magnitudes and varieties. As a result, the R squared metrics on each submodel vary from 25%-45%.&lt;/li&gt;
&lt;li&gt;I find 40% R square results to be significant as they predict human behaviors rather than more deterministic relationships.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The resulting dataset is available from the Open Environments at dataverse.harvard.edu. The supporting code is shared on github.com/OpenEnvironments under MIT open source licensing. The final data files are:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;YYYYblockgroupspending.csv where YYYY is the four digit year&lt;/li&gt;
&lt;li&gt;Get_FMLI.ipynb to collect the files&lt;/li&gt;
&lt;li&gt;Get_ACS.ipynb uses the Census API to download select ACS variables at the block group level&lt;/li&gt;
&lt;li&gt;modelvars.csv configures the requested variables and the Census definition of each.&lt;/li&gt;
&lt;li&gt;blockgroupspending_regression.ipynb trains the regression model.&lt;/li&gt;
&lt;li&gt;blockgroupspending_prediction.ipynb uses the two models to predict total expenditure and it&#39;s subcategories in each blockgroup.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Other files are include for posterity, including functions to download and process other CEX PUMD data files.&lt;/p&gt;
&lt;p&gt;All questions or feedback is most welcome by email at support@openenvironments.com&lt;/p&gt;
&lt;h2 id="citations"&gt;Citations&lt;/h2&gt;
&lt;p&gt;U.S. Bureau of Labor Statistics. Consumer Expenditure Survey. Summary Tables. &lt;a href="https://www.bls.gov/cex/tables/top-line-means.htm"&gt;https://www.bls.gov/cex/tables/top-line-means.htm&lt;/a&gt;. 2023 &lt;/p&gt;
&lt;p&gt;U.S. Bureau of Labor Statistics. Consumer Expenditure Survey. Public Use Microdata files. &lt;a href="https://www.bls.gov/cex/pumd.htm"&gt;https://www.bls.gov/cex/pumd.htm&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;U.S. Census Bureau, “American Community Survey 5-Year Estimates: Comparison Profiles 5-Year,” 2022, &lt;a href="http://api.census.gov/data/2022/acs/acs5"&gt;http://api.census.gov/data/2022/acs/acs5&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;Python Package Index - PyPI. Python Software Foundation. &quot;A simple wrapper for the United States Census Bureau’s API.&quot;. Retrieved from &lt;a href="https://pypi.org/project/census/"&gt;https://pypi.org/project/census/&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;The conda-forge Project: Community-based Software Distribution Built on the conda Package Format and Ecosystem. XGBoost. &lt;a href="https://anaconda.org/conda-forge/xgboost"&gt;https://anaconda.org/conda-forge/xgboost&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;The conda-forge Project: Community-based Software Distribution Built on the conda Package Format and Ecosystem. Tensorflow. &lt;a href="https://anaconda.org/conda-forge/tensorflow"&gt;https://anaconda.org/conda-forge/tensorflow&lt;/a&gt;&lt;/p&gt;
&lt;h2 id="controls"&gt;Controls&lt;/h2&gt;
&lt;p&gt;Pre-COVID estimates use means from 2020.&lt;/p&gt;
&lt;p&gt;&lt;a href="https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-income-before-taxes-2020.pdf"&gt;https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-income-before-taxes-2020.pdf&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-income-quintiles-before-taxes-2020.pdf"&gt;https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-income-quintiles-before-taxes-2020.pdf&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-income-deciles-before-taxes-2020.pdf"&gt;https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-income-deciles-before-taxes-2020.pdf&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-region-1-year-average-2020.pdf"&gt;https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-region-1-year-average-2020.pdf&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-population-area-size-2020.pdf"&gt;https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-population-area-size-2020.pdf&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-area-type-2020.pdf"&gt;https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-area-type-2020.pdf&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-composition-2020.pdf"&gt;https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-composition-2020.pdf&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-education-highest-2020.pdf"&gt;https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-education-highest-2020.pdf&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-housing-tenure-2020.pdf"&gt;https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-housing-tenure-2020.pdf&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-earners-2020.pdf"&gt;https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-earners-2020.pdf&lt;/a&gt;&lt;/p&gt;
&lt;p&gt;&lt;a href="https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-size-2020.pdf"&gt;https://www.bls.gov/cex/tables/calendar-year/mean-item-share-average-standard-error/cu-size-2020.pdf&lt;/a&gt;&lt;/p&gt;