DOI: 10.57783/bsh9-sq03
title: Nielsen Consumer Panel Data

description: For any questions, please contact: Eric Vlach, Associate Director of Cloud & Systems Engineering, Arts & Sciences IT, at ev2322@columbia.edu
description: Through a relationship with NielsenIQ, the Kilts Center at the University of Chicago Booth School of Business provides multiple consumer datasets to academic researchers around the world.



Columbia has an agreement with the Kilts Center. Authorized faculty, graduate students, and research staff can apply to access this dataset.



Annual funding of the dataset is shared between the Program for Economic Research and the Libraries.



Funding for data analysis using the Columbia Data Platform is provided by the Program for Economic Research.




description: The Consumer Panel Data comprise a representative panel of households that continually provide information about their purchases in a longitudinal study in which panelists stay on as long as they continue to meet NielsenIQ's criteria. NielsenIQ consumer panelists use in-home scanners to record all of their purchases (from any outlet) intended for personal, in-home use. Consumers provide information about their households and what products they buy, as well as when and where they make purchases.



**Years Available:** Starting with 2004 and including annual updates.



**Panel Size:** 40,000–60,000 active panelists (varies by year), projectable to the total United States using household projection factors.



**Panelists:** Household demographic, geographic, and product ownership variables are included, as well as select demographics for the heads of household and other members.



- Demographic variables include household income range, size, composition, presence and age of children, marital status, type of residence, race, and Hispanic origin. Male and female heads of household also report age range, birth date, hours employed, education, and occupation. For other family members, birth date, employment, and relationship/sex are reported.
- Geographic variables include panelist zip code, FIPS state and county codes, region (East, Central, South, West), and Scantrack Market code (assigned by NielsenIQ, and called Syndicated Major Markets (SMM) starting in 2021).
- Product Ownership variables include kitchen appliances, TV items, and internet connection.

<!-- -->

**Products:** From 2004 to 2020, all 10 NielsenIQ food and nonfood Departments (\~1.4 million UPC codes). These Departments are dry grocery, frozen foods, dairy, deli, packaged meat, fresh produce, nonfood grocery, alcohol, general merchandise, and health and beauty aids.



Starting in 2021 data, NielsenIQ implemented a new Product Hierarchy structure. The original Product Hierarchy of Departments, Product Groups, and Product Modules has been phased out. For now, Product Module Codes will still be available to make the transition smoother.



**Product Characteristics:** All products include UPC code and description, brand, multipack, and size, as well as NielsenIQ codes for Department, Product Group, and Product Module. Some products contain additional characteristics (e.g., flavor). Starting in 2021 data, NielsenIQ expanded the number of additional product characteristics.



**Purchases: **Each shopping trip contains the date, retail chain code, retail channel, first three digits of store zip code, and total amount spent. For each product purchased, the UPC code, quantity, price, and any deals/coupons are recorded. Note that retailer names are not available.



**Geographies:** Entire United States, divided into 62 major markets.



**Retail Channels:** All retail channels—grocery, drug, mass merchandise, superstores, club stores, convenience, health, and others.