National Center for Health Statistics

Comparison of Mental Health Estimates by 

Sociodemographic Characteristics in RANDS 3 and 

2019 NHIS

Leanna Moron, Katherine Irimata, Jennifer Parker

FCSM Research and Policy Conference

November 3, 2021
Research Motivation

• This research is motivated to better understand the comparability of 

major depressive disorder (depression) and generalized anxiety disorder 

(GAD) estimates between two different types of data sources:

– web-based panel survey, Research and Development Survey Round 3 

(RANDS 3)

– traditional household survey, 2019 National Health Interview Survey 

(2019 NHIS)

• Important for national surveys to accurately measure and report mental 

health estimates
Research Questions

1. How do national estimates of depression and generalized anxiety 

disorder symptomology compare between RANDS 3 and 2019 NHIS?

2. How do these mental health subgroup estimates by select 

sociodemographic characteristics (sex, age, race and Hispanic origin, 

education, Census region) compare across the two data sources?

3. How do associations by the select sociodemographic characteristics 

compare across the two data sources?
Data Sources
Research and Development Survey (RANDS)

• RANDS is a primarily web-based, series of probability-sampled commercial 

panel surveys, established by NCHS in 2015

• Survey questions focus on a range of health-related topics such as chronic 

health conditions, and access to healthcare and utilization

• The strengths of RANDS include its cost-effective and timely production of 

national health estimates

• RANDS has been used for methodological and evaluation research 

purposes i.e., calibration weights, question-response patterns
Research and Development Survey Round 3 (RANDS 3)

• RANDS round 3 (n=2,646) was collected in April 2019 using NORC at the 

Uミi┗ersit┞ of ChiIago’s Aマeri“peak® paミel

• The cumulative response rate was 18.1%

• Calibration: raking approach to create calibrated weights based on using 

the following variables from the 2019 NHIS as the reference data set

– Demographic characteristics: age, sex, race/ethnicity, education, 

household income, census region

– Health conditions: ever diagnosed high cholesterol, ever diagnosed 

asthma, ever diagnosed hypertension and ever diagnosed diabetes
National Health Interview Survey (NHIS 2019)

• The NHIS is a cross-sectional household interview survey and 

is one of the major data collection programs of NCHS

• The main objective of the NHIS is to monitor the health of the 

U.S. population through the collection and analysis of data on 

a broad range of health topics

• The U.S. Census Bureau was the data collection agent for the 

2019 NHIS (n= 31,997)

• The response rate for the sample adult data file was 59.1%
Methods
Methods

• Measures: 8-item Patient Health Questionnaire (PHQ-8), 7-item Generalized 

Anxiety Disorder scale (GAD-7) 

• Statistical analysis: descriptive statistics, Rao-Scott chi-squared tests

• Sociodemographic characteristics:

– Sex (Male, Female)

– Age (18-44 years, 45-64 years, 65+ years)

– Race and Hispanic origin (Hispanic, NH Black, NH White, NH Other)

– Education (High School Graduate or Less, Some College, BaIhelor’s Degree or 

Above)

– Census region (Northeast, Midwest, South, West)
Patient Health Questionnaire (PHQ-8)

The eight-item Patient Health Questionnaire (PHQ-8) was used to measure level of depressive 

symptomology by asking respondents how often they have been bothered by the following 

symptoms in the past 2 weeks: 

1. さLittle iミterest or pleasure iミ doiミg thiミgsざ

2. さFeeliミg do┘ミ, depressed, or hopelessざ 

3. さTrouHle falliミg or sta┞iミg asleep, or sleepiミg too マuIhざ

4. さFeeliミg tired or ha┗iミg little eミerg┞ざ

5. さPoor appetite or o┗ereatiミgざ

6. さFeeliミg Had aHout ┞ourself, or that ┞ou are a failure, or ha┗e let ┞ourself or ┞our faマil┞ 

do┘ミざ

7. さTrouHle IoミIeミtratiミg oミ thiミgs, suIh as readiミg the ミe┘spaper or ┘atIhiミg 

tele┗isioミざ 

8. さMo┗iミg or speakiミg so slo┘l┞ that other people Iould ha┗e ミotiIed? Or the opposite, 

Heiミg so fidget┞ or restless that ┞ou ha┗e Heeミ マo┗iミg arouミd a lot マore thaミ usualざ 
Scoring of PHQ-8: Measure of Depressive Symptomology

• Respoミse optioミs ┘ere さミot at allざ, さse┗eral da┞sざ, さマore thaミ half of 

da┞sざ, aミd さミearl┞ e┗er┞ da┞ざ ┘ith ┗alues raミgiミg froマ 0 to 3, respeIti┗el┞

• Summed scores were categorized as the follows: 

– none-minimal severity (scores of 0-4)

– mild severity (scores of 5-9)

– moderate severity (scores of 10-14)

– severe severity (scores of 15-24)

• Any level of depressive symptomology includes mild, moderate and severe 

levels 
Generalized Anxiety Disorder Scale (GAD-7)

The seven-item Generalized Anxiety Disorder scale (GAD-7) was used to measure level 

of generalized anxiety disorder symptomology by asking respondents how often they 

had been bothered by the following symptoms in the past 2 weeks: 

1. さFeeliミg ミer┗ous, aミ┝ious, or oミ edgeざ 

2. さNot Heiミg aHle to stop or Ioミtrol ┘orr┞iミgざ 

3. さWorr┞iミg too マuIh aHout differeミt thiミgsざ 

4. さTrouHle rela┝iミgざ 

5. さBeiミg so restless that it’s hard to sit stillざ

6. さBeIoマiミg easil┞ aミミo┞ed or irritaHleざ 

7. さFeeliミg afraid as if soマethiミg a┘ful マight happeミざ 
Scoring of GAD-7: Measure of GAD Symptomology

• Respoミse optioミs ┘ere さミot at allざ, さse┗eral da┞sざ, さマore thaミ half of 

da┞sざ, aミd さミearl┞ e┗er┞ da┞ざ ┘ith ┗alues raミgiミg froマ 0 to 3, respeIti┗el┞

• Summed scores were categorized as follows: 

– none-minimal severity (scores of 0-4)

– mild severity (scores of 5-9)

– moderate severity (scores of 10-14)

– severe severity (scores of 15-21)

• Any level of generalized anxiety disorder symptomology includes mild, 

moderate and severe levels
Findings
Comparison of National Estimates of Depression 

• The national estimate of the percent of U.S. adults experiencing any 

level of severity of depression symptoms was significantly higher 

based on RANDS 3 compared to 2019 NHIS 

• The use of calibrated weights in RANDS 3 improved the national 

estimate of depression symptoms by reducing the standardized bias

Data Source n Percent (SE) Standardized Bias

RANDS 3 (NORC-provided Weight) 499 39.6 (1.79) 0.54

RANDS 3 (NHIS-Calibrated Weight) 499 37.4 (1.87) 0.49

2019 NHIS 5,898 18.5 (0.33) N/A
Comparison of Depression Subgroup Estimates

0%

10%

20%

30%

40%

50%

60%

RANDS 3 (NORC-provided Weight) RANDS 3 (NHIS Calibrated Weight) 2019 NHIS
Sociodemographic Associations with Depression

• Based on conducting Rao-Scott chi-squared testing, both 

data sources found associations with depressive 

symptomology by sex, age, race and Hispanic origin, and 

education

• However, the 2019 NHIS also identified associations with 

depressive symptomology by Census region
Comparison of National Estimates of GAD 

• The national estimate of the percent of U.S. adults experiencing any 

level of generalized anxiety disorder symptoms was significantly 

higher based on RANDS 3 compared to the 2019 NHIS

• The use of calibrated weights in RANDS 3 improved the national 

estimate of GAD symptoms by reducing the standardized bias

Data Source n Percent (SE) Standardized Bias

RANDS 3 (NORC-provided Weight) 505 39.5 (1.84) 0.66

RANDS 3 (NHIS-Calibrated Weight) 505 36.7 (2.00) 0.58

2019 NHIS 4,815 15.6 (0.29) N/A
Comparison of GAD Subgroup Estimates

0%

10%

20%

30%

40%

50%

60%

RANDS 3 (NORC-provided Weight) RANDS 3 (NHIS Calibrated Weight) 2019 NHIS
Sociodemographic Associations with GAD

• Both data sources found associations with generalized anxiety 

disorder symptomology by age, race and Hispanic origin, and 

education, as well as no associations by Census region

• However, the 2019 NHIS also identified associations with 

generalized anxiety disorder symptomology by sex
Discussion & Summary
Discussion and Limitations

• Potential reasons for observed differences may include: 

– differences in response rates by data source (59.1% for 

2019 NHIS and 18.1% for RANDS 3)

– differences in survey mode; in-person and telephone 

survey mode for 2019 NHIS, and web-based panel survey 

mode for RANDS 3

• Limitations include that these results can not be generalized 

for other mental health measures or data sources
Summary 

• The national and subgroup mental health estimates were 

significantly higher for nearly all sociodemographic groups based on 

RANDS 3 compared to 2019 NHIS 

• In general, the calibrated weights improved the national and 

subgroup mental health estimates by reducing standardized bias

• Most (though not all) of the sociodemographic associations 

identified for depression and GAD symptoms were the same across 

the two data sources
For more information, contact CDC

1-800-CDC-INFO (232-4636)

TTY: 1-888-232-6348 www.cdc.gov

The findings and conclusions in this presentation are those of the authors and do not necessarily 

represent the official position of the National Center for Health Statistics, Centers for Disease Control 

and Prevention.

https://www.cdc.gov/nchs/rands

Leanna Moron

qfa8@cdc.gov