NSDUH Redesign

Presentation at CES Redesign 

Conference

July 17, 2012 

Joel Kennet, Joe Gfroerer 

Center for Behavioral Health Statistics and Quality, SAMHSA 
Overview of NSDUH 

• Annual household survey of civilian, 

noninstitutionalized population aged 12 

and older 

• Provides national, state, and substate 

estimates of substance abuse and 

mental health issues 
Current NSDUH Sample Design 

• 7,200 segments; 140,000 dwelling units 

(DUs) 

• 68,000 respondents each year, 

representative nationally and in each 

State 

• n=3,600 in 8 large states, n=900 in other 

states 

• Oversampling of young people 

• 1/3 of sample in each age group: 12-17, 18-

25, 26+ 

3 
NSDUH Data Collection 

• 12-month data collection period 

• “Dear Resident” letters mailed to DUs 

• FI visits DUs, does 5 min. screener 

• Select 0, 1, or 2 HH members 

• Face-to-face interview (1 hr., mainly 

ACASI) 

• $30 given to each participant 

4 
Why Redesign? 

• Update questionnaire and sample 

design 

• Implement more efficient data 

collection and estimation methods 
Redesign Plan 

• Assess data needs 

• Contact States, other data users to determine what data are 

needed

• Methods research and development 

• Explore various design alternatives in terms of cost and impact 

on data quality and analytic capability 

• Assess effects on estimates, and whether redesign could be 

done with a split sample to “bridge” old and new estimates 

• Field test promising data collection methods 

• Present plan to key stakeholders to gain acceptance 

• Implement new sample design in 2014; Qx in 2015 
Methods Studies-Sampling 

• USPS Frame Study 

• Field test using frame constructed from residential 

mailing lists 

• Sample Issues Study 

• Determine optimal cluster sizes 

• Assess pros and cons of biennial survey 

• Assess impact of changing sampling rates for 

demographic and geographic groups 

• Develop more flexible sample design 

• Assess feasibility of interviewing children under 12 

• Investigation of Census/ACS Options 

• Assess the use of the ACS for sampling, small area 

estimation, and weighting activities 

7 
Methods Studies- Estimation 

• Imputation and Editing Evaluation 

• Evaluating different methods to simplify 

procedures and reduce processing time 

• Weighting Assessment 

• Examining different predictor variables to 

improve weighting adjustments 

• Small Area Estimation 

• Studying ways to estimate change and trends 

more efficiently and evaluate the quality of 

substate estimates 
Methods Studies-Response 

Rates 

• Contact Materials Study 

• Develop improved informational documents 

that respondents receive (lead letter, study 

description, question and answer brochure, 

etc.) 
Methods StudiesQuestionnaire 

• Electronic Pill Cards/Calendar Study 

• Develop on-screen prescription pill photos 

and reference date calendar; assess usability 

• Debriefing Questions/Persuasive 

Statement Study 

• Test whether reinforcing confidentiality and 

requesting honesty improves responses 

• Assess usefulness of FI and respondent 

debriefing questions (e.g., privacy, 

comprehension, etc.) as indicators of data 

quality. 
Methods StudiesQuestionnaire 

• Questionnaire Structure Study 

• Test “ensemble” vs. “interleafed” format 

• Explore moving more questions to core 

• Clinical Validation Phase II 

• Compare Substance Dependence and Abuse module to 

a structured clinical interview 

• Prescription Drug Module Redesign 

• Update definitions for nonmedical use and therapeutic 

classes 

• Incorporate new drugs 

• Move Methamphetamine out of prescription drugs 

• Develop better trend measurement method 
Other Relevant Studies 

• Mode/setting/context effects studies 

• Effects can be large for sensitive data 

• Impact of incentives on reporting 

• Significant positive effect found in 2002 

along with reduction in overall cost 

• Interviewer Effects Analysis 

• Interviewer experience correlates negatively 

with respondent reporting of drug use 
Schedule for NSDUH Redesign 

• November 2007 – November 2008: Develop background 

materials, plan methods tests, receive input from subjectmatter experts & other data users 

• June 2008 – September 2011: Conduct methods studies, 

develop and modify questionnaire content 

• May 2010 - April 2011: Mailing list field test, conduct focus 

groups, usability testing, and cognitive interviewing 

• May 2011: SAMHSA approval of redesign plan 

• August 2011 – January 2012: Cognitive testing of new 

questionnaire 

• Sept 2012 - Nov 2012: Questionnaire field test data collection 

• June 2013 – August 2013: Dress rehearsal data collection 

• January 2014 - New sample design begins 

• January 2015: New questionnaire begins 

13
Dilemma: Change the Measure or 

Measure Change? 

We would like to: 

• Maintain valid trend data-high priority 

• Update questionnaire 

• Data priorities change 

• New phenomena 

• Update methodology 

• Improve data quality 

• Incorporate better methods 

• Reduce Costs 

• Future budgets unknown 
New NSDUH Design: Sampling 

• Age, state sample reallocation (2014) 

• Improve precision, efficiency; reduce cost 

• Increase cluster size in some states (2014) 

• Cost savings with little loss in precision 

• Continue field counting and listing for now 

• Uncertainties about cost and coverage with address listbased 

• No change in target population 

15
New NSDUH Design: 

Instrumentation 

• No change in data collection methods 

• Update/improve contact materials (2015) 

• Maintain $30 incentive 

• New questionnaire (2015) 

• Nearly identical for cigarettes, alcohol, marijuana, cocaine, 

inhalants, hallucinogens, heroin 

• Major revision for prescription drugs 

• A few topics added, deleted, based on new priorities 

• No change in dependence/abuse—wait for DSM-V 

16
Improvements to Prescription 

Drug Module 

• Update drugs covered (every year) 

• Methamphetamine moved to a separate 

module 

• On-screen pictures of pills 

• Probe past year use/misuse of specific drugs 

• Separate components of “nonmedical” use: 

17
Partial NSDUH Redesign 

• Pros 

• Might fix most egregious problems, e.g., prescription 

drug use, methamphetamine use 

• Most likely no break in trends for alcohol, tobacco, 

marijuana, cocaine 

• Cons/risks 

• Will still result in break in trends for some measures 

• Some questionnaire improvements not made 

• Possibility of break in trends for all drugs due to 

context effects, contact materials, or sample design 

(e.g., impact on field staff) 

18

18
Contact info 

• joel.kennet@samhsa.hhs.gov

• joe.gfroerer@samhsa.hhs.gov