Sample Design in 3-Stage Household Surveys 

Supplemented by Commercial Lists 

Alena Maze 

PhD Candidate JPSM 

1 FCSM 2015 
Household Surveys with Over- or Under-Sampling 

 Household surveys often target particular demographic subgroups 

O Blacks 

O Hispanics 

O Age groups 

 Different ways to obtain target sample sizes 

O Select equal probability sample of HUs, screen persons for

eligibility, retain at rates to obtain sample size 

O Stratify SSUs by census or ACS data related to target 

groups; sample SSU strata at different rates 

O Use commercial lists with demographic info on HUs 

2 FCSM 2015 
Pros and Cons 

(1) Equal probabillity with screening 

O Expensive if oversampling rates differ by group 

O Many HUs may be screened out and dropped 

(2) SSU stratification 

o More efficient than (1) if strata directly related to target groups 

o Info is at block group level not HU 

(3) Commercial lists 

o Info is at HU level 

o Only ~60% of HUs have demographic info & may be wrong

3 FCSM 2015 
Goals of Dissertation 

- Estimate accuracy of commercial lists for identifying households with 

certain characteristics (e.g. Hispanics, non-hispanic blacks, teens 

(15-19), females, etc.) 

- Determine how to allocate two and three stage samples 

supplemented with commercial lists accounting for: 

o Inaccuracy of listings 

o Costs at each stage of sampling 

o Target sample sizes and CVs for estimates of subgroups 

o Stratification of SSUs by area characteristics (e.g. density of 

blacks, hispanics, others) 

o Stratification of HU’s by list characteristics (e.g. Race/ethnicity, 

ages of persons in HU, etc.)7 

o Characteristics of different variables of interest 

4 FCSM 2015 
Goals of Dissertation (continued) 

- Study alternative variance component estimators 

O Design-based (ANOVA) 

O Anticipated variances 

O Bayes 

5 FCSM 2015 
Previous Literature 

Iannacchione, V., J. Staab, and D. Redden (2003), “Evaluating the Use of 

Residential Mailing Lists in a Metropolitan Household Survey,” Public 

Opinion Quarterly, 67(2), 202–210. 

Roth, S. B., J. Montaquila, and D. Han (2012), “The ABS Frame: Quality and 

Considerations,” Proceedings of the Section on Survey Research Methods, 

3779-3793. 

Roth, S. B., D. Han, and J. Montaquila (2013), “The ABS Frame: Quality and 

Considerations,” Survey Practice, 6(4), available at 

http://surveypractice.org/index.php/SurveyPractice/article/view/73/pdf. 

Shook-Sa, B., D. Currivan, J. McMichael, and V. Iannacchione (2013), 

“Extending the Coverage of Address-Based Sampling Frames: Beyond the 

USPS Computerized Delivery Sequence File,” Public Opinion Quarterly. 

doi:10.1093/poq/nft041. 

Valliant, R., Hubbard, F., Lee, S. and Chang, C. (2014), “Efficient use of 

commercial lists in U.S. Household Sampling. Journal of Survey Statistics 

and Methodology,” 2: 182- 209. 

6 FCSM 2015 
           

   

     

       

Example from Health & Retirement Study 

Example based on HRS; LBB = Late Baby Boomers 

Data from screening results in National Survey of Family Growth compared to 

commercial list records 

Commercial list stratum LBB; B LBB; H LBB; Other Not LBB Unoccupied Total

1 LBB; no race‐eth 0.0000 0.0125 0.5322 0.4065 0.0487 1

2 LBB; B 0.2213 0.0163 0.1586 0.5384 0.0654 1

3 LBB; H 0.0081 0.2730 0.1400 0.5336 0.0453 1

4 LBB; Other 0.0238 0.0101 0.4493 0.4657 0.0510 1

5 Has record; Not LBB 0.0139 0.0101 0.0566 0.8691 0.0503 1

6 Has record; No age info 0.0159 0.0198 0.0496 0.7995 0.1152 1

7 No record 0.0121 0.0136 0.0635 0.6933 0.2175 1

Total 0.0163 0.0159 0.0883 0.7553 0.1241 1.000

- Commercial list info accurate enough to be useful but far from perfect 

- MP allocation accounts for inaccuracies in finding sampling rates 

7 FCSM 2015 
Variance of an Estimator of Total 

- 3-stage sample 

o m PSUs selected with pps with replacement 

o na SSUs stratified and selected ppswr within stratum a

o qab HUs selected by stsrs with SSU stratum a, list stratum b



1 U1 pwr 1  1 D U 2ia pwr

V t pwr

ˆ  

S

2

2 2 i U a1 

S

2 



m t mt p n U U  i a 

2

1 D 1 1 B Qiajb 2 



    j Uia b1 q

S3iaj  i U a1  p n p i a j ia ab 

2 2 2 B D W2a D B W3ab 

m

a1 mn a1 b1mn q a a ab

8 FCSM 2015 
 

 

B

2 2 2 

- , W2a, and W3ab are relvariance components to be estimated 

- Random effects model 

yiajbk  iajbk i   iaj   iajbk

T iajbk  xiajbkβ

 ~ 0,

2 

,  ~ 0,

2 

,  ~ 0,

2 

i    iaj   a  iajbk   

- Anticipated variance 

Compute E V M

ˆ ;  t pwr 

Esimate model variance components via ML, REML, Bayes 

9 FCSM 2015 
 

 

Sample Allocation is a Math Programming Problem 

- Allocation problem 

Find m n, ,q  to min 



V tˆ 



a ab  pwr 

subject to 

- minimum values of m na

, ,qab

- CV constraints on subgroup estimates (e.g., Blacks, Hispanics,

Others) 

- deff   w  dmax for different subgroups 

Or, could minimize cost s.t. constraints on sample sizes, CVs, etc. 

10 FCSM 2015 
Data & Analysis 

 NSFG or HRS screening/interview results matched to commercial list 

information 

 Estimate 

- List accuracy 

- Variance components 

 Evaluate cost of MP allocations vs. 

- Equal probability allocation + screening 

- SSU stratification only + MP allocation 

- MP allocation to list strata and no SSU stratification 

11 FCSM 2015