National Center for Health Statistics

Measuring Disability: An Evaluation of Differences 

Between the American Community Survey and 

Washington Group Short Set Questions

Julie D. Weeks

James M. Dahlhamer, Aaron Maitland, Jennifer H. Madans

2023 FCSM Research & Policy Conference

College Park, Maryland

24-26 October 2023

This presentation does not represent official opinions or statistics of

the National Center for Health Statistics or the Centers for Disease Control and Prevention.
 The National Health Interview Survey has included items on 

functioning since its inception, with measures changing over time as 

methods and models of disability have evolved.

• American Community Survey disability set (ACS-6): 2009-2017

• Washington Group Short Set on Functioning (WG-SS): 2010-current

 In 2011 and 2012, both sets of questions were asked of a subset of 

adults

 Opportunity to assess the overlap or agreement in responses to two 

sets of disability questions

Background/Context
 Multi-purpose household health survey

 In-person, CAPI interview (telephone follow-up, if necessary)

 Three main interview modules:

• Family (ACS-6)

• Sample child

• Sample adult (WG-SS)

 Interviews conducted in ~35,000 households each year

 Analytic sample: 18,079 adults who responded to both family and adult 

modules

National Health Interview Survey: 2011-2012
Vision

ACS: Are you blind or do you have serious difficulty seeing even when wearing glasses?

WG: Do you have difficulty seeing, even if wearing glasses?

Hearing

ACS: Are you deaf or do you have serious difficulty hearing?

WG: Do you have difficulty hearing, even if using a hearing aid?

Mobility

ACS: Do you have serious difficulty walking or climbing stairs?

WG: Do you have difficulty walking or climbing stairs?

Cognition

ACS: Because of a physical, mental, or emotional condition, does this person have 

serious difficulty concentrating, remembering or making decisions?

WG: Do you have difficulty remembering or concentrating?

Self-Care

ACS: Do you have difficulty dressing or bathing?

WG: Do you have difficulty with self-care such as washing all over or dressing?

Comparing the Measures: The Question Stem
Comparing the Measures: The Response Options

WG-SS Response Options

1. No difficulty

2. Some difficulty

3. A lot of difficulty

4. Cannot do at all/unable to do

ACS-6 Response Options

1. Yes

2. No
Research Questions 1-2

 What is the overall prevalence of disability produced by, and the agreement 

between, the WG-SS and ACS-6 (across the five common domains)?

 Overall agreement; positive agreement; negative agreement; kappa; 

prevalence adjusted, bias adjusted kappa (PABAK)

ACS-6 Disability

Any domain = yes

WG-SS Disability

Any domain = a lot of difficulty or

cannot do at all
9.7

20.5

11.0

21.2

8.0

19.6

0

5

10

15

20

25

30

WG-SS ACS-6

Percent with disabilities

Total Female Male

SOURCE: National Health Interview Survey, 2011-2012.

Figure 1. Estimates of disability using the WG-SS and ACS-6 among 

respondents who answered both sets, 2011-2012 NHIS
39.8

23.8

8.1

19.8

11.6

3.2

0 10 20 30 40 50

65+ years

45-64 years

18-44 years

Percent with disabilities

WG-SS ACS-6

SOURCE: National Health Interview Survey, 2011-2012.

Figure 2. Estimates of disability using the WG-SS and ACS-6, by age group, 

among respondents who answered both sets, 2011-2012 NHIS
36.3

8.2

18.8

2.7

0 5 10 15 20 25 30 35 40

Not employed

Employed

Percent with disabilities

WG-SS ACS-6

SOURCE: National Health Interview Survey, 2011-2012.

Figure 3. Estimates of disability using the WG-SS and ACS-6, by employment 

status, among respondents who answered both sets, 2011-2012 NHIS
Table 1. Overall agreement between dichotomous ACS-6 and WG-SS, among 

respondents who answered both sets, 2011-2012 NHIS

ACS-6

WG-SS

Total

Yes No

Total 1,752 16,233 17,985

Yes 1,557

(8.66%)

2,125

(11.82%) 3,682

No 195

(1.08%)

14,108

(78.44%) 14,303

Overall agreement = 0.87

Negative agreement = 0.92

Positive agreement = 0.57

SOURCE: National Health Interview Survey, 2011-2012.

Kappa statistic: 0.50

Prevalence-adjusted bias-adjusted kappa: 0.74
Research Question 3

 Where does nonagreement occur when examining the ACS-6 

responses and WG-SS expanded response options?

• Examine the dichotomous ACS-6 responses across three WG-SS 

responses categories
ACS-6

WG-SS

A lot of 

difficulty or 

unable to do

Some 

difficulty No difficulty Total

Total 1,752 4,850 11,383 17,985

Yes 1,557 1,801 324 3,682

No 195 3,049 11,059 14,303

SOURCE: National Health Interview Survey, 2011-2012.

Table 2. Overall agreement between ACS-6 (dichotomy) and WG-SS 

(trichotomy) among respondents who answered both sets, 

unweighted, 2011-2012 NHIS

Of those with ACS-6 disability, 42.3% classified as with WG-SS disability (1557/3682).

Of those with ACS-6 disability but no WG-SS disability (the 1801+324=2125), 84.8% indicated “some” 

difficulty on WG-SS (1801/2125).
Research Question 4

 What are the factors associated with nonagreement?

• Overall nonagreement: binomial logistic regression

• Agreement (ref)

• Specific nonagreement: multinomial logistic regression

• Agreement (ref): ACS=No / WG=No , ACS=Yes / WG=Yes

• ACS=Yes / WG=No

• ACS=No / WG=Yes

Covariates

Sociodemographics: Age Sex Race, Hispanic origin

Education Employment status Poverty status

Health measures: Cognitive difficulties Reported health status Multiple chronic conditions
Covariate Overall

nonagreement

Specific

nonagreement

ACS=No /

WG=Yes

ACS=Yes /

WG=No

Age (Ref=18-44 years)

45-64 years 1.58* 1.48 1.59*

65 years and older 1.96* 1.81* 1.98*

Sex (Ref=male)

Female 0.90* 1.48* 0.85*

Race and Ethnicity (Ref=White, non-Hispanic)

Hispanic 0.95 1.09 0.94

Black, non-Hispanic 0.91 1.06 0.90

Other, non-Hispanic 0.77* 0.86 0.77*

Table 3. Multivariate logistic regression of nonagreement between ACS-6 and WG-SS 

among respondents who answered both sets, unweighted, 2011-2012 NHIS

SOURCE: National Health Interview Survey, 2011-2012.
Covariate Overall

nonagreement

Specific

nonagreement

ACS=No /

WG=Yes

ACS=Yes /

WG=No

Education (Ref=Bachelors or beyond)

Less than high school 1.44* 2.70* 1.37*

High school/GED 1.21* 1.76* 1.17

Some college/AA degree 1.35* 2.07* 1.30*

Employment status (Ref=Employed)

Not employed 1.63* 1.49* 1.64*

Poverty status (Ref=400% FPL or more)

Less than 100% FPL 1.23* 0.97 1.26*

100 ≤ 199% FPL 1.10 0.89 1.12

200 ≤ 399% FPL 1.07 1.24 1.05

SOURCE: National Health Interview Survey, 2011-2012.

Table 3. Multivariate logistic regression of nonagreement between ACS-6 and WG-SS 

among respondents who answered both sets, unweighted, 2011-2012 NHIS
Covariate Overall 

nonagreement

Specific

nonagreement

ACS=No /

WG=Yes

ACS=Yes /

WG=No

Cognitive difficulty (Ref=No)

Yes 1.82* 0.51 1.96*

Serious psychological distress in past 30 days 

(Ref=No)

Yes 1.11 1.62 1.08

Reported health status (Ref=very good/ 

excellent)

Poor/fair 2.55* 1.78* 2.65*

Good 1.99* 1.79* 2.01*

Multiple chronic conditions (Ref=No)

Yes 1.47* 2.05* 1.43*

SOURCE: National Health Interview Survey, 2011-2012.

Table 3. Multivariate logistic regression of nonagreement between ACS-6 and WG-SS 

among respondents who answered both sets, unweighted, 2011-2012 NHIS
Research Question 5

How does each measure correlate with participation outcomes known 

to be associated with disability?

• Examine strength of association between ACS and WG-SS and:

• Not employed

• Less than high school

• Less than 100% federal poverty level

• Rents home

• Odds of each outcome for those with disability compared to those 

without
Outcome

ACS WG-SS

Yes vs. No

A lot/Unable

vs. Some/No 

Difficulty

Some vs. No 

Difficulty

A lot/Unable

vs. No 

Difficulty

Unemployed (18-64 years)

5.40

(4.88-5.98)

7.43

(6.35-8.68)

1.97

(1.80-2.15)

8.94

(7.62-10.47)

≤ H.S. education (25+ years)

2.48

(2.25-2.72)

2.64

(2.36-2.94)

1.75

(1.58-1.93)

3.22

(2.85-3.63)

≤ 100% FPL

1.73

(1.58-1.89)

1.82

(1.63-2.04)

1.18

(1.08-1.28)

1.91

(1.70-2.15)

Rents home

1.18

(1.09-1.28)

1.46

(1.31-1.62)

0.89

(0.83-0.95)

1.41

(1.26-1.57)

Table 4. Bivariate logistic regressions of experiencing outcomes, by 

disability measure, among respondents who answered both sets, 

2011-2012 NHIS
Outcome

ACS WG-SS

Yes vs. No

A lot/Unable

vs. Some/No 

Difficulty

Some vs. No 

Difficulty

A lot/Unable

vs. No 

Difficulty

Unemployed (18-64 years)

5.40

(4.88-5.98)

7.43

(6.35-8.68)

1.97

(1.80-2.15)

8.94

(7.62-10.47)

≤ H.S. education (25+ years)

2.48

(2.25-2.72)

2.64

(2.36-2.94)

1.75

(1.58-1.93)

3.22

(2.85-3.63))

≤ 100% FPL

1.73

(1.58-1.89)

1.82

(1.63-2.04)

1.18

(1.08-1.28)

1.91

(1.70-2.15)

Rents home

1.18

(1.09-1.28)

1.46

(1.31-1.62)

0.89

(0.83-0.95)

1.41

(1.26-1.57)

Table 4. Bivariate logistic regressions of experiencing outcomes, by 

disability measure, among respondents who answered both sets, 

2011-2012 NHIS
Outcome

ACS WG-SS

Yes vs. No

A lot/Unable

vs. Some/No 

Difficulty

Some vs. No 

Difficulty

A lot/Unable

vs. No 

Difficulty

Unemployed (18-64 years)

5.40

(4.88-5.98)

7.43

(6.35-8.68)

1.97

(1.80-2.15)

8.94

(7.62-10.47)

≤ H.S. education (25+ years)

2.48

(2.25-2.72)

2.64

(2.36-2.94)

1.75

(1.58-1.93)

3.22

(2.85-3.63))

≤ 100% FPL

1.73

(1.58-1.89)

1.82

(1.63-2.04)

1.18

(1.08-1.28)

1.91

(1.70-2.15)

Rents home

1.18

(1.09-1.28)

1.46

(1.31-1.62)

0.89

(0.83-0.95)

1.41

(1.26-1.57)

Table 4. Bivariate logistic regressions of experiencing outcomes, by 

disability measure, among respondents who answered both sets, 

2011-2012 NHIS
Summary/Conclusions

 Overall agreement between the two sets of questions is fairly high, driven largely 

by agreement among those reporting no disability on both sets.

 Overall nonagreement is higher for those with lower SES status (lower levels of 

education, not employed, and in poverty).

 WG-SS includes more people likely at higher risk for social participation 

restrictions: WG-SS group has lower SES compared to ACS-6 group.

 Odds of being classified as with disability by ACS-6, but not the WG-SS, increases 

with age and cognitive difficulty.

 Nearly half (48.9%) of the ACS-6 with disability group answered “some difficulty” 

on WG-SS.

 WG-SS appears to identify a more homogenous group with higher levels of 

functional difficulty.
National Center for Health Statistics

Thank You

Julie D. Weeks

National Center for Health Statistics 

Centers for Disease Control and Prevention

Hyattsville, MD 20782

jweeks@cdc.gov

Citation: Weeks JD, Dahlhamer JM, Madans JH, Maitland A. Measuring disability: An examination of 

differences between the Washington Group Short Set on Functioning and the American Community Survey 

disability questions. National Health Statistics Reports; no 161. Hyattsville, MD: National Center for Health 

Statistics. 2021. DOI: https://dx.doi.org/10.15620/cdc:107202.