Center for Studying Disability Policy

Exploring Proxy Response Bias in a Large-Scale 

Survey of People with Disabilities

Eric Grau and Jason Markesich

November 4, 2021
Overview

• Health surveys often allow proxies to respond for people 

with disabilities who cannot respond for themselves

• Proxy response bias occurs when the responses provided 

by proxies are systematically different than those given by 

self-respondents

• Researchers should be concerned about the impact of 

proxy response bias on the validity of estimates of health 

and functional status

2
What do we know about proxy reporting?

• Proxy response more likely among older people, those with 

limited education, and with poorer physical and mental 

health

• Compared to self-response, proxy ratings of health status 

and functioning have been shown to be biased in different 

directions in different studies

• The relationship between the proxy and the person they 

answer for could affect the accuracy of proxy responses

3
Limitations to the existing research

• Few studies examined proxy response bias in surveys of 

people with disabilities

• Most were conducted with small convenience samples of 

people with specific health care needs

• Those sample members who actually needed a proxy 

because they could not respond for themselves are often 

not represented

4
National Beneficiary Survey (NBS)

• Sponsored by the Social Security Administration

• Sample of working age people who receive Supplemental 

Security Income (SSI) and/or Social Security Disability 

Insurance (SSDI)

• Seven rounds of data collection (we will be looking at two)

- Seven independent cross-sectional samples of beneficiaries

- Round 5—4000 completes, 2015, population as of 6/30/14

- Round 6—4000 completes, 2017, population as of 6/30/16

5
Proxies in the NBS

• Interviews in the NBS were completed by proxy for the 

following reasons: 

- The sample member did not pass a standardized cognitive screener

- A knowledgeable informant expressed that a proxy would be necessary

- The sample member needed to switch to a proxy during the course of 

the interview

• 26.5 percent of interviews in Rounds 5 and 6 of the NBS 

were completed by proxy

- 26 percent in Round 5

- 27 percent in Round 6

6
Key research questions

1. Is the use of proxy respondents in the NBS related to the 

demographic characteristics of the sample members?

2. What is the size and direction of the differences between 

proxy and self-reported responses to questions on health 

status and functional limitations?

3. To what extent does the proxy-sample member 

relationship affect the differences between proxy and 

self-reported responses?

7
Outline

• Address research question 1

- Round 5 and Round 6 separately

• Address research question 2

- Round 5 and Round 6 separately

- Cognitive and noncognitive disabilities separately

• Address research question 3

- Round 5 and Round 6 separately

- Cognitive and noncognitive disabilities separately

- Parent and non-parent proxies separately

8
Research Question #1:

Is the use of proxy respondents in the NBS 

related to demographic characteristics of the 

sample members?

9
Analysis plan

• Fit logistic models predicting the likelihood of proxy use 

accounting for the sample design 

• Identify characteristics for which proxy and self-reporting 

sample members differ 

10
List of potential confounders

11

Variable

Gender of sample member Disability type (cognitive, mental illness, physical)

Race (black, white, other) Age of disability (<18, 18-29, 30+)

Age (18-29, 30-39, 40-49, 50+) Has children (yes, no)

Employment status (working or not) Beneficiary status (SSI only, SSDI only, concurrent)

Education level (HS dropout, HS/GED, 

certificate/spec ed, some post-secondary

Living arrangement (live alone, with family, with 

friends/roommates, in group home)

Marital status (ever married, never married) Cohabitation status (living with someone or not)

Ethnicity BMI category
Finding

• Biggest difference is with disability type

- Sample members with cognitive disabilities are much more likely to 

require proxies than those with physical disabilities or mental illness

- The cognitive and noncognitive disability groups are very different, so it 

may be difficult to interpret results if the populations are combined

- Proceed by analyzing cognitive and noncognitive populations 

separately

12
Comparing characteristics of proxy users 

by disability type

• For sample members with either cognitive disabilities or 

noncognitive disabilities, proxy use was related to several 

demographic characteristics:

- Gender, education, living situation, marital status, number of children, 

and age of disability onset

• For those with only noncognitive disabilities, proxy use 

was also related to:

- Disability type, benefit type, age

13
Research Question #2:

What is the size and direction of the differences 

between proxy and self-reported responses to 

questions on health status and functional 

limitations?

14
NBS measures of health status and 

functioning

• Measures of physical and mental health status

- Eight self reported measures on a Likert scale, converted to binary for 

analysis

• Measures of activities of daily living (ADL), instrumental 

activities of daily living (IADL), and functional limitations 

assess difficulty with:

- Eight self-reported measures on a binary scale

15
Matching treatment and comparison 

groups to minimize bias

• Sample members with and without proxy responses differ 

on a lot of characteristics

• Estimating the proxy “effect” is potentially biased if not 

adjusted appropriately for these characteristics

• Propensity scoring aligns the distributions of those with 

and without proxies, reducing the propensity for bias due 

to observed confounders

16
Analysis plan 

• Estimate propensity score weights separately for each of 

the two disability group types, and each of the two rounds

• Assess the effect of the “treatment” on the selected 

outcomes using propensity score weighted logistic 

regression

• Analyze two rounds separately, assess commonalities and 

differences between rounds

17
Findings

• We obtained good covariate balance between the proxy 

respondents and self respondents with propensity score 

weights for both disability groups and both rounds

• Results between rounds were consistent for noncognitive 

disabilities, not so for cognitive disabilities

- The strongest result was for the outcome “difficulties doing errands 

alone” in both rounds

- Cognitive: proxies indicated more health problems in Round 6 only

- Noncognitive: proxies indicated less bodily pain, more problems lifting 

heavy objects or getting around home

18
Proxy vs. self-report odds ratios for those with 

cognitive disabilities

[>1: proxies indicate better health/fewer difficulties]

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

General health Limit activitiesphysicalDifficulty daily work Emotional problems Limit activitiesemotionalDifficulty stairs Difficulty lifting

heavy

Difficulty reaching

overhead

Difficulty stooping Difficulty errands

alone

round 5 round 6

*

*

*

* Statistically significant

*

*

*

*

*
Proxy vs. self-report odds ratios for those with 

noncognitive disabilities

[>1: proxies indicate better health/fewer difficulties]

20

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

General health Limit activities

physical

Limit daily work Bodily pain Limit energy Difficulty stairs Difficulty lifting

heavy

Difficulty

stooping

Difficulty

getting around

inside

Difficulty

errands alone

round 5 round 6

*

*

* Statistically significant

*

*

*

*

*
Research Question #3:

To what extent does the proxy-sample member 

relationship affect the differences between 

proxy and self-reported responses?

21
Analysis plan 

• Review proxy relationships to sample members and their 

response patterns on outcome variables

• Use similar regression model setup as for Q2 

• Assess the effect of the “treatments” on the selected 

outcomes

• Analyze two rounds separately, assess commonalities and 

differences between rounds

22
Proxy relationship

Number in 

NBS

Cognitive 

disabilities

Noncognitive 

disabilities

Total proxies 1,043 (26%) 477 (58%) 561 (18%)

Parent (mother or father) 635 283 350

Other proxies 410 194 211

Self-respondents 3,019 (74%) 340 (42%) 2,630 (82%)

Total 4,062 817 3,191

Proxy relationships in the NBS data, 

round 5

23
Proxy relationship

Number in 

NBS

Cognitive 

disabilities

Noncognitive 

disabilities

Total proxies 1,091 (27%) 525 (64%) 552 (18%)

Parent (mother or father) 661 325 331

Other proxies 465 207 221

Self-respondents 2,911 (72%) 295 (36%) 2,575 (82%)

Total 4,002 820 3,127

Proxy relationships in the NBS data, 

round 6

24
Findings

• Cognitive: In Round 6, non-parent proxies indicated more 

health problems and functional limitations, not in Round 5 

• Noncognitive: parent proxies indicated fewer health 

problems and functional limitations than comparable selfrespondents regardless of round—more than just bodily 

pain that we saw in result from RQ2

• Regardless of relationship and disability type, all proxies 

reported sample members had more difficulty doing 

errands alone than comparable self-respondents

25
Proxy vs. self-report odds ratios among those 

with cognitive disabilities, Round 5

[>1: proxies indicate better health/fewer difficulties]

26

0

0.2

0.4

0.6

0.8

1

1.2

1.4

General

health

Limit

activities

physical

Difficulty

work

Bothered

emotions

Limit

activities

emotional

Difficulty

stairs

Difficulty

lifting

Difficulty

reaching

Difficulty

stooping

Difficulty

errands alone

parents non-parents

*

*

* Statistically significant

*
Proxy vs. self-report odds ratios among those 

with cognitive disabilities, Round 6

[>1: proxies indicate better health/fewer difficulties]

27

0

0.2

0.4

0.6

0.8

1

1.2

General

health

Limit

activities

physical

Difficulty

work

Bothered

emotions

Limit

activities

emotional

Difficulty

stairs

Difficulty

lifting

Difficulty

reaching

Difficulty

stooping

Difficulty

errands alone

parents non-parents

*

*

*

*

*

* *

* Statistically significant

*

* *
Proxy vs. self-report odds ratios among those 

with noncognitive disabilities, Round 5

[>1: proxies indicate better health/fewer difficulties]

28

0

0.5

1

1.5

2

2.5

3

3.5

4

General health Limit activities

physical

Limit daily

work

Bodily pain Limit energy Difficulty stairs Difficulty

lifting

Difficulty

stooping

Difficulty

getting around

Difficulty

errands

parents non-parents

*

*

*

*

*

*

*

*Statistically significant

*

* *
Proxy vs. self-report odds ratios among those 

with noncognitive disabilities, Round 6

[>1: proxies indicate better health/fewer difficulties]

29

0

0.5

1

1.5

2

2.5

3

General health Limit activities

physical

Limit daily

work

Bodily pain Limit energy Difficulty stairs Difficulty

lifting

Difficulty

stooping

Difficulty

getting around

Difficulty

errands

parents non-parents

*

*

*

*

*

*

*Statistically significant

* *
Closing Remarks

30
Recap

• Proxy responses were consistently more likely for sample 

members who were men, living in group homes, unmarried, 

younger (particularly those under 30 years old), and either 

had no school, or had a special education certificate of 

completion

• From the analyses of RQ2 and RQ3, across the board, 

proxies reported more problems with doing errands alone

• Other results differed between cognitive and noncognitive 

disability types

31
Recap, cont.

• Cognitive: 

- From RQ2 analysis results, proxies reported were more likely 

to report functional limitations/poorer health than selfrespondents for sample members, but for one round only. 

- From RQ3 we saw that this was only true for non-parent 

proxies, again for one round only.

32
Recap, cont.

• Noncognitive: 

- For both RQ2 and RQ3, results were mostly consistent across 

rounds. 

- From RQ2, proxies were more likely to report less bodily 

pain than self respondents, and less likely to report difficulty 

lifting heavy things

- From RQ3, parent proxies were more likely to report fewer 

limitations/better health than self-respondents for a number 

of outcomes, particularly health outcomes, not just bodily 

pain, but it was non-parent proxies were more likely to report 

difficulty lifting heavy things

33
Limitations of research

• This is not a randomized control trial

- Propensity Score Weights are a vehicle that can be used to ensure proxy 

respondents and self-respondents are as alike as possible for the given 

confounders

- The differences that we observed may not be due to bias, but due to 

differences between the groups that are not captured by the 

confounders we have available

• We considered all proxy types together in the model—we 

did not break it down by the reason for proxy use

• The extent of self-report bias is unknown

34
Implications for future research

• Findings that show proxies report differing health and 

functional limitations than self respondents in populations 

of people with disabilities should be acknowledged but 

viewed with caution, as causal factors are not clear

• Consider implementing statistical correction procedures 

that reduce the impact of proxy response bias for 

estimates of health and functional status of people with 

disabilities

- Such procedures should account for known information about the 

relationship between the proxy and the sample member

35
Acknowledgements

• Paul O’Leary, Debra Tidwell-Peters, and Mark Trapani at 

SSA

• Ryan Callahan, Barbara Carlson, Jared Coopersmith, 

Daniela Golinelli, Gina Livermore, Kim McDonald, and 

Bevin Mory at Mathematica

• The NBS team of telephone interviewers, field locators and 

data collection supervisors

36
For more information

37

Eric Grau: EGrau@mathematica-mpr.com

Jason Markesich: JMarkesich@mathematica-mpr.com
Appendices

38
Variable

Cognitive odds 

ratio

Noncognitive 

odds ratio

Health Status Variables: In the past 4 weeks...

Physical health limited usual activities 0.75 (ns) 1.10 (ns)

Had difficulty doing daily work due to physical 

health

0.71 0.92 (ns)

Experienced at least moderate bodily pain 0.87 (ns) 1.30

ADL/IADL: Having difficulty with...

Lifting or carrying something heavy 0.65 0.82

Reaching over your head 0.56 1.07 (ns)

Getting around inside your home 0.97 (ns) 0.67

Doing errands alone 0.43 0.50

Odds Ratios (P-values < 0.05)

[>1: proxies indicate better health/fewer difficulties; ns = not significant]

39
Variable Parents odds ratio Non-parents odds ratio

Health Status Variables: In the past 4 weeks...

General health was fair or worse 1.06 (ns) 0.70 (ns)

Physical health limited usual activities 0.92 (ns) 0.58

Had difficulty doing daily work due to physical health 0.86 (ns) 0.58

Experienced at least moderate bodily pain 0.99 (ns) 0.73 (ns)

ADL/IADL: Having difficulty with...

Walking/climbing stairs 0.99 (ns) 0.66 (ns)

Lifting or carrying something heavy 0.73 (ns) 0.60

Reaching over your head 0.78 (ns) 0.43

Stooping, crouching, or kneeling 0.87 (ns) 0.69 (ns)

Doing errands alone 0.43 0.41

Odds Ratios for Those with Cognitive 

Disabilities (P-values<0.05)

[>1: proxies indicate better health/fewer difficulties; ns = not significant]

40
Odds Ratios for Those with Noncognitive 

Disabilities (P-values<0.05)

[>1: proxies indicate better health/fewer difficulties; ns = not significant]

Variable Parent odds ratios Non-parent odds ratios

Health Status Variables: In the past 4 weeks...

General health was fair or worse 1.40 0.67

Physical health limited usual activities 1.37 0.97 (ns)

Experienced at least moderate bodily pain 1.81 1.09 (ns)

Had little or no energy 1.34 0.77 (ns)

ADL/IADL: Having difficulty with...

Walking/climbing stairs (i29) 1.37 0.83 (ns)

Lifting or carrying something heavy (i35) 1.15 (ns) 0.71

Reaching over your head (i39) 1.39 0.97 (ns)

Stooping, crouching, or kneeling (i43) 1.28 0.90 (ns)

Getting around inside your home (i45) 0.79 (ns) 0.63

Doing errands alone (i47) 0.34 0.62

41
Propensity score weighting

• The nonparametric form of logistic regression is called 

“boosted logistic regression”

• The method puts a penalty on large coefficients

• We use a freeware software package developed at RAND 

called

- The Toolkit for Weighting and Analysis of Nonequivalent Groups 

(TWANG)

• This package includes the capacity for incorporating 

sampling weights into the final weights

42
Propensity score weighting

• There are multiple ways of measuring “distance” between 

treatment and comparison groups in the algorithm

• The algorithm minimizes the distance using these 

“stopping methods” subject to constraints

• We consider two:

- Minimize the maximum Kolmogorov-Smirnov statistic across 

confounders (KS-max)

- Minimize the absolute standardized mean effect size across 

confounders (ES-mean)

43