# Presentation: 244585

## Disability questions in censuses and surveys Methodological issues:  proxy response, non-response and mode of administration 

**21-23 September 2005**
***Montserrat López-Cobo***

**Fifth Meeting of the Washington Group**

## Are proxy-responses different from self-responses? Why?
Do all proxies provide the same quality responses? Who is the best proxy?
Is there an effect on the quality of data? Can we measure it? Can we correct it?

**Do all proxies provide the same quality responses? Who is the best proxy?**
**Is there an effect on the quality of data? Can we measure it? Can we correct it?**

**Proxy response**

##    It is generally accepted that proxy- and self- responses differ
Many studies found that proxy-respondents underreport disability-related information:
Disability
Impairments of emotion and Pain
Limitation of activities, long-term disability, need for assistance with ADL
Chronic condition, days of activity restriction, bed disability days
General health events and conditions

**Many studies found that proxy-respondents **_**underreport**_** disability-related information:**
**Disability**
**Impairments of emotion and Pain**
**Limitation of activities, long-term disability, need for assistance with ADL**
**Chronic condition, days of activity restriction, bed disability days**
**General health events and conditions**

**Proxy response****Level differences 1**

## ... While some other studies found that proxies overreport:
Cognitive status, Mild Cognitive Impairment
ADL disability
Functional health, when the self-respondent is demented

**Cognitive status, Mild Cognitive Impairment**
**ADL disability**
**Functional health, when the self-respondent is demented**

**Proxy response****Level differences 2**

## Differences in cognitive procedures (1/2)
Type and level of available information
Proxies report impairments or conditions that are more observable or mentioned to them
Proxy reports are more accurate for conditions that are serious, painful, persistent or potentially life threatening
If person under 65: proxies underreport
If person 65 and over: proxies overreport
Proxy information is more stable. Self information is more dynamic
Self-responses are less consistent across time

**Type and level of available information**
**Proxies report impairments or conditions that are more observable or mentioned to them**
**Proxy reports are more accurate for conditions that are serious, painful, persistent or potentially life threatening**
**If person under 65: proxies **_**underreport**_
**If person 65 and over: proxies **_**overreport**_
**Proxy information is more stable. Self information is more dynamic**
**Self-responses are less consistent across time**

**Proxy response****Understanding self-proxy differences 1**

## Differences in cognitive procedures (2/2)
Proxies engage in less extensive cognitive process. Provide heuristic-based responses
Proxies rely more on inferences and estimations.
Proxies overreport disabilities aparently related to a previously reported disability and underreport disabilities seemingly unrelated to the previously reported.
Proxies are less affected by social desirability concerns 
Proxies are more willing to report disability

**Proxies engage in less extensive cognitive process. Provide heuristic-based responses**
**Proxies rely more on inferences and estimations.**
**Proxies overreport disabilities aparently related to a previously reported disability and underreport disabilities seemingly unrelated to the previously reported.**
**Proxies are less affected by social desirability concerns **
**Proxies are more willing to report disability**

**Proxy response****Understanding self-proxy differences 2**

## Social relationship with the selected person
The closest the relationship, the most accurate the proxy-response
Spouse’s responses are better than other proxies’
Proffesionals and caregivers provide more accurate information than lay proxies
Length of relationship 
Positive effect in convergence self-proxy
Memory capacity of respondent
Affect to data quality 

**The closest the relationship, the most accurate the proxy-response**
**Spouse’s responses are better than other proxies’**
**Proffesionals and caregivers provide more accurate information than lay proxies**
**Length of relationship**** **
**Positive effect in convergence self-proxy**
**Memory capacity of respondent**
**Affect to data quality **

**Proxy response****The best proxy-respondent**

## Types of methods:
1) Traditional
First step: Measure the bias by comparing proxy- estimates with self- estimates
Second step: Introduce statistical control adjusting for demographic or health-related variables
2) New strategy
First: Set assumptions about the nature of differences between self- and proxy-responses
Second: Model and estimate these differences based on the assumptions

**1) Traditional**
**First step: Measure the bias by comparing proxy- estimates with self- estimates**
**Second step: Introduce statistical control adjusting for demographic or health-related variables**
**2) New strategy**
**First: Set assumptions about the nature of differences between self- and proxy-responses**
**Second: Model and estimate these differences based on the assumptions**

**Proxy response****Measuring and correcting bias**

## Mode of administration

## Differences in the sample
Population coverage
Response rates

Differences due to social context within which data are collected
Contact with interviewer
Non-verbal cues
Social desirability bias: Reduced with self-administered questionnaires

**Population coverage**
**Response rates**
**Differences due to social context within which data are collected**
**Contact with interviewer**
**Non-verbal cues**
**Social desirability bias: Reduced with self-administered questionnaires**

**Mode of administration****Sources of differences among modes 1**

## Differences intrinsic to the mode
Visual materials cannot be used by phone
Complexity of the questionnaire is limited by the mode 
Stimuli in a visual mode  Primacy effects Stimuli in a hearing mode  Recency effects
Differences in responses involving long lists
Order effects
Effects of the mode on the interviewer

**Visual materials cannot be used by phone**
**Complexity of the questionnaire is limited by the mode **
**Stimuli in a visual mode ******** Primacy effects Stimuli in a hearing mode ******** Recency effects**
**Differences in responses involving long lists**
**Order effects**
**Effects of the mode on the interviewer**

**Mode of administration****Sources of differences among modes 2**

## Telephone interviews
Do not underrepresent people with disabilities
Sometimes is not as well accepted by population as face-to-face interviews
Proved useful to assess mental health using recommended instruments (GHQ-12, CIS-R, CIDIS), the Expanded Disability Status Sacle
Telephone vs Mail
Mail responses report poorer health and more chronic conditions than Telephone
Differential non-response rates by age: 
		Elder: NR (Telephone) > NR (Mail) 
		Young: NR (Telephone) < NR (Mail) 
		General: Item NR (Telephone) < Item NR (Mail)

**Do not underrepresent people with disabilities**
**Sometimes is not as well accepted by population as face-to-face interviews**
**Proved useful to assess mental health using recommended instruments (GHQ-12, CIS-R, CIDIS), the Expanded Disability Status Sacle**
**Telephone *****vs***** Mail**
**Mail responses report poorer health and more chronic conditions than Telephone**
**Differential non-response rates by age: **
**	****	****Elder: NR (Telephone) > NR (Mail) **
**	****	****Young: NR (Telephone) < NR (Mail) **
**	****	****General: Item NR (Telephone) < Item NR (Mail)**

**Mode of administration****Results from comparative studies 1**

## Face-to-Face vs Mail
Health differences between respondents and non-respondents by mode: non-response bias
Non-respondents and late respondents to Mail are more cognitively impaired and more disabled than respondents (among elderly). 
Non-respondents to F-t-F are similar to respondents.

**Health differences between respondents and non-respondents by mode: non-response bias**
**Non-respondents and late respondents to Mail are more cognitively impaired and more disabled than respondents (among elderly). **
**Non-respondents to F-t-F are similar to respondents.**

**Mode of administration****Results from comparative studies 2**

## Telephone/CATI vs Face to Face
Differential non-response rates:   		      	NR (CATI) > NR (F-t-F)
Measurement bias: 					MBias (CATI) < MBias (F-t-F)
No differences between CATI and F-t-F for reports on chronic conditions, activity limitations and disability rates
CATI vs CAPI
Differential non-response rates by age:                              
		Elder: NR (CATI) > NR (CAPI)
		Young: NR (CATI) < NR (CAPI)

**Differential non-response rates:   ****	****	****      ****	****NR (CATI) > NR (F-t-F)**
**Measurement bias: ****	****	****	****	****	****MBias (CATI) < MBias (F-t-F)**
**No differences between CATI and F-t-F for reports on chronic conditions, activity limitations and disability rates**
**CATI *****vs***** CAPI**
**Differential non-response rates by age:                              **
**	****	****Elder: NR (CATI) > NR (CAPI)**
**	****	****Young: NR (CATI) < NR (CAPI)**

**Mode of administration****Results from comparative studies 3**

## CASI
Reduces social desirability bias
Enhances the feeling of privacy
Respondents generally like CASI
Technological possibilities have a positive influence on data quality (minimizing errors)
Respondent’s self-disclosure is higher in CASI

**Reduces social desirability bias**
**Enhances the feeling of privacy**
**Respondents generally like CASI**
**Technological possibilities have a positive influence on data quality (minimizing errors)**
**Respondent’s self-disclosure is higher in CASI**

**Mode of administration****Results from comparative studies 4**

## Disabilities: Physical, sensory, mental illness and mental retardation   
Challenges addressed: communication, fatigue and cognitive issues
Questionnaire design
Eliminate soft consonant sounds (s, z, t, f and g) to overcome high-frequency hearing loss 
Build in "breaks" for respondents to let them rest
Incorporate neutral encouragement to avoid drop-outs
Design checks for unexpected responses
Use structured probes for questions that might be difficult to understand

**Challenges addressed: communication, fatigue and cognitive issues**
**Questionnaire design**
**Eliminate soft consonant sounds (s, z, t, f and g) to overcome high-frequency hearing loss **
**Build in "breaks" for respondents to let them rest**
**Incorporate neutral encouragement to avoid drop-outs**
**Design checks for unexpected responses**
**Use structured probes for questions that might be difficult to understand**

**Mode of administration****An experience interviewing disabled people by telephone**

## Interviewer training and supervision
Usual background and purpose of study
Training on challenges likely to face
Sensitive exercise regarding the treatment to the disabled person
Guidance to overcome each of the challenges
Support to interviewers and reduction of stress
Other recommendations
Interviews take longer  Multiple sessions may be required

**Usual background and purpose of study**
**Training on challenges likely to face**
**Sensitive exercise regarding the treatment to the disabled person**
**Guidance to overcome each of the challenges**
**Support to interviewers and reduction of stress**
**Other recommendations**
**Interviews take longer ******** Multiple sessions may be required**

**Mode of administration****An experience interviewing disabled people by telephone**

## Non-response

## Components of non-response
Non-contact
Non-cooperation
Non-contact: the household pattern of disabled
Are older and likely to live in non-metropolitan areas  higher probability of being contacted
Less likely to live with children  lower prob. of contact
May be more fearful of opening doors to strangers  lower prob. of contact

**Non-contact**
**Non-cooperation**
**Non-contact: the household pattern of disabled**
**Are older and likely to live in non-metropolitan areas ******** higher probability of being contacted**
**Less likely to live with children ******** lower prob. of contact**
**May be more fearful of opening doors to strangers ******** lower prob. of contact**

**Non-response and Disability**

## Non-cooperation: exchange theory vs social isolation theory
Exchange theory: Persons who feel that the survey sponsor has provided (or could provide) benefits to them are more likely to cooperate
Persons with disabilities might be expected to be more cooperative with a government-sponsored survey
Social isolation theory: People who are isolated from the mainstream society feel less responsibility toward government and are less likely to cooperate 
Persons with disabilities might be expected to be less cooperative 

_**Exchange theory**_**: ****Persons who feel that the survey sponsor has provided (or could provide) benefits to them are more likely to cooperate**
**Persons with disabilities might be expected to be more cooperative with a government-sponsored survey**
_**Social isolation theory**_**: P****eople who are isolated from the mainstream society feel less responsibility toward government and are less likely to cooperate**** **
**Persons with disabilities ****might be expected to be less cooperative**** **

**Non-response and Disability**

## Persons with severe disability are more likely than persons with less severe disability to be contacted and to cooperate, but when they are interviewed they are less likely to answer for themselves; instead, proxy and assistant respondents tend to answer for them. 
Elder people with disabilities respond sooner than young and non-disabled to mail surveys.
These findings support Exchange theory
Self-perceived memory problems increase item non-response and “I don’t know” answers.
	Not preceived memory deficits implies inaccurate information on items requiring recall.

**Elder people with disabilities respond sooner than young and non-disabled to mail surveys.**
**These findings support Exchange theory**
**Self-perceived memory problems increase item non-response and “I don’t know” answers.**
**	****Not preceived memory deficits implies inaccurate information on items requiring recall.**

**Non-response and Disability****Results from studies 1**

## Health of non-respondents is worse than respondents’ in terms of: stroke, Basic ADL, mobility disabilities, self-rated health and mortality rates.
Respondents with poorer physical functioning and/or limiting long-term illness have higher non-response rates.
These findings support Social isolation theory

**Respondents with poorer physical functioning and/or limiting long-term illness have higher non-response rates.**
**These findings support Social isolation theory**

**Non-response and Disability****Results from studies 2**

## Recommendations for including disabled people in interview surveys

## 	Bias due to exclusion is amplified in surveys where disability is a key measure of interest

Recommendations:
Include institutionalised population in samples
Carry out the interview in a private environment
Provide adaptative technologies and procedures (of primary importance in visual, hearing and speech impairment).
Use simple and clear questions

**Recommendations:**
**Include institutionalised population in samples**
**Carry out the interview in a private environment**
**Provide adaptative technologies and procedures (of primary importance in visual, hearing and speech impairment).**
**Use simple and clear questions**

## Recommendations: 
Special training for interviewers
Provide alternative modes of administration
If proxy is used:
Respondent’s assessments of proxy’s answer is valuable
The proxy should be nominated by the selected respondent

**Special training for interviewers**
**Provide alternative modes of administration**
**If proxy is used:**
**Respondent’s assessments of proxy’s answer is valuable**
**The proxy should be nominated by the selected respondent**

## Proxy response validity depends on factors such as: proxy choice and distinction, topic investigated, health condition of the selected person...

Mode of administration (unique or a mix-mode) should be decided taking into account: topic investigated, population objective of the survey, expected acceptability of the mode by the population, technological possibilities...

**Mode of administration (unique or a mix-mode) should be decided taking into account: topic investigated, population objective of the survey, expected acceptability of the mode by the population, technological possibilities...**

**Final conclusion**

## Different patterns of response might be indicative of bias which can affect estimates. 

Non-response is one of the sources of data error. But not the unique... Emphasis should be given not only to minimazing non-response rates but also to estimation and control of measurement error.

**Non-response is one of the sources of data error. But not the unique... Emphasis should be given not only to minimazing non-response rates but also to estimation and control of measurement error.**

**Final conclusion**

## 21-23 September 2005

**Fifth Meeting of the Washington Group**