Disability questions in censuses and surveys: a literature review of
methodological issues - proxy response, non-response and mode of
administration

Within the context of internationally comparable data, there are many
challenges that need to be faced: validity, reliability, question
wording, cultural differences regarding health topics, translation
procedures, data management, mode of questionnaire administration and
data collection, self vs. proxy response, etc.

This paper is a first step to cope with three of these issues -proxy
response, mode of administration/data collection and non-response- by
means of a collection and summary of the vast literature concerning
them. As these matters are strongly related, many of the summarized
reports deal with two or even the three of them. However, an order has
been kept taking into account the main topic of the article.

SELF / PROXY RESPONSE

**Introduction**

Surveys that collect data on a specific person in a household (or on
every household member) need to arise certain methodological issues
regarding the response provider. Survey designers must make a decision:
how much effort to expend gathering data on individuals from the
individual himself or herself.

The general issue is to know if there are systematic differences in
quality between the responses obtained directly from the selected person
versus those obtained from some other respondent, particularly when the
selected person is disabled.

In particular it is desirable to find an answer to the following
questions:

Do survey data suffer in quality when eligible sampled persons do not
respond for themselves?

In what circumstances are proxy informants acceptable?

Who is allowed to be a proxy informant?

What is the relationship between proxy interviews and mode of
administration?

If there is a bias, is it measurable?

Even if we can measure it, what do we do about it?

A literature review has been undertaken in order to collect outcomes
from research on the field. The consulted papers are focused either on
the disability topic or on a more general health approach.

One of the most important characteristics that a self/proxy study must
accomplish is the control over the response status. Most surveys that
allow proxy response do not design the assignment of the response
status. On the contrary, the usual procedure is that persons at home at
the moment of the interview respond by themselves, and those not at
home, those who refuse to participate and those unable to answer provide
data via a proxy informant. In these studies based on
naturally-occurring response it may happen that differences due to
self/proxy response mix with self-selection bias. Thus, some results
from such studies might reflect the differences caused by the greater
probability to find less healthy people at home, which would lead to
obtain poorer health reports from self-respondents (Moore, 1988).

With respect to the quality assessment, the best way to know how well an
estimator reproduces the actual value of a characteristic in the
population is to know the true value of the parameter. Such a value
would be compared with the self and proxy responses estimators. Since
this is not feasible, the way most studies deal with quality assessment
is by simply comparing the aggregate responses of the two respondents
groups. Some studies assume that self-response is always better, while
others assume that for some health variables the *highest* estimator
gives the best approximation to the reality, and for another
characteristics the *lowest* estimator would be the best, no matter
whether it corresponds to the self or proxy respondents groups. New
approaches are oriented to estimate the bias components and correct the
healt-related estimates.

# We should not forget that in the terms used by WHO's International Classification of Functioning and Disability, self-response in a survey is an example of "participation," and substituting proxy or assisted response is a "participation restriction." Participation restrictions depend not only on medical conditions or impairments, but also on "external factors" that create "barriers" to full participation. Non-self-response, therefore, may be due to respondents' conditions, or barriers created by survey procedures, or both.

**Selection of literature**

1.  **Moore, J.C.: *Self/proxy response status and survey response
    quality*. Journal of Official Statistics. Vol.4, No.2, 1988, pp.
    155-172. Statistics Sweden \[1\].**

> Moore carries out a review of literature where several examples of
> self/proxy studies showing different results are provided:

[Report level differences]{.underline}

i.  Cartwright (1957): Comparison of husband's self-response with wife's
    proxy response. The number of reported illnesses was three times
    greater among self-respondents.

ii. Enterline and Capt (1959) found no significant differences for any
    of the ten specific chronic conditions examined.

iii. Nisselson and Woolsey (1959): In an experimental pre-test of the
     NHIS (USA) found that the overall illness rate was significantly
     greater for the self-response treatment, although the results are
     quite inconsistent across sex, age and illness categories. Reported
     ***days of disability*** were consistently lower in the
     self-response group. The authors' main conclusion, however, is that
     no conclusion can be made regarding possible biases in the use of a
     household respondent.

iv. Kovar and Wright (1973): Comparison of a maximum self-response
    treatment and a less strict "accept-proxy" treatment. Only the
    reports of ***limitation of activity*** and *recent doctor visits*
    show a significant treatment effect, with higher report levels for
    the self-respondents group.

v.  Mathiowetz and Groves (1983) elaborated an experiment to compare a
    randomly designated household respondent procedure with a
    self-selected "knowledgeable" household respondent procedure. Their
    findings show that most of the differences are in the direction of
    higher reported levels for the knowledgeable respondent than for the
    self-responses from the randomly designated respondents. Within the
    random respondent treatment there was also a consistent trend for
    higher reporting levels among proxies.

vi. Mooney (1962): Carried out a study through a reinterview design, in
    which the original proxy respondents were administered a
    self-response reinterview two weeks after the original interview.
    For all the health indicators examined (chronic conditions, acute
    conditions, restricted activity days, bed disability days), the
    self-response reinterview yielded a substantial increase in medical
    events and conditions over the original proxy reports. However, the
    reinterview also produced increased reporting among a sample of
    original self-respondents, although the percentage increase from
    original interview to reinterview was greater for the original proxy
    group.

vii. Koons (1973): Study several years' reinterviews in the NHIS (USA)
     also show increased reporting of health events and conditions in a
     self-response reinterview as compared to an original proxy
     interview, and the increases are typically larger than those
     observed in the group of respondents for self in the original
     interview.

> Moore concludes that the weight of evidence does not indicate
> consistent differences in the reporting levels of self and proxy
> respondents.

[Response quality differences]{.underline}

> Moore gives detail of two health-related studies in which controlled
> self/proxy response status was combined with and independent,
> comprehensive assessment of data quality, and states these studies do
> not reveal strong or consistent evidence to support the notion that
> proxy data are in general more biased than self-response data.

[Non-response effects]{.underline}

> Moore's review compares item non-response, person non-response and
> household non-response in several studies, to conclude that although
> the effects are neither large nor consistent, research evidence
> suggests that self/proxy status may have some reliable effects on
> response completeness. Self-response procedures produce higher
> household and person non-interview rates. However, differences seemed
> to be balanced by lower item non-response under procedures which
> maximize self-response.
>
> Moore's final conclusion is that there exists little support of the
> idea that self-response reports are of generally better quality that
> proxy reports. The most obvious conclusion is that there is really not
> enough evidence to draw solid conclusions, but there is some
> confidence that the responses of proxies are comparable in quality to
> what would have been obtained via self-response with additional
> effort. However, the lack of convincing evidence of quality
> differences is not synonymous with convincing evidence of no quality
> differences.

2.  **Lee S., Mathiowetz NA. and Tourangeau R.: *Perceptions of
    disability: the effect of self- and proxy response*. Journal of
    Official Statistics, Vol. 20, No. 4, 2004. Statistics Sweden
    \[2\].**

> The study compares estimates of persons with disabilities based on
> self versus proxy reports. It also examines the consistency of reports
> across two waves of data collection.
>
> The interviews were conducted in 2001 via CATI. The target population
> were households with two or more members aged 40 years old or older.
> Two waves of data collection were fielded two weeks apart. In both
> waves the respondent reported for him or herself as well as one other
> adult in the same household.
>
> [Cognitive processes and self- versus proxy respondents]{.underline}
>
> According to the literature, proxy respondents have been found to draw
> on different information than self-respondents in carrying out the
> cognitive tasks required in surveys, partly because proxy respondents,
> in general, do not have as much information about the target as
> self-respondents do. In contrast to self-respondents, who are aware of
> their private symptoms and conditions and are able to take account of
> these when reporting disability, proxy respondents may have
> information concerning only those impairments that are observable or
> that target has mentioned to the proxy.
>
> However, previous studies have failed to find consistent differences
> in favour of self-responses. This may be due to the shortcomings in
> past research, as (1) non-random selection of respondent status, (2)
> no distinction of different types of proxies and (3) variation in the
> survey topics on which self- and proxy reports were compared. This
> study compares estimates of persons with disabilities by respondent
> type where response status was randomised.
>
> Responses to the survey questions concerning whether a person has a
> disability may vary as a function of respondent type (self versus
> proxy) and as a function of the proxy's relation to the target person.
> The differences between the cognitive processes involved in answering
> questions about oneself and about someone else may explain why self-
> and proxy reports differ. The answering process has five components:
> encoding, comprehension, retrieval, judgement and communication.
> Survey respondents appear to take a systematically different route
> when they are answering for other people.
>
> The differences in the cognitive processes of self- and proxy
> respondents in disability surveys can be summarized as follows. On the
> one hand, we would expect information obtained via proxy respondents
> to be of lower quality, since proxies in general will have encoded
> less information than self-respondents. In addition, the information
> that proxies do have is likely to consist of stable characteristics as
> compared to self-respondents' more specific episodic information.
> Proxy respondents may engage in less extensive cognitive process than
> self-respondents and provide heuristic-based responses. On the other
> hand, if the classification of an individual as a person with a
> disability is sensitive, proxy respondents may be more willing to
> provide accurate information than self-respondents, because they are
> less likely to be affected by social desirability concerns.
>
> The findings of this study indicate that self-response tended to
> produce higher rates of persons with disabilities than proxy reports,
> although only in the second wave of data collection.
>
> [Reporting consistency and self- versus proxy respondents]{.underline}
>
> If proxy reports are more likely to be based on static
> characteristics, it might be anticipated that the consistency of
> reporting for self- and proxy respondents differs, with more
> consistent reports from proxies across time.
>
> The study states that self-respondents provided less consistent
> responses across the two interviews than proxies did. This finding
> supports theoretical and empirical literature that suggests that
> self-respondents rely on more dynamic information than proxies in
> responding to the survey questions.
>
> [Characteristics of proxy respondents]{.underline}
>
> The literature suggests two respondents characteristics that can
> affect the quality of proxy responses: social relationship and memory
> capability.
>
> Prior studies have found that spouse proxies agree with self-responses
> at a higher rate than other proxy respondents. Similarly, it has been
> noticed that the length of a relationship appears to have a positive
> effect in the convergence of self- and proxy reports. The memory
> capacity of the respondent has also been found to have an effect on
> quality of health-related reports. Elderly respondents are less able
> to retrieve accurate information from memory and are less likely to
> stay on the topic than younger respondents.
>
> Understanding the factors which lead to differences in reports by
> self- and proxy respondents is particularly important in the
> measurement of disability, since the use of proxy respondents may be
> confounded with the phenomenon on interest (e.g., impairment that
> limits participation in surveys).
>
> This study found that spouse proxies were significantly more likely to
> provide consistent answers than other proxies. Thus, non-spouse
> proxies may not have sufficient information on which to rely when
> making a judgement about the target's disability. This finding
> suggests that classifying all proxy reporters as one group may mask
> the effects of the respondents, since there seem to be significant
> effects of the proxy-target relationship and the characteristics of
> proxy respondents on the quality of the data. The duration of
> relationship was found to have no effect on the consistency of proxy
> reports.

3.  **Shields M.: *Proxy reporting of health information*. Statistics
    Canada, Catalogue 82-003. Health Reports, Vol. 15, No. 3. May 2004
    \[3\].**

> This article documents the extent of proxy reporting in Statistics
> Canada\'s National Population Health Survey (NPHS) and explores
> associations between reporting status and the prevalence and incidence
> of selected health problems. Data are from the household
> cross-sectional (1994/95, 1996/97 and 1998/99) and longitudinal
> (1994/95 to 2000/01) components of the NPHS. Supplemental data are
> from the 2000/01 Canadian Community Health Survey.
>
> Originally, the NPHS had two questionnaires: General and Health. The
> General questionnaire was used to collect socio-demographic
> information and basic health information (chronic conditions,
> long-term disability, two-week disability, and health care
> utilization) for every member of each household. Proxy by convenience
> was accepted. As well, one member of each household was chosen at
> random to respond to the Health questionnaire, which covered topics
> such as smoking, physical activity, medication use, social support and
> mental health. Proxy responses were accepted only by necessity.
>
> In cycle 3 information was collected directly from longitudinal panel
> members for both the General and Health questionnaires. In cycle 4 the
> both questionnaires were combined and proxy responses were accepted
> only by necessity.
>
> [Proxy reporting rates]{.underline}
>
> In the General file, the proxy reporting rates for the General
> questionnaire were fairly stable across survey cycles at approximately
> 35%. By contrast, in the Health file, the proxy rate for the
> information derived from the General questionnaire file dropped
> substantially from 31% in 1994/95 to 15% in 1998/99. Proxy reporting
> rates for the Health questionnaire were under 5% for all three cycles.
>
> [Effect on prevalence of health problems]{.underline}
>
> The results agree with previous studies that show that people
> responding on their own behalf are more likely to report health
> problems. In fact, in the 1994/95 cycle, when the proxy reporting
> rates were similar, the Health and General file estimates of chronic
> conditions, long-term disability and two-week disability are very
> close. On the contrary, estimates for 1998/99 based on the Health
> file, with its low proxy rate, are higher for the prevalence of
> non-food allergies, asthma, arthritis, back problems, migraine,
> stomach or intestinal ulcers, urinary incontinence, thyroid disorder,
> activity restriction, long-term disability, and cutting down on normal
> activities in the past two weeks because of illness or injury.
>
> Previous studies have found that proxy reports are most accurate for
> conditions that are serious, painful, persistent or potentially life
> threatening. Consistent with such findings, the 1998/99 General and
> Health file estimates do not differ significantly for diabetes,
> epilepsy, heart disease, need for assistance with activities of daily
> living, and having spent at least a day in bed during the previous two
> weeks because of illness or injury.
>
> [Changes in prevalence]{.underline}
>
> Apparent changes in prevalence might reflect variations in reporting
> status. If a proxy effect does exist, this reduction in the proportion
> of proxy responses across cycles might result in sharper increases in
> the prevalence of some conditions in the Health file, particularly
> conditions that are less serious and/or noticeable. And indeed, data
> from the Health file show larger increases in non-food allergies,
> asthma, arthritis, migraine, urinary incontinence, and cut-down day(s)
> in the previous two weeks. The prevalence of activity restriction and
> long-term disability decreased on both files, but the decrease was
> smaller for the Health file, again suggesting a proxy effect.
>
> Some studies have suggested that the proxy effect for seniors may be
> in the opposite direction---that is, proxy reporters for elderly
> people are more likely to report impairment than seniors would
> themselves. However, these studies often concerned institutional
> residents whose proxy reporters are typically younger people who,
> viewing chronic conditions from their own perspective, may be more
> likely to report a senior as being impaired. Estimates of the
> prevalence of health conditions based on NPHS data do not support this
> finding. When there were significant differences between the 1998/99
> General and Health files, the Health file estimates were higher. The
> reason may be that the NPHS sample pertains to the household
> population and that proxy reporters for elderly NPHS respondents
> tended to be of a similar age: close to three-quarters were that
> person's spouse.
>
> [Effect on incidence of health problems]{.underline}
>
> The proxy-reporting rate for the General questionnaire portion of the
> longitudinal file dropped from 33% in 1994/95 to 14% in 1998/99. In
> 2000/01, when the NPHS became strictly longitudinal and proxy
> responses were accepted only by necessity, the rate declined to 4%.
>
> The longitudinal file is frequently used to produce two-year incidence
> rates of health conditions. However, changes in reporting status from
> one cycle to the next could affect these estimates. In most of the
> cases, the reporting status remained the same. However, in about 5% of
> cases, reporting status changed from self to proxy, and in 15%, from
> proxy to self.
>
> If self-respondents have a greater tendency to report health problems,
> people whose status changed from proxy in one cycle to self in the
> next (proxy-self) might be more likely to report a new condition than
> those who reported on their own behalf in both cycles (self-self). To
> investigate this possibility, a multiple logistic regression model was
> run for each chronic condition for which the cross-sectional analyses
> suggested a proxy effect (non-food allergies, asthma, arthritis, back
> problems, migraine, ulcers, and incontinence), as well as for an
> activity restriction or long-term disability. Control variables such
> as sex, living arrangements, education, income and employment, as well
> as the health status through the Health Utilities Index, were
> introduced in the analysis.
>
> When the combined incidence of at least one of the seven conditions is
> considered, all reporting categories are significant and suggest a
> proxy effect, with the proxy-self group more likely to report a new
> condition, and the proxy-proxy and self-proxy groups less likely to do
> so.
>
> The regression models were rerun on the longitudinal file, this time
> excluding proxy by necessity. The results were similar to those for
> the full models, suggesting that accepting proxy responses by
> convenience has an effect on incidence measures of some health
> conditions.

4.  **Parsons JA., Baum S. and Johnson TP.: *Inclusion of disabled
    populations in social surveys: review and recommendations.* Survey
    Research Laboratory, December 2000 \[4\].**

> The authors carry out a comprehensive study of the issues and
> challenges of including disabled persons in interview surveys.
>
> Through a review of the literature and over fifty in-depth interviews
> with key disability advocates and survey methodologists, this paper
> reviews the issues (including sampling, proxy interviewing, and the
> use of adaptive technologies) and present a set of recommendations for
> including disabled persons in interview surveys.
>
> Many disabled persons are commonly excluded or discouraged from
> participating in general population surveys as a consequence of common
> survey design features that may produce unintended barriers. There are
> several expectations of the respondent across all modes of data
> collection. In face-to-face and telephone surveys, the respondent
> needs to be comfortable with the survey topic; be able to hear,
> understand, and respond to the questions being asked; and be
> physically able to focus for the duration of the interview. In mail or
> self-administered surveys there are the obvious requirements of
> literacy and having sufficient visual ability to read questions and
> directions. Because of these expectations, people with various
> disabilities could potentially be excluded from social surveys if
> appropriate accommodations are not provided. While the associated bias
> of the exclusion of this population is usually negligible for
> statistics on the general population, the bias is amplified in survey
> where disability is a key measure of interest.
>
> There are two aspects of surveying persons with disabilities: (1)
> including (or rather, not excluding) persons with disabilities in
> general household surveys and (2) surveying persons with disabilities
> about their disabilities. The aim of this paper is to present a
> general overview of issues relevant to increased inclusion of persons
> with disabilities in general population surveys, focusing on the key
> areas of sampling and respondent selection, the use of proxy
> respondents and adaptative technologies.
>
> *Recommendations for inclusion of persons with disabilities*
>
> *in general population surveys*
>
> [Sampling]{.underline}
>
> There is the issue of whether persons with disabilities enjoy an equal
> probability of selection, since many household surveys routinely
> exclude persons in-group homes and institutionalised populations. If
> appropriate representation of persons with disabilities is desired,
> researchers might consider augmenting sample frames with supplemental
> lists of long-term care and other health institutions.
>
> [Proxy interviewing]{.underline}
>
> There is considerable debate regarding the appropriateness of using
> proxy survey respondents to represent persons with disabilities in
> surveys. Survey methodologists approach proxy response from a
> measurement perspective: how much, if any, measurement error is
> introduced when proxy response is accepted? In contrast, disability
> advocates strongly feel that a proxy cannot give persons with
> disabilities adequate voice, given the subjective phenomenon of the
> disability experience.
>
> Traditionally, most survey researchers would consider the use of proxy
> respondents to be preferable to non-response, especially in situations
> where the selected respondent is willing but unable to participate in
> the interview. However, some disability experts strongly disagree and
> believe that non-response may under some circumstances be preferable
> to inaccurate reports from proxies.
>
> There are indeed several reports of differences between self-reports
> and proxy reports of health information. Studies examining self vs.
> proxy effects in the reporting of the activities of daily living
> (Andersen et al., 2000; Mathiowetz & Lair, 1994; Rodgers & Miller,
> 1997) indicate that self-reports of functional limitations are lower
> than those offered by proxies. Another study found close agreement
> between self and proxy responses on measures of overall health,
> functional status, social activity, and emotional health, but
> interestingly (in the context of disability studies), those proxies
> who spent more time helping the subject in activities of daily living
> tended to rate the subject's functional status and social activity as
> more impaired than did the subject (Epstein et al., 1989). None of
> these studies, however, randomly assign respondents to self vs. proxy
> conditions prior to the interview so that the conclusions must be
> considered with this bias in mind (Moore, 1988).
>
> A more recent review \[7\] has significant implications for the use of
> proxies in surveys about disability specifically. Through analysis of
> data from the two largest federal surveys that estimate disability for
> the U.S. population, the authors find significant differences between
> self-reports and proxy reports on the extent of disability among
> household members. Proxy respondents are likely to underreport
> disabilities for persons aged 65 years of age or younger but
> overreport disabilities for persons over 65. This is explained because
> disabilities which require help are more noticeable and thus likely to
> be overreported. But, as most disabilities as less observable, the
> general prevalence rate of disability would be underestimate using
> proxy reports.
>
> Disability advocates view the issue of proxy from a somewhat different
> perspective. Few would argue that a proxy respondent should never be
> used, as a no-proxy rule automatically would exclude those who may not
> be able to communicate directly with the interviewer. However, the
> message of the advocacy community is that researchers should always
> assume that the respondent could answer for himself or herself, even
> if the assistance of special technology or a translator is required.
> Most individuals, with or without disabilities, do not want families
> or other persons speaking for them. When a proxy is used, there should
> also be an attempt to incorporate the respondent's assessment of the
> proxy's answers.
>
> Another issue of debate is the selection of proxy respondents. In the
> experience of many of our key informants, choosing proxies is a
> delicate issue. At a minimum, asking the respondent to nominate a
> proxy would seem appropriate. People with disabilities experience a
> high rate of physical, sexual, and emotional abuse, frequently at the
> hands of those who care for them (Sobsey, 1994).
>
> Privacy issues should also be considered when interviewing persons
> with disabilities. Providing respondents with a private environment
> within which to answer highly sensitive survey questions is a topic
> receiving considerable methodological attention (Harrison & Hughes,
> 1997).
>
> [Adaptive technologies and procedures]{.underline}
>
> Issues related to adaptive technologies are of primary importance to
> persons who are deaf or hard-of-hearing, visually impaired, or who
> have speech impairments or speech disabilities. It is likely that a
> significant proportion of this population is being systematically
> excluded from participation in telephone surveys.
>
> Adaptive technologies are not limited to TTY (teletypewriter) services
> alone or to people who are deaf or hard of hearing. Self-administered
> surveys are as problematic as telephone surveys for some persons with
> disabilities. Possible steps to be taken to accommodate these persons
> in self-administered surveys are: offer the survey in a Braille
> version; offer the survey on audiocassette, with another blank
> audiocassette for the respondent to record his or her answers; offer a
> toll-free number the respondent can call to complete the survey by
> telephone; offer large-print versions of questionnaires; use
> high-contrast, low-glare paper; offer on-line or web-based surveys,
> particularly using voice-activated technologies; ensure that the
> questions are as clear and simple as possible, so that people with
> cognitive disabilities and mental illnesses can participate.
>
> It is also important to acknowledge that survey interviewers would
> benefit from training that addresses the communication needs of
> persons with disabilities.

5.  **Langlet ER.: *Estimation of proxy, time and content effect for the
    Health and Activity Limitation Survey*. Proceedings of the Survey
    Research Methods Section, American Statistical Association (2000)
    \[5\].**

> This study provides a method to measure proxy, time and content
> effects, by means of the comparison of results of a post-censal survey
> and the screening questions to identify disabled people through the
> Census questionnaire.
>
> The main problem of a post-censal survey is the identification of the
> population of interest through the Census questionnaire. Since the
> amount of questions on the Census form to identify the disabled
> population is usually relatively limited, differences in the
> populations identified will always persist with a detailed activity
> limitation survey. This fact implies almost inevitably the sampling of
> the Census \"no\" population. In fact, usually the most detailed the
> survey questionnaire, the most people with disability will be
> identified. The sampling of the \"no population\" is however very
> costly given the relative sample size required in the \"no
> population\" compared to the \"yes population\", (more than three
> times higher, as seen in the 1991 Health and Activity Limitation
> Survey (HALS)). The 1999 HALS test attempted to use new Census filter
> questions identifying a larger number of individuals limited in their
> activities. These new filter questions were expected to produce a
> larger proportion of false positives but more importantly a smaller
> proportion of false negatives. However, because of the very high rate
> of false negatives, this study did not permit to determine whether
> individuals with negative answers to the Census screening questions
> should be sampled or not.
>
> In this pilot test, two samples were designed. The larger one was to
> be used to measure the different components of change between the two
> phases. Comparing changes in terms of answers to the filter questions
> in both phases, it is possible to distinguish between proxy and time
> effect. Knowing who completed each phase and re-asking in Phase 2 the
> filter questions for both the Phase 2 and the Phase 1 reference
> periods makes this decomposition possible. Comparing changes in terms
> of answers to the filter questions in the first phase and ADL
> questions in the second phase, it is possible to distinguish between
> content effect (Census filter questions vs. ADL questions), the proxy
> and time effect. This sample will also be used to compare the
> characteristics of the false negatives and true positives. These
> analyses will permit to estimate the size and characteristics of the
> target population not covered by selecting only a \"yes\" sample.
>
> An effect not directly measured in the pilot test is the context
> effect. It is quite conceivable that respondents would be more likely
> to answer \"yes\" to filter questions in the context of an activity
> limitation survey than in the context of the Labour Force Survey.

6.  **Todorov, A. *Cognitive procedures for correcting proxy-response
    biases in surveys*. Appl. Cognit. Psychol. 17: 215-224 (2003)
    \[6\].**

> This paper identifies one source of systematic differences between
> self- and proxy-reports: proxies' higher reliance on inferences. Using
> data from the National Health Interview Survey on Disability (NHIS-D),
> proxy-response biases were modelled by independently collected
> measures of cognitive inferences. Conditional likelihood judgements
> about a number of disabilities (e.g. likelihood that a person has a
> disability given another disability) predicted the conditional
> disability reports for proxy- but not for self-respondents (e.g. the
> proportion of respondents who reported difficulty learning after
> reporting difficulty communicating). A model of self/proxy differences
> was estimated on data from the 1994 NHIS-D and tested against 1995
> data. The correlation between predicted and actual differences was
> 0.76. The correlation between predicted and actual proxy-reports was
> 0.95. Such research can be used to estimate and correct for systematic
> proxy-response biases.
>
> A number of studies have shown consistent differences between self and
> proxy-reports. Although both self- and proxy-reports are susceptible
> to reporting biases, self-reports tend to be more accurate. A typical
> strategy to disentangle the sources of differences between self- and
> proxy-responses is to introduce statistical control adjusting for
> demographic variables. An additional strategy is to start with
> specific assumptions about the nature of the differences between self-
> and proxy-reports and to model these differences based on the
> assumptions. This paper pursues the latter approach. From a cognitive
> point of view, there are two major differences between self- and
> proxy-respondents: (a) the information available for self- and
> other-judgements is different; and (b) the cognitive strategies of
> forming these judgements can be different.
>
> When people have to make a judgement about themselves, they can recall
> not only a variety of relevant behaviours but also feelings and
> beliefs. For example, the differential availability of information may
> lead to different judgements about what constitutes a disabling
> difficulty for oneself and what for another person. Moreover, the set
> of relevant behaviours is not accessible to the same extent to self-
> and proxy-respondents. One's behaviours are more salient and more
> accessible in memory than another's behaviours.
>
> Not only is the available information for self- and other-judgements
> different, but also the cognitive strategies of forming these
> judgements can differ. In general, when proxy-respondents do not have
> sufficient factual information about a specific question, they may try
> to infer the response by supplying assumptions about the question.
> Thus, given that proxy-respondents have less information than
> self-respondents, they should be more likely to rely on inferences
> when responding to survey questions.
>
> Todorov and Kirchner \[7\] have already identified one cognitive bias
> related to the different information available to self- and
> proxy-respondents. The less observable a disability, the larger the
> differences between self- and proxy-reports of the disability. This
> paper attempts to identify another bias related to proxies' higher
> reliance on inferences.
>
> If one of the sources of differences between proxy- and self-reports
> is the proxies' higher reliance on inferences, then measures of such
> inferences should predict proxy-reports. Specifically, likelihood
> judgements should be a significant predictor of proxy-reports when the
> analysis controls for self-reports, but they should not predict
> self-reports when the analysis controls for proxy-reports. Both the
> partial correlation analyses and the regression analyses confirmed
> this hypothesis. Lay judgements of the relatedness of disabilities
> predicted proxy-reports of disabilities measured in a nationally
> representative survey but did not predict self-reports. These
> judgements also predicted the differences between self- and
> proxy-reports. The higher the judged relatedness of disabilities, the
> more likely proxies' over-reporting of disabilities. The lower the
> judged relatedness of disabilities, the more likely proxies'
> under-reporting of disabilities.
>
> These findings suggest that proxy-reports are systematically biased.
> When respondents are asked to report about other people but do not
> have sufficient information, they appear to rely on inferences
> grounded in lay theories about the domain of questions. In the case of
> disabilities, respondents rely on theories about how disabilities are
> related to each other. This will lead to over-reporting of
> disabilities seemingly related to a previously reported disability and
> to under-reporting of disabilities seemingly unrelated to the
> previously reported disability.
>
> The variance of self/proxy differences accounted for by such
> informational and inference measures can be specifically attributed to
> systematic proxy-response biases. If these procedures prove to be
> reliable and valid, they can be used to estimate the unique variance
> attributable to proxy-response bias. For instance, the total variance
> of self/proxy differences can be partitioned into variance due to real
> differences, variance due to proxy-response biases, and error
> variance. Statistical analyses controlling for demographic differences
> can estimate the variance due to true differences, whereas analyses
> based on procedures as the ones in the current study can estimate the
> variance due to systematic proxy-response bias.
>
> These procedures seem to work in the health domain. For example,
> Todorov and Kirchner (2000) estimated that the use of proxy-reports in
> the NHIS-D underestimated the number of people with disabilities in
> the USA with 1,609,000. However, before such procedures are accepted,
> they should be validated in other survey domains (e.g. political and
> economic surveys). Further, one should use representative samples to
> estimate inferences or lay theories related to self/proxy differences.

**Other related studies**

-   A study attempting to identify differences in estimates due to mode
    of administration (self vs. interview) and type of interview
    (face-to-face vs. telephone) by evaluating self and proxy responses
    to two different questionnaires (MOS-HIV and EuroQol) showed that
    there were few differences in scores between self and interview
    administration and type of interview, but proxy respondents viewed
    patients as more impaired than did patients themselves on subjective
    aspects of health including mental health and health distress \[8\].

-   A study comparing agreement between self respondents and two types
    of proxy respondents \-- patient-designated lay proxy and
    professional proxy \-- when assessing the health status of patients
    with the SF-36 showed that professional proxies were better able to
    predict the patients\' responses than were the lay proxies. In
    professional proxies, the magnitude of the bias was absent or slight
    for six of the eight dimensions of the SF-36 with a small negative
    bias for the other two. Lay proxies showed a negative bias (i.e.
    they reported poorer function than did the patients themselves) for
    seven of the eight dimensions of the SF-36 (small in two and
    moderate in five). The study concludes that for group comparisons
    using the SF-36, professional proxies might be considered when
    patients cannot answer reliably for themselves \[9\].

-   Utilizing the 1990 Ontario Health Survey (OHS) inter-rater agreement
    was assessed for responses to questions that were answered both by
    individuals about themselves and by proxies on their behalf, as well
    as intra-rater agreement, assessing the effect of mode of survey
    administration (in-person interviews versus self-completed written
    questionnaires) on the self responses. The conclusions of the study
    are that proxy responses in the OHS for impairments of emotion and
    pain are not reliable indicators of self-response because proxy
    respondents consistently under-report the burden of morbidity. Also,
    levels of morbidity reported by subjects to interviewer-administered
    questionnaires may underestimate morbidity, relative to morbidity
    reported by subjects using self-administered questionnaires
    completed in privacy. It is also hypothesized that the relative
    magnitudes of inaccuracy introduced by interviewer administration
    relative to proxy reporting depends on the phenomenon being
    measured. When assessing pain, mode of administration is
    quantitatively a more important source of disagreement than type of
    respondent \[10\].

-   The aim of another study was to investigate two possible sources of
    bias inherent in using a household based postal questionnaire, the
    \"proxy effect\" and the \"saliency principle\" (reporting of only
    the most salient features). In surveys of symptoms and minor
    disability, a proxy effect is likely to be operative, underreporting
    disability. This effect is not apparent for obvious and
    long-standing problems such as dependence on others for help \[11\].

# The correspondence between respondent and proxy response was evaluated on 4 mental health measures (Affect Balance Scale, Centre for Epidemiological Studies Depression Scale, Mental Status Questionnaire, and Mini-Mental State Examination). Results indicated that respondent and proxy responses were strongly associated, particularly for the cognitive measures. This association was found even for respondents classified as depressed or cognitively impaired. Although there was evidence of proxy bias, with proxies underrating affective status and overrating cognitive status, the magnitude of the bias proved small for all scales but the Mental Status Questionnaire. Examination of response comparability by proxy characteristics showed that choice of proxy-affected agreement and bias \[12\].

# Proxy (designated by the subject) and subject responses to survey questions about chronic conditions, health symptoms, and physical and instrumental functioning were compared to determine the extent of disagreement, the direction of bias, and how disagreement and bias vary by proxy and subject characteristics. The authors observed substantial to almost perfect agreement for five of nine chronic conditions, no health symptoms, six of seven physical tasks of daily living, and seven of seven instrumental tasks of daily living. With few exceptions, proxies were more likely than subjects to report the presence of a condition, symptom, or functional problem. Variations in agreement and bias were noted by subject and proxy characteristics \[13\]. 

# Another study evaluates the response comparability between a group of elderly hip fracture patients and interviewer selected proxies on items pertaining to patients\' pre-fracture health and functional status. Proxies tend to overestimate patient disability relative to the patients themselves, especially with regard to capacity to perform instrumental activities of daily living. Although proxies who report the greatest contact with patients respond most comparably to the patients, when they do disagree, these proxies tend to overestimate patient disability \[14\].

# In a longitudinal study comparing self and family members\' reports of physical and functional health among patients with Alzheimer\'s disease and non-demented healthy older persons, functional health was consistently rated as more impaired by family caregivers of demented patients than by the patients themselves, a discrepancy not observed in the cognitively intact comparison group. Caregiver reports correlated significantly with declines in patients\' cognitive abilities as measured by formal testing, but self-reports did not. Patients did recognize deterioration in ADLs over time, despite progressively worsening cognitive ability. These data indicate that the capacity for self-observation is partially preserved in Alzheimer\'s patients in mild to moderate stages. Patient self-reports can provide valuable data for clinicians, but should be supplemented by detailed information from caregivers \[15\].

# In a study aimed at evaluating the predictive utility of self-reported and informant-reported functional deficits in patients with mild cognitive impairment (MCI) for the follow-up diagnosis of probable Alzheimer's disease (AD) showed that the patient's lack of awareness of functional deficits identified by informants strongly predicts a future diagnosis of AD. The analyses revealed that informant-reported deficits (but not self-reported deficits) and a discrepancy score indicating greater informant- than self-reported functional deficits significantly predicted the development of AD. These findings suggest that clinicians evaluating MCI patients should obtain both self-reports and informant reports of functional deficits to help in prediction of long-term outcome \[16\].

#  

MODE OF DATA COLLECTION

**Introduction**

Face-to-face interviews were the first mode of data collection used on
survey research, and they are still one of the most important sources of
data. However, factors such as cost and time constraints, new
technologies development, the steadily increase of number of surveys
targeted to the general population, etc., have lead to the expansion and
improvement of data collection modes.

In social research there exists a range of ways to administer a
questionnaire: paper-and-pencil face-to-face interview, computer
assisted personal interview (CAPI), computer assisted telephone
interview (CATI), web-based, computer assisted self-interview (CASI),
Audio-CASI, etc.

The choice of the mode of data collection is based on factors as the
objectives of the survey, the monetary and time constraints, the target
population, the availability of frames for sample selection, the
non-response rates associated to each mode of collection, etc. Mix-mode
surveys are being used more and more survey research.

While comparability across time and space is a main interest for survey
methodologists and because of the spread use of different modes of data
collection among surveys, it is important to ascertain the extent to
what differences in results are not due to real changes but to mode
effects or other bias components. Thus, it is necessary to study the
potential differential between modes as well as differences associated
to mix-mode data collection.

It is also important to detect if the various modes of data collection
are applicable to people with visual, hearing or cognitive impairments.

**Selection of literature**

1.  **Bradburn NM.: *Discussion of papers on mode effects from the
    Seventh Conference on Health Survey Research Methods.* Cynamon ML
    and Kulka RA, eds. Hyattsville, MD: National Center for Health
    Statistics. 2003 \[17\].**

> A general overview of the effects that different modes of data
> collection have on survey results is presented.
>
> The main kind of effects that one should worry about are (1)
> differences in the sample due to coverage or due to differential
> response rates, (2) differences in responses due to factors intrinsic
> to the mode of data collection, and (3) differences due to the social
> context within which the data are collected.
>
> [Factors related to the sample]{.underline}
>
> The most obvious difference between telephone and household based
> interviewing is in population coverage. Households without telephones
> cannot fall into the sample unless there is some mixed-mode provision
> to pick up the non-telephone households. This appears to be even more
> important for web-based surveys, due to the still-low proportion of
> households connected to the Internet.
>
> While not an effect intrinsic to the mode, differences in response
> rates are common between modes. This is a particularly difficult
> problem when the probability of having the characteristic of interest
> in the study is strongly correlated with the propensity to respond to
> the survey. For instance, if one wants to estimate rates for a disease
> and depends on different techniques for measuring prevalence, then one
> needs good quantitative estimates of the correlation between the
> measuring instrument and the propensity to respond in order not to
> underestimate the prevalence.
>
> [Factors intrinsic to modes of data collection]{.underline}
>
> There are factors intrinsic to mode that either prevent some types of
> questions being used altogether in one mode, or that interact with
> modal characteristics to produce response differences. The most
> obvious factor is the inability to use visual materials in telephone
> interviews, and as a consequence, showcards cannot be used either for
> complex response categories or as memory aids.
>
> Less obvious are interactions between mode of presentation and
> response bias. Stimuli presented in a visual mode are more susceptible
> to primacy effects; stimuli presented in auditory mode are more
> susceptible to recency effects. There is ample evidence to support the
> view that this interaction can cause significant modal differences in
> responses to questions involving lists for respondents to choose from
> or in multiple response categories.
>
> Order effects are difficult to overcome in any form of interviewing
> that depends on verbal interaction because, whether over the telephone
> or face-to-face, questions are asked one at a time in a specific
> order. Paper-and-pencil self-administered questionnaires may be
> superior in this regard because respondents can see all of the
> questions before they answer any of them, and, as a result, question
> and response alternative order effects are lower in mail
> questionnaires. The order effect benefit in self-administered
> questionnaires, however, would not hold true for CASI because the
> computer generally presents the questions one at a time.
>
> There also may be modal effects on the interviewers. In particular,
> different computer assistance programs have different functionality
> for the interviewer compared with paper-and-pencil questionnaires. For
> instance, programs differ in the ease with which interviewers can go
> back to previous questions and what they can do to change answers in
> light of subsequent questions.
>
> [Factors related to social context]{.underline}
>
> The third major factor that differs among modes is the kind and
> quality of social interaction involved in the data collection.
> Face-to-face interviewing involves the closest contact between
> interviewer and respondent, with telephone being somewhat more distant
> and the self-administered mode being the most remote. The development
> of CAPI, and the use of CASI and especially audio-CASI within a CAPI
> survey, have reduced the social desirability bias of the traditional
> face-to-face interview.
>
> A long line of research has shown that self-administered
> questionnaires reduce social desirability bias and produce more
> reports of negative behavior and unpopular opinions and fewer reports
> of positive behavior and socially approved opinions. The effect of
> telephone interviewing has not been consistent. There is more work to
> do before the effects of telephone interviewing on social desirability
> bias be understood.
>
> Telephone interviews tend to be shorter than fact-to-face interviews,
> and telephone interviewers do not have the non-verbal cues available
> to face-to-face interviewers to let them know when respondents may not
> understand a question or may have more to say. Thus, telephone
> interviews may result in less complete or well-thought-out answers.

2.  **CyBulski KA. and Ciemnecki AB.: *Interviewing populations with
    disabilities by telephone: survey design and operations.*
    Proceedings of the Survey Research Methods Section, American
    Statistical Association (2000) \[18\].**

> As part of the Health Care Financing Administration\'s evaluations of
> Section 1115 Medicaid Reform Demonstrations, Mathematica Policy
> Research, Inc. (MPR) conducted computer-assisted telephone interview
> (CATI) surveys in order to assess how Supplemental Security Income
> (SSI) recipients are faring in Medicaid managed care programs. The
> surveys addressed access to care, quality of care, and use of health
> services. MPR conducted more than 4,600 interviews. These were the
> first surveys of this type to be conducted solely by telephone for
> people with disabilities; therefore, care to accommodate their needs
> and to minimize proxy response was taken. SSI recipient's disabilities
> encompass physical or sensory disabilities (including blindness and
> deafness), mental illness and mental retardation. Respondents
> appreciated the opportunity to participate in these surveys.
>
> Those with physical or sensory disabilities were the most likely to
> self-respond (90 percent). The only notable source of non-response was
> the inability to locate sample members by telephone.
>
> This paper discusses the techniques used to design and conduct the
> surveys. To be successful, communication, stamina, and cognitive
> challenges had to be overcome. To accomplish this, (1) soft consonant
> sounds were eliminated to overcome high-frequency hearing loss, (2)
> \"breaks\" for respondents were built in, (3) neutral encouragement
> was incorporated, (4) checks for unexpected responses were designed,
> and (5) structured probes for questions that might be difficult to
> understand were used. The interviewers were trained on the challenges
> of interviewing people with disabilities by telephone and were
> provided with guidance for overcoming each challenge. Also sensitivity
> training regarding people with disabilities was provided.
>
> [Challenges in interviewing disabled people by telephone]{.underline}
>
> Survey researchers are unaware that a particular respondent has a
> disabling condition, especially if the interview is being conducted by
> telephone. The most common challenges were grouped in three broad
> categories:

-   Communications, including both hearing and speech impairments.

-   Stamina, including both mental and physical fatigue.

-   Cognitive barriers, including emotional disturbance, difficulty
    processing questions and responses, lack of complete or specific
    knowledge, and confusion about the purpose of the interview.

> [Design techniques used to overcome these challenges]{.underline}
>
> The questionnaires featured four techniques designed to overcome these
> challenges.
>
> First, to eliminate from the questionnaire high frequency sounds (s,
> z, t, f, and g). High-frequency hearing loss is common. By replacing
> high frequencies with low frequencies, we could make the interview
> easier to hear.
>
> Second, checkpoints for interviewers were built in to assess if the
> respondent needed encouragement or was becoming too fatigued to
> continue the interview. The survey had two versions: one that took, on
> average, 44 minutes to administer, and another taking 22 minutes.
> interview. If the interviewer noticed that the respondent was
> fatigued, they asked the respondent if they wanted to continue or
> preferred to schedule another time to compete the survey. While
> pretesting it was also found that it was useful to provide respondents
> with positive feedback about completing the survey task. Based on
> these preliminary findings, three checkpoints were included in the
> interviews. Despite their disabling conditions, about three-quarters
> of the respondents were able to complete the 22-minute interview
> without special intervention from the interviewer. About half of the
> respondents completed the 44-minute interview without needing special
> intervention. These differences persisted across all disabling
> conditions. Generally, the interviewers perceived that respondents
> were happy to be interviewed and were reluctant to stop. The
> interviewers strongly recommended the use of encouragement in future
> surveys of populations with disabilities.
>
> Third, we designed questions to \"double check\" unexpected responses.
> Of the 198 adult respondents who initially reported no doctor visit in
> the previous year, 64 percent changed their answer after the
> confirmation question. Overall, there was no difference between proxy
> and self-responses. Sample members with mental illness were the least
> likely to change their response when presented with the confirmation
> question (55 percent), while sample members with mental retardation
> were the most likely to change their response (72 percent). This may
> be because they were in the group that had the most difficulty
> understanding the original question or because they were most
> sensitive to providing socially desirable responses.
>
> Fourth, a series of structured probes were designed to keep the
> interview for the respondents with disabilities as comparable as
> possible to the interview administered to all types of Medicaid
> recipients. The majority of questions worked well for most of the
> respondents with disabilities. Nevertheless, some respondents needed
> to have concepts defined. The probes appeared on the CATI screens in
> the order that we wanted interviewers to use them. If none of the
> probes helped the respondent, the interviewer was allowed to rephrase
> the question in a way she thought the respondent would understand.
>
> [Interviewer selection, training, and supervision]{.underline}
>
> The training program consisted of giving the usual background and
> purpose of the study. In addition, trainers addressed challenges the
> interviewers were likely to face, as well as a sensitivity exercise
> designed to demonstrate that interviewers should be kind and have
> unconditional positive regard for respondents regardless of their
> limitations. Interviewers were encouraged to place their focus on the
> individual first and the disability last.
>
> Trainers presented guidance for overcoming each one of the challenges.
>
> To overcome hearing impairments, interviewers learned (1) to use a
> normal tone of voice and not restrict conversations to monosyllabic
> words, (2) to use controls on headsets to amplify outgoing sounds, and
> (3) to use a text telephone (TTY/TTD) relay operator if necessary.
>
> To overcome speech impairments, interviewers learned (1) to use
> controls on the headsets to amplify incoming sounds; (2) to not be
> afraid to ask the respondent to repeat what he or she said, (3) to be
> patient, because speech patterns become easier to discern after a few
> minutes; (4) to repeat aloud what they did hear and understand if
> clarification is needed; and (5) to not pretend to understand
> something they did not. Finally, as many people with speech
> impairments may be unable to monitor their tone of voice interviewers
> were asked not to make assumptions about people based on their tone of
> voice.
>
> To overcome stamina challenges, interviewers were trained to be aware
> of behaviours that might suggest the respondent is too fatigued to
> continue.
>
> To overcome cognitive challenges, interviewers (1) were showed how to
> keep the respondent free of distractions, (2) were instructed to say
> the respondent\'s name often, and (3) were suggested not to exaggerate
> the inflection or tone of their voice (such exaggerations call
> attention to themselves and can be distracting and confusing).
>
> Finally, it has to be recognized that conducting telephone interviews
> with people with disabilities would be challenging even for
> experienced, well-trained interviewers. Interviews take longer because
> questions need to be repeated and multiple sessions may be required.
> Extra efforts were made to support the interviewers and reduce stress
> and burnout. It was emphasized that the usual performance measures,
> such as hours per completed interview, are not as important as taking
> the time to be sure the respondent understands the question and is
> answering reliably. Supervisors reminded interviewers that break-offs
> are acceptable and desirable if respondents are fatigued. Supervisors
> and colleagues provided support during and after interviews and at
> regular debriefing sessions.
>
> Concluding, it has been demonstrated that, through careful instrument
> design and survey procedures, it is possible to conduct telephone
> interviews with persons with disabilities. Shorter interviews create
> less respondent burden and can be conducted with fewer break-offs and
> less need for encouragement from the interviewer than longer
> interviews. The vast majority of respondents could answer for
> themselves. Respondents were easily able to answer most questions
> about health functioning, demographics, access to care, satisfaction,
> and quality of care.

3.  **Biemer PP.: *Non-response bias and measurement bias in a
    comparison of face to face and telephone interviewing*. Journal of
    Official Statistics, Vol. 17, No. 2, 2001, pp. 295-320. Statistics
    Sweden \[19\].**

> This article provides a study design and analytic methodology for
> evaluating and comparing the quality of survey data in the case of
> face to face and telephone (CATI) interviewing. Under the proposed
> design, the mode differences are decomposed into measurement bias and
> non-response bias components.
>
> The estimation method for these two bias components in this article
> does not rely on the existence of gold standard estimates or
> assumptions such as 'more is better'. Latent class analysis (LCA) is
> used to estimate the classification error probabilities for the
> face-to-face and telephone modes and the subsequent obtention of the
> measurement bias component as a function of these estimates of the
> response probabilities. A key feature of the study measurement error
> evaluation design was the design of a telephone test-retest
> reinterview survey for both the CATI and face-to-face survey
> respondents. To obtain estimates of the non-response bias for the
> face-to-face survey a telephone follow-up survey of the face-to-face
> non-respondents was conducted. Non-response bias for the telephone
> survey is estimated using an error-corrected estimator of the true
> prevalence rate.
>
> The methodology is illustrated using data from a special study
> conducted for the U.S. 1994 National Health Interview Survey (NHIS).
>
> The analysis of mode biases suggests that neither CATI nor face to
> face interviewing is a uniformly superior mode of interview across all
> characteristics. For the characteristics considered in the analysis,
> measurement bias was considerable for both modes, often exceeding the
> non-response bias. The CATI non-response bias was often larger than
> that of the face-to-face interviewing on average, but was often offset
> by the measurement bias. On the other hand, the measurement bias for
> the face-to-face mode was often larger than that for the CATI mode. On
> average, the overall quality of estimates from both modes was much the
> same. These findings suggest that CATI survey can produce data that
> compare well in quality to those produced by a face-to-face survey for
> many characteristics, despite a difference in response rates of more
> than 20 percentage points.
>
> The article also provides possible ways to extend the investigation,
> such as extending LCA to the other variables in the survey, that would
> provide additional evidence regarding the relative quality of face to
> face and CATI data. Furthermore, there is still much to learn
> regarding the validity of the estimates from the LCA.
>
> The article states that high emphasis should be given to the
> estimation and control of measurement error in surveys, avoiding the
> general trend to judge the quality of a survey on the unique basis of
> the overall response rate. The results suggest that factors such as
> the mode of interview, the design of the data collection process and
> other features of the survey design that affect measurement error may
> be even more important than the final response rate in judging data
> quality.

4.  **Maffeo C., Frey W. and Kalton G.: *Survey design and data
    collection issues in the Disability Evaluation Study.* Proceedings
    of the Survey Research Methods Section, American Statistical
    Association (2000) \[20\].**

> This paper examines five major challenges to achieving the goals of
> the National Study of Health and Activity (NSHA). In particular, a
> number of methods to maximize response rates were tested in the pilot
> study for the NSHA.
>
> Response rates among three alternative data collection methods were
> compared. In the first and second methods a sample was selected using
> random-digit dial procedures (RDD). Initial screening was done by
> telephone with non-response follow-up in-person for eligible persons
> whose addresses could be obtained. The only difference between the
> first and second methods is the location where the interview and exam
> were conducted. The third method involved selecting an area
> probability sample, conducting the initial screener by telephone for
> cases where telephone numbers could be obtained and in-person for the
> others.
>
> The screener response rate in the RDD samples was low, despite the use
> of follow-up techniques. Besides standard follow-up phone calls, an
> attempt was made to obtain addresses for the respondents and send a
> refusal conversion letter that included a \$5.00 incentive. Because
> addresses could be obtained for only about one-half the sample, the
> improvement in the response rate was small, less than 10 percent.
>
> The initial screener response rate was substantially higher (about 25
> percentage points) in the area sample. Because addresses for the
> entire sample were available, refusal conversion efforts could be
> conducted in-person with the entire sample. While this is a more
> costly approach, the gain in response is quite large.
>
> Interestingly, response rates for the follow-up screener and interview
> and the exam were about the same in the RDD and area probability
> samples, although the overall response rate was much higher in the
> area sample because of the higher initial screener rates.
>
> The initial examination of response rates by rural/urban status shows
> higher rates in the rural areas than in the suburban or urban areas.
>
> Preliminary examination of follow-up screener/interview and exam
> response rates does not show a difference between the group initially
> screened by phone and the group initially screened in person.
>
> As a result of the pilot study findings, several design changes were
> considered, including selection of an area probability sample and
> initial screening done by telephone for cases where telephone numbers
> can be obtained.

5.  **St-Pierre M. and Béland Y.: *Mode effects in the Canadian
    Community Health Survey: a comparison of CAPI and CATI.* Statistics
    Canada \[21\].**

> The Canadian Community Health Survey (CCHS) consists of two
> cross-sectional surveys conducted over a two-year repeating cycle. The
> mode study was fully integrated as part of the CCHS cycle 2.1 to
> better understand potential differences caused by the two methods of
> collection used in the CCHS -- CAPI and CATI -- on survey estimates.
> It was anticipated that the increased number of CATI interviews in
> cycle 2.1 compared to cycle 1.1 would affect the comparability of some
> key health indicators over the two cycles either by artificially
> amplifying or masking a real change in behaviours.
>
> The mode study used a split-plot design with a unique sample frame
> where the secondary sampling units were randomly assigned to either
> CAPI or CATI. Response rates were high and very similar for both modes
> of collection (final response rate of 72,7% for CAPI vs. 75.1% for
> CATI).
>
> The main purpose of the mode study was to compare health indicators
> derived from data collected in person (CAPI) and those collected over
> the phone (CATI). Univariate and multivariate analyses were conducted,
> comparing the two modes of collection. First, chi-square tests for
> association were used to compare the two mode samples in terms of
> socio-demographic characteristics. Direct comparisons of several
> health indicators between the two modes are also presented, looking
> for significant differences between the estimates. A series of
> multiple logistic regressions were performed. This analysis evaluates
> the effect of the mode of collection on the prevalence of several
> health indicators when controlling for the socio-demographic and
> household variables.
>
> Although minor differences were detected in the socio-demographic
> profiles the two mode samples are representative of the target
> population and are comparable. Regarding the comparison of health
> indicators between CAPI and CATI modes, nearly no differences were
> found in point estimates for most of the health indicators measured by
> CCHS such as tobacco use (all ages), chronic conditions, activity
> limitations, fruit and vegetable consumption and others. Significant
> differences were however found between CAPI and CATI for some health
> indicators: self-reported height and weight, physical activity index,
> contact with medical doctors and self-reported unmet health care needs
> being the most notable. Although the multivariate analysis somewhat
> attenuated the impact of the mode effects when socio-demographic
> characteristics are considered, it is believed that any comparison of
> the above indicators over the two cycles should take into
> consideration the increased number of CATI in the second cycle. It is
> important to mention that other methodological (sample sizes,
> reference period, questionnaire, etc.) and contextual (changes in
> standards, true change, etc.) aspects should, as well, always be taken
> into consideration in any comparison of survey indicators over time.
>
> Comparison of non-respondents to CAPI and CATI yields no big
> differences between the two modes. Non-contact and refusal rates are
> similar. However, the age group distributions of person-level
> non-respondents show differences at the two ends. A response from
> elderly persons (65 and up) is much more difficult to obtain over the
> phone (13.9% non-response) than in person (8.9%) while the opposite is
> observed for the younger age group (12- 19). One could think that
> elderly persons with a physical condition might have difficulty to get
> to the phone. The same could be said with teenagers where the more
> physically active ones could be home less often and hence less
> available for a personal interview. This would however require further
> research.
>
> Extensive literature exists on comparisons between personal and
> telephone interview techniques and a great deal of inconsistencies in
> the results is certainly noticeable as these studies report varying
> magnitude of mode effects. It has been suggested that the
> inconsistency among results is probably caused by differences in the
> design of the studies. There is however unanimity on the presence of
> mode effects for some variables and the non-negligible biases on
> survey estimates.
>
> The authors of this paper think that the differences found in the mode
> study of the CCHS between CAPI and CATI are mainly caused by two
> confounding factors: social desirability and interviewer variability.
> Interviewer variability is inevitable in large surveys conducted by
> National Statistical Organisations. Some aspects of the work
> environments (e.g. supervision) of the two collection methods are
> simply so different that it is reasonable to believe that
> interviewers' behaviours could differ from one to the other and hence
> interviewer variability biases could be introduced.

6.  **Leeuw ED., Hox J., Kef S. and Van Hattum M.: *Overcoming the
    problems of special interviews on sensitive topics: computer
    assisted self-interviewing tailored for young children and
    adolescents.* 1997 Sawtooth Software Conference Proceedings.
    Pp.1-14. Sawtooth Software Inc. \[22\].**

> This study includes a mixed-mode CAPI and CASI (computer assisted
> self-interview) survey on personal networks, dating, and well-being of
> adolescents and young adults (14-24) with a visual handicap. This
> paper presents a literature review of data quality in CASI surveys,
> the special adaptations made for conducting the survey and presents
> empirical findings and general recommendations for the adaptation of
> computer assisted (self) interviewing for special populations.
>
> [Literature review]{.underline}
>
> Empirical research has shown that self-administered questionnaires
> when compared to interviews produce more valid reports of sensitive
> behavior and less social desirable answers in general. The
> self-administered procedure gives the respondent more time to
> understand the meaning of the question, and retrieve and compose an
> answer, which improves the quality of answers. This is especially
> important when surveying special populations, such as children,
> adolescents and elderly. Paper self-administered has a serious
> disadvantage: only relatively simple questionnaires can be used. CASI
> overcomes this problem, and also enhances the feeling of privacy of
> the self-administered form. Therefore CASI is specially suited for
> special population surveys on sensitive topics.
>
> Respondents generally like CASI. They find it interesting, easy to use
> and amusing. Only a few respondents complain about physical problems
> such as eyestrain. Furthermore, respondents tend to underestimate the
> time spent answering s computer assisted questionnaire.
>
> The technological possibilities of CASI have a positive influence on
> data quality. A consistent finding in the literature is that
> item-non-response caused by respondent- or interviewer errors is
> virtually eliminated, but there is little reduction on 'do not know'
> or 'no opinion' answers.
>
> Also respondent's self-disclosure is significantly higher in CASI as
> compared to paper self-administered questionnaires. The effect
> reported is larger when more sensitive information is asked.
>
> There are indications that time pressure interacts negatively with the
> perceptual and motor skills necessary for reading questions from a
> screen and typing in answers correctly. Therefore, respondents,
> especially when they are a special or difficult group should never be
> put under time pressure.
>
> [A mixed-mode CAPI and CASI survey of visually impaired and blind
> adolescents and young adults]{.underline}
>
> The survey was conducted to 354 blind and visually impaired aged 14 to
> 24 over the Netherlands. It was used a complex questionnaire of more
> than 260 questions.
>
> Especially the questions on the ego-centered network are very complex
> for interviewer to administer, so a CAPI procedure was developed for
> this part of the questionnaire. The questions on self-esteem,
> well-being, and loneliness are of a sensitive and private nature and
> CASI was the best choice for these questions. Specific adaptations of
> the procedures were made to accommodate the special needs of the blind
> and visually impaired respondents.
>
> The CASI part was a 'manual' Audio-CASI: The interviewer handed over
> the computer to the respondent, then the interviewer read the
> questions aloud to the respondent, who typed in the answers. A special
> hardboard template, with both Braille and magnified numbers, enabled
> the respondents to use the keyboard themselves.
>
> To support the respondent's memory, paper flash-cards with the
> response categories were developed (one version in Braille, one in
> very large magnification and one in little magnification).
>
> The questionnaire and the procedure were pre-tested extensively.
>
> An interview including the CASI part took on average 90 minutes.
>
> The experiences of the blind and visually impaired adolescents were
> very positive. In the Netherlands almost all blind and visually
> impaired young persons are very familiar with computers.
>
> The mixed-mode approach created interest and motivated the
> respondents. CASI gave the respondents more privacy while CAPI proved
> efficient with the complex network questions. The interviewers
> stressed that it was important to clearly verbally state that they
> were not looking at the screen during the CASI part.
>
> No significant differences in reliability were found between the blind
> and the visually impaired sub-groups. Thus, the hypothesis that it
> would be somewhat harder for the blind to use the CASI-part resulting
> in less consistent answers was not supported by the data.

**Other related studies**

-   In the fourth session of the Eighth Conference on Health Survey
    Research Methods, it is shown a big-picture view of where we are
    with respect to how to do general population health surveys in 2003.
    From a total survey design perspective, the discussion considers
    costs, the quality of sample frames, the rates and biases associated
    with non-response, and the issues of data quality and data
    comparability associated with alternative modes of data collection.
    A presentation deals with the present and future of surveys done
    solely based on random-digit dialling (RDD). Another presentation
    presents data from the US National Health Interview Survey on one of
    the threats to RDD surveys: households that have substituted
    individual cell phones for household telephone service. Several
    papers explore the potential of Internet, mail, and in-person
    interviewing, to complement or substitute for RDD-based telephone
    surveys. And finally, two aspects of the challenges currently faced
    are discussed: (1) issues related to non-observation due to
    limitations in the sample frames that are used and to non-response,
    and (2) the challenges related to measurement when data are
    collected using more than one mode \[23\].

-   In a study on inclusion of people with disabilities in telephone
    surveys, it was tested the hypothesis that a telephone survey would
    under represent adults with disabilities, and that the adults with
    disabilities who responded would report lower prevalence of sensory,
    mental, self-care, and multiple limitations than those observed in
    people with disabilities in the general population. The
    characteristics of Washington adults with disabilities identified by
    four different surveys were compared, three of them being telephone
    surveys. Contrary to expectations, post hoc analyses of all
    telephone surveys found significantly higher prevalence of
    disability in the Washington adult population than did the C2SS
    (Census 2000 Supplementary Survey). The hypothesis of more sensory,
    mental, and self-care limitation in telephone disability samples was
    supported in only 2 of 11 instances in which a disability sample was
    asked about 1 of these limitations. Findings were not explained by
    differences in disability definition or type of informant. These
    results suggest that population telephone surveys do not under
    represent adults with disabilities \[24\].

-   A study aimed at comparing face-to-face and telephone administration
    of the 12-item General Health Questionnaire (GHQ) and the Revised
    Clinical Interview Schedule (CIS-R) in a UK general practice sample
    suggests that telephone assessment of mental health using the GHQ
    and CIS-R is a reasonable method to be used in primary care research
    in the UK. It was found pointed the limitation that telephone
    responses from older people might be different from face-to-face
    assessments for the GHQ. However, telephone interviewing appeared
    less acceptable by participants \[25\].

-   A study was conducted to compare the results of a health-related
    quality of life measurement instrument, the Australian version of
    SF-36, conducted by mail and telephone. Modes were compared across a
    number of indices: data collection costs; consent rates and
    non-consent bias; data quality (completeness of data and internal
    consistency reliability); and response effects. Data collection
    costs were lower for the telephone mode. A significantly higher
    consent rate was achieved with the telephone mode. Those who were
    younger were more likely to refuse to participate when the mail mode
    was adopted, while older people were more likely not to consent to
    the telephone mode. The rate of missing responses was higher for the
    mail mode, while significant differences were found between modes in
    internal consistency reliability estimates. Health ratings were more
    favourable for the telephone administration \[26\].

-   To validate a questionnaire based on the Expanded Disability Status
    Scale (EDSS), patients in five different European centres were
    assessed independently by pairs of trained EDSS raters, first by
    telephone interview and a few days later by standardized
    neurological examination. EDSS assessment by telephone was highly
    correlated with the EDSS determined by physical examination. Best
    agreement could be found in higher EDSS scores, where assessment by
    telephone interview might be needed most. The telephone
    questionnaire is a valid tool to assess EDSS score in cases where
    the patient is unable to continue visiting a study centre or in
    long-term follow-up of trial participants \[27\].

-   A telephone survey with a sample of 893 residents from a Montreal
    catchment area was developed to asses mental disorders by the
    Composite International Diagnostic Interview Simplified (CIDIS), an
    instrument especially designed to be used in mail or telephone
    surveys. The prevalence rate for any mental disorder was lower in
    this study than in some large-scale epidemiological surveys
    reviewed. This could be explained by methodological differences,
    such as number of disorders covered and period of reference. With
    regard to specific mental disorders, results appeared very similar
    to those of other studies. In sum, aside from being lower in cost,
    telephone surveys can yield results comparable to those obtained in
    large-scale epidemiological surveys conducted by means of
    face-to-face interviews \[28\].

-   Another study had the objectives of (1) to analyse the refusal bias
    associated with postal questionnaires and face-to-face interviews
    and (2) to compare responses elicited from a postal questionnaire
    with those provided by a face-to-face interview in subjects over 75
    years of age. The non-respondents to the postal questionnaire were
    more cognitively impaired, more disabled, and showed a higher 1-year
    mortality rate (13.2 versus 5.2%) than respondents. Late respondents
    who needed a reminder letter were similar to non-respondents. The
    non-participants in the face-to-face interview were similar to the
    participants on age, sex, 1-year mortality rate, and responses given
    to all but two postal questions. Comparison of postal questionnaire
    responses and face-to-face responses revealed that agreement was
    substantial to moderate for factual information but poor for
    clinical information (depression, cognitive impairment, disability).
    The conclusion of the authors is that non-response bias is evidenced
    when postal questionnaires are utilized in the very elderly
    population and caution should be used when interpreting health data
    obtained by this technique. Refusal bias from face-to-face
    interviewing is less important in this population \[29\].

-   Using a national sampling frame of non-institutionalized adults from
    the General Social Survey, we randomly assigned adults to a mail
    survey (80%) or a computer-assisted telephone survey (20%). The
    surveys were designed to provide national norms for the SF-36 Health
    Survey. A significantly higher response rate was achieved among
    respondents randomly assigned to the mail (79.2%) than telephone
    survey (68.9%). Non-response bias was evident in both modes but,
    with the exception of age, was not differential between modes. The
    rate of missing responses was higher for mail than telephone
    respondents (1.59 vs. 0.49 missing items). Health ratings based on
    the SF-36 scales were less favourable, and reports of chronic
    conditions were more frequent, for mail than telephone respondents
    \[30\].

-   Another study had two primary aims: (1) to determine the usefulness
    of a telephone-administered health status questionnaire as an
    epidemiological survey instrument in groups at high risk for
    morbidity and disability; and (2) to evaluate the value of a
    telephone interview for making judgments about individual patient
    management. A structured telephone interview was compared with a
    face-to-face interview in 366 individuals with a stroke or an
    orthopaedic condition up to 5 years after discharge from a
    rehabilitation hospital. Standardized assessments including the
    Barthel Index, the Zung Scale, the Reintegration to Normal Living
    (RNL) Index, and the Pfeiffer Short Portable Mental Status
    Questionnaire (SPMSQ), along with questions on health, medication
    use, and health events were administered using both modes.
    Comparability between the modes was also assessed according to the
    type of respondent, self, or proxy. On the Barthel Index, the Zung
    Scale, the RNL Index, and the SPMSQ the percentage of patients
    indicated as having no disability was similar between the modes.
    Proxies\' responses were, in general, as consistent between the
    modes as patients\' responses. The reliability coefficients
    indicated moderate to substantial agreement between the modes on the
    majority of indices and health-related questions. Discord between
    modes, when present, was greatest for individuals with moderate and
    severe disability, with less frequent reporting of disability on the
    telephone. This study supports the use of telephone interviews in
    determining the prevalence of disability in the community and
    supports the use of the telephone interview in the case finding
    process \[31\].

-   Two low-cost mental health survey strategies (mail and telephone)
    were compared in terms of cost, response rate and quality of data
    obtained. A total of 1,074 persons agreed to participate in the
    study as a sample, one-half by telephone and the other half by mail.
    They completed the Diagnostic Interview Schedule Self-Administered,
    a questionnaire designed to be self-administered, which was used to
    assess specific mental disorders and to evaluate risk factors. In
    addition, 239 respondents who were selected according to the
    presence or absence of specific diagnoses were reinterviewed
    face-to-face using the Diagnostic Interview Schedule as an external
    criterion. The telephone method yielded a better response rate (15%
    higher) and better control over answers (for example, less missing
    data). The mail strategy was less expensive and appeared to yield
    data of slightly better quality, particularly for respondents
    suffering from anxiety disorders \[32\].

RESPONSE BIAS

**Introduction**

In conducting disability surveys or censuses including health-related
questions, we should know if respondents are different from
non-respondents in terms of such characteristics. In case they are not,
the measurement of non-response bias would be of great help to correct
health estimates resulted from survey data.

For too long, the efforts relating non-response have focused on rates,
but new strategies turn both research and practice to a focus on error.
While high response rates certainly reduces the risk of high
non-response bias, a low response rate does not necessarily mean high
non-response error or poor data quality.

The topic of response bias is strongly related with mode of data
collection and self- and proxy-response issues. The present selection of
literature attempts to shed light on these relationships and to provide
information on the quality of data that results when different
combinations of these issues are put together.

**Selection of literature**

1.  **Hendershot GE, Colpe LJ, Hunt PC: *Persons with disabilities:
    non-response and proxy response in the National Health Interview
    Survey on Disability.* International Conference on Survey
    Non-response. Portland OR, 28-31 October 1999 \[33\].**

> This study was designed to describe and analyse the association
> between levels of disability and survey non-response, the components
> of non-response (non-contact and non-cooperation), and
> non-self-response (proxy/assisted response).
>
> The study focuses on characteristics of the social environment and the
> respondent. Non-response can be seen as an outcome of this social
> interaction. The components of non-response analysed here are
> non-contact and non-cooperation. The principal factor affecting
> non-contact is the household pattern of staying at home. Households in
> which people are at home a lot are more likely to be contacted than
> other households. Persons with disabilities are older, on average,
> than persons without disabilities, and somewhat more likely to live in
> non-metropolitan areas, factors that increase the chances of their
> being contacted. On the other hand, they are less likely to live in
> households with young children, a factor decreasing the chances of
> contact. Disability itself might make it more difficult to be outside
> the home, which would increase the chances of contact. However, there
> is some evidence that persons with disabilities are more fearful of
> opening their doors to strangers, decreasing the chances of contact.
>
> There exit two theories that try to explain differences in
> non-cooperation rates by social status. In **exchange theory**,
> persons who feel that the survey sponsor has provided (or could
> provide) benefits to them are more likely to cooperate. In **social
> isolation theory**, however, people who are isolated from the
> mainstream society, such as the poor, feel less responsibility toward
> government and are less likely to cooperate. Persons with
> disabilities, by exchange theory, might be expected to be more
> cooperative with a government-sponsored survey, but by social
> isolation theory they might be expected to be less cooperative. The
> present study hypothesized that disability would be negatively related
> to both non-contact and non-cooperation. That is, it was hypothesized
> that as the severity of disability increases from mild to moderate to
> severe, the rates of non-contact and non-cooperation (given contact)
> will decrease.
>
> According to the World Health Organization International
> Classification of Functioning and Disability (WHO, 1999), "activity is
> the performance of a task or action by an individual," and "activity
> limitations are difficulties in performance of activities."
> Limitations are classified as mild, moderate, severe, and complete.
> The survey asked if a person had some difficulty, a lot, or was unable
> to lift a grocery bag, lift a glass, hold a pen, shake hands, bend to
> pick up a shoe, stand for 20 minutes, climb stairs, or walk three
> blocks.
>
> The analysis is designed to describe the relationship between the
> severity of disability, as measured by the number of activity
> limitations, and rates of non-contact (given eligibility),
> non-cooperation (given contact), and non-self-response (given
> cooperation). The analysis proceeds in two stages. First, the
> bivariate relationship between disability and response patterns is
> described. Second, the multivariate relationship between disability
> and response patterns is analysed with controls for other variables
> known or believed to affect survey contact, cooperation, and
> self-response.
>
> The study shows that non-response rates decline with the severity of
> disability. The same relationship is found for the two components of
> non-response, non-contact and non-cooperation: the more severe the
> disability, the lower the rates. However, a different relationship is
> found: the more severe the disability, the higher the rate of
> non-self-response. Thus, although persons with severe disability are
> more likely than persons with less severe disability to be contacted
> and to cooperate, when they are interviewed they are less likely to
> answer for themselves; instead, proxy and assistant respondents tend
> to answer for them.
>
> This study does not reveal the causes of the association between
> severity of disability and rates of contact and cooperation, but it
> does rule out as causes some possibly confounding factors, such as
> age, living arrangements, and metropolitan residence. At home patterns
> are presumed to be the major factor in contact rates, and there is
> reason to expect persons with more severe disability to be at home
> more often. The competing hypothesis, that persons at home are less
> likely to open the door to a stranger, is not supported by the data.
> The observed association between the severity of disability and rates
> of cooperation is consistent with the hypothesis based on exchange
> theory and inconsistent with the hypothesis based on social isolation
> theory.

2.  **Kahana B. et al.: *Mild cognitive impairment and accuracy of
    survey responses of the old old.* Health Survey Research Methods
    Conference, 1996 \[34\].**

> The term \"mild cognitive impairment\" or MCI describes a broad range
> of conditions, including limited dementia, mild dementia, age-related
> memory impairment, and benign senescent forgetfulness.
>
> In order to explore the role of MCI in affecting responses to health
> surveys, the relationships between four alternative indicators of
> possible cognitive deficits and response patterns to a health survey
> were examined.
>
> The report is based upon a longitudinal study of adaptation to frailty
> among a sample of healthy elders living in three Florida retirement
> communities. This research was undertaken in part to help establish
> appropriate and empirically based exclusion criteria based on a
> threshold at which inadequate responses are likely to pose threats to
> survey validity.
>
> The study presents data on the association between alternative
> cognitive impairment criteria and several dimensions of survey
> responses of participants in a longitudinal study. Each of the four
> alternative measures of cognitive impairment should show an
> association with the following outcome variables reflecting accuracy
> and reliability and completeness of survey responses: (a) the accuracy
> of height and weight estimates, (b) the reliability of physical health
> and psychological well-being measures, and (c) the frequency with
> which respondents fail to answer questions as part of physical health
> and psychological well-being measures. The four measures used are: (a)
> a self-report indicator of memory problems (metamemory), (b) an
> interviewer-rated measure of mental confusion, (c) the digit-symbol
> subtest of the Wechsler Adult Intelligence Scale (WAIS) (Wechsler,
> 1981) constituting a performance-based measure of fluid intelligence
> and (d) the 10-item Pfeiffer Short Portable Mental Status
> Questionnaire (SPMSQ; Pfeiffer, 1975).
>
> Findings of the study provide consistent indications of reliable and
> generally accurate survey responses of old old persons even when they
> exhibit MCI. Structured surveys may help elicit from the elderly
> accurate and valid responses by providing a meaningful context for
> information retrieval. Under such conditions, the elderly tend to show
> minimal deficits (Craik & Jennings, 1992). It is thus clear from the
> data that mild memory loss does not automatically translate into
> incapacitating confusion.
>
> Respondents\' memory problems, when self-perceived, may be reflected
> in increased \"I don\'t know\" responses or missing data.
> Alternatively, to the extent that respondents are unaware of memory
> deficits, inaccurate information on items requiring recall may be
> generated. Respondents may also deal with deficits by refusing to
> answer more demanding portions of the survey.
>
> In considering the findings of the study, it is noteworthy that they
> support the conclusion that older adults do not require special
> methods of data collection and are able to perform on a variety of
> complex tests and questionnaires. Our data do not support suggestions
> that older persons show lower internal consistency in handling complex
> scales, for example, those in which negatively and positively worded
> items are balanced.
>
> In evaluating implications of this study, it should be noted that in
> this research, as in other community-based surveys, there is a
> selection factor operating that excludes individuals who do not feel
> sufficiently cognitively or physically intact to participate in
> research.
>
> It can be concluded that survey response accuracy generally remains
> unaffected by mild cognitive deficit among the aged living in
> community. The findings thus add reassuring information to other
> recent studies regarding the limited evidence of non-response rates
> among the old old. It thus appears that concerns about conducting
> reliable and valid social research among the old old may have been
> exaggerated.

3.  **Cohen, G. and Duffy, J.C.: *Are non-respondents to health surveys
    less healthy than respondents?* Journal of Official Statistics,
    Vol. 18. No. 1, 2002. Statistics Sweden \[35\].**

> This is a study comparing the results of a health survey and a
> follow-up three years later with mortality registers five years after
> the first survey for respondents and non-respondents. Based on the
> idea that mortality is an indicator of health at the time of the
> survey, such comparison can inform estimates of the non-response bias
> on self-reported health variables.
>
> The study was based on a 1993 postal survey of an age-stratified
> sample of 4,869 people aged 55 and over, drawn from the Community
> Health Index in Lothian Region, Scotland. The follow-up, in 1996, was
> carried out of all 3,515 respondents to the first survey who were not
> known to have died or migrated.
>
> Self-reported health was measured in the first survey using the 1991
> census question on limiting long-term illness and the eight dimensions
> of the short form SF-36. Logistic regression was used to examine the
> association between non-response to the second survey and
> socio-economic and health variables measured at the first survey; and
> to analyse the mortality status at mid-1998 of respondents to the
> first survey in terms of health status in the first survey and
> response status to the follow-up.
>
> Results show that within each age-sex decennial group (up to 85+),
> except for the youngest males, non-responders had higher mortality
> rates than responders, the discrepancy increasing markedly for males
> over age 75 and females over age 65.
>
> To examine the influence of health and social background variables on
> the likelihood of non-response, a logistic regression was carried out.
> It was found a health effect, after adjusting for other significant
> variables (age group, home tenure and income support status). Those at
> the lower quartile on the physical functioning scale gave estimated
> odds of non-response 29% higher than those at the upper quartile.
> Regarding the mental health scale, the effect was of borderline
> significance, but still with 17% higher non-response rate for the
> lower quartile group.
>
> After controlling for self-reported status at first survey,
> non-responders to the follow-up showed higher mortality than
> responders. Both physical functioning and limiting long-term illness
> had independent highly significant effects.
>
> Among 1993 respondents non-response to 1996 follow-up was associated
> with approximately twice the risk of mortality in the next 18 months
> as compared to respondents to the follow-up.
>
> In conclusion, mortality up to mid-1998 was found to be significantly
> higher among non-respondents to the first survey. Non-response to the
> follow-up was associated with worse self-reported health three years
> earlier as well as with indicators of socio-economic status.

**Other related studies**

-   A study investigated to what extent differences in health status
    between respondents and dropouts affected the associations between
    cardiovascular diseases and functional status and self-rated health
    in a population-based longitudinal health survey in elderly men. The
    findings showed that the health of non-respondents was worse than
    that of respondents in terms of stroke, BADL and mobility
    disabilities and self-rated health. The authors conclude that
    selective non-response might lead to bias in the prevalence of
    disease, disabilities and self-rated health as well as in the
    associations between disease and functional status and self-rated
    health \[36\].

-   A research aimed at investigating the pattern of age specific
    non-response bias in a two phase survey of disablement in the
    community developed a study of the timing of response to a postal
    questionnaire that showed that patterns differed for different age
    groups. The estimated response prevalence of those aged 65 years and
    over who were dependent was steady over time whereas for those in
    the 16-64 age range the estimated prevalence fell as the survey
    progressed, indicating a tendency for those who were dependent to
    respond sooner. These findings raise questions about how different
    patterns of response might be indicative of bias which could
    differentially affect final age specific prevalence estimates
    \[37\].

-   Another study wished to describe non-responders and reasons for
    non-response and determine the magnitude and direction of
    non-response bias in connection with a postal questionnaire study of
    respiratory symptoms and disease among the elderly. The response
    rates were highest for the age group 70-74 years (76.6%) and fell
    dramatically with age (27.1% in age group 95+years). Men responded
    (63.7%) better than women (52.0%) and this was consistent at every
    age level. Late and non-responders had slightly lower frequencies of
    respiratory symptoms but higher frequencies of current-smoker
    status. All found, differences between responders and non-responders
    were small and would have only minimal effects on the final
    prevalence estimates for respiratory symptoms and disease. In
    comparing the results based on initial responders to results based
    on all responders, associations between predictor variables and
    outcome variables were essentially the same, showing that a reminder
    had little effect on the results of the study \[38\].

> **Reference list**

1.  Moore, J.C.: *Self/proxy response status and survey response
    quality*. Journal of Official Statistics. Vol.4, No.2, 1988, pp.
    155-172. Statistics Sweden.

2.  Lee S., Mathiowetz NA. and Tourangeau R.: *Perceptions of
    disability: the effect of self- and proxy response*. Journal of
    Official Statistics, Vol. 20, No. 4, 2004. Statistics Sweden.

3.  Shields M.: *Proxy reporting of health information*. Statistics
    Canada, Catalogue 82-003. Health Reports, Vol. 15, No. 3. May 2004.

4.  Parsons JA., Baum S. and Johnson TP.: *Inclusion of disabled
    populations in social surveys: review and recommendations*. Survey
    Reasearch Laboratory, December 2000.

5.  Langlet ER.: *Estimation of proxy, time and content effect for the
    Health and Activity Limitation Survey*. Proceedings of the Survey
    Research Methods Section, American Statistical Association (2000).

6.  Todorov, A. *Cognitive procedures for correcting proxy-response
    biases in surveys*. Appl. Cognit. Psychol. 17: 215-224 (2003).

7.  Todorov, A. and Kirchner, C.: *Bias in proxies\' reports of
    disability: data from the National Health Interview Survey on
    disability*. American Journal of Public Health. 2000 Aug; 90(8):
    1248-53.

8.  Wu AW, Jacobson DL, Berzon RA, Revicki DA, van der Horst C,
    Fichtenbaum CJ, Saag MS, Lynn L, Hardy D, Feinberg J. *The effect of
    mode of administration on medical outcomes study health ratings and
    EuroQol scores in AIDS*. Qual Life Res. 1997 Jan;6(1):3-10.

9.  Ball, A.E., Russell, E.M., Seymour, D.G., Primrose, W.R., Garratt,
    A.M.: *Problems in using health survey questionnaires in older
    patients with physical disabilities. Can proxies be used to complete
    the SF-36?* Gerontology. 2001 Nov-Dec; 47(6): 334-40.

10. Grootendorst PV, Feeny DH, Furlong W.: *Does it matter whom and how
    you ask? Inter- and intra-rater agreement in the Ontario Health
    Survey*. J. Clin. Epidemiol. 1997 Feb; 50(2): 127-35

11. Tennant, A., Badley, E.M. and Sullivan, M.: *Investigating the proxy
    effect and the saliency principle in household based postal
    questionnaires*. J Epidemiol Community Health. 1991 Dec;45(4):312-6.

12. Bassett, S.S., Magaziner, J. and Hebel, J.R.: *Reliability of proxy
    response on mental health indices for aged, community-dwelling
    women*. Psychol Aging. 1990 Mar; 5(1): 127-32.

13. Magaziner J, Bassett SS, Hebel JR, Gruber-Baldini A.: *Use of
    proxies to measure health and functional status in epidemiologic
    studies of community-dwelling women aged 65 years and older.* Am J
    Epidemiol. 1996 Feb 1;143(3):283-92.

14. Magaziner, J., Simonsick, E.M., Kashner, T.M. and Hebel, J.R.:
    *Patient-proxy response comparability on measures of patient health
    and functional status*. J Clin Epidemiol. 1988; 41(11): 1065-74.

15. Kiyak HA, Teri L, Borson S.: *Physical and functional health
    assessment in normal aging and in Alzheimer\'s disease: self-reports
    vs. family reports.* Gerontologist. 1994 Jun;34(3):324-30.

16. Tabert, MH. et al: *Functional deficits in patients with mild
    cognitive impairment. Prediction of Alzheimer's Disease*. Journal of
    Neurologic Physical Therapy, 2002.

17. Bradburn NM.: *Discussion of papers on mode effects from the Seventh
    Conference on Health Survey Research Methods.* Cynamon ML and Kulka
    RA, eds. Hyattsville, MD: National Center for Health Statistics.
    2003.

18. CyBulski KA. and Ciemnecki AB.: *Interviewing populations with
    disabilities by telephone: survey design and operations.*
    Proceedings of the Survey Research Methods Section, American
    Statistical Association (2000).

19. Biemer PP.: *Non-response bias and measurement bias in a comparison
    of face to face and telephone interviewing*. Journal of Official
    Statistics, Vol. 17, No. 2, 2001, pp. 295-320. Statistics Sweden.

20. Maffeo C., Frey W. and Kalton G.: *Survey design and data collection
    issues in the Disability Evaluation Study.* Proceedings of the
    Survey Research Methods Section, American Statistical Association
    (2000).

21. St-Pierre M. and Béland Y.: *Mode effects in the Canadian Community
    Health Survey: a comparison of CAPI and CATI.* Statistics Canada.

22. Leeuw ED., Hox J., Kef S. and Van Hattum M.: *Overcoming the
    problems of special interviews on sensitive topics: computer
    assisted self-interviewing tailored for young children and
    adolescents.* 1997 Sawtooth Software Conference Proceedings.
    Pp.1-14. Sawtooth Software Inc.

23. Fowler FJ.: *How to conduct general population surveys in the 21^st^
    century.* Introduction to session 4 of the Eighth Conference on
    Health Survey Research Methods. Cohen SB and Lepkowski JM, eds.
    Hyattsville, MD: National Center for Health Statistics. 2004.

24. Kinne S. and Topolski TD.: *Inclusion of People With Disabilities in
    Telephone Health Surveillance Surveys.* American Journal of Public
    Health. March 2005, Vol 95, No. 3 512-517.

25. Evans M., Kessler D., Lewis G., Peters TJ. and Sharp D.: *Assessing
    mental health in primary care research using standardized scales:
    can it be carried out over the telephone?* Psychol Med. 2004 Jan;
    34(1): 157-62.

26. Perkins JJ. and Sanson-Fisher RW. : *An examination of self- and
    telephone-administered modes of administration for the Australian
    SF-36.* J Clin Epidemiol. 1998 Nov;51(11):969-73.

27. Lechner-Scott J., Kappos L., Hofman M., Polman CH., Ronner H.,
    Montalban X., Tintore M., Frontoni M., Buttinelli C., Amato MP.,
    Bartolozzi ML., Versavel M., Dahlke F., Kapp JF. and Gibberd R.:
    *Can the Expanded Disability Status Scale be assessed by telephone?*
    Mult Scler. 2003 Mar; 9(2): 154-9.

28. Fournier L., Lesage AD., Toupin J. and Cyr M.: *Telephone surveys as
    an alternative for estimating prevalence of mental disorders and
    service utilization: a Montreal catchment area study*. Can J
    Psychiatry. 1997 Sep;42(7):737-43.

29. Hebert R., Bravo G., Korner-Bitensky N. and Voyer, L.: *Refusal and
    information bias associated with postal questionnaires and
    face-to-face interviews in very elderly subjects.* J Clin Epidemiol.
    1996 Mar; 49(3): 373-81.

30. McHorney CA., Kosinski M., Ware JE. Jr.: *Comparisons of the costs
    and quality of norms for the SF-36 health survey collected by mail
    versus telephone interview: results from a national survey*. Med
    Care. 1994 Jun;32(6):551-67.

31. Korner-Bitensky N., Wood-Dauphinee S., Siemiatycki J., Shapiro S.
    and Becker R.: *Health-related information postdischarge: telephone
    versus face-to-face interviewing*. Arch Phys Med Rehabil. 1994
    Dec;75(12):1287-96.

32. Fournier L. and Kovess V.: *A comparison of mail and telephone
    interview strategies for mental health surveys*. Can J Psychiatry.
    1993 Oct; 38(8): 525-33.

33. Hendershot GE, Colpe LJ, Hunt PC: *Persons with disabilities:
    non-response and proxy response in the National Health Interview
    Survey on Disability.* International Conference on Survey
    Non-response. Portland OR, 28-31 October 1999.

34. Kahana B. et al.: *Mild cognitive impairment and accuracy of survey
    responses of the old old.* Health Survey Research Methods
    Conference, 1996.

35. Cohen, G. and Duffy, J.C.: *Are non-respondents to health surveys
    less healthy than respondents?* Journal of Official Statistics,
    Vol. 18. No. 1, 2002. Statistics Sweden.

36. Hoeymans N., Feskens EJ., Van Den Bos GA. and Kromhout D.:
    *Non-response bias in a study of cardiovascular diseases, functional
    status and self-rated health among elderly men*. Age and Ageing, Vol
    27, 35-40, 1998.

37. Tennant, A. and Badley, E.M.: *Investigating non-response bias in a
    survey of disablement in the community: implications for survey
    methodology*. J Epidemiol Community Health. 1991 Sep; 45(3): 247-50.

38. Hardie, J.A., Bakke, P.S. and Morkve, O.: *Non-response bias in a
    postal questionnaire survey on respiratory health in the old and
    very old.* Scand J Public Health. 2003; 31(6): 411-7.