Extended Questions Sets: Purpose, Characteristics and Topic Areas

Barbara M. Altman

Emmanuelle Cambois

Jean-Marie Robine

The initial activity of the Washington Group was relatively simple and
straight forward. The development of an internationally comparable set
of questions for use in Censuses provided a natural set of limits to the
work. Censuses have limited space for questions so a minimum number of
questions could be considered. The challenge was first to decide on the
purpose of census questions on disability -- purposes of questions being
the guiding principle for selecting the conceptual elements of the
disablement process to be operationalized. The second challenge was to
identify domains relative to the agreed upon purpose and within those
domains the conceptual components that would best represent a broad
range of the population. As members of the WG know, the purpose was
established, and questions developed that are now being tested in a
variety of venues.

The next work focus for the Washington Group, to develop extended sets
of questions which are also culturally comparable, to be used in
surveys, censuses and other more varied venues, is a good deal more
complicated. There are a number of steps to consider including:

1.  **[Develop an organized process to create, test and approve extended
    sets]{.underline}**

2.  **[Select the purpose/s to be addressed and translated into
    content]{.underline}**

3.  **[Establish the characteristics of each of the extended
    set/s]{.underline}**

4.  **[Review existing measurement instruments relating to the defined
    objective and the literature describing the limits and robustness of
    these instruments]{.underline}**

**5.   [Review existing measurement instruments relating to the defined
objective and the literature describing the limits and robustness of
these instruments]{.underline}**

Furthermore, a key issue in this process of developing extended question
set/s is to make sure that the final proposal would be attractive enough
to be accepted by public health policy and survey makers. Past
experiences related to "official" recommendations provide good advice on
how to avoid failures.

*"Several years ago, the Regional WHO Office for Europe, as a part of
Health for All, recommended common health instruments that should be
included in European Health Interview Surveys. Most European countries
administer regular health interview surveys to monitor population
health. However the longest established surveys, such as the United
Kingdom General Household Survey, began before the current desire to
harmonize health information within the European Union, and, as a
result:*

-   *countries with the longest experience tend to be the most reluctant
    to implement the recommended instruments;*

-   *the relevance of previously recommended instruments was not always
    obvious to policy-makers who did however know the utility of their
    own national instruments;*

-   *instruments were rarely accompanied by recommendations on the
    specific study designs to contain them, thus producing a further
    obstacle for comparability of the collected information;*

-   *countries were not made aware of the implications when they amended
    the instruments (through question wording, selection of items,
    change in response categories)."*

The process to reach a consensual proposal for "recommended extended
instrument/s" should address these different issues. This paper aims to
discuss some of these points and make recommendations for consideration
by Washington Group members.

# Development of an organized process to create, test and approve extended sets

The possibilities that present themselves when considering extended sets
of questions to provide additional information about disability in
surveys are potentially unlimited. In order to approach this task in an
organized way, to minimize duplication and to use our resources
efficiently, we first need to develop a plan as to how we are going to
go about this process. As indicated earlier, the census questions task
provided natural boundaries for that activity, but this next step has
the potential for unlimited boundaries and wasted energies.

A work plan for this process should include a call for the development
of a reasonable number of extended question sets as well as
specification of the topic areas. A format for division of labor among
the group members to create the question sets would be useful, would
allow persons with expertise in a particular topic area to contribute in
that area and would speed the work. If work groups were provided with
general guidelines for purpose, length and other characteristics of
extended sets, this would facilitate work and also result in extended
sets with uniform characteristics.

The following discussion sets out some of the decision points necessary
for providing those guidelines and when summarized at the end of this
paper should be considered a first draft of a work plan.

# The purposes to be addressed and translated into content

The three general purposes for disability data discussed in the position
paper prepared for the general set of disability questions for Censuses
identified purposes equally relevant for the extended sets of questions.
We identified three major classes of purposes for collecting data on
persons with disabilities at the aggregate level. These purposes are
complementary and partially or fully addressed by the general questions
or by extended sets of questions: 1) to provide services, including the
development of programs and polices for service provision and the
evaluation of these programs and services (assessing the need for
assistance, technical devises, pensions...), 2) to monitor trends in the
level of functioning in the population (assessing the functional health
status of the population) or the trends in participation (assessing the
proportions employed or homeowners) and 3) to assess equalization of
opportunities for participation (being able to compare common indicators
on employment, resources, schooling... among those with functional
problems and those without such problems). In addition another purpose
that was not included in our first conceptualizations could be
considered relevant to extended questions sets. That would be 4)
identification of contextual circumstances that contribute to the
disability outcomes we observe. This purpose would be to understand the
factors in the environment that contribute to the lack of social
integration or participation that the person with the limitation in
functioning may experience. Rather than contributing to the
identification of the population of individuals who have risk factors
(such as functioning or task limitations) for participation limitations,
the purpose of collecting contextual information would be an analytic
one for explaining and understanding the disability process. This would
provide data to identify common barriers to full integration of people
with functional problems.

The development of extended measurement sets can be directly related to
the three original purposes. The ['equalization of
opportunity']{.underline} purpose implies, from the original position
paper, the full elaboration of all functioning limitations to identify
those more at risk of restricted participation. Equalization of
opportunity, the purpose behind the development of the general measures,
continues to be a leading concern for the use of the data to be
generated. It is associated with measurement of [basic actions or
functioning]{.underline} . The focus of a set of extended questions to
satisfy this purpose could take [two or more]{.underline} approaches.
The first approach would be to extend the number of domains covered in
order to fill in information about the functional health of the
population. In addition to the domains of sensory and physical
functioning, the domains of cognition and mental functioning would be
added. Subgroups with cognitive limitations or problems related to
psychological adjustment are equally at risk of facing barriers to
participation as those with limitations in walking or seeing, but are
not well covered in general questions. Another approach to extended
questions for the purpose of equalization of opportunity would be to
focus on a specific domain through the collection of more in depth
information . As an example, using sensory limitations , providing
detailed questions about seeing from a distance or up close, having
blindness in one eye, limits in peripheral vision, etc.

In addition to the objective of equalization of opportunity, there are
still very important [service needs]{.underline} facing governments that
collect this data and use it to make policy decisions. The purpose of
service provision addresses more specific, but important sub-populations
among those with disabilities. This topic has become a major public
health concern in recent years due to the lengthening of life expectancy
and the need for assistance in daily life for a part of the elderly
population. It is also of concern for the youngest population among whom
new conditions are being identified that require adaptation of education
programs. The data are necessary for the development and evaluation of
government programs. Extended sets to satisfy this purpose might be
directed toward specific age groups, such as children or elderly, or
towards persons with specific conditions such as the population with
mental health problems or wheel-chair users. Data necessary to
understand the need for services could be concerned with the limitations
with basic functioning that individuals experience in relation to the
need and the use of technical aids or assistive devices. The extended
set of question to assess the functional health status of the population
can contribute partly to this objective if questions are properly
formulated: for instance existing proposals allow us to distinguish
between the functional limitations experienced if people do not use any
aids or residual limitation considering the use of such aids. A more
specific way to contribute to this objective is to propose sets of
questions, as it is done in disability surveys, on the met and unmet
needs of the person in order to directly assess the level of assistance
used and needed.

The purpose of [trend monitoring]{.underline}, can relate to either of
the other two purposes addressed above, service needs or equalization of
opportunity. However, a primary use of trend monitoring ,, as commonly
used in developed countries today, is following the rates of
participation in social role behavior of persons with limitations in
functioning or basic actions. Trend monitoring of participation would
require more detailed questions about the types of activities available
to the population in general including school, work, leisure activities,
church participation, and participation in civic responsibilities.
Measurement for this purpose would be very difficult to keep
cross-culturally neutral since individual cultures can have very
different expectations for these kinds of behaviors.

For the purpose of documenting the environmental conditions in which a
person with a functional limitation is attempting to participate,
information about the individual's contextual situation needs to be
collected. This purpose is related to understanding the social milieu in
which the individual lives and how that milieu either facilitates or
impedes their full participation. Since all activity takes place within
the individual's physical, social and cultural context, it is essential
to have this information as well to understand the outcomes that occur.
In the past this part of the equation has been ignored, but new models
such as the World Health Organization's ICF Model and the model
developed in the U.S. by the Institute on Medicine, have recognized the
centrality of this element to the full picture of disability in any
culture. There are fewer questions sets for this purpose presently
developed and those that exist have been used for a relatively short
time.

Trend monitoring could also contribute to documenting the identification
of [contextual circumstances]{.underline} that contribute to the
disability outcomes. To do so, it should include a set of questions
about the environmental factors that would ease or make difficult
participation as describe above (school, work,...). Indeed, to plan for
improved environment to equalize opportunity, it is necessary to
identify what facilitates or prevents full participation for people with
impairment and functional limitations.

As discussed above, various purposes appear to be associated with one or
two of the levels of measurement defined in the first position paper and
so this could be a guiding factor for the development of the various
extended sets to satisfy those purposes.

# Establishing characteristics of extended questions set/s

There are many aspects of the questions sets that need to be considered
before jumping into creation of the questions. Among them, we thing the
WG should emphasis discussion on the 6 following issues**.**

> **a. Number of questions sets to be considered:** As described above,
> several different extended sets of question could be developed to meet
> different data needs. On the one hand, we cannot propose extended sets
> indefinitely, but on the other hand, we might want to consider
> providing at least one set to satisfy the four basic purposes
> discussed in the introduction. For the first set, the WG has already
> agreed to implement an equalization of opportunity purpose, which
> implies further identification of persons with limitations in basic
> actions necessary to accomplish basic activities and forms of
> participation. The immediate issue for this first set is to develop
> the characteristics of this extended set.
>
> In addition, we would propose that the group consider providing three
> more extended sets of questions to satisfy the objectives concerning
> the assessment of [services needs]{.underline}, of [trend in
> participation]{.underline} and of [environmental context]{.underline}.
>
> **b. Homogeneity or diversity:** Homogeneity or diversity of each
> extended set as well as level of specificity should be established
> After establishing the purpose of the extended set, thought must be
> given to the nature of the relationship of the questions. In some
> instances the extended set will want to capture as broad a range of
> coverage as possible, in other situations we might prefer to have in
> depth information about one type of limitation. An example for the
> purpose of equalization of opportunity would be to identify as many
> different kinds of domains of functional limitation one can in the
> space allowed. That may mean capturing a variety of physical and
> mental functioning problems, or alternatively from an in depth
> approach capturing all the possible cognitive and affective
> limitations associated with mental health problems.
>
> From a service provision purpose this diverse approach would be
> concerned with establishing need for assistive devices, along with
> limitations in task activities associated with self-care along with
> task activities associated with maintaining an independent life style.
> On the other hand, it could be more efficient to keep homogeneity
> within the service need set to be able to fully understand the exact
> limits in self care a respondent is experiencing.
>
> Diversity or homogeneity of questions for purpose of establishing
> trends is less important. What is important is that there be
> consistency in using the same questions over time. Detailed sets of
> question sometimes are build as a whole which cannot be split or
> changed in any way. It should also be taken into account when
> developing or adopting sets already in use that the number, order or
> wording of the questions cannot be changed without compromising the
> comparability over surveys or over time.
>
> However, diversity or homogeneity is again an issue for questions sets
> for the fourth purpose, environment. Question sets can be framed that
> will capture the experience of the environment in the total community
> and would include information on access/problems with transportation,
> buildings, local attitudes, local geography and a variety of other
> environmental factors that can impede or facilitate participation. Or
> a more homogeneous approach would seek to locate and describe all of
> the elements that either facilitate or impede an individual with a
> disability in the work environment.
>
> **c. Size of extended question set/s:** We cannot be frivolous with
> the entre we may have available to put these questions in surveys in a
> variety of countries. Since nations will be using our material
> voluntarily it is common courtesy to keep the data sets relatively
> small and specific to the purpose. A ten minute focused unit which can
> be included in a survey at various regular intervals (to provide trend
> data) will probably see much more use and acceptance than a 25 minute
> element that seeks to address all the issues. For that reason we would
> like to suggest to the group that the 4 proposed extended sets be
> limited to 10 or 15 minutes (at the most) that can be inserted in
> national data collection projects at regular intervals.
>
> **d. Sequencing or prioritizing of the extended sets for
> implementation and testing:** Depending if the proposal to develop 4
> extended question sets is accepted and how we organize that process,
> we need to prioritize the order in which sets will be developed and
> tested. It is probably not a good idea to work up and test the
> questions sets all at the same time to allow for the mistakes of
> working on the earlier groups to inform the latter groups. It would be
> appropriate to work out a proposed timeline for the question
> development and testing that also took into consideration developing
> grants to fund the projects, particularly the testing.
>
> **e. The development of question orientation, and wording, including
> answer categories.** The development of the orientation to the
> questions will influence the data that results. Careful thought must
> be given to that orientation to the questions and to the answer
> categories. With the equalization of opportunity extended set that
> focuses on functioning in basic actions, the issue of capacity *vs*
> performance is an important one for consideration. Also the range of
> functioning and the reasons for that particular range to be covered
> needs be explained. Question orientation related to the service needs
> purpose may want to consider an approach that will not be influenced
> by the differences in resources or technological sophistication of the
> countries. For the environmental purpose, this is particularly
> difficult problem because of the international differences in weather,
> topography, availability of water and many other factors.
>
> **f. The cultural compatibility of the conceptual elements chosen to
> be developed into questions.** This aspect will become all the more
> difficult as the questions sets about more complex activities are
> developed. The more complex the activity, the more likely that
> culturally specific ways have been developed to deal with it that
> differ across the international community. This issue imposes the need
> for reviews of the methodological literature which debate the
> different arguments related to wording selection, selection of items
> and questions and answer categories.

# The continued emphasis on international comparability with consideration for flexibility and special needs 

Along with the comparability issue that becomes much more difficult with
the increased complexity of the activity being measured, we must also
consider the need for special extended sets for particular
circumstances. As Davidson indicated last year in Bangkok in her section
of the position paper, a certain group of countries are experiencing the
effects of civil war, terrorism and other forms of violent action or the
results of earlier violence or wars. It is understandable that these
countries would find the general extended sets not adequate to serve
their needs. Do we want to consider some specialized extended sets or
components of extended sets to meet these data needs, or is that beyond
the responsibilities of this group.

Aspects of participation and environments can be so different between
national areas that we may want to consider adjusting the extended sets
in these area to account for at least some of the regional differences
that can be found between, for example, Europe and Africa. How much and
what kind of flexibility can we consider and still feel we are providing
appropriate extended sets to facilitate international comparison of the
data? At this point we don't have a recommendation , but feel it is
something that needs to be considered as the work moves forward.

# Census of existing instruments and literature is an important issue for several reason: 

-   Avoiding reinventing the wheel and keep close to already existing
    things

-   Benefiting from previous research on how to measure things

-   Ensuring that you do not impose new instruments in countries where
    valuable instruments are already in use.

The European experience so far shows that countries are more attracted
by the idea of using readily and comparable "modules of questions" taken
"off the shelves" to be implemented in their own survey whenever they
need comparable information on such aspects. For the time being four
different modules are proposed for use in that way This system allows
countries to simultaneously develop parts in the questionnaire devoted
to specific national interests and to insert the international modules
in some form of regularized pattern. In France, plans are to have a
health survey based on European modules every five years, complemented
alternatively with specific modules on handicap (barriers, met and unmet
needs...) and specific modules on health consumption (medicines,
care...).

[TO SUMMARIZE:]{.underline}

We suggest developing four extended sets of question in the framework of
the following selected purpose:

-   Assessing Equalization of opportunity

-   Assessing services needs

-   Assessing environmental context contributing to full/restraint
    social participation

-   Monitoring trends in participation

To facilitate one or several of these purposes, we suggest research on
the following type of survey instruments:

-   Functional health status: impairments, functional limitations,
    activity restriction

-   Use, need, satisfaction with technical devices and or human
    assistance financial support

-   Social participation at school, at work, at home, for leisure...

-   Environmental barriers or facilitators

Research on existing instruments will shorten our work, but selection
should be based on assessing measurement quality through literature and/
or test-Finding sets that are related to our purposes and meet our
requirements should not be difficult for the purposes of equalization of
opportunity or identifying service needs. However, there has been less
extensive development of instruments about full participation or
environmental context, so the latter two purposes may require further
work.