metadata: authors: 0: author: Harpham, Trudy
1: author: Timaeus, Ian
docna: 0: docna: Objectives and methods of a world health survey
docty: Policy Research Working Paper
subtopic: Health Care Services Industry,Health Service Management and Delivery,Nutrition,Law and Justice Institutions,Global Environment
teratopic: Gender
abstracts: cdata!: Many developing countries are trying to
            improve the routine collection of health information by
            strengthening surveys, censuses, and registration systems.
            At the international level, too, efforts are underway to
            provide information on health and health interventions,
            including statistical reporting programs of the U.N. and the
            World Bank. In view of the limited financial resources in
            the developing countries, would a world health survey
            complement these health information systems and contribute
            to long-term health care? This paper finds that although a
            series of coordinated country health studies could be
            valuable, there are many tradeoffs. Considering the variety
            of health problems and priorities in developing countries,
            it is probably more important to develop the expertise to
            conduct and analyze health studies than to devise a standard
            questionnaire to collect health data. As for the cost
            effectiveness of health programs, a world health survey is
            not the appropriate vehicle for such evaluations, but it
            could address such concerns as access, coverage, patient
            costs, and financing systems.
repnme: repnme: Objectives and methods of a world health survey
display_title: Objectives and methods of a world
            health survey
listing_relative_url: /research/1988/06/700705/objectives-methods-world-health-survey
topicv3: Nutrition m1326779 644,Health Services Delivery m1327294 1305,Environment, Natural Resources and Blue
            Economy m1327391 566,Mortality m1504365 3661,Health m1504661 3480
publishtoextweb_dt: 2010-07-01T00:00:00Z
historic_topic: Gender
origu: Policy Plan & Prog Review Dept
no_of_pages: 51
model_extractions: <!-- Start of picture text --> Poly bning, and Re"srch |<br>Poputldon, Health, and Nutrition<br><!-- End of picture text -->

Many developing countries are trying to improve the routine collection of health informiation by strengthening surveys, censuses, and registration systems. At the international level, too, efforts are under way to provide information on health and health interventions, including statistical reporting programs of the U.N. and the World Bank.

In view of the limited financial resources in the developing countries, would a world health survey complement these health infonmation systems and contribute to long-term health care? Is it reasonable to expect that such a survey could identify the pattems and causes of disease and at the same time measure the effectiveness of investments in health?

tradeoffs. Considering the variety of health problems and priorities in developing countries, it is probably more important to develop the expertise to conduct and analyze health studies than to devise a standard questionnaire to collect health &ktta. As for the cost-effectiveness of health programs, a world health survey is not the appropriate vehicle for such evaluations, but it can address such concems as access, coverage, patient costs, and financing systems.

This paper is a product of the Population, Health, and Nutrition Division, Population and Human Resources Department. Copies are available free from the World Bank, 1818 H Street NW, Washington DC 20433. Please contact Anelia Menciano, room S6278, extension 33612.

Although a series of coordinated country health studies could be valuable, there are many The PPR Working Paper Series disseminates the findings of work under way in the Bank's Policy, Planning, and Research Complex. An objective of the series is to get these findings out quickly, even if presentations are less than fully polished. The findings, interpretations, and conclusions in these papers do not necessarily represent official policy of the Bank.

Copyright i 1988 by the International Bank for Reconstruction and Development/The World Bank
|CONTEN|TS|PAGE|
|---|---|---|
|SUM|MARY|3|
|**1.**<br>INT|RODUCTION|5|
|2.<br>INV|ESTIGATION OF PATTERNS AND DETERMINANTS<br>OF HEALTH|E|
|2.1|Existing health information<br>systems in developing<br>|7|
||countries||
|2.2<br>|TheUseof surveys to investigate<br>patterns of health<br> <br> <br>|8<br>|
|2.3|Integration<br>of survey investigations<br>into existing<br>ttitil<br>t|10|
||sasca<br>sysems||
|2.4|The design of country health studies|11|
|2.5|Investigations<br>of the determinants<br>of health|13|
|3. MEA|SURING THE IMPACT OF INVESTMENTS<br>IN HEALTH|14|
|3.1|Economic evaluation|14|
|3.2|Costs|15|
|3.3<br>|Consequences<br>|16<br>|
|3.4|General problems of economic evaluation|20|
|3.5|Economic evaluation and the World Health Survey|21|
|4. SC|OPE AND LIMITATIONS<br>OF INTERVIEW-BASED<br>SURVEYS|23|
|4.1|Health interview<br>surveys- an introduction|23|
|4.2|Mortality- well established<br>methods|24|
|4.3|General morbidity, cause-specific<br>morbidity and<br>|26|
||use of services||
|4.4|Nutritional status indicators|31|
|4.5|Environmental<br>contamination|33|
|4.6|Explanatory<br>variables|33|
|4.7<br>|Comunity variables<br>|34<br>|
|4.8|Summary of core questionnaire<br>and modules|35|
|5. DE|SIGN AND ORGANISATIONAL<br>ISSUES|35|
|5.1|Selection of countries for the WHS|35|
|5.2<br>|Review of the existing health information<br>system<br>|37<br>|
|5.3|Sample sizes and design|38<br>|
|5.4|Repeat surveys|39<br>|
|5.5|In-depth studies|39|
|5.6<br>|Questionnaire<br>development and pre-testing<br>|40<br>|
|5.7|Piloting and training|41|
|<br>5.8|<br>Seasonality<br>and fieldwork operations|41|
|5.9|Validation and examinations|42|
|5.10|Data processing|42|
|5.11|Strategies for analysis|43|
|6. CO|NCLUSIONS|45|
|RE|FERENCES|47|

# REFERENCES

Trudy Harpham and Ian Timaeus, both with the Working Group on Health Impact Assessment, London School of Hygiene and Tropical Medicine, received contributions from David Bradley, Sandy Cairncross, Lucy Gilson, Wendy Graham, Allan Hill, Philip Payne, Max Price, David Ross and Andrew Tomkins.
# SUIKARY

This paper discusses the objectives and design of a proposed World Health Survey (WHS). Particular attention is given to two objectives which the World Bank identified for a WHS, namely, to ascertain patterns and determinants of disease and to develop a methodology for impact assessment.

Many developing countries are trying to improve existing health information systems. Vital registration systems are being strensthened and censuses and surveys used to improve knowledge about mortality. Routine health services reporting is being complemented by population based primary health care reporting networks. An international programme of health surveys must complement and not threaten these existing statistical systems. It should also take into account other international programes such as the Demographic and Health Surveys and the U.N. Household Survey Capability Programme health surveys.

The main attractions of survey-based investigations of health are that they could provide population-based and person-based estimates of the extent of ill-health. The variety of health problems and priorities in developing countries suggests that there is little scope for a standardised questionnaire to be used across countries. Instead the emphasis should be on providing a pool of expertise on the conduct and analysis of studies of health. The design of each country health study should be based on a review of existing data and statistical systems. However, a single-round, retrospective health survey would in many cases form an appropriate focus for the study. It could consist of a few core questions complemented by country-specific modules. The core questionnaire might include questions about mortality, general morbidity during the previous two weeks and action taken as a result, chronic ill-heaith among adults and anthropometry of pre-school children. This would be irn addition to questions on explanatory variables and the collection of community-level data. The 3
collection of information about specific health probiess, about coverage and use of particular health programmes should be tailored to country circumstances. In-depth studies of issues of importance to particular countries are also needed.

In turning to the second objective of 'impact assessment' the economic evaluation techniques of cost-benefit analysis and cost-effectiveness analysis are considered. Problems in measuring health outcomes and valuation continue to limit the application of these methods and a WHS would not be a suitable vehicle for such evaluations. On the other hand there are a range of health service concerns that can be more easily addressed, for example, access and coverage, patient costs, financing systems etc. Data related to these issues Ald be included as part of the WHS and specific suggestions are made.

Particular issues concerning the design and organisa of a WHS are considered including the selection of countries, -iaple sizes, questionnaire development, data processing and strategies for analysis.

It is concluded that a series of coordinated country health studies producing population-based estimates of health and its determinants would have a valuable role in improving the health information systems of some countries.

.4
# 1.INTRODUCTION

This paper discusses the objectives and design of a proposed World Health Survey (WHS). It was prepared at the request of World Bank as a background document for a meeting of staff of the World Bank, World Health Organisation (WHO) and invited experts to discuss the possibility of a WHS.

The terms of reference (TOR) for this paper defined the WHS as a programme to sponsor studies of health in a broad range of developing countries. They identified a need for an information system that could measure the impact of public health programmes and health services and also improve overall allocation and targeting of health interventions to specific population groups. Three specific objectives for the proposed WHS were identified: (1) To improve knowledge of patterns and determinants of health, disease and mortality in developing countries; (2) To develop appropriate methodology for measuring the impact on health status of investments in health as well as other sectors; (3) Tc conduct appropriately-designed surveys in several countries (a minimum of 5 and maximum of 20) over a 5 year period.

The TOR specified that the paper should provide a rationale for a WHS and discuss methodological options for country health studies. It was emphasised that the paper should discuss possible approaches to a WHS rather than presenting a detailed proposal for the implementation of any single study design.

Consideration of the TOR has led us to identify a number of additional issues which need to be taken into account both in deciding whether to conduct a WHS and in determining the design of the country health studies: Firstly, to what extent is it reasonable to expect that the two primary objectives stipulated, namely (1) ascertaining Patterns and deterinants of disease and (2) demonstrating methodology for impact 5
assessment, could be achieved by the same means?

Secondly, given their differing perspectives, can<sup>the needs of</sup> developing country institutions, donor organisations and the research community all be identified and accommodated within the same study design? It will be argued by the research community<sup>that the</sup> muitifactorial nature Gf many health problems in the developing world, particularly those associated with poverty, means<sup>that better</sup> knowledge of determinants can only come irom research aimed at elucidating the processes whereby environmental, behavioural and economic factors interact to cause disease and mortality: to the extent that this view is correct, it would seem that a<sup>survey approach</sup> is either inappropriate, or that at least it should be regarded as only one part of a more extensive strategy aimed at understanding mecharisms and processes. Consideration of the requirements of the other users, points to the need to decide on the emphasis to<sup>be placed</sup> on achieving internationally comparable data on health, particularly for those indicators which are considered to be useful for advocacy, or for allocation of aid resources between countries. This might conflict with the need of countries themselves for information which could improve their own policy-making capacity, or<sup>assist in the</sup> management of their own health infrastructure.

Finally, however these issues are resolved, there would need to be consideration of the relationship between any proposed strategy for a WHS, and existing international and national initiatives providing information on health and on health interventions, in particular:- surveys such as the Demographic and Health Survey (DHS), UN Household Survey Capabi'.ity Programme (UNHSCP) and World Bank's Living Standards Measurement Study (LSMS) - internatLonal and national initiatives to develop and strengthen routine and continuous collection and analysis of informatior. on health (eg development of primary health cara (PHC) reporting, use of sentinel monitoring, nutrition surveillance systems etc.) - methodologies and studies designed to measure the impact<sup>of</sup> 6
investments in health in the context of intervention projects (es EPI evaluation, case-control studies of diarrhoeal disease control measures etc.).

# 2. INVESTIGATION OF PATTERNS AND DETERMINANTS OF HEALTH

# 2.1 Existing health ipXz ation svstcms in developing countries

The first objective of the proposed WHS is to 'improve knowledge of patterns and determinants of health, disease and mortality in developing countries.' Developing countries are well aware of their lack of appropriate quantitative measures for planning, monitoring and evaluation in the health area. Many have been making strenuous activities to improve knowledge of mortality levels and differentials with additional questions in censuses, post enumeration surveys or special household studies. In another direction, most are striving to improve the accuracy and coverage of their vital registration siz*+ems. These two developments have in most cases taken place independently from the health sector with consequent loss of efficiency and of the gains which would certainly flow from comparison and combination of information from different sources.

A second area of activity is the improvement of routine health services reporting. This integrates the provision of information with the provision of services. As a result patterns and trends in health can readily be linked to input and output evaluations of the operational efficiency of programmes. With appropriate information systems, findings can readily be fed back to staff involved in providing health services and in collecting data. However such reporting systems have inherent disadvantages. Most notably they only provide information on that part of the population that is covered by and uses the health services. The poorer, rural and less educated sections of the population tend to be excluded from such statistics. By definition, they cannot measure unmet needs for health care. Secondly service-based reporting systems provide information on case 7
episodes as the unit of observation rather than the individual. The resulting statistics on disease incidence or prevalence may have misleading policy implications to the extent that they fail to reveal how far disease is concentrated in a small part of the population with multiple health problems.

An important initiative in many developing countries is the setting up of PHC reporting networks. These have the potential to provide a continuous source if community-based data on health. In so far as PHC systems succeed in extending the provision of health care beyond clinics and other facilities to the whole of the population, infvrnation can be collected that more closely reflects the health status of the community. However, for the full potential of such systems to be realised, greater efforts are needed to improve and standardise the information gathered and to assess possible selection biases that affect these data.

# 2.2 The use of surveys to investigate Patterns of health

The extent to which an international programme of surveys could complement existing health information systems needs to be assessed in terms of its capacity to improve the setting of priorities and the planning, implementation and evaluation of projects and programmes. Moreover in planning for a WHS it is important to consider the opportunity cost of conducting surveys in terms of the limited capacity of developing country statistical organisations as well as in financial terms. Population surveys should only be considered when they can contribute to the long-term development of surveillance of the population's health status and of the effectiveness of health care. In some countries it would be preferable to concentrate resources on the development of simple, low-cost community-based reporting systems that national institutions can maintain and fund themselves.

The brief review of alternative statistical systems in section 2.1 8
suggests two main reasons why survey-based investigations of health may be of value in some countries. Fir-t they could provide information on health status and service use for the whole population. Most countries collect some information on mortality of this type from existing sources. Fewer have reliable data or. other aspects of health. Second they could provide person-based rather than episodebased estimates of the extent of ill-health. Such information would be valuable in identifyins demographic and socio-economic differentials in health and thus to identify priorities based on need and to target future interventions. Data could be collected on the coverage and use of health facilities and prograines to identify groups of the population who lack adequate provision. By relating measures of health status to data on service use, survey-based estimates could be used to identify unmet needs and to clarify the biases affecting statistics derived from facility and PHC-based reporting systems. In addition, because survey investigations are independent of the provision of services they are particularly useful for examination of attitudes to and satisfaction with the services available.

Not every country will perceive the need for such a survey. M4any previous surveys have been slow to produce results and, when they have appeared, have not addressed the questions that policy makers need answering. Administrators engulfed with routine work may be unable to take a more holistic view of problems and priorities. In section 4 we describe the methodologies that are emerging for collecting useful information by means of health interview surveys. Nevertheless there are severe limitations to what can be learnt from them. In particular, respondents' assessments of their own health should not be expected to provide information comparable to that obtained from medical examinations. Even in developed countries, population surveys have tended to concentrate on examining ill-health in relation to restrictions on activity and patterns of service use.

Nevertheless many developing countries do see a role for national 9
household sample surveys as ona aspect of their health information systems and have conducted or are planning to conduct such an enquiry. Despite the example of the DHS and the efforts being made to improve methods by the UNHSCP, manj of these surveys are unsystematic, poorly planned and give insufficient attention to the developmnent and pretesting of questionnaires and procedures. The WHS could contribute to the development and application of more rigourous methodologies for investigating ill-health, the use of health services and factors associated with these.

# 2.3 Integration of survey investigations into existing statistical svstgms.

A more systematic approach to collecting mortality and morbidity insformation with a survey, perhaps repeated at regular intervals, could also be a useful way of demonstrating the links between the diverse sources of information on health which every country possesses. Part of the task of a WHS country study would be to demonstrate that the information to be collected can complement existing sources. In soma cases this might imply integration of the study with quite radical changes in the system of administration. For example, it may mean directing resources from the civil registration system towards more innovative services which could be better sources for the study of current morbidity and mortality trends and differentials. Where the stress is more on routinely-collected health statistics, the data from household surveys may be used to tegularly calibrate and adjust the other information. Moreover, bv stressing the need to identify trends in key indicators, particularly indicators of health status for whole communities, there is an opportunity to revise the information gathered by the reporting system itself. There is a good deal of iner,.a and redundancy in existing health information systems which tend to be multi-layered as a result of accretion over the years. Serious problems are being caused by the unintegrated, conflicting, duplicating and expensive (in time, labour and money) information-gathering activities already taking place in 10
many countries, especially in Africa. The situation has<sup>been</sup> aggravated by frequent revisions and introduction of new procedures. often being imposed on countries through demands for information from the international agencies. In the overall analysis of data from different sources, the value of a restricted number of indicators can be illustrated and some reforms instituted to ensure<sup>timely</sup> collection, analyses and use of these data in the future.

# 2.4 Th: desisn of country health studies

The information that is needed on patterns of health and service use in order to improve the provision of health care varies greatly between countries. Some health problems, such as infant diarrhoea, are major causes of sickness and death throughout the third world. Other diseases, such as malaria or neonatal tetanus, are of enormous significance in some countries but of little importance elsewhere. Moreover the importance of investigating any health problem depends not only on its impact on the population but also on the capacity of the health system to intervene effectively against it. Similarly, what information should be collected on the health services depends on their overall development, on the way in which health care<sup>is</sup> organised and the services that are provided and on the capacity of the system to respond to problems or issues that are revealed by new data.

Many of the specific issues of importance in particular countries can only be addressed by logistically-complex and expensive inves*igations. For example the assessment of the long-term implications of serious illness or the study of seasonal variations in nutritional status and disease incidence probably involve mounting follow up studies. Some health problems, such as parasitical infections or anaemia, can only be effectively investigated by means of medical examination and not by interview surveys. Such sophisticated, intrusive and expensive investigations can only be justified when the information that they will provide is of particular 11
importance.

The variety of health problems and priorities in developing countries suggests that there is little scope for surveys using standardised questionnaires and collecting a wide range of internationallycomparable information on patterns of disease and mortality. Instead the emphasis of the WHS should be placed on the development and dissemination of improved methodologies for conducting, processing and analysing health studies. Moreovex, it seems unlikely that detailed investigation of the informatio **n** eeds of a broad range of developing countries would justify a series of prospective surveys or surveys involving detailed medical examinations. We suggest that the focus of most country studies should consist of a single-round, retrospective health interview s. m. As we discuss in more detail in section 4, any core questionnaire should be relatively brief. It might include questions about mortality, sickness during the previous two weeks and actions taken as a result and perhaps about chronic ill-health among adults. Collection of anthropometric measures might also be considered a core activity. Questions about the demographic and socio-economic characteristics of respondents would also be collected in the core questionnaire. Conversely, the collection of information about specific health problems, about the coverage and use of particular health programmes and about living conditions should be oriented to the priorities of the country concerned. To some extent it might be possible to develop modules concerning particular topics. However, even these would probably need to be tailored to country circumstances. The structure of the health system and the living conditions of the population vary so much between developing countries that over-standardisati **o** fn questions about these subjects is likely to produce common-place or misleading findings. We would also emphasise the importance of in-depth studies of issues of importance in partirular countries. While we are sceptical about the scope for developing standardised protocols, more might be gained from devoting resources to such studies in certain countries than from conducting a larger number of more sup'erficial enquiries. Throughout the design of 12
each country study, a major concern would be to collect information that will be of most use while avoiding overloading the questionnaire or the capacity of the organisation responsible for implementation of the survey.

Another consideration that needs to he taken into account in the design of a WHS is the benefits that could be obtained. from repeating country health studies after an interval of a few years. An 'ad hoc' survey may do little to improve the planning of health interventions and its findings soon become out of date. It might be better to concentrate resources on developing the capacity of national institutions to conduct a continuing programme of field research in those countries in which the findings will be fully exploited. Repeated surveys could yield a much clearer view of the dynamics of the health situation and the improvements in health care resulting from various initiatives. It should be noted, however, that even in developed countries considerable difficulties have been encountered in interpreting trends in self-reported measures in ill-health (Wilson 1984).

# 2.5 Investigations of the determinants of health

The study of the determinants of health is beset with methodological problems. Research into this subject needs to elucidate the complex processes whereby environmental, behavioural and economic factors interact to cause disease and death. It is also necessary to distinguish the effects of multiple contexts on health. For example, the health of a child depends on its own constitution, on childrearing practices adopted by its mother and on the characteristics of the household and the community in which it grows up. Other problems apply to research into factors affecting the use of services. Complex interactions can be expected beteween service provision and individual and household characteristics in determining the use of services. Moreover specific problems arise from reciprocal relationships whereby differences in individual fraility influence the use of services as 13
weil as the uses of services affecting health (Schultz 1985).

No single approach to the provision of information or to analysis will be able to resolve the difficulty of research in this area. A great deal can be learnt from sophisticated analyses of retrospective survey data using statistical controls for confounding factors. However, such investigations have inherent limitations for the definite attribution of causality. Moreover it is impractical to collect information on all subjects of interest using such surveys. An important consideration for the WHS is that surveys should form at most only one aspect of a more extensive strategy aimed at understanding mechanisms and processes affecting health. Moreover it should be noted that the methodologically-sophisticated analyses needed to investigate determinants of health are likely to be beyond the capability of many developing country institutions.

# 3.MEASURING THE IMPACT OF INVESTMENTS IN HEALTH

# 3.1 Economic evaluation

The second objective of the proposed WHS is "to develop appropriate methodology for measuring the impact on health status of investments in health as well as other sectors". Economic evaluation techniques of cost-benefit analysis (CBA) and cost-effectiveness analysis (CEA) are methods for guiding monetary investment in any sector, and so are particularly appropriate to the stated WHS objective. In this section the problems of these techniques are examined at some length in order to identify data which could be collected and used to strengthen their methodology.

It is argued that the problems of such techniques undermine the usefulness of collecting data across countries for the purpose of evaluating the impact of investment on health. Although this is the commonly accepted objective of economic evaluation, even countryspecific studies of this nature should be treated with caution. On the 14
other hand, there are a range of health service concerns that could be more easily addressed, for example, access and coverage issues, financing systems etc. Data related to these options could be included as part of the WHS; specific suggestions are made.

# 3.2 Cgsts

For both CBA and CEA, monetary estimates are required of the inputs into the health programme, project or service being evaluated. These inputs or costs are various and can be summarised as below (Drummond and Stoddart 1985):

1. Organising and operating costs within the health care sector (eg.health care professionals' time, supplies, equipment, power and capital costs).

2. Costs borne by patients and their families - direct costs of outof-pocket expenses (transport and service fees) and patient and family inputs into treatment; indirect costs of time lost from work and psychic costs.

3. Costs borne externally to the health care sector, patients and their families.

Measurement of these costs is not the easy task it may seem and lack of appropriate information about costs to patients and their families often leads to their exclusion from analyses. Identifying the resources used by one programme, when they are shared with other programmes, is also a difficulty. Other informational problems relate to the budget:ng practices of developing countries, which result in little or no routine information that is disaggregated appropriately for economic evaluation purposes (eg. by programme). They can currently only be alleviated by special surveys or by accepting that the data is approximate. The problems of valuing indirect costs in monetary terms is considered in greater detail in relation to consequences.

15
# 3.3 Consequences

For CBA a monetary estimate of the benefits of a particular health - program_e, service or project is required in order to permit evaluation of widely differing alternative investments. For CEA, however, health output is measured quantatively and comparisons are made between differing strategies to achieve a stated objective. The consequences of any programme, project or service can be summarised as (Drummond and Stoddart 1985):

1. Changes in physical, social or emotional functioning (effects)

-

- 2. Changes in resource use for organising and operating services v .hin the health care sector both for the original and for unrelated conditions (direct benefits); relating to activities of patients and their families, in particular the savings in expenditure or leisure time (direct benefits) and savings in lost work time (indirect benefits).

3. Changes in the quality of life of patients and their families (utility).

In evaluating the consequences of health programmes, projeLts or. services there are two main problems: how to measure health effect and how to value health effects/benefits. For CEA measurement only is required, but for CBA both measurement and valuation is necessary.

Given the difficulty of determining other effects (changes in social/emotional functioning), changes in physical functioning are generally identified as the health effect of interventions. Choosing a measure for this effect is, of course, necessary but difficult. In some cases deaths averted or prevented may be suitable. However they could only be applied to fairly specific objectives such as reducing the deaths from a particular disease (the focus of most CEA analyses). In themselves, therefore, they inevitably bias evaluation away from less severe illnesses wh.cb may still have important, even lifelong, consequences. Further, preventive programmes which seek to reduce incidence (rather than fatality), are also less favoured by use of this measure. It wouid be m're useful to have a measure "that can be used to compare the effects of interventions targeted at different discases which have differing effects on mortality, morbidity and on 16
varioas age groups" (Mills 1985). For example, "healthy days<sup>of life</sup> lost" (Ghana Health Assessment Project Team 1981) - a form of a "quality adjusted life years" based on incidence rates, case fatality rates and the extent/duration of disability produced by disease - has been sugsested as an index of effectiveness. This measure has its<sup>own</sup> bias - in favour of rhildren - and judgements of "quality" are inevitably subjective.

In order to measure effects on physical health, better information<sup>is</sup> required about the mortality, morbidity and fertility of the community than is currently routinely available in developing countries. Specific measures which embody preferences for health now rather<sup>than</sup> .tater, fer preventing mortality over disability and for improving the health of different groups (children vs. productive adults vs. elderly) also require information about the~ relative value that individuals and coounities place on these health effects. Without such information the measures are likely to be both inappropriately value-laden and inaccurate.

All the measures so far considered focus on health outcomes - but<sup>the</sup> .'xplicit objectives of many PHC programmes are wider, encompassing self-reliance, increasing the dignity of community members, changing attitudes to health authorities etc. It is not clear how these could be measured or, if that were possible, whether the result could be added to the changes in health outcome to produce aggregate variables which would be comparable across programmes with differing objectives. Ignoring these effects inevitably undermines evaluation of programmes for which they are objectives.

Even if the focus on health impact is appropriate for evaluation of the programme under examination, other problems associated with measuring health outcomes further undermine use of the results<sup>to</sup> guide investment decisions. Many of the problems involved in investigating the determinants of health arise even more strongly in attempts to measure health impact. In the case of a few interventions, clearly defined measures of programme coverage can be obtained and differentials in health associated with these investisated using statistical controls for confounding factors. More 17
usually coverage is difficult to define, particularly in the case of broad-based PHC programes. To relate impact to inputs into<sup>an</sup> intervention usually requires consideration of the situation before and after that intervention. Country health studies could be repeated after several years but how long is an appropriate interval? A shorter period (2 - 5 years) of evaluation tends to favour<sup>those</sup> programmes which have an immediat **e** ffect (eg. immunisation, ORT etc.). Improvements in health services or sanitation may take a much longer period to reach maximum effect and over that longer period the effectiveness of the other types of interventions may decline. Can investment be delayed until long-term evaluations have been undertaken?

Ideally, meas. ement of health impact should be judged with<sup>reference</sup> to a control group, to determine whether changes observed are really due to a newly-introduced intervention or whether they would have occurred anyway eg. as a result of changing climate or improvements in agricultural extension. Where the health intervention is directed at the community, the samples for these control and intervention groups should be samples of communities and not of individuals, because individuals in a community receive the same health services/programmes, and the size of the samples is measured in numbers of communities rather than individuals. With<sup>the use of</sup> sample communities there is a serious problem of confounding variables - for example communities that have better health services are also likely to have better literacy, greater wealth, better leadership etc. In the 4 sence of randomised intervention studies very large samples would be required to overcome these confounding factors statistically.

Clearly, providing all the information required to<sup>measure</sup> consequences is a major task often beyond the efforts and time of evaluators. Instead recourse is often made to measures of intermediate output, such as the numbers of immunised children, or<sup>other</sup> utilisation and coverage data, for which information is more readily available and which are more clearly related to the intervention. Economic evaluation focused on such measures of health service output is perhaps more appropriate in the light of the difficulty of measuring health outcome.

18
The common practice for health projects is to identify much of their output by referring to the improvements in the availability and productivity of labour that result. However, the relationship between disease and productivity is very complex and measuring the full benefit of reducing disease is extremely difficult. This approach may also bias project selection in favour of the predominantly male labour force and ignore actions within the household, such as reallocation of tasks, which affect the impact of disease on productivity (Mills, 1985).

A new approach to valuation (eg Chernichovsky 1979, Popkin 1982), drawing on the economic theory of the household, has been suggested which identifies the household as the unit of analysis. The analysis traces the effects of disease on a) the avai ability and productivity of time spent in activities in the home and market, b) the demand for various subsistence commodities such as nutrition and health, and c) the demand for marketed goods such as health services and improved housing that are inputs into the production of better health. The focus of the analysis is on the time costs of disease rather than the impact on productive employment, as these are seen to capture more of the consequences of tropical disease.

Using either analysis, the benefits of additional labour time identified must be valued - and for CBA the valuation must be made in monetary terms. However, in developing countries such valuation is itself fraught with difficulties because the imperfections of their markets undermine the use of local prices as costs. Wages cannot, therefore, be used to value additional labour time without adjusting them in various ways. These adjustments are not easily made.

Other benefits must also be valued, such as savings on treatment costs and possibly the exploitation of natural resources previously restricted by health problems such as malaria. Identifying all relevant benefits (including non-health benefits) is, in itself, a difficult task and is compounded by the problem of valuation.

19
3.4 General Problems of economic evaluation Given such valuation problems, most economic evaluations of health programmes, projects or interventions are of cost-effectiveness rather then cost-benefits. As a result investment in health is not generally evaluated in relation to investment in other sectors, neither are investmDnts in different health interventions compared. Instead the cost-effectiveness of different strategies of achieving one specific objective are evaluated. Through such analysis, the least cost method of achieving the objective can be determined and/or the method which maximises the effectiveness of a given expenditure can be selected. Whilst CEA has an important role to play in improving the efficiency of health investments, it cannot answer wider questions about<sup>the</sup> relative importance of investments in health or even of investing in different parts of the health sector.

Attempts, moreover, to make international comparisons of economic evaluation studies should be treated with caution. Each evaluation is based on considerations specific to the country, and<sup>even Xthe</sup> community, in which it has been undertaken and so the results cannot be generalised. Programme costs, for example, will vary between settings depending on the organisation and scale of the programme, project or intervention, whilst the effectiveness of a programme will vary with the social, cultural and economic setting - affecting its acceptability and its benefits. Only, for example, by repeated studies of the same management strategies for the same intervention could general conclusions about the possible appropriateness of those strategies be drawn. (Such evaluation has been undertaken for immunisation programmes, one of the few cases where there is quite a strong link between intervention and health outcome - but the evaluations have, still, often used health service outcomes (eg.numbers of immunised children) as their focus.) 20
3.5 Economic evaluation and the World Health Survey The numerous problems involved in measuring the impact of investments in health have not been presented to dismiss the need for such evaluation, but to identify areas where the WHS could contribute most to improving methodology. It would obviously be best to focus on areas where amelioration is most possible and likely to result from the WHS.

Comparing investments in health with investments in other sectors is beyond current methodology and seeking to improve CBA techniques is not a top priority. As noted, the problems of measurement of health impact and valuation undermine the use of CBA in the health sector. Even if the WHS focused on some valuation issues (eg.tracing the labour time benefits of health programmes through data about the relationship between health and available labour at the household level), other problems of valuation and of measurement would still remain.

The key measurement problem for CEA (and CBA) of demonstrating the link between investment and health outcome is difficult to address in the context of a world health survey. Improvements in methodology could not be generated by data collection using standardised survey questions, as they could not hope to capture the differences between countries which have a direct bearing on the health impact of investment. Even if the survey's questions were made countryspecific, the detailed research required to investigate this link could only feasibly be undertaken within the context of the WHS for a few interventions with clearly definable coverage and for which confounding variables can be controlled for statistically at the individual or household level. Certainly controlled intervention trials are not needed routinely. Indeed, if the benefits of the intervention were well established, this would be ethically questionable. It is more feasible to organise such evaluations as 'ad hoc' activities tailored to the nature of the intervention and circumstances in which it i, made and taking advantage of opportunities for the application of innovative methods such as case control studies. It would be more appropriate to undertake such 21
research within - existing health intervention programmes methodological improvements are much Tnre likely to result from indepth study over many years linked to input-output monitoring of programe efficiency than from a "one-off' survey.

However, there are various issues related to health services which would form a more appropriate focus for investigations based on single-round or repeated single-round surveys. Access and coverage, for example, can be much more clearly identified as outcomes of investments in particular health programmes and so are more appropriate for economic evaluation than impact on health status. For the same reason, data on these issues can also be more easilv compared across countries. Although the dangers of transferring results of evaluations between countries without reference to differing country situations remains, the impact of such differences on the results are themselves also of interest to health planners. (How do different strategies of health service delivery affect numbers of population covered?). Data collection could concentrate both on areas where improvements In the methodology of economic evaluation might result, and on other useful economic/planning information relating to health services (for eacL country and to make international comparisons). Possible areas of interest would be: - data on patient costs for use of health services: differences between urban and rural areas, differences between age and sex groups; - data on the patient costs of different health financing mechanisms eg.user fees, tax financing and social security; - data on patient utilisation of health services/programmes: differences between urban and rural areas, differences between age and sex groups; effects of fees, transport costs and time cost; - data on the determinants of demand for health care; comparing demand for different types of providers (public, private-for-profit, traditional healers, charitabl **e** tc.); - data on the adequacy of care: relating adequacy to different factors such as level of facility, educational level of health worker, availability of drugs, availability of supervision and management.

Such household-base **d** ata would need to be complemented by data from the health system itself before it would form the basis for economic 22
evaluations. For example, data on health programe costs would be needed to complement data on patient costs.

A standardised questionnaire would, of course, still pose its own problems. Flexibility in wording, for example, would be required to ensure the relevance oO the questions to each individual country. Perhaps it would be better to undertake "add-on" studies of particular issues. Such studies could then be made appropriate to each country's situation and needs, and could collect both patient data and complementary health system data.

Clearly, the cost (both monetary and in terms of time, disruption to present systems etc.) of any data collection process additional to (or instead of) those in existence, must be weighed against the possible benefits. Health service issues have been identified as the most appropriate focus for a WHS, in terms of the possibility and relevance of generating benefits (ie. improvements in economic evaluation methodology). It should also be considered whether the same improvements (and benefits) might be better generated through improving existing data collection processes.

# 4. SCOPE AND LIMITATIONS OF INTERVIEW-BASED SURVEYS

# 4.1 Health interview surveys - an introduction

There is a growing amount of experience in the use of health interview surveys in developing countries. These typically include questions on general morbidity, cause-specific morbidity, mortalitv, causes of death, nutritional status, coverage and use of services, knowledge of and attitudes towards health practices/services, and an array of explanatory or independent variables (age, sex, residence, socioeconomic indicators, maternal education etc.). There are a number of theoretical models which attempt to classify the 'explanatory var,ables', and the 'proximate determinants' which affect an array of 'outcome variables' (see, for example, Mosley and Chen 1984 and van Norren and van Vianen 1986). Here, we do not follow any particular model but classify morbidity, mortality and nutritional status as outcome variables, consider a range of 'process' variables such as 23
knowledge, attitude and practice (KAP) and briefly comment on explanatory variables.

Several good methodological reviews of health interview surveys in developing countries have been produced recently (for example, Kroeger 1983, Ross & Vaughan 1984, Kroeger 1985, Gray 1986, WHO et al 1986) and the UNSO is currently sponsoring the preparation of operational guidelines for collecting health and mortality information using household surveys. We therefore do not attempt a full review of the methodological issues associated with the outcome, process and explanatory variables that one might cover in the WHS. Instead, we highlight recent debates and developments which should be considered in designing a WHS.

# 4.2 Mortality - well established methods

Several comprehensive reviews of the uses, collection and analysis of data on mortality are now available (eg. Vallin et al 1984, United Nations 1984). Here we restrict the discussion to a few issues concerning the collection of mortality data in surveys focused on health. Single-round surveys that have asked straightforward questions about deaths in the preceding 12 or 24 months have frequently yielded disappointing results. Both reference-period errors and omissions appear to be problems and data on young children are often particularly poor. Moreover, it is often difficult to collect reliable information about even the basic characteristics, such as age, of those who have died. An alternative approach is the multiround survey. Reference-period errors and omissions are greatly reduced as data is collected about deaths in previously-enumerated households during the intervals between rounds of the survey. Such studies involve prolonged fieldwork operations in which households must be first traced and then reinterviewed. Moreover to collect information in one or two years on adequate numbers of deaths to measure precisely mortality levels and patterns requires a sample of at least 20,000 households and preferably far more. Multiround surveys of mortality are complex and expensive operations and it is doubtful that they are justified in the context of a World Health Survey. While mortality is of particular significance for demographic 24
studies, for those concerned with health planning and evaluation mortality is only one aspect of the health situation. Information on morbidity and on coverage and use of health progra_es and services is probably of more significance.

Against the background of disillusionment with questions about recent deaths in the household, particularly among investigators with a background in medicine, it is important to emphasise that alternative methodologies for the retrospective collection of information on mortality have performed far better. Questions addressed to respondents about deaths among clearly-defined categories of their kin normally yield useful data. In particular the World Fertility Survey convincingly demonstrated that collecting detailed birth histories from women and asking about the ages at which deaths of their children occured could provide upexpectedly high-quality estimates (Hobcraft 1984). Such birth histories also provide information about the constitutional factors affecting early mortality (such as the age and parity of the mother and the length of birth intervals) and represent a framework within which dated information on child rearing practices that influence health and mortality (such as breastfeeding and weaning) can be collected. To minimise costs a WHS core questionnaire might collect truncated birth histories covering only the 5 years before the survey. A full birth history could be an optional module. Obtaining reliable data about adult mortality is more difficult. Questions about orphanhood and widowhood often, but not always, perform well (see, for example, Timaeus 1987). Methods for measuring long-term trends in mortality are now well established (Brass and Bamgboye 1981). In addition a number of approaches have been suggested that can define the period of exposure to risk much more closely. These include the supplementary questions proposed by Brass and by Chackiel and Orellana (1986) and procedures for analysing intersurvey changes in orphanhood (eg. Timaeus 1986).

In **some** countries the WHS could provide an opportunity to develop methodologies for assessing causes of death. This information is valuable both for health impact evaluation and for setting health priorities. It is generally agreed that open-ended questions addressed to relatives of a dead person do not provide reliable 25
information on cause of death (Gray 1985, 1986). Instead, symptomprompted questions with structured questionnaires which record the duration and severity o. the symptoms or signs prior to death are often addressed to surviving relatives. These sets of diagnostic questions (algorithms) can be used by lay interviewers. Gray (1985) provides a good description of diagnostic algorithms that can be used and discusses their validation and limitations. Promising results have been found in the diagnosis of neonatal tetanus, measles, diarrhoea/dysentery, and whooping cough. More validation of these algorithms is needed and algorithms for low birthweight, acute lower respiratory tract infections, tuberculosis, severe protein-calorie malnutrition and vitamin A deficiency, need testing and validating. Recall appears to be good up to 10-12 months after death (Gray 1986) but more country-specific research is needed to assess appropriate recall periods. New ideas for validation are needed as the traditional approach, of conducting retrospective interviews of mothers whose children died in medical institutions where the diagnosis was known, are inadequate. If a WHS was conducted in a country where there is good registration of death then more extensive validation could be done. For example, in Brazil, children have medically-certified causes of death. However, in this case one still faces the obstacle that the mothers will be more aware of the diagnosis through contact with the registration system.

# 4.3 General morbidity. cause-soecific morbidity and use of services

We know much more about levels and patterns of mortality than we do about morbidity. Curative health services affect mortality more than morbidity as in essence they prevent the transition from sickness to death. For eample, the dramatic decline in infant mortality in Kerala, India may be attributed to a reduction in the fatality rate which was a result of widely accessible and utilised health services (Krishnan 1984). It is notable that while mortality rates plummeted, morbidity rates and patterns appear to have remained the same as in the rest of India. Preventive public health programmes are more likely to affect morbidity and, with the introduction of preventive PHC policies in many developing countries, it is now particularly 26
appropriate to focus upon measures of morbidity. It is also timely to do so since a standardised methodology is now emerging which uses a 14 day recall period for general morbidity, allows for spontanecus and probed responses and links morbidity with illness sever .ty, functional disability, utilisation of services and attitudes towards services (WHO et al 1986 and Carlson 1985). A WHS core questionnaire might contain questions on general morbidity and associated use of services (including traditional healers and private practitioners) using a 14 day recall period. Optional modules could contain further questions on attitudes towards services, costs of services and the use of drugs etc. While household surveys in the 1970s and early 1980s produced levels of general morbidity which were not comparable (see reviews by Ross and Vaughan 1984, 1986 and Kroeser 1983, 1985) the standard methodology that is emerging enables us to make comparisons of general morbidity rates across countries. Many of the African countries participating in the UNHSCP are using aspects of the methodology described above in their national health surveys. It is important that any new international initiative in health surveys complements these existing surveys.

Although the use of general morbidity questions is becoming more common, the problems associated with them must not be forgotten. Extensive pre-testing is required to make questions culturallv appropriate; validation methods need to be developed; and, in order to present results in terms of standard epidemiological measures of frequencv, it is desirable to record and analyse the duration of completed and uncompleted episodes of illness separately. Ideally proxy-reporting should not be permitted, except for children under a defined age, when the mother should respond on their behalf. If any proxy-responding is allowed, results from proxy-interviews should be analysed separately from subject-interviews to allow comparisons to be made (Ross and Vaughan 1984).

Validation against facility-based or home-based health records is not very fruitful (Kroeger 1983, Ross and Vaughan 1986). A method of validation which has been little used but is probably the best of all, is the study of attenders at the various potential sources of health care. During the pre-testing stage a sample of attenders at a clinic, 27
hospital, pharmacy or traditional healer could be specifically selected for subsequent interview to validate their responses. Preferably these knewn attenders shoula be seeded among others<sup>to be</sup> interviewed.

It is not justified to extrapolate a morbidity rate found by<sup>using a</sup> recall period of less than 12 months into an annual rate,<sup>because of</sup> the seasonality and epidemicity of most conditions. Some countries have overcome this problem by extending dhe fieldwork of<sup>the health</sup> jurvey over a year (see section 5.8).

Morbidity results should be reported by symptoms or<sup>grouped into</sup> functional systems (eg. respiratory, digestive etc). It is probably futile to try to force them into medical diagnostic categories<sup>(eg.</sup> asthma, malaria etc) except for the limited number of conditions which are widely, and usually correctly, recognised by the survey population (Ross and Vaughan 1984). For these types of conditions it<sup>may be</sup> appropriate to have sets of questions in order to obtain<sup>cause</sup> specific morbidity.

In recent years, household surveys have increasingly been used for measuring the incidence or prevalence of specific diseases and the WHS could develop modules on cause-specific morbidity. Countries will have particular interests in certain diseases. In order to decide which topics to include in a national health survey it is often useful to use the kind of guide presented in Table 1. Questions on specific diseases should only be asked if interventions are to be planned around the results.

# Table I

How to select topics for inclusion in the World Health Survey (or, 'Setting Priorities in Public Health')

1. Prevalence or frequency

2. Severity 3. Ability to intervene (in terms of effectiveness, political and logistical feasibility) 4. Need for data to justify a programe 28
Rank each topic on a scale of 0 to 5.

VXAMPLK

|Tooic<br>Fre<br>Diarrhoea<br>in children|ouency<br>5|x Severitv<br>4|Ability to<br>x Interveng<br>2|<br>Need for info<br>x to iustify<br>**oro**<br>2|- Tota<br>80|
|---|---|---|---|---|---|
|Brain cancer|1|5|0|5|0|
|Anaemia|3|2|3|0|0|
|Infant formula<br>use|3|3|4|4|144|

Questions on cause-specific morbidity will typically follow questions on general morbidity. For certain diseases, questions on symptom/sign complexes have high sensitivity and specificity in most settings. These diseases tend to be easily recognised by the population and have clear symptoms eg. measles and diarrhoea. Kroeger (in WHO et al 1986) presents groups of diseases which can be elicited by means of interviews directly, indirectly or after having received a medical diagnosis. It is particularly important that questions on specific diseases are pre-tested for cultural appropriateness. In Bangladesh for example, it was discovered that an evaluation of an ORS programme was using a word for diarrhoea that meant cholera in certain parts of the country (Chowdhury 1986).

A problem that many developing countries are particularly interested in is long term, or chronic, disability. However, measuring disability through health interview surveys is fraught with difficulties. The main problems arise from the fact that both the I= and severitv of disability need to be defined and measured. In defining 'disability' much confusion arises from the casual use of different terms such as impairments, disabilities and handicaps. Previous surveys have been incomparable since different definitions were used. The International Classification of Impairments, Disabilities and Handicaps (ICIDH) goes some way to clarifying and standardising definitions and its use is to be encouraged in any survey tackling the topic of disability (WHO 1980). When it comes to severity of disability it is advisable to develop tests which lay 29
interviewers can rerform to assess the respondent's level of disability (es asking the respondent to say how many fingers the interviewer is raising while the interviewer stands at certain - distances away to test sight). No proxy responding can therefore be allowed. There is a need to develop simple methodologies and interviewer instructions for measuring disability which a WHS might apply. The strong points of a number of different previous disability surveys need to be identified and used. A WHS core questionnaire might contain a question on 1 year recall of chronic illness for adult members of the household.

In terms of specific behaviour rather than specific illness some countries may be interested in sets of questions on smoking or the use of alcohol (see ndO et al 1986 for an array of questions on these topics).

The WHS could be a useful vehicle for assessing the coverage of programmes which are either targeted at all individuals within the population or all individuals within a defined age group or sex (Ross in WHO et al 1986). The methodologies for each tvpe of programme will be different and an array of optional modules could be developed. The coverage of vaccination, oral rehydration therapy and malaria chemoprophylaxis are three obvious subjects of interest.

The determination of vaccination status is difficult excepting the use of scars to identify those who have received a BCG vaccination, and only vaccination cards or clinic records can provide reliable information.

Questions on treatment received for diarrhoeal episodes and source of treatment (eg. home, pharmacy, health facility) should directly follow questions on morbidity, severity and duration. The recall period should therefore be the same i.e. 14 days (this is accepted as the norm by the WHO Diarrhoeal Diseases Control Programme, see WHO 1981).

The recall period for questions on consumption of antimalarials should not exceed four weeks for weekly or biweekly drugs, and should be much shorter, probably two days, for daily preparations. Possession of 30
tablets should be checked and correlated against records or, where this is not possible, against the recalled date of supply (Ross in WHO !.tjal 1986).

# 4.4 Nutritional status indicators

Anthropometric measurements on pre-school age children are technically straightforward and, if well controlled, can be reliable and accurate. Such measurements could be part of the WIHS core questionnaire. The WHS will be able to make use of the excellent training manuals on anthropometry now available. The task of training interviewers to weigh and measure children is facilitated by well illustrated guides like the one produiced bv the IJNHSCP (United Nations 19$6), and the tape-slide sets produced by the LSHTM. However, indicators based on weight-for-age, height-for-age, weight-for-height, mid-upper arm circumference etc. are all best regarded as both non-specific and indirect. This is because body size at a given time is the end result of the interaction of nutritional and infectious disease processes and hence is an indicator of the combined or sequential effects of both. In the absence of other information (e.g. on disease status) therefore, anthropometric indices cannot be interpreted by themselves so as to implicate a deficiency of nutrients or energy supplv as primary causes of poor health status.

During the past few years, progress has been made in relating anthropometric measurements to other measures of outcome - in particular, with mortality experience. Thus, for all of the indicators mentioned above, we can say that (i) There is a non-linear relationship between anthropometric measurements and mortality during a fixed period subsequent to measurement. That is to say, the probability of death rises more rapidly with increasing deficit in body size (Kielmann and McCord - - (1978) weight-for-age; Chen et al (1980) weight and height-forage; Sommer and Lowenstein (1975) - arm circumference and height).

(ii) The shape of the function relating mortality with anthropometry is not constant, but depends upon the pattern of iafectious diseases to which the population is exposed.

31
Anthropometry of older children or of adults, may give more information about the adequacy of food/nutrient supply and could for, an optional module. Dugdale (1985) has suggested that measurements made on all members of households may be particularly useful in helping to differentiate food deficit situations from those in which poor health status is determined by exposure to disease.

Depending upon the general design, a WHS could contribute valuable information about the significance of anthropometric indicators in relation to other health indicators. In particular, there is a need for more information about the influence of different infectious disease backgrounds. In addition, extension of measurements of weight and height to adults would be extremely valuable in helping to validate the use of indices such as Body Mass Index (weight-height' ) in situations where chronic undernutrition of working populations is suspected as a determinant of poor health and performance. In addition, BMI may well prove to be an important basis for constructing health/nutrition indicators which provide a 'profile' covering all members of a household or family. All of these objectives would however entail the use of 'in depth studies' as part of the WHS design.

Other modules might consider specific nutritional deficiencies. Vitamin A and iodine deficiencies are still considerable public health problems in many developing countries. In the Asian countries, high prevalence and large populations at risk combine to produce an estimated half million cases per year of children with active corneal lesions and 5 million non-corneal xeropthalmias. About 800 million people are at risk of iodine deficiency diseases which include endemic cretinism as well as goitre. Although it is possible to show approximate geographical distributions of populations 'at risk', there is a shortage both of national representative studies and of withincountry distributions. Methodologies for measurement of status are technically demanding and expensive. However, the magnitude and persistence of these problems might justify a fairly intensive effort, at least in some carefully selected countries.

32
4.5 Environmental contamination In relation to the International Drinking Water Supply and Sanitation Decade WHO developed a 'Minimum Evaluation Procedure' (MEP) for water supply and sanitation projects (WHO 1983). A five page questionnaire was developed which focuses upon the evaluation of the functioning and use of water supply and sanitation facilities. Impact evaluations were explicitly avoided as they are generally more complex and costly to plan and carry out than function and use evaluations. The MEP can be shortened and culturally adapted and provides a useful starting point for an optional module on environmental contamination. Methods to measure personal hygiene (hand washing, food preparation etc), housing types and degree of crowding are also fairly well established now but the design of such measures needs anthropological input and careful pre-testing.

# 4.6 Explanatorv variables

Health depends upon a complex weave of socio-structural variables and processes including family relationships and power relationships within a community. However, we suggest that the WHS concentrates on explanatory variables which are subject to intervention or can be used to target interventions to improve health. Essentially this means restricting questions to indicators of social status and not exploring social processes. So, which explanatory variables should be included in a WHS?

Any health survey needs to obtain information on certain explanatorv, or independent, variables. Sex, age, and maternal education (measured by years of schooling) are essential, as are measures of socioeconomic status. The latter have generated much discussion. Although there is a need for information on household possessions, housing type, place of residence, access to food etc, it is all too easy to overload a household questionnaire with, for example, detailed questions on labour force participation.

Given the fact that current health depends very much on past health 33
and experiences it is important to consider the role of historical data in the WHS. For example it would certainly be pertinent to record how long people have lived in their present environment (ie length of residence). Similarly, information on past housing conditions, water supply, and sanitation facilities may be needed. In order to distinguish widowed and divorced mothers it may be necessarv to include a simplified marriage history in a questionnaire.

# 4.7 Community variables

So far the discussion has tended to assume that information on health and its determinants should be collected about individuals or households. However, many aspects of environmental risk and the availability and accessibility of services will be common to wider aggregates. Moreover the interuption of disease transmission may only occur once services are used widely throughout an area. Common and contextual variables also have the further analytic advantage of being clearly exogenous to household decisions about the use of services (Schultz 1985).

It could be verv valuable to supplement the household survey with collection of information directly from clinics. One area of interest would be in the provision of services. For example data could be collected on location, opening hours, availability of essential drugs and the history of service provision (Hobcraft 1985). Moreover clinic records could provide important supplementary information on morbidity. Exactly what might be possible would depend on the national context, but one area of interest could be life-threatening diseases with a low prevalence in the community.

It could be useful to compare information obtained on general morbidity through household questionnaires with community level data from use of the Delphi technique (Levine 1984, WHO 1986). This technique involves using panels of knowledgeable individuals to obtain information about the most frequently occuring and most important illnesses that affect the population in their area. This combination of household questionnaires complemented by key informant discussions is a well tested and attractive approach.

34
# 4.8 Summary of core questionnaire and modules

To summarise, the proposed topics for any core questionnaire and those for which optional modules might be developed are presented in Table 2. The core questionnaire would be fairly short and many of the optional modules would have to be country-specific.

# S. DESIGN AND ORGANISATIONAL ISSUES

# 5.1 Selection of countries for the WHS

The -OR for this report suggest that the WHS would be a programme to sponsor studies of health in a broad cange of developing countries. The experience of the World Fertility Survey suggests that comparative studies of some topics are best undertaken by including data from as many and as varied countries as possible while other investigations find it more profitable to concentrate on a limited number of countries with certain features in common. It should also be noted that only a programme of surveys that included such populous countries as China or India could be considered in any way representative of the developing world. It therefore seems important to decide at the outset whether the countries involved in a WHS should be selected to maximise their differences or whether country health studies should be concentrated where they are judged most important. At least four axes of variation seem relevant: existing health conditions, sociopolitical structure, the organisation of the health services and geographical location.

As was emphasised in section 2, survey-based investigations of health are not appropriate in every country. The value of such a survev depends on the nature of existing health information systems, the capacity of national institutions to conduct a survey without disrupting other activities and the extent to which the results will be used to improve planning and management of the health services.

In the selection of countries the WHS should avoid duplicating the efforts of the UNHSCP national health surveys. One possibility is that 35
A summary of proposed topics for a World Health Survey tore questionnaire Table 2 and optional modules

|Core questionnaire|Optional modules|
|---|---|
|Mortality<br>- five year birth history<br>- orphanhood for women aged 15-49|Mortality- complete birth history<br>- cause specific moetality eg. tetanus, measles,<br>whooping cough, diarrhoea|
|Morbidity- 14 day recall of general morbidity<br>for all household<br>members<br>- 1 year recall of chronic illness for<br>adults|Morbidity- cause specific morbidity eg. ARI, diarrhoea|
|Use of services- use associated<br>with two<br>week recall of morbidity|Use of services- attitudes towards services,<br>drugstaken<br>- coverage of specific programmes eg. ORT,<br>vaccination,<br>malaria prophylaxis<br>- costs of services (time and money)|
|Nutrition- anthropometry<br>of all preschool<br>children<br>Explanatory<br>variables (age, sex,<br>education,<br>houshold possessions,<br>place of residence, length of<br>residence,marital<br>status etc)<br>Community variables|Nutrition- anthropometry<br>of adults and older children<br>- nutritional<br>deficiencies<br>eg. Vit A and iodine<br>Environmental<br>contamination<br>- water supply, sanitation,<br>housing<br>type (past and present)<br>- personal hygiene practices<br>Knowledge and attitudes|
WHS staff could provide a central pool of expertise available to countries implementing such surveys.

# 5.2 Review of the existinx health information svstem

The first organisational step needed in a country where a study is to be conducted is a comprehensive review of past, current and future health data collection activities. This should include both facilitvbased and population based data. This kind of activity will sometimes have taken place already in which case a simple up-dating will be needed (eg. USAID recently undertook this exercise in Tanzania). Both the Ministry of Health and the Central Statistical Office need to be involved in this and every following stage. Too often national health surveys have been undertaken under the auspices of a Central Statistical Office without any liaison with or input from the Ministry of Health. The recent training workshop on health and nutrition survevs held in Zimbabwe aimed at overcoming this problem by bringing health planners and statisticians together to assess country health information needs and to design appropriate health survey methodologies (WHO et A 1986).

The choice of topics for inclusion in a country study should depend in part on the outcome of this review. The overall aim should be to produce information that complements that being provided by other sources. Duplication of data collection efforts should be avoided and in some countries it might be decided not to proceed with a survev. Questions about particular diseases should focus on the more important health problems in the country. Similarly questions about health services should reflect their organisation, for example the relative importance of vertical programmes and integrated PHC services. Attention should also be focused on matters pertinent to major future programmes that are being considered. Once the scope of the study has been decided, the review will provide an essential background to the detailed design of the questionnaire and to the development of plans for tabulation and the production oj reports.

37
# 5.3 Sample sizes and desian

Some of the measures that would be required from WHS country studies can be estimated from quite small samples. One example is service coverage and use. Collecting information on rarer health problems, for example mortality or adult morbidity requires much larger samples. Moreover national estimates are often of limited use to health planners who require information on the sub8roups of the population that particular programmes or geographically localised initiatives are aimed at. Many of the most important and policy-relevant research issues involve examination of the interactions between different determinants of health, for example factors that inhibit the use of services (Hobcraft 1984). Furthermore, if the WHS adopts the plan to repeat the studies in order to measure change, this again requires an increase in sample size.

Two relevant findings from the assessment of the World Fertility Survey (Cleland and Scott 1987) are that sample-size related fieldwork costs represent only a moderate part of total expenditure on a programme of international surveys and that rigorous methodologies, and in particular extensive training, can produce high-quality information even when large samples are interviewed. These findings, comibined with the multiple advantages to be gained, suggest that the country health studies should aim to cover samples of several thousand households. This would be adequate to investigate child mortality, to carry out detailed and disaggregated studies of the more common diseases and of service use and to measure substantively significant changes in programme coverage and health over a five year period. In large countries that are ecologically diverse or have a federal system of government it is likely that detailed geographical breakdown of the results would be needed if they were to be of use for health planning. This would indicate a correspondingly larger sample.

One issue that will require particular attention in the design of samples for a WHS is the extent to which thev are clustered. The loss of efficiency that results from cluster sampling is particularly severe in the case of health. This reflects the localised transmission of infectious disease and the fact that many services (eg. the 38
presence of a clinic) and facilities (eg. adequate sanitation) are often a characteristic of a whole village or neighbourhood. However the costs of carrying out intensive or follow-up studies on a widely dispersed sample would probably be prohibitive. It might be necessary to conduct them in a localised area or in a subsample of clusters, rather than individuals, included in the main survey.

# 3.4 Reneat surveys

If the proposal to repeat country health studies after several vears is adopted, attention will have to be given to this in the overall design of the WHS. It is probably impractical to attempt to conduct a panel study, that interviews the same households, after such a long interval. Some of the advantages of this could be obtained by conducting the second survey in the same sampling areas which were used for the initial study. For example, changes in community-level variables could be examined directly. To ensure that this approach was feasible, attention would have to be given to the production and preservation of the necessary documentation. Second, there is evidence that the results of health interview surveys are sensitive to even minor changes in methodology and changes in people's perceptions of illness over time (Ross and Vaughan 1986). The need for consistency will conflict with the desire to update the questionnaire to reflect changes in circumstances and improvements in methods. Explicit attention will have to be given to the resolution of this conflict of interests during preparation for the second round of fieldwork.

# 5.5 In-depth studies

Section 4 of this paper outlined the wide range of information on health and its determinants that can be collected in a single-round survey. Other kinds of information can only be collected prospectively or by conducting medical examinations. Many possibilities can be envisaged and the priority that should be assigned to them will depend on the country concerned. One theme that could be particularly valuable, both for assessing the benefits from health expenditure and for clarifying the determinants of health, would be follow up studies 39
of individuals experiencing ill-health or other medically relevant conditions at the time of Lhe main survey. This could yield a much better understanding of the longer-term implications of such experiences. Some specific possibilities are: to follow up pregnant women to examine determinants of their health status and pregnancy outcomes in the context of the use of MCH services and of their work activities. This study could be extended to examine the implications of a new b)irth for the health of the infant's older sibling; to follow up sick and malnourished children to clarify the interaction between morbidity and growth faltering and the process by which disease may develop into long-term disability;

-

to examine the economic and health consequences of serious illness among adults, in particular mothers and principal earners, both for the household as a whole and for the children in it.

# 5.6 Ouestionnaire development and ore-testina

The complexity of the social processes affecting health suggest that small-scale observational studies could represent a useful counterpart to the examination of existing statistical information (see section 5.2) during the planning of country health studies. These could be both clinic and community based. As well as suggesting relevant issues ano hypotheses, such studies have an important role to play in questionnaire development. For example, a good understanding of traditional concepts and terminology describing diseases and svmptoms is essential for the design of algorithms intended to identify particular diseases and causes of death. Moreover the sensitivity and appropriateness of questions about many issues vary between cultures. Such studies would also be useful for identifying and clarifying the nature of social aggregates (ie. communities) that are likely to be of relevance to the health status and health-related actions of individual households (see section 4.7).

40
5.7 Piloting and traininz Very thorough pilot tests will be needed of the whole organisational set up of the survey. If numerous respondents are used in each household what are the best arrangements for interviewers? What should the sex, age and background of interviewers be? If anthroponetric measurements are to be taken how is the work load to be split between interviewers? Will the mother be involved in the anthropometric measurements? In a health survey typically two interviewers proceed to each household. Particular attention should be paid to how measuring boards and scales are to be transported (if used). Experience from USAID's Demographic and Health Surveys (DHS) is useful here.

Many questions in a health survey are on sensitive topics and interviewers need thorough training to be able to deal constructively with potential problems.

Special procedures are needed in order to obtain blood, faecal, urine samples etc with specially trained staff made available. 5.8 Seasonalitv and fieldwork operations One potential methodological drawback of single-round survevs investigating health is the seasonal concentration of many infectious diseases and of food shortages. This needs to be taken into account in the design of country health studies. Combined with the need to use highly trained interviewers in order to collect reliable and consistent data on health, this suggests that fieldwork operations should follow a sinilar model to that adopted for the LSMS. In other words, a limited number of teams of interviewers should conduct a programme of work evenly spaced out across an entire calendar year. Entry, validation and editing of data should proceed in parallel. Even at the cost of adding somewhat to the expense of fieldwork, efforts should be made to ensure that interviews in each region and stratum, for example urban areas, are also spread across the whole year. However it should be recognised that conducting year-round fieldwork may be impractical in precisely those countries in which seasonal 41
variations in health are probably greatest and that drop-out of interviewers is a potential problem in such a long fieldwork period (Harpham 1987). Moreover this approach would not provide the individual-level data on seasonal variations in health needed to investigate this subject fully.

# 5.9 Validation and examinations

Laboratory investigations in communitv-based studies can for logistical and ethical reasons be divided broadly into those which assess and evaluate the delivery of health care services and those which examine health and nutritional status. The first group would include capillary blood samples taken for measurement of antibodies against tetanus, measles, and polio etc. In such studies blood samples would be collected as part of a sLngle round cross-sectional survey and brought, after suitable preparation 'in the field', to a central laboratory. There would be no need for immediate feedback of the information to the individual in order that treatment might be obtained. The second group would include capillary blood samples taken for measurement of haemoglobin or thyroid status etc where early treatment for those 'at risk' with very abnormal values would be indicated. This ethical consideration has been facilitated by the recent development of accurate, simple reproducible means for measuring haemoglobin 'cn site'; severe anaemia for instance could be treated immediately. The remainder of the blood sample could then be taken to the base laboratory.

# 5.10 Data vrocessing

Data processing problems are a most serious issue in manv countries. The analysis and dissemination of survey results are often delaved because all aspects of data processing have not been determined before the collection of data starts. However, there is now experience of using microcomputers and appropriate software to overcome such problems. The organisation of the Living Standards Measurement Studies (LSMS) of the World Bank is a useful example. The LSMS in the Ivory Coast (1600 households) produced a preliminary abstract of results just two months after the end of fieldwork (Spitzer 1986). This was achieved by making microcomputers the survey's hub around 42
which everything else revolved. The preparation of the questionnaire, data codebook and documentation, sample design and household selection, scheduling of interviews, data entry and cleaning, preliminary tables and analyses and statistical abstract were all carried out using microcomputers. Environmental problems were prevented bv the use of dust covers, surge guards and air conditioning. Editing ('cleaning') of data was performed at regional centres so it was possible for interviewers to re-visit households whose records had inconsistent or missing data. The 'WHS should make full use of such experience and plan processing at the earliest stage of each survey.

A related development of considerable importance that has been pioneered at ORSTOM and is also being investigated at LSHTM is the use of hand held computers for data capture. This eliminates the data entry stage of survey processing and appears to reduce errors that arise from interviewers and coders misunderstanding the skip instructions on the questionnaire. It has considerable potential for reducing the cost and speeding up the processing cf survey data.

# 5.11 Stratexies for analysis

A useful strategy for analysis, comparable with that adopted bv other international programmes of survey research, would be to aim to produce a first report rapidly, outlining the design and conduct of the survey and presenting descriptive tables and a brief commentary on them. The emphasis would be on providing information on each aspect of health and health-related behaviour rather than on examining their inter-relationship. Important topics would include mortalitv; incidence or prevalence of disease; the nutritional status of children; access to clinics, drug dispensaries and other services; coverage achieved by vertically-organised health programmes; and exposure to enironmental contamination. Each table should be produced according to a series of background characteristics, in particular those most useful for identifying differential need and use of services, for example age, sex, region and residence.

At least two priorities for further analysis can be identified. One is use of the data to assess the past or future effectiveness of in43
vestment in particular health programmes. The other is more general investigations of the processes affectins, and determinants of, health. Both fields of investigation are beset wita methodological problems and will be extremely demanding of the skills and ingenuity of the investigators. It is in the nature of such issues that no complete solutions exist. Nevertheless the WHS will only produce methodologically adequate studies and effective progress if considerable resources are committed to this final stage of the programme. The analysis of many surveys is more or less abandoned bv hard pressed organisations with responsibilities for other work once the initial report has been produced. Thus the central secretariat and outside researchers need to be involved. In addition to directly funding research, a great deal can be accomplished by rapid production of documented files for secondary analysis, organisation of meetings and active prograimes for the publication and dissemination of information. It should be noted, however, that some countries regard health as sensitive and confidential.

# 6. CONCLUSIONS

- (1) A series of coordinated country health studies, producing population-bas **e** stimates d of health and its determinants, has a valuable role to play in some countries as one aspect of efforts being made to imprwve health information systems in the developing world.

- (2) Improvement of methodology for assessing the health impact of interventions is of vital importance but data collection should probably focus on 'ad hoc' enquiries conducted in favourable circumstances. General-purpose health surveys are of limited use for evaluating impact but could provide information required to evaluate the cost effectiveness of sc:vice provision. Moreover a programme of surveys repeated at intervals of several years could form a focus for an overall assessment of progress being made in the health sector.

- (3) The main value of a WHS would arise from the establishment of a central secretariat accumulating expertise in the design, con-

44
duct, processing and analysis of country health studies and involved in dissemination and application of findings from them, rather than from attempts to collect internationally comparable information on health.

- (4) Country health studies should be set in the context of and supportive of the existing health information system. It is vital to involve Ministries of Health as well as national statistical offices. Major efforts should be put into drawing together, and perhaps conducting further analyses of, existing data before fielding any survey.

- (5) Country health studies should aim to provide complementarv information to existing initiatives to develop routine data collection, and other progra_es of surveys and methodologies for project evaluation. In many countries this is likely to be achieved by conducting health interview surveys that focus on morbiditv and children's nutritional status, but retrospective data on mortality should also be collected.

- (6) The variety of health problems and priorities in developing countries suggests that any core questionnaire should be fairly brief. Reviews of existing data should feed into the development of a design and questionnaire that reflect health problems, service provision and information needs in each country and will provide the data needed for planning.

- (7) Although much useful information can be collected in singleround surveys, other topics require a prospective studv or the conduct of medical examinations. Part of a WHS programme should consist of in-depth studies of health issues of particular importance in the countries concerned.

- (8) Experience of conducting and analysing general-purpose health surveys is limited. Although their feasibility in now established and many of the instruments designed, the WHS will need to devote adequate funds to methodological research. One

45
priority is the validation of interview-based questions. A second is analytic methodology needed to take full advantage of data to be collected.

46