Table 1. Key considerations for determining when and where to conduct coverage surveys. Primary purpose for coverage survey Method for selecting the survey area (e.g. district) Preferable timing in program To check if the data reporting system is working well Random Early (1st -2nd years of program), repeated in later years if early results show a large discrepancy To get a better estimate of coverage where there is reason to believe that the routine reporting is incorrect: a) in areas where the denominator (total population figure) is suspected to be grossly incorrect or out-of-date b) highly suspect reported coverage c) on-going incidence of morbidity d) failing a transmission or impact assessment survey Purposeful or random, depending on the number of areas with the suspected problem As need arises Evaluation of programme progress Random Mid-term evaluation of national Master Plan of Action (3rd – 4th years of program) To integrate a KAP1 survey with a coverage survey in order to: a) identify the reasons for low coverage b) test the effectiveness of social mobilization and communication strategies Purposeful if testing specific low coverage areas Random if testing across the program Following MDA with reported low coverage Following a new/change in social mobilization strategy 1Knowledge, Attitudes and Practices 1.2 TIMING RELATIVE TO THE MDA It is recommended that coverage surveys be implemented as soon after the MDA as possible to minimize recall bias. Coverage surveys should ideally be conducted after the reported coverage from the last MDA has been calculated and before planning for the next MDA has begun; typically no later than 3 – 6 months post-MDA. The following timelines are a suggestion that can be used to help with planning and budgeting: Planning for a coverage survey – 1-2 months Implementing a coverage survey – 2-3 weeks Interpreting and acting upon the coverage survey results to improve the next MDA– 1-2 months 1.3 COVERAGE EVALUATION SURVEY TEAM
1.5 TRAINING Training of the survey team should take place right before implementation of the coverage survey to ensure that the team does not forget the information learned. A suggested outline for training is as follows: Day 1 – In-class training on the segmentation approach, selection of households and practice with the questionnaire Day 2 – Practice the approach in 1 or 2 field sites (sites selected for field practice should not include any of the 30 sites selected for the actual survey) Day 3 (full or half day) – Discussion of experience from Day 2, review of the survey methods, and planning the field work 1.6 COST The cost of a typical coverage survey, including the cost of training, can vary substantially by country, as well as within countries according to the size and density of the survey area. Most coverage surveys cost between $2,000 - $10,000 per survey area/district to implement. 1.7 USE OF THE COVERAGE SURVEY BUILDER To aid survey planners in designing a coverage evaluation survey, an Excel-based tool called the Coverage Survey Builder (CSB) was developed. The CSB can help survey planners with the determination of a sample size appropriate for the survey, the selection of subunits, the segmentation and systematic selection of households within the selected subunits, and the compiling of results. The CSB is available for download at: and will be referred to in Sections 4, 5 and 6 below. SECTION TWO: SELECTION OF THE SURVEY AREA Coverage surveys should be conducted at the administrative geographic area in which MDA is conducted and for which PC coverage is collated and reported. It is referred to in this manual as the survey area. For many NTDs, this is typically a district or implementation unit. School-based distributions: School-age children (SAC), 5-14 years, are frequently treated through school because the use of existing school infrastructure makes treatment more efficient and reduces distribution costs; however, it is important to recognize that the SAC population requiring treatment is not limited to children who attend school. Therefore, it is important that coverage evaluation surveys include all school-age children – both in and out of school. To achieve this, coverage surveys evaluating
Example: Using the default values the sample size would be: SECTION FIVE: SYSTEMATIC SELECTION OF SUBUNITS For this protocol, 30 subunits should be randomly selected from among all within the survey area. Subunits should be administrative areas for which population figures are available. The ideal subunits are census enumeration areas (EAs), although villages, wards, localities or any other small administrative unit may also be used. Though EAs may require greater effort upfront than other choices of subunit, the use of EAs for survey sampling is advantageous because: they account for all households in a country in a jointly exhaustive and mutually exclusive fashion (i.e. each household in the country will fall into one and only one EA); they are among the smaller geographical units available, sometimes much smaller than villages and large towns, so they are easier to work in than larger units; they can be used in urban areas as well as rural and semi-urban areas; and, outline maps for each EA are often available, which can greatly aid in the creation of segments (Section 6). For the rest of this manual we will use the term “subunits” when referring to the first units of sample selection, be they EAs/villages/localities/etc. See the box below for a summary of this information. Enumeration Areas (EAs) Villages/Wards/Localities/Hamlets/etc. Designed to be comprehensive: jointly exhaustive and mutually exclusive A household can belong to multiple villages or no village Relatively consistent, small population size Population size can vary widely Can be used in rural, semi-urban, and urban areas Challenging to use in urban centers Maps are available, typically from the same governmental office that conducts the census Maps may not be available In order to improve the efficiency of household sampling, within each selected subunit a segment of households will be randomly selected. Sampling of households will take place only within that segment. The probability that any one subunit is selected is proportionate to the estimated number of segments it contains. The default segment size is 50 households; however, in some cases larger segments may be necessary if the expected number of households required to meet the sample size is >50 (segmentation
Consider using Independent Monitoring or the Coverage Supervision Tool with MDA mop-up during the next round to improve coverage Check the reasons for why the eligible population was not offered the drug Check the reasons for the eligible population not swallowing the drug Tailor corrective action according to reasons given which include strengthening: Drug supply chain Social mobilization and information and education campaigns (look at reasons given by those who did swallow the drug) Drug delivery platform used Training, supervision and motivation of drug distributors Communication on adverse events Capacity of national and/or district level staff Survey coverage is above the target coverage thresholdb Communities and drug distributors are motivated and the programme is functioning well Congratulate your teams. Sustain programme momentum for the next year to maintain coverage levels Comparison of survey coverage with reported coverage: To check if the data reporting system is working well Reported coverage is much higher than survey coveragec (i.e., routine reporting is likely overestimating true coverage) Drug distributors are incorrectly reporting on ingestion of the drugs Conduct a Data Quality or Data Quality Assessmentf to diagnose where the data reporting system is breaking down Improve the skills and motivation of drug distributors through better training and supervision; consider use of mHealth technologies Make improvements to the tally sheets and/or registers used The total population figure (e.g., the denominator) is incorrect or outdated, or people from outside the survey area are also taking the drugs and are being included in the total treatment tallies Determine if more accurate population estimates or projections are available and apply a correction factor to routine coverage estimates as appropriate
APPENDIX 2: RESULTS ENTRY FORM Drug: Order Subunit Name No. Females Interviewed No. Males Interviewed No. Females Offered No. Males Offered No. Females Swallowed No. Males Total Sample Size 0 Female Sample Size 0 Male Sample Size 0 Program Reach 0 Female Program Reach 0 Male Program Reach 0 Surveyed Coverage 0 Female Surveyed Coverage 0 Male Surveyed Coverage 0
APPENDIX 3: COVERAGE EVALUATION ACTION PLAN Problem/Weakness Identified Action Required (what tasks do you need to do to address the problem) Success Criteria (how will you identify your success) Time Frame (by when do you need to complete the tasks) Resources (who or what can help you complete the tasks)