Positive steps with negative studies in the WASH sector
Authors: John M Colford
Journal: The Lancet Global Health (2017)

Despite encouraging evidence about the health benefits of individual water, sanitation, and hygiene (WASH) interventions,1Clasen TF Alexander KT Sinclair D et al.Interventions to improve water quality for preventing diarrhoea.Cochrane Database Syst Rev. 2015; 10: CD004794Google Scholar, 2Wolf J Pruss-Ustun A Cumming O et al.Assessing the impact of drinking water and sanitation on diarrhoeal disease in low- and middle-income settings: systematic review and meta-regression.Trop Med Int Health. 2014; 19: 928-942Crossref PubMed Scopus (303) Google Scholar much less is known about the benefits of combined or packaged interventions delivered through public-health campaigns and programmes. Rigorous assessment of such combined intervention programmes has many challenges. For example, such assessments usually require a deep understanding of a complicated study design, typically entailing a cluster-randomised trial. These study designs frequently demand large sample sizes and geographically dispersed study units to avoid possible spillover effects that distort the noted estimates of effect.3Benjamin-Chung J Abedin J Berger D et al.Spillover effects on health outcomes in low- and middle-income countries: a systematic review.Int J Epidemiol. 2017; (published online April 24)https://doi.org/10.1093/ije/dyx039Crossref PubMed Scopus (30) Google Scholar As a result, the sample sizes demanded by these studies usually need substantial budgetary resources that can be alarming for funders without previous experience of this type of work. These challenges are compounded by the importance of maintaining high retention of participants and good adherence to the intervention, goals that further strain study resources and need (often humbling) insight into human behaviour. Finally, these studies might need to include an untreated control; such a design feature might be off-putting to communities and leaders, whose assent and participation is essential for the work. In The Lancet Global Health, Sheela Sinharoy and colleagues4Sinharoy SS Schmidt W-P Wendt R et al.Effect of community health clubs on child diarrhoea in western Rwanda: cluster-randomised controlled trial.Lancet Global Health. 2017; 5: e699-e709Summary Full Text Full Text PDF PubMed Scopus (28) Google Scholar show that, despite these challenges, rigorous assessment of combined WASH intervention programmes is possible and aligns appropriately with broader goals in science toward reproducibility and transparency in research.5Nuzzo R Fooling ourselves.Nature. 2015; 526: 182-186Crossref PubMed Scopus (193) Google Scholar, 6Nosek BA Alter G Banks GC et al.Promoting an open research culture.Science. 2015; 348: 1422-1425Crossref PubMed Scopus (1289) Google Scholar They did a cluster-randomised trial to compare two models of community health clubs, which are multi-session village-level gatherings designed to promote healthy behaviours mainly related to WASH, with a control (no intervention). Among children younger than 5 years, the prevalence of caregiver-reported diarrhoea in the previous 7 days (primary outcome) was 14% in control villages, 14% in villages allocated a shortened (eight sessions) intervention (prevalence ratio compared with control 0·97, 95% CI 0·81–1·16; p=0·74), and 14% in villages assigned a longer (20 sessions) intervention (0·99, 0·85–1·15; p=0·87). Moreover, no effects of the interventions were noted on secondary outcomes of height-for-age and weight-for-height Z scores and water quality. The rigour of their assessment makes the unexpected findings—ie, a null result for their health outcomes—credible and justifies the effort they invested in this work. The authors' approach to this assessment is instructive because of its adherence to many methodological practices that should now be standard, not only for assessment of WASH interventions but also for evaluation of many public-health intervention programmes. Several of these methodological practices merit particular attention. Sinharoy and colleagues produced a detailed analysis plan and showed that they adhered to it. They also agreed—as a condition of their funding—to provide deidentified study data publicly, to allow independent replication of their analyses. Additionally, although self-reported outcomes (typically related to diarrhoea or gastrointestinal symptoms) are especially common in the WASH sector, Sinharoy and colleagues also analysed so-called hard outcomes, including multiple anthropometric measures and water quality, in addition to diarrhoea. WASH studies are frequently done to assess short-term interventions and they have limited follow-up because of resource constraints, potentially missing an attenuation of effects that might be underway after interventions end, thereby denying the chance to assess the sustainability of the interventions. Sinharoy and colleagues continued follow-up for an acceptable duration, around 2 years. One final feature of their work is the separation of programme implementers from programme evaluators, to reduce potential biases arising from the human tendency to hope that a programme is effective by those implementing it. By agreeing to publish negative results, journals continue the fight against publication bias by choosing to publish well designed, implemented, and reported studies, even in the face of negative findings. Such negative studies are frequently disappointing to the authors, funders, and implementing organisations. Allowing methodologically sound but disappointingly negative results to receive widespread attention through publication in high-impact journals is the only way that the WASH sector can move forward to find interventions that truly are effective and to jettison conventional wisdom about effective programmes. Funders asking (hopefully, insisting) that investigators use these methods can create a virtuous cycle that reinforces our practice in the sector. The work of Sinharoy and colleagues4Sinharoy SS Schmidt W-P Wendt R et al.Effect of community health clubs on child diarrhoea in western Rwanda: cluster-randomised controlled trial.Lancet Global Health. 2017; 5: e699-e709Summary Full Text Full Text PDF PubMed Scopus (28) Google Scholar cannot answer whether the community health club model might be more successful if it included a broader package of interventions—perhaps, for example, including nutritional supplementation, educational efforts, or different and additional environmental interventions. Such an expanded package of interventions would itself best be studied in a subsequent trial and, if successful, replicated in other settings. This line of investigation into broader packages of health interventions done with methodological rigour and the willingness to document the absence of effect when none is present might be of particular interest to governmental and non-governmental funders. It is in the direction of such rigour that the efforts of these authors and those of similarly rigorous studies are leading the WASH sector. I declare no competing interests. Effect of community health clubs on child diarrhoea in western Rwanda: cluster-randomised controlled trialCommunity health clubs, in this setting in western Rwanda, had no effect on caregiver-reported diarrhoea among children younger than 5 years. Our results question the value of implementing this intervention at scale for the aim of achieving health gains. Full-Text PDF Open Access