Null is beautiful: On the importance of publishing null results

Science is built on the principle that it is self-correcting – false findings will eventually be identified through rigorous attempts to verify and reproduce novel results. In principle, of course, this is true. In practice, however, things may be rather different. Current incentive structures in biomedical science reward novelty and discovery over replication and confirmation. In the current climate of striving for publications and funding, an overworked scientist will struggle to justify investing time and effort into writing up null results, rather than focusing on collecting more data and writing up other, more “exciting” findings. The inevitable result is publication bias against null results – in many disciplines nearly all publications report “positive” findings (Fanelli, 2010), with psychology and psychiatry having the highest proportion of studies reporting support for the tested hypothesis. It might appear that scientists are blessed with powers of precognition (Bones, 2012); their experiments nearly always (appear to) work. The consequences of publication bias are profound and detrimental. Science cannot self-correct, at least not as efficiently as we would hope, if failures to replicate previous findings, or experiments that follow from theory but turn out to be blind alleys, are not published. Most senior academics will have experience of early career researchers setting out on a project that attempts to build on previous work and finding that they cannot replicate what appeared to be a robust finding. Only later do they discover that others are aware of this problem but have not published their own null results. This knowledge remains hidden from the public record, to the detriment of science. Several solutions to publication bias have been proposed, including pre-registration of study protocols and peer review of studies either blind to the results, or even before the study has been conducted (known as Registered Reports) (Chambers, 2013). Another solution is for journals to explicitly promote the publication of null results. The Journal of Psychopharmacology is therefore introducing a new submission format, Null Results in Brief. This will provide an efficient means by which scientists can publish null results with the minimum of effort. Only a brief rationale for the study and description of the methods is required, together with the results themselves and a short discussion focused on the implications of the null results, up to a maximum of 800 words, with eight references or fewer and up to two display items (figures and/or tables). In many ways, the bar for acceptance of these submissions will be set higher – within a null hypothesis significance testing framework, which remains the dominant approach to statistical inference in biomedical science, null results can only be interpreted if the study is based on a very clear and sound theoretical rationale, the study design is rigorous, and the statistical power is sufficiently high to exclude a theoretically or clinically meaningful effect. Submissions in the Null Results in Brief format will be peer reviewed in the usual way, with reviewers specifically asked to consider whether the results will add to current knowledge, and are likely to be useful to future investigators (either for inclusion in meta-analyses or to prevent attempts to test hypotheses which have already been shown to be unlikely to be correct). Submissions will need to demonstrate a clear biological rationale for the hypotheses tested, and the statistical power should be sufficient to ensure that the null results are interpretable (i.e. they can exclude a theoretically or clinically interesting effect size). Only submissions that meet these stringent criteria will be accepted for publication. By adopting a Null Results in Brief format, we hope to go some way towards addressing problems of publication bias, and prevent researchers from investing time and resources in questions that have already been addressed by others.