Perspectives on rigor and reproducibility in single cell genomics

Over the past ten years, single cell genomics has emerged as a powerful approach to cell biology whose impact relative to bulk tissue genomics can be likened to the transition from light to electron microscopy. Query of PubMed with the term “single cell genomics” reveals over 5,000 papers a year since 2012 for a field that is probably in its early growth phase. Analyses at the single cell level reveal properties of tissues that are barely approachable with bulk methods: the contributions of variable cell type abundance to normal and pathological variation, the identities of the cell-types that explain disease, the molecular basis of intercellular signaling, the trajectories of cells as they mature or senesce, and cellular resolution of regulatory mechanisms, are but five examples. None of this would be possible without the parallel emergence of powerful pipelines and algorithms for data analysis, and advances in high performance computing that facilitate routine handling of many-terabyte-sized datasets. Not a week goes by without the publication of a path-breaking study, job-prospects for trainees are outstanding, and fundamental transitions within the “knowledge machine” of modern biology [1] seem inevitable. Nevertheless, students new to the field can be bewildered by the diversity and complexity of computational methods, and soon realize that single cell data analysis is as much an art as a science. For this reason, there is some room for self-critical evaluation of the existing bioinformatics, and as a contribution to this end I here offer perspectives on five areas where robustness of inference can be improved. The science is going to proceed apace regardless, but I am concerned about a publishing culture that often glosses over the uncertainty that is necessarily intrinsic to analysis of what are among the most complex and voluminous datasets ever produced. We are lucky that the field is building on solid computational and statistical foundations, that attention to experimental quality control is well-informed by recent next-gen sequencing experience, and hence there is good reason to be confident that the vast majority of findings in the literature are correct. Nevertheless, at a time when funding bodies require applicants to address rigor and reproducibility explicitly, it is remarkable how little acknowledgment of analytical variability there is in the literature. First, though, it is worth emphasizing that there is much to be optimistic about with this young field, starting with the fact that it exists at all. Methods that are commonplace today were not yet even conceived two decades ago when microarray analysis ushered in the era of transcriptomic research, and it is really just a decade since the transition to RNAseq. For a few years it seemed that there were more publications presenting new methods for single cell analysis than experimental findings, but remarkably the field has quickly settled around a few basic pipelines such as Seurat [2], Monocle [3] and Scanpy [4], which is certainly contributing to comparability and accessibility. The current emphasis on generation of comprehensive atlases PLOS GENETICS