Effects of library size variance, sparsity, and compositionality on the analysis of microbiome data
Authors: Weiss Sophie J , Xu Zhenjiang , Amir Amnon , Peddada Shyamal , Bittinger Kyle , Gonzalez Antonio , Lozupone Catherine , Zaneveld Jesse R. , Vazquez-Baeza Yoshiki , Birmingham Amanda , Knight Rob 
Published: January 2018

Background: Data from 16S amplicon sequencing present challenges to ecological and statistical interpretation. In particular, library sizes often vary over several ranges of magnitude, and the data contains many zeroes. Also, since researchers sample a small fraction of the ecosystem, the observed sequences are relative abundances and therefore the data is compositional. Here we evaluate methods developed in the literature to address these three challenges in the context of normalization and ordination analysis, which is commonly used to visualize overall differences in bacterial composition between sample groups, and differential abundance analysis, which tests for significant differences in the abundances of microbes between sample groups. Results. Effects of normalization on ordination: Most normalization methods successfully cluster samples according to biological origin when many microbes differ between the groups. For datasets in which clusters are subtle and/or sequence depth varies greatly between samples, or for metrics in which rare microbes play an important role, rarefying outperforms other techniques. For abundance-based metrics, rarefying as well as alternatives like DESeq and metagenomeSeq’s cumulative sum scaling (CSS), seem to correctly cluster samples according to biological origin. With these normalization alternatives, clustering by sequence depth as a confounding variable must be checked for, especially for low library sizes. Effects of differential abundance testing model choice: We build on previous work to evaluate each statistical method using rarefied as well as unrarefied data. When the mean library sizes in the differential abundance groups differ by more than 2-3x, or the library sizes differ in distribution, our simulation studies reveal that each statistical method improved in its false positive rate when samples were rarefied. However, when the difference in library size mean is less than 2-3x, and the library sizes are similarly distributed, rarefying results in a loss of power for all methods. In this case, DESeq2 has the highest power to compare groups, especially for less than 20 samples per group. MetagenomeSeq’s fitZIG is a faster alternative to DESeq2, although it does worse for smaller sample sizes (<50 samples per group) and tends to have a higher false positives rate. For larger sample sizes (>50 samples), rarefying paired with a non-parametric test, such as the Mann-Whitney test, can also yield equally high sensitivity. Based on these results, we recommend a stepwise procedure in which sample groups are first tested for significant differences in library size. If there is a significant difference, we recommend rarefying with a non-parametric test. Otherwise, DESeq2 and/or fitZIG offer increased sensitivity, especially for rare OTUs and small sample numbers. Conclusions. These findings help guide which technique to use depending on the data characteristics of a given study.