A blind and independent benchmark study for detecting differentially methylated regions in plants
Authors: Stefan A Rensing
Noe Fernandez-Pozo
Nilay S Can
Ralf Schulze Bruening
Clemens Kreutz
Zsuzsanna Mérai
Rabea Meyberg
Journal: Bioinformatics (2020)
Discipline: BIOLOGY & BIOCHEMISTRY
Keywords: Bisulfite sequencing; DNA methylation; epigenetic regulation; differentially methylated regions; algorithms
Affiliations: Faculty of Medicine and Medical Center, Institute of Medical Biometry and Statistics, University of Freiburg, 79104 Freiburg, Germany., Centre for Integrative Biological Signalling Studies (CIBSS), University of Freiburg, 79104 Freiburg, Germany., Plant Cell Biology, Faculty of Biology, University of Marburg, 35043 Marburg, Germany., Gregor Mendel Institute of Molecular Plant Biology, Austrian Academy of Sciences, Vienna BioCenter (VBC), 1030 Vienna, Austria., Centre for Biological Signaling Studies (BIOSS), University of Freiburg, 79104 Freiburg, Germany.

Abstract:  Motivation: Bisulfite sequencing (BS-seq) is a state-of-the-art technique for investigating methylation of the DNA to gain insights into the epigenetic regulation. Several algorithms have been published for identification of differentially methylated regions (DMRs). However, the performances of the individual methods remain unclear and it is difficult to optimally select an algorithm in application settings. Results: We analyzed BS-seq data from four plants covering three taxonomic groups. We first characterized the data using multiple summary statistics describing methylation levels, coverage, noise, as well as frequencies, magnitudes and lengths of methylated regions. Then, simulated datasets with most similar characteristics to real experimental data were created. Seven different algorithms (metilene, methylKit, MOABS, DMRcate, Defiant, BSmooth, MethylSig) for DMR identification were applied and their performances were assessed. A blind and independent study design was chosen to reduce bias and to derive practical method selection guidelines. Overall, metilene had superior performance in most settings. Data attributes, such as coverage and spread of the DMR lengths, were found to be useful for selecting the best method for DMR detection. A decision tree to select the optimal approach based on these data attributes is provided. The presented procedure might serve as a general strategy for deriving algorithm selection rules tailored to demands in specific application settings. Availability and implementation: Scripts that were used for the analyses and that can be used for prediction of the optimal algorithm are provided at https://github.com/kreutz-lab/DMR-DecisionTree. Simulated and experimental data are available at https://doi.org/10.6084/m9.figshare.11619045. Contact: ckreutz@imbi.uni-freiburg.de. Supplementary information: Supplementary data are available at Bioinformatics online. 

Retracted: No

PubPeer comments:
 Hi, Unfortunately, I don t have access to Bioinformatics to read more about your article. However, I think there is a typo in your title: differeally -- differentially Thank You, Charles , 
 #1 Hi Charles, Thanks for catching that, hard to imagine it was there all the time. Please contact me if you would like a PDF. Kind regards, Stefan , 
 #2 Hi Stefan, Thank you very much for your response. I think you might need to contact the journal for a correction, but I am glad that you are aware about the typo. Best Wishes, Charles , 
 #3 Yes, of course, we ll do - we checked and it was still correct in the PDF proof, so no idea where it came from... , 
 Hi Stefan, Thank you for the additional explanation. I know that there was one situation where I could catch an additional error after the paper was accepted, but I think in time for the final version to be corrected. That said, I am going back to correct a number of my earlier papers (in addition to providing commentary for other papers and products). I am trying to find my most appropriate long-term workload. I think this applies other researchers, and figuring how how to carry out the most effective journal grant review is also important. I guess the good part is that fixing the problem should be easier than determining precisely what caused the problem. I hope this gets resolved soon, and you can make good progress with your research! Sincerely, Charles ,