Accurate and fast detection of copy number variations from short-read whole-genome sequencing with deep convolutional neural network
Authors: Jiajin Li
Stephen Hwang
Luke Zhang
Jae Hoon Sul
Venue:  (2021)
Keywords: copy number variation
deep learning
convolutional neural network
computational biology
DNA sequencing

Abstract: A copy number variant (CNV) is a type of genetic mutation where a stretch of DNA is lost or duplicated once or multiple times. CNVs play important roles in the development of diseases and complex traits. CNV detection with short-read DNA sequencing technology is challenging because CNVs significantly vary in size and are similar to DNA sequencing artifacts. Many methods have been developed but still yield unsatisfactory results with high computational costs. Here, we propose CNV-Net, a novel approach for CNV detection using a six-layer convolutional neural network. We encode DNA sequencing information into RGB images and train the convolutional neural network with these images. The fitted convolutional neural network can then be used to predict CNVs from DNA sequencing data. We benchmark CNV-Net with two high-quality whole-genome sequencing datasets available from the Genome in a Bottle Consortium, considered as gold standard benchmarking datasets for CNV detection. We demonstrate that CNV-Net is more accurate and efficient in CNV detection than current tools.

Under review as a conference paper at ICLR 2021
ACCURATE AND FAST DETECTION OF COPY NUM- BER VARIATIONS FROM SHORT-READ WHOLE-GENOME SEQUENCING WITH DEEP CONVOLUTIONAL NEURAL NETWORK
Anonymous authors Paper under double-blind review
ABSTRACT
A copy number variant (CNV) is type of genetic mutation where stretch of DNA is lost or duplicated once or multiple times. CNVs play important roles in the development of diseases and complex traits. CNV detection with short-read DNA sequencing technology is challenging because CNVs significantly vary in size and are similar to DNA sequencing artifacts. Many methods have been developed but still yield unsatisfactory results with high computational costs. Here, we propose CNV-Net, a novel approach for CNV detection using a six-layer convolutional neural network. We encode DNA sequencing information into RGB images and train the convolutional neural network with these images. The fitted convolutional neural network can then be used to predict CNVs from DNA sequencing data. We benchmark CNV-Net with two high-quality whole-genome sequencing datasets available from the Genome in a Bottle Consortium, considered as gold standard benchmarking datasets for CNV detection. We demonstrate that CNV-Net more accurate and efficient in CNV detection than current tools.
roles
INTRODUCTION
A copy number variant (CNV) is genetic mutation where stretch of DNA is completely lost or repeated more than once compared reference genome. CNV sizes range from 50 bases to 3 million bases or more with two major types: duplication if the DNA sequence is repeated and deletion if a DNA sequence is missing. CNVs are spread along a genome, which account for 4.8% to 9.5% of human genome (Zarrei al. 2015). They are known to influence many complex diseases including autism (Sebat et al.) (007). bipolar disorder (Green et al. 2016), schizophrenia (Stefansson eet al. 2008), and Alzheimer' disease (Cuccaro al. 2017) well as gene expression (Chiang al.) 2017). As CNVs may overlap with a large portion of a genome, such as one gene or even multiple genes, their effect on disease may be substantially larger than that of single nucleotide variants, and hence CNVs often play important roles in the genetic mechanisms of diseases and complex traits.
With the advent of next-generation DNA sequencing technologies over the past decade, the res- olution and scale of CNV detection has been greatly improved as large-scale sequencing studies become feasible. However, CNVs are still challenging detect from short-read next-generation DNA sequencing techniques due to the significantly varying sizes of CNVs and their similarity with common DNA sequencing artifacts. Many computational methods have been developed for the dis- covery of CNVs from short-read DNA sequencing data, but their performance is often unsatisfactory due to low accuracy and high computational cost (Kosugi et al. 2019). The main reason for this that previous methods (Rausch et al. 2012 Abyzov al. 2011] mainly rely the statistical anal- ysis of the signals from read alignments (a process that maps reads from DNA sequencing data to the reference genome), which often fail to employ all useful features of DNA sequencing data and require significant computational resources. Therefore, there is a need for a novel, sophisticated computational tool to improve CNV detection with higher accuracy and efficiency.
Here, we present CNV-Net, a new approach to identify CNVs from DNA sequencing data using six-layer deep convolutional neural network (CNN). CNV-Net first encodes the reads and their

---

Under review as a conference paper at ICLR 2021
related information from DNA sequencing data such as the type of base, base coverage (the number of reads covering a position), and base quality of each position into the RGB channels of images. false positives (not CNV) briakpoints. Note that a CNV is defined by two breakpoints, that is, its start and end positions. Although it has been shown that CNNs such
Then it uses deep CNN to predict these images deletions, duplications,
Variant (Poplin et al.) 2018) and Clairvoyante (Luo et al. 2019) can accurately detect single nucleotide variants from similarly encoded images or lensors, few CNNs have been designed for the identification of CNVs. The key advantage of CNV- Net over other methods is the application of deep learning, which allows learning features with higher complexity from DNA sequencing data. We train the network with the Genome in a Bottle Consortium NA12872 dataset (Pendleton et al.) 2015), and demonstrate CNV-Net performance on a bigh-quality benchmarking dataset acquired from a well-studied cell line, HG002 (Zook et al. 2020). We also compare its performance to that of previous methods.
METHODS
CNV-Net first encodes sequencing information into pileup images and classifies each image as a deletion, duplication, or false positive (not a CNV). In this section, we describe how we generate pileup images, that is, the input to CNV-Net, as well as the design of the CNN.
2.1 PILEUP IMAGE GENERATION
We convert the information of mapped reads from DNA sequencing data (provided in BAM file format) and reference genome information to 201 x55 pixel image representation of 201-base wide regions. Training data pileup images contain CNV breakpoint (start or end position) centered in the image, capturing the 100 positions left and right each CNV breakpoint. Using the RGB channels of an image, this snapshot is able capture features of the reads at specific locations in the reference genome.
Coverage [5) [cov. cov. 0}
Reference (base, 254, 240)
Reads
[45]
(base. quality, strand)
Figure I: 201 x55 pixel pileup image. Feature rows are labeled by brackets and RGB channels are labeled by curly braces. Reference rows are encoded in channel by base A: 250, G: 180, T: 100, C: 30, N: O. Coverage rows are encoded in RG channels by the number of mapped reads covering each position. Reads are encoded in RGB channels by base types, base quality, and strand directionality positive: 70, negative: 240.
As shown in Figure [ the first five rows the image capture the reference genome across a region of 201 base pairs by encoding the R channel of each pixel with an individual base in that position: A's as 250, G's as 180, T's as 100, C's as 30, and N's as O. Under the five rows representing the reference genome are another five rows that capture the read depth (coverage) at that position. The R and G channels are encoded with an adjusted read depth value between 0 and 255. The following 45 rows capture the mapped reads from DNA sequencing. The channel encodes for the nucleotide, which is the same the reference genome, and the G channel encodes for the base quality found in the BAM file, which is adjusted to between and 255. Lastly, the B channel encodes for strand directionality: 70 if positive and 240 if negative.
Each 201 x55 pixel pileup image, thus, captures coverage and reads specific to the 201-base region of the reference genome such that the sequencing data may be re-created from a series of consec- utive pileup images. there are more than 45 reads mapping to a specific region, those reads are discarded; instead compensate by capturing this information through the rows that encodes cov- erage. If no reads map to the region, the image is left blank below the first five rows for the reference genome.
2

---

Under review as a conference paper at ICLR 2021
2.2
DESIGN OF CONVOLUTIONAL NEURAL NETWORK
Input
Convl
Conv4
FC5
Output
15
Poo.
Prr
AucPooo
Augitol Flatten
Input (55x201x3)
AvgPool [2:2]
Convolution Dropout [3.3]
Fully Connected (3)
AvgRool
Convalution + Dropout
Convolution Dropout (27x100x64) [3.3]
(13x50x128)
13.31
Fully Connocted + Drogout (512)
Convolution + Drapout
(55x201x32) [3,3]
Figure 2: CNV-Net architecture. CNV-Net employs six-layer convolutional neural network for de- tecting copy number variations from DNA sequencing data. Descriptions under each layer indicate: type of layer, dimension of layer in parenthesis (input layer: height x width x channels, convolu- tional layer: height x width x filters, and fully connected layer: nodes), kernel size in brackets. The model predicts the probability of deletion ("DEL"), duplication ("DUP"), and false positive ("FP") breakpoint presence in each input pilcup image.
As presented in Figure 2 CNV-Net consists of four convolutional layers and two fully connected layers with total of 5, 100999 parameters. Each of the four convolutional layers uses x kernels and the RELU activation function. Each convolutional layer is followed by batch normalization, average pooling, and 0.4 dropout. Arrays are flattened in between the four convolutional layers and two fully connected layers followed by softmax output to predictions for CNV breakpoint presence in the 201-base window: duplication, deletion, or false positive (not a CNV).
In training CNV-net,
standard Adam optimizer (Kingma & Ba 2014) with sparse cate- gorical cross-entropy loss function. We use a cyclical learning rate initialized at Ic-5 and maximal 1e-2. To reduce overfitting, train the model until the accuracy cannot be improved in 15 epochs. The 201-base regions are centered around the CNV breakpoints, that is, the starts and ends of CNVs in the training set capturing the 100 positions directly before and after each CNV breakpoint. False positive images include random 201-bass regions outside of CNVs. From the NA12878 dataset, total of 52,243 pileup images labeled duplications, deletions, and false positives from the training set randomly split into 70%, 20%, 10% categories for training set, validation set, and test set for the network, respectively.
3 RESULTS
In this study, we use two standard structural variation benchmarking datasets: NA12878 and HG002. Both these benchmarking datasets are mapped to the GRCh37 reference genome. NA12878 is well-studied genome provided by the Genome in a Bottle Consortium. It is whole-genome se- quenced with 50x coverage and each read in the sequeneing data is 101 bases long. From the NA12878 image dataset (Pendletoon et al. 2015). we generate images for 22,936 duplications, 18,614 deletions, and 10,693 false positives (not CNVs) breakpoints. The HG002 dataset (Zook al. 2020) another robust benchmarking dataset derived by the Genome in a Bottle Consortium, which is quenced with 60x coverage. Reads the HG002 dataset are all 148 bases long. As we only consider CNVs passing the quality control filter, this dataset contains 174 duplications, 10,668 deletions, and 5,205 false positives breakpoints post-filten -iltteiin.
We train the convolutional neural network the NA NA12878 dataset and test the fitted model on the HG002 dataset. We also compare CNV-Net with popular existing tools for CNV detection, including Brecer (Chen al. 2009), CNVnatoo (Abyzov al. 201T), Delly (Rausch et al.
3

---

Under review as a conference paper at ICLR 2021
2012). Lumpy (Layer et al., 2014h, and Manta (Chen et al. 2016). It is important to note that CNV- Net makes predictions using the pileup images of CNVs where their start and end positions are given, and hence assume that candidate CNV positions other methods find the candidate CNV positions using the information on how reads are mapped to the reference genome and detect CNVs those candidate CNV positions.
given imput to CNV-Net. However,
The NA 12878 dataset is preprocessed and converted into RGB images as described in the Methods section. We split the CNVs in NA 12878 into 70%, 20%. 10% for training set, validation set, and test set, respectively. We train the CNV-Net with the training set and report its performance metrics on the test set. As shown in Table I CNV-Net identifies duplications with Fl-score of 0.7833, deletions with Fi-score of 0.7473, and false positives with Fl-score of 0.7124. This result shows that CNV-Net achieves high accuracy in the test set of the NA NA12878 dataset.
Table 1: Performance of CNV-Net on the test set of the NA12878 dataset
CNV types
Precision
Recall
F1-score
Duplications 0.8376
0.7356
0.7833
Deletions
0.7885
0.7102
0.7473
False positives 0.6046
0.8672
0.7124
Next, we evaluate our fitted CNV-Net on the HG002 dataset and compare with five existing tools for CNV detection. This training scheme represents the real data analysis we will train CNV- Net with CNVs from known genomes and testing it on different genomes. As demonstrated in Table 2 CNV-Net achieves the highest F1-score in CNV detection in the HG002 dataset among all six methods. It also has the highest recall and the second highest precision. This result shows that CNV-Net is more accurate in CNV detection than current methods.
Table 2: Performance of CNV-Net and other methods on the HG002 dataset
Tools
Precision
Recall
F1-score
Break
0.05437
0.6503
0.05923
CNV
0.3523
0.03143
0.05771
Delly
0.5841
0.06846
0.1226
Lumpy 0.7228
0,06815
0.1245
Manta
0.9282
0,3916
0.5508
CNV-Net 0.7960
0.6596
0.6912
We then compare the time cost of applying different methods to the HG002 dataset. CNV-Net is significantly faster than other methods (Table 3). That is because fitted convolutional neural net- work is efficient in making predictions, however, may be time-consuming during model training. Run time effectively front-loaded into model training instead of a concurrent statistical analysis. Another reason for this can be that CNV-Net makes predictions on candidate CNVs so it does not need spend time searching for candidate CNVs in DNA sequencing data as if other methods do.
Table 3: Runtime of CNV-Net and other methods on the HG002 dataset
Tools
Runtime
Brecer
92 minutes
CNVnator
51 minutes
Delly
565 minutes
Lumpy
46 minutes
Manta
36 minutes
CNV-Net
16.62 seconds
4

---

Under review as a conference paper at ICLR 2021
4 CONCLUSION
CNV-Net encodes DNA sequencing information into RGB images and applies deep convolutional neural network to learn complex features from DNA sequencing data to discover CNVs. We demon- strate that CNV-Net can detect CNVs breakpoints from DNA sequencing data with significantly higher accuracy and efficiency than existing tools.
To the best of our knowledge, CNV-Net is the first tool to use CNN to detect CNVs from DNA sequencing data and achieve high performance on the standard benchmarks. It changes the problem of CNV detection from a process of statistical modeling requiring professional expertise into pro- cess of optimizing a general deep learning model. Besides, although we benchmark CNV-Net using only human samples in this study, it can also be applied to DNA sequencing data of other species.
One limitation of CNV-Net its need to know candidate CNV positions, potential start and end po- sitions of CNVs, before applying the CNN. This is because CNV-Net makes predictions pileup images generated at the start and end positions of candidate CNVs. In our analysis, we used candi- date CNV positions in human genomes, which were previously identified by the Genome in a Bottle Consortium using a suite of other algorithms. In genome sequencing datasets where CNVss have not been analyzed, these candidate CNV positions may not be available. One solution to this problem may be using candidate CNV positions detected by other CNV detection algorithms, especially al- gorithms with high recall, as CNV-Net can remove false positive CNVs from those CNV positions. Additionally, we may also develop sensitive candidate CNV detection algorithm, which may have low precision but high recall. Our future research direction involves identifying candidate CNV positions from genome sequencing datasets and incorporating them with CNV-Net; nevertheless, CNV-Net demonstrates potential as valuable and effective tool for CNV detection.
REFERENCES
Alexej Abyzov, Alexander E Urban, Michael Snyder, and Mark Gerstein. CNVnator: an approach to discover, genotype, and characterize typical and atypical CNVs from family and population genome sequencing. Genome Res., 21(6):974-984, June 2011.
Ken Chen, John W Wallis, Michael D McLellan, David E Larson, Joelle M Kalicki, Craig S Pohl, Sean D McGrath, Michael C Wendl, Qunyuan Zhang, Devin P Locke, Xiaoqi Shi, Robert S Fulton, Timothy J Ley, Richard K Wilson, Li Ding, and Elaine R Mardis. BreakDancer: an algorithm for high-resolution mapping of genomic structural variation. Nat. Methods, 6(9):677- 681. September 2009.
Xiaoyu Chen, Ole Trieglaf,, Richard Shaw, Bret Barnes, Felix Schlesinger, Morten Källberg, Anthony Cox, Semyon Kruglyak, and Christopher T Saunders. Manta: rapid detection of struc- tural variants and indels for germline and cancer sequencing applications. Bioinformatics, 32(8): 1220-1222, April 2016.
Colby Chiang, Alexandra J Scott, Joe R Davis, Emily K Tsang, Xin Li, Yungil Kim, Tarik Hadzic, Farhan Damani, Liron Ganel, GTEx Consortium, Stephen B Montgomery, Alexis Battle, Don- ald F Conrad, and Ira M Hall. The impact of structural variation on human gene expression. Nat. Genet., April 2017.
Denis Cuccaro, Elvira Valeria De Marco, Rita Cittadella, and Sebastiano Cavallaro. Copy number variants in alzheimer's disease. J. Alzheimers. Dis., 55(1)337-52, 2017.
E K Green, E Rees, J T R Walters, K-G Smith, L Forty, D Grozeva, J L Moran, P Sklar, S Ripke, K D Chambert, G Genovese, S A McCarroll, I Jones, L Jones, M J Owen, M C O' Donovan, N Craddock, and G Kirov. Copy number variation in bipolar disorder. Mol. Psychiatry, 21(1): 89-93, January 2016.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. December 2014.
Shunichi Kosugi. Yukihide Momozawa, Xiaoxi Liu, Chikashi Terao, Michiaki Kubo, and Yoichiro Kamatani. Comprehensive evaluation of structural variation detection algorithms for whole genome sequencing. Genome Biol,, 20(1):117, June 2019.
5

---

Under review as a conference paper at ICLR 2021
Ryan M Layer, Colby Chiang, Aaron R Quinlan, and Ira M Hall. LUMPY: probabilistic framework for structural variant discovery. Genome Biol., 15(6):884, June 2014.
Ruibang Luo, Fritz Sedlazeck, Tak- Wah Lam, and Michael C Schatz. A multi-task convolutional
deep neural network for variant calling in single molecule sequencing. Nat. Commun., 10(1):998, March 2019.
Matthew Pendleton, Robert Sehra, Andy Wing Chun Pang, Ajay Ummat, Oscar Franzen, Tobias Rausch, Adrian M Stütz, William Stedman, Thomas Anantharaman, Alex Hastie, Heng Dai, Markus Hsi- Yaig Fritz, Han Cao, Ariella Cohain, Gintaras Deikus, Russell E Durrett, Scott C Blanchard, Roger Altman, Chen-Shan Chin, Yan Guo, Ellen E Paxinos, Jan O Korbel, Robert B Darnell, W Richard McCombie, Pui- Yun Kwok, Christopher E Mason, Eric E Schadt, and Ali Bashir. Assembly and diploid architecture of individual human genome via single-molecule technologies. Nat. Methods, (2(8):80--786 August 2015.
Ryan Poplin, Pi-Chuan Chang, David Alexander, Scott Schwartz, Thomas Colthurst, Alexander Ku, Dan Newburger, Jojo Dijamco, Nam Nguyen, Pegah T Afshar, Sam S Gross, Lizzie Dorfman, Cory McLean, and Mark A DePristo. A universal SNP and small-indel variant caller using deep neural networks. Nat. Biotechnol., 36(10)983366) November 2018.
Tobias Rausch, Thomas Zichner, Andreas Schlattl, Adrian M Stütz, Vladimir Benes, and Jan O Korbel. DELLY: structural variant discovery by integrated paired-end and split-read analysis. Bioinformatics, 28( (1(18)1333--339) September 2012.
Jonathan Sebat, B Lakshmi, Dheeraj Malhotra, Jennifer Troge, Christa Lese-Martin, Tom Walsh, Boris Yamrom, Seungtai Yoon, Alex Krasnitz, Jude Kendall, Anthony Leotta, Deepa Pai, Ray Zhang, Yoon-Ha Lee, James Hicks, Sarah J Spence, Annette T Lee, Kaija Puura, Terho Lehtimäki, David Ledbetter, Peter K Gregersen, Joel Bregman, James S Sutcliffe, Vaidehi Jobanputra, Wendy Chung, Dorothy Warburton, Mary-Claire King, David Skuse, Daniel H Geschwind, T Conrad Gilliam, Kenny Ye, and Michael Wigler. Strong association of de novo copy number mutations with autism. Science, 316(5823):445 449, April 2007.
Hreinn Stefansson, Dan Rujescu, Sven Cichon, Olli P H Pietiläinen, Andres Ingason, Stacy Stein- berg, Ragnheidur Fossdal, Engilbert Sigurdsson, Thordur Sigmundsson, Jacobine E Buizer- Voskamp, Thomas Hansen, Klaus D Jakobsen, Pierandrea Muglia, Clyde Francks, Paul M Matthews, Arnaldur Gylfason, Bjarni V Halldorsson, Daniel Gudbjartsson, Thorgeir E Thorgeirs- son, Asgeir Sigurdsson, Adalbjorg Jonasdottir, Aslaug Jonasdottir, Asgeir Bjornsson, Sigurborg Mattiasdottir, Thorarinn Blondal, Magnus Haraldsson, Brynja B Magnusdottir, Ina Giegling, Hans-Jürgen Möller, Annette Hartmann, Kevin V Shianna, Dongliang Ge, Anna C Need, Car- oline Crombie, Gillian Fraser, Nicholas Walker, Jouko Lonnqvist, Jaana Suvisaari, Annamarie Henrio-Hnnikssson, Tiina Paunio, Timi Toulopoulou. Elvira Bramon, Marta Di Forti, Robin Mur- ray, Mirella Ruggeri, Evangelos Vassos, Sarah Tosato, Muriel Walshe, Tao Li, Catalina Vasilescu, Thomas W Mühleisen, August Wang, Henrik Ullum, Srdjan Djurovic, Ingrid Melle, Jes Olesen, Lambertus A Kiemeney, Barbara Franke, GROUP, Chiara Sabatti, Nelson B Freimer, Jeffrey R Gulcher, Unnur Thorsteinsdottir, Augustine Kong, Ole A Andreassen, Roel A Ophoff, Alexander Georgi, Marcella Rietschel, Thomas Werge, Hannes Petursson, David B Goldstein, Markus M Nöthen, Leena Peltonen, David Collier, David Clair, and Kari Stefansson. Large recurrent microdeletions associated with schizophrenia. Nature, 455(7210):232 236, September 2008.
Mehdi Zarrei, Jeffrey R MacDonald, Daniele Merico, and Stephen W Scherer. A copy number variation map of the human genome. Nat. Rew. Genet., 17(31: March 2015.
Justin M Zook, Nancy F Hansen, Nathan D Olson, Lesley Chapman, James C Mullikin, Chunlin Xiao, Stephen Sherry, Sergey Koren, Adam M Phillippy, Paul C Boutros, Sayed Mohammad E Mason, Iman
Sahraeian, Vincent Huang, Alexandre Rouette, Noah Alexander, Christopher Hajirasouliha, Camir Ricketts, Joyce Lee, Rick Tearle, Ian T Fiddes, Alvaro Martinez Barrio, Jeremiah Wala, Andrew Carroll, Noushin Ghaffari, Oscar Rodriguez, Ali Bashir, Shaun Jack- man, John J Farrell, Aaron M Wenger, Can Alkan, Arda Soylev, Michael C Schatz, Shilpa Garg, George Church, Tobias Marschall, Ken Chen, Xian Fan, Adam C English, Jeffrey A Rosenfeld, Weichen Zhou, Ryan E Mills. Jay M Sage, Jennifer R Davis, Michael D Kaiser, John S Oliver, Anthony Catalano, Mark P Chaisson, Noah Spies, Fritz Sedlazeck, and Mare Salit. robust benchmark for detection of germline large deletions and insertions. Nat. Biotechnol,, June 2020.


Reviews:
[{"review_id": "lG8CleiuM2D", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "##########################################################################\n\nSummary:\nThe authors proposed CNV-Net, a deep learning-based approach for copy number variation identification. They encoded mapped DNA sequences into a pileup image that captures reference sequence, sequencing coverage, and mapped reads. Then, they used CNNs to classify it into deletions, duplications, or non-breakpoints. They benchmarked CNV-Net with two whole-genome sequencing datasets and claimed to obtain more accurate and efficient results than current tools.\n\n\n##########################################################################\n\nMajor comments:\nWhile the paper has its own merits, unfortunately, it has several issues that need to be addressed.\n- Although the authors claimed that CNV-Net is the first tool to use a CNN to detect CNVs, this is not true. Several previous works used CNNs for CNV detection. Furthermore, I think other machine learning and deep learning-based works should also be acknowledged. I’d recommend authors properly cite and compare previous works with the proposed method. Some of the previous works include the followings:\n(1) Zhang, Yun Xiang, et al. \"DL-CNV: A deep learning method for identifying copy number variations based on next generation target sequencing.\" Mathematical biosciences and engineering: MBE 17.1 (2019): 202-215.\n(2) Cai, Lei, Yufeng Wu, and Jingyang Gao. \"DeepSV: accurate calling of genomic deletions from high-throughput sequencing data using deep convolutional neural network.\" BMC bioinformatics 20.1 (2019): 665.\n(3) Hill, Tom, and Robert L. Unckless. \"A deep learning approach for detecting copy number variation in next-generation sequencing data.\" G3: Genes, Genomes, Genetics 9.11 (2019): 3575-3582.\n(4) Pounraja, Vijay Kumar, et al. \"A machine-learning approach for accurate detection of copy number variants from exome sequencing.\" Genome research 29.7 (2019): 1134-1143.\n- The main contribution of the paper would be using a pileup image of mapped reads and a CNN to detect CNVs. However, in my view, the novelty of the paper is quite limited. As stated in the introduction, the pileup image encoding and CNNs have already been used in a couple of previous works for SNV detection. I could find any significant methodological differences in CNV and SNV detections; it seems like rather a straightforward application of previous methods on another similar problem. Otherwise, please clarify the differences between the two problems and what authors have done to overcome the new obstacles.\n- The core issue I have with this paper is that I do not think the experiment settings are realistic. As stated by the authors, CNV-Net must know the candidate CNV positions. I think this is a serious issue that must be handled rather than leaving it as a limitation of the work. Currently, the CNV-Net is evaluated with mapped and pre-preprocessed reads with CNV centered breakpoints. Compared to the experiments conducted in the previous works, the experiments of the proposed work seem limited, unrealistic, and biased in favor of the proposed method. \n- In my view, the authors left out too much information. For examples, it is quite difficult to understand how they used other tools for the experiments; Do they only use the CNV breakpoints that passed the quality control filter as CNV-Net? Or do they use all the mapped reads? What tool-specific arguments did the authors use for each tool? Currently, it is extremely difficult to reproduce the results presented in the paper.\n\n##########################################################################\n\nMinor comments:\n- How did the authors choose the specific numbers to encode individual base into R channel (e.g. A with 250, G with 180)\n- In the results section, the authors stated that they only used CNVs passing the quality control filter. Please provide more details for the filter explaining the filtering criteria and how they chose them.\n- In Table 2, how did the authors obtain the metrics for the multi-class problem? Please state whether they are macro or micro averages of scores.\n\n##########################################################################\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "hYTP2pyQfUb", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "__Summary__\nThey authors describe a method to detect structural variation from aligned sequencing reads in a genome browser view. Their model encodes this genome browser view into an RGB image and applies a deep convolutional neural network to classify variant type (or no variant). They make use of curated variant annotations to train and test their model.\n\n__Major comments__\n* The RGB encoding is entirely arbitrary, unnecessary, and confusing. The authors should consider the actual range of the various input data and encode with a simple and interpretable strategy. For example, nucleotides are typically one hot encoded.\n\n* The improved accuracy on this task is abstract when only summarized in tables. Depicting an example of a structural variant whose prediction is improved by the authors’ method would be very valuable. Ideally, both a false positive turned true negative and a false negative turned true positive could be shown.\n\n* The requirement that this method be provided candidate structural variation start and end points means that it's actually a module that would need to be plugged into a pipeline that also specified how those candidates are obtained. I would encourage the authors to develop that strategy before publishing their method.\n\n* The authors have not clearly described how the predictions of the other methods were used to annotate these curated variants. Doing so involves critical parameters, such as the allowed distance between the method prediction and true specified variant break points. Setting these parameters to strict values would be very unfair given that the authors method is not required to produce such breakpoints de novo. The Zook et al. paper describes this process in detail and discusses several software packages to do it.\n\n__Minor comments__\n* Does the authors’ method make use of paired end read information?\n* How does Table 2 combine the accuracies for duplications and indels?", "extra_scores": {}, "extra_text": {}}, {"review_id": "bV13DskJMfC", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors in the paper describe a deep learning approach to detect copy number variants (CNVs) from DNA sequencing data, CNV-Net.  It described the approach by transforming the pileups into images and pass them through a CNN. This strategy has been proposed four or five years back to do SNPs and indels calling (DeepVariant).  It is challenging for SVs and CNVs as they can be arbitrarily large. However, there are also several existing models for this task (e.g. DeepSV, RDBKE). In this work, the authors only consider \"candidate CNV regions\" which are 201-bp small genomic regions that centered at the breakpoints. \n\nThe paper is only 4.5 pages. First, the authors did not explain or investigate many decisions in their model design. Then the experiments are flawed. I have many questions and concerns. \n1) First of all, this should not be called a CNV detector because it is in fact a breakpoint detector\n2) How did the authors do negative sampling? Were the negative samples randomly drawn from the genome? This creates bias as the sequence features might shift dramatically. \n3) Why are the negative samples not balanced with the positive samples?\n4) Are there differences in performance between deletion and duplications breakpoints? Does the model distinguish these two types? \n5) When splitting the training/validation/testing dataset, there will be significant data leakage if this is done randomly. Many SV/CNV regions have characteristic repeats patterns.\n6) The comparison between CNV-Net and other methods is not fair as the other methods are finding CNVs in the whole genome while CNV-Net is given a pre-defined set of \"candidate breakpoint regions\" in which positive samples have breakpoints perfectly centered in the middle. Moreover, such candidate regions might suffer from data leakage. \n7) Does HG002 really only has 174 duplications while NA12878 has 22,936? \n8) GIAB has more than two genomes available. Can we test on more genomes? \n9) Minor: RELU -> ReLU \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "3eVcbqOxWPe", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors present an innovative approach to CNV detection using CNNs. However, I believe that additional experiments need to performed before this paper is ready for publication. Below is a breakdown of strengths and weaknesses of this submission. \n\nStrengths: \n1. The empirical results seem encouraging compared to baselines. \n2. The method seems easy to implement and runs very fast. \n\nCons: \n\n1. There is only one comparison performed. I was wondering what the results are like in the opposite direction (training in HG002 and testing on NA12878)? Furthermore, the authors should consider other datasets, in particular the TCGA data where bam files, and copy number variations are available. The authors can train the method on one patient and test on another. \n\n2. There is lack of analysis on whether the method can recover known canonical copy number variations. In the TCGA dataset, and for certain cancer types, there are small and large copy-number variations that are known and have been validated in the lab. The authors should ensure that their method can recover these events using their method. \n\n3. The authors have considered baselines that do not take as input already known break points, which makes the comparison somewhat unfair. I was wondering if there is a way in which the authors can use the discovered breakpoints for each respective baseline and then determine whether their algorithm can improve the predictions? ", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Four knowledgeable referees have indicated reject. I agree with the most critical reviewer R4 that the model design lacks a clear and transparent motivation and that the experimental setup is not convincing, and so must reject.
Decision: Reject