DOI: 10.6078/d1gm63
title: Source code for StrVCTVRE: a supervised learning method to predict the pathogenicity of human genome structural variants

description: Whole genome sequencing resolves many clinical cases where standard
 diagnostic methods have failed. However, at least half of these cases
 remain unresolved after whole genome sequencing. Structural variants (SVs;
 genomic variants larger than 50 base pairs) of uncertain significance are
 the genetic cause of a portion of these unresolved cases. As sequencing
 methods using long or linked reads become more accessible and SV detection
 algorithms improve, clinicians and researchers are gaining access to
 thousands of reliable SVs of unknown disease relevance. Methods
 to predict the pathogenicity of these SVs are required to realize the full
 diagnostic potential of long-read sequencing. To address this emerging
 need, we developed StrVCTVRE to distinguish pathogenic SVs from benign SVs
 that overlap exons. In a random forest classifier, we integrated features
 that capture gene importance, coding region, conservation, expression, and
 exon structure. We found that features such as expression and conservation
 are important but are absent from SV classification guidelines. We
 leveraged multiple resources to construct a size-matched training set of
 rare, putatively benign and pathogenic SVs. StrVCTVRE performs accurately
 across a wide SV size range on independent test sets, which will allow
 clinicians and researchers to eliminate about half of SVs from
 consideration while retaining a 90% sensitivity. We anticipate clinicians
 and researchers will use StrVCTVRE to prioritize SVs in patients where no
 SV is immediately compelling, empowering deeper investigation into novel
 SVs to resolve cases and understand new mechanisms of disease. StrVCTVRE
 runs rapidly and is available at
 https://compbio.berkeley.edu/proj/strvctvre/.
description: Thank you for downloading the StrVCTVRE source code! StrVCTVRE
 was developed with Jupyter notebook, so most code is in ipynb files. I
 recommend running these files using jupyter, which can easily be installed
 in conda. The notebooks were developed in a python 2 environment, and
 likely will not work in a python 3 environment.  If you
 are interested in using pre-trained StrVCTVRE algorithm to score SVs,
 please see https://compbio.berkeley.edu/proj/strvctvre/ for instructions
 to run StrVCTVRE on your SVs. The following is required
 to run these notebooks. Python packages:<br> -
 numpy <br> - pandas <br> - joblib <br> -
 scikit-learn <br> - pybedtools<br> - cyvcf2<br> -
 pybigwig <br> - matplotlib <br> - scipy <br> -
 pprint<br> - pickle<br> Linux environment with:<br> -
 bedtools<br> - SVScore (optional; see below)<br> - VEP
 (optional; see below)<br> - AnnotSV (optional; see below)<br>
 - X-CNV (optional; see below)<br> - CADD-SV (optional; see
 below) Since there is some complexity to installing and
 running the SV annotation methods, they are optional, and I have provided
 post-annotation files so these methods do not need to be run. 
 If you prefer to run the SV annotation methods from scratch, you
 will need to uncomment the sections of the analysis where they are run.
   Due to the several methods being run, most of the
 paths in the code have been hardcoded. These can either be replaced with
 your path of preference, or, perhaps preferably, you can create the file
 path /data/andrewsharo-S/thesis/Aim1 on your own system and put all files
 in that folder. Additionally, all figures will output to
 /h/andrewsharo/JupyterProjects/Thesis/Aim1/SVClassifier/Figs/, so it may
 be easiest to create that path on your system.   To
 recreate the entire manuscript, the notebooks must be run in the following
 order:<br> DataSelectionFinal.ipynb<br>
 MultipleTrainingSets-ClinVarOnly.ipynb<br>
 MultipleTrainingSets-5Categories.ipynb<br>
 CorrelationMatrix-v2.ipynb<br> RawTestOnChrs1357.ipynb<br>
 GeneDistributionAnalysis-v4(2020ClinVar)-ForShamil-OneVarPerGene.ipynb<br> MultipleTrainingSets-ClinVarOnly-geneNormalizedOneVarPerGene.ipynb<br> MultipleTrainingSets-5Categories-rawTestOnNormalizedOneVarPerGene-chrs1357.ipynb<br> CompareToSVScoreAndVeP-v3-ClinVar2020-chrs1357-OneVarPerGene.ipynb<br> CompareToSVScoreAndVeP-v3-ClinVar2020-chrs1357.ipynb<br> MultipleTrainingSets-5Categories-ClinVar vs All Data.ipynb<br> MultipleTrainingSets-5Categories-AllChrs-OutputRFs.ipynb<br> readDecipherVariants-HighContribution-ForShamil.ipynb  (Decipher data must be requested)<br> MultipleTrainingSets-5Categories-AllChrs-TestDecipher.ipynb (Decipher data must be requested)<br> CompareToAnnotSV.ipynb<br> DominantRecessiveModel-ForShamil.ipynb<br> MultipleTrainingSets-ClinVarOnly-SizeDistributionFixed-DomRec-ForShamil.ipynb<br> MultipleTrainingSets-ClinVarOnly-SizeDistributionFixed-DomRec-ForShamil-ConfidentGenesOnly.ipynb<br> MultipleTrainingSets-5Categories-DomRec-ForShamil.ipynb<br> MultipleTrainingSets-5Categories-DomRec-ForShamil-OnlyConfidentGenes.ipynb<br> MultipleTrainingSets-5Categories-DomRec-ForShamil-UnifiedFigure.ipynb<br> AnalyzeODonnellData-v2.ipynb (full SVs must be requested from author)<br> GeneDistributionAnalysis-v2.ipynb<br> ThousandGenomeAnalysis.ipynb<br> Conservation Distribution.ipynb<br> RawTestOnChrs1357NoDedupNoDeCommonNoSizeLimit.ipynb<br> RawTestOnChrs1357&lt;1Mb.ipynb<br> RawTestOnChrs1357NoDedupNoDeCommon.ipynb<br> CompareSVScoreAndStrVCTVRE_Runtime.ipynb<br> MultipleTrainingSets-5Categories-geneNormalized-testonRaw-Reviewers.ipynb<br> MultipleTrainingSets-5Categories-ClinVar vs All Data-forReviewer.ipynb<br> Compare To CADD-SV and XCNV chrs1357.ipynb<br> MultipleTrainingSets-5Categories-ClinVar vs All Data-CADD-SV and X-CNV.ipynb<br> readDecipherVariants-HighContribution-ForShamil-XCNV-only.ipynb<br> readDecipherVariants-HighContribution-ForShamil-CADDSV.ipynb Here is a list of figures from the Sharo et al. manuscript mapped to their corresponding notebook:<br> 1a: CorrelationMatrix-v2.ipynb<br> 1b: MultipleTrainingSets-5Categories.ipynb<br> 2: MultipleTrainingSets-5Categories-ClinVar vs All Data.ipynb<br> 3: MultipleTrainingSets-5Categories.ipynb<br> 4: readDecipherVariants-HighContribution-ForShamil.ipynb  (Decipher data must be requested)<br> 5a: ThousandGenomeAnalysis.ipynb<br> 5b: readDecipherVariants-HighContribution-ForShamil.ipynb  (Decipher data must be requested)<br> S1: Conservation Distribution.ipynb<br> S2: GeneDistributionAnalysis-v2.ipynb<br> S3: DataSelectionFinal.ipynb<br> S4: MultipleTrainingSets-5Categories.ipynb<br> S5: CompareToSVScoreAndVeP-v3-ClinVar2020-chrs1357.ipynb<br> S6: MultipleTrainingSets-5Categories-ClinVar vs All Data-CADD-SV and X-CNV.ipynb<br> S7a: RawTestOnChrs1357NoDedupNoDeCommon.ipynb<br> S7b: RawTestOnChrs1357NoDedupNoDeCommonNoSizeLimit.ipynb<br> S7c: RawTestOnChrs1357&lt;1Mb.ipynb<br> S8: MultipleTrainingSets-5Categories-ClinVar vs All Data-forReviewer.ipynb<br> S9a: readDecipherVariants-HighContribution-ForShamil-XCNV-only.ipynb<br> S9b: readDecipherVariants-HighContribution-ForShamil-CADDSV.ipynb<br> S10a: readDecipherVariants-HighContribution-ForShamil-XCNV-only.ipynb<br> S10b: readDecipherVariants-HighContribution-ForShamil-CADDSV.ipynb<br> S11: CompareToAnnotSV.ipynb<br> S12: CompareSVScoreAndStrVCTVRE_Runtime.ipynb<br> S13: DominantRecessiveModel-ForShamil.ipynb<br> S14: MultipleTrainingSets-5Categories-DomRec-ForShamil-UnifiedFigure.ipynb To retrain StrVCTVRE on updated structural variants, run the following notebooks:<br> DataSelectionFinal.ipynb - provide new structural variants to this notebook.<br> MultipleTrainingSets-5Categories-AllChrs-OutputRFs.ipynb - The output of this notebook should replace the existing trained rf files.  List of other findings in the paper mapped to corresponding notebook:<br> All numbers relating to training data: DataSelectionFinal.ipynb <br> Matching algorithm for pathogenic and benign variants: MultipleTrainingSets-ClinVarOnly.ipynb, MultipleTrainingSets-5Categories.ipynb<br> Performance on recently discovered SVs from undiagnosed neuromuscular and retinal disorder cases: AnalyzeODonnellData-v2.ipynb (full SVs must be requested from author)<br> 1000 Genomes Project performance: ThousandGenomeAnalysis.ipynb<br> Determination of StrVCTVRE Odds Pathogenicity cut-off: MultipleTrainingSets-5Categories-AllChrs-TestDecipher.ipynb Directory of Data present:<br> 20191219_dbVar_pathogenic_NR_SV.formatted.sorted.bed - SVs trained on by AnnotSV<br> ALL.wgs.mergedSV.v8.20130502.svs.genotypes.END.only.SVScored.vcf - subset of 1KGP SVs, only dels and dups<br> ALL.wgs.mergedSV.v8.20130502.svs.genotypes.GRCh38.vcf.gz - all of 1KGP SVs<br> clinvar_atleast2_2plus_AD_fromShamil.tsv - list of confidently dominant genes<br> clinvar_atleast2_2plus_AR_fromShamil.tsv - list of confidently recessive genes<br> diagnoses_for_Andrew.tsv - public subset of pathogenic CMG SVs<br> exons_Appris_featurized_transcript_Chr1-Y_loeuf_domino_arad.bed - annotated list of exons<br> exons_Appris_featurized_transcript_Chr1-Y_loeuf_domino.bed - annotated list of exons<br> exons_Appris_featurized_transcript_Chr1-Y_loeuf.sorted.bed - annotated list of exons<br> exons_Appris_featurized_transcript_Chr1-Y.sorted.bed - annotated list of exons<br> exons_CDS_Chr1-Y.sorted.bed - annotated list of exons<br> fileList.txt - list of files in directory<br> forSVScore.2020.AllChrs.header.vcf - header file<br> forSVScore.header.scored.20200302.tsv - output file from SVScore<br> forSVScore.header.scored.20200302.vcf - output file from SVScore<br> forSVScore.header.scored.2020.oneVarPerGene.tsv - output file from SVScore<br> forSVScore.header.scored.2020.vcf - output file from SVScore<br> forSVScore.header.vcf - input file for SVScore<br> forSVScore.vcf - input file for SVScore<br> forVeP.GRCh38.header.vcf - input file for VeP<br> forVeP.GRCh38.header.vep.vcf - input file for VeP<br> forVeP.GRCh38.vcf - input file for VeP<br> fromLO.bed - liftover output file<br> fromLO.err - liftover error file<br> fromLO.nocomments.err - modified liftover error file<br> genes_Appris_Chr1-Y.bed - annotated list of genes<br> genes_Appris_Chr1-Y.sorted.bed - annotated list of genes<br> gnomad_v2_sv.sites.hg19.full.filtered.header.bed - subset of gnomAD SVs<br> gnomad_v2_sv.sites.hg38.bed - full gnomAD SVs<br> GRCh38_hg38_variants_2016-08-31.txt - Database of Genomic Variants SVs<br> great_ape_sv_gl_and_grch38b_EVA_gs.vcf.gz - Great Ape SVs<br> h - header file<br> hg19ToHg38.over.chain.gz - liftover file<br> hg38chromsizes.tsv - list of chromosomes ranked by size<br> hg38.phastCons100way.bw - phastCons scores<br> hg38.phyloP100way.bw - phyloP scores<br> hg38ToHg19.over.chain.gz - liftover file<br> liftOver - liftover program<br> path.header - header file<br> rep12tadsMergedhg38.bed - TADs used to annotate SVs<br> score_all_final_19.02.19.txt - Domino (dominant gene predictor) scores<br> summary_exon_usage_hg38.sorted.bed - exons annotated by exon usage and expression<br> variant_summary.txt.gz - ClinVar SVs Please direct all questions to sharo@berkeley.edu, brenner@compbio.berkeley.edu