Package: digcnv
DigCNVA machine learning model to quality control genotyped CNVsFalse CNVs bring noise to analysis and could distort a diagnosis.
CNV calling produce false negative and positive identifications.
To remove false negative it's recommended to use multiple CNV caller at a time.
So we present here, a statistical approach to clean CNV results coming from two calling algorithms,PennCNVorQuantiSNPThis machine learning can be used in two different ways:a first one-line script to annotate any new CNVs on our pre-trained model. Trained on 38,000 CNVs coming from 7 different genotyping technologies.
This model have AUC > 90% for most technologies (already trained-on or  new technologies)a second approach line by line to train and test our machine learning model on your own visualized dataset and performe some statistics.DigCNV modelsModel presented in IAMDRS congress available at:https://murena.io/s/xEsyae6gxfMEnWJMost up-to-date model available athttps://murena.io/s/xEsyae6gxfMEnWJClassify CNVs with one line# To run only once to install packagepipinstalldigcnv

python3-mdigcnv[-v]<Pathtoconfigfile>Config fileExample of config file needed for the one line execution. Example can be download with functiongetConfigFileExample(output_path)[Inputs]
pc_output_path = Path to the PennCNV output file
pc_qc_path = Path to the PennCNV microarray quality file 
qs_output_path = Path to the QuantiSNP output file


[Output]
Save_to_file = True
Output_path = /home/thomas/Documents/scripts/DigCNV/temp_data/DigCNV_pred.tsv

[DigCNV]
model_path": Path of the downloaded model. Available at :Run DigCNV with more optionsPrepare datafromdigcnvimportCNVision,dataPreparation# Import and merge CNV coming from two CNV calling algorthims (only PennCNV and QuantiSNP in this version)cnvs=CNVision.mergeMultipleCNVCallingOutputs("<list of PennCNV and QuantiSNP output pathways>",["PennCNV","QuantiSNP"])# Add microarray quality data to the list of CNVscnvs=dataPreparation.addMicroArrayQualityData(cnvs,"<path to the PennCNV microarray quality file>")# Compute derived features and add it to the CNV listcnvs=dataPreparation.addDerivedFeatures(cnvs)# Add CallRate data to CNV listcnvs=dataPreparation.addCallRateToDataset(cnvs,call_rate_path="<Pathway to the callrate file>",callrate_colname="<CallRate column name>",individual_colname="<Individual column name>")# Add Chromosomic information such as centromere and Segmental Duplications overlapcnvs=dataPreparation.addChromosomicAnnotation(cnvs)# Add number of probes used with the technology (Useful for model trained on multiple datasets and multiple technologies)cnvs=dataPreparation.addNbProbeByTech(cnvs,pfb_file_path="<Pathway to the PFB file>")Run DigCNV from the pre-trained modelMore information at"article"fromdigcnvimportdigCnvModel,dataVerif# Create an empty DigCNV modelmodel=digCnvModel.DigCnvModel()# Open pre-trained model and update objectmodel_path=join(split(__file__)[0],'data','DigCNV_model_multiple_technos.pkl')model.openPreTrainedDigCnvModel(model_path)# Check if mandatory columns for the DigCNV model exist and have right formatsdataVerif.checkIfMandatoryColumnsExist(cnvs,post_data_preparation=True)dataVerif.checkColumnsformats(cnvs,post_data_preparation=True)# Optional plot a correlation heatmap between different predictors used in modeldataVerif.plotCorrelationHeatMap(cnvs,list_dim=model._dimensions,output_path="<Pathway where output plot (.pdf or .png)>")# Check NaN data within mandatory columns and split data into two dataframes: first for CNVs with all information available# and a second one with all CNVs with at least one missing data (can't be used for prediction)cnvs,cnvs_with_na=dataVerif.computeNaPercentage(cnvs,dimensions=model._dimensions,remove_na_data=True)# Discriminate true from false CNVs from CNVs with all data, then produce a list of classespredicted_cnvs=model.predictCnvClasses(cnvs)cnvs["DigCNVpred"]=predicted_cnvsTrain your own DigCNV modelfromdigcnvimportdigCNVModel# Uses CNVs created in Prepare data section :# Add your own annotation to your CNVs (0 for false CNVs and 1 for true)cnvs["visualized_class"]=["<classes of each CNV visualized>"]# Dimensions used to classify CNVs (Recommended dimensions you can use your own)predictors=["",""]# Remove CNVs with at least one missing values in used predictors or in visualized columncnvs,removed=DigCnvPreProcessing.removeLinesWithNA(cnvs,dimensions=predictors+["visualized_class"])# Split dataset into two groups a training dataset and a testing dataset (70% - 30%)X_train,y_train,X_test,y_test=DigCnvPreProcessing.createTrainingTestingDatasets(cnvs,X_dimension="visualized_class")# If ratio between the two classes is too unbalanced uniformize classes by split majoritary class and adding new pseudo CNVs to minority classX_train,y_train=DigCnvPreProcessing.uniformizeClassesSizes(X_train,y_train,17,0.4,0.5)# Create a DigCNV modelmodel=digCnvModel.DigCnvModel()model.createDigCnvClassifier()# Train the DigCNV model with the given training datasetmodel.trainDigCnvModel(training_data=X_train,training_cat=y_train)# Analyse classification accuracy for the trained modelpredicted_cnvs=model.evaluateCnvClassification(testing_df=X_test,expected_values=y_test,images_dir_path="<Path to the image output directory>")Tune DigCNV hyperparametersfromdigcnvimportDigCNVTunningInput files format examplePennCNV outputchr20:44356194-44378577       numsnp=7      length=22,384      state2,cn=1 /path/to/finalreport/10001 startsnp=rs232258 endsnp=rs380421 conf=16.163
chr9:17583310-17622213        numsnp=21     length=38,904      state5,cn=3 /path/to/finalreport/10001 startsnp=rs1028594 endsnp=rs3808750 conf=101.052
chr10:47543322-47703613       numsnp=47     length=160,292     state5,cn=3 /path/to/finalreport/10001 startsnp=rs11259779 endsnp=rs4128664 conf=156.227
chr6:4263349-4472587          numsnp=69     length=209,239     state2,cn=1 /path/to/finalreport/10002 startsnp=rs6937085 endsnp=rs7746329 conf=120.225
chr6:80608294-80611616        numsnp=6      length=3,323       state2,cn=1 /path/to/finalreport/10002 startsnp=rs17833835 endsnp=rs1887571 conf=20.441
...PennCNV quality outputSampleID        LRR_mean        LRR_median      LRR_SD  BAF_mean        BAF_median      BAF_SD  BAF_DRIFT       WF      GCWF
10001   -0.0045 0.0000  0.1474  0.5028  0.5000  0.0268  0.000036        -0.0141 -0.0002
10002   0.0056  0.0000  0.1588  0.5027  0.5000  0.0259  0.000000        0.0194  0.0018
10003   -0.0090 0.0000  0.1564  0.5031  0.5000  0.0308  0.000000        0.0132  0.0053
10004   0.0014  0.0000  0.1494  0.5032  0.5000  0.0280  0.000000        0.0184  0.0018
10005   0.0031  0.0000  0.1412  0.5025  0.5000  0.0272  0.000000        0.0170  0.0001
...QuantiSNP outputSample Name     Chromosome      Start Position (bp)     End Position (bp)       Start Probe ID  End Probe ID    Length (bp)     No. Probes      Copy Number     Max. Log BF     Log BF: State 0 Log BF: State 1 Log BF: State 2 Log BF: State 3     Log BF: State 4 Log BF: State 5 Log BF: State 6
10001   1              31943355        31943355 rs7545865       rs7545865                     1 1       1       1.2152  -24.0258        -2.5507 0       1.2152  -16.1517        -19.4123        -25.013
10001   1             111930916       111934304 rs12077338      rs4839132                  3389 4       1       4.57286 -42.9409        4.57286 0       2.93527 -29.2398        -35.802 -44.9483
10001   2              44082362        44096010 rs6718187       rs6752551                 13649 4       3       0.954579        -62.543 -41.1676        0       -14.003 0.954579        0.576184        -60.5049
10001   2              92308395        92308395 rs4509760       rs4509760                     1 1       1       0.0218224       -26.3579        -4.38718        0       0.0218224       -19.0042        -21.9322        -27.2609
10001   3              59820539        59821071 rs1905866       rs17362486                  533 2       4       0.33986 -9.54639        -22.8223        0       -3.94108        -6.15468        0.33986 -8.45724
...