Package: variant-extractor
VariantExtractorDeterministic and standard extractor of indels, SNVs and structural variants (SVs)from VCF files built under the frame ofEUCANCan's second work package. VariantExtractor is a Python package (requires Python version 3.6 or higher) and provides a set of data structures and functions to extract variants from VCF files in adeterministic and standardway whileadding informationto facilitate afterwards processing. It homogenizesmultiallelic variants,MNPsandSVs(includingimprecise paired breakendsandsingle breakends). The package is designed to be used in a pipeline, where the variants are ingested from VCF files and then used in downstream analysis. Check theavailable documentationfor more information.While there is somewhat of an agreement on how to label the SNVs and indels variants, this is not the case for the structural variants. In the current scenario, different labeling between variant callers makes comparisons between structural variants difficult. This package provides an unified interface to extract variants (included structural variants) from VCFs generated by different variant callers. Apart from reading the VCF file, VariantExtractoradds a preprocessing layer to homogenize the variantsextracted from the file. This way, the variants can be used in downstream analysis in a consistent way. For more information about the homogenization process, check thehomogenization rulessection.Table of contentsGetting startedInstallationUsageVariantRecordVariantTypeBreakendSVRecordShorthandSVRecordHomogenization rulesMultiallelic variantsSNVsStructural variantsBreakend vs shorthand notationPaired breakendsInferred breakend pairsImprecise paired breakendsSingle breakendsDependenciesGetting startedInstallationVariantExtractor is available on PyPI and can be installed usingpip:pipinstallvariant-extractorUsage# Import the packagefromvariant_extractorimportVariantExtractor# Create a new instance of the classextractor=VariantExtractor('/path/to/file.vcf')# Iterate through the variantsforvariant_recordinextractor:print(f'Found variant of type{variant_record.variant_type.name}:{variant_record.contig}:{variant_record.pos}')# Import the packagefromvariant_extractorimportVariantExtractor# Create a new instance of the classextractor=VariantExtractor('/path/to/file.vcf')# Save variants to a CSV fileextractor.to_dataframe().drop(['variant_record_obj'],axis=1).to_csv('/path/to/output.csv',index=False)For a more complete list of examples, check theexamplesdirectory. This folder also includes an example of ascript for normalizing VCF filesfollowing thehomogenization rules.VariantRecordTheVariantExtractorconstructor returns a generator ofVariantRecordinstances. TheVariantRecordclass is a container for the information contained in a VCF record plus some extra useful information.PropertyTypeDescriptioncontigstrContig nameposintPosition on the contigendintEnd position of the variant in the contig (same asposfor TRN and SNV)lengthintLength of the variantidOptional[str]Record identifierrefstrReference sequencealtstrAlternative sequencequalOptional[float]Quality score for the assertion made in ALTfilterList[str]Filter status.PASSif this position has passed all filters. Otherwise, it contains the filters that failedinfoDict[str, Any]Additional informationformatList[str]Specifies data types and order of the genotype informationsamplesDict[str, Dict[str, Any]]Genotype information for each samplevariant_typeVariantTypeVariant type inferredalt_sv_breakendOptional[BreakendSVRecord]Breakend SV info, present only for SVs with breakend notation. For example,G]17:198982]alt_sv_shorthandOptional[ShorthandSVRecord]Shorthand SV info, present only for SVs with shorthand notation. For example,<DUP:TANDEM>VariantTypeTheVariantTypeenum describes the type of the variant. For structural variants, it is inferredonlyfrom the breakend notation (or shorthand notation). It does not take into account anyINFOfield (SVTYPEnorEVENTYPE) that might be added by the variant caller afterwards.REFALTVariant nameDescriptionAGSNVSingle nucleotide variantAGTGADELDeletionAA[1:20[ or <DEL>DELDeletionAACCT or <INS>INSInsertionA]1:20]A or <DUP>DUPDuplicationAA]1:20] or [1:20[AINVInversion.<INV> is a special caseA<CNV>CNVCopy number variationAA]X:20] or A[X:20[ or ]X:20]A or [X:20[ATRNTranslocationAA. or .ASGLSingle breakendBreakendSVRecordTheBreakendSVRecordclass is a container for the information contained in a VCF record for SVs with breakend notation.PropertyTypeDescriptionprefixOptional[str]Prefix of the SV record with breakend notation. For example, forG]17:198982]the prefix will beGbracketstrBracket of the SV record with breakend notation. For example, forG]17:198982]the bracket will be]contigstrContig of the SV record with breakend notation. For example, forG]17:198982]the contig will be17posintPosition of the SV record with breakend notation. For example, forG]17:198982]the position will be198982suffixOptional[str]Suffix of the SV record with breakend notation. For example, forG]17:198982]the suffix will beNoneShorthandSVRecordTheShorthandSVRecordclass is a container for the information contained in a VCF record for SVs with shorthand notation.PropertyTypeDescriptiontypestrType of the SV record with shorthand notation. One of the following,'DEL','INS','DUP','INV'or'CNV'. For example, for<DUP:TANDEM>the type will beDUPextraList[str]Extra information of the SV. For example, for<DUP:TANDEM:AA>the extra will be['TANDEM', 'AA']Homogenization rulesVariantExtractor provides a unified interface to extract variants (included structural variants) from VCF files generated by different variant callers. The variants are homogenized and returned applying the following rules:Multiallelic variantsAn entry with multipleALTsequences (multiallelic) is divided into multiple entries with a singleALTfield. This entries with a singleALTfield are then processed with the rest of the homogeneization rules. For example:CHROMPOSIDREFALTFILTER21multiallelic_1GC,TPASSis returned as:CHROMPOSIDREFALTFILTERVariantType21multiallelic_1_0GCPASSSNV21multiallelic_1_1GTPASSSNVSNVsEntries withREF/ALTof the same lenghts are treated like SNVs. If theREF/ALTsequences are more than one nucleotide (MNPs), they are divided into multiple atomic SNVs. For example:CHROMPOSIDREFALTFILTER21snv_1CGPASS23mnp_1TAGAGTPASSare returned as:CHROMPOSIDREFALTFILTERVariantType23snv_1CGPASSSNV23mnp_1_0TAPASSSNV24mnp_1_1AGPASSSNV25mnp_1_2GTPASSSNV|Structural variantsVariantExtractor returns one entry per structural variant (one entry per breakend pair). This helps to avoid the ambiguity of the notation and keeps the process deterministic. For this reason, in case of paired breakends, the breakend with the lowest chromosome and/or position is returned. If a breakend is not the lowest chromosome and/or position and is missing its pair, its pair isinferred and returned.Breakend vs shorthand notationEntries with the same information, either described with shorthand or breakend notation, will be returned the same way. Here is an example for a DEL entry:CHROMPOSIDREFALTFILTERINFO13000event_1_oAA[1:5000[PASSSVTYPE=BND15000event_1_hA]1:3000]APASSSVTYPE=BND13000event_1AA[1:5000[PASSSVTYPE=DEL13000event_1A<DEL>PASSSVTYPE=DEL; END=500013000event_1AGTCACAA...APASSare returned as one entry (each one of them with their ownALTfield), but with the sameVariantRecord.endandVariantType:CHROMPOSIDREFALTFILTERINFOVariantTypeVariantRecord.end13000event_1A...PASS...DEL5000The special case of INV<INV> is a special case of shorthand notation because it represents two paired breakends. For example, the following shorthand notation:CHROMPOSIDREFALTFILTERINFO2321682event_1T<INV>PASSSVTYPE=INV;END=421681is equivalent to the following breakends:CHROMPOSIDREFALTFILTERINFOVariantType2321681event_1_0NN]2:421681]PASSSVTYPE=INVINV2321682event_1_1T[2:421682[TPASSSVTYPE=INVINVIn this case, VariantExtractor converts internally <INV> to two entries with breakend notation (one for each breakend pair). Note that theNwill be replaced with the correct nucleotide iffasta_refis provided to VariantExtractor.Paired breakendsForpaired breakends, breakends are paired using theINFOfieldsMATEIDorPARID. If these fields are not available, they are paired using their coordinates (contig+position). The breakend with the lowest chromosome and/or position is returned. For example:CHROMPOSIDREFALTFILTERINFO23000event_1_oT]3:5000]TPASSSVTYPE=BND35000event_1_hGG[2:3000[PASSSVTYPE=BND13000event_2_oAA[1:5000[PASSSVTYPE=BND15000event_2_hA]1:3000]APASSSVTYPE=BNDare returned as one entry per variant:CHROMPOSIDREFALTFILTERINFOVariantType23000event_1_oT]3:5000]TPASSSVTYPE=BNDTRN13000event_2_oAA[1:5000[PASSSVTYPE=BNDDELInferred breakend pairsIfallbreakends are missing their pair, the breakends with the lowest chromosome and/or position are inferred and returned. For example:CHROMPOSIDREFALTFILTERINFO35000event_1_hGG[2:3000[PASSSVTYPE=BND15000event_2_hA]1:3000]APASSSVTYPE=BNDare returned as their inferred breakend pair with the lowest chromosome and/or position:CHROMPOSIDREFALTFILTERINFOVariantType23000event_1_hN]3:5000]NPASSSVTYPE=BNDTRN13000event_2_hAA[1:5000[PASSSVTYPE=BNDDELNote that theNwill be replaced with the correct nucleotide iffasta_refis provided to VariantExtractor. The following equivalencies are applied:CHROM1POS1REF1ALT1CHROM2POS2REF2ALT21500NN[7:800[7800N]1:500]N1500N]7:800]N7800NN[1:500[1500N[7:800[N7800N[1:500[N1500NN]7:800]7800NN]1:500]Imprecise paired breakendsImprecise breakends do not match exactly with their pair in coordinates. In this case, they are paired using theINFOfieldsMATEIDorPARID. As with the rest of variants, for each breakend pair, only the breakend with the lowest chromosome and/or position is returned. However, it is important to notice that theCIPOSfield is lost for the other breakend. For example:CHROMPOSIDREFALTFILTERINFO23010event_1_oTT[3:5000[PASSSVTYPE=BND;CIPOS=0,50;PARID=event_1_h35050event_1_hA]2:3050]APASSSVTYPE=BND;CIPOS=0,100;PARID=event_1_oare paired and the entry with the lowest chromosome and/or position is returned:CHROMPOSIDREFALTFILTERINFOVariantType23010event_1_oTT[3:5000[PASSSVTYPE=BND;CIPOS=0,50;PARID=a_hTRNSingle breakendsSingle breakends cannot be matched with other breakends because they lack a mate. They may be able to be matched later in downstream analysis. That is why each one is kept as a different variant. For example:CHROMPOSIDREFALTFILTERINFO23000event_sTT.PASSSVTYPE=BND35000event_mG.GPASSSVTYPE=BNDare returned as two entries:CHROMPOSIDREFALTFILTERINFOVariantType23000event_sTT.PASSSVTYPE=BNDSGL35000event_mG.GPASSSVTYPE=BNDSGLDependenciesThe dependencies are covered by their own respective licenses as follows:Python/Pysam package(MIT license)