Package: allein_zu_haus
This pacakge implements aNeedleman–Wunschglobal alignment withaffine gap penalties, taking into account the confidence in each base pair of the read sequence.The classic Needleman-Wunsch algorithm finds the optimal global match assuming all read nucleotides have been identified with certainty. In practice,NGS (next generation sequencing)identifies nucleotides with varying levels of qualities, and popular formats, e.g.,FASTQare expressely built for describing these qualities. This package modifies the algorithm to take qualities into account (seemathematical_rationalebelow).This package is designed to be used in conjunction with a read aligner such asBowtie2orGEM. Ideally, these tools would be configured to take basepair quality into account when searching for hits. Since this is currently not the case, one can use the allein_zu_haus package in order to realign reads to reference sequences at the positions reported by the read aligner, and extract more accurate alignment scores andCIGARstrings.The package handles ambiguous base codes by weighing the different options according to the base priors provided. Read sequences must be drawn from the {A, C, G, T, N} alphabet, but reference sequences may contain any basepairs defined by the IUPAC nucleotide ambiguity code. Quality values are ignored for ambiguous read basepairs.Usage example:Minimal Exampleimportallein_zu_hausimportnumpyasnpmax_read_len=100# Max length of read to be alignedmax_ref_len=2*max_read_len# Max length of reference subsequence to be globally aligned (where start position is determined by read aligner output)mismatch_penalty=4# Penalty for mismatched nucleotidesgap_open=6# Gep opening penaltygep_extend=3# Gap extension penaltyaligner=allein_zu_haus.Aligner(max_read_len,max_ref_len,mismatch_penalty,gap_open,gap_extend)read=np.array(['A','C','G','T','A'],dtype=bytes)# Read sequence, given as numpy array, dtype=bytesref=np.array(['A','G','G','T','A'],dtype=bytes)# Reference subsequence to be aligned, given as numpy array, dtype=bytesread_bp_probs=np.array([0.9,0.99,0.8,0.99,0.99])# Confidence in each read basepair. See below on how to extract such values from read quality stringsbase_probs=np.array([0.25]*4)# Basepair prior probabilities. Here, assuming uniform distribution on nucleotides.# See below example for more biologically relevant priorsmax_gaps=2# Maximal number of gaps allowed. Use small values to improve run timescore,cigar=aligner.match(read,read_bp_probs,base_probs,ref,max_gaps)#Aligner returns alignment score and CIGAR stringSetting Up Priors & QualitiesOne way of extracting more meaningful basepair priors is by using the nucleotide frequencies observed in reference sequences. Assuming _ref_seq is a string variable holding the reference genome of interest this can be done by:defget_priors():return[(k,v/float(len(_ref_seq)))for(k,v)incollections.Counter(_ref_seq).items()]Naturally, more intricate priors can easily be designed.Basepair probabilities cab be extracted from quality strings reported in FASTQ files / read aligner output using this formula:\begin{equation*}
1 - 10^\frac{-q}{10}
\end{equation*}where q is the ascii value of the quality character - 33 (assuming qualities are goven in Phred+33)defget_read_bp_probs(read_quality_string):# read quality_string: FASTQ quality string as reported in FASTQ file or read aligner outputreturn1-np.power(10,-np.array([ord(e)-33foreina])/10.)As you can see, read_p now holds per basepair probabilities (1 - P_error). For example, for quality string ‘??CII’ output will be [0.999, 0.999, 0.996, 0.9999, 0.9999]Aligning multiple readsThis package is optimized for multiple alignments, e.g., when iterating over the results ofGEMor a FASTQ file, and checking the score of each result relative to some corresponding reference subsequence. For this reason, to use it, first create an object with the parameters relevant to all matches.
Since reads usually have similar lengths, and since read aligners provide the position in the reference sequence to which the read was aligned, both length of read and reference sequence can be easily bound.importallein_zu_hausimportnumpyasnpmax_read_len=100max_ref_len=2*max_read_lenmismatch_penalty=4gap_open=6gep_extend=3aligner=allein_zu_haus.Aligner(max_read_len,max_ref_len,mismatch_penalty,gap_open,gap_extend)Then use the aligner repeatedly for each match, providing only the match specific parameters:forread,read_quality_string,ref,max_gapsin...:# read, ref should be of type np.array with dtype=bytesread_bp_probs=get_read_bp_probs(read_quality_string)# Function for exatracting basepair confidence from FASTQ quality strings. See example above.base_probs=get_priors()# Function for computing priors from reference and/or read sequences. See example abovescore,cigar=aligner.match(read,read_bp_probs,base_probs,ref,max_gaps)# multiple calls to aligner, with relevant sequences and priorsMathematical RationaleSuppose we wish to find the optimal match between a readDand a referenceF. Unfortunately, we cannot observeDdirectly, and instead only seeD’, which is the sequence outputted by some imperfectsequencing process. Say that at some point in the alignment algorithm we consider whether a nucleotide fromD’matches a nucleotide fromF. Define:bD’is the nucleotide reported by the sequencing process.bDis the true (unknown) uncleotide.bRis the reference nucleotide.The classic algorithm would assign the penalty\begin{equation*}
\mbox{penalty}(b_{D'}, b_R)
\end{equation*}whereas the correct penalty should be\begin{equation*}
\mbox{penalty}(b_{D}, b_R) \simeq
\sum_b \left[ P\left( B_D = b | B_{D'} = b_{D'} \right) \cdot \mbox{penalty}(b, b_R) \right]
\end{equation*}ByBayes’ Theorem,\begin{equation*}
P\left( B_D = b | B_{D'} = b_{D'} \right)
=
\frac
{
    P\left( B_{D'} = b_{D'} | B_D = b \right)
    \cdot
    P\left( B_D = b \right)
}
{
    \sum_{k = 'A', 'C', 'G', 'T}
    P\left( B_{D'} = b_{D'} | B_D = k \right)
    \cdot
    P\left( B_D = k \right)
}
\end{equation*}For evaluating these terms, note that\begin{equation*}
P\left( B_D = b \right)
\end{equation*}is the prior over the nucleotides (which must be given by the user), and\begin{equation*}
P\left( B_{D'} = b_{D'} | B_D = b \right)
=
\begin{cases}
    1 - P_{\mbox{err}} ,& \text{if } b_{D'} = b, \\
    \frac{P_{\mbox{err}}}{3}, & \text{otherwise}
\end{cases}
\end{equation*}wherePerris the probability for error determined by the reported quality for this nucleotide.IssuesFeel free to open tickets athttps://bitbucket.org/taliraveh/allein_zu_haus/issues.