BACKGROUND
rna interference  is a naturally occurring phenomenon by which rna duplexes known as short interfering rna  can reduce gene expression through enzymatic cleavage of a target mrna mediated by the rna-induced silencing complex .
the ability of synthetic sirna to inhibit targeted genes with near specificity makes it an extremely powerful tool for functional genomics that has drawn considerable interest recently  <cit> .
rnai is commonly achieved by introducing chemically synthesized sirna 19– <dig> mers into cells by transfection.
however, many cells and cell lines are either refractory to or adversely affected by transfection, and the transient nature of this methodology renders it unsuitable for the generation of long-term cell lines of the desirable phenotype.
two alternatives to synthetic sirna are dna-vector mediated rnai production  <cit> , and most recently viral-mediated sirna synthesis  <cit> .
for the latter technologies, sense and antisense strands can be expressed from different promoters  <cit> .
alternatively, short hairpin  rnas, expressed from a single promoter, are processed into sirnas by dicer or a homologous double strand rnase  <cit> .
one caveat of sirna design is that not all 19– <dig> base rna duplexes will cleave their target with efficacy, and much effort has gone towards identifying a set of rules for selecting an effective sirna target site within a gene.
recent findings  <cit>  offered the first clue towards the development of guidelines for selecting an sirna target site.
these studies showed that the risc complex is asymmetric and favors the strand of the sirna duplex with the least thermodynamically stable 5' terminus.
subsequently, reynolds et al.
designed an algorithm based on statistical data showing patterns of efficacy for sirna oligonucleotides containing specific residues at defined positions within the 19-mer  <cit> .
a limitation of their study is that a small number of genes were tested.
several additional algorithms for designing effective sirnas have been published since those initial reports with surprisingly disparate results, making the determination of which residues are generally favorable for sirna efficacy a point of controversy  <cit> .
additionally, whether any of the algorithms developed for synthetic sirna oligonucleotides apply to the design of shrna expressed stably from a vector has not been well explored.
in the present report, we construct and analyze a set of  <dig> shrnas for  <dig> different human genes.
to our knowledge this is the largest individual set of data published for shrna 19-mers.
we describe a method for simultaneously preparing wild type and control mutant shrna vectors that is time and cost efficient, and show that sequencing of shrna plasmids can be quite problematic due to the intrinsic secondary structure of the hairpin.
we examine several different strategies for overcoming this problem including the use of modified bigdye chemistries and the addition of agents known to relax dna structure.
the knockdown efficacy for each of the  <dig> shrnas was evaluated against six published algorithms for sirna oligonucleotide design by linear regression and roc curve analyses.
we describe a modification of three of the algorithms that provides fair-to-good prediction of shrna efficacy, and confirm the significance of the modified algorithms using a pooled set of shrnas from previous publications.
these findings should be of general applicability in the design and construction of shrna vectors.
RESULTS
design and preparation of shrna plasmids
to address the question of how shrna sequence correlates with knockdown efficacy,  <dig> shrna vectors from  <dig> different genes were designed and constructed .
target sequences were selected in the coding region of each gene and were designed to broadly conform to the seminal studies of sequence features for sirna oligomer efficacy  <cit> .
accordingly, sequences are low in runs and have a g/c ratio of about 50%.
the shrnas were designed to target sites that are devoid of single nucleotide polymorphisms, and correspond to all splice variants amplified by our real time pcr primer sets.
since sirnas can have off-target effects, it is important for functional assays to make a specific mutant with one or more base mismatch within the target recognition site as a control  <cit> .
to conserve time and cost, we have developed a method of making wild-type and mutant shrna vectors simultaneously .
gene knockdown results for four wild-type/mutant shrna pairs are shown in figure  <dig>  these results demonstrate the utility of this method in providing a point mutant shrna vector that can serve as a loss-of-function control for gene knockdown by wild type shrnas.
though detailed protocols have been published for construction of shrna vectors  <cit> , this is the first protocol for producing wild-type and mutant vectors simultaneously and should facilitate the implementation of highly controlled system for shrna.
strategy for accurate sequencing through hairpin structures
verifying the sequence of an shrna hairpin is essential since mismatch of even one nucleotide within the target sequence can ablate knockdown  an issue that is frequently encountered in the preparation of shrna vectors is that many are difficult to sequence due to the intrinsic secondary structure of the hairpin.
one strategy recently proposed to overcome this issue involves engineering a restriction site within the loop/stem region of the hairpin to physically separate the inverted repeats by digestion, and then piecing together sequence using sense and antisense primers  <cit> .
however, the ability to achieve sequencing of shrna constructs without modifying stem/loop sequence would be of clear advantage.
to address this possibility, we evaluated modified sequencing reactions for improvement in the read-through of the hairpin secondary structure in three shrna hairpins.
modifications include adding agents known to relax dna structure including dmso, betaine, pcrx enhancer and thermofidelase i; and adding increasing amounts of dgtp bigdye terminator  chemistry to the standard bigdye v <dig>   chemistry which contains ditp rather than dgtp.
sequencing results for each of the three dna constructs are summarized in table  <dig>  read-through of the hairpin structure was measured as the ratio of the peak height about  <dig> bases after the hairpin structure to the signal about  <dig> bases before the hairpin structure.
a ratio of  <dig> indicates no loss in signal and  <dig> indicates complete loss of read-though.
in the absence of any additive to bd chemistry, the hairpin caused a reduction in peak height ratio for our less tightly structured hairpin, phspg-shmuttlr <dig>  to  <dig> , and a complete loss in read through for the other two plasmids.
this can be visualized as an abrupt stop in the sequence peak profile for phspg-shtlr <dig> .
among the dna relaxing agents, 5% dmso,  <dig>  m betaine and  <dig> × pcrx enhancer each improved the sequence read significantly for some constructs.
however, the addition of  <dig>  m betaine plus  <dig> × pcrx enhancer to bd chemistry was found to sequence most consistently, with peak height ratios of  <dig> – <dig>  .
the addition of 10: <dig> bd:dgtp chemistries alone also improved read through somewhat, with peak height ratios of  <dig> – <dig>  .
the sub-optimal peak height ratio for 10: <dig> bd:dgtp can be attributed to a visible step in the sequence peak profile after the secondary structure region where the signal is reduced .
increasing the dgtp chemistry content to 5: <dig> and 3: <dig> bd:dgtp or using straight dgtp chemistry increased the peak height ratio and reduced the step somewhat .
however, the mixed incorporation of ditp and dgtp resulted in worse peak broadening as the amount of dgtp used increased , and dgtp only chemistry caused severe sequence compressions .
the best overall results were observed by combining betaine plus pcrx and 10: <dig> bd:dgtp mixed chemistries together.
this combination reduced the step with less peak broadening and increased peak height ratios to  <dig> – <dig>  .
thermofidelase i, a dna destabililizing enzyme that is frequently used to improve sequencing of genomic dna  <cit> , did not improve sequencing of any of the three hairpins in straight bd chemistry , and actually reduced the peak height ratio significantly in 10: <dig> bd:dgtp chemistries for all three shrna constructs, causing the reappearance of a stop at the hairpin structure .
in summary, the combination of 10: <dig> bd:gtp chemistries,  <dig>  m betaine, and  <dig> × pcrx enhancer provided optimal sequencing, and mixed bd:dgtp chemistries, betaine, pcrx enhancer, and dmso each had some positive effects on their own.
thermofidelase i, however, probably should be avoided for shrna vectors with difficult intrinsic secondary structure.
correlation between shrna knockdown efficiency and published algorithms for sirna design
to determine whether the efficacy of knockdown by shrna vectors correlates with published rules for the design of effective sirna oligonucleotides, shrnas were evaluated for their ability to knockdown gene expression.
the shrnas were transduced stably into either thp <dig> or jurkat human cell lines as detailed in table  <dig>  first two columns.
the average knockdown was determined from rna collected on three or more different days and is listed for each shrna .
knockdown was shown to be reproducible for cell lines that were independently transduced and sorted, suggesting that knockdown is a function of the shrna target sequence rather than features of the viral transduction .
more than one third of the shrna vectors constructed were unable to suppress transcription , despite comparable growth rates and long term expression of the gfp marker at high levels in these cell lines.
furthermore, great variations in knockdown efficacy for several shrnas made against many of the same genes  argue against any simple biological reasons for differences in efficacy for these genes.
many of the ineffective shrnas have negative 5' ΔΔg values and high reynolds scoring, each which have been hypothesized to correlate with sirna knockdown efficacy   <cit> .
conversely, among the shrnas that were able to confer gene knockdown, several had either positive 5'ΔΔg values or low reynolds scores.
these findings indicate that 5'ΔΔg and reynolds scoring algorithm for sirna may not provide positive correlative criteria for shrna design.
to determine whether other published algorithms for sirna oligonucleotide design can be applied to shrna vectors, each of the shrna target sites was evaluated by four additional algorithms, and scores were plotted against the percent knockdown for each shrna .
for each algorithm plot a best fit line was drawn and the r <dig> value calculated as an indication of whether the variance in knockdown efficacy can be explained by the algorithm scoring.
results confirm a poor association between shrna efficacy and either 5' ΔΔg  considerations  <cit>  or the reynolds et al.
algorithm  <cit> , and also demonstrate a poor association with the hsieh et al.
algorithm  <cit> , with each in fact showing a weak reverse correlation with the data.
the algorithms of amarguizoui et al.
<cit> , ui-tei et al.
<cit> , and takasaki et al.
<cit> , correlate directly with shrna efficacy.
however, none of the algorithm scores explain a significant percentage of the variance in knockdown efficacy.
among the algorithms tested, the takasaki et al.
scoring system shows the highest association, with an r <dig> value of  <dig> .
because these results suggest that a linear relationship does not strongly apply to shrna knockdown for any of the six algorithms, we evaluated each of the algorithms by roc curve analysis to determine whether any algorithm is superior to the others at identifying effective shrnas.
the roc curve is a plot of sensitivity  versus  <dig> minus the specificity  that is generated by varying the decision threshold between the minimum and maximum algorithm score.
the diagonal of the roc plot represents the roc curve for an algorithm that is no better at discrimination than random selection.
algorithms that are poor discriminators have roc curves that track along the diagonal and have an area under the roc curve  that is not significantly different from the auc of the diagonal .
algorithms that are good discriminators have roc curves with strong convex deviation from the diagonal and aucs that approach  <dig> and are significantly different from the auc of the diagonal.
the hsieh et al.
algorithm had a concave roc curve  indicating unacceptable sensitivity and specificy in discriminating effective from ineffective shrnas.
the roc curves for all other algorithms  tracked near the diagonal of the roc plot and had aucs that were not significantly different from the auc of the diagonal .
thus, none of the algorithms showed a statistically significant ability to discriminate between effective and ineffective shrnas.
the takasaki et al.
algorithm  showed the most promise as a discriminator of effective from ineffective shrnas.
however, this algorithm suffered from a relatively high false positive fraction for decision thresholds near the maximum score as indicated by the weak, erratic deviation from the diagonal near the origin of the roc curve .
this indicated that the algorithm assigned a high score to a number of ineffective shrnas.
inspection of the data revealed that two of the three high-scoring ineffective shrnas targeted genes whose expression was successfully knocked-down by other shrnas .
thus it is unlikely that the inefficacy of the shrnas is a consequence of selective pressure against the stable suppression of gene expression.
it is more likely that the takasaki et al.
algorithm does not account for a critical feature of effective shrnas.
application of an algorithm modification based on the stability of the  <dig> central bases of each shrna
inspection of the physical properties of the high scoring ineffective shrnas revealed that the average stability of the duplex formed by the  <dig> central bases of the shrnas  was greater than the average stability of high scoring effective shrnas .
based on this observation, the takasaki et al.
algorithm was modified such that shrnas with a central duplex Δg equal to or less than - <dig>  kcal/mol were assigned a minimum score .
this modification assigned minimum scores to five shrnas, four which were ineffective, thus increasing the specificity of the algorithm without a significant loss in sensitivity.
a minimum score assigned to one effective shrna , indicates that other properties in addition to central duplex stability influence efficacy.
nevertheless, the addition of this modification eliminated the weak erratic deviation of the roc curve from the diagonal for high decision thresholds and increased the auc to  <dig>  .
similar modification of the amarzguioui et al.
and ui-tei et al.
algorithms also raised the aucs of their roc curves .
with this modification, the aucs of the roc curves for all three modified algorithms were significantly different from the auc of the diagonal , indicating statistically significant predictive capability.
differences between aucs of the roc curves for the modified algorithms were not significant, so on statistical grounds all three of the modified algorithms were of equal utility.
the 5' ΔΔg, reynolds et al, and the hsieh et al.
algorithms were not improved to a statistically significant predictive capability by applying the central duplex Δg modification .
to address the possibility that the improvement achieved by the modification of the amarzguioui et al, ui-tei et al, and takasaki et al.
algorithms is a consequence of overfitting our set of shrnas, an independent set of  <dig> shrnas pooled from previous publications  were subjected to analysis.
while none of the roc curves for the three unmodified algorithms had an auc significantly different from that of the diagonal , all of the modified algorithms yielded roc curves with aucs significantly different from the auc of the diagonal .
on statistical grounds, all three of the modified algorithms were of equal utility as the aucs of the roc curves for the modified algorithms were all significantly different from the auc of the diagonal, but not significantly different from each other.
this analysis of an independent set of shrnas suggests that the modification of the algorithms is of general validity.
because minimizing the false positive rate is the primary concern in shrna design, we recommend using the modified ui-tei et al.
algorithm, which had the lowest high false positive fraction at decision thresholds near the maximum score as indicated by the strong deviation from the diagonal near the origin of the roc curve .
using a decision threshold of  <dig> limits selection of shrnas to a region of the roc curve where the sensitivity was acceptable , while the specificity was very good .
by setting this decision threshold, the false positive fraction was minimized, while  <dig> – 33% of the effective shrnas were identified from our shrnas and the published set of shrnas respectively.
should the sensitivity need to be increased, we recommend using a decision threshold of  <dig>  this threshold had a sensitivity of  <dig>  –  <dig>  and a specificity of  <dig>  –  <dig> .
if the decision threshold was further relaxed to  <dig>  the sensitivity increased to  <dig>  –  <dig> , but the specificity fell to  <dig>  –  <dig> .
we recommend using the highest of these decision thresholds possible.
though statistically small, this study has the advantage to our knowledge of being the largest published set of 19-mer based shrnas to date.
in addition, unlike other shrna studies that are necessarily skewed toward effective shrnas, our study includes both functional and non-functional shrnas.
we have shown that modified ui-tei et al., amarzguioui et al.
and takasaki et al.
algorithms are fair to good predictive tools that distinguish effective from ineffective shrnas.
however, significant shortcomings still exist in the modified algorithms.
a direct assessment of the algorithm modifications using shrnas designed according to each original and modified algorithm would lend support to these findings.
these algorithms are meant to reduce the number of false positive shrnas selected, not completely eliminate them altogether, and thus this would require a large number of shrnas to obtain a statistically significant difference in false positive rate.
the availability of larger shrna data sets should support the development of algorithms with improved sensitivity and specificity.
additionally, several software applications for sirna oligonucleotide design that were not considered in this study may be of use in the design of shrnas  <cit> .
criteria for designing functional sirna oligonucleotides remain controversial as evidenced by the large number of studies still being devised for sirna design, and since we did not test these sequences as sirnas it cannot be established whether the modification of these algorithms also applies in the context of sirna oligonucleotides.
shrna has an added layer of complexity over sirna oligonucleotides since the hairpin needs to be processed within the cell before entering the risc complex.
moreover, selective pressure against the stable expression of shrnas that are deleterious to cell growth would be expected to lend an additional constraint to the stable expression of certain shrnas.
despite these complexities, our findings begin to bring insight into the ability to apply sirna algorithms for design of functional shrnas.
CONCLUSIONS
we have provided several important strategies that should facilitate the generation of effective shrna vectors for gene knockdown in mammalian cells.
the ability to produce wild-type and mutant shrna vectors simultaneously using mixed oligonucleotide pairs provides an efficient method to generate a specific control vector with little added time or cost.
this strategy should be particularly useful in generating specific controls in high throughput applications.
difficulty in sequencing through the high intrinsic secondary structure of some hairpin vectors also has presented a major constraint in the construction of shrna vectors, and the knowledge that sequencing issues can be resolved by modifying bigdye chemistries and adding betaine and other dna relaxing agents should be valuable regardless of the method of shrna design and construction.
using data from  <dig> shrnas that we have constructed we have performed an analysis of the ability of published algorithms for sirna oligonucleotide target selection to predict knockdown efficacy.
our results show that shrna efficacy cannot strictly be explained by any of the six algorithms tested.
we provide a modification, however, that greatly improves the predictability of the ui-tei et al., amarzguioui et al.
and takasaki et al algorithms.
results were confirmed using data from  <dig> previously published shrnas.
these findings should be of significant applicability in the design and preparation of functional shrnas.

 rna interference is a naturally occurring phenomenon by which rna duplexes known as short interfering rna can reduce gene expression through enzymatic cleavage of a target mrna mediated by the rna-induced silencing complex. the ability of synthetic sirna to inhibit targeted genes with near specificity makes it an extremely powerful tool for functional genomics that has drawn considerable interest recently. rnai is commonly achieved by introducing chemically synthesized sirna 19– mers into cells by transfection. however, many cells and cell lines are either refractory to or adversely affected by transfection, and the transient nature of this methodology renders it unsuitable for the generation of long-term cell lines of the desirable phenotype. two alternatives to synthetic sirna are dna-vector mediated rnai production, and most recently viral-mediated sirna synthesis. for the latter technologies, sense and antisense strands can be expressed from different promoters. alternatively, short hairpin rnas, expressed from a single promoter, are processed into sirnas by dicer or a homologous double strand rnase. one caveat of sirna design is that not all 19– base rna duplexes will cleave their target with efficacy, and much effort has gone towards identifying a set of rules for selecting an effective sirna target site within a gene. recent findings offered the first clue towards the development of guidelines for selecting an sirna target site. these studies showed that the risc complex is asymmetric and favors the strand of the sirna duplex with the least thermodynamically stable 5' terminus. subsequently, reynolds et al. designed an algorithm based on statistical data showing patterns of efficacy for sirna oligonucleotides containing specific residues at defined positions within the 19-mer. a limitation of their study is that a small number of genes were tested. several additional algorithms for designing effective sirnas have been published since those initial reports with surprisingly disparate results, making the determination of which residues are generally favorable for sirna efficacy a point of controversy. additionally, whether any of the algorithms developed for synthetic sirna oligonucleotides apply to the design of shrna expressed stably from a vector has not been well explored. in the present report, we construct and analyze a set of shrnas for different human genes. to our knowledge this is the largest individual set of data published for shrna 19-mers. we describe a method for simultaneously preparing wild type and control mutant shrna vectors that is time and cost efficient, and show that sequencing of shrna plasmids can be quite problematic due to the intrinsic secondary structure of the hairpin. we examine several different strategies for overcoming this problem including the use of modified bigdye chemistries and the addition of agents known to relax dna structure. the knockdown efficacy for each of the shrnas was evaluated against six published algorithms for sirna oligonucleotide design by linear regression and roc curve analyses. we describe a modification of three of the algorithms that provides fair-to-good prediction of shrna efficacy, and confirm the significance of the modified algorithms using a pooled set of shrnas from previous publications. these findings should be of general applicability in the design and construction of shrna vectors.  design and preparation of shrna plasmids to address the question of how shrna sequence correlates with knockdown efficacy, shrna vectors from different genes were designed and constructed. target sequences were selected in the coding region of each gene and were designed to broadly conform to the seminal studies of sequence features for sirna oligomer efficacy. accordingly, sequences are low in runs and have a g/c ratio of about 50%. the shrnas were designed to target sites that are devoid of single nucleotide polymorphisms, and correspond to all splice variants amplified by our real time pcr primer sets. since sirnas can have off-target effects, it is important for functional assays to make a specific mutant with one or more base mismatch within the target recognition site as a control. to conserve time and cost, we have developed a method of making wild-type and mutant shrna vectors simultaneously. gene knockdown results for four wild-type/mutant shrna pairs are shown in figure these results demonstrate the utility of this method in providing a point mutant shrna vector that can serve as a loss-of-function control for gene knockdown by wild type shrnas. though detailed protocols have been published for construction of shrna vectors, this is the first protocol for producing wild-type and mutant vectors simultaneously and should facilitate the implementation of highly controlled system for shrna. strategy for accurate sequencing through hairpin structures verifying the sequence of an shrna hairpin is essential since mismatch of even one nucleotide within the target sequence can ablate knockdown an issue that is frequently encountered in the preparation of shrna vectors is that many are difficult to sequence due to the intrinsic secondary structure of the hairpin. one strategy recently proposed to overcome this issue involves engineering a restriction site within the loop/stem region of the hairpin to physically separate the inverted repeats by digestion, and then piecing together sequence using sense and antisense primers. however, the ability to achieve sequencing of shrna constructs without modifying stem/loop sequence would be of clear advantage. to address this possibility, we evaluated modified sequencing reactions for improvement in the read-through of the hairpin secondary structure in three shrna hairpins. modifications include adding agents known to relax dna structure including dmso, betaine, pcrx enhancer and thermofidelase i; and adding increasing amounts of dgtp bigdye terminator chemistry to the standard bigdye v chemistry which contains ditp rather than dgtp. sequencing results for each of the three dna constructs are summarized in table read-through of the hairpin structure was measured as the ratio of the peak height about bases after the hairpin structure to the signal about bases before the hairpin structure. a ratio of indicates no loss in signal and indicates complete loss of read-though. in the absence of any additive to bd chemistry, the hairpin caused a reduction in peak height ratio for our less tightly structured hairpin, phspg-shmuttlr to, and a complete loss in read through for the other two plasmids. this can be visualized as an abrupt stop in the sequence peak profile for phspg-shtlr. among the dna relaxing agents, 5% dmso, m betaine and × pcrx enhancer each improved the sequence read significantly for some constructs. however, the addition of m betaine plus × pcrx enhancer to bd chemistry was found to sequence most consistently, with peak height ratios of –. the addition of 10: bd:dgtp chemistries alone also improved read through somewhat, with peak height ratios of –. the sub-optimal peak height ratio for 10: bd:dgtp can be attributed to a visible step in the sequence peak profile after the secondary structure region where the signal is reduced. increasing the dgtp chemistry content to 5: and 3: bd:dgtp or using straight dgtp chemistry increased the peak height ratio and reduced the step somewhat. however, the mixed incorporation of ditp and dgtp resulted in worse peak broadening as the amount of dgtp used increased, and dgtp only chemistry caused severe sequence compressions. the best overall results were observed by combining betaine plus pcrx and 10: bd:dgtp mixed chemistries together. this combination reduced the step with less peak broadening and increased peak height ratios to –. thermofidelase i, a dna destabililizing enzyme that is frequently used to improve sequencing of genomic dna, did not improve sequencing of any of the three hairpins in straight bd chemistry, and actually reduced the peak height ratio significantly in 10: bd:dgtp chemistries for all three shrna constructs, causing the reappearance of a stop at the hairpin structure. in summary, the combination of 10: bd:gtp chemistries, m betaine, and × pcrx enhancer provided optimal sequencing, and mixed bd:dgtp chemistries, betaine, pcrx enhancer, and dmso each had some positive effects on their own. thermofidelase i, however, probably should be avoided for shrna vectors with difficult intrinsic secondary structure. correlation between shrna knockdown efficiency and published algorithms for sirna design to determine whether the efficacy of knockdown by shrna vectors correlates with published rules for the design of effective sirna oligonucleotides, shrnas were evaluated for their ability to knockdown gene expression. the shrnas were transduced stably into either thp or jurkat human cell lines as detailed in table first two columns. the average knockdown was determined from rna collected on three or more different days and is listed for each shrna. knockdown was shown to be reproducible for cell lines that were independently transduced and sorted, suggesting that knockdown is a function of the shrna target sequence rather than features of the viral transduction. more than one third of the shrna vectors constructed were unable to suppress transcription, despite comparable growth rates and long term expression of the gfp marker at high levels in these cell lines. furthermore, great variations in knockdown efficacy for several shrnas made against many of the same genes argue against any simple biological reasons for differences in efficacy for these genes. many of the ineffective shrnas have negative 5' ΔΔg values and high reynolds scoring, each which have been hypothesized to correlate with sirna knockdown efficacy. conversely, among the shrnas that were able to confer gene knockdown, several had either positive 5'ΔΔg values or low reynolds scores. these findings indicate that 5'ΔΔg and reynolds scoring algorithm for sirna may not provide positive correlative criteria for shrna design. to determine whether other published algorithms for sirna oligonucleotide design can be applied to shrna vectors, each of the shrna target sites was evaluated by four additional algorithms, and scores were plotted against the percent knockdown for each shrna. for each algorithm plot a best fit line was drawn and the r value calculated as an indication of whether the variance in knockdown efficacy can be explained by the algorithm scoring. results confirm a poor association between shrna efficacy and either 5' ΔΔg considerations or the reynolds et al. algorithm, and also demonstrate a poor association with the hsieh et al. algorithm, with each in fact showing a weak reverse correlation with the data. the algorithms of amarguizoui et al., ui-tei et al., and takasaki et al., correlate directly with shrna efficacy. however, none of the algorithm scores explain a significant percentage of the variance in knockdown efficacy. among the algorithms tested, the takasaki et al. scoring system shows the highest association, with an r value of. because these results suggest that a linear relationship does not strongly apply to shrna knockdown for any of the six algorithms, we evaluated each of the algorithms by roc curve analysis to determine whether any algorithm is superior to the others at identifying effective shrnas. the roc curve is a plot of sensitivity versus minus the specificity that is generated by varying the decision threshold between the minimum and maximum algorithm score. the diagonal of the roc plot represents the roc curve for an algorithm that is no better at discrimination than random selection. algorithms that are poor discriminators have roc curves that track along the diagonal and have an area under the roc curve that is not significantly different from the auc of the diagonal. algorithms that are good discriminators have roc curves with strong convex deviation from the diagonal and aucs that approach and are significantly different from the auc of the diagonal. the hsieh et al. algorithm had a concave roc curve indicating unacceptable sensitivity and specificy in discriminating effective from ineffective shrnas. the roc curves for all other algorithms tracked near the diagonal of the roc plot and had aucs that were not significantly different from the auc of the diagonal. thus, none of the algorithms showed a statistically significant ability to discriminate between effective and ineffective shrnas. the takasaki et al. algorithm showed the most promise as a discriminator of effective from ineffective shrnas. however, this algorithm suffered from a relatively high false positive fraction for decision thresholds near the maximum score as indicated by the weak, erratic deviation from the diagonal near the origin of the roc curve. this indicated that the algorithm assigned a high score to a number of ineffective shrnas. inspection of the data revealed that two of the three high-scoring ineffective shrnas targeted genes whose expression was successfully knocked-down by other shrnas. thus it is unlikely that the inefficacy of the shrnas is a consequence of selective pressure against the stable suppression of gene expression. it is more likely that the takasaki et al. algorithm does not account for a critical feature of effective shrnas. application of an algorithm modification based on the stability of the central bases of each shrna inspection of the physical properties of the high scoring ineffective shrnas revealed that the average stability of the duplex formed by the central bases of the shrnas was greater than the average stability of high scoring effective shrnas. based on this observation, the takasaki et al. algorithm was modified such that shrnas with a central duplex Δg equal to or less than - kcal/mol were assigned a minimum score. this modification assigned minimum scores to five shrnas, four which were ineffective, thus increasing the specificity of the algorithm without a significant loss in sensitivity. a minimum score assigned to one effective shrna, indicates that other properties in addition to central duplex stability influence efficacy. nevertheless, the addition of this modification eliminated the weak erratic deviation of the roc curve from the diagonal for high decision thresholds and increased the auc to. similar modification of the amarzguioui et al. and ui-tei et al. algorithms also raised the aucs of their roc curves. with this modification, the aucs of the roc curves for all three modified algorithms were significantly different from the auc of the diagonal, indicating statistically significant predictive capability. differences between aucs of the roc curves for the modified algorithms were not significant, so on statistical grounds all three of the modified algorithms were of equal utility. the 5' ΔΔg, reynolds et al, and the hsieh et al. algorithms were not improved to a statistically significant predictive capability by applying the central duplex Δg modification. to address the possibility that the improvement achieved by the modification of the amarzguioui et al, ui-tei et al, and takasaki et al. algorithms is a consequence of overfitting our set of shrnas, an independent set of shrnas pooled from previous publications were subjected to analysis. while none of the roc curves for the three unmodified algorithms had an auc significantly different from that of the diagonal, all of the modified algorithms yielded roc curves with aucs significantly different from the auc of the diagonal. on statistical grounds, all three of the modified algorithms were of equal utility as the aucs of the roc curves for the modified algorithms were all significantly different from the auc of the diagonal, but not significantly different from each other. this analysis of an independent set of shrnas suggests that the modification of the algorithms is of general validity. because minimizing the false positive rate is the primary concern in shrna design, we recommend using the modified ui-tei et al. algorithm, which had the lowest high false positive fraction at decision thresholds near the maximum score as indicated by the strong deviation from the diagonal near the origin of the roc curve. using a decision threshold of limits selection of shrnas to a region of the roc curve where the sensitivity was acceptable, while the specificity was very good. by setting this decision threshold, the false positive fraction was minimized, while – 33% of the effective shrnas were identified from our shrnas and the published set of shrnas respectively. should the sensitivity need to be increased, we recommend using a decision threshold of this threshold had a sensitivity of – and a specificity of –. if the decision threshold was further relaxed to the sensitivity increased to –, but the specificity fell to –. we recommend using the highest of these decision thresholds possible. though statistically small, this study has the advantage to our knowledge of being the largest published set of 19-mer based shrnas to date. in addition, unlike other shrna studies that are necessarily skewed toward effective shrnas, our study includes both functional and non-functional shrnas. we have shown that modified ui-tei et al., amarzguioui et al. and takasaki et al. algorithms are fair to good predictive tools that distinguish effective from ineffective shrnas. however, significant shortcomings still exist in the modified algorithms. a direct assessment of the algorithm modifications using shrnas designed according to each original and modified algorithm would lend support to these findings. these algorithms are meant to reduce the number of false positive shrnas selected, not completely eliminate them altogether, and thus this would require a large number of shrnas to obtain a statistically significant difference in false positive rate. the availability of larger shrna data sets should support the development of algorithms with improved sensitivity and specificity. additionally, several software applications for sirna oligonucleotide design that were not considered in this study may be of use in the design of shrnas. criteria for designing functional sirna oligonucleotides remain controversial as evidenced by the large number of studies still being devised for sirna design, and since we did not test these sequences as sirnas it cannot be established whether the modification of these algorithms also applies in the context of sirna oligonucleotides. shrna has an added layer of complexity over sirna oligonucleotides since the hairpin needs to be processed within the cell before entering the risc complex. moreover, selective pressure against the stable expression of shrnas that are deleterious to cell growth would be expected to lend an additional constraint to the stable expression of certain shrnas. despite these complexities, our findings begin to bring insight into the ability to apply sirna algorithms for design of functional shrnas.  we have provided several important strategies that should facilitate the generation of effective shrna vectors for gene knockdown in mammalian cells. the ability to produce wild-type and mutant shrna vectors simultaneously using mixed oligonucleotide pairs provides an efficient method to generate a specific control vector with little added time or cost. this strategy should be particularly useful in generating specific controls in high throughput applications. difficulty in sequencing through the high intrinsic secondary structure of some hairpin vectors also has presented a major constraint in the construction of shrna vectors, and the knowledge that sequencing issues can be resolved by modifying bigdye chemistries and adding betaine and other dna relaxing agents should be valuable regardless of the method of shrna design and construction. using data from shrnas that we have constructed we have performed an analysis of the ability of published algorithms for sirna oligonucleotide target selection to predict knockdown efficacy. our results show that shrna efficacy cannot strictly be explained by any of the six algorithms tested. we provide a modification, however, that greatly improves the predictability of the ui-tei et al., amarzguioui et al. and takasaki et al algorithms. results were confirmed using data from previously published shrnas. these findings should be of significant applicability in the design and preparation of functional shrnas. 
analysis of an independent set of data from shrnas pooled from previous publications confirms these findings. our results show that none of the scoring algorithms can explain a significant percentage of variance in shrna knockdown efficacy as assessed by linear regression analysis or roc curve analysis. application of a modification based on the stability of the central bases of each shrna provides fair-to-good predictions of knockdown efficacy for three of the algorithms. we show that sequencing through shrna vectors can be problematic due to the intrinsic secondary structure of the hairpin, and we determine a strategy for effective sequencing by using a combination of modified bigdye chemistries and dna relaxing agents. the addition to sequencing reactions of a combination of mixed ditp/dgtp chemistries and dna relaxing agents enables read through the intrinsic secondary structure of problematic shrna vectors. however, application of a modification based on the central shrna stability should provide a useful improvement to the design of effective shrna vectors. we demonstrate an efficient method for preparing wild-type and mutant control shrna vectors simultaneously using oligonucleotide hybrids. the efficacy of knockdown for the shrna vectors was evaluated against six published algorithms for sirna oligonucleotide design. the use of mixed oligonucleotide pairs provides a time and cost efficient method of producing wild type and mutant control shrna vectors. much controversy has surrounded the development of rules for the design of effective sirna oligonucleotides; and whether these rules apply to shrna is not well characterized. to determine whether published algorithms for sirna oligonucleotide design apply to shrna, we constructed shrnas from human genes expressed stably using retroviral vectors. six published algorithms for sirna oligonucleotide design that were tested in this study show little or no efficacy at predicting shrna knockdown outcome.
 rna interference technology is a powerful methodology recently developed for the specific knockdown of targeted genes. rnai is most commonly achieved either transiently by transfection of small interfering rna oligonucleotides, or stably using short hairpin rna expressed from a dna vector or virus. much controversy has surrounded the development of rules for the design of effective sirna oligonucleotides; and whether these rules apply to shrna is not well characterized.  to determine whether published algorithms for sirna oligonucleotide design apply to shrna, we constructed shrnas from human genes expressed stably using retroviral vectors. we demonstrate an efficient method for preparing wild-type and mutant control shrna vectors simultaneously using oligonucleotide hybrids. we show that sequencing through shrna vectors can be problematic due to the intrinsic secondary structure of the hairpin, and we determine a strategy for effective sequencing by using a combination of modified bigdye chemistries and dna relaxing agents. the efficacy of knockdown for the shrna vectors was evaluated against six published algorithms for sirna oligonucleotide design. our results show that none of the scoring algorithms can explain a significant percentage of variance in shrna knockdown efficacy as assessed by linear regression analysis or roc curve analysis. application of a modification based on the stability of the central bases of each shrna provides fair-to-good predictions of knockdown efficacy for three of the algorithms. analysis of an independent set of data from shrnas pooled from previous publications confirms these findings.  the use of mixed oligonucleotide pairs provides a time and cost efficient method of producing wild type and mutant control shrna vectors. the addition to sequencing reactions of a combination of mixed ditp/dgtp chemistries and dna relaxing agents enables read through the intrinsic secondary structure of problematic shrna vectors. six published algorithms for sirna oligonucleotide design that were tested in this study show little or no efficacy at predicting shrna knockdown outcome. however, application of a modification based on the central shrna stability should provide a useful improvement to the design of effective shrna vectors. 