BACKGROUND
transcriptome sequencing projects for non-model organisms are popular because they cost less and are more computationally tractable than full genome sequencing projects, but still yield sufficient information to meet the requirements of many research programs.
traditionally, transcriptome projects have been based on sanger dideoxy-sequenced expressed sequence tags , but, because second-generation sequencing technologies provide much higher throughput than sanger sequencing at a lower cost per base, these new technologies are increasingly used.
for model organisms where a wealth of genomic information is available, the massively parallel, short-read  sequencing technologies  are most frequently used, as transcriptome reads can be mapped to the reference genome or transcriptome.
however, most published non-model organism projects have used the roche  <dig> pyrosequencing platform  <cit> , because the longer reads generated  are more amenable to de novo assembly and annotation.
a transcriptome project progresses through phases of data acquisition, assembly of the sequence reads to define putative transcripts, and then annotation and exploitation of the assembled data.
the transcriptome assembly problem is not simple.
individual reads can have errors and polymorphisms that complicate recognition of overlaps, and individual transcripts  can have several orders of magnitude variation in abundance, and thus in effective coverage.
previous analyses of transcriptome data generated by roche  <dig> pyrosequencing have almost always used just one software program for assembly .
only two studies report the results of more than assembler.
the first  <cit>  briefly compared the length of assembled consensuses from newbler  <cit> , cap <dig>  <cit> , and stackpack  <cit>  on an arabidopsis thaliana transcriptome dataset, whereas the second  <cit>  demonstrated the comprehensive est2assembly pipeline on a phylogenetically diverse sample of insects, but only compared results for newbler and mira  <cit> .
neither study provides a systematic comparison of assemblers, and so here we present such a comparison to assist the growing number of transcriptome projects using roche  <dig> pyrosequencing data.
here we compare the performance of five assemblers: newbler, cap <dig>  mira, seqman  <cit> , and clc's assembly cell  <cit>  .
these assemblers differ in the algorithms used  strategy, while clc uses de bruijn graph path finding) and how they treat individual reads .
we tested two versions of newbler because we found the frequently-used, public release version  to have several undesirable features  and thus contacted the developers to discuss these.
they provided a pre-release version  that addresses some of the issues identified.
we did not include tgicl  <cit>  or stackpack  <cit>  because both are essentially wrappers for the cap <dig> assembler.
although velvet  <cit>  and abyss  <cit>  are popular assemblers for second-generation sequence data and can use roche  <dig> pyrosequencing reads, they are primarily assemblers for genome sequence data from short-read platforms that rely on the high and even coverage depths afforded by these massively parallel technologies.
in preliminary experiments on our data we were only able to generate very short contigs using short-read assemblers and so we have not compared velvet, oases , and abyss.
* i.e.
data from one read can appear in multiple contigs
† olc: overlap-layout-consensus
our target organism, litomosoides sigmodontis, is a model filarial nematode, closely related to the causative agents of human filariases .
it originally derives from cotton rat hosts, but can be maintained in laboratory rodents  and is thus a tractable experimental system in which to investigate the dynamics of immune response induction and modulation, and test vaccine and drug candidates  <cit> .
we expect l. sigmodontis to have a transcriptome similar to that of the filarial nematode b. malayi and the more distantly related rhabditid nematode caenorhabditis elegans, with ~ <dig>  to  <dig>  protein-coding genes generating ~ <dig>  different transcripts with mean length ~ <dig>  kb.
the transcriptome project is part of a larger investigation into l. sigmodontis genomics, and detailed analyses of the content and biology of the transcriptome data will be published elsewhere.
RESULTS
l. sigmodontis transcriptome data
we used a roche  <dig> flx instrument to generate 'standard chemistry'  and 'titanium chemistry'  reads from cdna libraries from three different lifecycle stages of l. sigmodontis: microfilaria , adult males, and adult females .
reads were trimmed for adapters leaving a total of  <dig>  reads with  <dig> , <dig> bases used in all assembly experiments .
comparison of assemblers
for each assembler, we used the default parameters recommended for transcriptome assembly .
after assembly, contigs less than  <dig> bases in length and singletons were discarded for subsequent analyses.
we compared the assemblies using the following standard metrics: total number of reads used in the assembly, number of contigs >  <dig> bases generated, n <dig> length of contigs , maximum contig length, summed contig length, and approximate time taken to perform analysis .
we include the n <dig> as a measure even though it is not strictly appropriate for transcriptome assemblies .
we also assessed assembly integrity and completeness by comparison to four reference datasets.
† only contigs >  <dig> bases were assessed
* on a dual quad-core  <dig> ghz xeon workstation with  <dig> gb ram
** on a dual core  <dig>  ghz mac mini server with  <dig> gb ram
numbers of reads used in assembly
the optimal assembler will use all the reads given, and will deliver assemblies with unambiguous mappings of reads to contigs, and thus putative transcripts.
each assembler has a different way of reporting the number of reads utilised.
for example, mira reported only  <dig>  singletons but classified  <dig>  unassembled reads as 'debris'.
similarly, newbler  <dig>  and newbler  <dig>  generate separate lists of singletons, repetitive reads, and 'outliers' .
cap <dig> and seqman only report assembled and unassembled reads.
clc does not track reads at all and maps reads back to the assembly to estimate where they might belong.
therefore, to compare the assemblies, we mapped all the reads back to each assembly using ssaha <dig>  <cit>  and the clc reference aligner , as recommended in a study comparing short-read aligners  <cit> .
ssaha2's default settings for roche  <dig> reads are clearly tuned for sensitivity, because the number of reads with multiple matches is very high compared to the clc aligner.
for both aligners, the highest numbers of reads were mapped back to the seqman assembly, while fewest were mapped to the newbler  <dig>  assembly.
cap <dig>  clc, and mira were comparable in terms of the number of reads used.
clc had the fewest reads with multiple matches by far, indicating that it was the least redundant of the six assemblies.
number, mean length, and summed length of contigs
the optimal assembler will produce the longest summed length of contigs, while avoiding over-assembly of reads into in silico chimaeras, and avoiding the production of near-identical, largely overlapping contigs from allelic copies or error-rich data.
it will also produce a transcriptome estimate with a mean and variance in contig length similar to that expected from the whole transcriptome.
newbler  <dig>  generated an assembly with the largest n <dig> and the longest mean contig length , but it also produced the smallest assembly span of the six programmes tested.
inspection of contig assemblies using the next-generation sequence assembly visualisation software tablet  <cit>  showed that newbler  <dig>  was discarding read overlap information in deriving the assembled contig sequence.
the overall assembly sizes of cap <dig>  clc and newbler  <dig>  are comparable to each other , as are the assembly sizes of mira, newbler  <dig>  and seqman .
newbler  <dig>  has an assembly that is 39% larger than the newbler  <dig>  assembly, and seems to have solved the previous version's problem of discarding sequence data.
of all the six assemblers, newbler  <dig>  produced the highest number of contigs longer than  <dig> kb, and the most bases in contigs longer than  <dig> kb .
mira and seqman both generated comparable assembly spans, but with at least  <dig>  more contigs than newbler  <dig> , indicating that they have shorter contigs overall.
speed of analyses
the optimal assembler will complete analyses in a short period of time, and the time taken will scale well with increasing data volumes.
while speed itself is not an overriding optimality criterion, rapid analyses will permit robust and extensive exploration of parameter space.
in addition, some algorithms can make efficient use of multi-threaded processors or cluster computing, effectively reducing their wall-clock run time.
the speed metric is not linked to assembly quality.
clc was astonishingly fast  compared to mira .
this speed comes at a significant cost however, as clc does not track read placement in the assembly.
tracking of read placement in an assembly is a very valuable feature, as it allows inspection of the data underpinning suspect assemblies, and thus clc sacrifices speed for verifiability.
assembly redundancy, shared and novel bases
in comparison to other assemblies, the optimal assembly will include the largest proportion of the unique bases present in the sum of all assemblies.
thus an assembler that produced a large assembly that included many contigs that were redundant could be worse that an assembly that was shorter but only included unique bases.
to determine if the differing assembly sizes were due to novel sequences in each assembly, or just due to repetitive and redundant assemblies, we used blat  <cit>  with default parameters to pairwise align all six assemblies .
these pairwise comparisons show that the seqman assembly had the most novel bases when compared to the others, and the others five assemblies had the fewest novel bases when compared to seqman.
at the other end of the spectrum, the newbler  <dig>  assembly had the fewest novel bases and almost all its bases were seen in the other assemblies.
for example, the seqman assembly had over  <dig> mb of novel sequence compared to the assembly from newbler  <dig> , and the latter had only  <dig>  novel bases in the reciprocal comparison.
cap <dig>  clc, mira, and newbler  <dig>  were all very similar in that they generated assemblies that had about 1- <dig> mb of novel sequence compared to each other.
seqman, mira and newbler  <dig>  all generated assemblies in excess of  <dig> mb, 25-50% larger than the 14- <dig> mb assemblies produced by cap <dig>  clc, and newbler  <dig> .
however, these additional contigs and bases did not represent an excess of novel sequence, except in the case of seqman, suggesting that mira and newbler  <dig>  produce some redundant contigs .
clc generated the least redundant assembly, a direct consequence of using a de bruijn graph algorithm with small fragments  that collapses most repeats.
alignments to reference sequences
another optimality criterion for a novel de novo assembled transcriptome is how well it recapitulates previously determined sequences for the target species, and how well it represents sequences from related organisms.
the best assembler will return contigs that match previous data well, and will deliver a high coverage of the conserved proteome of related taxa.
we used four comparator datasets:
a set of  <dig> sanger dideoxy sequenced ests from l. sigmodontis , assembled into  <dig> clusters spanning  <dig>  mb .
we expect the roche  <dig> pyrosequencing assemblies to faithfully reflect these longer reads and clusters;
the proteome predicted from the genome sequence of b. malayi  <cit> .
we expect the b. malayi refseq protiens  to be a good model for the proteome of l. sigmodontis, and thus proportional coverage of this proteome should reflect assembly quality;
gene families conserved across the phylum nematoda derived from mcl-tribe  <cit>  analysis of  <dig> species assembled in nembase <dig> database .
we selected  <dig>  tribes, containing  <dig>  est clusters, that had representatives in species spanning the inferred base of the phylum.
they represent an estimate of the conserved core of genes that may be present in all nematode proteomes, and again proportional coverage should reflect assembly quality.
a set of  <dig>  c. elegans proteins, grouped into  <dig>  eukaryotic ortholog groups , that are known to have orthologs in the proteomes of six other eukaryotic genomes  <cit> .
we expect mrnas for these conserved proteins to be present in the l. sigmodontis data, and thus proportional coverage should reflect assembly quality.
these datasets are not independent, but each offers a different assessment of assembly quality.
tables  <dig>   <dig>   <dig> and  <dig> show the results of aligning the contigs from each assembly to each of these four datasets.
in all four cases, the best assembler was either newbler  <dig>  or seqman.
newbler  <dig>  had the lowest number of significant matches to known reference and related sequences.
however, to identify which assembler had the largest number of well-assembled long contigs, we did a second assessment of each dataset where only those hits that covered at least 80% of a target sequence were considered.
with this restriction, clc had the fewest hits in all cases except hits to conserved nematode proteins , suggesting that the longer contigs from the clc assembly were not as accurate.
* indicates a value significantly lower than the others, using a huber m-estimator.
note: e-value cutoff 1e-5
* indicates a value significantly lower than the others , using a huber m-estimator.
note: e-value cutoff 1e-5
* indicates a value significantly lower than the others , using a huber m-estimator.
note: e-value cutoff 1e-5
* indicates a value significantly lower than the others , using a huber m-estimator.
merging assemblies to improve credibility
as each assembler uses a different algorithm to derive final contigs, and each of these algorithms may model different portions of the true transcriptome with different accuracies, we reasoned that combining assemblies might identify the subset of the assemblies that had high credibility, as it was found by both  approaches.
we checked if pairs of assemblies performed better than individual assemblies under the various optimality criteria proposed above, particularly in their recapitulation of the reference data.
we combined two assemblies at a time by treating their  contigs as pseudo-reads and assembled these using a traditional olc assembler .
only second-order contigs that contained first-order contigs from both constituent assemblies were considered for further analysis because two independent assemblers had agreed on the consensus sequence in that contig: we term these "robust contigs".
whereas previously the six assemblies had generated from  <dig>  to  <dig>  contigs and spanned  <dig>  mb to  <dig>  mb , we found that merging assemblies resulted in a much narrower range of contig numbers and total span .
the n <dig> and mean contig sizes also increased, approaching that predicted for a complete transcriptome.
these robust contigs also performed better in the tests assessing representation of reference datasets .
this was especially true for matches filtered to include only those covering at least 80% of the reference sequence.
the co-assemblies of  and  matched the most est clusters.
the  combined assembly matched the most b. malayi peptides.
although there was no consistent pattern indicating a clear superiority of any pair of assemblies , these merged assemblies show that it is possible to get more long, robust contigs that align better to reference sequences by combining the outputs of two different assemblies.
co-assembling three primary assemblies at a time and considering only the contigs that had 'reads' from all three assemblies gave smaller total contig numbers than pairwise co-assemblies.
the number of stringent matches to reference sequences went up by less than 1% in all cases , suggesting that the additional effort involved in generating and combining these assemblies may not be worthwhile.
summary
we compared six assemblers by aligning their assembly contigs to four reference sequence sets.
the newbler  <dig>  assembly scored worst in each comparison, probably because it had the smallest span of all the assemblies.
under stricter comparison parameters, the clc assembly was poorer, suggesting that this assembly is more fragmented than the others.
although no single assembler was optimal in every case, newbler  <dig>  and seqman had the best alignments to related reference sequences overall.
however, seqman generated over  <dig> contigs extra for approximately the same total number of assembled bases .
we were able to improve the assembly by merging two assemblies at a time using a traditional olc assembler .
this approach generated more high quality alignments to our reference sets.
discussion and
CONCLUSIONS
we compared six programmes  for the task of de novo assembly of transcriptome data from an organism with little or no previous genomic resources.
we tested each using default or minimally adjusted parameters, as has been the common practice in previous  <dig> pyrosequencing transcriptome studies .
it may be that it is possible to generate better assemblies for particular datasets by exploring the parameter space for each assembler.
however, as sequencing becomes cheaper and more accessible, the vast majority of  <dig> transcriptome assemblies will probably be done by researchers who are new to assembly and just want something that works with a few sensible default parameters.
of the six de novo transcriptome assemblers tested, newbler  <dig>  had the best contig length metrics for our data, and, along with seqman, had the best alignments to related reference sequences.
each assembler has certain advantages and disadvantages that are presented in more detail in the supplementary materials .
versions  <dig>  and  <dig>  of newbler were the only assemblers that explicitly attempted to reconstruct and group alternative transcripts and isoforms.
if only one assembler had to be used because of time or resource constraints, we would currently recommend newbler  <dig>  overall.
however, none of the other assemblers are deprecated apart from newbler  <dig> .
we strongly recommend redoing transcriptome assemblies that were performed with newbler  <dig>  or earlier versions, if the goal is to get as complete an assembly as possible.
it is possible that different raw data sets may be better assembled by different programs , and thus researchers should perhaps cross-compare the best available ones on their data using the optimality criteria used here.
merging assemblies performed with different programs is a frequently used approach in genome assembly projects, especially those that employ multiple sequencing technologies.
application of this strategy to the problem of de novo transcriptome assembly appears particularly useful.
while a merged assembly may simply sum the errors made by both programs, filtering the resultant second stage contigs on the basis of their containing first stage contigs from both of the starting sets generates a transcriptome assembly that is on average longer and that better represents known or assumed reference sequences than do either of the starting contig sets.
a preferred assembly strategy is thus to perform initial assemblies with multiple high-quality assemblers and then to merge these using a traditional olc assembler such as cap <dig>  second-order contigs that have support from multiple first-order assemblies are much more likely to be accurate.
need for new assemblers
all the de novo assemblers in this study, with the exception of clc, use the olc assembly strategy.
clc uses de bruijn graphs.
as read lengths and throughput increase and sequencing costs come down, an average transcriptome assembly project for a non-model organism may comprise several million  <dig> base reads.
under the olc paradigm, the computational time for assembling more reads rises exponentially with data complexity because the number of pairwise comparisons to detect overlaps will increase, and the layout graphs will be harder to resolve into a consensus sequence.
the clc de novo assembler is clearly a step in the right direction because its de bruijn graph algorithm achieves reasonable results in very little time on large datasets.
other de bruijn graph assemblers such as oases  <cit>  and abyss  <cit>  are promising but are currently not suitable for most non-normalised  <dig> pyrosequencing de novo transcriptome assembly projects because many transcripts have very low coverage that cannot be assembled reliably.
perhaps the way forward is to use the de bruijn graph method for transcripts with high coverage and the olc method for transcripts with low coverage.
we have shown that an assembly-merging strategy delivers robust contigs from intermediate assemblies produced by current programs, and this strategy is likely to be of utility in deriving the best assemblies from future programs as well.

 transcriptome sequencing projects for non-model organisms are popular because they cost less and are more computationally tractable than full genome sequencing projects, but still yield sufficient information to meet the requirements of many research programs. traditionally, transcriptome projects have been based on sanger dideoxy-sequenced expressed sequence tags, but, because second-generation sequencing technologies provide much higher throughput than sanger sequencing at a lower cost per base, these new technologies are increasingly used. for model organisms where a wealth of genomic information is available, the massively parallel, short-read sequencing technologies are most frequently used, as transcriptome reads can be mapped to the reference genome or transcriptome. however, most published non-model organism projects have used the roche pyrosequencing platform, because the longer reads generated are more amenable to de novo assembly and annotation. a transcriptome project progresses through phases of data acquisition, assembly of the sequence reads to define putative transcripts, and then annotation and exploitation of the assembled data. the transcriptome assembly problem is not simple. individual reads can have errors and polymorphisms that complicate recognition of overlaps, and individual transcripts can have several orders of magnitude variation in abundance, and thus in effective coverage. previous analyses of transcriptome data generated by roche pyrosequencing have almost always used just one software program for assembly. only two studies report the results of more than assembler. the first briefly compared the length of assembled consensuses from newbler, cap, and stackpack on an arabidopsis thaliana transcriptome dataset, whereas the second demonstrated the comprehensive est2assembly pipeline on a phylogenetically diverse sample of insects, but only compared results for newbler and mira. neither study provides a systematic comparison of assemblers, and so here we present such a comparison to assist the growing number of transcriptome projects using roche pyrosequencing data. here we compare the performance of five assemblers: newbler, cap mira, seqman, and clc's assembly cell. these assemblers differ in the algorithms used strategy, while clc uses de bruijn graph path finding) and how they treat individual reads. we tested two versions of newbler because we found the frequently-used, public release version to have several undesirable features and thus contacted the developers to discuss these. they provided a pre-release version that addresses some of the issues identified. we did not include tgicl or stackpack because both are essentially wrappers for the cap assembler. although velvet and abyss are popular assemblers for second-generation sequence data and can use roche pyrosequencing reads, they are primarily assemblers for genome sequence data from short-read platforms that rely on the high and even coverage depths afforded by these massively parallel technologies. in preliminary experiments on our data we were only able to generate very short contigs using short-read assemblers and so we have not compared velvet, oases, and abyss. * i.e. data from one read can appear in multiple contigs † olc: overlap-layout-consensus our target organism, litomosoides sigmodontis, is a model filarial nematode, closely related to the causative agents of human filariases. it originally derives from cotton rat hosts, but can be maintained in laboratory rodents and is thus a tractable experimental system in which to investigate the dynamics of immune response induction and modulation, and test vaccine and drug candidates. we expect l. sigmodontis to have a transcriptome similar to that of the filarial nematode b. malayi and the more distantly related rhabditid nematode caenorhabditis elegans, with ~ to protein-coding genes generating ~ different transcripts with mean length ~ kb. the transcriptome project is part of a larger investigation into l. sigmodontis genomics, and detailed analyses of the content and biology of the transcriptome data will be published elsewhere.  l. sigmodontis transcriptome data we used a roche flx instrument to generate'standard chemistry' and'titanium chemistry' reads from cdna libraries from three different lifecycle stages of l. sigmodontis: microfilaria, adult males, and adult females. reads were trimmed for adapters leaving a total of reads with, bases used in all assembly experiments. comparison of assemblers for each assembler, we used the default parameters recommended for transcriptome assembly. after assembly, contigs less than bases in length and singletons were discarded for subsequent analyses. we compared the assemblies using the following standard metrics: total number of reads used in the assembly, number of contigs > bases generated, n length of contigs, maximum contig length, summed contig length, and approximate time taken to perform analysis. we include the n as a measure even though it is not strictly appropriate for transcriptome assemblies. we also assessed assembly integrity and completeness by comparison to four reference datasets. † only contigs > bases were assessed * on a dual quad-core ghz xeon workstation with gb ram ** on a dual core ghz mac mini server with gb ram numbers of reads used in assembly the optimal assembler will use all the reads given, and will deliver assemblies with unambiguous mappings of reads to contigs, and thus putative transcripts. each assembler has a different way of reporting the number of reads utilised. for example, mira reported only singletons but classified unassembled reads as'debris'. similarly, newbler and newbler generate separate lists of singletons, repetitive reads, and'outliers'. cap and seqman only report assembled and unassembled reads. clc does not track reads at all and maps reads back to the assembly to estimate where they might belong. therefore, to compare the assemblies, we mapped all the reads back to each assembly using ssaha and the clc reference aligner, as recommended in a study comparing short-read aligners. ssaha2's default settings for roche reads are clearly tuned for sensitivity, because the number of reads with multiple matches is very high compared to the clc aligner. for both aligners, the highest numbers of reads were mapped back to the seqman assembly, while fewest were mapped to the newbler assembly. cap clc, and mira were comparable in terms of the number of reads used. clc had the fewest reads with multiple matches by far, indicating that it was the least redundant of the six assemblies. number, mean length, and summed length of contigs the optimal assembler will produce the longest summed length of contigs, while avoiding over-assembly of reads into in silico chimaeras, and avoiding the production of near-identical, largely overlapping contigs from allelic copies or error-rich data. it will also produce a transcriptome estimate with a mean and variance in contig length similar to that expected from the whole transcriptome. newbler generated an assembly with the largest n and the longest mean contig length, but it also produced the smallest assembly span of the six programmes tested. inspection of contig assemblies using the next-generation sequence assembly visualisation software tablet showed that newbler was discarding read overlap information in deriving the assembled contig sequence. the overall assembly sizes of cap clc and newbler are comparable to each other, as are the assembly sizes of mira, newbler and seqman. newbler has an assembly that is 39% larger than the newbler assembly, and seems to have solved the previous version's problem of discarding sequence data. of all the six assemblers, newbler produced the highest number of contigs longer than kb, and the most bases in contigs longer than kb. mira and seqman both generated comparable assembly spans, but with at least more contigs than newbler, indicating that they have shorter contigs overall. speed of analyses the optimal assembler will complete analyses in a short period of time, and the time taken will scale well with increasing data volumes. while speed itself is not an overriding optimality criterion, rapid analyses will permit robust and extensive exploration of parameter space. in addition, some algorithms can make efficient use of multi-threaded processors or cluster computing, effectively reducing their wall-clock run time. the speed metric is not linked to assembly quality. clc was astonishingly fast compared to mira. this speed comes at a significant cost however, as clc does not track read placement in the assembly. tracking of read placement in an assembly is a very valuable feature, as it allows inspection of the data underpinning suspect assemblies, and thus clc sacrifices speed for verifiability. assembly redundancy, shared and novel bases in comparison to other assemblies, the optimal assembly will include the largest proportion of the unique bases present in the sum of all assemblies. thus an assembler that produced a large assembly that included many contigs that were redundant could be worse that an assembly that was shorter but only included unique bases. to determine if the differing assembly sizes were due to novel sequences in each assembly, or just due to repetitive and redundant assemblies, we used blat with default parameters to pairwise align all six assemblies. these pairwise comparisons show that the seqman assembly had the most novel bases when compared to the others, and the others five assemblies had the fewest novel bases when compared to seqman. at the other end of the spectrum, the newbler assembly had the fewest novel bases and almost all its bases were seen in the other assemblies. for example, the seqman assembly had over mb of novel sequence compared to the assembly from newbler, and the latter had only novel bases in the reciprocal comparison. cap clc, mira, and newbler were all very similar in that they generated assemblies that had about 1- mb of novel sequence compared to each other. seqman, mira and newbler all generated assemblies in excess of mb, 25-50% larger than the 14- mb assemblies produced by cap clc, and newbler. however, these additional contigs and bases did not represent an excess of novel sequence, except in the case of seqman, suggesting that mira and newbler produce some redundant contigs. clc generated the least redundant assembly, a direct consequence of using a de bruijn graph algorithm with small fragments that collapses most repeats. alignments to reference sequences another optimality criterion for a novel de novo assembled transcriptome is how well it recapitulates previously determined sequences for the target species, and how well it represents sequences from related organisms. the best assembler will return contigs that match previous data well, and will deliver a high coverage of the conserved proteome of related taxa. we used four comparator datasets: a set of sanger dideoxy sequenced ests from l. sigmodontis, assembled into clusters spanning mb. we expect the roche pyrosequencing assemblies to faithfully reflect these longer reads and clusters; the proteome predicted from the genome sequence of b. malayi. we expect the b. malayi refseq protiens to be a good model for the proteome of l. sigmodontis, and thus proportional coverage of this proteome should reflect assembly quality; gene families conserved across the phylum nematoda derived from mcl-tribe analysis of species assembled in nembase database. we selected tribes, containing est clusters, that had representatives in species spanning the inferred base of the phylum. they represent an estimate of the conserved core of genes that may be present in all nematode proteomes, and again proportional coverage should reflect assembly quality. a set of c. elegans proteins, grouped into eukaryotic ortholog groups, that are known to have orthologs in the proteomes of six other eukaryotic genomes. we expect mrnas for these conserved proteins to be present in the l. sigmodontis data, and thus proportional coverage should reflect assembly quality. these datasets are not independent, but each offers a different assessment of assembly quality. tables and show the results of aligning the contigs from each assembly to each of these four datasets. in all four cases, the best assembler was either newbler or seqman. newbler had the lowest number of significant matches to known reference and related sequences. however, to identify which assembler had the largest number of well-assembled long contigs, we did a second assessment of each dataset where only those hits that covered at least 80% of a target sequence were considered. with this restriction, clc had the fewest hits in all cases except hits to conserved nematode proteins, suggesting that the longer contigs from the clc assembly were not as accurate. * indicates a value significantly lower than the others, using a huber m-estimator. note: e-value cutoff 1e-5 * indicates a value significantly lower than the others, using a huber m-estimator. note: e-value cutoff 1e-5 * indicates a value significantly lower than the others, using a huber m-estimator. note: e-value cutoff 1e-5 * indicates a value significantly lower than the others, using a huber m-estimator. merging assemblies to improve credibility as each assembler uses a different algorithm to derive final contigs, and each of these algorithms may model different portions of the true transcriptome with different accuracies, we reasoned that combining assemblies might identify the subset of the assemblies that had high credibility, as it was found by both approaches. we checked if pairs of assemblies performed better than individual assemblies under the various optimality criteria proposed above, particularly in their recapitulation of the reference data. we combined two assemblies at a time by treating their contigs as pseudo-reads and assembled these using a traditional olc assembler. only second-order contigs that contained first-order contigs from both constituent assemblies were considered for further analysis because two independent assemblers had agreed on the consensus sequence in that contig: we term these "robust contigs". whereas previously the six assemblies had generated from to contigs and spanned mb to mb, we found that merging assemblies resulted in a much narrower range of contig numbers and total span. the n and mean contig sizes also increased, approaching that predicted for a complete transcriptome. these robust contigs also performed better in the tests assessing representation of reference datasets. this was especially true for matches filtered to include only those covering at least 80% of the reference sequence. the co-assemblies of and matched the most est clusters. the combined assembly matched the most b. malayi peptides. although there was no consistent pattern indicating a clear superiority of any pair of assemblies, these merged assemblies show that it is possible to get more long, robust contigs that align better to reference sequences by combining the outputs of two different assemblies. co-assembling three primary assemblies at a time and considering only the contigs that had'reads' from all three assemblies gave smaller total contig numbers than pairwise co-assemblies. the number of stringent matches to reference sequences went up by less than 1% in all cases, suggesting that the additional effort involved in generating and combining these assemblies may not be worthwhile. summary we compared six assemblers by aligning their assembly contigs to four reference sequence sets. the newbler assembly scored worst in each comparison, probably because it had the smallest span of all the assemblies. under stricter comparison parameters, the clc assembly was poorer, suggesting that this assembly is more fragmented than the others. although no single assembler was optimal in every case, newbler and seqman had the best alignments to related reference sequences overall. however, seqman generated over contigs extra for approximately the same total number of assembled bases. we were able to improve the assembly by merging two assemblies at a time using a traditional olc assembler. this approach generated more high quality alignments to our reference sets. discussion and  we compared six programmes for the task of de novo assembly of transcriptome data from an organism with little or no previous genomic resources. we tested each using default or minimally adjusted parameters, as has been the common practice in previous pyrosequencing transcriptome studies. it may be that it is possible to generate better assemblies for particular datasets by exploring the parameter space for each assembler. however, as sequencing becomes cheaper and more accessible, the vast majority of transcriptome assemblies will probably be done by researchers who are new to assembly and just want something that works with a few sensible default parameters. of the six de novo transcriptome assemblers tested, newbler had the best contig length metrics for our data, and, along with seqman, had the best alignments to related reference sequences. each assembler has certain advantages and disadvantages that are presented in more detail in the supplementary materials. versions and of newbler were the only assemblers that explicitly attempted to reconstruct and group alternative transcripts and isoforms. if only one assembler had to be used because of time or resource constraints, we would currently recommend newbler overall. however, none of the other assemblers are deprecated apart from newbler. we strongly recommend redoing transcriptome assemblies that were performed with newbler or earlier versions, if the goal is to get as complete an assembly as possible. it is possible that different raw data sets may be better assembled by different programs, and thus researchers should perhaps cross-compare the best available ones on their data using the optimality criteria used here. merging assemblies performed with different programs is a frequently used approach in genome assembly projects, especially those that employ multiple sequencing technologies. application of this strategy to the problem of de novo transcriptome assembly appears particularly useful. while a merged assembly may simply sum the errors made by both programs, filtering the resultant second stage contigs on the basis of their containing first stage contigs from both of the starting sets generates a transcriptome assembly that is on average longer and that better represents known or assumed reference sequences than do either of the starting contig sets. a preferred assembly strategy is thus to perform initial assemblies with multiple high-quality assemblers and then to merge these using a traditional olc assembler such as cap second-order contigs that have support from multiple first-order assemblies are much more likely to be accurate. need for new assemblers all the de novo assemblers in this study, with the exception of clc, use the olc assembly strategy. clc uses de bruijn graphs. as read lengths and throughput increase and sequencing costs come down, an average transcriptome assembly project for a non-model organism may comprise several million base reads. under the olc paradigm, the computational time for assembling more reads rises exponentially with data complexity because the number of pairwise comparisons to detect overlaps will increase, and the layout graphs will be harder to resolve into a consensus sequence. the clc de novo assembler is clearly a step in the right direction because its de bruijn graph algorithm achieves reasonable results in very little time on large datasets. other de bruijn graph assemblers such as oases and abyss are promising but are currently not suitable for most non-normalised pyrosequencing de novo transcriptome assembly projects because many transcripts have very low coverage that cannot be assembled reliably. perhaps the way forward is to use the de bruijn graph method for transcripts with high coverage and the olc method for transcripts with low coverage. we have shown that an assembly-merging strategy delivers robust contigs from intermediate assemblies produced by current programs, and this strategy is likely to be of utility in deriving the best assemblies from future programs as well. 
as different assemblers use different underlying algorithms to generate contigs, we also explored merging of assemblies and found that the merged datasets not only aligned better to reference sequences than individual assemblies, but were also more consistent in the number and size of contigs. seqman assemblies performed best on the criterion of recapitulating known transcripts, and had more novel sequence than the other assemblers, but generated an excess of small, redundant contigs. combining differently optimal assemblies from different programs however gave a more credible final product, and this strategy is recommended. although no single assembler performed best on all our criteria, newbler gave longer contigs, better alignments to some reference sequences, and was fast and easy to use. roche pyrosequencing has become a method of choice for generating transcriptome data from non-model organisms. we have carried out a systematic comparison of five assemblers to establish best practices for transcriptome assemblies, using a new dataset from the parasitic nematode litomosoides sigmodontis. transcriptome assemblies are smaller than genome assemblies and thus should be more computationally tractable, but are often harder because individual contigs can have highly variable read coverage.
 roche pyrosequencing has become a method of choice for generating transcriptome data from non-model organisms. once the tens to hundreds of thousands of short reads have been produced, it is important to correctly assemble these to estimate the sequence of all the transcripts. most transcriptome assembly projects use only one program for assembling pyrosequencing reads, but there is no evidence that the programs used to date are optimal. we have carried out a systematic comparison of five assemblers to establish best practices for transcriptome assemblies, using a new dataset from the parasitic nematode litomosoides sigmodontis.  although no single assembler performed best on all our criteria, newbler gave longer contigs, better alignments to some reference sequences, and was fast and easy to use. seqman assemblies performed best on the criterion of recapitulating known transcripts, and had more novel sequence than the other assemblers, but generated an excess of small, redundant contigs. the remaining assemblers all performed almost as well, with the exception of newbler, which generated assemblies that had significantly lower total length. as different assemblers use different underlying algorithms to generate contigs, we also explored merging of assemblies and found that the merged datasets not only aligned better to reference sequences than individual assemblies, but were also more consistent in the number and size of contigs.  transcriptome assemblies are smaller than genome assemblies and thus should be more computationally tractable, but are often harder because individual contigs can have highly variable read coverage. comparing single assemblers, newbler performed best on our trial data set, but other assemblers were closely comparable. combining differently optimal assemblies from different programs however gave a more credible final product, and this strategy is recommended. 