Package: needletail
NeedletailNeedletail is a MIT-licensed, minimal-copying FASTA/FASTQ parser andk-mer processing library for Rust.The goal is to write a fastandwell-tested set of functions that more specialized bioinformatics programs can use.
Needletail's goal is to be as fast as thereadfqC library at parsing FASTX files and much (i.e. 25 times) faster than equivalent Python implementations atk-mer counting.Exampleexterncrateneedletail;useneedletail::{parse_fastx_file,Sequence,FastxReader};fnmain(){letfilename="tests/data/28S.fasta";letmutn_bases=0;letmutn_valid_kmers=0;letmutreader=parse_fastx_file(&filename).expect("valid path/file");whileletSome(record)=reader.next(){letseqrec=record.expect("invalid record");// keep track of the total number of basesn_bases+=seqrec.num_bases();// normalize to make sure all the bases are consistently capitalized and// that we remove the newlines since this is FASTAletnorm_seq=seqrec.normalize(false);// we make a reverse complemented copy of the sequence first for// `canonical_kmers` to draw the complemented sequences from.letrc=norm_seq.reverse_complement();// now we keep track of the number of AAAAs (or TTTTs via// canonicalization) in the file; note we also get the position (i.0;// in the event there were `N`-containing kmers that were skipped)// and whether the sequence was complemented (i.2) in addition to// the canonical kmer (i.1)for(_,kmer,_)innorm_seq.canonical_kmers(4,&rc){ifkmer==b"AAAA"{n_valid_kmers+=1;}}}println!("There are {} bases in your file.",n_bases);println!("There are {} AAAAs in your file.",n_valid_kmers);}InstallationNeedletail requiresrustandcargoto be installed.
Please use either your local package manager (homebrew,apt-get,pacman, etc) or install these viarustup.Once you have Rust set up, you can include needletail in yourCargo.tomlfile like:[dependencies]needletail="0.4"To install needletail itself for development:gitclonehttps://github.com/onecodex/needletail
cargotest# to run testsPythonDocumentationFor a real example, you can refer totest_python.py.The python library only raise one type of exception:NeedletailError.There are 2 ways to parse a FASTA/FASTQ: one if you have a string (parse_fastx_string(content: str)) or a path to a file
(parse_fastx_file(path: str)). Those functions will raise if the file is not found or if the content is invalid and will return
an iterator.fromneedletailimportparse_fastx_file,NeedletailError,reverse_complement,normalize_seqtry:forrecordinparse_fastx_file("myfile.fastq"):print(record.id)print(record.seq)print(record.qual)exceptNeedletailError:print("Invalid Fastq file")A record has the following shape:classRecord:id:strseq:strqual:Optional[str]defis_fasta(self)->booldefis_fastq(self)->booldefnormalize(self,iupac:bool)Note thatnormalize(seehttps://docs.rs/needletail/0.4.1/needletail/sequence/fn.normalize.htmlfor what it does) will mutateself.seq.
It is also available as thenormalize_seq(seq: str, iupac: bool)function which will return the normalized sequence in this case.Lastly, there is also areverse_complement(seq: str)that will do exactly what it says. This will not raise an error if you pass some invalid
characters.BuildingTo work on the Python library on a Mac OS X/Unix system (requires Python 3):pipinstallmaturin# finally, install the library in the local virtualenvmaturindevelop--cargo-extra-args="--features=python"To build the binary wheels and push to PyPI# The Mac build requires switching through a few different python versions
maturin build --features python --release --strip

# The linux build is automated through cross-compiling in a docker image
docker run --rm -v $(pwd):/io ghcr.io/pyo3/maturin:main build --features=python --release --strip -f
twine upload target/wheels/*Getting HelpQuestions are best directed as GitHub issues. We plan to add more documentation soon, but in the meantime "doc" comments are included in the source.ContributingPlease do! We're happy to discuss possible additions and/or accept pull requests.AcknowledgementsStarting from 0.4, the parsers algorithms is taken fromseq_io. While it has been slightly modified, it is mainly
coming from that library. Links to the original files are available insrc/parser/fast{a,q}.rs.