Package: itrs
ItrsRust's iterators in python, implemented in Rust!Itrsis a python native module that exposes to python anIteratorinterface similar to Rust'sIteratortrait.This means that, given the following innocent fibonacci generator:>>>deffibo():...yield1...yield2...current,next=1,2...whileTrue:...current,next=next,current+next...yieldnext...instead of the following python code:>>>importitertools>>># Euler problem #2>>>sum(xforxinitertools.takewhile(lambdax:x<4_000_000,fibo())ifx%2==0)4613732you can now write the following:>>>fromitrsimportItrs>>>(Itrs(fibo())....take_while(lambdax:x<4_000_000)....filter(lambdax:x%2==0)....sum(0)...)4613732But why, you madman?I dislike python's default iterator syntax that uses functions as combinators, because it becomes very difficult to read when using several of them. Some useful combinators and folding functions (such ascount) are also missing, or available only in itertools.A second reason is as an exercice in usingPyO3and rust'sIteratortrait. This also allowed me to play withResultandOptiontypes so I could get the type I wanted in all situations.Also, because I'm a madman 🤪!How do I use this?From PyPIBuild from PyPI is only supported for x64 Linux at the moment.Usingcargoyour favorite python installation method from PyPI:$pipinstallitrsThen, from the virtualenv in which you --obviously-- executed the previous command, open a python interpreter and:>>>fromitrsimportItrs>>>>>># you can create Itrs objects from any iterable>>>it_array=Itrs([0,1,2])>>>it_str=Itrs("Intel the Beagle")>>>defi_yield():>>>foriinrange(10):>>>yieldi>>>it_yield=Itrs(i_yield())>>>>>># you can iterate on any itrs object using normal python syntax>>>foreleminit_array:>>>print(elem)012>>># Iterating a second time on a exhausted iterator yields no further result>>>foreleminit_array:>>>print(elem)>>># you can call methods on Itrs objects to create new iterators or produce results.>>># That's the whole selling point of this!>>>it_str.filter(lambdax:x==' ').count()2>>>[xforxinit_yield.skip(1).filter(lambdax:x%2==0).map(lambdax:x*x)][4,16,36,64]Compiling the library from RustCompiling from sourceshouldwork with Linux, Windows or OSX indifferently, but was only tested under Linux.
As PyO3 requires nightly at the moment, a nightly toolchain of Rust is required to compile this repository.The repository contains arust-toolchainfile containing a nightly toolchain that is known to work with the project.If usingrustup, you can install that toolchain with:$rustuptoolchaininstallnightly-2019-09-04Then, after cloning the repository, just:$cargobuild--releaseAfter the build completes, you will need to rename the produced binary by dropping thelibprefix, so it can be imported by python as intended:$mvtarget/release/{lib,}itrs.soAlternatively, you can also build a wheel usingmaturin$pipinstallmaturin
$maturinbuild--releaseand then install the wheel with:$pipinstalltarget/wheels/itrs-0.1.0-cp38-cp38-manylinux1_x86_64.whlYou're done! You can now importitrsfrom a python shell with thetarget/releasedirectory in your python path.Should I use this in production?No.Even if python iterators suck, they are the standard, and using an external library to reimplement iterators should trigger a "WAT?" reaction in everyone around youUsing python iterators, you can write new function combinators. Adding new combinators toItrswould involve monkey patching, and is not possible sinceItrsis an extension type. Note that in rust, new combinators can be added using method syntax thanks to the trait system. This is what theitertools cratedoes, for example.BTW, all iterator combinators aren't added as of yet.Performance is bad. There are several reasons for this:Rust uses monomorphization of generic type parameters. This avoid the indirection of runtime polymorphism, and allows to inline the iterator code, which can then enable efficient optimizations at compile time. However, Python being an interpreted language without generics,it cannot take advantage of monomorphization, so the current design performs one allocation per combinator, which is far from efficient.The current implementation is naive and only attempts to reuse the functions defined by theIteratortrait as much as possible. It also usesRcrather than PyO3's providedPyObject, which might be inefficient considering we're already in the python runtime?