Package: tubo
Tubo is a library that provides a simple pipeline system for Python.Unix pipe system is an excellent example of the concept ofseparation of responsibility. Each utility does a single thing well. This increases readability, maintainability of code and code reuse. Tubo wants to bring this abstraction to Python.InstallationpipinstalltuboUsageYou have a source of iterable items and you want to perform various operations on them. In a Unix-like system you would write something like this:catfoo.txt|op1|op2|op3Using Tubo, instead, you would use Python and you would write something like this:>>>output=tubo.pipeline(file('foo.txt'),op1,op2,op3)And the output would be available for you, to print it or to further transform it as you prefer. The advantage is that you can write the operations in Python, giving you a lot of flexibility.Create a pipelineThe central part of Tubo is the methodtubo.pipeline. It accepts an arbitrary number of arguments, the first being adata sourceand the following beingoperationson iterable data, defined using python generators.Each operation should `yield` something, so that the following operation can work.Example: capitalize words that contain ailetter.text=['italy','germany','brazil','france','england','argentina','peru','united states','australia','sweden','china','poland','portugal']defcapitalize(lines):forlineinlines:forwordinline.split(","):yieldword.capitalize()deffilter_wordwith_i(words):forwordinwords:if'i'inword:yieldwordoutput=tubo.pipeline(text,filter_wordwith_i,capitalize,)At this point, output is an iterable, and we can do anything we want with it. We can print it or further transform it.Merge two or more iterablesSometimes, you need to write functions that take two or more inputs, and process them. In this case, you need to write an operation that accepts alist of iterables.Example: interleave lines from two or more files (such as the utilitypaste)definterleave(listoflines):forlinesinitertools.izip(*listoflines):yield''.join(lines)output=tubo.pipeline((file('file1.txt'),file('file2.txt')),interleave)Consume iterators at C-speedOnce you have your pipeline, it’s time to consume it.tubo.consume(output)# Equivalent to:## for element in output:#     passThis consumes the iterator at C-speed, and usesthis recipe.ExamplesReverse text of unique lines, append the number of linesdefuniq(lines):seen=set()forlineinlines:iflinenotinseen:seen.add(line)yieldlinedefreverse_string(lines):forlineinlines:yield''.join(reversed(line))defappend_nlines(lines):fornlines,lineinenumerate(lines):yieldlineyield"\nTotal Number of lines:{}".format(nlines+1)output=tubo.pipeline(open(filename),uniq,reverse_string,append_nlines,)Concatenate two files 1st wordsWhen we need to merge two inputs, or two results of different pipes, we will use the functionsmergeandmerge_longest, which willdefselect_Nth_word(N,lines):forlineinlines:yieldline.split(' ')[N]select_first_word=functools.partial(select_Nth_word,0)select_second_word=functools.partial(select_Nth_word,1)defconcatenate(words):forword1,word2inwords:yield"{}{}".format(word1,word2)pipeline1=tubo.pipeline(open(fname1),select_first_word,)pipeline2=tubo.pipeline(open(fname2),select_second_word,)output=tubo.pipeline(tubo.merge(pipeline1,pipeline2,),concatenate)CreditsThe library was inspired from apost by Christoph Rauch.History0.1.1 (2014-08-02)Now works for Python3. Wheel added.0.1.0 (2014-08-02)Initial concept.