Package: pipely
The light pipeline librarypipelycan execute any class or any sequence of classes in any order. Install with:pipinstallpipely1. Quick StartTo build a pipeline with executable classes, create a config.yamlfile in the following format:steps:
    [step_name_1]:
        exec: [relative path to a file]:[class to execute]
    [step_name_2]:
        exec: [relative path to a file]:[class to execute]
    [step_name_3]:
        exec: [relative path to a file]:[class to execute]
        depends_on:
        - [step_name_1]
        - [step_name_2][steps]names should be unique;depends_ondefines order and enables pipely to detect independent steps and execute them in parallel;the executable classes should have a__call__method (see example below);Then trigger the pipeline in cli:python-mpipelyfrom-pipeline<file.yaml>[dict.json]<file.yaml>(required) is the path to a yaml config file (any format):../../file.yaml, orpath/to/file.yaml, orfile.yaml.[dict.json](optional argument) is the path to a shared dictionaryjsonfile if value exchange between classes is needed (more in Section 1.2.)1.1. ExampleLet's create atest.yamlconfig file:steps:
    a1_print:
        exec: src/file1.py:firstA
    a2_print:
        exec: src/file1.py:secondA
    final_print:
        exec: src/file2.py:printDone
        depends_on:
        - a1_print
        - a2_printdepends_onparameter sets the following order for pipely:firstly executea1_printanda2_printin paralleland then executefinal_printLet's look at executable classes. To use pipely, the executable classes should have a__call__method, as shown below (checkexample/src/):#example/src/file1.pyclassfirstA:defrun(self):a=32print(a)def__call__(self):#call methodself.run()classsecondA:defrun(self):a=12print(a)def__call__(self):#call methodself.run()#example/src/file2.pyclassprintDone:defrun(self):print("Done.")def__call__(self):#call methodself.run()To start pipely, type in cli:python-mpipelyfrom-pipelinetest.yaml1.2. Example w/ a shared dictionaryLet's create atestContext.yamlconfig file:steps:
    a_first:
        exec: src/file1_shared.py:firstA
    a_second:
        exec: src/file1_shared.py:secondA
    a_sum:
        exec: src/file2_shared.py:aSum
        depends_on:
        - a_first
        - a_second
    a_sum_print:
        exec: src/file3_shared.py:aSumPrint
        depends_on:
        - a_sumdepends_onparameter sets the following order for pipely:executesa_firstanda_secondin parallelthen executesa_sum, which sums up both a's in the previous stepsfinally executesa_sum_print, which prints the final result previously calculated ina_sumLet's look at executale classes to understand how values are transferred between them (checkexample/srcfolder):#example/src/file1_shared.pyclassfirstA:defrun(self):a=32self.result=adef__call__(self,context):#include contextself.run()context["a1"]=self.result#to save into shared dictionaryclasssecondA:defrun(self):a=12self.result=adef__call__(self,context):#include contextself.run()context["a2"]=self.result#to save into shared dictionaryNow we can use previously saved valuesa1anda2in another class, as shown below:#example/src/file2_shared.pyclassaSum:defrun(self,context):#include contexta1=context["a1"]#to extract from shared dictionarya2=context["a2"]#to extract from shared dictionaryself.result=a1+a2def__call__(self,context):#include contextself.run(context)#to run the functioncontext["aSum"]=self.result#and save into shared dictionaryThen trigger the pipeline in cli with an optional second argument--context-path, which the path to a shared dictionaryexample_context.json:python-mpipelyfrom-pipelinetestContext.yaml--context-pathexample_context.json2. Imperative wayPipely can also trigger a specific class from a specific .py file.python-mpipelyfrom-class<path/to/file.py>:<TestClass>Below is an example of a command that triggers aprintDoneclass fromsrc/file4.pyfile.python3-mpipelyfrom-classsrc/file4.py:printDoneIf your class needs to operate on a shared dictionary, the commandfrom-classcould use an optional second argument--context-path. This argument awaits a path to a json representing the shared dictionary.python-mpipelyfrom-classsrc/file4.py:printShared--context-pathexample_context.json