Package: sequent
Table of ContentsOverviewSequent InterfaceDistributed OperationRecoveryResourcesAdditional InformationOverviewSequentprovides programmer with interface to create flow.Sequentis useful when there are multiple batch processes that should link together in processing flow.For example, programs A1, A2 extract data from database, program B process the extracted files, and programs C1 and C2 transmit process information to remote client.Sequentcan be used to create the flow where A1, A2 would run concurrently, then program B would run.  When B is successful, programs C1 and C2 would run.SequentusesEventor(https://github.com/Acrisel/eventor) as its underline infrastructure. With that, it gainsEventor’s recovery capabilities.  When running in recovery, successful steps can be skipped.Considering the above example, this can be useful when C2 transmission program fails. Recovery run would only execute it with out redoing potentially expensive work that was already completed (A1, A2, B, and C1)It would be easier to show an example. In this example,Step s2would run afterStep s1would loop twice. When they are done,Step s3will run.  This flow would be repeat twice.Simple Example1importsequentasseq2importlogging34defprog(progname,success=True):5logger=logging.getLogger(os.getenv("SEQUENT_LOGGER_NAME"))6logger.info("doing what{}is doing".format(progname))7ifnotsuccess:8raiseException("{}failed".format(progname))9returnprogname1011myflow=seq.Sequent(config={'sleep_between_loops':0.05,12'LOGGING':{'logging_level':logging.DEBUG}})1314s1=myflow.add_step('s1',repeats=[1,2])1516s11=s1.add_step('s11',repeats=[1,2,])1718s111=s11.add_step('s111',func=prog,kwargs={'progname':'prog1'})19s112=s11.add_step('s112',func=prog,kwargs={'progname':'prog2'},20requires=((s111,seq.STEP_SUCCESS),))2122s12=s1.add_step('s12',func=prog,kwargs={'progname':'prog3'},23require=s((s11,seq.STEP_SUCCESS),))2425s2=myflow.add_step('s2',func=prog,kwargs={'progname':'prog4'},26requires=((s1,seq.STEP_SUCCESS),))2728myflow.run()Example OutputThe above example with provide the following log output. Note more detailed logging activities was stripped off.  Only actual shows actual program activity is shown.1[2016-12-0711:14:29,761][INFO][Eventorstorefile:/sequent/example/runly00.run.db]2...3[2016-12-0711:14:29,962][INFO][doingwhatprog1isdoing]4...5[2016-12-0711:14:30,124][INFO][doingwhatprog2isdoing]6...7[2016-12-0711:14:30,358][INFO][doingwhatprog1isdoing]8...9[2016-12-0711:14:30,587][INFO][doingwhatprog2isdoing]10...11[2016-12-0711:14:30,908][INFO][doingwhatprog3isdoing]12...13[2016-12-0711:14:31,234][INFO][doingwhatprog1isdoing]14...15[2016-12-0711:14:31,407][INFO][doingwhatprog2isdoing]16...17[2016-12-0711:14:31,657][INFO][doingwhatprog1isdoing]18...19[2016-12-0711:14:31,894][INFO][doingwhatprog2isdoing]20...21[2016-12-0711:14:32,240][INFO][doingwhatprog3isdoing]22...23[2016-12-0711:14:32,565][INFO][doingwhatprog4isdoing]24...25[2016-12-0711:14:32,713][INFO][Processingfinishedwith:success]Code HighlightsFlow diagram:+--S1----------------------+|||+--S11----------+|||||||S111->S112|->S12|->S2|+---------------+|+--------------------------+For simplicity, code definition of prog (line 6) serves as reusable activity for all the steps in this example.ASequentobject is defined (line 12) to host myflow. By default, Sequent’s Eventor loops on events and steps.  By defaults it sleeps one second between loops. Here ‘sleep_between_loops’ changes this setting to 0.05 seconds.myflow contains two steps,s1ands2.s1is a container step that would repeat twice (defined on line 15).s2is a processing step (defined on line 26).s1contains two steps.s11(line 17) iscontainerstep ands12is a processing step.s11contains two processing stepss111ands112(lines 19-20).Finally, on line 29 the flow is executed usingmyflow().loggeris set with in step program (line 5) to direct step logging into its dedicated log.Sequent InterfaceSequent Class InitiatorSequent signature in its most simplistic format:Sequent(name='',store='',run_mode=SEQ.SEQUENT_RESTART,recovery_run=None,config={},config_tag='')DescriptionSequent, when instantiated, provides interface to build program flow. When called upon,Sequentsteps are translated toEventorsteps andStep’srequiresare translated toEventor’sEventsandSteps’triggers.Sequent instantiation arguments are the same asEventor’s.Argsname: string id for Sequent object initiated.store: path to file that would store runnable (sqlite) information; if ‘:memory:’ is used, in-memory temporarystorage will be created.  If not provided, calling module path and name will be used
with db extension instead of ‘py’.run_mode: can be eitherRUN_RESTART(default) orRUN_RECOVER; in restart, new instance or the runwill be created. In recovery,recovery_run: ifRUN_RECOVERis used,recovery_runwill indicate specific instance of previously recoveryrun that would be executed.If not provided, latest run would be used.config: keyword dictionary of default configurations. Available keywords and their default values:NameDefault
ValueDescriptionworkdir/tmpplace to create necessry artifacts (not in use)logdir/var/log/eventorplace to create debug and error log filestask_construct‘process’method to use for execution of stepsmax_concurrent1maximum concurrent processing, if value <1, no
limit will be posestop_on_exceptionTrueif an exception occurs in a step, stop
all processes. If True, new processes will not
start. But running processes will be permitted
to finishsleep_between_loops1seconds to sleep between iteration of checking
triggers and tasksconfig_tag: key with in config where Sequent configuration starts.Sequent add_event methodadd_event(require=None)Argsrequires: logical expression ‘sqlalchemy’ style to automatically raise this expression.syntax:requires:(requires,requires,...)|or_(requires,requires,...)|eventif expression is of the first style, logicalandwill apply.the second expression will apply logicalor.the basic atom in expression isevenwhich is the product of add_event.ReturnsEvent object to use are require inadd_step.Sequent add_step methodadd_step(name,func,args=(),kwargs={},hosts=[],requires={},delay=0,acquires=[],releases=None,recovery={},config={})Argsname: string unique id for stepfunc: callable object that would be call at time if step executionargs: tuple of values that will be passed tofuncat callingkwargs: keywords arguments that will be passed tofuncat callinghosts: list of hosts step should run on. If not provided,localhostwill be used.ifrequires: mapping of step statuses such that when set of events, added step will be launched:statusdescriptionSTEP_READYset when task is ready to run (triggered)STEP_ACTIVEset when task is runningSTEP_SUCCESSset when task is successfulSTEP_FAILUREset when task failsSTEP_COMPLETEstands for success or failure of taskdelay: seconds to wait before executing step once is requires are available.  Actual executionmay be delayed further if resources needs to be acquired.acquires: list of tuples of resource pool and amount of resources to acquire before starting.releases: list of tuples of resources pool and amount of resources to release once completed.If None, defaults toacquires.  If set to empty list, none of the acquired resources would
be released.recovery: mapping of state status to how step should be handled in recovery:statusdefaultdescriptionSTEP_READYSTP_RERUNif in recovery and previous status is ready, rerunSTEP_ACTIVESTP_RERUNif in recovery and previous status is active, rerunSTEP_FAILURESTP_RERUNif in recovery and previous status is failure, rerunSTEP_SUCCESSSTP_SKIPif in recovery and previous status is success, skipconfig: keywords mapping overrides for step configuration.namedefaultdescriptionstop_on_exceptionTruestop flow if step ends with ExceptionReturnsStep object to use in add_assoc method.Sequent run methodrun(max_loops=-1)when callingrunmethod, information is built and loops evaluating events and task starts are executed.
In each loop events are raised and tasks are performed.  max_loops parameters allows control of how many
loops to execute.In simple example,myflow.run()engage Sequent’s run() method.Argsmax_loops: max_loops: number of loops to run.  If positive, limits number of loops.defaults to negative, which would run loops until there are no events to raise and
no task to run.ReturnsIf there was a failure that was not followed by event triggered, result will be False.Distributed OperationSequentcan operate Steps on distributed environment. A step can be associated with hosts usinghostsargument inadd_step.Sequentuses SSH to submit steps to remote host. This means that cluster needs to be configured with SSH keys. To set up the environment forSequentdistributed operation:Host from whichSequentprogram would be initiated, should be able to SSH to participating hosts without only using keys.SSH authorized_keys on each target host should has propercommandto initiate the right operation environment.  This may include activating the correct virtualenv.Optionally, set SSH backdoor to originated host. In the futureSequentmay use this backdoor, as callback.Software needs to be uniformly installed on all participating machines.Sequentmust be initiated with database configuration that is accessible from all participating hosts.Sequentand its remote agents would use that database to share  operation information. The database user needs to have permissions to create schema (if the associated schema is not created.) It also needs to have create table permissions.Anything passed to Sequent, predominately withadd_step, needs to be importable. For example in simple example:importexample_progsasexamples111=s11.add_step('s111',func=example.prog,kwargs={'progname':'prog1'})RecoveryRecovery allows rerun of a program in a way that it will skip successful steps.  To use recovery, store mast be physical (cannot use in-memory).According to step recovery setup, when in recovery, step may be skipped or rerun.  By default, only success statuses are skipped.Here is an example for recovery program and run.Recovery Example1importsequentassqnt2importlogging34appname=os.path.basename(__file__)5logger=logging.getLogger(appname)67defprog(flow,progname,step_to_fail=None,iteration_to_fail=''):8logger=logging.getLogger(os.getenv("SEQUENT_LOGGER_NAME"))9step_name=flow.get_step_name()10step_sequence=flow.get_step_sequence()11logger.info("doing what{}is doing (}/{})".format(progname,step_name,step_sequence))12ifstep_to_fail==step_nameandstep_sequence==iteration_to_fail:13raiseException("{}failed ({}/{})".format(progname,step_name,step_sequence))14returnprogname1516defbuild_flow(run_mode=sqnt.RUN_RESTART,run_id=None,step_to_fail=None,iteration_to_fail=''):17myflow=sqnt.Sequent(name=appname,run_mode=run_mode,run_id=run_id,config={'sleep_between_loops':0.05,},)1819s1=myflow.add_step('s1',repeats=[1,2])2021s11=s1.add_step('s11',repeats=[1,2,])2223s111=s11.add_step('s111',func=prog,kwargs={'flow':myflow,'progname':'prog1',24'step_to_fail':step_to_fail,25'iteration_to_fail':iteration_to_fail,})26s112=s11.add_step('s112',func=prog,kwargs={'flow':myflow,'progname':'prog2',27'step_to_fail':step_to_fail,28'iteration_to_fail':iteration_to_fail,},29requires=((s111,sqnt.STEP_SUCEESS),))3031s12=s1.add_step('s12',func=prog,kwargs={'flow':myflow,'progname':'prog3',32'step_to_fail':step_to_fail,33'iteration_to_fail':iteration_to_fail,},34requires=((s11,sqnt.STEP_SUCEESS),))3536s2=myflow.add_step('s2',func=prog,kwargs={'flow':myflow,'progname':'prog4',37'step_to_fail':step_to_fail,38'iteration_to_fail':iteration_to_fail,},39requires=((s1,sqnt.STEP_SUCEESS),))40returnmyflow4142# creating flow simulating failure43myflow=build_flow(step_to_fail='s1_s11_s111',iteration_to_fail='1.2.2')44myflow.run()4546run_id=myflow.run_id4748# creating recovery flow49myflow=build_flow(run_mode=RUN_RECOVER,run_id=run_id)50myflow.run()Example Output1[2016-12-0714:49:24,437][INFO][Eventorstorefile:/sequent/example/runly04.run.db]2...3[2016-12-0714:49:24,645][INFO][doingwhatprog1isdoing(s1_s11_s111/1.1.1)]4...5[2016-12-0714:49:24,805][INFO][doingwhatprog2isdoing(s1_s11_s112/1.1.1)]6...7[2016-12-0714:49:25,047][INFO][doingwhatprog1isdoing(s1_s11_s111/1.1.2)]8...9[2016-12-0714:49:25,272][INFO][doingwhatprog2isdoing(s1_s11_s112/1.1.2)]10...11[2016-12-0714:49:25,587][INFO][doingwhatprog3isdoing(s1_s12/1.1)]12...13[2016-12-0714:49:25,909][INFO][doingwhatprog1isdoing(s1_s11_s111/1.2.1)]14...15[2016-12-0714:49:26,073][INFO][doingwhatprog2isdoing(s1_s11_s112/1.2.1)]16...17[2016-12-0714:49:26,321][INFO][doingwhatprog1isdoing(s1_s11_s111/1.2.2)]18[2016-12-0714:49:26,323][INFO][[Steps1_s11_s111/1.2.2]Completed,status:TaskStatus.failure]19[2016-12-0714:49:26,397][ERROR][Exceptioninrun_action:20<Task(id='15',step_id='s1_s11_s111',sequence='1.2.2',recovery='0',pid='10276',status='TaskStatus.failure',created='2016-12-07 20:49:26.300030',updated='2016-12-07 20:49:26.311884')>]21[2016-12-0714:49:26,397][ERROR][Exception('prog1 failed (s1_s11_s111/1.2.2)',)]22[2016-12-0714:49:26,397][ERROR][File"/eventor/eventor/main.py",line63,intask_wrapper23result=step(seq_path=task.sequence)24File"/eventor/eventor/step.py",line82,in__call__25result=func(*func_args,**func_kwargs)26File"/sequent/example/runly04.py",line34,inprog27raiseException("%sfailed (%s/%s)"%(progname,step_name,step_sequence))]28[2016-12-0714:49:26,397][INFO][Stoppingrunningprocesses]29[2016-12-0714:49:26,401][INFO][Processingfinishedwith:failure]30[2016-12-0714:49:26,404][INFO][Eventorstorefile:/sequent/example/runly04.run.db]31...32[2016-12-0714:49:27,921][INFO][doingwhatprog1isdoing(s1_s11_s111/1.2.2)]33...34[2016-12-0714:49:28,159][INFO][doingwhatprog2isdoing(s1_s11_s112/1.2.2)]35...36[2016-12-0714:49:28,494][INFO][doingwhatprog3isdoing(s1_s12/1.2)]37...38[2016-12-0714:49:28,844][INFO][doingwhatprog4isdoing(s2/1)]39[2016-12-0714:49:28,845][INFO][[Steps2/1]Completed,status:TaskStatus.success]40[2016-12-0714:49:29,002][INFO][Processingfinishedwith:success]Example HighlightsThe functionbuild_flow(code line 14) build a Sequent flow similarly to simple example above.  Since no specific store is provided in Sequent instantiation, a default runner store is assigned (code line 15). In this build, steps will use default recovery directives whereby successful steps are skipped.The first build and run is done in lines 42-43. In this run, a parameter is passed to cause steps111in its fourth iteration to fail.  As a result, flow fails.  Output lines 1-29 is associated with the first run.The second build and run is then initiated.  In this run, parameter is set to a value that would pass steps111and run mode is set to recovery (code lines 45-46). Eventor skips successful steps and start executing from failed steps onwards. Output lines 30-40 reflects successful second run.Note that the second run required arun_idof the run that is reactivated.run_idis fetched from its corresponding attribute inSequentObjects.For prog to know when to default, it uses the following methods flow.get_step_name() and flow.get_step_sequence() (lines 7-8). Those Sequent methods allow access to Eventor step attributes. Another way
to access these attributes is via os.environ:name=os.getenv('SEQUENT_STEP_NAME')sequence=os.getenv('SEQUENT_STEP_SEQUENCE')recovery=os.getenv('SEQUENT_STEP_RECOVERY')logger_name=os.getenv('SEQUENT_LOGGER_NAME')Distributed ExampleResourcesadd_stepallows association of step with resources.  If acquires argument is provided, before step starts,Eventorwill attempt to reserve resources.  Step will be executed only when resources are secured.Whenreleaseargument is provided, resources resources listed as its value will be released when step is done.  If
release is None, whatever resources stated byacquireswould be released.  If the empty list is set as value, no
resource would be released.To use resources, program to use Resource and ResourcePool from acris.virtual_resource_pool.  Example for such definitions are below.Example for resources definitions1importsequentassqnt2fromacrisimportvirtual_resource_poolasvrp34classResources1(vrp.Resource):pass5classResources2(vrp.Resource):pass67rp1=vrp.ResourcePool('RP1',resource_cls=Resources1,policy={'resource_limit':2,}).load()8rp2=vrp.ResourcePool('RP2',resource_cls=Resources2,policy={'resource_limit':2,}).load()910myflow=sqnt.Sequent(config={'sleep_between_loops':0.05,},)11s1=myflow.add_step('s1',repeats=[1,2],acquires=[(rp1,2),])Additional InformationSequent github project (https://github.com/Acrisel/sequent) has additional examples with more complicated flows.