Package: liteflow.core
LiteFlowLiteFlow is a Python library for running workflows.  Think: long running processes with multiple tasks that need to track state.  It supports pluggable persistence and concurrency providers to allow for multi-node clusters.InstallationBasic ConceptsStepsPassing data between stepsEventsFlow controlParallel ForEachWhile conditionIf conditionHostPersistenceMulti-node clustersSamplesInstallationInstall the "liteflow.core" package> pip install liteflow.coreBasic ConceptsStepsA workflow consists of a series of connected steps.  Each step produces an outcome value and subsequent steps are triggered by subscribing to a particular outcome of a preceeding step.
Steps are usually defined by inheriting from the StepBody abstract class and implementing therunmethod.First we define some stepsfromliteflow.coreimport*classHello(StepBody):defrun(self,context:StepExecutionContext)->ExecutionResult:print("Hello world")classGoodbye(StepBody):defrun(self,context:StepExecutionContext)->ExecutionResult:print("Goodbye")returnExecutionResult.next()Then we define the workflow structure by composing a chain of steps.classMyWorkflow(Workflow):defid(self):return"MyWorkflow"defversion(self):return1defbuild(self,builder:WorkflowBuilder):builder\.start_with(Hello)\.then(Goodbye)Theidandversionproperties are used by the workflow host to identify a workflow definition.Each running workflow is persisted to the chosen persistence provider between each step, where it can be picked up at a later point in time to continue execution.  The outcome result of your step can instruct the workflow host to defer further execution of the workflow until a future point in time or in response to an external event.The first time a particular step within the workflow is called, the persistenceData property on the context object isNone.  The ExecutionResult produced by therunmethod can either cause the workflow to proceed to the next step by providing an outcome value, instruct the workflow to sleep for a defined period or simply not move the workflow forward.  If no outcome value is produced, then the step becomes re-entrant by setting persistenceData, so the workflow host will call this step again in the future buy will popluate the persistenceData with it's previous value.Passing data between stepsEach step is intended to be a black-box, therefore they support inputs and outputs.  Each workflow instance carries a data property for holding 'workflow wide' data that the steps can use to communicate.The following sample shows how to define inputs and outputs on a step, it then shows how to map the inputs and outputs to properties on the workflow data property.#Our workflow step with inputs and outputsclassAddNumbers(StepBody):def__init__(self):self.input1=0self.input2=0self.output=0defrun(self,context:StepExecutionContext)->ExecutionResult:self.output=self.input1+self.input2returnExecutionResult.next()#A class to hold workflow wide dataclassMyData:def__init__(self):self.value1=0self.value2=0self.value3=0#Our workflow definition with mapped inputs & outputsclassMyWorkflow(Workflow):defbuild(self,builder:WorkflowBuilder):builder\.start_with(Hello)\.then(AddNumbers)\.input('input1',lambdadata,context:data.value1)\.input('input2',lambdadata,context:data.value2)\.output('value3',lambdastep:step.output)\.then(Goodbye)EventsA workflow can also wait for an external event before proceeding.  In the following example, the workflow will wait for an event called"event1"with a key of"key1".  Once an external source has fired this event, the workflow will wake up and continue processing.classMyWorkflow(Workflow):defbuild(self,builder:WorkflowBuilder):builder\.start_with(Hello)\.wait_for('event1',lambdadata,context:'key1')\.then(Goodbye)#External events are published via the host#All workflows that have subscribed to event1, key1, will be passed "hello"host.publish_event('event1','key1','hello')#Data from the published event can be captured and mapped to the workflow data object with an output on the WaitFor stepclassMyWorkflow(Workflow):defbuild(self,builder:WorkflowBuilder):builder\.start_with(Hello)\.wait_for('event1',lambdadata,context:'key1')\.output('captured_value',lambdastep:step.event_data)\.then(Goodbye)Flow ControlParallel ForEachclassDoStuff(StepBody):defrun(self,context:StepExecutionContext)->ExecutionResult:print(f"doing stuff...{context.execution_pointer.context_item}")returnExecutionResult.next()classMyWorkflow(Workflow):defbuild(self,builder:WorkflowBuilder):builder\.start_with(Hello)\.for_each(lambdadata,context:["abc","def","xyz"])\.do(lambdax:\x.start_with(DoStuff))\.then(Goodbye)While conditionclassMyWorkflow(Workflow):defbuild(self,builder:WorkflowBuilder):builder\.start_with(Hello)\.while_(lambdadata,context:data.value1<3)\.do(lambdado:\do.start_with(DoStuff)\.input('my_value',lambdadata,context:data.value1)\.output('value1',lambdastep:step.your_value))\.then(Goodbye)If conditionclassMyWorkflow(Workflow):defbuild(self,builder:WorkflowBuilder):builder\.start_with(Hello)\.if_(lambdadata,context:data.value1>3)\.do(lambdax:\x.start_with(DoStuff))\.then(Goodbye)HostThe workflow host is the service responsible for executing workflows.  It does this by polling the persistence provider for workflow instances that are ready to run, executes them and then passes them back to the persistence provider to by stored for the next time they are run.  It is also responsible for publishing events to any workflows that may be waiting on one.UsageWhen your application starts, create a WorkflowHost service usingconfigure_workflow_host,  callregister_workflow, so that the workflow host knows about all your workflows, and then callstartto fire up the event loop that executes workflows.  Use thestart_workflowmethod to initiate a new instance of a particular workflow.fromliteflow.coreimport*host=configure_workflow_host()host.register_workflow(MyWorkflow())host.start()wid=host.start_workflow("MyWorkflow",1,None)PersistenceSince workflows are typically long running processes, they will need to be persisted to storage between steps.
There are several persistence providers available as separate packages.Memory Persistence Provider(Default provider, for demo and testing purposes)MongoDB(more to come soon...)Multi-node clustersBy default, the WorkflowHost service will run as a single node using the built-in queue and locking providers for a single node configuration.  Should you wish to run a multi-node cluster, you will need to configure an external queueing mechanism and a distributed lock manager to co-ordinate the cluster.  These are the providers that are currently available.Queue ProvidersSingleNodeQueueProvider(Default built-in provider)AzureRabbitMQ(coming soon...)Distributed lock managersLocalLockProvider(Default built-in provider)AzureRedis Redlock(coming soon...)SamplesHello WorldPassing DataEventsParallel ForEachIf conditionWhile loopAuthorsDaniel Gerlag-Initial workLicenseThis project is licensed under the MIT License - see theLICENSE.mdfile for details