Package: durabletask
Durable Task Client SDK for PythonThis repo contains a Python client SDK for use with theDurable Task Framework for GoandDapr Workflow. With this SDK, you can define, schedule, and manage durable orchestrations using ordinary Python code.⚠️This SDK is currently under active development and is not yet ready for production use.⚠️Note that this project isnotcurrently affiliated with theDurable Functionsproject for Azure Functions. If you are looking for a Python SDK for Durable Functions, please seethis repo.Supported patternsThe following orchestration patterns are currently supported.Function chainingAn orchestration can chain a sequence of function calls using the following syntax:# simple activity function that returns a greetingdefhello(ctx:task.ActivityContext,name:str)->str:returnf'Hello{name}!'# orchestrator function that sequences the activity callsdefsequence(ctx:task.OrchestrationContext,_):result1=yieldctx.call_activity(hello,input='Tokyo')result2=yieldctx.call_activity(hello,input='Seattle')result3=yieldctx.call_activity(hello,input='London')return[result1,result2,result3]You can find the full samplehere.Fan-out/fan-inAn orchestration can fan-out a dynamic number of function calls in parallel and then fan-in the results using the following syntax:# activity function for getting the list of work itemsdefget_work_items(ctx:task.ActivityContext,_)->List[str]:# ...# activity function for processing a single work itemdefprocess_work_item(ctx:task.ActivityContext,item:str)->int:# ...# orchestrator function that fans-out the work items and then fans-in the resultsdeforchestrator(ctx:task.OrchestrationContext,_):# the number of work-items is unknown in advancework_items=yieldctx.call_activity(get_work_items)# fan-out: schedule the work items in parallel and wait for all of them to completetasks=[ctx.call_activity(process_work_item,input=item)foriteminwork_items]results=yieldtask.when_all(tasks)# fan-in: summarize and return the resultsreturn{'work_items':work_items,'results':results,'total':sum(results)}You can find the full samplehere.Human interaction and durable timersAn orchestration can wait for a user-defined event, such as a human approval event, before proceding to the next step. In addition, the orchestration can create a timer with an arbitrary duration that triggers some alternate action if the external event hasn't been received:defpurchase_order_workflow(ctx:task.OrchestrationContext,order:Order):"""Orchestrator function that represents a purchase order workflow"""# Orders under $1000 are auto-approvediforder.Cost<1000:return"Auto-approved"# Orders of $1000 or more require manager approvalyieldctx.call_activity(send_approval_request,input=order)# Approvals must be received within 24 hours or they will be canceled.approval_event=ctx.wait_for_external_event("approval_received")timeout_event=ctx.create_timer(timedelta(hours=24))winner=yieldtask.when_any([approval_event,timeout_event])ifwinner==timeout_event:return"Canceled"# The order was approvedyieldctx.call_activity(place_order,input=order)approval_details=approval_event.get_result()returnf"Approved by '{approval_details.approver}'"As an aside, you'll also notice that the example orchestration above works with custom business objects. Support for custom business objects includes support for custom classes, custom data classes, and named tuples. Serialization and deserialization of these objects is handled automatically by the SDK.You can find the full samplehere.Feature overviewThe following features are currently supported:OrchestrationsOrchestrations are implemented using ordinary Python functions that take anOrchestrationContextas their first parameter. TheOrchestrationContextprovides APIs for starting child orchestrations, scheduling activities, and waiting for external events, among other things. Orchestrations are fault-tolerant and durable, meaning that they can automatically recover from failures and rebuild their local execution state. Orchestrator functions must be deterministic, meaning that they must always produce the same output given the same input.ActivitiesActivities are implemented using ordinary Python functions that take anActivityContextas their first parameter. Activity functions are scheduled by orchestrations and have at-least-once execution guarantees, meaning that they will be executed at least once but may be executed multiple times in the event of a transient failure. Activity functions are where the real "work" of any orchestration is done.Durable timersOrchestrations can schedule durable timers using thecreate_timerAPI. These timers are durable, meaning that they will survive orchestrator restarts and will fire even if the orchestrator is not actively in memory. Durable timers can be of any duration, from milliseconds to months.Sub-orchestrationsOrchestrations can start child orchestrations using thecall_sub_orchestratorAPI. Child orchestrations are useful for encapsulating complex logic and for breaking up large orchestrations into smaller, more manageable pieces.External eventsOrchestrations can wait for external events using thewait_for_external_eventAPI. External events are useful for implementing human interaction patterns, such as waiting for a user to approve an order before continuing.Continue-as-new (TODO)Orchestrations can be continued as new using thecontinue_as_newAPI. This API allows an orchestration to restart itself from scratch, optionally with a new input.Suspend, resume, and terminateOrchestrations can be suspended using thesuspend_orchestrationclient API and will remain suspended until resumed using theresume_orchestrationclient API. A suspended orchestration will stop processing new events, but will continue to buffer any that happen to arrive until resumed, ensuring that no data is lost. An orchestration can also be terminated using theterminate_orchestrationclient API. Terminated orchestrations will stop processing new events and will discard any buffered events.Retry policies (TODO)Orchestrations can specify retry policies for activities and sub-orchestrations. These policies control how many times and how frequently an activity or sub-orchestration will be retried in the event of a transient error.Getting StartedPrerequisitesPython 3.8A Durable Task-compatible sidecar, likeDapr WorkflowInstalling the Durable Task Python client SDKInstallation is currently only supported from source. Ensure pip, setuptools, and wheel are up-to-date.python3-mpipinstall--upgradepipsetuptoolswheelTo install this package from source, clone this repository and run the following command from the project root:python3-mpipinstall.Run the samplesSee theexamplesdirectory for a list of sample orchestrations and instructions on how to run them.DevelopmentThe following is more information about how to develop this project. Note that development commands require thatmakeis installed on your local machine. If you're using Windows, you can installmakeusingChocolateyor use WSL.Generating protobufsProtobuf definitions are stored in the./submodules/durabletask-protodirectory, which is a submodule. To update the submodule, run the following command from the project root:gitsubmoduleupdate--initOnce the submodule is available, the corresponding source code can be regenerated using the following command from the project root:makegen-protoRunning unit testsUnit tests can be run using the following command from the project root. Unit testsdon'trequire a sidecar process to be running.maketest-unitRunning E2E testsThe E2E (end-to-end) tests require a sidecar process to be running. You can use the Dapr sidecar for this or run a Durable Task test sidecar using the followingdockercommand:dockerrun--namedurabletask-sidecar-p4001:4001--env'DURABLETASK_SIDECAR_LOGLEVEL=Debug'--rmcgillum/durabletask-sidecar:lateststart--backendEmulatorTo run the E2E tests, run the following command from the project root:maketest-e2eContributingThis project welcomes contributions and suggestions.  Most contributions require you to agree to a
Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
the rights to use your contribution. For details, visithttps://cla.opensource.microsoft.com.When you submit a pull request, a CLA bot will automatically determine whether you need to provide
a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions
provided by the bot. You will only need to do this once across all repos using our CLA.This project has adopted theMicrosoft Open Source Code of Conduct.
For more information see theCode of Conduct FAQor
contactopencode@microsoft.comwith any additional questions or comments.TrademarksThis project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft
trademarks or logos is subject to and must followMicrosoft's Trademark & Brand Guidelines.
Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.
Any use of third-party trademarks or logos are subject to those third-party's policies.