Package: avikom-camunda-client
avikom-camunda-clientThis fork ofyogeshrnaik/camunda-external-task-client-python3makes use ofasync/awaitand operates workers asynchronously rather than threaded. Additionally, it contains some convenience functions that wrap the Camunda REST API and speed up development.InstallationThe easiest way to install this client is viapip:pipinstallgit+https://github.com/OpenAvikom/camunda-external-task-client-python3If you want to run the examples, it might be easier to first clone the project and run everything locally:gitclonehttps://github.com/OpenAvikom/camunda-external-task-client-python3.gitcdcamunda-external-task-client-python3
pipinstall-e.# install module editableHello World ExampleThis example will show you how to let a Python client communicate with Camunda.Firs, we assume that Camunda is up and running. If that's not the case, you might want to have a look at theCamunda Platform Docker Images.
Second, we assume that Camunda is running atlocalhost:8080. If not, you need to adapt the example below accordingly.
Third, you need a way to deploy and run a BPMN process. TheCamunda Modeleris probably a good point of departure.The ModelThe filebpmn_process/hello_world.bpmncontains a very simple model:We have a start event (left circle), a ServiceTask to conduct and an end event (right circle). When you hit the 'Start Current Diagram' (the 'play' button in the modeler) nothing much will happen.
The process will be running though. You can checklocalhost:8080to make sure.Not let's have a closer look at the 'Hello World' external task (the square symbol with the cogs at the top right).
The interesting fields in the property panel areImplementationandTopic:Implementationtells Camunda, that the task should be resolved by anExternalworker andTopicis a string that more or less describes the task to be conducted.
We see that Camunda is now waiting for an external worker that is capable of conducting a task with the topicHelloWorldTask.
The Camunda Cockpit will show you a running process waiting forHelloWorldto be completed by an external process.
Now let's create a Python client which will subscribe to that topic and do tasks by just returning a success event.
The file can be found inexamples/hello_world.py:importaiohttpimportasynciofromcamunda.external_task.external_taskimportExternalTaskfromcamunda.external_task.external_task_workerimportExternalTaskWorkerfromcamunda.external_task.external_task_resultimportExternalTaskResultasyncdefmain():# let's create an async http context with aiohttp# aiohttp will close the connection when the worker returns (it won't though)asyncwithaiohttp.ClientSession()assession:# We create a worker with a task id and pass the http session as well as the REST endpoint of Camunda.# You need to change 'base_url' in case your Camunda engine is configured differently.worker=ExternalTaskWorker(worker_id=1,base_url="http://localhost:8080/engine-rest",session=session)print("waiting for a task ...")# Subscribe is an async function which will block until the worker is cancelled with `worker.cancel()`,# In this example, no one will do this. We will stop the program with Ctrl+C instead# When the worker detects a new task for the topic assigned to `topic_name` it will trigger the# function/method passed to `action`.awaitworker.subscribe(topic_names="HelloWorldTask",action=process)# this will be called when a task for the subscribed topic is availableasyncdefprocess(task:ExternalTask)->ExternalTaskResult:print("I got a task!")# To communicate the successfull processing of a task, we return an ExternalTaskResult created by `task.complete` .# If we call `task.failure` instead, Camunda will publish the task again until# some client finally completes it or the maximum amount of retries is reached.returntask.complete()# run the main taskasyncio.run(main())You can run that example from the project folder with:pythonexamples/hello_world.pyYou should see something like this in your terminal:python ./examples/hello_world.py
waiting for a task ...
I got a task!If you don't see the second line you probably need to start the Camunda process (in the modeler).Working with dataThe example above is quite trivial since our client just returns a success event/result but nothing actually happens.
In the next example, we will let the python worker decide whether a passed number is odd or even.
First, let's have a look at the BPMNbpmn_process/odd_number.bpmn:We have to externalServiceTaskslabeledNumber is even!andNumber is odd!.
Both use the same topicEchoTaskwhich we will use to print text to the terminal.
Have a look in the property panel and click theInput/Outputtab of one of the tasks.
You should see something like this:The task defines one input parameter namedtextof typeTextwith a value ofNumber is even!(or odd).
Next, let's have a look atNumber Check:<bpmn:serviceTaskid="Activity_NumberCheck"name="Number Check"camunda:type="external"camunda:topic="NumberCheckTask"><bpmn:extensionElements><camunda:inputOutput><camunda:inputParametername="number">42</camunda:inputParameter><camunda:outputParametername="isOdd">${result}</camunda:outputParameter></camunda:inputOutput></bpmn:extensionElements><bpmn:incoming>Flow_Start</bpmn:incoming><bpmn:outgoing>Flow_OutCome</bpmn:outgoing></bpmn:serviceTask>Now you might say 'Wait! Where does this come from?'.
You can use the Modeler GUI to edit values but you can also edit the BPMN-XML directly.
In the lower left you will see two tabs labelledDiagramandXML. This is where you change the view.We see that the topiccamunda:topichas been set toNumberCheckTaskand that the service task contains two parameters in thecamunda:inputOutputblock.
We assign42tonumberand assign the received out parameterresultto the environment variableisOdd.
Of course, you can change the value ofnumberas you like.
You also don't need to do it in the XML view.
Just change back toDiagramand look for theInput/Outputtab in the property panel ofNumber Check.Last, we have a look at the Gateway (the diamond-shaped box with the big X).
The gateway itself has just a label.
This does nothing but can be used to clarify what the gateway will evaluate.
We will evaluate our previously assignedisOddvariable.
To do so, we need to set conditions on the flows leaving the gateway.
If you click on the flow to the bottom (labeled with"true"), the property panel will look like this:Thenameparameter is again just something that makes your model easier to comprehend.
TheCondition TypeandExpressionwill determine if a Flow is considered valid.
We choseExpressionsince we want to check the value ofisOdd.
In Camunda, this is done as seen in the picture above.
WhenisOdd == "true"the service taskNumber is odd!will be executed and the process will end after this.
The flow above the gateway has a line crossing the flow which means that this is the default flow when no other condition is met.
So, what we do is we check whetherisOddis equal to"true", executeNumber is odd!if that is the case orNumber is even!otherwise.Now let's see howexamples/odd_number.pylooks like:importaiohttpimportasynciofromcamunda.external_task.external_taskimportExternalTask,Variablesfromcamunda.external_task.external_task_workerimportExternalTaskWorkerfromcamunda.external_task.external_task_resultimportExternalTaskResultasyncdefmain():asyncwithaiohttp.ClientSession()assession:# a new parameter! We can pass 'asyncResponseTimeout' to Camunda to configure how long a 'fetch and lock' connection# is kept alive. If this value is rather large, clients do not need to reconnect that often but it will take longer for a# worker to properly shut down. The default value is 30 seconds (300000)worker=ExternalTaskWorker(worker_id=4,base_url="http://localhost:8080/engine-rest",session=session,config={"asyncResponseTimeout":5000},# wait 5 seconds before timeout)# Our worker will now subscribe to two topics now# We will create a new task with `asyncio.create_task` and await only the second subscribeasyncio.create_task(worker.subscribe(topic_names="NumberCheckTask",action=number_check))awaitworker.subscribe(topic_names="EchoTask",action=echo)asyncdefnumber_check(task:ExternalTask)->ExternalTaskResult:try:number=task.context_variables["number"]print(f"We received{number}for checking...")# We set a locally scoped variable 'result' to 'true' or 'false'task.local_variables.set_variable("result","true"ifint(number)%2!=0else"false",Variables.ValueType.STRING)# We pass the variables object as LOCAL variables which will only be available in the context of the task# that called the external task worker. The result must be assigned in case it should be used somewhere else.# Just have a look at the odd_number.bpmn to see how.returntask.complete()# If your input could not be parsed with `int()` the task will fail# and another external service could try to do better.exceptExceptionaserr:print(f"Oh no! Something went wrong:{err}")returntask.failure(error_message=err.__class__.__name__,error_details=str(err),max_retries=3,retry_timeout=5000,)asyncdefecho(task:ExternalTask)->ExternalTaskResult:print(f"Camunda wants to say:{task.context_variables['text']}")returntask.complete()# run the main taskasyncio.run(main())We now have two callbacks and our worker will subscribe them the appropriate topic as seen in the BPMN model.
We also reduce theasyncResponseTimeoutof the worker to prevent already closed python clients to accidently fetch tasks.
This is just a workaround for the sake of simplicity.
It is recommended to actually wait for worker subscriptions to return (by awaitingworker.cancel()) before shutting down the whole process.Innumber_checkand inecho, we see how to retrieve variables from theExternalTaskobjecttask.
Note that retrieving variables withtask.context_variables["key"]will raise aKeyErrorifkeydoes not exists.
To deal with optional variables you can usetask.context_variables.get_variable("key")which will returnNoneifkeycannot be found.
Innumber_check, there is also shown how aVariablesobject is created, a value is assigned and how this object is passed as alocalvariables object.
Local variables can only be used in the scope of the service task.
This is why we have to assignresultto an output parameter in Camunda.
We could also passvariablestoglobal_variablesinstead.
This way,resultcould be used in the whole process.
However, in more complex scenarios this might clutter the environment and may even lead to colliding variable definitions.
Keeping things local increases the control by keeping variable scopes narrow and alsovisiblein the BPMN model.When you run the example, you should see an output like this after you have started the Camunda process:python ./examples/odd_number.py
We received 42 for checking...
Camunda wants to say: Number is even!Managing multiple workers/subscriptionsThe above mentions some issues when clients are stopped and restarted.
Camunda connections might not have been properly released the next time a task is scheduled by Camunda.
This may cause the already stopped/inactive instance to lock the task.
One way to deal with this is to catch a KeyboardInterrupt or any other kind of event that signalizes a shutdown and awaitworker.cancel()as shown inexamples/manage_tasks.py:importaiohttpimportasynciofromcamunda.external_task.external_taskimportExternalTask,Variablesfromcamunda.external_task.external_task_workerimportExternalTaskWorkerfromcamunda.external_task.external_task_resultimportExternalTaskResultclassWorker:def__init__(self):self.worker=Noneself.loop=Nonedefstart(self):"""Run the worker and block forever"""self.loop=asyncio.get_event_loop()self.loop.run_until_complete(self._run())asyncdef_run(self):asyncwithaiohttp.ClientSession()assession:self.worker=ExternalTaskWorker(worker_id=4,base_url="http://localhost:8080/engine-rest",session=session)# dispatch the first subscriptionself.loop.create_task(self.worker.subscribe(topic_names="NumberCheckTask",action=number_check))# and block the current task with the second subscription againawaitself.worker.subscribe(topic_names="EchoTask",action=echo)defstop(self):self.loop.run_until_complete(self.worker.cancel())asyncdefnumber_check(task:ExternalTask)->ExternalTaskResult:try:number=task.context_variables["number"]print(f"We received{number}for checking...")task.local_variables.set_variable("result","true"ifint(number)%2!=0else"false",Variables.ValueType.STRING)returntask.complete()exceptExceptionaserr:print(f"Oh no! Something went wrong:{err}")returntask.failure()asyncdefecho(task:ExternalTask)->ExternalTaskResult:print(f"Camunda wants to say:{task.context_variables['text']}")returntask.complete()# run the main tasktry:worker=Worker()worker.start()exceptKeyboardInterrupt:# Stopping workers might take a while.# How long it will take depends on the chosen asyncResponseTimeout (default is 30000 ms)print(f"Stopping workers...")worker.stop()print(f"All done!")The code above basically does the same as theodd_numberexample before but we wrapped the asynchronous bits into aWorkerclass and added methods to start and stop workers and their subscriptions. Depending on how long you are willing to wait for a shutdown you might want to adjustasyncResponseTimeout.