Achieving Rule-based observability using Sidekick and Camunda
Tags: opensource, productivity, devops, java
Published: 2022-09-12 13:15:29

## Collect what you need, only when you need it

Sidekick is an [**open source**](https://github.com/runsidekick/sidekick) live application debugger that comes with many built-in features to make data collection both easier and more efficient. Collecting snapshots and generating logs from running applications are powerful for learning how your code works as long as you collect the data you need.

Logging everything is not a smart move as you can find yourself searching in a log pile full of mostly unnecessary logs in a very short time. Similar case applies for the snapshots & traces. That is why we have included conditional data generation in Sidekick. Combined with hit and time limits, they allow you to collect only what you need, leaving a graveyard of useless data behind.

In addition to our out-of-the-box conditional limitation features, Sidekick’s API’s and clients let you to bring your own rules which leads to a relatively new approach to observability, a rule based one. With this approach we will be able to put tracepoints when rules met and start collecting data automatically based on rule engine’s output.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vjqvqneoyjk8cz33chdw.gif)



## Rule Engines

There are plenty of different rule engines & workflow automation tools that we can use for such a scenario. A few can be listed as:

{% embed https://camunda.com/ %}

{% embed https://www.drools.org/ %}

{% embed https://www.npmjs.com/package/json-rules-engine %}

{% embed https://www.npmjs.com/package/node-rules %}

We will be using Camunda for this example for broader use cases as Camunda is more than a rule engine it is a universal process orchestrator. For a usage example of Camunda you can check out:

[Use Camunda as an easy-to-use REST-based orchestration and workflow engine (without touching Java)](https://blog.bernd-ruecker.com/use-camunda-without-touching-java-and-get-an-easy-to-use-rest-based-orchestration-and-workflow-7bdf25ac198e)


## Using Sidekick with Camunda

Camunda provides external task clients that allow developers to implement tasks to run when a workflow hits to the related step. We will be using Node.js client but cases below can also be achieved with different clients of Camunda and Sidekick’s upcoming Python & Java clients.

{% embed https://www.npmjs.com/package/camunda-external-task-client-js %}

Example task method:

```

// Subscribe to the topic: 'topicName'  
client.subscribe("topicName", async function({ task, taskService }) {  
  // Put your business logic  
  // Complete the task  
  await taskService.complete(task);  
});

```

{% embed https://www.npmjs.com/package/@runsidekick/sidekick-client %}


Task method with Sidekick Put tracepoint event:
```
// Subscribe to the topic: 'topicName'  
client.subscribe("topicName", async function({ task, taskService }) {    const params= {  
      applicationFilters: [  
          {  
            name: "Demo application",  
            version: "v1.0",  
            stage: "prod"  
          }  
        ],  
      fileName: "gitlab.com/repos/...",  
      lineNo: 23,  
      expireSecs: -1,  
      expireCount: -1,  
      enableTracing: true,  
      persist: true  
  }  
  apiClient.putTracepoints(params);  
  await taskService.complete(task);  
});
```
# Example use cases

## Case 1 — Activating Tracepoints based on error scenarios

Camunda’s DMN engine provides us ability to define rules in an excel like interface.

![](https://miro.medium.com/max/1400/1*NiZSUANcysBNOJwAzeNO6A.png)

Sidekick DMN

As you can observe at the DMN above we can define where we put tracepoints when a certain type of error occurs.

Triggering workflows with error types makes a great use case as it enables us to automatically start collecting extra information based on different cases. Enabling different tracepoints at once is especially useful for spotting state changes.

> Note: In addition to this use case Sidekick Node.js agent has ability to automatically capture error stacks, without any tracepoint. check out: [https://www.npmjs.com/package/@runsidekick/sidekick-agent-nodejs](https://www.npmjs.com/package/@runsidekick/sidekick-agent-nodejs)

![](https://miro.medium.com/max/1400/1*PtgBVnGlhUHaG54ioHJrYg.png)

Example Sidekick BPMN

## Case 2 —Putting Tracepoints on workflow errors

![](https://miro.medium.com/max/1400/1*4GYMm9hF1EzWD4R2MPzIsw.png)

[image from: https://blog.bernd-ruecker.com/use-camunda-without-touching-java-and-get-an-easy-to-use-rest-based-orchestration-and-workflow-7bdf25ac198e](https://blog.bernd-ruecker.com/use-camunda-without-touching-java-and-get-an-easy-to-use-rest-based-orchestration-and-workflow-7bdf25ac198e)

Camunda comes with an error handling mechanism that lets you run actions in cases of errors. In addition to Sidekick client, we can also attach Sidekick agents to task services to observe their state changes and debug them effectively.

## Case 3 — Activating Tracepoints on certain events

![](https://miro.medium.com/max/1400/1*RsCLAeQS9H4dOBF0J1MPPw.png)

Last but not least, Sidekick Actions bring more than debugging to the table. They are also great tools to understand how the system works. Especially for the cases like understanding why and how edge cases occur or understanding when does a certain step is reached.

We have briefly explained how Sidekick can be used with Camunda to achieve rule based observability, for better understanding systems, having a smoother debug experience and lowering monitoring costs with conditional data collection. The 3 cases we have explained are just the tip of the iceberg and there is a lot more to discover here.

Please let us know your invaluable ideas because they will guide us on our path to discovering the future of application observability. You can get in touch with us through [**Twitter**](https://twitter.com/RunSidekick), and join our [**Discord community**](https://www.runsidekick.com/discord-invitation).