Exploring the intersection of BPMN with Process Mining: Insights from integrating bpmn-visualization with pm4py
Tags: processmining, analytics, visualization, bpmn
Published: 2023-05-30 11:15:00

## Introduction

You know what they say: “_you can't improve what you can't measure,_” and that's where [process mining](https://link.springer.com/book/10.1007/978-3-662-49851-4) has a role. Process mining is a powerful set of techniques for analyzing and optimizing business processes based on _data_. Similar to machine learning and data analytics, process mining uses data-driven methodologies to extract insights from _process execution data_, also known as _event data_. 

One of the key requirements for effective process mining is the ability to **visualize** and **interact** with the results of analysis on process models. This is especially crucial as process mining findings are often used by business experts who may not have a technical background. However, many of the available industrial process mining tools have limited support for BPMN-based visualization, despite [BPMN](https://www.bpmn.org/) being a widely-used standard for process modeling and execution. As a result, there is a clear need for tools and libraries that can provide more comprehensive BPMN-based visualization capabilities.

> There is an evident requirement for process mining tools and libraries that can offer enhanced visualization capabilities based on BPMN (Business Process Model and Notation).

To fill this need, [Process Analytics](https://process-analytics.dev/?utm_source=dev.to&utm_medium=display&utm_campaign=blog) - an open-source initiative under Apache 2.0 license led by [Bonitasoft](https://www.bonitasoft.com/) - is committed to developing developer-centric BPMN-based visualization components that can be seamlessly integrated into process mining projects. A pivotal component of this initiative is the [bpmn-visualization](https://process-analytics.github.io/bpmn-visualization-js/) library which empowers developers to visualize the results of process mining techniques on BPMN process models. With support for both [TypeScript](https://github.com/process-analytics/bpmn-visualization-js/) and [R](https://github.com/process-analytics/bpmn-visualization-R/), _bpmn-visualization_ provides developers with a straightforward means to render, style, and interact with BPMN elements.

Note that the popular Python process mining library _[pm4py](https://pm4py.fit.fraunhofer.de/)_ offers a range of methods for process discovery, conformance checking, and performance analysis. By integrating _bpmn-visualization_ with _pm4py_, we aim to take a step towards enabling BPMN-based process mining and visualization. The source code and documentation of this integration is available on [GitHub](https://github.com/process-analytics/bpmn-visualization-pm4py), which offers an open opportunity for testing, exploration and contribution.

In this article, we will describe our experience in integrating these two libraries, and discuss the challenges we encountered due to the limited support for BPMN-based process mining. Our goal is to provide high-level insights and takeaways for the process mining community, including researchers, process mining experts, editors, and developers. Furthermore, we aim to engage anyone interested in the fusion of BPMN with process mining, including individuals from the BPM domain or other related fields. By doing so, we hope to raise awareness about the importance of native BPMN-based process mining and its potential impact across different domains. 

---

## Integration


### Architecture

The integration involves the development of two main components - the **frontend** and the **backend**. 

![Figure 1. High level architecture of the integration between bpmn-visualization and pm4py](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5oxyml89mlksva6hkol0.png)

The frontend component is written in JavaScript and leverages the capabilities of _bpmn-visualization_ to visualize BPMN process models and the result of the backend analysis. 

The backend component is built using _pm4py_ and is responsible for processing data and performing process mining analysis. More details about the architecture, the technologies used, and the application setup are available on [GitHub](https://github.com/process-analytics/bpmn-visualization-pm4py).


### Use cases

Three process mining techniques are used in this integration example: process discovery, conformance checking, and enhancement.

**Process discovery**: The [inductive miner algorithm](https://www.researchgate.net/publication/242348456_Discovering_Block-Structured_Process_Models_from_Event_Logs_-_A_Constructive_Approach) is used to automatically create a process model from event data stored in an event log using the [XES storage standard](https://xes-standard.org/).

**Conformance checking**: The event log is compared with the BPMN process model to detect deviations or non-conformities. 

**Enhancement**: The BPMN process model is enriched with statistics data computed from the event log, such as the frequency of execution of BPMN elements and the conformance data obtained from the conformance checking technique.

Figure 2 below shows an excerpt of a BPMN process model discovered from a [simple synthetic event log](https://processmining.org/old-version/event-book.html) and enriched with frequency data using our developed integration example.



![Figure 2. Excerpt of a BPMN process model discovered from an event log and enriched with frequency data](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/roxr53mn8cg6f9wld46v.png)

### Approach 

BPMN-based process mining is an emerging topic that is still under development in the research world, and not fully exported to the industrial world. This is because most process mining techniques originate from research efforts which are based on [Petri net](https://en.wikipedia.org/wiki/Petri_net), a formalism for modeling concurrent systems. This is true of pm4py, as most of its implemented algorithms are based on Petri nets. 

> BPMN-based process mining is a developing research area that has not yet fully transitioned to industry. The integration process involves converting between BPMN and Petri nets, enabling the utilization of existing process mining algorithms designed for Petri nets and visualizing the results on BPMN diagrams.

Therefore, to develop the integration, we followed the approach described in [enabling BPMN-based process mining](http://www.padsweb.rwth-aachen.de/wvdaalst/publications/p901.pdf). Generally, this approach consists of converting between BPMN and Petri nets. This means that existing process mining algorithms developed for Petri nets can be used and the obtained results are then processed to be visualized on BPMN.

To give a sense of this approach, Figure 3 below illustrates the workflow of generating the visualization shown in Figure 2 above (the discovered BPMN enriched with frequency information). 

![Figure 3. Generating a visualization of the discovered process](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7ganayls51ivbldxny26.png)

The first step consists of importing an event log. In the backend, **Process Discovery** and **Enhancement** components are required to discover and compute frequency information, respectively. 

The **Process Discovery** component first creates a Petri net from the imported event log, which is then converted to BPMN. In our integration, this conversion is performed by the BPMN variant of the inductive miner using the [discover_bpmn_inductive](https://pm4py.fit.fraunhofer.de/static/assets/api/2.7.4/generated/pm4py.discovery.discover_bpmn_inductive.html) method. It's important to note that newer process discovery algorithms that can directly create BPMN process models are emerging (e.g. see the [Model Generation Application](https://process-analytics.dev/model-generation-application/) developed as part of the Process Analytics project or the Split miner used by [Apromore](https://apromore.com/research-lab/)). 

Next, a layout generation algorithm is employed to determine the positions (i.e., x and y coordinates) of elements on the BPMN diagram. In our example, we used the [bpmn-layout-generators](https://github.com/process-analytics/bpmn-layout-generators) library developed as part of the Process Analytics project. Finally, this component outputs a serialized BPMN xml file that is sent to _bpmn-visualization_ for rendering.

The **Enhancement component** enables the enrichment of the BPMN model with statistical data, such as frequency and performance information. To achieve this, a mapping between the event log and the process should be first established using [replay algorithms](https://www.researchgate.net/publication/229124749_Replaying_History_on_Process_Models_for_Conformance_Checking_and_Performance_Analysis). Once the mapping is established, the statistical data, such as frequency and performance data, can be computed for Petri net elements. The computed statistics then need to be mapped back to the corresponding BPMN elements. Finally, _bpmn-visualization_ is utilized to visualize the results on top of the BPMN model.

---

## Challenges & limitations


### No native support for BPMN-based process mining

One of the main challenges we encountered is the lack of native support for BPMN-based process mining. While most process mining techniques rely on Petri nets, these can be challenging for non-technical business users to comprehend and utilize effectively. Existing industry-based solutions are primarily built on non-formal representations of [process maps](https://fluxicon.com/book/read/mapview/). Although process maps offer initial insights, they have limitations in representing the full process behavior and cannot support thorough and detailed analysis. In contrast, BPMN already has widespread acceptance within the business community, making it a more suitable and comprehensive approach for process mining.

> Converting BPMN to Petri nets for BPMN-based process mining has limitations such as information loss, performance issues, and maintenance overhead.

The approach of converting BPMN to Petri nets is not a perfect solution for BPMN-based process mining due to several limitations:

**Loss of information**: The conversion process from BPMN to Petri nets may result in a loss of information, which can affect the accuracy of the analysis. In the BPMN-based process mining approach, a subset of BPMN elements is used to ensure a conversion with no loss.

**Performance issues**: The conversion process can be computationally intensive, which can lead to performance issues, especially when dealing with large process models. 

**Maintenance overhead**: The use of Petri nets requires additional maintenance overhead, as the models need to be updated whenever changes occur in the business process. 

Furthermore, the **enhancement** process mining technique, which involves establishing a mapping between the process model and the event log and computing various statistics, relies on replay algorithms that have been developed for Petri nets. Unfortunately, to the best of our knowledge, there is currently no known open-source implementation available for BPMN. Since event logs cannot be directly replayed against BPMN models, they must first be converted to Petri nets before the replay can be performed.

Despite these limitations, the conversion approach is currently the most viable solution for BPMN-based process mining, given the lack of native support for BPMN in the process mining community.


### BPMN layout generation

Another challenge in BPMN-based process mining is generating a layout for the discovered process model. While process discovery algorithms can automatically extract a process model from event logs, they do not generate a visual layout that represents the model in a way that is easily understandable by humans. Process mining has primarily relied on existing hierarchical [graph layout algorithms](https://www.graphviz.org/pdf/dotguide.pdf) that do not take into account specific process properties, such as centralizing the most frequent path or the most performant path. 

Recent efforts have emerged to address these issues in process maps, such as the [Tracy algorithm](https://ceur-ws.org/Vol-2374/paper5.pdf) by UIPath. Notably, there are currently no proposed layout algorithms specifically tailored for BPMN. In our [bpmn-layout-generators](https://github.com/process-analytics/bpmn-layout-generators) library, we currently focus on experimenting and generating more comprehensible BPMN layouts. In the future, we will continue exploring opportunities to enhance the layouting of BPMN models, including considerations for process properties.  

---


## Conclusion & takeaways

Despite the current lack of native support for BPMN-based process mining, integrating BPMN with process mining techniques offers valuable insights that outweigh the discussed limitations.

BPMN, with its higher level of detail and formality, surpasses the limitations of process maps, which are commonly used by existing process mining tools for result visualization. At Process Analytics, we are dedicated to developing BPMN-based visualization libraries like _bpmn-visualization_ and _bpmn-layout-generators_. 

> We envision a future, as [hinted by Prof. Wil van der Aalst](https://medium.com/@process-analytics/interview-with-professor-wil-van-der-aalst-at-icpm-2022-81dc4d23df4c), where there will be a seamless transition from process maps to BPMN. 

As this transition unfolds, we anticipate a growing demand for BPMN-based process mining tools capable of harnessing the full potential of this formalism, providing more accurate and comprehensive insights.

If you want to stay on top of the latest news and releases from the Process Analytics project, follow us through:



* Website: [https://process-analytics.dev](https://process-analytics.dev/?utm_source=dev.to&utm_medium=display&utm_campaign=blog) 
* Twitter: [@ProcessAnalyti1](https://twitter.com/ProcessAnalyti1)
* GitHub: [https://github.com/process-analytics](https://github.com/process-analytics)
* Discord: [https://discord.com/invite/HafnBYsRXd](https://discord.com/invite/HafnBYsRXd)
* YouTube: [Process Analytics YouTube channel](https://www.youtube.com/@process-analytics)


## Useful resources

* [Official process mining website](https://www.processmining.org/) 
* [Getting started tutorial to _bpmn-visualization_](https://medium.com/@process-analytics/getting-started-with-bpmn-visualization-a54fbafae2df) 
* [Limitations of process maps from a practitioner’s perspective by Wil van der Aalst](https://www.sciencedirect.com/science/article/pii/S1877050919322367?via%3Dihub) 
* More articles about BPMN process discovery and layout generation by Process Analytics project can be found on: (https://process-analytics.dev/model-generation-application/)