# Predictive Business Process Monitoring Approach Based on Hierarchical Transformer
## 2 Literature Review  
Predictive Process Monitoring is different from traditional process mining tasks such as process discovery and consistency testing. It focuses not only on the existing execution of the business process but also on the future execution trends of the business process, contributing to the early determination of potential risks and bottlenecks in the business process and providing a basis for immediate optimization of the business process. Predictive business monitoring research can be broadly divided into formal modeling-based and data-driven approaches. The data-driven approach can better utilize the complex business process execution information contained in the massive event logs, overcoming the lack of flexibility of various formal approaches and gradually becoming one of the main streams of predictive business monitoring research. In recent years, deep learning techniques have shown increasing success in various real-world tasks such as natural language processing, recommender systems, and computer vision. The predictive business monitoring tasks are closer to the common applications of deep learning, so many researchers have started conducting relevant research using deep learning techniques and achieve better results. The earliest research on predictive business monitoring based on deep learning focuses on applying RNNs. [PERSON] et al. [PERSON] et al. (2018) used RNNs to construct a basic framework for business process prediction, consisting of a two-layer unidirectional LSTM. [PERSON] et al. [PERSON] et al. (2018) used the idea of parameter sharing to design a multi-task business process prediction method based on a multi-layer LSTM. [PERSON] et al. [PERSON] et al. (2018) used another simplified variant of RNN, GRU, to design a business process classification method. To further improve the accuracy of RNN-based predictive business monitoring, researchers have proposed a variety of proven methods. For example, [PERSON] et al. [PERSON] et al. (2019) conducted feature engineering for subtypes, continuous-type activity attributes, and time series, respectively, and used the extracted features as input to the LSTM; [PERSON] et al. [PERSON] et al. (2018) introduced attention, multi-layer stacking, and bidirectional connection and other mechanisms to effectively improve the accuracy of remaining time prediction; [PERSON] et al. [PERSON] et al. (2019) designed a transfer learning based model training framework targeting the length characteristics of business process instances. In addition to RNNs, researchers use other types of deep learning models for predictive business monitoring. For example, [PERSON] et al. [PERSON] et al. (2019) designed a method for predicting the next activity in a business process using convolutional neural networks with the help of modeling ideas from the image domain, so it can capture the spatial structure of the business process logs and enable parallelization training; [PERSON] et al. [PERSON] et al. (2019) proposed a multi-stage deep learning approach, which extracts features using feature hashing and deep stacked autoencoders and then uses multi-layer feed-forward neural networks to construct a next activity prediction model; [PERSON] et al. [PERSON] et al. (2019) used deep auto-encoders to extract event sequence features and variational system structure features to achieve a more comprehensive feature representation, followed by residual time prediction using a multi-layer perceptron. There are relatively few studies on predictive business monitoring based on the Transformer. [PERSON] et al. [PERSON] et al. (2019) used the self-attention mechanism in the Transformer to construct a business process next activity prediction model and achieved better prediction results than the traditional LSTM. [PERSON] et al. [PERSON] et al.