# Predictive Business Process Monitoring Approach Based on Hierarchical Transformer
## 3 Preliminaries and Methods
### Concept Drift Detection  
The first layer Transformer model obtains the encoding vector of each event that provides a basis for concept drift detection. Therefore, in contrast to the direct application of existing concept drift detection methods, this paper proposes a concept drift detection method based on the encoding representation. The basic idea of this method is splitting the time series of business process instances to obtain different subsequences. Then, we conduct significance tests on the data distribution of each subsequence to determine whether concept drift has occurred between the subsequences. Through this method, many candidate concept drift points may be obtained and may thus apply the Optics algorithm to cluster on the candidate concept drift points. The most representative point in the cluster of the candidate concept drift points will be retained as the segmentation point, and the sequence segmentation is completed. In the candidate concept drift point search stage, supposing the encoding representations of two subsequences are \(\mathbf{W}_{1}=\left(\mathrm{e}_{1,1},\mathrm{e}_{1,2},\ldots,\mathrm{e}_{1,\mu_{1 }}\right)\) and \(\mathbf{W}_{2}=\left(\mathrm{e}_{2,1},\mathrm{e}_{2,2},\ldots,\mathrm{e}_{2,\mu_{ 2}}\right)\). Let \(\mu_{1}\) and \(\mu_{2}\) be the mean of the encoding vectors of the subsequence \(\mathbf{W}_{1}\) and \(\mathbf{W}_{2}\), and let \(\lambda_{1}\) and \(\lambda_{2}\) be the trace of the covariance matrix of the subsequence \(\mathbf{W}_{1}\) and \(\mathbf{W}_{2}\). From the perspective of the differences test, the conceptual drift detection task is to test whether the subsequence \(\mathbf{W}_{2}\) has changed significantly in distribution compared to the subsequence \(\mathbf{W}_{1}\). The following statistic testing is first constructed to quantitatively assess the difference between the two subsequences. \[s=\frac{\left|\mu_{2}-\mu_{1}\right|}{\sqrt{\frac{\lambda_{1}}{n_{1}}+\frac{ \lambda_{2}}{n_{2}}}} \tag{14}\]  
After that, let the original hypothesis be that the difference between the two subsequences is not significant, i.e., no conceptual drift has occurred. Given a confidence threshold \(\delta\), the acceptance region of the original hypothesis can be expressed formally as follows:  
\[\Pr\left(\frac{\left|\mu_{2}-\mu_{1}\right|}{\sqrt{\frac{\lambda_{1}}{n_{1}}+ \frac{\lambda_{2}}{n_{2}}}}\geq\varepsilon\right)\leq\delta \tag{15}\]  
Equation (15) can be equivalently written as:  
\[\Pr\left(\left|\mu_{2}-\mu_{1}\right|\geq\varepsilon\sqrt{\frac{\lambda_{1}}{ n_{1}}+\frac{\lambda_{2}}{n_{2}}}\right)\leq\delta \tag{16}\]  
According to [PERSON]'s Inequality [28] and its corollaries [29], that the following holds:  
\[\Pr\left(\left|\mu_{2}-\mu_{1}\right|-\left|E(\mu_{2})-E(\mu_{1})\right|\geq \varepsilon\right)\leq\exp\left(\frac{-2\varepsilon^{2}}{\left(\frac{1}{n_{1}} +\frac{1}{n_{2}}\right)\left|\mathbf{b}-\mathbf{a}\right|^{2}}\right) \tag{17}\]  
where \(E(\mu_{2})\) and \(E(\mu_{1})\) denote the expectation of the subsequence encoding representation, respectively. \(\mathbf{b}\) and \(\mathbf{a}\) denote the upper and lower deterministic bounds of the event encoding vector, respectively. These values are usually not directly computable. However, considering that the event encoding is learned through the previously described event-level Transformer by random initialization, it can be assumed that: \(E(\mu_{2})=E(\mu_{1})=0\). \(\left|\mathbf{b}-\mathbf{a}\right|^{2}\) can be written as a fixed constant \(\Delta^{2}\), so we can obtain Equation (18). Transformer for Subsequence  
The second layer Transformer model takes the event sub-sequence obtained after concept drift detection as input, which can be formally represented as \(\mathbf{M}=\langle\mathbf{e}_{s},\mathbf{e}_{s+1},\ldots,\mathbf{e}_{s+l-1}\rangle \in\mathbb{R}^{l\times d}\) after encoding by the event-level Transformer model, where \(l\) denotes the length of the sub-sequence, and \(d\) denotes the event encoding vector dimension.