# Predictive Business Process Monitoring Approach Based on Hierarchical Transformer
## 3 Preliminaries and Methods
### Hierarchical ProcessTransformer
#### 3.4.1 First Layer: Event-Level Transformer  
Events are the fundamental building blocks of business process cases, and valid event coding is one of the keys to the task of predictive business process monitoring. According to the definition of an event in Section 3.1, an event is a composition of multiple attributes, so the encoding of events requires effective integration of event-related attribute information. To make the HiP-Transformer model proposed in this paper more adaptable, all attributes are divided into discrete and continuous attributes. Different encoding methods are designed for these two types of attributes. In general, assuming that an event contains a discrete and a continuous attribute, i.e., \(e=(a,t,m_{1},m_{2})\in A\times T\times M_{1}\times M_{2}\), where \(m_{1}\) and \(m_{2}\) denote discrete and continuous attributes, respectively. The subsequent way of encoding the attributes can be easily  
Figure 1: HiP-Transformer Model Framework. extended to zero or more attributes. The discrete attribute \(m_{1}\) is first represented as a one-hot coding vector \(\widetilde{\mathbf{m}}_{1}\in\{0,1\}^{|M_{1}|}\) according to its value domain \(M_{1}\). A learnable coding matrix is constructed for the attribute \(\mathbf{M}_{1}\in\mathbb{R}^{|M_{1}|\times d}\), and the encoding vector for this attribute can be obtained as \(\mathbf{m}_{1}=\widetilde{\mathbf{m}}_{1}\mathbf{M}_{1}\). For the continuous attribute \(m_{2}\), attribute learnable encoding vector \(\widetilde{\mathbf{m}}_{type(m_{2})}\) is constructed, and then the specific values of the attribute are normalized, which in turn obtains its encoding vector as \(\mathbf{m}_{2}=m_{2}\odot\widetilde{\mathbf{m}}_{type(m_{2})}\), where \(\odot\) denotes the operation of the product of a scalar and a vector. To easily understand, further examples of encoding discrete and continuous attributes are given. Suppose that the discrete attribute of an event is \"activity executor is [PERSON]\" and the continuous attribute is \"execution cost is 0.5 (after normalization)\", i.e., \(m_{1}=^{\prime}\) Bob\({}^{\prime}\), \(m_{2}=0.5\). The activity executor takes the value space as \(M_{1}=\{\prime\text{[PERSON]}t\), \(\prime\text{[PERSON]}\), \(\prime\text{[PERSON]}\prime\}\). Let the encoding dimension of each attribute be 4. The encoding process for this discrete attribute is as follows. \[\widetilde{\mathbf{m}}_{1}=(1,0,0)\]  
\[\mathbf{M}_{1}= \begin{pmatrix}e_{1,1},e_{1,2},e_{1,3},e_{1,4}\\ e_{2,1},e_{2,2},e_{2,3},e_{2,4}\\ e_{3,1},e_{3,2},e_{3,3},e_{3,4}\end{pmatrix} \]  
\[\mathbf{m}_{1}=\widetilde{\mathbf{m}}_{1}\mathbf{M}_{1}=(e_{1,1},e_{1,2},e_{1,3},e_{1,4})\]  
The encoding process for this continuous attribute is as follows. \[\widetilde{\mathbf{m}}_{cost}=(h_{1},h_{2},h_{3},h_{4})\]  
\[\mathbf{m}_{2}=m_{2}\odot\widetilde{\mathbf{m}}_{cost}=(0.5\times h_{1},0.5\times h_{2 },0.5\times h_{3},0.5\times h_{4})\]  
The activity \(a\) performed in the event is, in essence, a discrete attribute of the event, and the activity vector \(\mathbf{a}\) can be obtained by the encoding method of the discrete attribute described above. The encoding vector \(\mathbf{e}\) of the entire event \(e\) is obtained in the event-level Transformer using the self-attention mechanism. Considering that the activity is the main part of an event, the activity vector is first used as the query vector in the self-attention mechanism to calculate the weight vector of each attribute concerning the activity, and the vectors of the attributes are fused in a weighted manner. Let \(\mathbf{X}=(\mathbf{m}_{1};\mathbf{m}_{2})\), the process is calculated as follows:  
\[\mathbf{m}=softmax\bigg{(}\frac{\mathbf{q}\mathbf{k}^{T}}{\sqrt{d^{(K)}}}\bigg{)}\mathbf{V} \tag{12}\]  
\[\mathbf{q}=\mathbf{a}\mathbf{W}^{(Q)},\ \mathbf{K}=\mathbf{X}\mathbf{W}^{(K)},\ \mathbf{V}=\mathbf{X}\mathbf{W}^{(V)} \tag{13}\]  
where \(\mathbf{W}^{(Q)}\), \(\mathbf{W}^{(K)}\), and \(\mathbf{W}^{(V)}\) are the linear transition matrices shown in Equation (6).