# POP-ON: Prediction of Process Using One-Way Language Model Based on NLP Approach
## 3 Pop-On
### Data Pre-Processing  
In our research, concrete data pre-processing was performed to improve the process prediction precision. As a result, the inclusion of our data pre-processing approach in the POP-ON model improved the prediction precision. Figure 3 visually shows the data pre-processing process for the event log datasets. In Figure 3, the sub sequence, label, and attribute are extracted from each case in the left event log. At the top right, special tokens are inserted and tokenized for each subsequence, and at the bottom right, one-hot encoding and min-max scaling are performed for each attribute type.  
Tokenization is the breakdown of the flow of strings into meaningful tokens that are distinguished from each other [29]. The tokenization of sentences is essential for processing natural language. In this study, a series of events belonging to each case of the dataset was reported as one sentence and tokenized for each event. Tokenized events of case \(C_{i}\) are called an event sequence \(E_{i}\). For case \(C_{i}\) and event sequence \(E_{i}=\big{(}e_{i1},\ e_{i2},\ \ldots,\ e_{il_{i}}\big{)}\) with length \(l_{i}\geq 2\), the subsequence of \(C_{i}\) is shown in Equation (1):  
\[E_{i_{sub}}=\big{(}e_{i1},\ e_{i2}\big{)},\ (e_{i1},\ e_{i2},\ e_{i3}\big{)},\ \ldots,\ \big{(}e_{i1},\ e_{i2},\ \ldots,\ e_{il_{i}}\big{)} \tag{1}\]  
The purpose of this study is to predict the next event in the process. Therefore, if \(l_{i}=1\) or the length of the subsequence is 1, since in this case, it is a subsequence that cannot be used for both training and verification, it was excluded from the set of all subsequences. Therefore, for every case \(C_{i}\) of length \(l_{i}\geq 2\), there are \(l_{i}-1\) subsequences each. The target token corresponding to each subsequence is the last token of the subsequence as shown in Equation (2):  
\[T_{i_{sub}}=\big{(}e_{i2},e_{i3},\ldots,e_{il_{i}}\big{)} \tag{2}\]  
Figure 3: Overall data pre-processing.  
The last token (target token) of the subsequence is removed, and the beginning of sequence token at the beginning and the sequence with the end of sequence token inserted at the end becomes the input sequence of the model. A padding token is inserted into each input sequence according to the length of the longest sequence among the entire input sequence set. Finally, input sequences are encoded as integers according to the vocabulary. In the vocabulary, unique integers are allocated for each token. The model outputs the result of predicting the next token at each location in the input sequence as an output sequence.  
Attributes are related to the occurrence time of an event such as the amount of a request, organization, importance, product, and timestamp. In this study, attribute data was divided into categorical attributes and numerical attributes. A categorical attribute is a variable that has label values such as 'product' and 'organization involved', and a numerical attribute is data that can be expressed numerically, such as 'case:AMOUNT_REQ (the amount requested by customers).  
Data normalization is a process of standardizing attribute data to a smaller range by scaling. If data is normalized, the same weight can be given to all attributes. The normalization process is particularly effective for neural networks or distance-based models. By normalizing the input data of a classification problem using neural networks, it helps to accelerate the learning phase [30]. In this research, one-hot encoding was applied to the categorical attribute and min-max scaling was applied to the numerical attribute to normalize all data to the [0, 1] range.