# POP-ON: Prediction of Process Using One-Way Language Model Based on NLP Approach
## 5 Conclusions  
In this paper, we investigated the efficiency of a one-way language model approach among fully attention-based transformer models for predicting future process events of the current process instance being executed. In addition, we proposed POP-ON, a process prediction model. As far as we know, this was the first time this approach has been applied in the area of business process prediction. The research goal was to apply the newly proposed approach to log data generated by the manufacturing process and software, and to evaluate the predictive performance of the next process event. To improve the prediction accuracy, we performed one-hot encoding and min-max scaling in the data pre-processing step, and made it possible to use categorical data. In the data pre-processing stage, after categorical data was applied to the experiment, the hyperparameters were adjusted to determine the impact on the prediction results, and hyperparameter optimization was applied to find the optimal configuration. Based on GPT-2, the proposed POP-ON model adds a linear network that uses the attribute of an event as an input to the residual structure of the multi-head attention layer within the Transformer Decoder Layer so that elements other than the sequence can be reflected in the prediction.  
In Section 4, the predictive performance of the proposed approach and the existing deep learning approach were compared and evaluated. The experimental results suggest that the proposed method achieved good results in the evaluation index and outperformed the state-of-the-art approach in most experiments. In addition to its excellent predictive performance, the proposed Transformer approach demonstates that powerful predictive models of the latest trends used in NLP could also be used in process mining based on business process log data.  
Future research aims to solve the 'black box' characteristic of the deep learning method used in the process prediction analysis process and provide post-mortem explanations so that domain experts can interpret the prediction. This is important for ensuring reliability of model results and applying them to actual processes in enterprises. Our future research goal is to make the process prediction model easier to understand, and to use the reliable predictive model for enterprise. In addition to the one-way language-based model in the transformer, we intend to analyze the process with a powerful model that considers the state of events in both directions such as BERT. In process prediction, a one-way language model was used as needed. However, in addition to forecasting, business process management requires a more versatile monitoring component, and another approach can be applied for this.  
Conceptualization, J.M. and J.J.; methodology, J.M.; software, [PERSON] and [PERSON]; validation, [PERSON] and [PERSON]; formal analysis, [PERSON]; investigation, [PERSON] and [PERSON]; resources, [PERSON]; data curation, [PERSON]; writing--original draft preparation, [PERSON] and [PERSON]; writing--review and editing, [PERSON]; visualization, [PERSON]; supervision, [PERSON] project administration, [PERSON] All authors have read and agreed to the published version of the manuscript.  
This research was supported by the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2020-2018-0-01417) supervised by the IITP (Institute for Information & Communications Technology Planning & Evaluation).  
Not applicable.  
Data Availability Statement: BPI Challenge 2012 (www.win.tue.nl/bpi/doku.php?id=2012:challenge), BPI Challenge 2013 (www.win.tue.nl/bpi/doku.php?id=2013:challenge), Helpdesk (data.mendeley.co m/datasets/39 bp3 vv62t/1)  
This work was supported by the Smart Factory Technological R&D Program S2727115 funded by Ministry of SMEs and Startups (MSS, Korea).  
The authors declare no conflict of interest.