# Detecting and Responding to Concept Drift in Business Processes
## 4 Methodology
### Model Management Using MDP  
We use tuple \(\langle\mathbf{P_{SO}},\mathbf{G_{SO}}\rangle\) and \(\langle\mathbf{P_{SN}},\mathbf{G_{SN}}\rangle\) to represent the transition and duration matrices of the old and new data, respectively. Furthermore, we use \(H_{O}\) and \(H_{N}\) as their high-level representation. We use tuple \(\langle H_{O},H_{N}\rangle\) as the MDP state that contains information about the environment evolution. Naturally, we say there is no concept drift if the old and new data have the same high-level representation, i.e., \(H_{O}=H_{N}\). Otherwise, there is a high possibility for concept drift. During the MDP learning stage, depending on the current state, one of the actions is selected to update the current/old SMP model. The reward of the selected action is then calculated to update the decision-making policy. During the MDP testing stage, given the current state, i.e., the high-level environment representation derived from the old and new data, we select the action that achieves the highest long-term reward learned in the training process. In the scenario that we use two K-means models, respectively, for the transition and duration matrix to have a two-dimensional high-level representation, actions can be made individually for transition probabilities and gamma mixture models, leading to nine actions in total. Consequently, the actions are 'keep_keep', 'keep_adapt', 'keep_retrain', 'adapt_keep', '\(\cdots\) and'retrain_retrain'. For example, action 'keep_retrain' means that we maintain the current transition probabilities but retrain duration data into gamma mixture models. On the other hand, if the high-level representation is obtained using one K-means model, we have three actions: keep, adapt or retrain both the transition probabilities and gamma mixture models. In other words, the three actions are 'keep_keep', 'adapt_adapt' and'retrain_retrain'. Regarding actions for transition probabilities, if action 'keep' is selected, we consider there is no drift in the process control-flow and therefore keep the current transition probability matrix (\(\mathbf{P_{SO}}\)) unchanged, i.e., \(\mathbf{P_{SO}^{k+1}}=\mathbf{P_{SO}^{k}}\). \(k\) and \(k+1\) refer to the transition probabilities before and after updating. If action 'adapt' is selected, we consider there is a gradual drift, and therefore, we update the model using the observed new data. That is, \(\mathbf{P_{SO}^{k+1}}=\frac{\mathbf{P_{SO}^{k}}+\mathbf{P_{SN}^{k}}}{2}\). We consider there is a sudden drift if action'retrain' is selected and the current transition matrix is replaced by the new data. That is \(\mathbf{P_{SO}^{k+1}}=\mathbf{P_{SN}^{k}}\). We denote the current gamma mixture models and the gamma mixture models derived from the new data batch as \(\mathbf{ga-mm_{O}}\) and \(\mathbf{ga-mm_{N}}\), respectively. For duration actions, if action 'keep' is selected, we keep the current mixture model unchanged, i.e., \(\mathbf{ga-mm_{O}^{k+1}}=\mathbf{ga-mm_{O}^{k}}\). As a result, the low-level duration matrix is the same, i.e., \(\mathbf{G_{SO}^{k+1}}=\mathbf{G_{SO}^{k}}\). If action 'adapt' is selected, we merge the old and new data to refit gamma mixture models. Otherwise, if action'retrain' is selected, we refit the gamma mixture models using the new observation data. The immediate reward of selecting action 'a' at state's' (\(\langle H_{O},H_{N}\rangle\)) is calculated as Equation (6). It has three components: (1) the accuracy of updated transition probabilities, (2) the goodness of fit of updated gamma mixture models and (3) a penalty for using a specific action. For the first component, we use the multinomial test [49] to measure the difference in each state transition between the updated transition probabilities and transition probabilities from the new data batch. In other words, the multinomial test is applied between each row of the updated transition matrix and the new data transition matrix.