# Data Preprocessing Method and API for Mining Processes from Cloud-Based Application Event Logs
## 5 CPA-PM Application Programming Interface (API)
* **keepLastEvent** (table DataSet, groupID, String eventAttribute, String eventName): similar to the previous one, but keeps the _last_ occurrence of an event (i.e., the last occurrence of several repeated events). * **mergeRows** (table DataSet, list groupByVariables, list columnArguments): aggregates several rows into one row. _groupByVariables_ represent the variables to group the dataframe by, and _columnArguments_ represents the new column arguments based on aggregation functions. Each column of the aggregated rows requires one predefined aggregation operator from: mean, median, min, max, sum, first, last, n (count), of logical operators. String concatenation is supported too. Those aggregation operators are defined by the analyst. As mergeRows() is the most complex function of this API, we are illustrating its application on a simple sequence of events extracted from a real event log. This function works by splitting the data into groups, applying some analysis to each group of data, and then combing the results. It works by aggregating several rows for each column. For instance, the first three rows of Table 1 target the same activity of the same case identifier. The analyst may decide to keep only one such event (as in the patterns described earlier). The issue here is to decide how to aggregate the information in the various columns. When aggregating the rows grouping by _case_id_ and _activity_ attributes, the analyst may decide to keep the _timestamp_ and _weekday_ of the first event; produce lists of unique _activity_, _city_, _client_, and _device_ values for their respective columns; calculate the sum of the merged elements in the _num_items_ column, and keep all the values of the _product_category_ \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Case\_id** & **Timestamp** & **Activity** & **City** & **Weekday** & **Client** & \
\begin{tabular}{c} **Num\_** \\ **Items** \\ \end{tabular} &
\begin{tabular}{c} **Product\_** \\ **Category** \\ \end{tabular} & **Device** \\ \hline \hline
121 & 2018-08-01 & Main home & Ottawa & Monday & Phone App & 3 & B & Android \\
121 & 00:04:22 & page & Ottawa & Monday & N/A & 4 & B & Android \\
121 & 2018-08-01 & Main home & Ottawa & Monday & N/A & 6 & D & Android \\
121 & 00:04:25 & page & Ottawa & Monday & Phone App & 4 & A & Android \\
121 & 2018-08-01 & \
\begin{tabular}{c} Home page \\ \end{tabular} &
\begin{tabular}{c} Great \\ website \\ search \\ \end{tabular} & Ottawa & Monday & Phone App & 4 & N/A & Android \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Simple event log example, with targeted events to be merged highlighted in gray. column (without uniqueness). The invocation of this API in R on the log named _EventLogs_ is as follows, with the result shown in Table 2.  
mergeRows (EventLogs,. (case_id, activity), summarise,  timestamp = first(timesamp),  activity = paste(unique(activity), collapse = ','),  city = paste(unique(city), collapse=','),  weekday = first(weekday),  client = paste(unique(client), collapse=','),  num_items = sum(num_items),  product_category = paste((product_category), collapse=','),  device = paste(unique(device), collapse=',')) -> EventLogs  
This API contains basic functions to load/save CSV files and filter rows and columns, but also advanced ones that support the various steps of CPA-PM more directly and efficiently. For example, deleteTruncatedTracesStart() and deleteTruncatedTracesEnd() are essentially used to support Step 1.7 of the CPA-PM method (Section 4.1). These functions exclude from the event log the incomplete traces (i.e., all the events for one case) that either started too early or finished too late (or did not finish) according to the time frame where the log was produced, see Figure 3.  
This API addresses challenges #2 to #9 from the literature review. It is currently implemented in R, but a Python version is also being considered.