# Parallel-Structure Deep Learning for Prediction of Remaining Time of Process Instances
## 3 Proposed Method
### Network Architecture
#### 3.3.1 Multi-Layer Perceptron with a Data-Aware Entity-Embedding Layer  
The first component of our proposed method is a multi-layer perceptron with a data-aware entity-embedding layer (MLP + DAEE), which accepts an SOP as input. Table 2 describes the details of layer names, layer types, input and output shapes, and their connectivity of the embedding component. Table 3 describes the details of layer names, layer types, input and output shapes, and their connectivity of MLP with the Data-Aware Entity-Embedding component. Note that \"n\" and \"m\" in the tables indicate the number of categorical variables and continuous variables, respectively. Rectified linear unit (ReLU) [27] is the most widely used activation function in the world right now due to its simplicity but should be applied only to the hidden layer. For both the MLP + DAEE and CNN components, we use the ReLU as an activation function, except for the output layer, where we use the linear activation function because we are dealing with regression problems. We chose ReLU because of its two significant advantages. The first is its ability to reduce the likelihood of gradients to vanish because of its constant gradient growth, unlike the case with sigmoid, where the gradient value gets smaller. The constant gradient in ReLU also causes the learning process of the network to be faster. The second advantage is sparsity because it limits its value to no less than zero, unlike the case with sigmoid, which produces non-zero values, thereby causing dense representation. Figure 4: Converting the set of prefixes into a chart image. A deep NN that is trained on relatively few datasets tends to overfit on training data easily. Generalization error increases due to overfitting. We adopted Dropout [28] for several hidden layers of our network architecture to help to cope with the overfitting problem. As with other hyperparameters in deep learning, the dropout percentage rate (which represents the percentage of neurons that will be dropped out) is also chosen based on trial-and-error experiments in general. Because the problem we were dealing with in this study is a regression problem, the suitable loss function to be used in the output layer was a choice between Root Mean Squared Error (RMSE) and MAE. We chose MAE because it is more robust to outliers [29]. MAE is obtained from the average of the absolute difference between the actual and predicted values. Dealing with categorical variables in machine learning is challenging. One known method in machine learning for treating categorical variables is to encode them into integer labels. However, that is insufficient because our machine-learning model will assume that a \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Layer (Type)** & **Input Shape** & **Output Shape** & **Connected to** \\ \hline embed (Embedding) & (\(|D|\), 50) & (1, \(|D|\times\) 50) & input\_dis\_1 \\  & & & input\_dis\_2 \\  & & &... \\  & & & input\_dis\_n \\ dense\_1 (Dense) & (1) & (1) & input\_con\_1 \\ dense\_2 (Dense) & (1) & (1) & input\_con\_2 \\ dense\_3 (Dense) & (1) & (1) & input\_con\_3 \\... &... &... &... \\ dense\_m (Dense) & (1) & (1) & input\_con\_m \\ reshape (Reshape) & (1, \(|D|\times\) 50) & (\(|D|\times\) 50) & embed[1] \\  & & & embed[2] \\  & & & embed[3] \\  & & &... \\  & & & embed[n] \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Input layer for SOP and connectivity of embedding component.