## 4 Performance Analysis
### Model Evaluation  
This section quantitatively explains that the model learning method proposed in this paper can produce meaningful results in the manufacturing process. We used two algorithms for AI learning. First, the model was trained LSTM, and the hidden characteristics of data were reflected using Bi-LSTM with a reverse LSTM layer added. As described above,  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Case ID** & **Activity Machine** & \multicolumn{2}{c}{**Start Time**} & \multicolumn{2}{c}{**Complete Time**} & **Qty** \\ \hline Case 9 & Turning—Machine 9 & 2012-01-30 & 14:38 & 2012-01-30 & 15:59 & 20 \\ Case 9 & Turning—Machine 9 & 2012-01-30 & 16:07 & 2012-01-30 & 17:34 & 20 \\ Case 9 & Turning—Machine 9 & 2012-01-30 & 17:35 & 2012-01-30 & 21:03 & 20 \\ Case 9 & Turning Q.C. & 2012-02-01 & 07:23 & 2012-02-01 & 07:30 & 20 \\ Case 9 & Lapping—Machine 1 & 2012-02-01 & 09:54 & 2012-02-01 & 11:18 & 20 \\ Case 9 & Lapping—Machine 1 & 2012-02-14 & 15:13 & 2012-02-14 & 17:18 & 20 \\ Case 9 & Round Grinding—Manual & 2012-02-16 & 06:58 & 2012-02-16 & 08:09 & 20 \\ Case 9 & Round Grinding—Manual & 2012-02-16 & 08:09 & 2012-02-16 & 11:11 & 20 \\ Case 9 & Laser Marking—Machine 7 & 2012-02-16 & 11:29 & 2012-02-16 & 11:30 & 20 \\ Case 9 & Final Inspection Q.C. & 2012-02-19 & 13:20 & 2012-02-19 & 15:00 & 20 \\ Case 9 & Final Inspection Q.C. & 2012-02-20 & 09:07 & 2012-02-20 & 09:08 & 20 \\ Case 9 & Packing & 2012-02-21 & 00:00 & 2012-02-21 & 01:00 & 20 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Metal processing event log of Case 9.  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Parameter** & **Description** \\ \hline CPU & Intel core i7 \\ Ram & 64 GB \\ Graphics Card & Geforce 1660 TI \\ \hline \hline \end{tabular}
\end{table}
Table 5: Description of Parameters.  
the data set used for this experiment is a metal processing process data set, and each feature is a combination of the machine model name and function used for the manufacturing process. As a hyperparameter in this experiment, batch size was set to 40. Epochs were unified as 1000, and the learning rate was set to 0.001. MAE was used as an evaluation index, and as a result of the experiment, both the LSTM model and the Bi-LSTM model were able to obtain positive results as the number of learning increased and showed convergence when a specific epoch was exceeded. In addition, a comparison of the results of the model using the validation set and the model using the train set shows that there is no overfitting to one side, indicating that the learning results of the corresponding model are significant. The LSTM model converges to 0.45 (hour), and when the Bi-LSTM algorithm is used, the LSTM model converges to 0.15 (hour).  
Figure 12 is the result of training using the LSTM algorithm to predict the manufacturing process time. As the epoch increases, they all move downwards to the right. The graph on the left shows the results of the Train Model, while the graph on the right shows the results of the Test Model. When the two models are compared, they can be checked for not being overfitted, such as biased to one side; this validates the corresponding model. In the case of the model using the LSTM algorithm, it is evident that the MAE decreases rapidly up to 200 epochs, and the lowest MAE value is 0.45 (hour), which converges at the 0.45 level through the graph.  
Figure 13 is the result of learning to predict the manufacturing process time using the BI-LSTM algorithm, as shown in the previous figure. Both the Train Model and the Test Model showed a downward trend and showed convergence after a certain epoch. Furthermore, a comparison of the two graphs shows that the learning was properly accomplished through the fact that overfitting, such as biasing, did not occur. The Test Model graph shows that it gradually converges from 150 to 200 Epoch, and the best result was 0.15 (hour).  
Figure 12: Comparison between train data and test data of the model using the LSTM algorithm.