## 4 Proposed Framework  
Currently, there is a gap in the literature concerning the performance evaluation of AI algorithms in this domain. In this study, we proposed a comprehensive framework that \begin{table}
\begin{tabular}{l l l} \hline \hline
**Column ID** & **Description** & **Values** \\ \hline Incident number & The unique internal code of the incident & INC123 xxxx \\ Assignment group & The group to which the incident has been assigned. & 17 March 2020 \\ Opened at & Date/Timestamp of when created the incident record. & 18 March 2020 \\ Closed at & Date/Timestamp of when the incident record was closed. & (1—High; 2—Medium; 3—Low; 4—None) \\ Incident severity & The level of impact for each incident. & 18 March 2020 \\ \hline CMDB & The name of the configuration management database associated with the incident & \\ Category & The category associated with the incident & \\ Short description & Brief information about the incident. & \\ Description & Detailed information about the incident. & \\ Status & The manual mapping from problem to incident. & (0—MIR; 1—Non-MIR). \\ MIR opened at & Date/timestamp of when created the MIR record. & 17 March 2020 \\ Incident opened at & Date/timestamp of when the incident record was opened. & 18 March 2020 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Data dictionary. compared various ML and DL approaches, specifically for IT incident prediction, using our proprietary industry-level data. We developed a computational pipeline that began with preprocessing, followed by feature extraction, training, and evaluation (a detailed description of the pipeline is presented in Section 5.1) and shown in Figure 3.  
Our comparative pipeline is shown in Figure 4. The first phase of the pipeline was data preparation. As discussed earlier in Section 3, we had performed data augmentation and synthetic resampling on our original dataset. These datasets were split (70% for training and 30% for testing). Using the training data, we had performed supervised training using all 11 classifiers. Specific to the DL-based classifiers, we allocated 20% of our training dataset for training validation using the default Keras model's _fit_ function. The training phase was performed for 10 epochs with _binary_crossentropy_ as the loss function and AUC as the performance metric. We selected AUC as a metric to prevent over-fitting due to the imbalanced characteristics of our dataset. Finally, the models were tested with the test data (30% of the dataset). Similarly, the AUC score was determined to evaluate the performance of all 11 classifiers. Figure 3: Proposed IT incident risk framework. An essential element of the prediction framework was the utilization of various state-of-the-art classifiers. In this work, we categorized these classifiers as (1) conventional ML methods, including NB, SVM, gradient boosting (GB), extreme gradient boosting (XGBoost), and categorical boosting (CatBoost); (2) DL-based approaches, including gated recurrent unit (GRU), CNN, and Bi-LSTM; and lastly, (3) transformers, including BERT, robustly optimized BERT (RoBERTa), and enhanced representation through knowledge integration (ERNIE 2.0). A brief description of each classifier is provided below. * **NB**[42]: Naive Bayes classifier is based on Bayes theorem and is one of the popular ML algorithms for classification. NB is a simple probabilistic model that assumes that each feature or variable of the same class makes an independent and equal contribution to the outcome. Our dataset was divided into a feature matrix and a target vector. The feature matrix (\(X\)) contained all the vectors (rows) of the dataset, in which each vector consisted of the value of dependent features. We assumed that the number of features were \(d\), \(X=(x_{1},x_{2},x_{3},\ldots,x_{d})\) while the target vector (\(y\)) contained the value of the class variable for each row, according to its feature matrix. The feature matrix (\(X\)) was the incident description reported in the database system. The target (\(y\)) value would either be MIR or NMIR.