## 3 Improving Multi-Class Classification for Recognition of the Prioritized Classes Using the AHP
The AHP has the potential to be modified for use in machine-based decision-making processes that include complicated multi-criteria challenges, such as security and machine learning, among others [31; 32]. The choice of the AHP is based on the following characteristics of the method [33]:  
* The AHP provides the ability to check the consistency of the evaluations provided by decision makers;
* The AHP allows the use of heterogeneous measurement scales for different criteria;
* The AHP eases multi-objective decision-making by using exclusively pairwise comparisons of the alternatives, which provides increased reliability of the results;
* The most controversial step of manual weight assignment to different criteria is avoided. A three-level hierarchical structure for the AHP, aimed at the evaluation of the machine learning model's performance, is presented in Figure 2.  
Level one is the objective of the process, which in our case is to improve (or at least to not decrease) the performance of the ML model. The second level is the criteria. This level is based on the personal preferences of the final user of the model and is completed by the field experts. The main input used in this step is the criteria pairwise comparison matrix, also called the judgment matrix, which is manually generated before the AHP process is started. For example, assume that the intrusion detection dataset involved in the machine learning process contains four classes: Normal, DDoS, Worms, and Exploits. Then four criteria should be considered at level 2 of AHP. The 4 \(\times\) 4 judgment matrix should be  
Figure 2: A hierarchical framework for the AHP. constructed by the experts, comparing the importance of each class in the context of the concrete infrastructure. The learning process begins with the provision of the judgment matrix, which includes all the necessary weights for each criterion. The third level is alternatives. These always include only two because the AHP is used to decide whether to include (or not) the newly calculated changes into the main ML model. The AHP process compares both alternatives using each criterion in Level 2 separately. This is an effortless task, as all criteria used in ML are numerical, and the process of constructing judgment matrices in Level 2 is based on comparing numerical estimates of the criteria and converting the differences into numerical values suitable for the AHP (i.e., the real numbers in the range of 0-9). The whole process of ML model evaluation is summarized in Figure 3.  
One of the intriguing application areas of the AHP score-based machine learning model performance evaluation is multi-criteria adaptive incremental learning. The main steps in applying AHP in the process of adaptive incremental learning are as follows:  
1. Construct an AHP framework according to the structure provided in Figure 2 and include all the criteria suitable for the concrete dataset and application area. 2. Load the judgment matrix representing the priorities of the concrete organization with the results of pairwise comparisons of all criteria. 3. Obtain the performance characteristics of the current and the new machine learning models. 4. For each criterion, repeat the following: * Construct the weight coefficient matrix using both alternatives (i.e., metrics of the current and the new ML model). The elements of the matrix are calculated using a comparison function, which compares performance metrics of the current and the new model and converts them into real numbers from the interval (0, 9). 5. Provide all the created matrices with the standard AHP decision-making method to obtain the estimated weights of both alternatives. 6. Check the consistency of the matrices using AHP consistency indicators. Choose the best alternative as the final decision. Figure 3: The proposed method of the AHP decision-making process. We suggest using the score function \(scr(x)\) to compare the performance metrics of individual ML models and to create the judgment matrices needed in step 5:  
\[scr(x)=\left\{