# Enhancing associative classification on imbalanced data through ontology-based feature extraction and resampling

Joel Mba Kouhoue, Jerry Lonlac, Alexis Lesage, Arnaud Doniec, Stéphane Lecoeuche

► To cite this version:

Joel Mba Kouhoue, Jerry Lonlac, Alexis Lesage, Arnaud Doniec, Stéphane Lecoeuche. Enhancing associative classification on imbalanced data through ontology-based feature extraction and resampling. Knowledge-Based Systems, 2025, 309, pp.112897. 10.1016/j.knosys.2024.112897. hal-04855987

HAL Id: hal-04855987

https://hal.science/hal-04855987v1

Submitted on 13 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Enhancing associative classification on imbalanced data through
ontology-based feature extraction and resampling

Joel Mba Kouhoue a,*, Jerry Lonlac a, Alexis Lesage b, Arnaud Doniec a, Stéphane Lecoeuche c

<sup>a</sup> IMT Nord Europe, Institut Mines-Télécom, University of Lille, Lille, France

<sup>b</sup> Intent Technologies, Toulouse, France

<sup>c</sup> IMT Mines Alès, Ales, France

A B S T R A C T

Associative classification models are valuable for discovering relationships within heterogeneous data systems, making them particularly useful for data integration tasks. However, they struggle with imbalanced and sparse data. This paper addresses the problem of imbalanced classification in building maintenance data by providing several updates based both on algorithms and preprocessing. Experiments conducted on real maintenance datasets demonstrate significant improvements in accuracy and precision.

Keywords:

Imbalanced associative classification

Ontologies

Feature extraction

Oversampling

Building maintenance

1. Introduction

Several efforts have aimed to standardize building data exchange,
yet no standard has emerged for the operation and maintenance phase.
Existing solutions still require manual data matching and sharing
among owners, service providers, and occupants, which is tedious due
to the high data volume. Therefore, it is essential to find new ways to
find matching between different data views. In this work, we assume
to represent matching as association rules. These rules take the form
X → Y, linking the subset of values X of a view V1 with the conclusion
Y of a view V2.

This paper extends the work in [1], where we introduced the CARRT
framework (Classification based on Association rules Ranking with
Topsis method) for associative classification. While we achieved good
results for elevator maintenance data, the models were less effective
in other building maintenance domains due to under-information and
class imbalances. Our research aims to address these issues to improve
the performance of associative classification models in this context.

To achieve these objectives, we implements several updates based
both on algorithms and preprocessing. Especially, in terms of algorithm
updates, we provide additional criteria, such as *Lift* and *Conviction*, in
the rule ranking step, which are less sensitive to the class distribu-
tion. For preprocessing, we match textual descriptions of maintenance
operations with structured concepts in the Brick ontology [2]. This
ontology-based feature engineering generates additional features for
our classification models. We also use the SMOTEN (Synthetic Minority

Over-sampling Technique for Nominal) method [3], which generates
synthetic data for minority classes and is suited for multi-class and
categorical data.

We experimented with eleven associative classification algorithms
on real building maintenance datasets from *Intent Technologies*¹. For
each dataset, we assessed the model's performance initially, after on-
tology feature extraction, and post-oversampling. Results indicate that
our approach achieves the best results in terms of precision. Regarding
data augmentation strategies, we observed enhanced accuracy follow-
ing feature extraction and a notable improvement in precision after
resampling.

The rest of the paper is organized as follows. Section 2 provides a
common background about associative classification. We also discuss
related work and define the problem. Section 3 outlines our proposed
method, detailing the approach adopted to address the identified chal-
lenges. Section 4 presents experimental evaluation and results. Finally,
Section 5 provides some statistical tests of our results, highlighting the
effectiveness of our approach.

2. Related work and problem definition

2.1. *Common background*

Let $I = \{x_1, x_2, \dots, x_m\}$ be a set of elements called items. A set $X \subseteq I$ is called an *itemset*. Let $T = \{t_1, t_2, \dots, t_n\}$ be another set of elements

* Correspondence to: 46 rue durutte, 3ieme etage porte droite, 59500 Douai, France.
*E-mail addresses: joel.mba-kouhoue@imt-nord-europe.fr (J. Mba Kouhoue), jerry.lonlac@imt-nord-europe.fr (J. Lonlac), a.lesage@intent-technologies.eu (A. Lesage), arnaud.doniec@imt-nord-europe.fr (A. Doniec), stephane.lecoeuche@mines-ales.fr (S. Lecoeuche).

¹ https://intent.tech
called transaction identifiers (tids). A set $T \subseteq \mathcal{T}$ is called a tidset. Let $\mathbf{D}$ be a binary relation on the set of tids and items, i.e $\mathbf{D} \subseteq \mathcal{T} \times I$. Thus, $(t, x) \in \mathbf{D}$ if the transaction $t$ contains the item $x$. Let $\mathbf{f} : 2^{\mathcal{T}} \to 2^I$ be a function defined as follows: $\mathbf{f}(T) = \{x | \forall t \in T, (t, x) \in \mathbf{D}\}$ i.e., $\mathbf{f}(T)$ is the set of items that are common to all the transactions in the tidset $T$. Let $\mathbf{g} : 2^I \to 2^\mathcal{T}$ be a function defined as follows: $\mathbf{g}(X) = \{t | \forall x \in X, (t, x) \in \mathbf{D}\}$ i.e., $\mathbf{g}(X)$ is the set of tids that contain all the items in the itemset $X$. The support of an itemset $X$, denoted $supp(X)$, is the number of transactions that contain $X$, i.e., $supp(X) = |\mathbf{g}(X)|$. An itemset $X$ is said to be frequent if $supp(X) \geq minsup$, where $minsup$ is a user-defined threshold. The problem of mining all frequent itemsets can be described as follows: determine all subsets $X \subseteq I$ such that $supp(X) \geq minsup$. An association rule (AR) is an expression of the form $X \to Y$, where $X$ and $Y$ are itemsets and $X \cap Y = \emptyset$. $X$ is called antecedent and $Y$ is called consequent of the rule. The support of an association rule $X \to Y$ is the number of transactions that contain the itemsets $X$ and $Y$: $supp(X \to Y) = supp(X \cap Y)$. The confidence of an association rule $X \to Y$ measures its predictive accuracy and is given by $conf(X \to Y) = supp(X \cap Y)/supp(X)$. A rule is considered a strong rule if $conf(X \to Y) \geq minconf$, where $minconf$ is a user-defined threshold. However, it is sometimes useful to extract co-occurrences between a set of items and a specific target attribute of interest. These rules are known as class association rules (CARs). Associative classification (AC) is a variant of association rule discovery where only the class attribute appears on the rule's right-hand side (consequent).

## 2.2. Associative classification algorithms

Among the methods used for associative classification, CBA (Classification Based on Association, [4]) is the first proposed algorithm. CBA uses the Apriori algorithm for the generation of association rules that satisfy predefined support and confidence thresholds. For the prediction phase, CBA uses a heuristic algorithm described as follows: for each learning case, the rule with the highest confidence among the rules covering it is stored [5]. The stored rules are ranked in decreasing order by confidence and the selection of the rule depends on which one covers a new case with the highest confidence. CBA2 [6] is an improved version of CBA which addresses the problem of unbalanced classes by using multiple class minsups in rule generation (i.e. each class is assigned a different minsup), rather than using only a single minsup as in CBA. To improve the efficiency of the Apriori candidate generation step, CMAR (Classification based on Multiple Association Rules) [7] and L3 [8] AC methods use approaches based on the Frequent Pattern (FP)- growth method [9] to discover rules, and store these rules in a prefix tree data structure known as a CR-tree. Algorithms that use the CR-tree consider the common attribute values contained in the rules, which use less storage if compared with CBA. In addition, rules can be retrieved efficiently as CR-tree indices rules [10]. Recently, [11] proposed several variations of these associative classification algorithms that take into account data consisting of several features of different types. [12] proposed a new associative classification method to handle data with soft labels. This extension introduces imprecise classification association rules (ICAR) to accommodate flexible consequences, and proposes support and confidence measures for ICAR. The developed method, association rule-based soft labeled classification method (ARC-SL), utilizes entropy-based adaptive partitioning, Apriori-based rule mining, and rule pruning to construct a more accurate classification model, validated through experiments on benchmark datasets and facial expression recognition. However, the current algorithms and ranking methods only use objective criteria such as support and confidence, neglecting subjective or qualitative knowledge held by business experts. To address these limitations, [1] propose a new associative classification framework called CARRT (Classification based on Association Rule Ranking with Topsis method). CARRT used TopKClassRules algorithm [13] at the association rules discovery step. At the ranking stage, CARRT implemented some ranking strategy based

<table><caption>**Table 1**
Example of fields associated with building maintenance data.</caption><thead><tr><td>Fields</td><td>Data type</td></tr></thead><tbody><tr><td>Reference, customer and service provider codes</td><td>Alphanumeric</td></tr><tr><td>Description</td><td>Unstructured Text</td></tr><tr><td>WorkType, Topic</td><td>Structured Text</td></tr><tr><td>Criticality, Estimated and Actual resolution duration</td><td>Numeric</td></tr><tr><td>Start and end dates</td><td>Date</td></tr></tbody></table>

on TOPSIS (Technique for Order Preference by Similarity to Ideal Solution) method [14], which is a multi-criteria decision making technique that has the advantage of taking into account both objective criteria (support, confidence, lift, conviction, size of rules) and the subjective knowledge held by domain experts. The classifier is then built by taking only the best rules leading to each class.

## 2.3. Building maintenance data issues

In real-world applications with actual data, machine learning models encounter various challenges. Particularly in the building maintenance sector, the primary hurdles are information scarcity and data imbalance. The objective of applying machine learning in this sector is to construct precise classifiers for predicting customer service codes using data supplied by service providers from log data. Table 1 represents the main data fields contained in a maintenance log. The prediction quality remains low due to insufficient information, particularly the absence of data fields regarding equipment and maintenance activity, crucial for predicting the customer's service code. This essential information can be inferred from the “description” field, which contains detailed descriptions of maintenance operations provided by operators. However, this unstructured text data varies across service providers, necessitating intermediate processing before inputting it into our classification models. To address these issues, several approaches, such as fuzzy ontology and type-2 fuzzy ontology [15] have been proposed for information retrieval using ontologies. Additionally, the work presented in [16] involved designing a system aimed at extracting information about hotels using fuzzy queries made by users in natural language and performing opinion mining on the features of those hotels.

## 2.4. Association rule-based classifier for imbalanced classification

Imbalanced classification problems are attracting the attention of the research community because they are prevalent in real-world problems. In the context of associative classification, it is unlikely to find rules describing the small classes, as the combination of items characterizing the small classes occurs too infrequently to pass certain significance tests for detecting association patterns. Consequently, classification rules obtained from the discovered association patterns for predicting the small classes are rare and weak [17]. In general, methods used to address this problem can be grouped into three main categories: data resampling methods [18,19], which involve either oversampling the minority classes or undersampling the majority classes; algorithmic approaches, which involve modifying existing algorithms or creating new ones to directly tackle imbalance problems; and hybrid approaches, which combine the previous two methods. CBA2 [6] is one of the earliest associative classification algorithms designed to handle class imbalance, employing a multi-support strategy to assign specific minsup values for each class. A new Fuzzy Association Rule-based Classifier was proposed by [20], specifically designed to address imbalanced classification problems. FARCI incorporates modifications throughout the FARC-HD [21] algorithm, focusing on lift-based rule generation, pattern weighting, and fitness function adaptation.

Despite various approaches proposed in the literature, no single solution addresses all issues related to the building maintenance sector. In the following section, we outline the method implemented to tackle these issues, involving feature extraction from a building ontology and resampling using the SMOTEN method.
<figure><img src="image_2.png" /><figcaption>Fig. 1. Proposed methodology.</figcaption></figure>

## 3. Proposed method

### 3.1. Rules ranking strategy

Ranking association rules is a key process in any associative classification task. Given the very large number of rules extracted during the association rule mining phase, it is important to filter them to keep only the best rules. The particularity of the association rule ranking approach proposed in [1] is that we used multi-criteria analysis to rank these rules. Specifically, we relied on the TOPSIS (Technique for Order Preference by Similarity to Ideal Solution) method, which is a commonly used technique in multi-criteria decision-making. The advantage of this approach is that it allows for the combination of both objective and subjective quality measures in the ranking process. Regarding the objective criteria, we initially used support, confidence and rule size as criteria. However, in the context of imbalanced classification, these criteria may be insufficient. Indeed, support can be very low for minority classes, as it measures absolute frequency. As a result, rules with low support may be unreliable, as they are based on limited data. Conversely, confidence may be misleading. Indeed, a rule may have high confidence simply because the consequent is very rare and therefore, the total number of cases where the consequent is observed is low. To ensure the robustness of the CARRT Framework in the context of imbalanced classes, additional criteria have been added for the evaluation of rule quality, namely the lift and the conviction.

* Lift [22] measures the strength of the association between X and Y by comparing the observed frequency of X and Y together with the expected frequency if X and Y were independent.

$$Lift(X \to Y) = \frac{P(X \cap Y)}{P(X) \cdot P(Y)} \qquad (1)$$

Lift is often considered one of the most robust criteria for imbalanced datasets, as it evaluates the strength of the association independently of absolute frequencies.

* Conviction [22] measures the strength of the association between the antecedent and the consequent by considering how often the antecedent occurs without the consequent being present.

$$Conviction(X \to Y) = \frac{1 - P(Y|X)}{1 - P(Y)} \qquad (2)$$

Conviction is useful for measuring how much the presence of one item increases or decreases the probability of another item, offering a different perspective than lift or confidence. However, in practice, it is recommended to use a combination of these criteria to obtain a more comprehensive and balanced evaluation of association rules, taking into account the specifics of the dataset and the analysis objectives.

### 3.2. Ontology-based features extraction and data resampling for imbalanced classification

Fig. 1 shows an overview of the process carried out in order to solve the both problem of under-information and imbalanced class. We first use Brick ontology [2] in the feature engineering step to extract concept from text description attribute of each maintenance operation. Then, imbalanced datasets are divided into training data for association rule mining and testing data for testing classification models. After that, the oversampling process is carried out using the SMOTEN method to balance the data. Then, the resulting data is used for the associative classification process using CBA [4], CBA2 [6], L3 [8], CARRT [1] algorithms, etc. The final stage is to evaluate each method used to measure the performance of the resulting classification. (3.1)

#### 3.2.1. Ontology-based featured engineering

The objective of this step is to add new features as input to our associative classification models, specifically the features related to equipment and maintenance activities, based on the analysis of the textual description of maintenance operations. To achieve this, we proceed in following steps.
<table><caption>**Table 2**
Description of datasets.</caption><thead><tr><th>Dataset</th><th>N.F.</th><th>N.F.O.</th><th>N.C.</th><th>N.S.</th><th>N.S.S.</th><th>N.S.MC.</th><th>N.S.MC.S.</th><th>N.S.mC.</th><th>N.S.mC.S.</th><th>L.ratio</th><th>L.ratio.S</th></tr></thead><tbody><tr><td>Multi_1</td><td>11</td><td>15</td><td>8</td><td>194</td><td>355</td><td>74</td><td>74</td><td>6</td><td>28</td><td>12.34</td><td>2.64</td></tr><tr><td>Multi_2</td><td>15</td><td>16</td><td>6</td><td>594</td><td>1044</td><td>303</td><td>303</td><td>9</td><td>99</td><td>33.67</td><td>3.06</td></tr><tr><td>Multi_3</td><td>10</td><td>12</td><td>4</td><td>101</td><td>143</td><td>71</td><td>71</td><td>5</td><td>19</td><td>14.2</td><td>3.73</td></tr><tr><td>Multi_4</td><td>15</td><td>20</td><td>14</td><td>218</td><td>485</td><td>93</td><td>93</td><td>6</td><td>24</td><td>15.5</td><td>3.9</td></tr><tr><td>Heat_1</td><td>14</td><td>19</td><td>15</td><td>194</td><td>634</td><td>79</td><td>127</td><td>6</td><td>31</td><td>13.17</td><td>4.0</td></tr><tr><td>Heat_2</td><td>13</td><td>15</td><td>6</td><td>250</td><td>341</td><td>110</td><td>110</td><td>6</td><td>28</td><td>18.34</td><td>3.9</td></tr><tr><td>Comf</td><td>18</td><td>22</td><td>8</td><td>627</td><td>1044</td><td>294</td><td>294</td><td>6</td><td>64</td><td>49</td><td>4.5</td></tr><tr><td>Vent</td><td>17</td><td>19</td><td>6</td><td>504</td><td>802</td><td>290</td><td>290</td><td>6</td><td>64</td><td>48.34</td><td>4.5</td></tr></tbody></table>

**Preprocessing (Part-of-speech tagging):** In this step, we used en_core_web_trf<fnref n="2" /> model from Spacy [23] with has 91% accuracy for the part-of-speech tagging. We assume that the names mentioned in the description may potentially match with a subclass of the 'Equipment' class in the Brick ontology [2], and that the verbs mentioned in the description may match with a maintenance activity in the ontology [24]. For example, if we consider the following description: 'Hot water heater out of service, to be replaced,' we obtain the following result: ('Hot': 'ADJ', 'water': 'NOUN', 'heater': 'NOUN', 'out': 'ADP', 'service': 'NOUN', 'be': 'AUX', 'replaced': 'VERB'). After that, we only keep the terms corresponding to nouns and verbs to the next step.

**Syntactic matching:** After tagging terms in the description, we then perform SPARQL queries [25] to search the matching class from ontologies. For the nouns, we search the sub-classes of the 'Equipment' class of Brick ontology that match, and for the verbs, we search the classes that match in the maintenance activity ontology proposed by [24]. The quality of results after this matching step really depends on the nature of the input text data. So if the input data contains less useful information, there is probably no chance to get any matching concepts from ontologies.

**Dimensionality reduction:** The results of the previous matching process are stored in a two-dimensional table, where the first dimension (columns) consists of all ontology resources, and the second dimension specifies the textual description data. The values of this matrix are binary (Yes or No), indicating whether a textual expression matches with the given ontology resource. Given the large number of ontological resources to be tested, we filter the columns to keep only those with a coverage greater than 10%. The resulting columns are then used as input for our classification models.

Finally, we input both the initial structured data and the features derived from operation descriptions into our classification models. However, the issue of class imbalance still remains. In the next section, we will present the oversampling technique used to address the class imbalance problem.

### 3.2.2. Data splitting and sampling

The initial stage of the unbalanced dataset is divided into two parts. In several studies on imbalanced classes that have been carried out before, a comprehensive scheme for dividing training data and testing data uses the stratified splitting technique [26]. In this study, the data will be divided as follows, 50% of the data will be used as training data for building our classifier (data to be re-sampled) and 50% for the evaluation of the classifier performance (test data). The sampling was done using the SMOTE (Synthetic Minority Over-sampling Technique) method, which is widely used for addressing data imbalance issues [27]. This technique consists of adding new data to the minority class derived from the resulting artificial data to balance the amount of data between the majority and minority classes. Considering a sample $x_i$, a new sample $x_{new}$ will be generated considering its k nearest-neighbors. Then, one of these nearest-neighbors $x_{zi}$ is selected and a sample is generated using Eq. (3).

$$X_{new} = x_i + \lambda * (x_{zi} - x_i) \quad (3)$$

where, $x_{new}$ represents the new synthetic sample, $x_i$ is an original sample from the minority class, $x_{zi}$ is one of the k nearest neighbors of $x_i$ and $\lambda$ is a random parameter between 0 and 1, indicating the amount of interpolation between $x_i$ and $x_{zi}$.

Since the maintenance data we are working with are categorical, in this study we used the SMOTEN algorithm [3], which is similar to SMOTE but it expects that the data to resample are only made of categorical features, and supports multi-class re-sampling.

## 4. Experimental evaluation and results

### 4.1. Datasets

All the data used in this work is provided by the company Intent Technologies<fnref n="3" /> which develop a digital platform to collect, manage, and share building data over the different stakeholders of real estate industry. These data consist of height datasets about maintenance activities carried out by some service providers on the perimeter of their customers. Table 2 shows the summary of these datasets. The first 4 datasets are data relating to the multiservice business. Datasets 5 and 6 represent maintenance data for heating systems, while the last two are related to the maintenance of comfort equipment and the ventilation system respectively. These datasets do not have exactly the same features but have the same column label, which is the customer service code that we want to predict based on maintenance operations features provided by services providers. For each dataset, we display the following information: The name of the dataset (column 1), the number of features before and after Ontology-based features extraction (columns 2 and 3), the number of classes and the total number of samples (columns 4 and 5), the number of samples after overSampling (column 6), the number of samples in majority class before and after overSampling (columns 7 and 8), the number of samples in minority class before and after overSampling (columns 9 and 10), and the Imbalance ratio between the majority class and minority class, before and after overSampling (columns 11 and 12).

### 4.2. Algorithms summary

The experiments here used the java implementations of ACAC (Associative classification based on all-confidence) [28], ACCF (Associative classification based on closed frequent itemsets) [29], ACN (An associative classifier with negative rules) [30], ADT [31], CBA [4], CBA2 [6], CMAR [7], CPAR(Classification based on predictive association rules), L3 [8], and MAC [32] available at the LAC software library [33]. Our own implementation (CARRT [1]) written in Python was used. CARRT used the TopKClassRules available in SPMF [34]. Since CARRT does not set a support threshold as input, we set the support threshold of other algorithms quite low (5%) to ensure fair comparisons. We notice that the implementation code source of CARRT and all the datasets used in these experiments are available at github repository.<fnref n="4" />

<footnote n="2">² https://spacy.io/models/en</footnote>

<footnote n="3">³ https://intent.tech/</footnote>

<footnote n="4">⁴ https://github.com/joelMba/carrt</footnote>
<table><caption>**Table 3**
Accuracy results on initial data.</caption><thead><tr><th rowspan="2">Minconf</th><th rowspan="2">Algorithm</th><th colspan="8">Accuracy</th></tr><tr><th>Comf</th><th>Heat_1</th><th>Heat_2</th><th>Multi_1</th><th>Multi_2</th><th>Multi_3</th><th>Multi_4</th><th>Vent</th></tr></thead><tbody><tr><td rowspan="12">30</td><td>acac</td><td>10.70</td><td>32.99</td><td>58.77</td><td>32.99</td><td>38.38</td><td>56.86</td><td>42.20</td><td>30.56</td></tr><tr><td>accf</td><td>84.39</td><td>34.02</td><td><strong>79.82</strong></td><td>57.73</td><td>80.13</td><td><strong>70.59</strong></td><td>41.28</td><td>90.48</td></tr><tr><td>acn</td><td>84.39</td><td>36.08</td><td><strong>79.82</strong></td><td><strong>59.79</strong></td><td>79.80</td><td><strong>70.59</strong></td><td>42.20</td><td>90.48</td></tr><tr><td>adt</td><td><strong>86.43</strong></td><td>40.21</td><td>48.25</td><td>38.14</td><td>50.84</td><td>70.59</td><td>42.20</td><td>57.54</td></tr><tr><td>cba</td><td>84.71</td><td><strong>42.27</strong></td><td>72.81</td><td>48.45</td><td>79.80</td><td>66.67</td><td><strong>44.04</strong></td><td>91.27</td></tr><tr><td>cba2</td><td>84.90</td><td>26.80</td><td>68.42</td><td>36.08</td><td>77.78</td><td>56.86</td><td>33.03</td><td>87.70</td></tr><tr><td>cmar</td><td>75.67</td><td>39.18</td><td>38.60</td><td>26.80</td><td>35.69</td><td>56.86</td><td>31.19</td><td>51.59</td></tr><tr><td>cpar</td><td>84.71</td><td>29.90</td><td>78.95</td><td>36.00</td><td>8.08</td><td>70.00</td><td>28.44</td><td><strong>91.67</strong></td></tr><tr><td>L3</td><td>84.59</td><td>32.99</td><td>78.07</td><td>56.70</td><td><strong>80.47</strong></td><td><strong>70.59</strong></td><td>41.28</td><td>91.27</td></tr><tr><td>mac</td><td>77.96</td><td>40.21</td><td>79.82</td><td><strong>59.79</strong></td><td>73.74</td><td><strong>70.59</strong></td><td>42.20</td><td>91.27</td></tr><tr><td>carrt</td><td>84.39</td><td>40.21</td><td>78.94</td><td><strong>59.79</strong></td><td>76.09</td><td>66.67</td><td>39.45</td><td>90.87</td></tr><tr><td rowspan="12">45</td><td>acac</td><td>34.97</td><td>36.08</td><td>50.88</td><td>43.30</td><td>50.84</td><td><strong>70.59</strong></td><td>42.20</td><td>48.41</td></tr><tr><td>accf</td><td>84.39</td><td>34.02</td><td><strong>79.82</strong></td><td>58.76</td><td><strong>80.13</strong></td><td><strong>70.59</strong></td><td>41.28</td><td>90.48</td></tr><tr><td>acn</td><td>84.39</td><td>36.08</td><td><strong>79.82</strong></td><td><strong>59.79</strong></td><td>79.80</td><td><strong>70.59</strong></td><td>42.20</td><td>90.48</td></tr><tr><td>adt</td><td><strong>86.50</strong></td><td>40.21</td><td>48.25</td><td>38.14</td><td>50.84</td><td><strong>70.59</strong></td><td>42.20</td><td>57.54</td></tr><tr><td>cba</td><td>84.71</td><td><strong>42.27</strong></td><td>72.81</td><td>48.45</td><td>79.80</td><td>66.67</td><td><strong>44.04</strong></td><td>91.27</td></tr><tr><td>cba2</td><td>84.90</td><td>26.80</td><td>68.42</td><td>36.08</td><td>77.78</td><td>56.86</td><td>33.03</td><td>87.70</td></tr><tr><td>cmar</td><td>68.09</td><td>36.08</td><td>58.77</td><td>44.33</td><td>54.88</td><td>56.86</td><td>41.28</td><td>51.59</td></tr><tr><td>cpar</td><td>84.71</td><td>29.90</td><td>78.95</td><td>36.00</td><td>8.08</td><td>70.00</td><td>28.44</td><td><strong>91.67</strong></td></tr><tr><td>L3</td><td>84.59</td><td>32.99</td><td><strong>79.82</strong></td><td>56.70</td><td><strong>80.13</strong></td><td><strong>70.59</strong></td><td>41.28</td><td>91.27</td></tr><tr><td>mac</td><td>77.96</td><td>41.24</td><td><strong>79.82</strong></td><td><strong>59.79</strong></td><td>73.74</td><td><strong>70.59</strong></td><td><strong>44.04</strong></td><td>91.27</td></tr><tr><td>carrt</td><td>84.39</td><td>40.21</td><td>78.95</td><td><strong>59.79</strong></td><td>76.09</td><td>68.63</td><td>44.04</td><td>90.87</td></tr><tr><td rowspan="12">60</td><td>acac</td><td>63.69</td><td>7.22</td><td>59.65</td><td>27.84</td><td>62.29</td><td><strong>70.59</strong></td><td>20.18</td><td>61.11</td></tr><tr><td>accf</td><td>84.39</td><td>36.08</td><td><strong>79.82</strong></td><td>58.76</td><td><strong>80.13</strong></td><td><strong>70.59</strong></td><td><strong>42.20</strong></td><td>90.48</td></tr><tr><td>acn</td><td>84.39</td><td><strong>40.21</strong></td><td><strong>79.82</strong></td><td><strong>59.79</strong></td><td>79.80</td><td><strong>70.59</strong></td><td><strong>42.20</strong></td><td>90.48</td></tr><tr><td>adt</td><td><strong>86.18</strong></td><td><strong>40.21</strong></td><td>48.25</td><td>0.00</td><td>0.00</td><td><strong>70.59</strong></td><td><strong>42.20</strong></td><td>57.54</td></tr><tr><td>cba</td><td>84.71</td><td><strong>40.21</strong></td><td>72.81</td><td>48.45</td><td>79.80</td><td>66.67</td><td><strong>42.20</strong></td><td>91.27</td></tr><tr><td>cba2</td><td>84.90</td><td>29.90</td><td>68.42</td><td>36.08</td><td>77.78</td><td>56.86</td><td>33.03</td><td>87.70</td></tr><tr><td>cmar</td><td>71.34</td><td>0.00</td><td>52.63</td><td>49.48</td><td>54.88</td><td>52.94</td><td>25.69</td><td>62.30</td></tr><tr><td>cpar</td><td>84.71</td><td>29.90</td><td>78.95</td><td>36.00</td><td>8.08</td><td>70.00</td><td>28.44</td><td><strong>91.67</strong></td></tr><tr><td>L3</td><td>84.59</td><td>21.65</td><td><strong>79.82</strong></td><td>52.58</td><td>79.80</td><td><strong>70.59</strong></td><td>35.78</td><td>91.27</td></tr><tr><td>mac</td><td>84.71</td><td><strong>40.21</strong></td><td><strong>79.82</strong></td><td><strong>59.79</strong></td><td>73.74</td><td><strong>70.59</strong></td><td><strong>42.20</strong></td><td>91.27</td></tr><tr><td>carrt</td><td>84.39</td><td>19.58</td><td>79.84</td><td>47.42</td><td>78.11</td><td>68.63</td><td>33.95</td><td>91.66</td></tr></tbody></table>

### 4.3. Methodology

This section overviews the methodology used to evaluate the algorithms presented in Section 4.2 using datasets presented in Section 4.1. The methodology is applied in the same manner for each of the training datasets. Usually, the associative classification algorithms have two common parameters: the minimum confidence (minconf) and minimum support (minsup). In our experiments, we set the minsup very low (with the aim of obtaining enough rules, including those leading to poorly represented classes) and fix three values for minconf (30, 45, and 60%). Then for each dataset and parameter value, each algorithm is run and a classifier is constructed using the rules mined by that algorithm. These classifiers are then applied to evaluate each testing dataset. The results of the classification are evaluated using accuracy and precision. In the following section, we present the results obtained on the initial data, and the both improvements achieved after the ontology-based feature extraction and SMOTEN sampling processes, as presented in Section 3.2.

### 4.4. Classification performance

Classification performance is evaluated using accuracy and precision. An overview of the results for the algorithms with three different minimum confidence values (30, 45 and 60) across our eight datasets is provided in the tables below. Tables 3 and 4 present the accuracy and precision results on the original or initial data, respectively. Note that the best result for a given minimum confidence is marked in bold, while the best result across all minimum confidence values is marked in bold and italic. For accuracy, very good results are observed on some datasets such as “Comf”, with an average accuracy of over 80%, except for the ACAC algorithm, which achieves accuracy of 10.70, 34.97, and 63.69 for minimum confidence values of 30%, 45%, and 60%, respectively. However, we do not achieve very good results

on certain datasets like “Heat_1”, which has an average accuracy of less than 50%. However, the accuracy results cannot be fully trusted because the dataset’s condition is unbalanced. Regarding precision (Table 4), the results are not very good, except for the ADT algorithm, which achieves good precision on the “Comf” dataset. Tables 5 and 6 show respectively the accuracy and precision results after adding features derived from textual descriptions using the Brick ontology. We observe that adding features derived from textual descriptions using the Brick ontology increases the accuracy of our classifiers, in general. Regarding the precision, there is also a slight improvement, but results on precision remain very poor without oversampling. So for correct classification, datasets are over-sampled to generate synthetic samples of minority classes. Tables 7 and 8 show respectively the accuracy and precision results after both ontology-based feature engineering and oversampling. We observed some good improvement in the precision and a small degradation on accuracy. In the next section, we present the statistical tests of the results.

## 5. Statistical significance test of the results

We used the staTDS (Statistical Tests for Data Science) [35] library to compare the performance of different algorithms and to evaluate the impact of various data augmentation strategies. Table 9 shows the p-values of the Wilcoxon signed-ranks test for the comparison of the accuracy results considering the different values of the parameter minconf used in our experiments, where ID(Initial Data), IDO(Initial Data Ontology) and IDOS (Initial Data Ontology over-Sampling) respectively correspond to the test results on the original data, test results after adding features derived from textual descriptions using the Brick ontology and test results after oversampling. 30 vs 45 stands for the alternative hypothesis that the accuracy for minconf = 30 is significant different for the accuracy for minconf = 45 (equivalently for the other columns of the table). With the parameter alpha = 0.05,
<table><caption>**Table 4**
Precision results on initial data.</caption><thead><tr><th rowspan="2">Minconf</th><th rowspan="2">Algorithm</th><th colspan="8">Precision</th></tr><tr><th>Comf</th><th>Heat_1</th><th>Heat_2</th><th>Multi_1</th><th>Multi_2</th><th>Multi_3</th><th>Multi_4</th><th>Vent</th></tr></thead><tbody><tr><td rowspan="10">30</td><td>acac</td><td>1.72</td><td>3.84</td><td>17.31</td><td>69.50</td><td>7.60</td><td>16.86</td><td>3.01</td><td>5.28</td></tr><tr><td>accf</td><td>18.75</td><td>3.37</td><td>44.57</td><td>15.68</td><td>44.48</td><td>17.65</td><td>3.15</td><td>30.37</td></tr><tr><td>acn</td><td>18.75</td><td>4.45</td><td>44.57</td><td>16.61</td><td>43.66</td><td>17.65</td><td>3.01</td><td>30.37</td></tr><tr><td>adt</td><td><strong>70.24</strong></td><td>2.68</td><td>8.04</td><td>47.70</td><td>8.47</td><td>17.65</td><td>3.01</td><td>9.59</td></tr><tr><td>cba</td><td>29.92</td><td>4.12</td><td><strong>49.88</strong></td><td>18.92</td><td>43.66</td><td>23.80</td><td>4.58</td><td>30.67</td></tr><tr><td>cba2</td><td>41.07</td><td>4.22</td><td>44.07</td><td>18.34</td><td><strong>46.14</strong></td><td>24.61</td><td>5.39</td><td>30.29</td></tr><tr><td>cmar</td><td>17.04</td><td>2.67</td><td>11.79</td><td><strong>79.80</strong></td><td>11.05</td><td><strong>28.39</strong></td><td>6.08</td><td>27.08</td></tr><tr><td>cpar</td><td>29.92</td><td><strong>5.34</strong></td><td>45.00</td><td>20.00</td><td>1.99</td><td>22.00</td><td><strong>6.18</strong></td><td><strong>30.68</strong></td></tr><tr><td>L3</td><td>18.81</td><td>4.38</td><td>44.51</td><td>16.63</td><td>44.58</td><td>17.65</td><td>3.38</td><td>30.58</td></tr><tr><td>mac</td><td>17.76</td><td>2.68</td><td>44.57</td><td>16.61</td><td>43.42</td><td>17.65</td><td>3.01</td><td>30.58</td></tr><tr><td>carrt</td><td>25.38</td><td>2.96</td><td>44.73</td><td>16.61</td><td>45.52</td><td>17.35</td><td>6.09</td><td><strong>30.68</strong></td></tr><tr><td rowspan="10">45</td><td>acac</td><td>4.52</td><td>4.70</td><td>12.74</td><td>17.53</td><td>8.47</td><td>17.65</td><td>3.35</td><td>8.92</td></tr><tr><td>accf</td><td>18.75</td><td>3.37</td><td>44.57</td><td>16.62</td><td>44.48</td><td>17.65</td><td>3.00</td><td>30.37</td></tr><tr><td>acn</td><td>18.75</td><td>4.45</td><td>44.57</td><td>16.61</td><td>43.66</td><td>17.65</td><td>3.01</td><td>30.37</td></tr><tr><td>adt</td><td><strong>70.85</strong></td><td>2.68</td><td>8.04</td><td>4.77</td><td>8.47</td><td>17.65</td><td>3.01</td><td>9.59</td></tr><tr><td>cba</td><td>29.92</td><td>4.12</td><td><strong>49.88</strong></td><td>18.92</td><td>43.66</td><td>23.80</td><td>4.58</td><td>30.67</td></tr><tr><td>cba2</td><td>41.07</td><td>4.22</td><td>44.07</td><td>18.85</td><td><strong>46.14</strong></td><td>25.44</td><td>5.39</td><td>30.29</td></tr><tr><td>cmar</td><td>16.15</td><td>4.88</td><td>17.43</td><td>19.44</td><td>25.61</td><td>21.61</td><td>3.35</td><td>27.08</td></tr><tr><td>cpar</td><td>29.92</td><td><strong>5.34</strong></td><td>45.00</td><td><strong>20.00</strong></td><td>1.99</td><td>22.00</td><td>6.18</td><td><strong>30.68</strong></td></tr><tr><td>L3</td><td>18.81</td><td>4.42</td><td>44.57</td><td>16.63</td><td>44.53</td><td>17.65</td><td>3.38</td><td>30.58</td></tr><tr><td>mac</td><td>17.76</td><td>3.84</td><td>44.57</td><td>16.61</td><td>43.42</td><td>17.65</td><td>4.75</td><td>30.58</td></tr><tr><td>carrt</td><td>25.38</td><td>3.09</td><td>44.57</td><td>16.61</td><td>45.52</td><td><strong>30.21</strong></td><td><strong>6.19</strong></td><td>30.66</td></tr><tr><td rowspan="10">60</td><td>acac</td><td>14.61</td><td>3.89</td><td>18.75</td><td>17.29</td><td>26.24</td><td>17.65</td><td>5.07</td><td>25.11</td></tr><tr><td>accf</td><td>18.75</td><td>3.47</td><td>44.57</td><td>16.62</td><td>44.48</td><td>17.65</td><td>3.01</td><td>30.37</td></tr><tr><td>acn</td><td>18.75</td><td>2.68</td><td>44.57</td><td>16.61</td><td>43.66</td><td>17.65</td><td>3.01</td><td>30.37</td></tr><tr><td>adt</td><td><strong>69.22</strong></td><td>2.68</td><td>8.04</td><td>0.00</td><td>0.00</td><td>17.65</td><td>3.01</td><td>9.59</td></tr><tr><td>cba</td><td>29.92</td><td>2.68</td><td><strong>49.88</strong></td><td>18.92</td><td>43.66</td><td>23.80</td><td>3.01</td><td>30.67</td></tr><tr><td>cba2</td><td>41.07</td><td>3.70</td><td>44.07</td><td>18.34</td><td><strong>46.14</strong></td><td>24.61</td><td>5.29</td><td>30.29</td></tr><tr><td>cmar</td><td>16.44</td><td>0.00</td><td>19.44</td><td>16.45</td><td>24.74</td><td>21.21</td><td>3.77</td><td>26.85</td></tr><tr><td>cpar</td><td>29.92</td><td>5.34</td><td>45.00</td><td><strong>20.00</strong></td><td>1.99</td><td>22.00</td><td>6.18</td><td><strong>30.68</strong></td></tr><tr><td>L3</td><td>18.81</td><td>3.73</td><td>44.57</td><td>16.35</td><td>44.51</td><td>17.65</td><td>3.98</td><td>30.58</td></tr><tr><td>mac</td><td>29.92</td><td>2.68</td><td>44.57</td><td>16.61</td><td>43.42</td><td>17.65</td><td>3.01</td><td>30.58</td></tr><tr><td>carrt</td><td>25.38</td><td><strong>6.27</strong></td><td>44.73</td><td>18.93</td><td>44.22</td><td><strong>30.21</strong></td><td><strong>9.02</strong></td><td>30.66</td></tr></tbody></table>

<table><caption>**Table 5**
Accuracy results after ontology-based feature extractions.</caption><thead><tr><th rowspan="2">Minconf</th><th rowspan="2">Algorithm</th><th colspan="8">Accuracy</th></tr><tr><th>Comf</th><th>Heat_1</th><th>Heat_2</th><th>Multi_1</th><th>Multi_2</th><th>Multi_3</th><th>Multi_4</th><th>Vent</th></tr></thead><tbody><tr><td rowspan="10">30</td><td>acac</td><td>3.82</td><td>50.00</td><td>51.75</td><td>20.62</td><td>18.97</td><td>56.86</td><td>42.20</td><td>31.35</td></tr><tr><td>accf</td><td><strong>84.71</strong></td><td>58.82</td><td>79.82</td><td><strong>59.79</strong></td><td>68.01</td><td>70.59</td><td>58.72</td><td>90.87</td></tr><tr><td>acn</td><td>84.39</td><td>55.88</td><td>79.82</td><td>58.76</td><td>65.90</td><td>70.59</td><td>54.13</td><td>90.87</td></tr><tr><td>adt</td><td>46.82</td><td>47.06</td><td>48.25</td><td>38.14</td><td>44.06</td><td>70.59</td><td>42.20</td><td>57.54</td></tr><tr><td>cba</td><td>84.08</td><td>63.24</td><td>85.09</td><td>52.58</td><td>73.37</td><td>66.67</td><td>67.89</td><td>90.87</td></tr><tr><td>cba2</td><td>78.66</td><td>55.15</td><td>85.09</td><td>43.30</td><td>78.93</td><td>56.86</td><td><strong>71.56</strong></td><td>84.92</td></tr><tr><td>cmar</td><td>76.75</td><td>50.00</td><td>40.35</td><td>31.96</td><td>23.75</td><td>43.14</td><td>33.94</td><td>59.52</td></tr><tr><td>cpar</td><td><strong>84.71</strong></td><td><strong>65.44</strong></td><td><strong>88.60</strong></td><td>38.00</td><td>26.00</td><td>45.00</td><td>66.06</td><td><strong>91.67</strong></td></tr><tr><td>L3</td><td>84.08</td><td>65.44</td><td>85.96</td><td><strong>59.79</strong></td><td>75.10</td><td><strong>74.51</strong></td><td>67.89</td><td>91.27</td></tr><tr><td>mac</td><td><strong>84.71</strong></td><td>59.56</td><td>85.09</td><td>56.70</td><td>65.52</td><td>70.59</td><td>66.06</td><td><strong>91.67</strong></td></tr><tr><td>carrt</td><td>84.39</td><td>63.23</td><td>82.45</td><td>51.54</td><td><strong>80.34</strong></td><td>68.62</td><td>67.89</td><td>91.66</td></tr><tr><td rowspan="10">45</td><td>acac</td><td>36.62</td><td>39.71</td><td>59.65</td><td>36.08</td><td>46.36</td><td>70.59</td><td>42.20</td><td>48.41</td></tr><tr><td>accf</td><td><strong>84.71</strong></td><td>58.82</td><td>79.82</td><td><strong>59.79</strong></td><td>68.01</td><td>70.59</td><td>58.72</td><td>90.87</td></tr><tr><td>acn</td><td>84.39</td><td>55.88</td><td>79.82</td><td>58.76</td><td>65.90</td><td>70.59</td><td>54.13</td><td>90.87</td></tr><tr><td>adt</td><td>46.82</td><td>47.06</td><td>48.25</td><td>0.00</td><td>44.06</td><td>70.59</td><td>42.20</td><td>57.54</td></tr><tr><td>cba</td><td>84.08</td><td>63.24</td><td>85.09</td><td>52.58</td><td>73.37</td><td>66.67</td><td>67.89</td><td>90.87</td></tr><tr><td>cba2</td><td>78.66</td><td>55.15</td><td>85.09</td><td>43.30</td><td><strong>78.93</strong></td><td>56.86</td><td><strong>71.56</strong></td><td>84.92</td></tr><tr><td>cmar</td><td>71.34</td><td>58.09</td><td>65.79</td><td>27.84</td><td>25.86</td><td>52.94</td><td>48.62</td><td>59.52</td></tr><tr><td>cpar</td><td><strong>84.71</strong></td><td><strong>65.44</strong></td><td><strong>88.60</strong></td><td>38.00</td><td>26.00</td><td>45.00</td><td>66.06</td><td><strong>91.67</strong></td></tr><tr><td>L3</td><td>84.08</td><td>64.71</td><td>86.84</td><td><strong>59.79</strong></td><td>75.10</td><td><strong>74.51</strong></td><td>66.97</td><td>91.27</td></tr><tr><td>mac</td><td><strong>84.71</strong></td><td>59.56</td><td>85.09</td><td>56.70</td><td>65.52</td><td>70.59</td><td>66.06</td><td><strong>91.67</strong></td></tr><tr><td>carrt</td><td>84.39</td><td>64.70</td><td>84.21</td><td>50.51</td><td>76.09</td><td>68.62</td><td>50.45</td><td>91.66</td></tr><tr><td rowspan="10">60</td><td>acac</td><td>66.88</td><td>47.06</td><td>75.44</td><td>34.02</td><td>50.77</td><td>59.79</td><td>51.38</td><td>57.54</td></tr><tr><td>accf</td><td><strong>84.71</strong></td><td>58.82</td><td>79.82</td><td><strong>59.79</strong></td><td>68.01</td><td>59.79</td><td>58.72</td><td>90.87</td></tr><tr><td>acn</td><td>84.39</td><td>55.88</td><td>79.82</td><td>58.76</td><td>65.90</td><td>59.79</td><td>54.13</td><td>90.87</td></tr><tr><td>adt</td><td>46.82</td><td>47.06</td><td>48.25</td><td>0.00</td><td>40.42</td><td>59.79</td><td>42.20</td><td>57.54</td></tr><tr><td>cba</td><td>84.08</td><td>63.24</td><td>85.09</td><td>52.58</td><td>72.22</td><td>66.67</td><td>59.63</td><td>90.87</td></tr><tr><td>cba2</td><td>78.66</td><td>55.15</td><td>85.09</td><td>43.30</td><td><strong>78.93</strong></td><td>56.86</td><td><strong>70.64</strong></td><td>84.92</td></tr><tr><td>cmar</td><td>69.75</td><td>38.24</td><td>59.65</td><td>30.93</td><td>24.14</td><td>50.98</td><td>51.38</td><td>60.32</td></tr><tr><td>cpar</td><td><strong>84.71</strong></td><td>65.44</td><td><strong>88.60</strong></td><td>38.00</td><td>26.00</td><td>55.00</td><td>66.06</td><td><strong>91.67</strong></td></tr><tr><td>L3</td><td>84.08</td><td>61.76</td><td>86.84</td><td>56.70</td><td>71.26</td><td><strong>70.59</strong></td><td>66.97</td><td>91.27</td></tr><tr><td>mac</td><td><strong>84.71</strong></td><td>59.56</td><td>85.09</td><td>56.70</td><td>60.92</td><td>59.79</td><td>66.06</td><td><strong>91.67</strong></td></tr><tr><td>carrt</td><td>84.39</td><td><strong>66.91</strong></td><td>86.85</td><td>51.54</td><td>78.11</td><td>68.62</td><td>50.45</td><td>91.66</td></tr></tbody></table>
<table><caption>**Table 6**
Precision results after ontology-based features extractions.</caption><thead><tr><th rowspan="2">Minconf</th><th rowspan="2">Algorithm</th><th colspan="8">Precision</th></tr><tr><th>Comf</th><th>Heat_1</th><th>Heat_2</th><th>Multi_1</th><th>Multi_2</th><th>Multi_3</th><th>Multi_4</th><th>Vent</th></tr></thead><tbody><tr><td rowspan="10">30</td><td>acac</td><td>0.80</td><td>5.93</td><td>28.69</td><td>5.43</td><td>6.52</td><td>16.86</td><td>3.19</td><td>5.37</td></tr><tr><td>accf</td><td>21.17</td><td>13.27</td><td>44.73</td><td>16.70</td><td>63.52</td><td>17.65</td><td>11.68</td><td>30.47</td></tr><tr><td>acn</td><td>21.11</td><td>7.34</td><td>44.57</td><td>16.71</td><td>53.95</td><td>17.65</td><td>7.10</td><td>30.56</td></tr><tr><td>adt</td><td>5.85</td><td>3.14</td><td>8.04</td><td>4.77</td><td>73.03</td><td>17.65</td><td>3.01</td><td>9.59</td></tr><tr><td>cba</td><td>21.07</td><td>11.25</td><td>61.60</td><td>21.62</td><td>69.44</td><td>22.93</td><td>13.00</td><td>30.65</td></tr><tr><td>cba2</td><td><strong>27.59</strong></td><td>17.73</td><td>56.50</td><td>23.97</td><td><strong>76.15</strong></td><td>23.56</td><td><strong>27.22</strong></td><td>29.75</td></tr><tr><td>cmar</td><td>19.43</td><td>7.93</td><td>29.56</td><td>9.31</td><td>28.69</td><td>23.07</td><td>8.57</td><td>27.08</td></tr><tr><td>cpar</td><td>21.17</td><td>15.83</td><td>59.64</td><td>10.41</td><td>30.19</td><td>30.07</td><td>12.31</td><td><strong>30.68</strong></td></tr><tr><td>L3</td><td>21.07</td><td><strong>19.32</strong></td><td><strong>64.84</strong></td><td><strong>44.36</strong></td><td>71.90</td><td><strong>43.37</strong></td><td>25.05</td><td>30.58</td></tr><tr><td>mac</td><td>21.17</td><td>7.90</td><td>57.26</td><td>18.83</td><td>62.24</td><td>17.65</td><td>13.32</td><td><strong>30.68</strong></td></tr><tr><td>carrt</td><td>25.38</td><td><strong>19.32</strong></td><td>55.10</td><td>26.05</td><td>45.52</td><td>17.50</td><td>11.96</td><td><strong>30.68</strong></td></tr><tr><td rowspan="10">45</td><td>acac</td><td>5.21</td><td>3.08</td><td>25.39</td><td>8.94</td><td>43.07</td><td>17.65</td><td>3.01</td><td>8.92</td></tr><tr><td>accf</td><td>21.17</td><td>13.27</td><td>44.73</td><td>16.70</td><td>63.52</td><td>17.65</td><td>11.68</td><td>30.47</td></tr><tr><td>acn</td><td>21.11</td><td>7.34</td><td>44.57</td><td>16.71</td><td>53.95</td><td>17.65</td><td>7.10</td><td>30.56</td></tr><tr><td>adt</td><td>5.85</td><td>3.14</td><td>8.04</td><td>0.00</td><td>73.49</td><td>17.65</td><td>3.01</td><td>9.59</td></tr><tr><td>cba</td><td>21.07</td><td>11.25</td><td><strong>61.60</strong></td><td>21.62</td><td>69.44</td><td>22.93</td><td>13.00</td><td>30.65</td></tr><tr><td>cba2</td><td><strong>27.59</strong></td><td>17.73</td><td>56.50</td><td>23.97</td><td><strong>76.15</strong></td><td>23.56</td><td><strong>27.22</strong></td><td>29.75</td></tr><tr><td>cmar</td><td>18.67</td><td>7.33</td><td>46.62</td><td>15.52</td><td>26.30</td><td>17.31</td><td>8.49</td><td>27.08</td></tr><tr><td>cpar</td><td>21.17</td><td>15.83</td><td>59.64</td><td>16.41</td><td>29.19</td><td>30.07</td><td>12.31</td><td><strong>30.68</strong></td></tr><tr><td>L3</td><td>21.07</td><td>18.46</td><td>59.34</td><td><strong>32.65</strong></td><td>71.90</td><td><strong>43.37</strong></td><td>17.67</td><td>30.58</td></tr><tr><td>mac</td><td>21.17</td><td>7.90</td><td>57.26</td><td>18.83</td><td>62.24</td><td>17.65</td><td>13.32</td><td><strong>30.68</strong></td></tr><tr><td>carrt</td><td>25.38</td><td><strong>24.94</strong></td><td>56.14</td><td>23.77</td><td>45.52</td><td>17.50</td><td>17.74</td><td><strong>30.68</strong></td></tr><tr><td rowspan="10">60</td><td>acac</td><td>17.61</td><td>5.81</td><td>36.21</td><td>22.17</td><td>44.22</td><td>17.65</td><td>6.62</td><td>9.59</td></tr><tr><td>accf</td><td>21.17</td><td>13.20</td><td>44.73</td><td>16.43</td><td>63.27</td><td>17.65</td><td>11.68</td><td>30.47</td></tr><tr><td>acn</td><td>21.11</td><td>7.54</td><td>44.57</td><td>16.71</td><td>53.95</td><td>17.65</td><td>7.07</td><td>30.56</td></tr><tr><td>adt</td><td>5.85</td><td>3.14</td><td>8.04</td><td>0.00</td><td><strong>76.58</strong></td><td>17.65</td><td>3.01</td><td>9.59</td></tr><tr><td>cba</td><td>21.07</td><td>11.25</td><td><strong>61.60</strong></td><td>21.62</td><td>61.54</td><td>22.93</td><td>11.47</td><td>30.65</td></tr><tr><td>cba2</td><td><strong>27.59</strong></td><td><strong>17.73</strong></td><td>56.50</td><td>23.97</td><td>76.15</td><td>23.56</td><td><strong>25.79</strong></td><td>29.75</td></tr><tr><td>cmar</td><td>19.30</td><td>5.96</td><td>48.35</td><td>5.95</td><td>35.91</td><td>22.03</td><td>9.25</td><td>26.71</td></tr><tr><td>cpar</td><td>21.17</td><td>15.83</td><td>59.64</td><td>6.41</td><td>35.19</td><td>30.07</td><td>12.31</td><td><strong>30.68</strong></td></tr><tr><td>L3</td><td>21.07</td><td>14.19</td><td>59.34</td><td><strong>30.52</strong></td><td>72.04</td><td><strong>42.86</strong></td><td>14.32</td><td>30.58</td></tr><tr><td>mac</td><td>21.17</td><td>8.34</td><td>57.26</td><td>18.83</td><td>51.20</td><td>17.65</td><td>13.39</td><td><strong>30.68</strong></td></tr><tr><td>carrt</td><td>25.38</td><td>17.42</td><td>58.30</td><td>18.05</td><td>44.22</td><td>17.50</td><td>17.03</td><td><strong>30.68</strong></td></tr></tbody></table>

<table><caption>**Table 7**
Accuracy results after ontology-based features extractions and resampling.</caption><thead><tr><th rowspan="2">Minconf</th><th rowspan="2">Algorithm</th><th colspan="8">Accuracy</th></tr><tr><th>Comf</th><th>Heat_1</th><th>Heat_2</th><th>Multi_1</th><th>Multi_2</th><th>Multi_3</th><th>Multi_4</th><th>Vent</th></tr></thead><tbody><tr><td rowspan="10">30</td><td>acac</td><td>12.45</td><td>31.23</td><td>13.45</td><td>10.11</td><td>49.08</td><td>23.61</td><td>19.75</td><td>18.95</td></tr><tr><td>accf</td><td>62.64</td><td>56.78</td><td>83.63</td><td>63.48</td><td><strong>79.66</strong></td><td>56.94</td><td>61.73</td><td>10.97</td></tr><tr><td>acn</td><td>59.20</td><td>39.12</td><td>75.44</td><td>61.24</td><td><strong>79.66</strong></td><td>52.78</td><td>41.98</td><td>10.97</td></tr><tr><td>adt</td><td>45.21</td><td>60.57</td><td>49.12</td><td>43.26</td><td>50.92</td><td>50.00</td><td>52.26</td><td>0.00</td></tr><tr><td>cba</td><td>71.84</td><td>56.15</td><td>86.55</td><td><strong>73.03</strong></td><td>9.00</td><td>68.06</td><td>56.79</td><td>0.25</td></tr><tr><td>cba2</td><td><strong>81.03</strong></td><td><strong>72.56</strong></td><td>83.63</td><td>71.91</td><td>9.39</td><td>68.06</td><td><strong>77.37</strong></td><td>2.74</td></tr><tr><td>cmar</td><td>41.57</td><td>18.61</td><td>52.05</td><td>24.16</td><td>52.44</td><td>37.50</td><td>36.21</td><td>0.00</td></tr><tr><td>cpar</td><td>71.84</td><td>68.45</td><td><strong>87.13</strong></td><td>3.37</td><td>8.24</td><td>69.44</td><td>65.43</td><td>5.99</td></tr><tr><td>L3</td><td>74.14</td><td>69.40</td><td>85.38</td><td>66.29</td><td>13.41</td><td><strong>70.83</strong></td><td>74.07</td><td>10.97</td></tr><tr><td>mac</td><td>65.13</td><td>43.22</td><td>82.46</td><td>60.11</td><td>73.61</td><td><strong>70.83</strong></td><td>49.38</td><td>10.97</td></tr><tr><td>carrt</td><td>74.52</td><td>56.15</td><td>84.21</td><td>71.34</td><td>73.56</td><td>62.50</td><td>70.78</td><td><strong>78.05</strong></td></tr><tr><td rowspan="10">45</td><td>acac</td><td>44.06</td><td>35.33</td><td>40.94</td><td>37.08</td><td>50.92</td><td>40.28</td><td>25.51</td><td>10.97</td></tr><tr><td>accf</td><td>62.64</td><td>56.78</td><td>83.63</td><td>63.48</td><td><strong>79.66</strong></td><td>56.94</td><td>56.79</td><td>10.97</td></tr><tr><td>acn</td><td>59.20</td><td>38.80</td><td>75.44</td><td>61.24</td><td><strong>79.66</strong></td><td>52.78</td><td>34.57</td><td>10.97</td></tr><tr><td>adt</td><td>45.21</td><td>60.57</td><td>49.12</td><td>43.26</td><td>50.92</td><td>50.00</td><td>52.26</td><td>0.00</td></tr><tr><td>cba</td><td>71.84</td><td>47.32</td><td>86.55</td><td><strong>73.03</strong></td><td>9.00</td><td>68.06</td><td>44.86</td><td>0.25</td></tr><tr><td>cba2</td><td><strong>81.03</strong></td><td>72.56</td><td>83.63</td><td>71.91</td><td>9.39</td><td>68.06</td><td><strong>77.37</strong></td><td>2.74</td></tr><tr><td>cmar</td><td>54.98</td><td>34.07</td><td>58.48</td><td>24.16</td><td>61.01</td><td>33.33</td><td>28.81</td><td>0.00</td></tr><tr><td>cpar</td><td>71.84</td><td><strong>68.45</strong></td><td><strong>87.13</strong></td><td>3.37</td><td>8.24</td><td>69.44</td><td>65.43</td><td>5.99</td></tr><tr><td>L3</td><td>74.14</td><td>66.56</td><td>85.38</td><td>66.29</td><td>13.41</td><td><strong>70.83</strong></td><td>66.26</td><td>10.97</td></tr><tr><td>mac</td><td>65.13</td><td>47.32</td><td>82.46</td><td>60.11</td><td>73.61</td><td><strong>70.83</strong></td><td>37.45</td><td>10.97</td></tr><tr><td>carrt</td><td>75.28</td><td>62.14</td><td>87.13</td><td>63.48</td><td>66.74</td><td>50.00</td><td>60.49</td><td><strong>71.57</strong></td></tr><tr><td rowspan="10">60</td><td>acac</td><td>44.06</td><td>13.56</td><td>48.54</td><td>41.57</td><td>64.03</td><td>52.78</td><td>19.34</td><td>10.97</td></tr><tr><td>accf</td><td>62.64</td><td>42.59</td><td>83.63</td><td>63.48</td><td><strong>79.66</strong></td><td>56.94</td><td>56.38</td><td>10.97</td></tr><tr><td>acn</td><td>59.20</td><td>31.86</td><td>75.44</td><td>61.24</td><td><strong>79.66</strong></td><td>52.78</td><td>28.81</td><td>10.97</td></tr><tr><td>adt</td><td>45.21</td><td>60.25</td><td>49.12</td><td>43.26</td><td>0.00</td><td>0.00</td><td>52.26</td><td>0.00</td></tr><tr><td>cba</td><td>71.84</td><td>34.07</td><td>86.55</td><td><strong>73.03</strong></td><td>9.00</td><td>68.06</td><td>37.86</td><td>0.25</td></tr><tr><td>cba2</td><td><strong>81.03</strong></td><td><strong>72.24</strong></td><td>83.63</td><td>71.91</td><td>9.39</td><td>68.06</td><td><strong>76.95</strong></td><td>2.74</td></tr><tr><td>cmar</td><td>59.77</td><td>13.88</td><td>52.05</td><td>27.53</td><td>57.98</td><td>15.28</td><td>25.51</td><td>0.00</td></tr><tr><td>cpar</td><td>71.84</td><td>68.45</td><td><strong>87.13</strong></td><td>3.37</td><td>8.24</td><td>69.44</td><td>65.43</td><td>5.99</td></tr><tr><td>L3</td><td>74.33</td><td>42.27</td><td>84.21</td><td>66.29</td><td>13.22</td><td>68.06</td><td>60.91</td><td>10.97</td></tr><tr><td>mac</td><td>65.13</td><td>32.81</td><td>82.46</td><td>58.43</td><td>73.61</td><td><strong>70.83</strong></td><td>36.21</td><td>10.97</td></tr><tr><td>carrt</td><td>74.32</td><td>48.89</td><td>79.53</td><td>59.55</td><td>63.60</td><td>54.16</td><td>58.84</td><td><strong>77.55</strong></td></tr></tbody></table>
<table><caption>**Table 8**
Precision results after ontology-based feature extractions and resampling.</caption><thead><tr><th rowspan="2">Minconf</th><th rowspan="2">Algorithm</th><th colspan="8">Precision</th></tr><tr><th>Comf</th><th>Heat_1</th><th>Heat_2</th><th>Multi_1</th><th>Multi_2</th><th>Multi_3</th><th>Multi_4</th><th>Vent</th></tr></thead><tbody><tr><td rowspan="10">30</td><td>acac</td><td>5.52</td><td>13.35</td><td>9.64</td><td>9.94</td><td>7.17</td><td>34.38</td><td>2.47</td><td>8.38</td></tr><tr><td>accf</td><td>44.25</td><td>45.15</td><td>86.02</td><td>63.58</td><td>37.70</td><td>53.12</td><td>53.27</td><td>1.85</td></tr><tr><td>acn</td><td>35.94</td><td>15.57</td><td>77.73</td><td>56.87</td><td>37.68</td><td>52.12</td><td>17.53</td><td>1.96</td></tr><tr><td>adt</td><td><b>83.84</b></td><td><b>80.40</b></td><td>86.20</td><td><b>86.14</b></td><td>7.27</td><td>12.50</td><td><b>83.43</b></td><td>0.00</td></tr><tr><td>cba</td><td>51.11</td><td>35.54</td><td>89.01</td><td>75.42</td><td>2.73</td><td>68.75</td><td>29.59</td><td>0.07</td></tr><tr><td>cba2</td><td>80.46</td><td>77.38</td><td>83.79</td><td>76.05</td><td>19.48</td><td>68.75</td><td>78.34</td><td>0.73</td></tr><tr><td>cmar</td><td>32.22</td><td>6.48</td><td>59.28</td><td>27.82</td><td>25.85</td><td>38.92</td><td>15.81</td><td>0.00</td></tr><tr><td>cpar</td><td>63.52</td><td>69.65</td><td><b>89.82</b></td><td>4.41</td><td>3.19</td><td>76.07</td><td>61.88</td><td>1.40</td></tr><tr><td>L3</td><td>59.01</td><td>70.11</td><td>83.65</td><td>68.28</td><td>2.89</td><td>68.67</td><td>67.08</td><td>1.89</td></tr><tr><td>mac</td><td>39.64</td><td>14.93</td><td>84.85</td><td>59.63</td><td>37.22</td><td>67.75</td><td>23.60</td><td>1.83</td></tr><tr><td>carrt</td><td>65.04</td><td>42.54</td><td>87.36</td><td>73.96</td><td><b>69.89</b></td><td><b>76.84</b></td><td>64.80</td><td><b>55.89</b></td></tr><tr><td rowspan="10">45</td><td>acac</td><td>18.52</td><td>13.05</td><td>21.33</td><td>25.53</td><td>7.29</td><td>11.33</td><td>7.96</td><td>2.53</td></tr><tr><td>accf</td><td>44.25</td><td>45.19</td><td>86.02</td><td>63.60</td><td>37.70</td><td>53.12</td><td>49.40</td><td>1.85</td></tr><tr><td>acn</td><td>35.94</td><td>16.02</td><td>77.73</td><td>56.87</td><td>37.68</td><td>52.12</td><td>14.36</td><td>1.96</td></tr><tr><td>adt</td><td><b>83.84</b></td><td><b>80.40</b></td><td>86.20</td><td><b>86.14</b></td><td>7.27</td><td>12.50</td><td><b>83.43</b></td><td>0.00</td></tr><tr><td>cba</td><td>51.11</td><td>23.92</td><td>89.01</td><td>75.42</td><td>2.73</td><td>68.75</td><td>19.54</td><td>0.07</td></tr><tr><td>cba2</td><td>80.46</td><td>77.38</td><td>83.79</td><td>76.05</td><td>19.48</td><td>68.75</td><td>78.34</td><td>0.73</td></tr><tr><td>cmar</td><td>40.34</td><td>17.08</td><td>61.69</td><td>19.43</td><td>24.26</td><td>28.17</td><td>15.38</td><td>0.00</td></tr><tr><td>cpar</td><td>63.52</td><td>69.65</td><td><b>89.82</b></td><td>4.41</td><td>3.19</td><td><b>76.07</b></td><td>61.88</td><td>1.40</td></tr><tr><td>L3</td><td>59.01</td><td>61.29</td><td>83.65</td><td>68.28</td><td>2.89</td><td>68.67</td><td>64.18</td><td>1.89</td></tr><tr><td>mac</td><td>39.64</td><td>23.61</td><td>84.85</td><td>59.63</td><td>37.22</td><td>67.75</td><td>18.38</td><td>1.83</td></tr><tr><td>carrt</td><td>77.69</td><td>54.15</td><td>86.24</td><td>68.41</td><td><b>62.33</b></td><td>48.65</td><td>53.57</td><td><b>39.94</b></td></tr><tr><td rowspan="10">60</td><td>acac</td><td>23.55</td><td>12.07</td><td>40.77</td><td>34.41</td><td>22.68</td><td>45.08</td><td>5.41</td><td>1.83</td></tr><tr><td>accf</td><td>44.25</td><td>33.62</td><td>86.02</td><td>64.18</td><td>37.70</td><td>53.12</td><td>49.79</td><td>1.85</td></tr><tr><td>acn</td><td>35.94</td><td>13.79</td><td>77.73</td><td>56.87</td><td>37.68</td><td>52.12</td><td>13.00</td><td>1.96</td></tr><tr><td>adt</td><td><b>83.84</b></td><td><b>80.34</b></td><td>86.20</td><td><b>86.14</b></td><td>0.00</td><td>0.00</td><td><b>83.43</b></td><td>0.00</td></tr><tr><td>cba</td><td>51.11</td><td>17.50</td><td>89.01</td><td>75.42</td><td>2.73</td><td>68.75</td><td>15.45</td><td>0.07</td></tr><tr><td>cba2</td><td>80.46</td><td>76.94</td><td>83.79</td><td>76.05</td><td>19.48</td><td>68.75</td><td>78.13</td><td>0.73</td></tr><tr><td>cmar</td><td>36.45</td><td>14.78</td><td>58.49</td><td>24.00</td><td>30.65</td><td>23.13</td><td>14.94</td><td>0.00</td></tr><tr><td>cpar</td><td>63.52</td><td>69.65</td><td><b>89.82</b></td><td>4.41</td><td>3.19</td><td><b>76.07</b></td><td>61.88</td><td>1.40</td></tr><tr><td>L3</td><td>58.75</td><td>44.99</td><td>86.98</td><td>70.07</td><td>2.89</td><td>68.77</td><td>66.16</td><td>1.89</td></tr><tr><td>mac</td><td>39.64</td><td>16.23</td><td>84.85</td><td>61.92</td><td>37.22</td><td>67.75</td><td>14.20</td><td>1.83</td></tr><tr><td>carrt</td><td>75.33</td><td>46.65</td><td>78.73</td><td>62.06</td><td><b>59.50</b></td><td>56.12</td><td>54.43</td><td><b>61.52</b></td></tr></tbody></table>

<table><caption>**Table 9**
* *p*-values of the Wilcoxon signed-ranks test for the comparison of the accuracy results considering the different values of the parameter *minconf* used in our experiments.</caption><thead><tr><th rowspan="2">Algorithm</th><th colspan="3">ID</th><th colspan="3">IDO</th><th colspan="3">IDOS</th></tr><tr><th>30 vs 45</th><th>30 vs 60</th><th>45 vs 60</th><th>30 vs 45</th><th>30 vs 60</th><th>45 vs 60</th><th>30 vs 45</th><th>30 vs 60</th><th>45 vs 60</th></tr></thead><tbody><tr><td>acac</td><td><b>0.0425</b></td><td>0.4609</td><td>0.7353</td><td><b>0.0425</b></td><td><b>0.0156</b></td><td><b>0.028</b></td><td><b>0.0547</b></td><td>0.1484</td><td>0.6002</td></tr><tr><td>accf</td><td>0.3173</td><td>0.1088</td><td>0.1797</td><td>0.5001</td><td>0.5001</td><td>0.5001</td><td>0.3173</td><td>0.1797</td><td>0.1797</td></tr><tr><td>acn</td><td>0.5001</td><td>0.3173</td><td>0.3173</td><td>0.5001</td><td>0.5001</td><td>0.5001</td><td>0.1797</td><td>0.1797</td><td>0.1797</td></tr><tr><td>adt</td><td>0.3173</td><td>0.1088</td><td>0.1088</td><td>0.3173</td><td>0.1797</td><td>0.3173</td><td>0.5001</td><td>0.1088</td><td>0.1088</td></tr><tr><td>cba</td><td>0.5001</td><td>0.1797</td><td>0.1797</td><td>0.5001</td><td>0.1797</td><td>0.1797</td><td>0.1797</td><td>0.1797</td><td>0.1797</td></tr><tr><td>cba2</td><td>0.5001</td><td>0.3173</td><td>0.3173</td><td>0.5001</td><td>0.3173</td><td>0.3173</td><td>0.5001</td><td>0.1797</td><td>0.1797</td></tr><tr><td>cmar</td><td>0.1159</td><td>0.6406</td><td>0.398</td><td>0.1282</td><td>0.5469</td><td>0.4609</td><td>0.173</td><td>0.7532</td><td>0.2367</td></tr><tr><td>cpar</td><td>0.5001</td><td>0.5001</td><td>0.5001</td><td>0.5001</td><td>0.3173</td><td>0.3173</td><td>0.5001</td><td>0.5001</td><td>0.5001</td></tr><tr><td>L3</td><td>0.6547</td><td>0.138</td><td>0.0679</td><td>0.593</td><td><b>0.0464</b></td><td>0.0679</td><td>0.1797</td><td><b>0.0464</b></td><td><b>0.0464</b></td></tr><tr><td>mac</td><td>0.1797</td><td>0.3173</td><td>1.0</td><td>0.5001</td><td>0.3173</td><td>0.3173</td><td>0.6547</td><td>0.1088</td><td>0.1088</td></tr><tr><td>carrt</td><td>0.1088</td><td>0.499</td><td>0.3454</td><td>0.5002</td><td>1.0</td><td>0.0679</td><td>0.1094</td><td><b>0.0078</b></td><td>0.3828</td></tr></tbody></table>

we observed that *acac* with *minconf* = 30 was significantly different to *acac* with *minconf* = 45 on Initial Data (ID). For IDO, *acac* was significantly different for each value of *minconf*, *L3* with *minconf* = 30 was significantly different to *L3* with *minconf* = 60. For IDOS, *acac* with *minconf* = 30 was significantly different to *acac* with *minconf* = 45, *L3* with *minconf* = 60 was significantly different to *L3* with *minconf* = 30 and *minconf* = 45, *carrt* with *minconf* = 30 was significantly different to *carrt* with *minconf* = 60. We also observed that *accf*, *acn*, *adt*, *cba*, *cba2*, *cmar*, *cpar* and *mac* had negligible sensitivity to *minconf*.

Even if the Wilcoxon test indicates whether the differences in accuracy for different values of *minconf* are significant or not, it does not tell us which *minconf* value is the best. Fig. 2 shows the results of the Friedman-Nemenyi test on accuracy across different *minconf* values. The plot compares the performance of various algorithms with *minconf* values of 30, 45, and 60. For the ID datasets, we observe that *acac* performs best with a *minconf* threshold of 45, followed by 60 and 30. For IDO, *acac* performs best when *minconf* is 60, while *L3* performs best

with a *minconf* of 30. For IDOS, both *L3* and *carrt* perform better when *minconf* is set to 30.

Considering these results, our further statistical significance tests for accuracy considers *minconf* = 45 for ID. For IDO, *minconf* = 60 for *acac*, *minconf* = 30 for *L3* and *minconf* = 45 for the other algorithms. For IDOS, *minconf* = 30 for *L3* and *carrt*, and *minconf* = 45 for other algorithms.

Table 10 shows the *p*-values of the Wilcoxon signed-ranks test for the comparison of the precision results considering the different values of the parameter *minconf* used in our experiments. We observed that on ID there is no significantly difference between parameters. For IDO, *acac* with *minconf* = 60 was significantly different to *acac* with *minconf* = 30 and *minconf* = 45, *L3* with *minconf* = 30 was significantly different to *L3* with *minconf* = 60. For IDOS, *acac* with *minconf* = 30 was significantly different to *acac* with *minconf* = 60. We also observed that all algorithms in ID had negligible sensitivity to *minconf*.
<figure><img src="image_3.png" /><figcaption>Fig. 2. Friedman-Nemenyi test on accuracy across different *minconf* values.</figcaption></figure>

<caption>**Table 10**</caption>

p-values of the Wilcoxon signed-ranks test for the comparison of the precision results considering the different values of the parameter *minconf* used in our experiments.

<table><thead><tr><th rowspan="2">Algorithm</th><th colspan="3">ID</th><th colspan="3">IDO</th><th colspan="3">IDOS</th></tr><tr><th>30 vs 45</th><th>30 vs 60</th><th>45 vs 60</th><th>30 vs 45</th><th>30 vs 60</th><th>45 vs 60</th><th>30 vs 45</th><th>30 vs 60</th><th>45 vs 60</th></tr></thead><tbody><tr><td>acac</td><td>0.7422</td><td>0.1953</td><td>0.063</td><td>0.1953</td><td><strong>0.0156</strong></td><td><strong>0.018</strong></td><td>0.6406</td><td><strong>0.0547</strong></td><td>0.1094</td></tr><tr><td>accf</td><td>0.6547</td><td>0.593</td><td>0.1797</td><td>0.5001</td><td>0.1088</td><td>0.1088</td><td>1.0</td><td>0.285</td><td>1.0</td></tr><tr><td>acn</td><td>0.5001</td><td>0.3173</td><td>0.3173</td><td>0.5001</td><td>0.6547</td><td>0.6547</td><td>0.6547</td><td>0.1797</td><td>0.1797</td></tr><tr><td>adt</td><td>0.6547</td><td>0.1088</td><td>0.1088</td><td>0.6547</td><td>0.6547</td><td>0.3173</td><td>0.5001</td><td>0.1088</td><td>0.1088</td></tr><tr><td>cba</td><td>0.5001</td><td>0.1797</td><td>0.1797</td><td>0.5001</td><td>0.1797</td><td>0.1797</td><td>0.1797</td><td>0.1797</td><td>0.1797</td></tr><tr><td>cba2</td><td>0.1797</td><td>0.1797</td><td>0.0679</td><td>0.5001</td><td>0.3173</td><td>0.3173</td><td>0.5001</td><td>0.1797</td><td>0.1797</td></tr><tr><td>cmar</td><td>0.7353</td><td>0.5469</td><td>0.4609</td><td>0.8658</td><td>1.0</td><td>0.4609</td><td>0.8658</td><td>0.8658</td><td>0.7353</td></tr><tr><td>cpar</td><td>0.5001</td><td>0.5001</td><td>0.5001</td><td>0.6547</td><td>0.6547</td><td>0.6547</td><td>0.5001</td><td>0.5001</td><td>0.5001</td></tr><tr><td>L3</td><td>0.593</td><td>0.5002</td><td>0.4652</td><td>0.0679</td><td><strong>0.0464</strong></td><td>0.0796</td><td>0.1797</td><td>0.9165</td><td>0.6002</td></tr><tr><td>mac</td><td>0.1797</td><td>0.3173</td><td>1.0</td><td>0.5001</td><td>1.0</td><td>1.0</td><td>0.6547</td><td>1.0</td><td>0.285</td></tr><tr><td>carrt</td><td>0.5002</td><td>0.1159</td><td>0.138</td><td>0.2733</td><td>0.8927</td><td>0.2249</td><td>0.3828</td><td>0.1484</td><td>0.6406</td></tr></tbody></table>

Even if the Wilcoxon test indicates whether the differences in precision for different values of *minconf* are significant or not, it does not tell us which *minconf* value is the best. Fig. 3 shows the results of the Friedman-Nemenyi test on precision across different *minconf* values. The plot compares the performance of various algorithms with *minconf* values of 30, 45, and 60. For IDO datasets, *acac* performs best when *minconf* = 60, while *L3* is more effective with *minconf* set to 30. For IDOS, *acac* performs best when *minconf* is set to 60%.

Considering these results, our further statistical significance tests for precision considers *minconf* = 45 for each ID dataset. For IDO, *minconf* = 60 for *acac*, *minconf* = 30 *L3* and *minconf* = 45 for the other algorithms. For IDOS, *minconf* = 60 for *acac* and *minconf* = 45 for other algorithms.

Table 11 shows the p-values of the Wilcoxon signed-ranks test for the comparison of the accuracy and precision results considering the different data augmentation process used in our experiments. For the accuracy, we observed that there is no significantly difference between ID and IDO for all algorithms. IDO and IDOS was significantly different for *acac* and *acn*. ID and IDOS was significantly different for *acac*, *acn* and *cmar*. For the precision, ID and IDO was significantly different for

*acac*, *accf*, *acn*, *L3* and *mac*. IDO and IDOS was significantly different for *accf* and *carrt*. ID and IDOS was significantly different for *accf*, *cba2*, *cpar*, *L3* and *carrt*.

Even if the Wilcoxon test indicates whether the differences in accuracy and precision after various data augmentation processes are significant, it does not clearly show whether these processes improve or deteriorate model performance. Fig. 4 presents the results of the Friedman-Nemenyi test conducted on the results to determine the most relevant augmentation strategies. For accuracy, we observe that model performance improves after the feature extraction process based on ontology (IDO), while performance deteriorates after the addition of synthetic data for minority classes. For precision, models perform better after adding both features derived from ontology and oversampling (IDOS).

Considering these results, our further statistical significance tests considers IDO for the accuracy and IDOS for the precision. Fig. 5 shows the Friedman-Nemenyi test for comparison of the accuracy results considering the different algorithms used in our experiments. *L3* was significantly better than 3 algorithms (*adt*, *cmar*, and *acac*). *mac* was significantly better than 2 algorithms (*adt* and *cmar*) and was not
<figure><img src="image_4.png" /><figcaption>Fig. 3. Friedman-Nemenyi test on precision across different *minconf* values.</figcaption></figure>

<caption>**Table 11**</caption>

p-values of the Wilcoxon signed-ranks test for the comparison of the accuracy and precision results considering the different data augmentation process used in our experiments.

<table><thead><tr><th rowspan="2">Algorithm</th><th colspan="3">Accuracy</th><th colspan="3">Precision</th></tr><tr><th>ID vs IDO</th><th>IDO vs IDOS</th><th>ID vs IDOS</th><th>ID vs IDO</th><th>IDO vs IDOS</th><th>ID vs IDOS</th></tr></thead><tbody><tr><td>acac</td><td>0.1282</td><td><strong>0.0391</strong></td><td><strong>0.0581</strong></td><td><strong>0.018</strong></td><td>0.4609</td><td>0.1953</td></tr><tr><td>accf</td><td>0.173</td><td>0.4609</td><td>0.9453</td><td><strong>0.018</strong></td><td><strong>0.0581</strong></td><td><strong>0.0547</strong></td></tr><tr><td>acn</td><td>0.6858</td><td><strong>0.0499</strong></td><td><strong>0.0581</strong></td><td><strong>0.0277</strong></td><td>0.25</td><td>0.1094</td></tr><tr><td>adt</td><td>0.2733</td><td>0.8438</td><td>0.7422</td><td>1.0</td><td>0.1094</td><td>0.1094</td></tr><tr><td>cba</td><td>0.2367</td><td>0.1953</td><td>0.9453</td><td>0.25</td><td>0.5469</td><td>0.3125</td></tr><tr><td>cba2</td><td>0.1282</td><td>0.8438</td><td>0.8438</td><td>0.1953</td><td>0.25</td><td><strong>0.0391</strong></td></tr><tr><td>cmar</td><td>0.8438</td><td>0.1484</td><td><strong>0.0391</strong></td><td>0.398</td><td>0.25</td><td>0.3125</td></tr><tr><td>cpar</td><td>0.173</td><td>0.25</td><td>0.9453</td><td>0.1282</td><td>0.1094</td><td><strong>0.0581</strong></td></tr><tr><td>L3</td><td>0.1282</td><td>0.4609</td><td>0.9453</td><td><strong>0.018</strong></td><td>0.4609</td><td><strong>0.0581</strong></td></tr><tr><td>mac</td><td>0.2367</td><td>0.1953</td><td>0.6406</td><td><strong>0.018</strong></td><td>0.3125</td><td>0.1094</td></tr><tr><td>carrt</td><td>0.3454</td><td>0.8658</td><td>0.6406</td><td>0.2489</td><td><strong>0.0078</strong></td><td><strong>0.0078</strong></td></tr></tbody></table>

<figure><img src="image_5.png" /><figcaption>Fig. 4. Friedman-Nemenyi test conducted on the results to determine the most relevant augmentation strategies.</figcaption></figure>

significantly better or worse than other algorithms. Also, *accf*, *cpar*, *cba*, *carrt*, *acn*, and *cba2* were not significantly better or worse than any algorithms.

Fig. 6 shows the Friedman-Nemenyi test for comparision of the precision results considering the different algorithms used in our experiments. *carrt* and *cba2* were significantly better than *acac* and *cmar*, and was not significantly worse than any algorithms. Also, *adt*, *cpar*, *accf*, *accf*, *L3*, *cba*, *mac* and *acn* were not significantly better or worse than other algorithms. Finally, *acac* and *cmar* were not significantly better or than any algorithms.

## 6. Example of rules

The resulting classifiers are good decision support tools for business experts responsible for the data matching process, due to their clarity. This is why white-box models are particularly appreciated for

their interpretability. Table 12 shows an example of classifier rules produced from the “Multiservice_3” dataset, which includes the history of operations (interventions carried out by the service provider) of a given service provider on the perimeter of its client. These operations are considered as inputs and the objective is to determine the association rules to predict the corresponding category (service code) at the customer or property manager system level. For example, the first and second rules indicate respectively that if the work performed by the service provider is related to a corrective operation or if the processing duration of the performed operation is less than 1 h, then the corresponding customer service code will be “Panne Robinetterie”. We can also deduce from the other rules that if the description provided by the service provider during the intervention contains the keyword “Leak”, then this generally corresponds to the category “Panne chauff ind” in the customer system.
<figure><img src="image_6.png" /><figcaption>Fig. 5. Friedman-Nemenyi test for comparision of the accuracy results considering the different algorithms used in our experiments.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Fig. 6. Friedman-Nemenyi test for comparision of the precision results considering the different algorithms used in our experiments.</figcaption></figure>

<table><caption>**Table 12**
Final classifiers with our method.</caption><thead><tr><th>Rules</th></tr></thead><tbody><tr><td><i>workType</i> = corrective → <i>Class</i> = Panne Robinetterie</td></tr><tr><td><i>processingDuration</i> = [0 - 1] → <i>Class</i> = Panne Robinetterie</td></tr><tr><td><i>Leak</i> = True ∧ <i>criticalLevel</i> = -1 → <i>Class</i> = Panne chauff ind</td></tr><tr><td><i>Leak</i> = True ∧ <i>temporary_repair</i> = True → <i>Class</i> = Panne chauff ind</td></tr><tr><td><i>Leak</i> = True ∧ <i>processingDuration</i> = [1 - 2] → <i>Class</i> = Panne chauff ind</td></tr><tr><td><i>criticalLevel</i> = -1 ∧ <i>resolutionDuration</i> = [1 - 2] → <i>Class</i> = Panne chauff ind</td></tr></tbody></table>

## 7. Conclusion

The challenges posed by real-world data in application contexts are numerous, and two major issues are class imbalance and under-information. Class imbalance introduces biases in model outcomes, while under-information restricts the algorithms' ability to extract meaningful insights. To address these challenges, we proposed in this paper an ontology-based approach to extract additional features from textual data, aiming to enrich the information available for modeling. Furthermore, to tackle the issue of class imbalance, we implements several updates based on algorithms and preprocessing. In terms of algorithm updates, we provide additional criteria, such as *Lift* and *Conviction*, in the rule ranking step, which are less sensitive to the class distribution. For preprocessing, we adopted a data oversampling approach using the SMOTEN method. This approach allowed the generation of synthetic data for minority classes, thereby enhancing the robustness of the models. The experiments conducted on building maintenance data show that our approach performs best results on precision, a clear improvement in the accuracy of the models after the ontology-based feature extraction process, which is performed using textual descriptions of maintenance operations, and a significant improvement in precision after the oversampling process. Finally, we illustrated an application example of using the classifiers obtained for matching service codes. However, the efficient utilization of these classifiers and association rules in a business context remains a challenge, primarily

due to the high number of classifiers that need to be managed by business users and experts. In the continuation of our work, we plan to semantically represent these association rules in knowledge graphs to interconnect different classifiers and generalize the data-matching process.

### CRediT authorship contribution statement

**Joel Mba Kouhoue:** Writing – original draft, Validation, Resources, Methodology, Conceptualization.
**Jerry Lonlac:** Validation, Methodology.
**Alexis Lesage:** Resources, Investigation, Conceptualization.
**Arnaud Doniec:** Validation, Supervision, Methodology.
**Stéphane Lecoeuche:** Validation, Supervision, Methodology.

### Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Joel Mba Kouhoue reports was provided by IMT Nord Europe - Douai. Joel Mba Kouhoue reports a relationship with IMT Nord Europe - Douai that includes: employment. If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

### Acknowledgment

This work is co-financed by IMT Nord Europe and the company Intent Technologies, through the ANR - Data IA program.

### Data availability

Data will be made available on request.
References

[1] J. Mba Kouhoue, J. Lonlac, A. Lesage, A. Doniec, S. Lecoeuche, A comprehensive analysis on associative classification in building maintenance datasets, in: International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, Springer, 2023, pp. 45–58.

[2] B. Balaji, A. Bhattacharya, G. Fierro, J. Gao, J. Gluck, D. Hong, A. Johansen, J. Koh, J. Ploennigs, Y. Agarwal, et al., Brick: Towards a unified metadata schema for buildings, in: Proceedings of the 3rd ACM International Conference on Systems for Energy-Efficient Built Environments, 2016, pp. 41–50.

[3] N.V. Chawla, K.W. Bowyer, L.O. Hall, W.P. Kegelmeyer, SMOTE: synthetic minority over-sampling technique, J. Artif. Intell. Res. 16 (2002) 321–357.

[4] B. Liu, W. Hsu, Y. Ma, Integrating classification and association rule mining, in: KDD, 1998, pp. 80–86.

[5] E. Bahri, S. Lallich, Proposition d'une méthode de classification associative adaptative, Rev. Nouvelles Technol. l'Inf. EGC, RNTI-E-19 (2010) 501–512.

[6] B. Liu, Y. Ma, C.-K. Wong, Classification using association rules: weaknesses and enhancements, Data Mining Sci. Eng. Appl. (2001) 591–605.

[7] W. Li, J. Han, J. Pei, CMAR: Accurate and efficient classification based on multiple class-association rules, in: ICDM, 2001, pp. 369–376.

[8] E. Baralis, P. Garza, A lazy approach to pruning classification rules, in: ICDM, 2002, pp. 35–42.

[9] J. Han, J. Pei, Y. Yin, Mining frequent patterns without candidate generation, ACM Sigmod Record 29 (2) (2000) 1–12.

[10] F. Thabtah, A review of associative classification mining, Knowl. Eng. Rev. 22 (1) (2007) 37–65.

[11] K.H.T. Dam, T. Given-Wilson, A. Legay, R. Veroneze, Packer classification based on association rule mining, Appl. Soft Comput. 127 (2022) 109373.

[12] X. Geng, Y. Liang, L. Jiao, ARC-SL: Association rule-based classification with soft labels, Knowl.-Based Syst. 225 (2021) 107116.

[13] P. Fournier-Viger, C.-W. Wu, V.S. Tseng, Mining top-k association rules, in: Advances in Artificial Intelligence, vol. 7310, 2012, pp. 61–73.

[14] R. Zulqarnain, M. Saeed, N. Ahmad, F. Dayan, B. Ahmad, Application of TOPSIS method for decision making, Int. J. Sci. Res. Math. Statist.Sci. 7 (2) (2020).

[15] I. Qasim, M. Alam, S. Khan, A.W. Khan, K.M. Malik, M. Saleem, S.A.C. Bukhari, A comprehensive review of type-2 fuzzy ontology, Artif. Intell. Rev. 53 (2) (2020) 1187–1206.

[16] F. Ali, E.K. Kim, Y.-G. Kim, Type-2 fuzzy ontology-based opinion mining and information extraction: A proposal to automate the hotel reservation system, Appl. Intell. 42 (2015) 481–500.

[17] Y. Sun, A.K. Wong, M.S. Kamel, Classification of imbalanced data: A review, Int. J. Pattern Recognit. Artif. Intell. 23 (04) (2009) 687–719.

[18] G.E. Batista, R.C. Prati, M.C. Monard, A study of the behavior of several methods for balancing machine learning training data, ACM SIGKDD Explor. Newslett. 6 (1) (2004) 20–29.

[19] A. Fernández, S. Garcia, F. Herrera, N.V. Chawla, SMOTE for learning from imbalanced data: progress and challenges, marking the 15-year anniversary, J. Artif. Intell. Res. 61 (2018) 863–905.

[20] J. Sanz, M. Sesma-Sara, H. Bustince, A fuzzy association rule-based classifier for imbalanced classification problems, Inform. Sci. 577 (2021) 265–279.

[21] J. Alcalá-Fdez, R. Alcala, F. Herrera, A fuzzy association rule-based classification model for high-dimensional problems with genetic rule selection and lateral tuning, IEEE Trans. Fuzzy Syst. 19 (5) (2011) 857–872.

[22] S. Brin, R. Motwani, C. Silverstein, Beyond market baskets: Generalizing association rules to correlations, in: Proceedings of the 1997 ACM SIGMOD International Conference on Management of Data, 1997, pp. 265–276.

[23] Y. Vasiliev, Natural language processing with Python and spaCy: A practical introduction, No Starch Press, 2020.

[24] C. Woods, M. Selway, T. Bikaun, M. Stumptner, M. Hodkiewicz, An ontology for maintenance activities and its application to data quality, Semantic Web (Preprint) 1–34.

[25] M. Schmidt, M. Meier, G. Lausen, Foundations of SPARQL query optimization, in: Proceedings of the 13th International Conference on Database Theory, 2010, pp. 4–33.

[26] C. Kaope, Y. Pristyanto, The effect of class imbalance handling on datasets toward classification algorithm performance, MATRIK: J. Manajemen, Teknik Informat. Rekayasa Komputer 22 (2) (2023) 227–238.

[27] R.R. Rao, K. Makkithaya, Learning from a class imbalanced public health dataset: a cost-based comparison of classifier performance., Int. J. Electr. Comput. Eng. (2088-8708) 7 (4) (2017).

[28] Z. Huang, Z. Zhou, T. He, X. Wang, ACAC: Associative classification based on all-confidence, in: ICGC, 2011, pp. 289–293.

[29] X. Li, D. Qin, C. Yu, ACCF: Associative classification based on closed frequent itemsets, in: 2008 Fifth International Conference on Fuzzy Systems and Knowledge Discover, vol. 2, IEEE, 2008, pp. 380–384.

[30] G. Kundu, M.M. Islam, S. Munir, M.F. Bari, ACN: An associative classifier with negative rules, in: 2008 11th IEEE International Conference on Computational Science and Engineering, IEEE, 2008, pp. 369–375.

[31] K. Wang, S. Zhou, Y. He, Growing decision trees on support-less association rules, in: ACM SIGKDD, 2000, pp. 265–269.

[32] N. Abdelhamid, A. Ayesh, F. Thabtah, S. Ahmadi, W. Hadi, MAC: A multiclass associative classification algorithm, J. Inf. Knowl. Manag. 11 (02) (2012) 1250011.

[33] F. Padillo, J.M. Luna, S. Ventura, LAC: Library for associative classification, Knowl.-Based Syst. 193 (2020) 105432.

[34] P. Fournier-Viger, J.C.-W. Lin, A. Gomariz, T. Gueniche, A. Soltani, Z. Deng, H.T. Lam, The SPMF open-source data mining library version 2, in: Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2016, Riva Del Garda, Italy, September 19-23, 2016, Proceedings, Part III 16, Springer, 2016, pp. 36–40.

[35] C. Luna, A.R. Moya, J.M. Luna, S. Ventura, StaTDS library: Statistical tests for data science, Neurocomputing 595 (2024) 127877.