# Complement-Class Harmonized Naive Bayes Classifier
## 1 Introduction
\[P(x|c)=\prod_{j=1}^{m}P(x_{j}|c), \tag{2}\]  
This is the core concept of Naive Bayes (NB) classifier which uses Equation (3) to classify a test instance (\(x\)), where \(a_{i}\) is the \(i\)-th attribute value:  
\[c(x)=\operatorname*{argmax}_{c\in C}P(c)\prod_{i=1}^{m}P(a_{i}|c) \tag{3}\]  
Equation (3) is simple because the conditional independence assumption is made for efficiency reasons and to make it possible to estimate the values of all probability terms, since in practice, many attribute values are not represented in training data in sufficient numbers. However, the performance of NB degrades in domains where the independence assumption is not satisfied [19; 20] or where the training data are scarce [21; 22]. Various methods and approaches have been proposed to address the first problem and relax the attributes' conditional independence assumption by extending NB structure [23; 24], attribute selection [25; 26], and attribute weighting methods [3; 27; 28; 29; 30; 31; 32; 33; 34]. To alleviate the second problem, other methods are proposed that act in very different ways on scarce data, such as instance cloning [35; 36], instance weighting [37; 38], and fine-tuning Naive Bayes [1; 39]. However, to the best of our knowledge, most existing approaches for alleviating attributes' conditional independence assumption and the data scarcity problem have one or both of the following problems: 1. Overfitting due to increased model complexity, especially on small or imbalanced datasets, 2. The absence of profound identification of potential discriminative attribute (feature) value in the presence of scant data. Consequently, the improvement of the enhanced NB classifier will be limited due to not targeting the right potential discriminative attributes for improving its representations in the data and its predictive power. For example, current state-of-the-art attribute weighting [30; 34; 40] and fine-tuning [39] Naive Bayes classifiers are fine-grained boosting of attribute values, however, the complexity of the methods increases their tendency to overfit the training data and become less tolerant to noise [1; 3; 41]. In addition, the methods are either class-independent [30], where it assigns each attribute value the same weight for all classes, or class-dependent [34; 39; 40], but not considering the attribute value distribution divergence between different classes simultaneously. Thus, an attribute value that is equally distributed but highly correlated with two or more classes is considered a discriminative attribute and enjoys the highest attribute weights in case of attribute weighting or the largest probability term update amount in case of fine-tuning algorithms. We proposed a new fine-tuning approach of NB; we call it the complement-class harmonized NB classifier (CHNB), which is different from the original fine-tuning algorithm FTNB [39] in capturing the attribute value inter-correlation (between classes) and intra-correlation (within the class). The aim is to improve the estimation of conditional probability and mitigate the effect of conditional independent assumption, especially for domains with scant and imbalanced data. In the proposed CHNB, the fine-tuning update amount is computed gradually to increase or decrease impacted probability terms, therefore, CHNB creates a more dynamic and accurate distribution for each rare class attribute value which would eliminate diversity and overlap the drawbacks of the synthetic sample generation of SMOTE and its variants. Moreover, CHNB can be integrated with any data-level approaches for class imbalanced problems, such as SMOTE. We hypothesize that this approach will improve asymptotic accuracy, especially in domains with scarce data, without reducing the accuracy in domains with sufficient data. We conducted extensive experiments to compare our proposed method with state-of-the-art attribute weighting and fine-tuning NB methods on 41 general benchmark datasets, and with imbalanced ensemble methods on three imbalanced benchmark datasets.