# Complement-Class Harmonized Naive Bayes Classifier
## 1 Introduction  
Machine learning (ML) is a data-driven approach that has emerged as a useful tool for rapid and accurate prediction. However, under-sampled or non-representative data can lead to incomplete information about a concept, making it difficult to make accurate predictions, causing overfitting problems. In overfitting, the ML model is over-optimized to the training data and fails to generalize unseen examples. This problem becomes worse if the data is high-dimensional or if the model has multiple tunable parameters, such as in deep learning or boosted models [1, 2, 3, 4]. The challenges posed by scarce data have been recognized and extensively discussed in the research community for some time. In general, existing approaches apply data-level, model-level, or combined techniques that act in very different ways. For example, under-sampling, over-sampling [5], cleaning-sampling [6], or hybrid [7] are data-level methods that can deal with data scarcity. Recent research combines these resampling techniques with ensemble models because of the flexible characteristics of ensemble models, such as reducing prediction errors, and reducing bias and/or variance. Each phase of ensemble models provides a chance to make the model better for classifying the minority class by taking a base learning algorithm and training it on a different training set. Different algorithms using different resampling methods for building ensemble models were proposed [8, 9, 10, 11], SMOTE [5] is the most influential data-level technique for class-imbalance problems [12], which generates synthetic rare class samples based on the sample of k nearest neighborswith the same class. However, SMOTE and its variants have two main drawbacks in synthetic sample generation [13]; Rare classes' probability distributions are not considered, and in many cases, the generated minority class samples lack diversity and overlap heavily with major classes. Many recent published works addressed these drawbacks. [PERSON] et al. [13] proposed a weighted kernel-based SMOTE, which generates synthetic rare class samples in a feature space. The authors in [14] proposed a SMOTE-based, class-specific, extreme learning machine, which exploits the benefits of both the minority oversampling and class-specific regularization to overcome the limitation of the linear interpolation of SMOTE. In [2], a generalized Dirichlet distribution was used as a prior for the multinomial NB classifier to find non-informative generalized Dirichlet priors so that its performance on high-dimensional imbalanced data could be largely improved compared with generating synthetic instances in a high-dimensional space. Naive Bayes (NB) classifier is a well-known classification algorithm for high-dimensional data because of its computational efficiency, robustness to noise [15], and support of incremental learning [16; 17; 18]. This is not the case for other machine learning algorithms, which need to be retrained again from scratch. In the Bayesian classification framework, the posterior probability is defined as:  
\[P(c|x)=\frac{P(x|c)P(c)}{P(x)} \tag{1}\]  
where \(x\) is the feature vector, \(c\) is the classification variable, \(P(x)\) is the evidence, \(P(x\,|\,c)\) is the likelihood probability distribution, and \(P(c\,|\,x)\) is the posterior probability. However, we cannot obtain reliable estimates of the likelihood \(P(x\,|\,c)\) due to the curse of dimensionality. However, if we assume that, given a class label, each attribute is conditionally independent of each other and all attributes are equally important, then the computation of \(P(x\,|\,c)\) is made feasible and is obtained simply by multiplying the probability for each individual attribute, Equation (2).