## 2 Materials and Methods
### Oversampling Methods
#### 2.3.3 Adasyn  
The Adaptive Synthetic (ADASYN) algorithm can be described as an upgraded version of SMOTE algorithm. The initial dataset consist of \(m\) samples that can generally be written as:  
\[\{\mathbf{x}_{i},y_{i}\},i=1,\ldots,m, \tag{1}\]  
where \(\mathbf{x}_{i}\) and \(y_{i}\) represent an instance in the n-dimensional feature space \(\mathbf{X}\) and class identity label associated with \(x_{i}\). The total number of dataset samples is grouped based on classes where \(S_{s}\) represents the minority class samples and \(S_{l}\) represents the majority class samples. The conditions for the definition of minority and majority class samples are  
* the number of minority samples must be less or equal to the number of majority samples,
* the sum of the minority and majority number of samples must be equal to the total number of samples.  
The algorithm execution begins by inspecting the degree of class imbalance using the expression:  
\[d=\frac{S_{s}}{S_{l}}. \tag{2}\]  
The degree of class imbalance can be in the range of 0 to 1. The value of d is compared with a preset threshold for the maximum tolerated degree of class imbalance ratio. If the value is below the threshold the ADASYN algorithm will be applied. The next step is to calculate the number of synthetic data samples that have to be generated in the case of the minority class. To calculate the number of samples the following expression is utilized:  
\[G=(S_{l}-S_{s})\times\beta, \tag{3}\]  
Figure 6: The results of SMOTE oversampling technique application on binary and multiclass dataset. (**a**) SMOTE oversampled Binary Dataset; (**b**) SMOTE Oversampled Multiclass Dataset.  
In the previous equation, the \(\beta\) is the parameter used to define the balance level after the generation of synthetic data. The value of this parameter can be in the range from 0 to 1 and if \(\beta=1\) this means that the dataset is fully balanced after the generalization process.  
The K nearest neighbors are found for each dataset minority class sample based on Euclidean distance in n-dimensional space and calculate the ratio between the number of examples \(\delta_{i}\) in the \(K\) nearest neighbors of \(x_{i}\) that are from majority class using the expression:  
\[r_{i}=\frac{\Delta_{i}}{K},\quad i=1,\ldots,S_{s}. \tag{4}\]  
The range of \(r_{i}\) is between 0 and 1. After \(r_{i}\) is obtained it has to be normalized so that the density distribution is equal to 1. The normalization is done using the expression:  
\[\hat{r}_{i}=\frac{r_{i}}{\sum_{i=1}^{S_{s}}r_{i}} \tag{5}\]  
The next step is to calculate the number of synthetic data samples that have to be generated for each minority sample:  
\[g_{i}=\hat{r}_{i}\times G \tag{6}\]  
Finally, for each minority class data sample generate synthetic data samples in following steps in range from 1 to \(g_{i}\).  
* Randomly choose one data sample \(x_{zi}\) from the K nearest neighbors for data \(x_{i}\)
* Generate the synthetic data sample using the expression: \[s_{i}=x_{i}+(x_{zi}-x_{i})\times\lambda\] (7) where \((x_{zi}-x_{i})\) is the difference vector in n dimensional space and \(\lambda\) a random number between 0 and 1.  
After the minority classes are oversampled in both cases using the ADASYN method the kernel PCA was utilized for better dataset visualization. The results are shown in Figure 7.  
Figure 7: Cont.  
In case of ADASYN (Figure 7) the minority classes in both cases are oversampled. However, the areas of class samples overlap. This is especially evident in multiclass case where samples of healthy patients overlap with samples of Hepatitis C and Fibrosis patients. The samples of cirrhosis patients slightly overlap with healthy, hepatitis C and fibrosis patients.