## 3 Results
### Data Exploration, Outlier Detection, and Elimination  
In order to obtain the underlying structure of the data, we apply Principal Component Analysis (PCA) on Gram-negative and Gram-positive datasets separately. PCA is a dimensionality reduction technique that maps the data in high dimensional space (here each dimension corresponds to a physico-chemical property of a peptide) to a lower dimensional space (usually 2D or 3D) preserving the original structure of the data [96]. This technique is commonly used to highlight variation in a dataset and to capture strong patterns. Hence, PCA helps to visualize the data and the outliers. PCA has been applied to antimicrobial peptide data in several studies for data exploration and outlier detection purposes [97; 98; 99; 100]. In our study, we also applied PCA to our dataset for visualizing the AMP and non-AMP samples. In Figure 6, we present PCA results of the Gram-negative dataset (Figure 6a,c) and of the Gram-positive dataset (Figure 6b,d). While Figure 6a,b refer to the PCA results in 3D, Figure 6c,d refer to the PCA results in 2D. Interactive 3D plots are provided as Supplementary Material. We observe in Figure 6 that there are some outlier samples (peptides) in both Gram-negative and Gram-positive datasets. The presence of outliers can result in a poor fit and lower predictive modeling performance in classification or regression problems. For most machine learning datasets, due to the large number of input variables, the identification and removal of outliers is challenging by only using simple statistical methods. There are different computational approaches for outlier detection. One of those approaches depends on novelty detection based on machine learning [101], more specifically on one-class approaches [102; 103; 104; 105; 106]. In this study, in order to have a more homogenous group of peptides having antimicrobial activities, we wanted to eliminate outlier samples (peptides) if one of their physico-chemical features acts as an outlier. To see the distribution of the attributes in positive class (AMP) and negative class (non-AMP), we plotted the histograms for each feature. Figure 7 presents two histograms drawn for the Net Charge feature of the Gram-positive dataset for (a) AMP class and (b) Non-AMP class. It can be observed from Figure 7 that while the net charge values are in the range of [0, 31] for AMP class, it is in the range of [\(-\)6, 16] for the negative class. Based on our analysis using such histograms, we define a certain range of values for each feature for the positive class (AMP, the peptides having antimicrobial activity). We perform this analysis separately for the Gram-positive dataset and the Gram-negative dataset, and we eliminate the peptides in the positive class if their physico-chemical properties are outside of this predefined range. The range for each attribute is shown in Table 4.  
Figure 5: Feature ranking according to their importances in classification using RF model in Gram-positive dataset. Figure 6: Principal component analysis results for Gram-negative dataset are shown in (**a**,**c**); for Gram-positive dataset are shown in (**b**,**d**). While 3D plots are presented in (**a**,**b**), 2D plots are presented in (**c**,**d**). Figure 7: Graphical representation of Net Charge feature of the Gram-positive dataset. Histogram of (**a**) AMP class and (**b**) non-AMP class. At the end of the outlier elimination step, we obtain 194 non-AMPs and 88 AMPs for the Gram-positive dataset; 114 non-AMPs and 90 AMPs for the Gram-negative dataset. In Figure 8, we present PCA results of the Gram-negative dataset (shown in a,c), and of the Gram-positive dataset (shown in b,d) after outlier detection and elimination. While PCA plots are presented in 3D in (Figure 8a,b), they are presented in 2D in (Figure 8c,d). While the red colors refer to non-AMPs, blue colors indicate AMPs. Compared with Figure 6, Figure 8 implies that the positive class members are better separated from negative class members for both datasets after outliers are eliminated.