## 4 Experimental Results and Analysis
### Comparison with Other RBP Prediction Methods on Independent Validation Datasets  
To further evaluate the performance of the proposed CnnRBP method, we compare our method with existing RBP prediction methods, including SPOT-Seq-RNA [14], RNAPPred [10], RBPPred [13], Deep-RBPPred [21], TriPepSVM [15] and iDRBP_MMC [24]. Among the six methods, TriPepSVM is an SVM model based on the same short sequence motifs as ours. Deep-RBPPred is a CNN model with features derived from five physico-chemical properties. iDRBP_MMC is a multi-label learning model with a CNN based on PSSMs and structural motifs. For each species, we train a CnnRBP model on the training dataset with the hyperparameters optimized by the 10-fold cross-validation and then perform predictions on the independent validation dataset. For SPOT-Seq-RNA, RNAPPred, RBPPred and TriPepSVM, their evaluation data are cited from the paper that proposed the method TriPepSVM. For Deep-RBPPred, we executed the publicly released source code to make an evaluation on the independent validation dataset. For iDRBP_MMC, we fed testing samples into its web server and then calculated evaluation indexes from the prediction results. The evaluation indexes for all methods are shown in Table 4. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline
**Predictor** & **ACC (\%)** & **PRE (\%)** & **SEN (\%)** & **SPE (\%)** & **F1 (\%)** & **BACC (\%)** & **MCC** & **AUC** \\ \hline \multicolumn{8}{c}{Human dataset} \\ \hline SPOT-Seq-Pred & 85.70 & 38.89 & 23.20 & 94.52 & 29.07 & 58.86 & 0.22 & - \\ RNAPPred & 49.67 & 18.74 & 87.57 & 44.09 & 30.88 & 65.83 & 0.22 & 0.72 \\ RBPPred & 65.63 & 24.08 & 75.69 & 64.12 & 36.53 & 69.91 & 0.27 & 0.70 \\ Deep-RBPPred & 30.25 & 14.73 & 90.61 & 21.18 & 25.35 & 55.90 & 0.10 & 0.69 \\ TriPepSVM & 88.95 & 58.75 & 51.93 & 94.45 & 55.13 & 73.23 & 0.49 & 0.83 \\ iDRBP\_MMC & 89.29 & 56.57 & 78.45 & 90.93 & 65.73 & 84.69 & 0.61 & 0.92 \\ CnnRBP * & 92.64 & 72.07 & 71.27 & 95.85 & 70.67 & 83.56 & 0.67 & 0.91 \\ \hline \multicolumn{8}{c}{_c. coli_ dataset} \\ \hline SPOT-Seq-Pred & 87.28 & 100.00 & 29.03 & 100.00 & 45.00 & 64.52 & 0.50 & - \\ RNAPPred & 66.67 & 32.00 & 80.00 & 63.83 & 45.71 & 71.91 & 0.34 & 0.75 \\ RBPPred & 80.92 & 47.73 & 67.74 & 83.80 & 56.00 & 75.77 & 0.45 & 0.77 \\ Deep-RBPPred & 61.02 & 20.71 & 78.85 & 85.85 & 32.80 & 68.72 & 0.24 & 0.72 \\ TriPepSVM & 92.34 & 69.39 & 65.38 & 96.04 & 67.32 & 80.71 & 0.63 & 0.92 \\ iDRBP\_MMC & 92.57 & 75.00 & 57.00 & 97.36 & 64.77 & 77.18 & 0.62 & 0.90 \\ CnnRBP * & 93.04 & 69.64 & 75.00 & 95.51 & 72.22 & 85.26 & 0.68 & 0.96 \\ \hline \multicolumn{8}{c}{Salmonella dataset} \\ \hline SPOT-Seq-Pred & 92.11 & 100.00 & 34.15 & 100.00 & 51.43 & 67.31 & 0.56 & - \\ RNAPPred & 49.18 & 17.25 & 86.27 & 44.18 & 28.76 & 65.23 & 0.20 & 0.79 \\ RBPPred & 81.44 & 35.11 & 63.46 & 83.91 & 45.21 & 73.68 & 0.37 & 0.81 \\ Deep-RBPPred & 60.12 & 27.38 & 74.19 & 57.04 & 40.00 & 65.62 & 0.24 & 0.70 \\ TriPepSVM & 90.17 & 85.00 & 54.83 & 97.89 & 66.66 & 76.36 & 0.63 & 0.86 \\ iDRBP\_MMC & 87.86 & 67.85 & 61.29 & 93.66 & 64.41 & 77.48 & 0.58 & 0.90 \\ CnnRBP * & 92.49 & 84.62 & 70.97 & 97.18 & 77.19 & 84.08 & 0.73 & 0.91 \\ \hline \hline \end{tabular}  
* Training one model per species.  
\end{table} 
Table 3: The top and last selected features for different species. From the results in Table 4, it can be found that our method CnnRBP achieves the best overall prediction performance among all methods. Specifically, on the human dataset we can find that two deep-learning-based methods (i.e., CnnRBP and iDRBP_MMC) are significantly superior compared to other shallow methods, according to the overall performance indexes, such as the F1 measure, BACC, MCC and AUC. Among the shallow methods, TriPepSVM performs best with an AUC of 0.83, MCC of 0.49 and BACC of 73.23%. RNAPred and SPOT-Seq-Pred appear to have a large disparity in SEN and SPE, leading to a poor performance in BACC and MCC, which shows that they cannot deal with the class imbalance problem.