Nearest Neighbor Classifiers Versus Random Forests and Support Vector Machines Recent experimental studies have shown that the support vector machine and the random forest are stand out models that outperform most other supervised learning methods in benchmarked evaluations on realworld data sets. Studies have also established that both these models are closely related to nearest neighbor classifiers. Due to this connection, one would expect a similar performance from a nearest neighbor classifier; however, the latter lags greatly behind the support vector machine and the random forest in most evaluations. On the contrary, the nearest neighbor classifier has great theoretical potential because of its ability to model complex decision boundaries with low bias. Furthermore, it theoretically guarantees an error of no more than twice the Bayes error rate on an infinite data set. In this paper, we examine this obvious disconnect between the great theoretical promise and empirical reality of the nearest neighbor classifier. An additional point is that the random forest and support vector machine can be considered eager forms of the (lazy) nearest neighbor classifier. Inspired by the connections between nearest neighbor classifiers, support vector machines and random forest models, we propose a nearest neighbor classifier that is competitive to these models.

tTree and tForest: Decision Tree and Random Forest Algorithms Including the Relevance Factor with Applications in Bioinformatics The increase in popularity of machine learning with its application as a prediction tool in bioinformatics is enabling biologists to save laboratory resources. Decision trees and random forests are nonparametric machine learning algorithms which work by selecting the best features for dividing a dataset at each iteration. Division continues until the entire dataset is classified based on all possible output classes. The selection of the best features, that is, for creating the split point, is based on evaluating a cost function for each feature. One of the most popular cost functions is called the Gini index. In this paper we investigate the use of the relevance of features to the output classes at each division point in addition to the use of the Gini index. The relevance of features for our algorithms, termed tTree and tForest, is measured using pvalues, readily obtained via the ttest and accessible for many bioinformatics problems. We apply our solution to the type IV secretion system (T4SS) effector prediction problem and perform experiments of our proposed algorithms and two variations of them using two different datasets for T4SS effector and noneffector proteins. We show that the use of the tTree and tForest algorithms results in an increase in classification accuracy as well as an increase in the time needed to build both trees and forests.