# Prediction of Transcription Factor Binding Sites of SP1 on Human Chromosome1
## 1 Introduction
### Background  
Logistic Regression [4] is a commonly used technique in classification problems that is applied when the values are completely different from each other. It is a statistical model in which a logistic curve is fitted to the dataset. It calculates the probability of the default class based on the features. If the probability turns out to be greater than a specified threshold, it predicts the value of the target value as the default class. Linear Discriminant Analysis (LDA) [5] is mainly used for dimensionality reduction. It finds a linear combination of features that separates two or more classes of objects. After the dimensionality reduction, it could be viewed as a classification algorithm. K-Nearest Neighbor (KNN) [6] classifies a new data point by searching the entire training set for the k most similar instances by being closest to the test data point. The value for k is preferred to be an odd number and it has a major effect on the algorithm's performance. If k is too small, the model becomes sensitive to outliers. Decision Trees [6; 7] use the divide and conquer approach. They are not affected much by outliers and can deal with linearly inseparable data. They split the data based on the features, and there are many splitting criteria available (e.g., Gini coefficient, entropy metrics, etc.). Errors propagate through trees, which becomes a big problem as the number of classes increases. Without proper pruning, the Decision Tree can easily overfit. Random Forest [8] is a bagging ensemble algorithm, meaning that it uses multiple different algorithms and generates a final result based on them. Random Forest trains many Decision Trees and returns the class that had the majority in the trees' decisions. With the increase in the number of trees the algorithm becomes computationally slow. Naive Bayes [6; 9] is a simple yet powerful classification algorithm. It is a conditional probability model in which it assumes that a value of a particular feature is independent of the value of any other feature. This assumption is unrealistic in real life data; however, the algorithm is still effective in multiple problems. An advantage of this algorithm is that it requires a small number of training data to estimate the parameters for classification, and it is considered extremely fast when compared to other more advanced algorithms. The Naive Bayes model can be directly calculated by calculating the probability of each class of the problem and calculating the conditional probability for each class given each x value. Support Vector Machine (SVM) [6; 10] is considered a complex algorithm but can provide high accuracy. It works well even if data are not linearly separable in feature space, given the appropriate kernel. The algorithm is based on the concept of maximizing the minimum distance from hyperplane to the nearest sample point. Its performance is dependent on choice of features. AdaBoost [10; 11; 12], short for adaptive boosting, is an ensemble boosting algorithm. A boosting algorithm uses weak learner classifiers and transforms their output into a strong learner. AdaBoost adapts multiple Decision Trees by using them consecutively; each tree improves on the results of the previous one by attempting to correct its errors. Models are added until the training set is predicted perfectly, or a maximum number of models are added. Predictions of the last model is the sum of weights of the predictions made by all of the previous models. Gradient Boosting [13] is also an ensemble boosting algorithm that is similar to AdaBoost. However, the main difference is the technique used to identify the errors of the weak learners. Gradient Boosting identifies the shortcoming of weak classifiers by using gradients in the loss function. The Extra Trees [14] algorithm is short for extremely randomized trees.