Machine Learning in Bioinformatics
By: S Joel Franklin
Tags: Machine Learning, Data Science, Bioinformatics, Supervised Learning, Classification

Machine Learning in Bioinformatics

Classification of genes & Performance comparison of common classifiers

Image by Arek Socha from Pixabay

Bioinformatics is a field of study that uses computation to extract knowledge from biological data. It includes the collection, storage, retrieval, manipulation and modeling of data for analysis, visualization or prediction. Here we would use machine learning to classify genes of E. Coli bacteria.

Let us understand the basics of Genetics. DNA or deoxyribonucleic acid is the hereditary material in humans and almost all other organisms. The DNA is made up of four chemical bases: Adenine(A), Guanine(G), Cytosine(C), and Thymine(T). Adenine pairs up with Thymine and Guanine pairs up with Cytosine. Each base is also attached to a sugar molecule and a phosphate molecule. A base, sugar, and phosphate together form the nucleotide.

Genes are made up of very long nucleotide sequences. Genes can be classified into 2 categories (Promoter & Non-Promoter) based on the nucleotide sequence.

It is impractical to look at the long nucleotide sequences. Hence we would be looking at the short nucleotide sequence(57 sequences) of genes of E. Coli bacteria and predict whether the gene is Promoter or Non-Promoter.

The data set ‘genes’ is of UIC machine learning data base.

The necessary packages are imported.

# Importing the necessary packages import numpy as np

import pandas as pd

The data is read into ‘genes’.

# Import UCI molecular biology (promoter gene sequences) dataset url = ‘https://archive.ics.uci.edu/ml/machine-learning-databases/molecular-biology/promoter-gene-sequences/promoters.data'

columns = [‘Class’,’id’,’Sequence’] # Naming the columns

genes = pd.read_csv(url,names = columns) # Reading the data # Naming the columns# Reading the data

Let us explore the dataset.

genes.shape # Prints the shape of ‘df’

The ‘genes’ dataframe is of shape (106,3) which implies 106 cases and 3 columns.

genes.nunique() # Prints number of unique elements in each column

Class :- 2, id :- 106, Sequence :- 106

There are 2 unique classes, 106 unique ids and 106 unique sequences. We know there are 106 cases. So each of the ‘id’ and ‘Sequence’ is unique.

genes[‘Class’].unique() # Prints the unique elements in ‘Class’ column

The 2 unique classes are labelled as ‘+’ (Promoter) and ‘-’ (Non-Promoter).

genes.head() # Displays first 5 rows of ‘genes’ dataframe

Some of the ‘Sequences’ begin with ‘\t’ special character which denotes a tab space. The preprocessing step would be to remove these special characters so that there are only bases(a,g,c,t).

# Removing ‘\t’ in ‘Sequence’ column for i in range(genes.shape[0]):

genes[‘Sequence’][i] = genes[‘Sequence’][i].replace(‘\t’, ‘’)

Let us verify this by checking the sequence and length of sequence of 5 randomly selected sequences.

for i in np.random.randint(0,106,5):

print(genes[‘Sequence’][i])

print(len(genes[‘Sequence’][i]))

print(‘

’)

‘\t’ characters have been removed and length of each sequence is 57.

Only the ‘Sequence’ column is important in classifying the gene. Let us make a dataframe of the ‘Sequence’ column.

# Initializing a dataframe ‘nucleotide_sequence’ of size (106,57) nucleotide_sequence = pd.DataFrame(np.random.randn(106,57)) # Storing each base of ‘Sequence’ in ‘nucleotide_sequence’ for i in range(genes.shape[0]):

nucleotide_sequence.loc[i] = list(genes[‘Sequence’][i]) # Storing 'Class' column of 'Sequence' as a column in 'nucleotide_sequence' nucleotide_sequence['Class'] = genes['Class']

Let us look at the dataframe ‘nucleotide_sequence’

nucleotide_sequence.head() # Displays first 5 rows of ‘nucleotide_sequence’ dataframe

plt.figure(figsize=(8,8))

plt.xticks(fontsize=20)

plt.yticks(fontsize=20)

plt.xlabel('Class',fontsize=20)

plt.ylabel('count',fontsize=20)

sns.countplot(nucleotide_sequence['Class']) # Displays count of each element in 'Class' column

There are equal number of Promoters and Non-Promoters in the dataset given.

Machine learning algorithms can be applied only on numerical values. Hence we convert the string characters of ‘nucleotide_sequence’ into numerical values using ‘get_dummies’ function.

# Switch to numerical data using pd.get_dummies() function numerical_nucleotide = pd.get_dummies(nucleotide_sequence)

Let us print the ‘numerical_nucleotide’ dataframe.

print(numerical_nucleotide.head()) # Prints first 5 rows of 'numerical_nucleotide'

There are 2 class columns :- ‘Class_+’ and ‘Class_-’. The class label ‘Promoter’ or ‘Non-Promoter’ can be found by just looking at one of the column (either ‘Class+’ or ‘Class-’). So one of them is dropped and the other is renamed as ‘Class’.

# Remove one of the class columns and rename the other to simply ‘Class’ numerical_nucleotide.drop(‘Class_-’,axis = 1,inplace = True)

numerical_nucleotide.rename(columns = {‘Class_+’:’Class’},inplace = True)

Now let us print the ‘numerical_nucleotide’ dataframe.

print(numerical_nucleotide.head()) # Prints first 5 rows of 'numerical_nucleotide'

Below given is the snippet of code that serves the same purpose as get_dummies() function (Optional)

Now that we have processed the data, the next step is train the data using machine learning classifiers.

The dataset is split into training and test sets.

# Import train_test_split from sklearn.model_selection import train_test_split # X is input and y is output X = numerical_nucleotide.drop(‘Class’,axis = 1)

y = numerical_nucleotide[‘Class’] # Split into train and test sets X_train,X_test,y_train,y_test = train_test_split(X,y)

The necessary classifiers and metrics are imported.

# Performance comparison of 9 classifiers from sklearn.neighbors import KNeighborsClassifier

from sklearn.neural_network import MLPClassifier

from sklearn.tree import DecisionTreeClassifier

from sklearn.ensemble import RandomForestClassifier, AdaBoostClassifier

from sklearn.naive_bayes import GaussianNB

from sklearn.svm import SVC

from sklearn.metrics import classification_report,accuracy_score

from sklearn import model_selection

The Cross validation scores of the classifiers are calculated.

The Cross validation scores of classifiers are displayed.

Classifier rankings based on Cross validation scores :-

(1 Neural Net (MLP Classifier) & SVM Linear

(2 Naive Bayes

(3 AdaBoost

(4 SVM rbf

(5 K Nearest Neighbors

(6 Decision Tree

(7 Random Forest

(8 SVM Sigmoid

Neural Net (MLP Classifier) & SVM Linear have the best cross validation score of 0.9625. But it is not right to generalize the performance of classifier on new unseen data based on cross validation score. Hence we check the performance of classifier models on the test set.

for i in range(len(models)):

a = models[i].fit(X_train,y_train) # Fitting the model

y_predict = a.predict(X_test) # Predicting using trained model

print(accuracy_score(y_predict,y_test))

print(classification_report(y_predict,y_test))

K-Nearest Neighbors

Neural Net (MLP Classifier)

Decision Tree

Random Forest

AdaBoost

Naive Bayes

SVM Linear

SVM rbf

SVM Sigmoid

Classifier rankings based on Test scores :-

(1 Neural Net (MLP Classifier), Naive Bayes & SVM Linear

(2 K-Nearest Neighbors & SVM rbf

(3 AdaBoost & SVM Signoid

(4 Decision Tree

(5 Random Forest

Neural Net (MLP Classifier), Naive Bayes & SVM Linear have the highest accuracy score and f1-score.

The performance of the classifier models can be further improved by hyperparameter tuning. If you are curious, please do read hyperparameter tuning of Neural Networks.

Happy Reading!