Machine Learning For Genomics. How to transform your genomics data to... | by Anuradha Wickramarachchi | Towards Data Science

Machine learning has become popular. However, it is not a common use case in the field of Bioinformatics and Computational Biology. There are very few tools that use machine learning techniques. Most of the tools are developed on top of deterministic approaches and algorithms. In this article, I will present how we can arrange our data so that they would be used effectively in a machine learning model.
There are many scenarios in genomics that we might use machine learning. The major areas of Clustering and Classification can be used in Genomics for various tasks. A few of them are as follows:
Binning of Metagenomics ContigsIdentification of Plasmids and ChromosomesClustering reads into chromosomes for better assemblyClustering of reads as a preprocessor for assembly of reads
Binning of Metagenomics Contigs
Identification of Plasmids and Chromosomes
Clustering reads into chromosomes for better assembly
Clustering of reads as a preprocessor for assembly of reads
Classifying shorter sequences into classes (phylum, genus, species, etc)Phylogenetic inference of the sequencesDetection of Plasmids and ChromosomesFinding coding regionsChromosome prediction in human genomics
Classifying shorter sequences into classes (phylum, genus, species, etc)
Phylogenetic inference of the sequences
Detection of Plasmids and Chromosomes
Finding coding regions
Chromosome prediction in human genomics
The list could possibly extend a lot further, though I have listed a few areas which I have had the experience in.
Out of the two steps, transformation and model selection, I would consider the first to be of higher importance. This is because without a solid base for the data representation we might not get the maximum out of the model. However, having clean and information-rich data could perform reasonably better even if our model seems to be poor. We could consider data under two categories.
Sequential Data refers to data where the ordering of the data fed to the model corresponds to the actual order of data in the dataset. Let's have a look at the following example.
Prediction of Protein Coding RegionsIn this scenario, we will be looking at consecutive nucleotide bases and their order. Otherwise, it would not make sense. In such a scenario, we could transform data as sentences having consecutive trimer words.Taxonomic Label PredictionIn this scenario, we need to have a higher precision as oligonucleotide frequencies could be very similar among species and have a very low uniqueness between certain species. Therefore, we would be looking at creating k-mer sentences of k values above 5 or 7 as sentences.
Prediction of Protein Coding RegionsIn this scenario, we will be looking at consecutive nucleotide bases and their order. Otherwise, it would not make sense. In such a scenario, we could transform data as sentences having consecutive trimer words.
Taxonomic Label PredictionIn this scenario, we need to have a higher precision as oligonucleotide frequencies could be very similar among species and have a very low uniqueness between certain species. Therefore, we would be looking at creating k-mer sentences of k values above 5 or 7 as sentences.
Let us consider the following example. I will be using the two species Pseudomonas aeruginosa (CP007224.1) and Lactobacillus fermentum (AP008937.1) to demonstrate the transformation into k-mer word sentences (First 50 bases shown).
Pseudomonas aeruginosaTTTAAAGAGACCGGCGATTCTAGTGAAATCGAACGGGCAGGTCAATTTCCLactobacillus fermentumTTGACTGAGCTCGATTCTCTTTGGGAAGCGATCCAAAATTCATTCCGTAA
Now, we can transform this as trimer words to like the following output.
Pseudomonas aeruginosaTTT TTA TAA AAA AAG AGA GAG AGA GAC ACC CCG CGG GGC GCG CGA GAT ATT TTC TCT CTA TAG AGT GTG TGA GAA AAA AAT ATC TCG CGA GAA AAC ACG CGG GGG GGC GCA CAG AGG GGT GTC TCA CAA AAT ATT TTT TTC TCCLactobacillus fermentumTTG TGA GAC ACT CTG TGA GAG AGC GCT CTC TCG CGA GAT ATT TTC TCT CTC TCT CTT TTT TTG TGG GGG GGA GAA AAG AGC GCG CGA GAT ATC TCC CCA CAA AAA AAA AAT ATT TTC TCA CAT ATT TTC TCC CCG CGT GTA TAA
Now that we have sentences of words, the processing becomes similar to that of sentiment analysis. The key idea is to preserve the sentence length, which you could decide based on the average length of sequences.
Since the order of k-mers matter in the above scenario, we can easily use a Recurrent Neural Network (RNN) or a Long Short Term Memory model (LSTM). This is because of their ability to keep the order of items as a temporal relationship.
However, in the above scenario, you must use tokenization followed by a word embedding layer. Be sure to pickle the encoders and tokenizers to a file (serialise) so that you’d have the encoder for predictions later on. Here kmer_strings are the sentences I created and classes are the designated label.
tokenizer = Tokenizer()tokenizer.fit_on_texts(kmer_strings)vocab_size = len(tokenizer.word_index) + 1encoded_docs = tokenizer.texts_to_sequences(kmer_strings)binarizer = preprocessing.LabelBinarizer()labels_encoded = binarizer.fit_transform(classes)
Following is one model that I used for a sequence classification a few days ago using the Keras Sequential API. I am embedding the words into 5 dimensions from the initial 32 dimensions (there are 32 unique trimers merging the reverse complements to the lower strand).
model = Sequential()model.add(Embedding(vocab_size, 5))model.add(Bidirectional(LSTM(5)))model.add(Dense(30, activation = 'relu'))model.add(Dropout(0.2))model.add(Dense(20, activation = 'relu'))model.add(Dropout(0.2))model.add(Dense(2, activation = 'softmax'))
If your classification is multi-label, you would need to use sigmoid activation function in-place of the softmax function.
For scenarios where relatedness between similar sequences is more important than the unique identifiability of sequences, we tend to use unordered data. These are usually represented in the form of oligonucleotide frequency vectors. For example, the normalized frequencies for Pseudomonas aeruginosa (CP007224.1) and Lactobacillus fermentum (AP008937.1) would look like follows.
We can see that they follow drastically different patterns. Note that we have lost the actual order of trimers but left with a representation to identify the two species uniquely. However, at k=3 you are very likely to have collisions between closely related species. This is an interesting article on how to cluster based on these patterns;
towardsdatascience.com
And the following article might help on how to use DBSCAN pretty effectively.
towardsdatascience.com
Now that you have seen the data, you could use a model similar to the following to do classification.
model = Sequential()model.add(Dense(30, activation = 'relu', input_shape=(32,)))model.add(Dropout(0.2))model.add(Dense(20, activation = 'relu'))model.add(Dropout(0.2))model.add(Dense(2, activation = 'softmax'))model.compile(loss='binary_crossentropy', optimizer='adam')
Note that your input dimension will increase in powers of 4 as you decide on longer k-mers for this. Furthermore, it would take a very long time to compute such long vectors too. You might want to read on the following article that explains how to do that using a small script that you can use very easily.
medium.com
Now that you have seen how one might use genomic sequences of variable lengths in a machine learning model, let me show few tools that actually do this.
PlasClass (published 2020, PLOS)Uses logistic regression on k-mer frequency vectors to detect whether they originate from a plasmid sequence or a chromosomal segment. This is a binary classification based tool.PlasFlow (published 2018, Nucleic Acid Research)This tool predicts the phylum level classification and predicts whether a given contig is a plasmid or a chromosome. Uses a neural network on top of k-mer frequency vectors.MetaBCC-LR (published, 2020, Bioinformatics)Uses t-distributed Stochastic Neighbour Embedding (t-SNE) to dimension reduce genomic long reads to perform binning of metagenomic reads’ trimer vectors.
PlasClass (published 2020, PLOS)Uses logistic regression on k-mer frequency vectors to detect whether they originate from a plasmid sequence or a chromosomal segment. This is a binary classification based tool.
PlasFlow (published 2018, Nucleic Acid Research)This tool predicts the phylum level classification and predicts whether a given contig is a plasmid or a chromosome. Uses a neural network on top of k-mer frequency vectors.
MetaBCC-LR (published, 2020, Bioinformatics)Uses t-distributed Stochastic Neighbour Embedding (t-SNE) to dimension reduce genomic long reads to perform binning of metagenomic reads’ trimer vectors.
There are many more tools in different areas of research other than these few (which I used and the 3rd one I authored). LSTMs are used in gene prediction and coding region detection.
However, there is tremendous potential in the area for machine learning techniques to show off.
I hope you had some useful reading. Cheers!