rahuls472/DNABERT-Promoter-Classifier

🧬 DNABERT Promoter Classifier

A fine-tuned DNABERT model for binary promoter sequence classification. Given a DNA sequence, the model predicts whether it belongs to a Promoter or Non-Promoter region.

Unlike conventional NLP models, DNABERT treats DNA as a language. During preprocessing, each DNA sequence is converted into overlapping 6-mers, enabling the Transformer to learn biologically meaningful sequence patterns.

This project demonstrates the application of Transformer-based deep learning for computational genomics and serves as part of my Bioinformatics and Machine Learning portfolio.

---

🚀 Overview

Promoters are regulatory DNA elements responsible for initiating gene transcription. Accurate promoter identification plays a critical role in:

• Genome annotation
• Gene regulation studies
• Functional genomics
• Bioinformatics research

This model fine-tunes DNABERT, a Transformer model pre-trained on genomic sequences represented as 6-mers.

Property	Value
Base Model	DNABERT (DNA-BERT-6)
Task	Binary DNA Sequence Classification
Framework	Hugging Face Transformers
Backend	PyTorch
Precision	FP16 Mixed Precision
Training Platform	Google Colab GPU

---

🧬 Sequence Preprocessing

Before tokenization, every DNA sequence is converted into overlapping 6-mers, matching the representation used during DNABERT pretraining.

Example:

DNA sequence

ATGCGTACGA

6-mer representation

ATGCGT TGCGTA GCGTAC CGTACG GTACGA

The generated 6-mer sentence is then tokenized using the DNABERT tokenizer.

Important: The same preprocessing must be applied during inference. Raw DNA sequences should always be converted to 6-mers before being passed to the model.

---

👨‍💻 Author

Rahul Kumar Singh

M.Sc. Bioinformatics
Deen Dayal Upadhyaya Gorakhpur University

• GitHub: https://github.com/rahuls472
• Hugging Face: https://huggingface.co/rahuls472
• LinkedIn: https://www.linkedin.com/in/rahul-kumar-singh-1796b5332/

---

🎯 Classification Task

Label	Class
0	Non-Promoter
1	Promoter

---

📊 Model Performance

The model was evaluated on the held-out test dataset.

Metric	Score
Accuracy	90.8%
Weighted Precision	0.92
Weighted Recall	0.91
Weighted F1-score	0.91
ROC-AUC	0.958
Average Precision (PR-AUC)	0.973

---

📈 Classification Report

Class	Precision	Recall	F1-score
Non-Promoter	0.85	0.98	0.91
Promoter	0.98	0.85	0.91

Overall Test Accuracy: 90.8%

---

📊 Confusion Matrix

---

📈 ROC Curve

ROC-AUC = 0.958

---

📉 Precision–Recall Curve

Average Precision = 0.973

---

⚙️ Training Configuration

Parameter	Value
Learning Rate	2e-5
Batch Size	8
Epochs	5
Weight Decay	0.01
Optimizer	AdamW
Mixed Precision	FP16
Evaluation Strategy	Every Epoch
k-mer Size	6
Maximum Sequence Length	512 Tokens

---

💻 Example Usage

from transformers import AutoTokenizer
from transformers import AutoModelForSequenceClassification
import torch

def generate_kmers(sequence, k=6):
    return " ".join(
        sequence[i:i+k]
        for i in range(len(sequence)-k+1)
    )

model_name = "rahuls472/DNABERT-Promoter-Classifier"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

sequence = "ATCGGCTAGCTAGCTAGCTAGCTA"

sequence = generate_kmers(sequence)

inputs = tokenizer(
    sequence,
    return_tensors="pt",
    truncation=True,
    padding="max_length",
    max_length=512
)

with torch.no_grad():
    outputs = model(**inputs)

prediction = torch.argmax(outputs.logits, dim=1).item()

print("Promoter" if prediction == 1 else "Non-Promoter")

---

🔬 Training Pipeline

DNA Sequence
      │
      ▼
Generate overlapping 6-mers
      │
      ▼
DNABERT Tokenizer
      │
      ▼
DNABERT Encoder
      │
      ▼
Sequence Classification Head
      │
      ▼
Promoter / Non-Promoter

---

📚 Applications

• Promoter prediction
• DNA sequence classification
• Genome annotation
• Functional genomics
• Computational biology research
• Educational projects

---

⚠️ Limitations

This model is intended for research and educational purposes only.

It is not intended for:

• Clinical diagnosis
• Medical decision making
• Disease prediction
• Variant interpretation

---

🚀 Future Work

• Multi-species promoter prediction
• Cross-validation experiments
• Hyperparameter optimization
• Multi-class genomic element classification
• Explainability using attention visualization
• Integration into genomics analysis pipelines

---

🙏 Acknowledgements

This project was developed using:

• Hugging Face Transformers
• PyTorch
• Google Colab
• DNABERT
• Genomic Benchmarks Dataset

Special thanks to the open-source bioinformatics and machine learning communities for providing the datasets, pretrained models, and tools that made this work possible.