File size: 825 Bytes
5d65aea 0964ff1 5d65aea 0964ff1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
---
tags:
- DNA
license: mit
---
## MiniDNA model
This is a distilled version of [DNABERT](https://github.com/jerryji1993/DNABERT) by using MiniLM technique. It has a BERT architecture with 6 layers and 768 hidden units, pre-trained on 6-mer DNA sequences. For more details on the pre-training scheme and methods, please check the original thesis report _[link to be added]_.
## How to Use
The model can be used to fine-tune on a downstream genomic task, e.g. promoter identification.
```python
import torch
from transformers import BertForSequenceClassification
model = BertForSequenceClassification.from_pretrained('Peltarion/dnabert-minilm')
```
More details on how to fine-tune the model, dataset and additional source codes are available on [GitHub](https://github.com/joanaapa/Distillation-DNABERT-Promoter). |