|
--- |
|
tags: |
|
- DNA |
|
license: mit |
|
--- |
|
|
|
## MiniDNA model |
|
|
|
This is a distilled version of [DNABERT](https://github.com/jerryji1993/DNABERT) by using MiniLM technique. It has a BERT architecture with 6 layers and 768 hidden units, pre-trained on 6-mer DNA sequences. For more details on the pre-training scheme and methods, please check the original thesis report _[link to be added]_. |
|
|
|
|
|
## How to Use |
|
|
|
The model can be used to fine-tune on a downstream genomic task, e.g. promoter identification. |
|
|
|
```python |
|
import torch |
|
from transformers import BertForSequenceClassification |
|
model = BertForSequenceClassification.from_pretrained('Peltarion/dnabert-minilm') |
|
``` |
|
|
|
More details on how to fine-tune the model, dataset and additional source codes are available on [github.com/joanaapa/Distillation-DNABERT-Promoter](https://github.com/joanaapa/Distillation-DNABERT-Promoter). |
