Zi-yangChen/PRIME_BPE

VERD： Plant Regulatory Information Mining Engine

This repository hosts a foundation DNA language model specifically pre-trained on plant cis-regulatory elements (CREs). The model is designed to capture sequence patterns and motif features of CREs, enabling downstream applicationssuch as regulatory element identification, functional prediction and de novo design.

GitHub repository: VERD

Model Highlights

Domain-specific pre-training: Trained exclusively on plant cis-regulatory element sequences.

Model size: ~90 MB parameters.

Tokenizer: BPE (Byte Pair Encoding) with a vocabulary size of nearly 4000 tokens.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10.0

Framework versions

• Transformers 4.45.2
• Pytorch 2.5.1
• Datasets 3.6.0
• Tokenizers 0.20.1