FunDLM-Medium-1024bp-Centromere

Model summary

Base architecture
Max input length	1024 bp
Task	Sequence classification (centromere prediction)
Number of classes
Labels

Installation

Environment setup follows the same steps as PlantCAD2.

Install core dependencies

pip install torch==2.12.0 transformers==5.15.0 accelerate==1.14.0 tokenizers==0.22.2
safetensors==0.8.0 huggingface_hub==1.27.0 datasets==3.1.0 peft==0.13.2

Install Mamba/Caduceus-specific dependencies (required — this model uses a

Mamba-based state-space backbone, same as PlantCAD/PlantCAD2)

pip install mamba_ssm==2.3.2.post1 causal_conv1d==1.6.0 --no-build-isolation

FunDLM-Medium-1024bp-Centromere/ ├── code/ │ └── lora_fine_tune.py # CLI entry point: tokenize / train / evaluate / predict ├── dataset/ # TSV data (sequence, label columns), split into train/valid/test └── model/ # Base FunDLM-Medium-1024bp checkpoint (or HF model id)

## Running classification

Classification is run through `lora_fine_tune.py`, which supports four subcommands:
`tokenize`, `train`, `evaluate`, and `predict`. This fine-tunes/evaluates a LoRA adapter
on top of the FunDLM-Medium-1024bp base model for the centromere classification task.

Set up your paths first:

```bash
PROCESSED_DIR="/path/to/FunDLM/centromere/FunDLM-Medium-1024bp"   # output dir for tokenized data + checkpoints
TRAIN_DATA_SRC="/path/to/FunDLM/centromere/dataset/data1024bp"     # train/valid TSVs
TEST_DATA_SRC="/path/to/FunDLM/centromere/dataset/data1024/test"      # test TSVs
EVAL_SCRIPT="/path/to/FunDLM/code/lora_fine_tune.py"
MODEL_PATH="/path/to/FunDLM/models/FunDLM-Medium-1024bp"                # base model checkpoint or HF model id

Step 1: Tokenize the data

Each of train / valid / test needs to be tokenized separately. Input TSVs must have a sequence column and a label column (class labels for classification).

python "$EVAL_SCRIPT" tokenize \
  --data_dir "$PROCESSED_DIR/train.tsv" \
  --output_path "$PROCESSED_DIR/train.parquet" \
  --model_name "$MODEL_PATH" \
  --sequence_length 1024 \
  --task_type classification \
  --seq_column sequence \
  --label_column label

python "$EVAL_SCRIPT" tokenize \
  --data_dir "$PROCESSED_DIR/valid.tsv" \
  --output_path "$PROCESSED_DIR/valid.parquet" \
  --model_name "$MODEL_PATH" \
  --sequence_length 1024 \
  --task_type classification \
  --seq_column sequence \
  --label_column label

python "$EVAL_SCRIPT" tokenize \
  --data_dir "$PROCESSED_DIR/test.tsv" \
  --output_path "$PROCESSED_DIR/test.parquet" \
  --model_name "$MODEL_PATH" \
  --sequence_length 1024 \
  --task_type classification \
  --seq_column sequence \
  --label_column label

Step 2: LoRA fine-tuning

python "$EVAL_SCRIPT" train \
  --train_dir "$PROCESSED_DIR/train.parquet" \
  --valid_dir "$PROCESSED_DIR/valid.parquet" \
  --output_dir "$PROCESSED_DIR" \
  --model_name "$MODEL_PATH" \
  --task_type classification \
  --train_batch_size #train batch size \
  --eval_batch_size #eval batch size \
  --eval_num_samples #eval_num_samples \
  --max_steps #max_steps \
  --gradient_accumulation_steps #gradient_accumulation_steps \
  --learning_rate 1e-5 \
  --warmup_steps #warmup_steps \
  --lr_scheduler_type cosine \
  --weight_decay 0.01 \
  --bf16 True \
  --fp16 False \
  --eval_strategy steps \
  --eval_steps 2000 \
  --save_strategy steps \
  --save_steps 2000 \
  --logging_steps 100 \
  --seed 42 \
  --use_wandb True \
  --wandb_project lora-fine-tune \
  --max_grad_norm 0.5 \
  --remove_unused_columns False

This saves periodic checkpoints under $PROCESSED_DIR/checkpoint-*. --use_wandb True logs training to Weights & Biases — set --use_wandb False to disable, or make sure you're logged in (wandb login) first.

Step 3: Evaluation

Find the most recent checkpoint and evaluate it on the held-out validation set:

export OUTPUT_DIR="$PROCESSED_DIR"

# Find the latest checkpoint
CHECKPOINT_PATH=$(ls -td ${OUTPUT_DIR}/checkpoint-* | head -1)
echo "Using checkpoint: $CHECKPOINT_PATH"

export CUDA_VISIBLE_DEVICES=0

python "$EVAL_SCRIPT" evaluate \
    --checkpoint_dir "$CHECKPOINT_PATH" \
    --data_dir "$PROCESSED_DIR/valid.parquet" \
    --output_dir "$OUTPUT_DIR/eval-valid" \
    --model_name "$MODEL_PATH" \
    --task_type "classification" \
    --batch_size 4 \
    --bf16 True \
    --seed 42

Step 4: Prediction (inference on new/test sequences)

export CUDA_VISIBLE_DEVICES=0

python "$EVAL_SCRIPT" predict \
     --checkpoint_dir "$CHECKPOINT_PATH" \
     --data_dir "$PROCESSED_DIR/test.parquet" \
     --output_file "$OUTPUT_DIR/eval-test/predictions_centromere_medium_1024bp.csv" \
     --model_name "$MODEL_PATH" \
     --task_type "classification" \
     --batch_size 32 \
     --bf16 True \
     --seed 42 \
     --seq_column "sequence"