xxl0001/TFBS-COD-PlantGFM

COD PlantGFM for Arabidopsis TFBS Classification

COD PlantGFM is a multi-label transcription factor binding-site (TFBS) classifier for Arabidopsis thaliana. It predicts binding probabilities for 359 transcription factors from a DNA sequence.

The model is the distilled student from the MTEDF TFBS classification experiment. It uses the original PlantGFM backbone plus a mean-pooling classification head (LayerNorm(1024) + Linear(1024, 359)).

Distillation setup

• Teachers: PlantGFM, PlantBiMoE, and NTv3-650M
• Soft target: arithmetic mean of the three teacher probabilities
• Student initialization: original PlantGFM backbone
• Objective: 0.8 * BCE(soft targets) + 0.2 * BCE(hard labels)
• Selected checkpoint: epoch 7
• Selection metric: median PRAUC on the difficult low124 TF subset

Results

Split / scope	Mean AUC	Median AUC	Mean PRAUC	Median PRAUC
Validation, all 359	0.980266	0.987306	0.765171	0.819582
Validation, low124	0.967084	0.978159	0.607084	0.649835
Test, all 359	0.974748	0.986734	0.743845	0.802300
Test, low124	0.961044	0.979873	0.577787	0.610877

The test split contains 54,866 sequences. low124 contains the 124 TFs for which the reference deepTFBS-ML PRAUC is below 0.5.

Usage

This repository contains custom PlantGFM model code. Review the code and load with trust_remote_code=True.

import torch
from transformers import AutoModel, AutoTokenizer

repo_id = "xxl0001/TFBS-COD-PlantGFM"
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModel.from_pretrained(repo_id, trust_remote_code=True).eval()

dna = "ACGTN" * 100
# Raw continuous DNA is accepted; spacing is handled by the custom tokenizer.
inputs = tokenizer(
    dna,
    return_tensors="pt",
    padding="max_length",
    truncation=True,
    max_length=1024,
)

with torch.no_grad():
    logits = model(
        input_ids=inputs["input_ids"],
        attention_mask=inputs["attention_mask"],
    ).logits
probabilities = logits.sigmoid()[0]

# Output index -> Arabidopsis TF gene identifier.
id2label = model.config.id2label
top = torch.topk(probabilities, k=10)
for score, index in zip(top.values.tolist(), top.indices.tolist()):
    print(id2label[index], score)

See load_example.py for a complete local/Hub example.

Input and output

• Input alphabet: A, C, G, T, N
• Input format: raw continuous DNA, e.g. "ACGTN"; the custom tokenizer transparently converts it to PlantGFM's single-nucleotide token format
• Fine-tuning maximum length: 1,024 tokenizer positions
• Output: logits with shape (batch_size, 359)
• Probabilities: apply sigmoid independently to all 359 logits
• Label mapping: config.json contains id2label and label2id

The repository's CODPlantGFMTokenizer removes whitespace, converts DNA to uppercase, validates the A/C/G/T/N alphabet, and inserts the single-nucleotide separators expected by PlantGFM. N is encoded with PlantGFM's native token ID 23. The tokenizer supports both one sequence and batches of sequences. This model performs multi-label classification; do not apply softmax across TFs.

Model files

model.safetensors contains only inference model weights. Optimizer and scheduler state from the original 2.6 GB training checkpoint were deliberately removed. The released weights are about 943 MB.

Limitations

• The model was trained and evaluated on the archived Arabidopsis TFBS dataset; performance may not transfer to other species or assay distributions.
• The 359 outputs are specific to the TF ordering stored in config.json.
• Training used sequences truncated/padded to 1,024 positions even though the PlantGFM backbone supports longer contexts.
• Predicted probabilities are not guaranteed to be calibrated.

Provenance

• Backbone: hu-lab/PlantGFM
• Upstream code: hu-lab-PlantGLM/PlantGLM
• Task code: deepTFBS-based MTEDF experiment archive
• Checkpoint source: selected epoch-7 COD PlantGFM student

License

The companion deepTFBS code is GPL-3.0. That code license must not be assumed to cover PlantGFM weights, the fine-tuned weights, or the dataset. The archived PlantGFM model card did not state an explicit model-weight license. Confirm the PlantGFM and dataset terms before publishing this repository.