GenoJEPA-Tiny

GenoJEPA-Tiny is the lightweight checkpoint of GenoJEPA, a genomic representation learning framework based on joint-embedding predictive architecture.

GenoJEPA learns semantic representations of DNA sequences through latent-space alignment instead of nucleotide-level reconstruction. The Tiny model is designed for efficient embedding extraction, probing, and downstream genomic sequence classification.

Model usage

from transformers import AutoModel, AutoTokenizer

model_name = "ChengsenWang/GenoJEPA-Tiny"

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModel.from_pretrained(model_name, trust_remote_code=True)

sequences = [
    "ACGTACGTNNNNACGTACGT",
    "TTGCAAGTCCGATCGATCGA",
]

embeddings = model.encode(sequences, tokenizer=tokenizer, batch_size=64)
print(embeddings.shape)

The extracted embeddings can be used with lightweight classifiers such as Logistic Regression for downstream genomic prediction tasks.

For full training, evaluation, and probing scripts, please refer to the GitHub repository.

Citation

If you use this model, please cite GenoJEPA:

@article{
  title   = {From nucleotides to semantics: genomic representation learning via joint-embedding predictive architecture},
  author  = {Wang, Chengsen and Qi, Qi and Sun, Haifeng and Zhuang, Zirui and He, Bo and Liu, Siying and Liao, Jianxin and Wang, Jingyu},
  journal = {bioRxiv},
  year    = {2026}
}