CodonLLM-1.5B

CodonLLM-1.5B is an autoregressive language model fine-tuned for general synonymous codon optimization. The model translates arbitrary protein amino acid sequences into biologically viable DNA coding sequences while strictly preserving 100% amino acid sequence fidelity and open reading frames.

Model Overview

• Model Name: CodonLLM-1.5B
• Base Architecture: unsloth/Qwen2.5-Coder-1.5B-Instruct-bnb-4bit (Qwen2.5-Coder-1.5B)
• Fine-Tuning Method: 4-bit Quantized Low-Rank Adaptation (QLoRA) via Unsloth
• LoRA Parameters: Rank $r=16$, Alpha $\alpha=32$, Dropout $0$
• Target Projections: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Task: Sequence-to-sequence biological translation (Amino Acid $\to$ Synonymous Coding DNA)
• Optimization Scope: General codon optimization across standard genetic code tables

Intended Use

• Translating target protein sequences into viable, synthesis-ready DNA coding sequences.
• Preserving 100% amino acid sequence identity with zero missense or nonsense mutations.
• Modeling context-dependent codon selection and translation cadence rather than static position-independent table substitution.

How to Use

Loading via Unsloth

from unsloth import FastLanguageModel

# Load model and tokenizer
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="<your-hf-username>/CodonLLM-1.5B",
    max_seq_length=2048,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

# Target protein sequence
protein = "MSKGEELFTGVVPILVELDGDVNGHKFSVSGEGEGDATYGKLTLKFICTTGKLPVPWPTLVTTFSYGVQCFSRYPDHMKQHDFFKSAMPEGYVQERTIFFKDDGNYKTRAEVKFEGDTLVNRIELKGIDFKEDGNILGHKLEYNYNSHNVYIMADKQKNGIKVNFKIRHNIEDGSVQLADHYQQNTPIGDGPVLLPDNHYLSTQSALSKDPNEKRDHMVLLEFVTAAGITHGMDELYK"

# Instruction prompt
prompt = f"""Below is an instruction that describes a biological task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
Optimize the following protein sequence into a high-expression DNA coding sequence.

### Input:
{protein}

### Response:
"""

inputs = tokenizer([prompt], return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=492)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print("Generated Coding DNA:\n", response.strip())

Loading via Standard Hugging Face PEFT / Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_name = "unsloth/Qwen2.5-Coder-1.5B-Instruct-bnb-4bit"
adapter_repo = "<your-hf-username>/CodonLLM-1.5B"

tokenizer = AutoTokenizer.from_pretrained(adapter_repo)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
model = PeftModel.from_pretrained(base_model, adapter_repo)

Performance and Validation

• Translation Fidelity: 100% Sequence Identity across benchmark sequences with zero amino acid mismatches.
• Reading Frame Preservation: Strict $3 \times L$ nucleotide length correspondence with intact codon triplets.
• GC Content: Stable distribution within natural physiological bounds (~47% to 52%).
• Memory Footprint: Executable on single consumer GPUs (< 6 GB VRAM).

Training Details

• Training Objective: Cross-entropy loss computed strictly over completion tokens using response-only loss masking (train_on_responses_only).
• Quantization: 4-bit NormalFloat (NF4) with 8-bit AdamW optimizer.
• Context Length: 2048 tokens.

License

This model and its adapter weights are distributed under the Apache License, Version 2.0.