Gemma-Cure — Drug Discovery LoRA Adapter

Gemma 4 E2B fine-tuned on 225K drug–target pairs for novel small-molecule generation.

Built for the Kaggle Gemma 4 Good Hackathon 2026 by DLYog Lab.

---

Model Description

Gemma-Cure takes a protein target name, amino-acid sequence, and measured binding affinity, then generates:

1. A structured scientific rationale (binding pocket analysis, key residues, physicochemical reasoning)
2. A novel SMILES string for a drug-like small molecule

The model is designed for educational drug discovery — explaining reasoning in plain English accessible to early researchers and high school students.

Live Demo

Try it now: Deep2Lead Platform — PathoHunt 3D game uses this model live

---

Training Details

Parameter	Value
Base model	unsloth/gemma-4-E2B-it-unsloth-bnb-4bit
Architecture	Gemma 4 E2B (5.2B parameters)
Adapter type	RS-LoRA (Rank-Stabilised LoRA)
LoRA rank (r)	32
LoRA alpha	64
Trainable params	62M / 5.2B (1.2%)
Training dataset	225,000 drug–target binding pairs
Data sources	BindingDB, ChEMBL, MOSES
Training framework	Unsloth + HuggingFace TRL SFTTrainer
Hardware	NVIDIA GB10 Grace Blackwell (122GB VRAM)
Training time	~3 hours total (iterative runs)
Final LR	2e-5 (cosine scheduler)
Batch size	8 × 16 gradient accumulation = 128 effective
Quantization	4-bit (BnB NF4)

Auto-Evaluation Loop

Training used a custom EvalAndStopCallback that evaluates drug quality every 100 steps on 3 benchmark targets and stops automatically when a pharmaceutical quality gate passes:

• Validity gate: ≥ 67% of generated SMILES must be chemically valid (RDKit)
• QED gate: Average QED (Quantitative Estimate of Drug-likeness) ≥ 0.55

---

Evaluation Results

Evaluated on 3 benchmark drug targets using RDKit SMILES validation and QED scoring:

Target	SMILES	QED
SARS-CoV-2 Main Protease	O=C(O)c1ccc(-n2cc(Nc3ccccc3)cn2)o1	0.761
EGFR Kinase	CN(C)c1ccc(-n2cc(NC(=O)[C@](Cc3ccccc3)N=O)nc2OC)cn1	0.591
BACE1 Alzheimer target	CN(C)c1ccc(-n2cc(N[C@@H]3CC4CCN(CCc5ccccc5Cl)CC4CCC3)nc2O)[nH]1	0.421

Metric	Base Gemma 4	v2 baseline	Gemma-Cure (final)
SMILES validity	0%	33%	100%
Average QED	0.000	0.116	0.591
Composite score	0.000	0.224	0.795

Quality gate passed at step 100 of run 2 (41 minutes of training).

---

How to Load

Requirements

pip install unsloth

Inference

from unsloth import FastModel

model, processor = FastModel.from_pretrained(
    model_name="dlyog/gemma-cure",   # downloads base + adapter automatically
    load_in_4bit=True,
    max_seq_length=2048,
)
FastModel.for_inference(model)

SYSTEM = (
    "You are Deep2Lead's drug discovery AI v2. When given a protein target or biological "
    "context, first reason about the binding pocket geometry, key residues, and desired "
    "physicochemical profile (2-3 sentences), then output novel drug-like SMILES molecules. "
    "Always label your reasoning as 'Rationale:' and your molecules as 'SMILES:'. Explain "
    "choices in plain English suitable for high school students and early researchers. "
    "Prioritize selectivity, low toxicity, and synthetic accessibility."
)

messages = [
    {"role": "system", "content": [{"type": "text", "text": SYSTEM}]},
    {"role": "user",   "content": [{"type": "text", "text": (
        "Target: EGFR Kinase\n"
        "Protein sequence (first 150 AA): MRPSGTAGAALLALLAALCPASRALEEKKVCQGTSNKLTQLGTFEDHFLSLQ"
        "RMFNNCEVVLGNLEITYVQRNYDLSFLKTIQEVAGYVLIALNTVERIPLENLQIIRGNMYYENSYALAVLSNYDANKTGLKELPMRNLQEILHGAVR\n"
        "Measured binding affinity: IC50 = 2.0 nM\n"
        "Design a small molecule drug candidate with high binding affinity to this target. "
        "First explain your structural reasoning, then provide the SMILES.\n"
        "Requirements: MW 200-500 Da, QED > 0.50, SAS <= 5.0, Lipinski Ro5 compliant."
    )}]},
]

inputs = processor.apply_chat_template(
    messages, tokenize=True, return_dict=True,
    return_tensors="pt", add_generation_prompt=True,
).to("cuda")

with __import__("torch").no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=350,
        temperature=0.9,
        top_p=0.92,
        top_k=50,
        repetition_penalty=1.3,
        do_sample=True,
    )

response = processor.decode(
    outputs[0][inputs["input_ids"].shape[1]:],
    skip_special_tokens=True
)
print(response)

Expected Output Format

Rationale: The EGFR kinase active site contains a conserved ATP-binding hinge region
with Cys797 as a key covalent anchor point. A pyrimidine-aniline scaffold provides
optimal hinge binding geometry while maintaining MW within the 200-500 Da window
and favorable LogP for cell penetration.

SMILES: CN(C)c1ccc(-c2nc(Nc3ccccc3F)c(C#N)cn2)cc1

---

Prompt Format

The model expects this exact format (use processor.apply_chat_template):

System: You are Deep2Lead's drug discovery AI v2...

User:
Target: <target name>
Protein sequence (first 150 AA): <sequence>
Measured binding affinity: <Ki/IC50/Kd value>
Design a small molecule drug candidate...
Requirements: MW 200-500 Da, QED > 0.50, SAS ≤ 5.0, Lipinski Ro5 compliant.

---

Dataset

Training data: 225,000 drug–target binding pairs curated from:

Source	Records	Description
BindingDB	~150K	Experimental binding affinities (Ki, IC50, Kd)
ChEMBL	~50K	Bioactive molecules with target annotations
MOSES	~25K	Drug-like SMILES diversity scaffold

Each record: {target_name, protein_sequence_150AA, binding_affinity, smiles, rationale}

---

Technical Notes

• Why RS-LoRA: Uses α/√r scaling (vs standard α/r), stabilising gradients across different rank sizes. With r=32 and α=64, the effective scale is α/√r ≈ 11.3 vs standard α/r = 2.0 — higher signal without the instability of large α.
• Why Unsloth: 2× faster training, 80% less VRAM via custom CUDA kernels and gradient checkpointing optimisations. Enables 4-bit training on the full 5.2B model with only 62M trainable LoRA params.
• repetition_penalty=1.3: Critical for SMILES generation — without it the model loops on fragments like c1c1c1c1.... Matches deployment params.
• TRANSFORMERS_OFFLINE=1: Set during training to use local HF cache and prevent mid-training download attempts.

---

Citation

@misc{gemma-cure-2026,
  title        = {Gemma-Cure: Drug Discovery LoRA Adapter for Gemma 4 E2B},
  author       = {Tarun Kumar Chawdhury},
  year         = {2026},
  howpublished = {HuggingFace Model Hub},
  url          = {https://huggingface.co/dlyog/gemma-cure},
  note         = {Fine-tuned on 225K drug-target pairs. Kaggle Gemma 4 Good Hackathon 2026.}
}

---

License

This adapter is released under the Gemma Terms of Use. The base model weights remain subject to the original Gemma license.