TxGravity-30B-A5B
Model Summary
TxGravity-30B-A5B is a therapeutics-focused language model fine-tuned from the Gravity-30B-A5B-base. It is trained to predict a broad range of therapeutic properties โ small-molecule ADMET, toxicity, drugโtarget interaction, proteinโprotein and peptideโMHC interaction, and more โ following the instruction format of the Therapeutic Data Commons (TDC) benchmark.
The model is supervised-fine-tuned on the TDC therapeutic instruction-tuning data (single-turn
instruction โ answer), covering 57 tasks. Answers are formatted as (A)/(B) for binary
classification or a normalized 000โ1000 bin for regression.
| Property | Value |
|---|---|
| Total Parameters | 29.56B |
| Active Parameters | 5.34B |
| Architecture | GravityMoE (DeepSeek-V3-compatible) |
| Layers | 52 |
| Routed / Shared Experts | 64 (top-8) / 1 |
| Context Length | 8,192 tokens |
| Precision | bf16 |
| Base model | Gravity-30B-A5B |
โ ๏ธ This is a task-specialized property predictor, not a general instruction-tuned or safety-aligned assistant. Its outputs are intended for the TDC-style therapeutic prediction prompts it was trained on. Predictions may be inaccurate, biased, or incomplete and must be independently verified before any experimental, clinical, or decision-making use.
Training
- Base / init: Gravity 30B base checkpoint.
- Data: TDC therapeutic instruction-tuning set (single-turn), 57 tasks. Nine tasks without a paper-reported comparison / generation-only tasks were excluded from training.
- Objective: supervised fine-tuning (next-token, assistant span only), sequence packing at seq_len 8,192.
- Epochs: 2.
- Optimizer: Muon, lr 2e-5, cosine schedule (min-ratio 0.1), warmup 50 steps, grad clip 1.0, weight decay 0.
- Compute: 1ร node of 8 GPUs, FSDP (dp_shard 8), full activation checkpointing, bf16.
Evaluation
Evaluated on the Therapeutic Data Commons (TDC) test sets with a corrected, generation-based
pipeline: classification tasks are scored by generating an answer and reading the first-token
log-probability of (A)/(B) (โ AUROC / AUPRC / Accuracy); regression tasks parse the
predicted normalized bin (โ PCC / MAE / Spearman). Both columns are measured with this same
pipeline.
Nine TDC tasks (disgenet, gdsc1, gdsc2, mirtarbase, phase1/2/3, uspto, uspto_yields) were excluded
from training, so they are omitted here.
On the 57 evaluated TDC tasks, TxGravity-30B-A5B outperforms TxGemma-27B on 53/57 tasks.
Best value per row in bold (โ = higher is better, โ = lower is better).
| Task | Metric | TxGemma-27B | TxGravity-30B-A5B |
|---|---|---|---|
| ADME / Pharmacokinetics | |||
| BBB Penetration (Martins) | AUROC โ | 0.805 | 0.869 |
| Bioavailability (Ma) | AUROC โ | 0.642 | 0.707 |
| Caco-2 (Wang) | MAE โ | 0.613 | 0.434 |
| Clearance Hepatocyte (AZ) | Spearman โ | 0.287 | 0.387 |
| Clearance Microsome (AZ) | Spearman โ | 0.281 | 0.502 |
| CYP1A2 Inhibition (Veith) | AUPRC โ | 0.876 | 0.909 |
| CYP2C19 Inhibition (Veith) | AUROC โ | 0.790 | 0.898 |
| CYP2C9 Inhibition (Veith) | AUPRC โ | 0.579 | 0.800 |
| CYP2C9 Substrate (CarbonMangels) | AUPRC โ | 0.275 | 0.438 |
| CYP2D6 Inhibition (Veith) | AUPRC โ | 0.490 | 0.688 |
| CYP2D6 Substrate (CarbonMangels) | AUPRC โ | 0.628 | 0.697 |
| CYP3A4 Inhibition (Veith) | AUPRC โ | 0.726 | 0.834 |
| CYP3A4 Substrate (CarbonMangels) | AUROC โ | 0.573 | 0.599 |
| Half Life (Obach) | Spearman โ | 0.153 | 0.450 |
| HIA (Hou) | AUROC โ | 0.861 | 0.979 |
| Lipophilicity (AZ) | MAE โ | 0.838 | 0.611 |
| PAMPA Permeability (NCATS) | AUROC โ | 0.618 | 0.707 |
| Pgp Inhibition (Broccatelli) | AUROC โ | 0.822 | 0.897 |
| PPBR (AZ) | MAE โ | 12.249 | 8.078 |
| Solubility (AqSolDB) | MAE โ | 1.110 | 0.942 |
| VDss (Lombardo) | Spearman โ | 0.351 | 0.516 |
| Toxicity | |||
| Ames Mutagenicity | AUROC โ | 0.720 | 0.821 |
| Carcinogens (Lagunin) | Accuracy โ | 0.786 | 0.839 |
| ClinTox | AUROC โ | 0.566 | 0.698 |
| DILI | AUROC โ | 0.682 | 0.867 |
| hERG | AUROC โ | 0.726 | 0.866 |
| hERG (Karim) | Accuracy โ | 0.676 | 0.780 |
| hERG Central | AUROC โ | 0.786 | 0.873 |
| LD50 (Zhu) | MAE โ | 0.975 | 0.699 |
| Skin Reaction | AUROC โ | 0.689 | 0.600 |
| Tox21 | AUROC โ | 0.782 | 0.844 |
| ToxCast | AUROC โ | 0.821 | 0.882 |
| DrugโTarget Interaction | |||
| BindingDB IC50 | Spearman โ | 0.404 | 0.780 |
| BindingDB Kd | PCC โ | 0.258 | 0.647 |
| BindingDB Ki | PCC โ | 0.002 | 0.736 |
| BindingDB Patent | PCC โ | 0.007 | 0.578 |
| DAVIS | MSE โ | 2.296 | 0.752 |
| KIBA | MSE โ | 1.101 | 1.213 |
| High-Throughput Screening | |||
| Butkiewicz HTS | AUROC โ | 0.705 | 0.717 |
| HIV | AUROC โ | 0.707 | 0.758 |
| SARS-CoV-2 3CLPro (Diamond) | AUROC โ | 0.692 | 0.713 |
| SARS-CoV-2 in vitro (Touret) | AUROC โ | 0.511 | 0.531 |
| Drug Synergy | |||
| DrugComb Bliss | MAE โ | 4.491 | 3.390 |
| DrugComb CSS | MAE โ | 11.556 | 6.615 |
| DrugComb HSA | MAE โ | 4.405 | 3.266 |
| DrugComb Loewe | MAE โ | 11.114 | 5.793 |
| DrugComb ZIP | MAE โ | 3.936 | 2.739 |
| OncoPolyPharmacology | PCC โ | 0.330 | 0.552 |
| PeptideโMHC | |||
| MHC-I (IEDB/Nielsen) | AUROC โ | 0.929 | 0.913 |
| MHC-II (IEDB/Jensen) | AUROC โ | 0.781 | 0.858 |
| Antibody | |||
| Protein SAbDab | MAE โ | 1.680 | 0.975 |
| SAbDab (Chen) | AUPRC โ | 0.349 | 0.560 |
| TAP (Antibody) | MAE โ | 5.796 | 6.032 |
| ProteinโProtein | |||
| HuRI (PPI) | AUPRC โ | 0.618 | 0.760 |
| Gene Editing | |||
| CRISPR Repair (Leenay) | Spearman โ | 0.087 | 0.216 |
| Reaction | |||
| Buchwald-Hartwig Yield | PCC โ | 0.512 | 0.933 |
| Misc | |||
| Weber (ADR) | AUROC โ | 0.676 | 0.719 |
Quickstart
Installation
pip install "transformers>=4.45" accelerate safetensors
Using Transformers
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "trillionlabs/TxGravity-30B-A5B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id, torch_dtype=torch.bfloat16, device_map="auto"
)
# TxGravity expects TDC-style therapeutic prompts (Instruction / Context / Question).
prompt = (
"Instructions: Answer the following question about drug properties.\n"
"Context: As a membrane separating circulating blood and brain extracellular fluid, "
"the blood-brain barrier (BBB) is the protection layer that blocks most foreign drugs "
"from reaching the brain.\n"
"Question: Given a drug SMILES string, predict whether it\n"
"(A) does not cross the BBB (B) crosses the BBB\n"
"Drug SMILES: <smiles>CC(C)Cc1ccc(cc1)C(C)C(=O)O</smiles>\n"
"Answer:"
)
messages = [{"role": "user", "content": prompt}]
inputs = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)
out = model.generate(inputs, max_new_tokens=8, do_sample=False)
print(tokenizer.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))
Limitations
- Task-specialized: it targets the TDC therapeutic prediction prompts it was trained on and is not a general conversational assistant.
- Predictions are model estimates and may be wrong; do not use for clinical or experimental decisions without independent validation.
Acknowledgements
This work was supported by the AI Specialized Foundation Model Project (์ธ๊ณต์ง๋ฅ ํนํ ํ์ด๋ฐ์ด์ ๋ชจ๋ธ ํ๋ก์ ํธ), funded by the Ministry of Science and ICT (๊ณผํ๊ธฐ์ ์ ๋ณดํต์ ๋ถ, MSIT) and managed by the National IT Industry Promotion Agency (NIPA, ์ ๋ณดํต์ ์ฐ์ ์งํฅ์).
License
This model is released under the Apache License 2.0.
Citation
@misc{trillionlabs2026txgravity,
title = {TxGravity-30B-A5B},
author = {Trillion Labs},
year = {2026},
url = {https://huggingface.co/trillionlabs/TxGravity-30B-A5B}
}
Contact
For questions, please contact Trillion Labs.
- Downloads last month
- 11