Ibis0-30B-A3B (classifier)

paper · code · data

Ibis0 models read a biological hypothesis together with a set of experimental results (evidence stubs from GWAS Catalog associations, DepMap CRISPR screens and Tahoe-100M perturbation data) and report the probability that a held-out measurement supports the hypothesis. They are trained on EDB-Bio v1.0.0.

This model is Qwen3-30B-A3B with a scalar head on the last token, trained with binary cross-entropy.

Branches

Branch	Training domains	Step
`main`	Tahoe, DepMap, GWAS	2400
`holdout-tahoe`	DepMap, GWAS	1700
`holdout-depmap`	Tahoe, GWAS	1300
`holdout-gwas`	Tahoe, DepMap	1800

The leave-one-domain-out branches give the paper's transfer results. Load one with revision="holdout-tahoe".

Use

The prompt-building code lives in EdisonScientific/ibis0. Clone and install it first (Python 3.12 and uv required; running this classifier locally requires Linux and a CUDA GPU):

git clone --recurse-submodules https://github.com/EdisonScientific/ibis0.git
cd ibis0
uv sync --frozen --all-groups

uv sync installs the local ibis_env_utils package, which loads EDB-Bio datasets, and the domain packages tahoe_meta, gwas_meta, and depmap_meta, which render episodes into the system and user messages used during training. These packages come from the code repository.

The architecture is defined in modeling_ibis0.py in this model repository, so loading requires trust_remote_code=True. Save the following as example.py in the cloned code repository and run uv run python example.py. This builds a Tahoe test prompt and predicts its probability with the classifier (about 61 GB of bf16 weights; use an 80 GB+ GPU):

import tahoe_meta as env
import torch
from ibis_env_utils import Split, load_dataset_split
from transformers import AutoModelForSequenceClassification, AutoTokenizer

df = load_dataset_split("hf://EdisonScientific/EDB-Bio/testsets/primary/tahoe", Split.TEST, revision="v1.0.0")
row = df.iloc[0]
record = env.build_record(row, len(row["context_pool"]), prompt_style="direct")
messages = [
    {"role": "system", "content": env.system_prompt_for("direct")},
    {"role": "user", "content": record["question"]},
]

repo = "EdisonScientific/Ibis0-30B-A3B-Classifier"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(
    repo, trust_remote_code=True, dtype=torch.bfloat16, attn_implementation="flash_attention_2"
).to("cuda").eval()
ids = tok.apply_chat_template(
    messages, add_generation_prompt=True, enable_thinking=False,
    return_tensors="pt", return_dict=True,
)["input_ids"].to(model.device)
with torch.inference_mode():
    p = torch.sigmoid(model(input_ids=ids).logits.float()).item()
print(p)

For another domain, use import gwas_meta as env or import depmap_meta as env and change the dataset path's final component to gwas or depmap respectively.

scripts/export/predict_classifier.py scores the published test set. bf16 MoE inference is not bit-deterministic: repeated evaluations of the same checkpoint differ per episode (logit s.d. about 0.2–0.4), while aggregate metrics agree closely.

Licence

Apache-2.0. Fine-tuned from Qwen/Qwen3-30B-A3B (Apache-2.0).

Citation

@article{ibis0,
  title   = {Do Language Models Know What Compelling Scientific Evidence Looks Like?},
  author  = {Igoe, Conor and Grand, Gabriel and Koch, Zane and Bou, Albert and Moss, Eli and Hinks, Michaela M. and Rodriques, Samuel G. and White, Andrew and Narayanan, Siddharth},
  journal = {arXiv preprint arXiv:ARXIV_ID},
  year    = {2026}
}