Ibis0 BED outcome simulator

paper · code · data

The outcome simulator from the Bayesian experimental design (BED) experiments in "Do Language Models Know What Compelling Scientific Evidence Looks Like?". Given a candidate experiment with its result hidden, the hypothesis θ′ that the experiment itself tests, and a requested valence, it generates a plausible outcome: either compelling support for θ′ or a clear null. During experiment selection the two outcomes are weighted by the belief model's p(θ′ | evidence) to estimate expected information gain. The belief model is Ibis0-30B-A3B-Classifier.

Clone and install ibis0

The prompt builder and BED scripts below live in the EdisonScientific/ibis0 code repository. Install Python 3.12 and uv, then clone and install the repository:

git clone --recurse-submodules https://github.com/EdisonScientific/ibis0.git
cd ibis0
uv sync --frozen --all-groups

Run the referenced scripts from this repository with uv run python <script-path>. Local GPU inference requires Linux and a CUDA GPU.

Use

build_theta_prime_sft.py builds the simulator's prompts from an experiment, its hypothesis, and the requested outcome type. bed_thetaprime.py uses the simulator and classifier together to select experiments.

The simulator returns JSON with three fields: supports_hypothesis ("yes" or "no"), weight_of_evidence_decibans, and y, which contains the simulated measurements for the domain.

Licence

Apache-2.0. Fine-tuned from Qwen/Qwen3-30B-A3B (Apache-2.0).

Citation

@article{ibis0,
  title   = {Do Language Models Know What Compelling Scientific Evidence Looks Like?},
  author  = {Igoe, Conor and Grand, Gabriel and Koch, Zane and Bou, Albert and Moss, Eli and Hinks, Michaela M. and Rodriques, Samuel G. and White, Andrew and Narayanan, Siddharth},
  journal = {arXiv preprint arXiv:ARXIV_ID},
  year    = {2026}
}
Downloads last month
2
Safetensors
Model size
31B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for EdisonScientific/Ibis0-BED-Simulator

Finetuned
(88)
this model

Dataset used to train EdisonScientific/Ibis0-BED-Simulator

Collection including EdisonScientific/Ibis0-BED-Simulator