TheJackBright/verisci-autoscientist-adapter

VeriSci AutoScientist Adapter

LoRA adapter trained with Adaption AutoScientist for verifier-grounded scientific reasoning and Math-Code tasks.

Status: diagnostic adapter checkpoint is available and packaged. The clean Mistral run completed at 52.45% best win rate, so this artifact is useful for reproducibility and public eligibility proof, but it is not strong enough to claim as a winning final model without another stronger run or a clearly framed fallback submission.

What It Does

The model is trained to:

• compute unit-checked mechanics and thermodynamics quantities,
• solve circuit power, chemistry, solution concentration, and half-life decay calculations,
• handle unit conversion, vector components, and two-point linear models,
• perform explicit Euler ODE steps,
• apply trapezoid-rule integration,
• update 1D heat-equation finite-difference states,
• classify dimensional consistency,
• write small Python functions that satisfy edge-case tests,
• abstain when a scientific or code prompt is underspecified,
• emit final answers as machine-readable JSON.

Training

Field	Value
Base model requested in Adaption	mistralai/Mistral-7B-Instruct-v0.2
Adapter config base model	togethercomputer/Mistral-7B-Instruct-v0.2
12k full-run recommendation	google/gemma-3-4b-it
Training type	LoRA
Candidate AutoScientist run ID	7ea2c71b-94d7-472d-8288-795ab5e0a2c3
Candidate 8k dataset ID	3103c6ac-7d61-4271-af62-41cb023de85e
Candidate latest status	Succeeded, 5/5 iterations
Candidate final best win rate	52.45%
Completed at	2026-08-07T11:24:15.731Z
Checkpoint SHA-256	e03d0528384ede3ac7e0903c41ccc96ce4a12425874dc271e10fa92193b78070
Diagnostic Llama run ID	43b5486d-0bfc-4e9f-869e-a9892a679386
Diagnostic Llama best win rate	49.48%; not the final candidate
12k source preflight dataset ID	13b1c92a-a96b-4ce7-810b-4368ad6aa234
Max iterations	5
Target win rate	0.82
Final best win rate	52.45%; below target
LoRA rank	16
LoRA alpha	32
Learning rate	1e-5
Epochs	1

Low-credit pilot result: Adaption enhanced_completion output failed the final-JSON preservation audit, while source prompt/completion columns passed. The 12k source preflight also passed row-count and final-JSON audit. The clean candidate run maps AutoScientist to Adaption's raw-upload columns, original_prompt and original_completion.

Evaluation

Evaluation	Base	Adapted	Delta
Adaption held-out category win rate	Pending	Pending	Pending
VeriSci verifier suite accuracy	Pending	Pending	Pending
Numeric/unit tasks	Pending	Pending	Pending
PDE/ODE tasks	Pending	Pending	Pending
Python code tests	Pending	Pending	Pending
Abstention tasks	Pending	Pending	Pending

The public dataset currently reports 8,000 unique prompts, 0 duplicate prompts, 0 train/validation/test prompt leakage rows, and 100% gold-verifier pass rate. The weak pre-fix Llama checkpoint is retained only as diagnostic evidence. The clean Mistral checkpoint is packaged for transparency, but the submission remains gated on stronger final evidence, Kaggle mirrors, and base-vs-adapted verifier results.

Usage

After the adapter files are uploaded, use the LoRA adapter with the base model named in adapter_config.json:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "togethercomputer/Mistral-7B-Instruct-v0.2"
adapter_id = "TheJackBright/verisci-autoscientist-adapter"

tokenizer = AutoTokenizer.from_pretrained(base_id)
base = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")
model = PeftModel.from_pretrained(base, adapter_id)

Intended Use

Research and educational use for scientific reasoning, exact-answer evaluation, and model adaptation experiments.

Out Of Scope

Do not use this model as the sole authority for engineering, lab, medical, legal, financial, or safety-critical decisions.

License

Apache-2.0 for the adapter and generated data. Base model terms also apply.