You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

FORM-8B: A Fine-Tuned Specialist for FORM Symbolic Algebra Code Generation

Model description

FORM-8B is a QLoRA fine-tune of Qwen3-8B (unsloth/qwen3-8b-unsloth-bnb-4bit) specialised for zero-shot FORM code generation. FORM is the symbolic manipulation language used for large perturbative QFT calculations (Grassmann variables: multi-loop Feynman diagram algebra).

The model is trained exclusively on execution-verified data: every training example was validated by running the generated FORM program through the FORM binary (deterministic-output match) plus an LLM consistency audit.

Available adapter variants

Adapter	Training data	Size
`adapters/form-qwen3-8b-v3c`	full mix (2,465 deterministic + 2,168 open/Q&A/tutorial) — recommended	178 MB
`adapters/form-qwen3-8b-v3cn`	full mix, no docs-in-context rows	178 MB
`adapters/form-qwen3-8b-v3b`	deterministic-only (2,465 examples)	178 MB

Training config: QLoRA r=16, alpha=32, lr=2e-4, 5 epochs, 4-bit NF4 quantised base.

Training data: the training corpus is not included in this release; it is available from the author upon request.

Intended use

Generating complete FORM programs from plain-English instructions
Assisting physicists with symbolic-algebra scripting (gamma traces, tensor contractions, pattern matching, preprocessor metaprogramming)
Agentic workflows: pair with a FORM execution sandbox and route generation/verification to the model

Out of scope: general chat, non-FORM languages, guaranteed mathematical correctness (generation is syntax-verified at inference time by the caller).

Performance (see paper, Table 3)

All numbers from the revalidated (v2) benchmarks; frontier models evaluated with docs, FORM-8B zero-shot.

Benchmark	FORM-8B (v3c)	GLM-5.3 +docs	GLM-5.2 756B +docs	DeepSeek-V4 Flash 304B +docs
664 deterministic tasks (output match)	97.7%	67.8%	75.5%	62.3%
100 open-ended tasks, v2 (syntax pass)	83.0%	65.0%	55.0%	48.0%
89 output-determined of those (strict output match, lower bound)	18.0%	5.6%	5.6%	3.4%
44 tutorial tasks, v2 (syntax pass)	43.2%	54.5%	31.8%	25.0%
44 tutorial tasks, v2 (strict output match, 41 adjudicable)	14.6%	19.5%	22.0%	14.6%
32 open-ended tasks (rubric judge)	56.2%	65.6%	31.2%	53.1%
22 output-determined open-ended tasks (strict output match)	50.0%	59.1%	50.0%	54.5%
32 open-ended tasks, no docs (rubric judge)	56.2%	46.9%	40.6%	37.5%

Under the strict output-match metric, apparent tutorial differences between models are statistically insignificant (overlapping 95% CIs); on the 32-task rubric benchmark the differences between FORM-8B, GLM-5.3 and DeepSeek-V4 Flash are likewise within statistical noise, and the strict re-adjudication of its 22 output-determined tasks confirms GLM-5.3's lead there is genuine (13/22 solved vs. our 11/22, overlapping CIs). The Instruct-100 strict rate is a conservative lower bound: many of its instructions underspecify the target computation, so a correct-in-spirit program can fail the check (the same grading is applied to all models).

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained(
    "unsloth/qwen3-8b-unsloth-bnb-4bit", load_in_4bit=True, device_map="auto")
model = PeftModel.from_pretrained(base, "bakar-chargeishvili/FORM-8B",
                                  subfolder="adapters/form-qwen3-8b-v3c")
tok = AutoTokenizer.from_pretrained("unsloth/qwen3-8b-unsloth-bnb-4bit")

msgs = [{"role": "user", "content":
         "Compute the trace of a product of gamma matrices contracted with "
         "two external vectors, and simplify using the metric tensor in 4D."}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True,
                               enable_thinking=False)

Note: thinking mode is disabled — the model directly emits FORM code.

Limitations

Trained on synthetic + tutorial-derived data; may not cover exotic FORM features (parallel/local expressions, #procedure edge cases)
Output is syntactically valid FORM but semantic correctness must be verified by executing the program (this is by design — see paper §3)
General reasoning/coding capability preserved to within ~2.6 pp of base (MMLU, GSM8K, HumanEval) but not unchanged

License

Adapters inherit Apache-2.0 from the Qwen3-8B base model. Benchmark data: CC-BY-4.0.

Citation

See the accompanying paper: "Verification-Driven Fine-Tuning of Small Language Models for Zero-Shot FORM Symbolic Algebra Code Generation."

Downloads last month: -