Astheral V3

Built with Llama. Real 8.03B merged BF16 weights, fine-tuned from Astheral V2. This is a modest instruction-following update, not a claim of broadly improved reasoning or coding.

Measured results

Both final models were evaluated in BF16 with identical prompts and decoding settings.

Suite	V2	V3
Task-family holdout (60 cases)	41/60	43/60
GSM8K test subset	6/8	6/8
HumanEval subset	6/8	6/8

The 60-case suite uses unseen values within training-related task families; it is not a broad benchmark. Public benchmark subsets are small and not evidence of state-of-the-art performance. Raw prompts, outputs, scoring code and failures are included in paired_bf16_outputs.json, holdout_outputs.json and holdout_evaluation.json.

The quantized adapter scored 49/60 before merging; the shipped BF16 weights scored 43/60 after saving and reloading. The shipped score is the relevant result.

Training and reproducibility

A10G-small, listed at $1/hour (USD). Rank-16 NF4 QLoRA updates 41,943,040 adapter parameters. Dataset: 1,713 token-filtered examples; 180 optimizer steps, approximately 0.42 epochs. Includes generated arithmetic/formatting/function examples plus retained V2 data. Source and exact data are in v3/ and v3-results/. Reports include the parent revision, seed, timings and memory use.

V2 parent: d9aa899722be4788f57107c2951c72d0ec6614ee. It remains available at tag v2-before-v3.

Use

Load SEE7/Astheral with Transformers using BF16 on an appropriate GPU. The merged weights require about 16 GB before context/cache overhead. Existing inference.py supports instruct and thinking modes and reads settings.json. For the separate adapter, use the exact V2 parent revision specified in adapter/adapter_config.json, not the V3 merged weights.

Licensing

Llama 3.1 licensing applies. See LICENSE, UPSTREAM_LLAMA3_LICENSE, NOTICE and the existing data attribution files. Retained V2 examples keep their original source terms; synthetic examples are provided with the reproducible generation script.

Extended V3 training and standard evaluation

A second real training pass updated 83,886,080 rank-32 adapter parameters for 360 optimizer steps (822.7 seconds of optimizer training). It was merged, saved and reloaded. The 60-case BF16 task-suite score stayed at 43/60. Four standard lm-evaluation-harness 0.4.13 tasks used the first 100 evaluation examples each, zero-shot BF16 likelihood scoring with default task prompts and no chat template. These are subsets, not full benchmark scores.

Task	Deployed V3 accuracy	Expanded candidate accuracy
ARC-Easy	87/100	86/100
PIQA	79/100	80/100
SciQ	94/100	94/100
WinoGrande	79/100	79/100

The expanded candidate showed no overall improvement, so the existing V3 weights remain deployed. Expanded full weights, training data, source, per-example benchmark results and reports are preserved on branch v3-expanded-20261113. See completion_report.json and expanded-evaluation/. The Space now permits up to 4,096 new tokens, defaults to 1,024, and allows 110 seconds generation inside a 120-second ZeroGPU allocation with the 180-second quota reserve.