sn92-gen-v1

Generalist decision model for the REAX subnet (Bittensor SN92). It is a fine-tune of perplexity-ai/pplx-decider-v1.2 at revision cce0f3f80033e9d89d80831d5281fd72124aacce, which is itself a fine-tune of Qwen/Qwen3.8-27B. It keeps that model's protocol-v3 joint inline readout (joint_config.json, decision_config.json, readout.safetensors) and its configuration and tokenizer files unchanged. The readout head (readout.safetensors) is the base's head multiplied by 3, which equals dividing the calibrated temperature by 3: a sharper readout, chosen for the REAX scoring rule, where a yes/no answer counts only when its probability is at least 0.8 or at most 0.2. decision_config.json is the base's.

How it was made

• Training: LoRA on the language model's attention, linear-attention and MLP projections, trained through the checkpoint's own encoder, noncausal backbone and frozen readout head. Loss: cross-entropy of the readout against the gold option with label smoothing 0.05, plus the expected-level error for score questions; anchor rows use cross-entropy against the unmodified model's own answer distribution. AdamW, 16,384-token batches, cosine schedule.
• Runs:
  • g4c: stage 1 (g4, from the unmodified base): LoRA rank 32 (alpha 64), learning rate 5e-05, the checkpoint selected at step 1200 of a one-epoch schedule; 43,685 sn92-specialists train rows (at most 2,500 per category) + 3,000 anchors; stage 2, continued from that checkpoint with a fresh optimizer: LoRA rank 32 (alpha 64), learning rate 2e-05, the checkpoint selected at step 300 of a one-epoch schedule; a replay of 6,000 of stage 1's sn92 rows + 3,000 anchors + 7,200 area rows.
• Merge, in fp32 and stored in bf16: W = W(base) + 1 x dW(g4c).

Data

• decisionmodels/sn92-specialists train rows (revision e2ff3bb099553fd9fd93f806152d5f261579bf3e, CC BY 4.0), 20 categories. Their states are disjoint from the dev split.
• Anchors, to keep the base's general knowledge and arithmetic (self-distillation; the target is the base's own answer distribution): 2,000 questions of the MMLU auxiliary_train split (cais/mmlu, revision c30699e8356da336a370243923dbaf21066bb9fe, MIT) and 1,000 problems of the AQuA-RAT train split (deepmind/aqua_rat, revision 33301c6a050c96af81f63cad5562cb5363e88971, Apache-2.0).
• Area rows (1,800 per area, gold labels), for areas the sn92 categories do not cover: math & dates (our own procedural generator: date differences, weekdays, business days, deadlines, time zones, percent change, unit conversion); tool use (glaiveai/glaive-function-calling-v2, revision e7f4b6456019f5d8bcb991ef0dd67d8ff23221ac, Apache-2.0: which function the assistant called first, and whether it called one); language understanding (MASSIVE 1.1 by Amazon, CC BY 4.0: utterance intents drawn from its 51 locales, rewritten as multiple-choice questions); knowledge (tau/commonsense_qa, revision 94630fe30dad47192a8546eb75f094926d47e155, MIT).

Selection

Stage 1's checkpoint (step 1,200 of 1,595) was chosen by the mean log-probability of the gold option on an 800-row subset of the sn92-specialists dev split; stage 2's (step 300 of 355) on that subset plus 800 held-out area rows. The final model and the readout factor were chosen by reading the candidates out through the validators' joint-readout runtime (reax-subnet v1.2.0-rc10 spec/joint.py: the REAX prompt, the last prompt position, readout rows A.., temperature 2) on 5,750 rows: 150 per category of the dev split, of held-out MMLU and AQuA-RAT test questions, of held-out area rows, of BFCL function-calling questions and of MMLU / MMMLU test questions in 15 languages. REAX score (0.75 x competence + 0.25 x calibration) on those rows: 86.70 with the head x 3.0, against 74.65 for the unmodified base. All these rows are disjoint from the training rows, but they were used to make these choices, so they are not an independent test.

Licence

Apache-2.0 (LICENSE). NOTICE carries the base model's notice and the data attributions.