Hugging Face's logo

damianborek
/
polaris-2

PEFT
Safetensors
English
lora
classification
agents
decision
nimble
Model card Files
xet
 Community

Polaris 2

Polaris 2 (polaris-2) is the autonoxis decision model: a LoRA adapter for bespokelabs/Bespoke-Nimble-9B (revision 594dfdcfb6f94e3d0c0db7535180d3c71689169a), itself built on Qwen3.5-9B. It makes "conductor" decisions for autonomous coding agents. It does not generate text. Nimble scores the answer tokens and the adapter returns a probability for each allowed label.

It replaces Polaris 1. On 96 real orchestrator packets (v9) it scores 72% (69/96), against 59% (57/96) for Polaris 1, and it answers every packet of v5–v8 correctly.

Two tracks, one question per request:

track labels
decision: what should the conductor do next with this lane? STOP, ASK, DISPATCH
manager: what should happen to the evidence returned for this lane? ACCEPT, VERIFY, REJECT, REOPEN, ESCALATE

The question wording and the label criteria the adapter was trained on are in conductor-questions.json. They differ from Polaris 1's questions, so use this file (or plugin versions that ship it) with this adapter. The adapter's autonoxis.json carries its name (polaris-2), which autonoxis-server reports.

Confidence is not calibrated on real packets. Five v9 mistakes are unsafe (a wrong ACCEPT or DISPATCH) at confidence ≥ 0.99, so the 0.8 gate does not stop them. Treat ACCEPT as "verify first".

Links

Use with the autonoxis plugins

The easiest way to use Polaris 2 is through the autonoxis server and one of its two clients: a Pi extension (pi-autonoxis-model 0.4.0 or later) and a Claude Code plugin (autonoxis-model 0.3.0 or later). Those versions ship the Polaris 2 questions. Both clients send the trained call convention for you, act only at confidence ≥ 0.8 (never lower: Pi uses a fixed 0.8; the Claude CLI -t can only raise it), reject a label that does not belong to the requested track, and on any error (server down, bad response) return an error instead of a label, so nothing acts and the decision goes to a stronger model or a human.

Polaris 1 still works with the older plugin versions (pi-autonoxis-model 0.3.x, autonoxis-model 0.2.x).

1. Run the server. server.py and its full setup (NVIDIA GPU, about 20 GB in bf16; torch, transformers, peft, huggingface_hub) are in pi-autonoxis-model/server:

git clone https://github.com/damian87x/pi-autonoxis-model && cd pi-autonoxis-model
git clone https://github.com/bespokelabsai/nimble server/nimble
hf download bespokelabs/Bespoke-Nimble-9B --revision 594dfdcfb6f94e3d0c0db7535180d3c71689169a --local-dir base
hf download damianborek/polaris-2 --local-dir adapter
echo '{"model_path": "base", "max_input_tokens": 2048}' > nimble-model.json
python server/server.py --model-config nimble-model.json --adapter adapter --port 8765

It listens on http://127.0.0.1:8765 (loopback only, no authentication). Both clients read AUTONOXIS_URL and default to that address. GET /health returns {"status": "ok", "model": "polaris-2"}.

2. Pi.

pi install git:github.com/damian87x/pi-autonoxis-model

Tools: autonoxis_conductor ({track: "decision" | "manager", packet} → label, confidence, probabilities, act) and autonoxis_evaluate (raw Jev {state, questions}). Command: /autonoxis-model status | test.

3. Claude Code.

/plugin marketplace add damian87x/autonoxis-model
/plugin install autonoxis-model@autonoxis-model

(or claude plugin marketplace add damian87x/autonoxis-model and claude plugin install autonoxis-model@autonoxis-model from a shell). Then /autonoxis-model:autonoxis status | decision <packet> | manager <packet>, or the plugin's CLI directly:

python3 scripts/autonoxis.py conductor --track manager --file packet.txt
# exit 0 = ACT (conf >= 0.8), 3 = ESCALATE (below 0.8), 2 = error (never act)

4. Raw HTTP. One question with id label, state {"packet": ...}, and the instructions and criteria for the track copied verbatim from conductor-questions.json. With raw HTTP, apply the 0.8 gate yourself.

How to use without the plugins

You need the Nimble prompt builder and scorer from github.com/bespokelabsai/nimble: nimble.scoring.parallel_schema.prepare_prompts and nimble.training.schema_train.candidate_logits.

The call convention matches training. Use it exactly:

  • ask one question per request, with the field id label;
  • the state is the JSON {"packet": <packet text>};
  • the field's description and choice descriptions come from conductor-questions.json (instructions and criteria for that track).

Load the base model and attach the adapter unmerged (merging in bf16 shifts the probabilities).

import json, torch
from transformers import AutoTokenizer, AutoModelForImageTextToText
from peft import PeftModel
from huggingface_hub import hf_hub_download
from nimble.scoring.parallel_schema import prepare_prompts
from nimble.training.schema_train import candidate_logits

BASE, REV = "bespokelabs/Bespoke-Nimble-9B", "594dfdcfb6f94e3d0c0db7535180d3c71689169a"
ADAPTER = "damianborek/polaris-2"

tok = AutoTokenizer.from_pretrained(ADAPTER)
model = AutoModelForImageTextToText.from_pretrained(BASE, revision=REV, dtype=torch.bfloat16).to("cuda")
model = PeftModel.from_pretrained(model, ADAPTER).eval()   # do not merge
questions = json.load(open(hf_hub_download(ADAPTER, "conductor-questions.json")))

def ask(packet: str, track: str) -> dict:
    q = questions[track]
    field = {"type": "enum", "description": q["instructions"],
             "choices": list(q["criteria"]), "choice_descriptions": q["criteria"]}
    p = prepare_prompts(tok, json.dumps({"packet": packet}, ensure_ascii=False), {"label": field}, 2048)
    ids, cands = p.full_ids[0], p.candidate_ids[0]
    batch = {"input_ids": torch.tensor([ids], device="cuda"),
             "attention_mask": torch.ones(1, len(ids), dtype=torch.long, device="cuda"),
             "candidate_ids": torch.tensor([cands], device="cuda"),
             "candidate_mask": torch.ones(1, len(cands), dtype=torch.bool, device="cuda")}
    with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
        probs = candidate_logits(model, batch)[0].double().softmax(-1).tolist()
    return dict(zip(q["criteria"], probs))

Confidence gate. Compute conf = (n * max_prob - 1) / (n - 1), where n is the number of labels. Act on the top label only when conf >= 0.8. Below that, escalate to a stronger model or a human.

Results

"Unsafe" means the model predicted DISPATCH or ACCEPT where the reference label is different. The evaluation sets are not released. eval-summary.json holds the per-set metrics for this adapter (no packet text).

set Polaris 2 Polaris 1 Polaris 1 + newer plugin questions Jev contract prompt
v5 (40) 40/40 40/40 40/40 not measured
v6 (48) 48/48 48/48 48/48 not measured
v7 (48) 48/48 48/48 48/48 not measured
v8 (60) 60/60 58/60 58/60 60/60
v9 (96, real packets) 69/96 (72%) 57/96 (59%) 64/96 (67%) 70/96 (73%)
  • "Polaris 1" is the released Polaris 1 with its own questions. "Polaris 1 + newer plugin questions" is Polaris 1 as served by pi-autonoxis-model 0.3.0 / autonoxis-model 0.2.0. "Jev contract prompt" is untrained TypeSafe Jev given the same questions Polaris 2 was trained with.
  • v9 is 96 packets from real orchestrator runs (48 decision, 48 manager). Its reference labels are model labels, not human labels: Opus 5.5 and Grok 4.7 labelled every packet blind and agreed on 73; for the 23 disagreements the Opus 5.5 label is used. Polaris 2 gets 63 of the 73 agreed packets and 6 of the 23 disagreement packets. By track: decision 33/48, manager 36/48.
  • On v9, Polaris 2 roughly matches Jev with the same questions (69 vs 70), but runs locally at no per-call cost.
  • v9 unsafe: 5 packets (Polaris 1: 6). Four VERIFY packets answered ACCEPT (confidence 0.9995–1.0) and one ASK packet answered DISPATCH (0.998).
  • v9 with the 0.8 gate: keeps 93 of 96 at 74.2% accuracy. With a 0.95 gate: keeps 87 at 73.6%.
  • v5–v8 reference labels are Fable 5 (claude-fable-5). v8 was never used for training or selection, but it is in-distribution. v5–v7 are not clean held-out sets (training examples were written from earlier mistakes on them). On v5–v8 Polaris 2 makes no unsafe predictions.
  • Four training seeds were run. v9 per seed: 66, 67, 69, 67 (mean 67.25); a probability ensemble of all four also scores 69/96. This adapter was chosen by the v5–v8 total alone (196/196); v9 was never used for selection.

What changed from Polaris 1

  • New question wording for both tracks (the conductor contract prompt that improved untrained Jev on real packets). The adapter is trained on these questions, and the plugins now send them.
  • 320 new training rows: packets written across eight domains (web, games, mobile, infrastructure, data pipelines, docs, ML, mixed), labelled blind by Opus 5.5 and Grok 4.7. They agreed on 318; the other 2 use the Opus 5.5 label. None of them is a v9 packet, and each was checked for text overlap against every evaluation set.
  • Same training recipe as Polaris 1.

Limitations

  • Confident misses. All five v9 unsafe answers are at confidence ≥ 0.99, four of them ACCEPT where VERIFY was expected. The 0.8 gate does not catch them. Treat ACCEPT as "verify first", and keep hard vetoes in code.
  • The gate is weaker on real packets. Above 0.8 confidence, accuracy is 74% on v9, against 100% on v5–v8.
  • Model labels. v9 labels come from two frontier models, and 23 of 96 are contested. Human labels could move the numbers either way.
  • Small seed count. Four seeds (a fifth crashed at start); v9 spread across seeds is 66–69.
  • The questions do much of the work. Polaris 1 given the same new questions without retraining also reaches 69/96 on v9 (60/60 on v8). Compared with that, Polaris 2 makes one fewer unsafe answer on v9 (5 vs 6) and passes more packets through the 0.8 gate (93 vs 88) at about the same accuracy (74.2% vs 75.0%).
  • Prompts are limited to 2048 tokens. Longer packets are rejected, not truncated. English only.
  • Narrow domain: conductor and manager decisions for autonomous coding lanes under one decision contract. The labels will not transfer to other policies.
  • The adapter is trained for the single-question convention above.

Training

  • Recipe (unchanged from Polaris 1): LoRA r=16, alpha=32, dropout 0.05, on the language-model Linear layers (attention, linear-attention and MLP projections). Candidate cross-entropy over the answer tokens. lr 5e-5, effective batch 8, 3 epochs, linear schedule with 10% warmup, bf16.
  • Questions: conductor-questions.json in this repository, for both tracks.
  • Data: 1396 rows. The 1076 Polaris 1 rows (lab history plus drafted contrastive rows aimed at the hard boundaries, with label disagreements adjudicated by Fable 5, claude-fable-5) plus the 320 new rows above (labels by Opus 5.5 and Grok 4.7, Opus 5.5 winning disagreements). No evaluation packet is in the training data (checked at build time).
  • The training data is not released.

License

Apache-2.0, the same as the base model. Bespoke-Nimble-9B is itself a LoRA-merged Qwen/Qwen3.5-9B, which is also Apache-2.0.

Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for damianborek/polaris-2

Base model

Qwen/Qwen3.5-9B-Base
Finetuned
Qwen/Qwen3.5-9B
Adapter
bespokelabs/Bespoke-Nimble-9B
Adapter
(2)
this model