ItsVej/pea-1

Pea 1

Pea 1 is a 2B text decision model. It returns a probability over outcomes you supply. It does not generate text, score answer tokens, or execute tools.

Request access on this page. Downloads stay closed until that request is approved.

This repository is one package:

• The repository root is the serving export. vllm serve ItsVej/pea-1 needs no extra flags.
• checkpoint/ is the native loader checkpoint. Use it with Vej's own loader.

The serving export has the adapter merged into the backbone. The native checkpoint keeps the adapter separate. They belong to the same release and are not byte-identical.

Serve it

vllm serve ItsVej/pea-1

Requires vLLM >= 0.27 and bfloat16. The export is a sequence-classification model with one score per candidate. The endpoint returns raw scores. Apply the softmax over a question's candidates in your own code. A raw score is not a probability and is not comparable across questions.

The position limit is pinned to 2048 tokens per row. Overlength input is an error. Inputs are not silently truncated.

On a GPU shared with a desktop session, set --gpu-memory-utilization to the memory that is actually free.

Use it

Send one request row per candidate to /classify. Each row is a JSON string:

{"state": "<state as text, or canonical JSON>", "instruction": "<the question>", "candidate": "<one candidate>"}

curl localhost:8000/classify -H 'Content-Type: application/json' -d '{
  "model": "ItsVej/pea-1",
  "input": [
    "{\"state\": \"Dashboard is blank after login.\", \"instruction\": \"Which team should handle this?\", \"candidate\": \"incident: Production incident response\"}",
    "{\"state\": \"Dashboard is blank after login.\", \"instruction\": \"Which team should handle this?\", \"candidate\": \"billing: Billing and refunds\"}"
  ]
}'

Repeat the full state in every row. A free-form prompt returns a float that is not a decision.

The native checkpoint can run without vLLM. The hosted API loads pea-1-temperature.json:

vej serve --backend local --checkpoint checkpoint --calibration pea-1-temperature.json --device cpu

Temperature is 1.3335214321633249. It was fitted on 2,026 held-out calibration decisions and checked on 1,016 further decisions. It was not fit on the 14,286-decision test. Omitting --calibration leaves the temperature at 1. The file matches the native checkpoint hashes and is rejected for the merged serving export. Divide vLLM logits by the same temperature before the softmax to use that scale. Top-1 decisions do not change.

That command needs the Vej source package and its locked dependencies. This repository contains the weights and the temperature file, not that source package.

Agreement with the native loader

Measured against the unmerged native loader, over 3 requests, 8 questions and 25 candidates:

metric	value
top-1 decision agreement	1.0
max candidate probability delta	7.15e-3
max Score expectation delta	5.46e-3
max candidate logit delta	1.01e-1

Exact scalars are not portable across dtype, kernel and device. Re-measure on your own target before relying on specific values. Raw logit deltas include a per-question constant shift that cancels in the softmax. The probability deltas are the measure that matches the decision contract.

Held-out test accuracy for the native checkpoint is 76.60% exact match on 14,286 decisions.

Limitations

Temperature 1.334 scales Choice, Noul and Score probabilities from the native loader. A stated probability is still not a guaranteed frequency on a new workflow, and it is not evidence of factual accuracy or instruction following. vLLM scores stay raw logits until you divide them by that temperature. The temperature file does not load on the merged serving export.

Text only. The softmax assigns mass even when every offered candidate is wrong, so include an explicit "none of these" candidate where that matters. Separate per-question answers do not mean the underlying events are independent. Probabilities are not permission to act.

Apache-2.0 covers Vej code and this model adaptation. Upstream model assets and training texts keep their own terms. See LICENSE and NOTICE. The training records are not in this repository.