scairesearch/kieve-jev-base

Kieve-JeV (base)

Kieve-JeV is an encoder-based typed decision engine. It takes a state (any text) and a set of typed questions and returns, for every question, one answer with a calibrated probability distribution over the supplied options (or a probability for the yes/no noul primitive).

Two question primitives are supported:

• choice - pick one criteria option; the model emits a full distribution.
• noul - a single yes/no probability (a learned no-answer head).

The model never invents options: the distribution is defined over the options you pass in, so it works with arbitrary, task-specific answer sets.

Architecture

• Backbone: answerdotai/ModernBERT-base (frozen) with LoRA adapters (rank 16).
• Variant: jev - state/question encoders plus a paired cross-attention option scorer and a separate noul head.
• Operators: fused neural operators (use_ops) enabled (MultiScaleFNOField + ImprovedGeoGNP + GateFuse when enhanced is on).
• Trainable parameters: ~21.9M LoRA (rank 16) + heads (total checkpoint parameters: ~273.7M).

Files

File	Description
checkpoint_latest.pt	Training checkpoint. model_state holds the weights; temperature the calibrated softmax temperature.
config.json	Inference config consumed by Kieve (params + fixed).

Usage

As a decision-index engine

python -m decision_index pipeline \
  --engine decision_index.engines.kieve:KieveEngine \
  --option model=scairesearch/kieve-jev-base \
  --out runs/kieve-jev-base

Via the Kieve decision server (/v1/systemone)

python -m kieve.serve \
  --checkpoint checkpoint_latest.pt \
  --config config.json \
  --host 0.0.0.0 --port 8000

Then POST {"state": ..., "questions": [...]} to /v1/systemone.

Evaluation

Trained on the mixed JevBench slices aegis2, atbench500, jailbreak_classification, prompt_injections, scienceqa_text, pubmedqa (3,000 cases, 700 steps, LoRA rank 16, ModernBERT-base tokenizer).

Decision Index public run, edition 0.2.1 (scores.json, complete: true, 150,317 requests scored):

Metric	Value
Public index (chance-corrected, 0-100)	0.67
Raw index	22.29
Latency median / p95 / mean	75 ms / 709 ms / 146 ms

Decision Index

Submitted to the Decision Index board. Public run dataset: scairesearch/decision-index-results-kieve-jev-base. Engine: decision_index.engines.kieve:KieveEngine.

Known issue: this checkpoint was trained on a broken backbone

This checkpoint was trained before the backbone was fixed, and the encoder it contains is effectively random. Three defects were in the backbone:

• the RoPE tables stored the raw rotation angles instead of their cosine and sine, so the "rotation" was q*angle + rotate_half(q)*angle - not a rotation, unbounded in the position index, and it made the residual stream and its gradients grow with depth (gradient norm ~1e10 against ~0.35 for the reference implementation);
• the key-padding mask never matched the collate's pad id, so the encoder attended to padding;
• the MLP width did not match the ModernBERT checkpoint, so the MLP weights were silently dropped.

With gradients clipped to norm 1.0, every update was scaled to nothing: the model could not learn, and the Decision Index public run reflects that (public index 0.67, i.e. chance).

Use scairesearch/kieve-jev-v0.4 instead. It is the first release trained on a working backbone, and it is the one submitted to the board.

License

Apache-2.0. ModernBERT backbone is subject to its own license.