scairesearch/kieve-jev-large

Kieve-JeV v0.4

Kieve-JeV is an encoder-based typed decision engine. It takes a state (any text) and a set of typed questions and returns, for every question, one answer with a calibrated probability distribution over the supplied options (or a probability for the yes/no noul primitive).

Two question primitives are supported:

• choice - pick one criteria option; the model emits a full distribution.
• noul - a single yes/no probability (a learned no-answer head).

The model never invents options: the distribution is defined over the options you pass in, so it works with arbitrary, task-specific answer sets.

Architecture

• Backbone: answerdotai/ModernBERT-base (frozen) with LoRA adapters (rank 16).
• Variant: jev - state/question encoders plus a paired cross-attention option scorer and a separate noul head.
• Operators: fused neural operators - multi-scale FNO (MultiScaleFNOField), low-rank GNP (ImprovedGeoGNP) and GateFuse. The operator branches are zero-initialised, so the pretrained backbone is preserved at step 0.
• Trainable parameters: 22.0M (total checkpoint parameters: 174.4M).

Files

File	Description
checkpoint_latest.pt	Training checkpoint. model_state holds the weights; temperature the calibrated softmax temperature.
config.json	Inference config consumed by Kieve (params + fixed).

Usage

As a decision-index engine

python -m decision_index pipeline \
  --engine decision_index.engines.kieve:KieveEngine \
  --option model=scairesearch/kieve-jev-v0.4 \
  --out runs/kieve-jev-v0.4

Via the Kieve decision server (/v1/systemone)

python -m kieve.serve \
  --checkpoint checkpoint_latest.pt \
  --config config.json \
  --host 0.0.0.0 --port 8000

Then POST {"state": ..., "questions": [...]} to /v1/systemone.

Evaluation

Trained on 313,908 public multiple-choice cases (MMLU auxiliary train 99,842, MedMCQA 100,000, HellaSwag 39,905, WinoGrande 40,398, SciQ 11,679, CommonsenseQA 9,741, OpenBookQA 4,957, ARC 3,370) plus the JevBench text slices, for 3,500 steps with LoRA rank 16 on a frozen ModernBERT-base and the enhanced operator stack (multi-scale FNO, low-rank GNP, gated fusion).

Held-out multiple-choice evaluation (400 cases per set, never trained on; chance in brackets):

Set	Accuracy	Chance
ARC-Challenge	0.285	0.25
ARC-Easy	0.260	0.25
HellaSwag	0.230	0.25
MedMCQA	0.293	0.25
MMLU	0.258	0.25
OpenBookQA	0.355	0.25
WinoGrande	0.468	0.50
macro average	0.307	0.25

Mixed in-training evaluation (512 rows, 31% accuracy, NLL 1.319, ECE 0.063).

Decision Index

A complete public 0.3 suite run of this model is on the Decision Index board: scairesearch/decision-index-results-kieve-jev-large (140,178 requests scored, scores.json says "complete": true).

Metric	Value
Public index (chance-corrected, 0-100)	1.85
Raw index	21.78
Breadth skill	1.71
Per-area skill	knowledge 0.017, tools 0.059, arts 0.015, retrieval 0.010, language 0.000

For scale: the board's top-10 cutoff is 67.48 and the 400M-class models sit at 22-25, so an encoder of this size is a bottom-of-the-board entry. The earlier Kieve submissions scored 0.67 (chance) because their backbone could not learn; see the model cards of those releases.

Engine: decision_index.engines.kieve:KieveEngine. The engine reads the backbone geometry, the LoRA rank and the operator stack back from the checkpoint, so this model loads with no missing or unexpected tensors.

License

Apache-2.0. ModernBERT backbone is subject to its own license.