oidlabs/cue-reflex-4B

CueReflex-E4B

CueReflex-E4B is a fast, calibrated System-1 decision model fine-tuned from google/gemma-4-E4B-it. Given a structured or unstructured state, a decision question, and bounded criteria/options, it returns a calibrated probability distribution over the options in a single forward pass without text generation.

v0.1 Release Goal

This initial release aims to establish a reproducible baseline for independent evaluation and future improvements, rather than claim state-of-the-art performance. The reported JevBench public results are local diagnostics, not official leaderboard scores. The inference runtime and reproduction instructions are forthcoming. Future improvements will be released as separately versioned checkpoints.

Model Summary

• Developed by: OidLabs Team
• Base model: google/gemma-4-E4B-it
• Architecture: LoRA adapter (r=32, alpha=64) + frozen answer-letter readout (decision_head.pt) with logit softcapping (30.0)
• Supported decision types:
  • choice: Categorical selection over 2 to 16 options
  • noul: Binary verification (no / yes)
  • score: Ordinal grading across ordered rubric levels (0 .. K-1)
• Max sequence length: 4,096 tokens
• Max options: 16 (A–P)
• Temperature calibration: Identity (T = 1.0 across choice, noul, and score, selected on held-out validation with a no-sharpening T >= 1.0 policy)
• License: Apache-2.0

Training & Contamination Disclosure

• Training data: Fine-tuned on 76,800 curated, commercial-friendly structured decision examples covering choice, noul, and score tasks.
• External benchmark evaluation: The 231 public JevBench items were used for external diagnostic evaluation, not included in training or used to fit model parameters or temperature calibration.

Diagnostic Evaluation (JevBench Public 231)

Local diagnostic results on the 231 public JevBench items (original.jsonl, easy.jsonl, hard.jsonl):

Metric	Zero-Shot (gemma-4-E4B-it)	CueReflex-E4B
Accuracy	64.94% (150 / 231)	80.52% (186 / 231)
Log Loss	1.1725	0.4738
Brier Score	0.5639	0.2607
ECE (10-bin)	0.2569	0.0684
Coverage	100.0%	100.0%

Accuracy by Public Tier

Tier	Items	Zero-Shot	CueReflex-E4B
Easy	48	48 / 48 (100.00%)	48 / 48 (100.00%)
Original	72	65 / 72 (90.28%)	72 / 72 (100.00%)
Hard	111	37 / 111 (33.33%)	66 / 111 (59.46%)