kartikg1/tevro-decider

Tevro Decider

Tevro scores decisions from a supplied state. It supports multiple-choice, yes/no (noul), and ordinal-score questions.

The architecture combines the first 20 Qwen3-0.6B layers with rank-8 LoRA on the last 14 retained layers and a two-layer bidirectional Tevro post-head (approximately 26.2 million head parameters).

Results

Metric	Development accuracy
Overall	80.90%
Multiple choice	79.45%
Yes/no	88.78%
Ordinal score	69.45%

Evaluated on 5,000 development questions, with results verified after checkpoint reload.

On a separate held-out test split of 5,000 questions never used in development, accuracy is 80.2% (multiple choice 80.8%, yes/no 88.0%, score 65.8%; yes/no AUC 0.953). On the public JevBench set it scores 57.6% (133/231; easy 97.9%, standard 66.7%, hard 34.2%).

See evaluation.json for the full metrics. This development set was used during model development.

Training

Trained on Tevro Decision v1, a 250K-question decision dataset, for 12,000 updates with effective batch size 64. Architecture and training settings are included in config.yaml.

Use

This repository includes the LoRA adapter, custom head in safetensors, and Python inference code. The loader also downloads the Qwen backbone; use the provided loader rather than a standard chat or text-generation pipeline.

pip install huggingface_hub
hf download kartikg1/tevro-decider --local-dir tevro-decider
cd tevro-decider
pip install -r requirements.txt
python predict.py example.json --device cpu
# Optional, on an available CUDA device:
python predict.py example.json --device cuda

Review the included source before running it. CPU loading and single-example inference were smoke-tested. choice returns an option key, noul a yes probability, and score an expected ordinal value.

Confidence is uncalibrated and no abstention threshold was fitted. Validate the model on representative application data before deployment.

Files

• head.safetensors and lora/: trained weights.
• meta.json and config.yaml: architecture and training configuration.
• predict.py: single-question inference entry point.
• evaluation.json: checkpoint reload evaluation.
• release_manifest.json: payload SHA-256 hashes.
• NOTICE.md: dependency notices.