Khide/mini-jev-rlcd

Mini-Jev: Calibrated DevOps Tool-Calling Agent

Mini-Jev is a research project. It is a distilled version of the Beam-1.0-501B model. It is not officially affiliated with or endorsed by the creators of the original Beam model.

The Problem: Overconfidence

Most small models are "overconfident"—they will guess a tool even when they are unsure. In a production DevOps environment, a wrong tool call (like triggering a rollback by mistake) can be catastrophic.

The Solution: RLCD

This model was trained using Reinforcement Learning for Calibrated Decisions (RLCD). Instead of just training for accuracy, we optimized for:

1. Cross-Entropy: Getting the right answer.
2. Brier Score: Ensuring the probability matches the empirical truth.
3. KL Divergence: Distilling the epistemic uncertainty of a 501B parameter teacher.

Calibration

The model is calibrated using post-hoc Temperature Scaling ($T^*$). This ensures that when the model says it is 70% confident, it is actually correct approximately 70% of the time.

Usage (System 1 / System 2 Cascade)

This model is designed to act as System 1.

• If confidence $\hat{p} \ge 0.7$, execute the tool immediately.
• If confidence $\hat{p} < 0.7$, escalate to a larger expert model (System 2).

from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained("khide/mini-jev-rlcd")
FastLanguageModel.for_inference(model)
# Use the calibrated logits to route your requests!