samatv256/mini-Jev

ODM Mini v1

Open-weight local Choice decision head for agent tool selection, built on frozen Qwen3-0.6B.

ODM Mini takes an agent state plus candidate action descriptions and directly ranks the candidates. It produces candidate scores, grouped-softmax probabilities, a selected action, confidence, decision margin, and measured latency. It does not generate text.

Model architecture

Frozen Qwen3-0.6B
→ candidate-description mean pooling
→ Linear(1024,256)
→ GELU
→ Linear(256,1)
→ grouped softmax

Only description-token hidden states are pooled. The candidate ID, prompt headers, state tokens, and padding are excluded from the pooling mask. The Qwen backbone remains frozen.

Size and required backbone

The ODM-specific trained DecisionHead is approximately 1.1 MB. The complete runnable model is substantially larger because it additionally requires the separately downloaded Qwen/Qwen3-0.6B backbone. This repository does not redistribute Qwen weights.

Installation and usage

pip install torch transformers safetensors huggingface_hub

The code below downloads this five-file release, imports its deterministic loader without trust_remote_code=True, separately obtains Qwen3-0.6B, and performs a Choice:

import sys
from pathlib import Path

from huggingface_hub import snapshot_download

release_dir = Path(
    snapshot_download(
        repo_id="samatv256/mini-Jev",
        allow_patterns=[
            "README.md",
            "model.safetensors",
            "config.json",
            "odm_mini.py",
            "LICENSE",
        ],
    )
)
sys.path.insert(0, str(release_dir))

from odm_mini import ODMMiniModel

model = ODMMiniModel.from_pretrained("samatv256/mini-Jev")

state = {
    "user_request": "Find the current weather in Boston.",
    "available_context": "The user has not provided weather data.",
}
candidates = [
    {
        "id": "weather.lookup",
        "description": "Look up current weather for a specified city.",
    },
    {
        "id": "calendar.list",
        "description": "List upcoming calendar events for the user.",
    },
    {
        "id": "control.finish",
        "description": "Finish because the request has already been completed.",
    },
]

choice = model.predict_choice(state=state, candidates=candidates)
print("selected candidate:", choice.selected)
print("probabilities:", choice.probabilities)
print("confidence:", choice.confidence)
print("decision margin:", choice.decision_margin)
print("latency (ms):", choice.latency_ms)

ODMMiniModel.from_pretrained() accepts either the Hub model ID or a local snapshot directory. On CUDA it defaults to BF16 backbone inference; on CPU it defaults to FP32. The DecisionHead is always evaluated in FP32.

Training

• 50,000 synthetic training decisions
• frozen Qwen/Qwen3-0.6B backbone
• DecisionHead-only training
• grouped cross-entropy objective
• selected seed: 41

The selected head is the seed-41 epoch-4 checkpoint. Temperature remains at 1.0; its maximum grouped-softmax output is reported as confidence, not as a universally calibrated probability.

Evaluation

Synthetic held-out evaluation:

Metric	Result
Semantic Choice accuracy	72.97%
Stress Choice accuracy	67.64%
Counterfactual pair consistency	67.12%

Performance measurements used an NVIDIA GH200, BF16 backbone inference, and the shared-prefix KV-cache serving path. For short and medium contexts of approximately 256–1,024 state tokens, total latency was approximately 76–85 ms for 3–16 candidates (measured range: 76.11–82.36 ms). Offline representation extraction reached 831.3 candidates/second at candidate batch size 512 while caching 430,072 candidates. These local measurements are hardware- and workload-specific and are not a direct speed comparison with Jev or any hosted service.

Limitations and intended use

ODM Mini v1 is a research/hackathon prototype and is not ready for unmonitored production agent control.

In a real shadow-agent evaluation covering 75 multi-step trajectories and 243 decisions, ODM Mini achieved:

• action accuracy: 27.98%
• controller agreement: 26.34%

The major known failure is high-confidence premature completion on intermediate multi-step trajectories. After partial progress, the model can over-index on successful receipts and choose control.finish before remaining steps have been completed. Training used synthetic, static decision snapshots, so the published metrics should not be assumed to transfer to arbitrary tools, domains, or agent loops.

Recommended uses are research, offline evaluation, single-step Choice experiments, and shadow-mode analysis with an independent controller. Do not use this checkpoint as the sole decision-maker for consequential actions or autonomous production control.

Released files

• model.safetensors — ODM Mini DecisionHead tensors only
• config.json — architecture and backbone reference
• odm_mini.py — inference-only loader
• README.md — model card and working example
• LICENSE — Apache License 2.0

License

ODM Mini-specific code and weights in this repository are released under Apache-2.0. The separately downloaded Qwen backbone is governed by its own repository terms.