kushalpatil/jevify-gemma4-e4b

google/gemma-4-E4B-it fine-tuned (LoRA, merged) to give honest probabilities when asked typed questions about a piece of state — the model behind jevify, a local, Jev-compatible probabilistic decision API.

Nothing is generated: one prefill, read the next-token distribution over the answer labels, done.

Use

pip install "jevify[transformers] @ git+https://github.com/kushalpatil07/jevify"
jevify serve --model kushalpatil/jevify-gemma4-e4b                     # POST /v1/systemone, Jev wire format

from jevify import Jevify, Noul, Choice, Score
jev = Jevify.from_transformers("kushalpatil/jevify-gemma4-e4b")        # or serve it with vLLM and use Jevify.from_runtime("vllm", "kushalpatil/jevify-gemma4-e4b")
jev.system_one("Help! My payouts have been failing for 3 days.", {
    "urgent": Noul("Does this convey urgency?"),
    "team":   Choice("Which team should handle this?", {"billing": None, "technical": None, "sales": None}),
    "mood":   Score("How frustrated is the customer?", ["calm", "frustrated", "furious"]),
})

Training

LoRA r=64 on attention projections, 2 epochs over ~47k (state, question, target-distribution) items from 16 sources: hard-labeled classification sets with randomized option subsets and order, multi-annotator sets with real human label distributions, and constructed long states (up to 24k tokens) with answers known by construction. Loss = KL(target || label distribution). No teacher model. Recipe: train/ in the jevify repo.

Results

Held-out, out-of-distribution (6 datasets not used in training; 307 items), no post-hoc scaling:
ood_massive      n=  50  acc=0.900 nll=0.308 brier=0.140 ece=0.065 conf=0.932
ood_trec         n=  50  acc=0.940 nll=0.201 brier=0.095 ece=0.097 conf=0.874
ood_paws         n=  50  acc=0.540 nll=0.915 brier=0.622 ece=0.262 conf=0.792
ood_sms_spam     n=  50  acc=0.980 nll=0.121 brier=0.056 ece=0.046 conf=0.934
ood_app_reviews  n=  50  acc=0.740 nll=0.901 brier=0.427 ece=0.126 conf=0.705
ood_subj         n=  50  acc=0.940 nll=0.218 brier=0.109 ece=0.128 conf=0.841
spike            n=   7  acc=1.000 nll=0.051 brier=0.021 ece=0.044 conf=0.956
ALL              n= 307  acc=0.844 nll=0.435 brier=0.237 ece=0.043 conf=0.849

In-distribution held-out (800 items), raw -> trained:
  step 0  acc=0.745 conf=0.962 ece=0.217 nll=1.827 brier=0.412
  step 1209  acc=0.823 conf=0.823 ece=0.028 nll=0.438 brier=0.193