dimajyg/homemade-jev

homemade jev (MiniCPM5-2B LoRA)

3-way NLI head (contradiction / entailment / neutral). Not TypeSafe Jev, not Loop openjev/openjev.

Adapter only (~96 MB). Base: openbmb/MiniCPM5-2B (Apache-2.0).

Labels: 0 contradiction, 1 entailment, 2 neutral (dleemiller). Template: Premise: {p}\nHypothesis: {h}.

v0 (this commit)

train	AllNLI 200k + ANLI r3 + WANLI, 1 epoch, LoRA r=16, fp32, A100 80 GB, 71.2 min
MNLI-m 2k seed 42	acc 0.8885, ECE 15-bin 0.0104, Brier 0.082
smoke guitar P(ent)	0.940
smoke refund noul yes	0.462 (weak; hard-mix run is in progress)

Hard-mix continue (ANLI r1/r2, SciTail, QNLI, FEVER, policy/numbers) will overwrite these files. Check eval.json.

Load

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer
from peft import PeftModel

BASE = "openbmb/MiniCPM5-2B"
ADAPTER = "dimajyg/homemade-jev"
ID2LABEL = {0: "contradiction", 1: "entailment", 2: "neutral"}

tok = AutoTokenizer.from_pretrained(ADAPTER, use_fast=True)
if tok.pad_token is None:
    tok.pad_token = tok.eos_token
model = AutoModelForSequenceClassification.from_pretrained(
    BASE, num_labels=3, id2label=ID2LABEL,
    label2id={v: k for k, v in ID2LABEL.items()},
    torch_dtype=torch.float32,
)
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

def nli(premise, hypothesis):
    text = f"Premise: {premise}\nHypothesis: {hypothesis}"
    batch = tok(text, return_tensors="pt", truncation=True, max_length=256, return_token_type_ids=False)
    with torch.no_grad():
        p = torch.softmax(model(**batch).logits.float(), dim=-1)[0]
    return {ID2LABEL[i]: round(float(p[i]), 4) for i in range(3)}

Space (CPU Basic, slow): https://huggingface.co/spaces/dimajyg/homemade-jev

Do not treat softmax as calibrated for a new domain. Merge of LoRA into base is not in this repo (Air 16 GB).