Mímir (impacte/mimir-lfm-openjev)

Mímir — the Norse being of wisdom and counsel who keeps the Well of Knowledge beneath Yggdrasil's root; even Odin trades an eye for a drink from Mímir's head. This model is our drink: a single judge that every other model in the homelab consults before it acts.

Mímir is LiquidAI/LFM2.5-8B-A1B turned into a jev cross-encoder following the recipe of AlexWortega/openjev: one model that reads a premise and a hypothesis and answers contradiction / entailment / neutral. No task-specific heads, no per-task training — the single NLI primitive is reused zero-shot for answer reranking, answer grading, policy guarding, and agentic state checks.

Architecture	Lfm2MoeForSequenceClassification — custom 3-way head over the Lfm2MoeModel backbone, last-token pooling, 8.3B total / ~1.5B active params (32 experts, top-4, hybrid conv + attention)
Labels	0 = contradiction, 1 = entailment, 2 = neutral
Template	Premise: {premise}\nHypothesis: {hypothesis} (stored in config.nli_template)
Training	two stages, LoRA r=16 (details below), cross-entropy over 3 classes
License	LFM Open License v1.0 (inherited from the base model — see below)

Credits — please read

This model exists because of two lines of work, and both deserve the credit:

1. openjev by Alex Wolf (AlexWortega) — the entire jev approach: turning a causal LLM into a single NLI cross-encoder used zero-shot for reranking, grading, guarding, and agent control. Mímir is a faithful port of that recipe (labels, template, two-phase training curriculum, zero-shot usage) onto a different backbone. openjev's code is MIT licensed; our trainer/eval scripts are adapted from it. If you use Mímir for anything interesting, you are using Alex's idea — go star openjev.
2. LiquidAI/LFM2.5-8B-A1B by Liquid AI — the base model: a hybrid MoE (short-conv + attention + 32-expert FFN) with ~1.5B active parameters at 8.3B total, which is what makes this jev cheap enough to run on consumer GPUs. Mímir's weights are a derivative of the LFM2.5 base and are released under the same LFM Open License v1.0.

Nothing here is trained per task; the capability comes from the base model plus the jev recipe.

How it works

Given a premise and a hypothesis, the model outputs P(contradiction), P(entailment), P(neutral):

• Rerank — premise = query, hypotheses = candidate answers; pick the argmax-entailment option.
• Grade — premise = reference answer, hypothesis = candidate answer; entailment ≈ correct.
• Guard — premise = policy text, hypothesis = user request; contradiction ≈ violation.
• Agents — premise = environment/agent state dump, hypothesis = candidate action; argmax entailment is the decision (openjev demos this on Flappy Bird, Doom and Minecraft).

Results (zero-shot, n=1000 per benchmark)

Stage 1 trains on AllNLI (SNLI + MNLI), stage 2 on the openjev-style hard mixture (adversarial NLI, evidence-grounded NLI, long-document haystack, agentic traces):

Benchmark	Mímir stage 1	Mímir (stage 2, this repo)	openjev v1 (Qwen3.5-4B)	openjev v2 (Qwen3.5-4B)
MNLI-m	0.898	0.897	0.91	0.91
MNLI-mm	0.889	0.882	—	—
ANLI r1	0.568	0.744	—	—
ANLI r2	0.436	0.609	—	—
ANLI r3	0.392	0.555	0.42	0.63
WANLI	0.607	0.741	0.63	0.77
SciTail	0.733	0.943	—	—
ARC-C rerank	0.54	0.562	0.59	0.72
MMLU rerank	0.428	0.433	0.47	0.53

With ~1.5B active parameters (vs openjev's 4B dense), Mímir matches or beats openjev v1 across the board and approaches v2 on adversarial NLI. The remaining rerank gap is expected: openjev v2 includes the (gated) xlam function-calling corpus that we could not download, and trains on the full mixture rather than a 500k-row sample.

Use it

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model = AutoModelForSequenceClassification.from_pretrained(
    "impacte/mimir-lfm-openjev", trust_remote_code=True, dtype=torch.bfloat16, device_map="auto")
tok = AutoTokenizer.from_pretrained("impacte/mimir-lfm-openjev")

pairs = [
    ("The Eiffel Tower is in Paris.", "The Eiffel Tower is in France."),   # entailment
    ("The Eiffel Tower is in Paris.", "The Eiffel Tower is in Berlin."),   # contradiction
    ("The Eiffel Tower is in Paris.", "The tower is made of iron."),       # neutral
]
texts = [model.config.nli_template.format(premise=p, hypothesis=h) for p, h in pairs]
enc = tok(texts, return_tensors="pt", padding=True).to(model.device)
probs = torch.softmax(model(**enc).logits, -1)
for (p, h), row in zip(pairs, probs):
    label = ["contradiction", "entailment", "neutral"][row.argmax()]
    print(f"{label:14s} {row.tolist()}")

trust_remote_code=True loads modeling_lfm2_moe_seqcls.py from this repo (the transformers release has no lfm2_moe sequence-classification head; the module implements one and wires it into the backbone). For a higher-level API (batch predict, rerank, grade, guard, and latents for training per-task MLP heads), see the LfmJevCrossEncoder wrapper in the openjev training codebase this model was built with.

VRAM note: the bf16 model needs ~16 GB. On a single ≥24 GB GPU, device_map="auto" just works. On smaller cards, shard it explicitly across GPUs, e.g. max_memory={0: "13GiB", 1: "7GiB"} for a 16 GB + 8 GB pair — on tight dual-GPU setups the automatic balancer can under-provision the smaller card during MoE weight conversion, and explicit caps avoid that.

Training

	stage 1 — AllNLI	stage 2 — hard mixture
Data	SNLI + MNLI train, 200k rows	500k rows sampled from an 857k-row balanced mixture (988k raw → class-balanced → MNLI-val leakage-filtered)
Mixture	—	ANLI 163k · nli_fever 146k · MNLI 120k · SNLI 120k · WANLI 103k · QNLI 60k · long-doc haystack 80k · LingNLI 45k · SciTail 24k · agentic (AgentTraj-L, AgentInstruct, When2Call, synthetic state predicates) ~92k · ConTRoL/bAbI ~27k
Method	LoRA r=16 α=32 on attention + short-conv projections (q_proj,k_proj,v_proj,out_proj,in_proj), score head fully trained; expert router untouched	same
Optimization	bs 16×2, lr 1e-4 (stage 1) / 1e-4 (stage 2), cosine, 3% warmup, wd 0.01, bf16, 1 epoch, gradient checkpointing
Hardware	RTX 5060 Ti 16 GB + RTX 4060 Ti 8 GB (bf16 model sharded across both, grouped_mm MoE kernels)
Eval	MNLI validation_matched during training; all benchmarks strictly zero-shot (eval sets are banned from training by the mixture builder)

Note on quantization: bitsandbytes QLoRA is not applicable to this backbone — the ~8.1B expert weights are stored as raw nn.Parameters (not nn.Linear), which bnb cannot quantize. The bf16 model therefore needs ~16 GB of VRAM in total and is sharded across GPUs for training/inference.

Limitations

• Trained on English NLI data (multilingual transfer is untested, though LFM2.5 itself is multilingual).
• Rerank scores are calibrated only implicitly; thresholds on P(entailment) should be tuned per task.
• The neutral class absorbs "not stated" long-document rows; treat P(neutral) as a soft "unsupported" signal, not a factuality judgment.
• Sequence length during training was capped at 256 tokens; longer inputs work (the backbone supports up to 128k) but are out-of-distribution.

License

Mímir's weights are a derivative work of LiquidAI/LFM2.5-8B-A1B and are distributed under the same LFM Open License v1.0 (permitted for commercial use below Liquid AI's revenue threshold; see the license text). The adapted training/eval code follows openjev's MIT license.

Citation

@misc{mimir2026,
  title  = {Mímir: LFM2.5-8B-A1B as a jev cross-encoder},
  author = {oamazonasgabriel},
  year   = {2026},
  url    = {https://huggingface.co/impacte/mimir-lfm-openjev},
  note   = {openjev recipe applied to LiquidAI LFM2.5-8B-A1B}
}

@misc{wortega2025openjev,
  title        = {openjev: one small model that can do everything (zero-shot NLI judge)},
  author       = {Alex Wolf (AlexWortega)},
  howpublished = {\url{https://huggingface.co/AlexWortega/openjev}},
  year         = {2025}
}

@misc{liquidai2025lfm25,
  title        = {LFM2.5-8B-A1B},
  author       = {Liquid AI},
  howpublished = {\url{https://huggingface.co/LiquidAI/LFM2.5-8B-A1B}},
  year         = {2025}
}