sectionme/granite-clm-g2

granite-clm g2 — a System-1.5 candidate ranker, in the CLM tradition

Part of the agentic-loop Granite ranking path (agentic-loop-ranking-granite). Unrelated to the FerrisRes runtime, which is a separate backend.

model.onnx is a fine-tuned derivative of ibm-granite/granite-embedding-97m-multilingual-r2. The token embeddings and the bottom 8 of 12 transformer layers are frozen; only the top 4 layers plus the heads.safetensors projection were trained.

file	what
model.onnx	encoder, opset 17, input_ids/attention_mask (int64) -> last_hidden_state [batch, seq, 384]
granite_encoder.json	pooling + dimension contract for the runtime
heads.safetensors	g2 variant: 2-branch MLP heads, 384 -> 512 -> 256, L2-normalised
manifest.json	training provenance, digests, hyper-parameters

Pooling is not in the graph: the runtime applies masked-mean then L2 normalisation.

Provenance: this is a CLM, at 1/80th the encoder

The idea is Contrastive Language Models (CLM, Apache-2.0) — "a new class of System One model trained with a contrastive learning objective that connects states and actions." Three properties of that design are what this checkpoint inherits:

1. States and actions are disaggregated. Each side gets its own projection head, so an action's embedding is cached and reused across every state that might select it. Ranking a state against N candidates costs one state embedding plus N cheap dot products, not N forward passes.
2. The answer distribution is a softmax over the candidate set. CLM's typed questions are a state plus a closed set of options, and the softmax over the scores is the answer distribution — no per-task setup.
3. InfoNCE both ways. A state is pulled toward the action actually taken and pushed off the others, and vice versa.

This is not CLM-8B and is not comparable to it in quality. It is the same shape of idea, rebuilt small enough to live inside an agent process that already owns its encoder.

	CLM-8B	this checkpoint
encoder	Qwen3-8B, frozen, last-token pooling	Granite 97M, top 4 layers fine-tuned, masked-mean
encoder dim	4096	384
projection dim	512	256
head params	20M per side	660k total (g2)
training data	60M Q&A + 30M hard negatives + 1M agentic trajectories	3,000 synthetic states
extra residency	~16 GB, a second model	390 MB, reuses an encoder already loaded
typed questions (Noul/Choice/Score)	yes	no — ranking only

That last row is a real scope limit. This implements CLM's rank primitive and nothing else; the typed-question weights are not here and there is no Choice/Noul/Score API.

Where it sits: System-1.5, and why the name is "1.5"

CLM is a System One model: fast, cheap, and able to answer bounded control questions without waking a large reasoning model. This one is System-1.5 — it may act on its own, but only inside a narrow envelope, and the envelope is enforced outside the model:

state text  ──► Granite 97M (this repo) ──► 384-d ──► state head ──┐
                                                            │  logit_scale · dot
candidate   ──► (cached base vectors)     ──► 384-d ──► action head ┘
                                                                  │
                                                    softmax over the candidate set
                                                                  │
                                                     abstention  (never optional)
                                                                  │
                                            Selected ────────────┴──── Escalated
                                                                          │
                                                                    System-2 / human

The single most important property: the score is relative to the supplied candidate set. exp(logit_scale) · cos(state, candidate) answers "which of these fits this state best" — never "is the top candidate correct." If every candidate is bad, this model still puts one first. A top-1 number is not evidence that anything is right, and the softmax probability is not a confidence.

So abstention is not an add-on bolted onto the ranker; it is a separate layer that says "none of these" using signals that do not depend on the backend being calibrated — top-1−top-2 margin, distribution entropy, an explicit __escalate__ candidate that competes on merit, and caller-supplied deterministic safety checks. In agentic-loop that is agentic-loop-ranking::abstain, resolving at control-plane layer 7. A CLM-style prob is the number to feed the abstention layer, not a threshold to compare against.

Results — and what they do not show

Test split, n=2370: top-1 0.6245, top-2 0.9840, MRR 0.8094 (frozen-encoder baseline top-1 0.5194, so +0.1051).

That clears the ranking gate, and it is the only gate cleared.

The decision gate is not met. No backend meets a selected-error budget — the best is .413 at full coverage against a 10% budget. This checkpoint is evidence of direction, not a production result, and it was trained on 3,000 states for 3 epochs on 6 CPU cores. Treat it as a research artifact. Do not serve decisions from it.

Reproduce with --max-train 3000 --max-len 256 --batch-size 4 --epochs 3, --lr 3e-3 --encoder-lr 2e-5 --train-top-layers 4; the full command line and corpus digest are in manifest.json.

Usage

The contract

step	what	where
pool	masked mean over token states, then L2 normalise	the runtime, not the graph
project	LayerNorm → Linear → GELU, per layer	heads.safetensors
score	logit_scale · dot(state_proj, candidate_proj)	in the file; trained 15.634, init 20
answer	softmax over the candidate axis	caller

The graph emits token-level last_hidden_state on purpose. A graph that pooled internally would be pooled twice and would embed differently from every vector the heads were trained on. granite_encoder.json says so; trust it over the habit of loading this as a sentence-transformer.

Python

Reference implementation, agreeing to 1e-7 with an independent re-implementation of the Rust head contract:

import numpy as np, onnxruntime as ort
from transformers import AutoTokenizer
from safetensors.numpy import load_file
from huggingface_hub import snapshot_download

d    = snapshot_download("sectionme/granite-clm-g2")
T    = load_file(f"{d}/heads.safetensors")
tok  = AutoTokenizer.from_pretrained(d)
onnx = ort.InferenceSession(f"{d}/model.onnx")

def gelu(x):                       # tanh approximation, as in the Rust head
    return 0.5 * x * (1 + np.tanh(0.7978845608028654 * (x + 0.044715 * x**3)))

def embed(texts, side):            # side is "state" or "action"
    ids = tok(texts, return_tensors="np", padding=True,
              truncation=True, max_length=256)
    h = onnx.run(None, {"input_ids": ids["input_ids"],
                        "attention_mask": ids["attention_mask"]})[0]
    m = ids["attention_mask"][..., None].astype(h.dtype)
    v = (h * m).sum(1) / m.sum(1)                          # masked mean
    v = v / np.linalg.norm(v, axis=-1, keepdims=True)     # L2
    for i in (0, 1):                                      # pre-norm per layer
        mu = v.mean(-1, keepdims=True)
        v = (v - mu) / np.sqrt(((v - mu)**2).mean(-1, keepdims=True) + 1e-5)
        v = v * T[f"{side}.norms.{i}.weight"] + T[f"{side}.norms.{i}.bias"]
        v = v @ T[f"{side}.layers.{i}.weight"].T + T[f"{side}.layers.{i}.bias"]
        v = gelu(v)                       # after the last layer too
    return v / np.linalg.norm(v, axis=-1, keepdims=True)

def rank(state, candidates):             # -> one probability per candidate
    s  = embed([state], "state")[0]                      # 256-d
    lg = T["logit_scale"] * (embed(candidates, "action") @ s)   # (N,)
    e  = np.exp(lg - lg.max())
    return e / e.sum()                       # normalised over the candidate set

On a held-out state — one positive, three negatives:

1. p=0.4032  margin=+0.0620  advance            <- strong-positive
2. p=0.3413                 no_op_edit-00       <- negative
3. p=0.2092                 repeat_patch-02     <- hard-negative
4. p=0.0462                 broad_refactor-01   <- negative

The right answer wins — and the margin over the runner-up is +0.062, barely above the default min_margin of 0.05. That is the honest summary of where this model is: it discriminates, but not by enough to act unattended.

In agentic-loop

The intended path is the Rust ranker, not Python. Loading is fail-closed — wrong encoder, wrong width, unsupported manifest version or wrong asset digest refuses to construct a ranker rather than producing confident nonsense.

hf download sectionme/granite-clm-g2 --local-dir models/granite-clm/g2-ft
cargo run --release --features granite --bin granite-calibrate -- \
  --granite-dir models/granite-clm/g2-ft --asset models/granite-clm/g2-ft \
  --shards <embedding-shard> --dataset <corpus.jsonl> --splits test

granite_encoder.json declares encoder_revision: finetune-be69086397ed, a local artifact rather than an upstream Granite revision. The loader verifies encoder identity against it, so this encoder and these heads must be kept together — do not pair these heads with stock ibm-granite/granite-embedding-97m-multilingual-r2 weights, which is exactly the mismatch the fail-closed contract exists to catch.

License

AGPL-3.0-or-later, matching the agentic-loop project this is trained from. Derived from ibm-granite/granite-embedding-97m-multilingual-r2 (Apache-2.0). The idea is CLM (Apache-2.0); no CLM weights are included here.