kev-0.5b-merged: unofficial merged derivative of Kev

This is an unofficial derivative of jaredpalmer/kev-0.5b by Jared Palmer, prepared by Avartha. It is not endorsed by the Kev author. Kev and its weights are Apache-2.0, as are the Qwen2.5 (attention-only) base weights; see LICENSE (Kev) and LICENSE-QWEN (base).

Modifications: the Kev LoRA adapter is merged into the base weights; the pointer head is also provided as safetensors.

Sources (exact revisions)

	repo	revision
Kev release	jaredpalmer/kev-0.5b	9ce2fd39db3a397c89733f94af948e3d1fdfffcd
Base	Qwen/Qwen2.5-0.5B	060db6499f32faf8b98477b0a26969ef7d8b9987
Kev code (encoder, head, merge rule)	github.com/jaredpalmer/kev@fe64b1274ea7f80d4095866df90666abb03e9cf6	Apache-2.0

head.pt records no base revision; this is Qwen/Qwen2.5-0.5B main, unchanged since 2024-09-25, i.e. before Kev existed (inferred, not recorded upstream).

What this repository is

• A Qwen2.5 (attention-only) backbone in bf16 with the Kev fine-tune already applied. Kev never uses the LM head.
• head.safetensors: the Kev pointer head (fp32). head.pt is the unchanged original, and kev_head.json describes the contract:

tensor	shape	dtype
q.weight	[256, 896]	float32
q.bias	[256]	float32
k.weight	[256, 896]	float32
k.bias	[256]	float32
temperature	[]	float32

• Readout: logit_j = ((W_k h_opt_j + b_k) . (W_q h_decide + b_q)) / sqrt(256) / T, then a softmax over the question's options. h is the backbone's final-norm hidden state.
• Temperature T: 1.0: head.pt carries no temperature, and kev-src's loader then applies 1.0 (the upstream card quotes T=1.47, which is not stored in head.pt and therefore not applied by Kev's loader).
• Token layout: <|fim_prefix|> state, <|fim_middle|> question, <|box_start|>/<|box_end|> option, <|fim_suffix|> decide. There is no chat template; each question is a causal row continuing the state.
• The tokenizer, config.json and preprocessor files come from the base at the pinned revision. That is what Kev's loader uses: it always loads the tokenizer from the base.

Procedure

Merged with upstream/kev_merge.py (sha256 260f41d81bb9901b6991b36869dabe0930611c360ab3c0c1a4d7eb7de9331eed), streaming one base shard at a time on CPU:

• For each LoRA-adapted Linear: W_bf16 = bf16( fp32(W) + (B @ A) * 2.0 ). Here 2.0 = lora_alpha / r = 32 / 16 comes from the adapter's adapter_config.json (plain LoRA: no DoRA, no rsLoRA, no rank/alpha patterns). This is exactly kev-src kev/checkpoint.py:296-312: an fp32 base, PEFT merge_and_unload, then a single cast to bf16.
• Adapter keys base_model.model.<module>.lora_{A,B}.weight map to base keys model.<module>.weight. The prefix was chosen as the only candidate under which every adapted module exists in the base index.
• 168 of 168 adapted modules were applied. The script fails if any adapted module is missing.
• Every other tensor is copied bit for bit, including tensors the base stores in fp32.
• Nothing dropped: the base has no MTP or vision tensors.
• Shard file names follow the base. Output: 290 tensors, 988,065,536 bytes.

Verification (CPU, before upload)

Run with upstream/kev_verify.py through kev-src's own encode(), rows_of(), DecisionModel.probs() and PointerHead. It used 5 short System One requests built with kev.api.to_record (token counts [48, 50, 43, 35, 64]). No GPU was used.

• Inventory vs base: 290 tensors. Names, shapes and dtypes equal the base's minus the 0 dropped tensors (ok=True).
• Bitwise vs Kev's own bf16 merge: all 290 backbone tensors of kev-src's PEFT fp32 merge, cast to bf16, equal this artifact bit for bit (ok=True). This covers all 168 adapted modules. 0 tensors are stored in fp32 and compared in fp32.
• Head conversion: head.safetensors tensors equal head.pt's (ok=True). The temperature equals the loader's, rounded to fp32 (ok=True).

arm vs Kev reference (fp32, adapter unmerged, head.pt)	hidden max abs	hidden max rel L2	probs max abs	argmax agree
fp32 merged (PEFT merge_and_unload) vs fp32 unmerged	0.000229	3.19e-06	1.55e-06	7/7
this artifact, bf16 weights, fp32 compute	2	0.0203	0.00152	7/7
this artifact, bf16 weights, bf16 compute (served form)	13.3	0.136	0.0187	7/7

The bf16 drift is inherent to bf16 serving. Kev's own cards report served bf16 within 0.017 of fp32 for the 4B. Here the served-form max |dp| (0.0187) exceeds that figure; with bf16 weights but fp32 compute it is 0.0015, so most of the drift comes from bf16 arithmetic, not from the merge rounding. Not measured here: accuracy on Kev's evaluation suites, and GPU end-to-end serving.

Files

file	bytes	sha256
LICENSE	11,343
LICENSE-QWEN	11,343
config.json	681
generation_config.json	138
head.pt	1,839,295	0d59bd4608e38f32b4f86c85eb29e8187e14f2117ca99fe765e799b290c52a66
head.safetensors	1,837,524	ffd98f6744bd9d0d8d7d71193a98f862a652f5d4af346acfcc6c93b3c05fe131
kev_head.json	2,688
merge_manifest.json	8,317
merges.txt	1,671,839
model.safetensors	988,097,824	2101d19e18a1e0c54d434bc1217e6a7042b3eb0b2b86296a0697b650be840201
tokenizer.json	7,031,645
tokenizer_config.json	7,228
upstream/adapter_config.json	1,286
upstream/kev_head.py	4,878
upstream/kev_merge.py	7,700
upstream/kev_model_card.md	12,454
upstream/kev_verify.py	10,381
verification.json	2,106
vocab.json	2,776,833
README.md	(this file)