kimgreg/efficient-lm-head-reliability

Efficient LM-head reliability: compressed heads and reference states

This repository contains auxiliary research tensors, not a standalone language model. The three heads/<model>/head.pt files are packed ARCHead output heads built from the pinned FlashHead checkpoints. The ten reference_states/<model>/<source>/*.pt files are dense-model hidden states used for matched-state candidate-retrieval experiments. The heads and states are tied to the source models and tokenizers; they cannot be loaded with AutoModel.from_pretrained on their own.

Contents

Folder	Models	Purpose
heads/	Qwen3 0.6B and 1.7B, Llama 3.2 1B	Packed ARCHead heads, with build manifests
reference_states/*/states/	All four models	Dense natural/tool trajectories
reference_states/*/bfcl_stage2/	Qwen3 and Llama	Dense BFCL trajectories
reference_states/*/forced_formats_h/	Qwen3 and Llama	Selected format-probe states

The BFCL, natural-prompt and format-probe metadata, experiment code, and per-example results are maintained in the companion research repository. That repository is currently private. The full released FlashHead checkpoints and HNSW indexes are downloaded from their upstream repositories; they are not duplicated here.

Source checkpoints

Model key	Released checkpoint revision
qwen3-0.6b	embedl/Qwen3-0.6B-FlashHead@a3ed38667e683dcf90c38f5a23d8ef139494badb
qwen3-1.7b	embedl/Qwen3-1.7B-FlashHead@399929e7f68464fa43395fa3b28416725bfd7c44
llama-3.2-1b	embedl/Llama-3.2-1B-Instruct-FlashHead@927ce57183e92f16a569cc705e6d806fc548d154
gemma-3-1b	embedl/gemma-3-1b-it-FlashHead@f19052159425d8db5154c1068e7add46d653332b

The ARCHead files use the released code at suayptalha/archead@29002720037fa4215822933e3ec50636882a7cc2. Each head's manifest.json records its calibration sources, compression configuration and SHA-256 digest.

Download and integrity

hf download kimgreg/efficient-lm-head-reliability --local-dir ./hf-assets
cd hf-assets
sha256sum -c SHA256SUMS

The companion repository's fetch_research_data.sh restores the same files to the paths expected by its experiment scripts. SHA256SUMS covers the 13 large tensors. Verify the upstream checkpoint terms before reusing a derived head with a different distribution or purpose.