kompress-v5

Token compression classifier fine-tuned from PeetPedro/kompress-v4 (ModernBERT-base, 149M params). Trained as part of the ultrawhale fine-tuning loop.

Kompress classifies each token in a message as keep (1) or drop (0). Used by the headroom proxy to compress LLM context before it reaches the model.

Eval results (heretic adversarial benchmark)

Heretic-style prompts generate responses maximally dense with must-keep tokens (chemical formulas, CVE identifiers, memory addresses, line numbers). The benchmark measures what fraction of those tokens survive compression.

Metric	Value
heretic exact_pct	0.961
keep_rate	—
override_delta	0.000
base model	kompress-v4

Full progression across all versions

Training

Second self-labeling iteration. mk_in_ref ~0.86 but heretic regressed slightly (0.967 to 0.961). Loop converged — further self-labeling adds noise rather than signal. v4 remains the recommended production checkpoint.

Usage

# Via headroom proxy (recommended)
# ANTHROPIC_BASE_URL=http://localhost:8787 claude

# Direct library use
from headroom import compress, CompressConfig
result = compress(messages, config=CompressConfig(kompress_model="PeetPedro/kompress-v5"))

CONCLUSION

Loop converged. Second self-labeling iteration added noise. Convergence criterion met.

USECASE

Proof that self-labeling converges. Not for production use.

Series

Version	heretic	keep_rate	Notes
v3	0.942	0.728	first self-label
v3.1	0.925	—	domain data
v3.2	0.929	—	domain refined
v3.3	0.942	—	domain-only, overfit
v4	0.967	0.823	override internalized
v5	0.961	—	loop converged
v6	0.962	0.854	agent-distribution

Training code: ultrawhale