PeetPedro/kompress-v31

kompress-v3.1

Token compression classifier fine-tuned from PeetPedro/kompress-v3 (ModernBERT-base, 149M params). Trained as part of the ultrawhale fine-tuning loop.

Kompress classifies each token in a message as keep (1) or drop (0). Used by the headroom proxy to compress LLM context before it reaches the model.

Eval results (heretic adversarial benchmark)

Heretic-style prompts generate responses maximally dense with must-keep tokens (chemical formulas, CVE identifiers, memory addresses, line numbers). The benchmark measures what fraction of those tokens survive compression.

Metric	Value
heretic exact_pct	0.925
keep_rate	—
override_delta	+0.002
base model	kompress-v3

Full progression across all versions

Training

Added 600 synthetic domain pairs (code_diff, log_stream, json_tool_output, agent_error). Must-keep token coverage improved over v3.

Usage

# Via headroom proxy (recommended)
# ANTHROPIC_BASE_URL=http://localhost:8787 claude

# Direct library use
from headroom import compress, CompressConfig
result = compress(messages, config=CompressConfig(kompress_model="PeetPedro/kompress-v31"))

CONCLUSION

Domain data improved mk_in_ref but regressed heretic. Domain specialization trades generality.

USECASE

Domain-specific compression experiment. Reference for domain routing PR.

Series

Version	heretic	keep_rate	Notes
v3	0.942	0.728	first self-label
v3.1	0.925	—	domain data
v3.2	0.929	—	domain refined
v3.3	0.942	—	domain-only, overfit
v4	0.967	0.823	override internalized
v5	0.961	—	loop converged
v6	0.962	0.854	agent-distribution

Training code: ultrawhale