kompress-v3
Token compression classifier fine-tuned from chopratejas/kompress-v2-base (ModernBERT-base, 149M params). Trained as part of the ultrawhale fine-tuning loop.
Kompress classifies each token in a message as keep (1) or drop (0). Used by the headroom proxy to compress LLM context before it reaches the model.
Eval results (heretic adversarial benchmark)
Heretic-style prompts generate responses maximally dense with must-keep tokens (chemical formulas, CVE identifiers, memory addresses, line numbers). The benchmark measures what fraction of those tokens survive compression.
| Metric | Value |
|---|---|
| heretic exact_pct | 0.942 |
| keep_rate | 0.728 |
| override_delta | +0.027 |
| base model | kompress-v2-base |
Full progression across all versions
Training
Self-labeled references via kompress-v2-base, 1802 training pairs. mk_in_ref improved from ~0.5 (Q&A labels) to 0.720. First iteration of the ultrawhale self-labeling loop.
Usage
# Via headroom proxy (recommended)
# ANTHROPIC_BASE_URL=http://localhost:8787 claude
# Direct library use
from headroom import compress, CompressConfig
result = compress(messages, config=CompressConfig(kompress_model="PeetPedro/kompress-v3"))
Series
| Version | heretic | keep_rate | Notes |
|---|---|---|---|
| v3 | 0.942 | 0.728 | first self-label |
| v3.1 | 0.925 | — | domain data |
| v3.2 | 0.929 | — | domain refined |
| v3.3 | 0.942 | — | domain-only, overfit |
| v4 | 0.967 | 0.823 | override internalized |
| v5 | 0.961 | — | loop converged |
| v6 | 0.962 | 0.854 | agent-distribution |
Training code: ultrawhale