KVEraser Qwen3-8B

LoRA adapter for KVEraser: Learning to Steer KV Cache for Efficient Localized Context Erasing, after pre-training and joint task-specific fine-tuning.

The adapter acts on the Qwen3 transformer during KV erasing. Generation uses the unchanged backbone; use the KVEraser evaluation code to route both operations through one shared backbone.

Usage

From the KVEraser code repository, install its requirements and run on a CUDA GPU:

CUDA_VISIBLE_DEVICES=0 python evaluate.py \
  --checkpoint Graph-COM/KVEraser \
  --measure_performance --output_dir outputs/pretrained

The adapter, backbone, and evaluation data download automatically. Use --checkpoint_revision to select a model revision. The code README also provides pre-training and fine-tuning commands.

Configuration

Base model: Qwen/Qwen3-8B, revision b968826d9c46dd6066d109eabc6255188de91218. LoRA rank 16, alpha 32, dropout 0; targets: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj. Adapter parameters: 43,646,976.

Both training phases use seed 42, one epoch, AdamW, LR 1e-5, weight decay 0.01, and gradient clipping 1.0; gradient accumulation is 8 for pre-training and 1 for fine-tuning. See training_config.json. Inference uses BF16 and SDPA.

Evaluation

Results for this adapter on 1,522 test samples using one RTX 6000 Ada 48GB GPU; EM and F1 are percentages. Evaluation uses greedy decoding with up to 32 new tokens. The paper reports results on A100 GPUs.

Task	Samples	EM	F1
NIAH 1K	100	100.00	—
NIAH 2K	100	100.00	—
NIAH 4K	100	100.00	—
NIAH 8K	100	100.00	—
NIAH 16K	100	100.00	—
NIAH 32K	100	99.00	—
2Wiki	344	90.70	91.63
MuSiQue	300	84.67	86.67
IIRC	178	91.57	92.46
BFCL	100	85.00	—

On one RTX 6000 Ada 48GB, 32K NIAH online latency averages 514.22 ms and peak allocated memory is 22.91 GiB. Timing includes erasing, query replay, and generation; it excludes loading and initial prefill, with three warmup samples excluded per dataset. Memory includes the backbone and initial prefill. Full metrics and environment are in evaluation.json.