EviSuff-EvidencePlanner-8B

EviSuff-EvidencePlanner-8B is the trained model release from EviSuff: Learning When Evidence Is Enough for Deep-Search Agents. This repository contains three PEFT/LoRA variants targeting evidence-aware answering, evidence acquisition, and stopping in deep-search agents. The release-facing name describes the model's evidence-planning role and scale without embedding the upstream model-family name; the complete base-model lineage remains documented explicitly below.

Publication gate: the files are locally staged, but the licence for the adapter weights and the final Hugging Face repository identifier still require author/institutional confirmation. Do not make the repository public before resolving LICENSE_PENDING.md.

Released checkpoints

Directory	Role	Required base
answer-sft/	Answer-form warm-up	Qwen/Qwen3-8B
no-gate/	Update-matched process tuning without gate-decision samples	Qwen3-8B with answer-sft merged
full-evisuff/	Process tuning with EviSuff gate-decision supervision	Qwen3-8B with answer-sft merged

Each directory contains only adapter_config.json and adapter_model.safetensors. Every adapter has 174,587,904 trainable parameters across 504 tensors and uses LoRA rank 64, alpha 128, dropout 0.05, and the Qwen projection layers as targets.

Important loading order

no-gate and full-evisuff are second-stage adapters. Applying either one directly to the unmodified Qwen3-8B base is incorrect. The required sequence is:

1. Load Qwen/Qwen3-8B.
2. Apply answer-sft and merge it into the base model.
3. Apply either no-gate or full-evisuff.

The repository provides scripts/load_model.py, which implements this sequence for local files or a Hugging Face repository.

pip install -r requirements.txt

# Run from a local clone
python scripts/load_model.py --adapter full-evisuff --prompt "What evidence is still missing?"

# After publication, run from the Hugging Face repository
python scripts/load_model.py \
  --repo-id <HF_ORG>/EviSuff-EvidencePlanner-8B \
  --adapter no-gate \
  --prompt "Summarize the supported conclusion and its uncertainty."

Downloading the base model is subject to the Qwen model's access requirements and terms. Loading an 8B model also requires suitable accelerator memory; the script does not quantize the model by default.

Training summary

All three adapters use PEFT LoRA with r=64, alpha=128, dropout=0.05, no bias, and all linear projection layers. Seeds and data seeds are 42. Training used bfloat16 and gradient checkpointing.

Adapter	Train	Validation	Maximum length	Learning rate	Epoch/update rule	Gradient accumulation	Final step
Answer-SFT	5,223	530	8,192	1e-4	2 epochs	4	1,306
No-gate	15,247	1,714	40,960	2e-5	update-matched to 2,058 steps	8	2,058
Full EviSuff	16,462	1,846	40,960	2e-5	1 epoch	8	2,058

Sanitized training summaries are provided under training_configs/. Training examples and teacher trajectories are not distributed in this release.

Evaluation results

Core evaluation uses 157 genuinely held-out EviSuff-BoundaryBench test/stress tasks after a message-level leakage audit. Human metrics are percentages; O is the paper's aggregate score.

System	CSR	SPC	U-Cal	Stop	O
Answer-SFT	33.923	38.217	43.312	28.025	0.694
No-gate	39.155	43.312	48.408	35.669	0.708
Full EviSuff	49.257	50.318	56.051	57.962	0.734

Full EviSuff improves stop accuracy by 22.293 percentage points over the update-matched No-gate adapter. The result is deliberately bounded: the untuned Base system retains the highest strict claim-support rate, so Full EviSuff is not uniformly best on every answer-quality metric.

On SEAL-HARD, official-compatible accuracy is 6.693% for No-gate and 9.449% for Full EviSuff. Full also abstains more often. SEALQA is third-party data and is not redistributed here.

Intended use

• Research on evidence sufficiency and stopping in deep-search agents.
• Reproduction of the EviSuff model comparisons.
• Analysis of answer-form, tool-use, and gate-decision supervision.

Limitations

• Results cover one base-model family and a leakage-audited but imbalanced held-out set.
• The adapters do not guarantee factuality, complete evidence, or safe autonomous web use.
• The models can inherit limitations and biases from Qwen3-8B and the training trajectories.
• Correct operation depends on the surrounding search, citation, and gate runtime; a standalone text-generation prompt is only a smoke test.
• Dynamic web evidence can change after training and evaluation.

Files not included

• The full or merged Qwen3-8B model weights.
• Optimizer, scheduler, RNG, trainer-state, and training-argument binary files.
• Raw logs, caches, model outputs, and private filesystem paths.
• Training examples, teacher trajectories, test/stress gold annotations, and SEALQA files.

Licences

BASE_MODEL_LICENSE is the Apache License 2.0 text shipped with the local Qwen3-8B base-model copy. It applies to the base model, not automatically to these adapter weights. The adapter-weight licence remains pending; see LICENSE_PENDING.md.

Validation and citation

Run python scripts/validate_release.py after cloning. File hashes are listed in checksums.sha256. Cite the paper and the model release using CITATION.cff once the final repository URL and paper identifier are available.