Instructions to use ZBox008003/AgentForesight-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ZBox008003/AgentForesight-7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="ZBox008003/AgentForesight-7B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("ZBox008003/AgentForesight-7B")
model = AutoModelForCausalLM.from_pretrained("ZBox008003/AgentForesight-7B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ZBox008003/AgentForesight-7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ZBox008003/AgentForesight-7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ZBox008003/AgentForesight-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/ZBox008003/AgentForesight-7B

SGLang

How to use ZBox008003/AgentForesight-7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ZBox008003/AgentForesight-7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ZBox008003/AgentForesight-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ZBox008003/AgentForesight-7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ZBox008003/AgentForesight-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use ZBox008003/AgentForesight-7B with Docker Model Runner:
```
docker model run hf.co/ZBox008003/AgentForesight-7B
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

AgentForesight-7B

Online auditor for early failure prediction in multi-agent systems.

AgentForesight-7B watches a multi-agent trajectory as it unfolds. At every step it sees only the prefix observed so far and must either let the run continue or raise an alarm at the decisive error — the step whose failure would otherwise propagate and lock in a wrong outcome. This is a strictly harder contract than post-hoc failure attribution, which inspects a completed trajectory with full hindsight.

📄 Paper: AgentForesight: Online Auditing for Early Failure Prediction in Multi-Agent Systems
🌐 Project page: https://zbox1005.github.io/agent-foresight/
💻 Code: https://github.com/ZBox1005/AgentForesight
📊 Dataset: ZBox008003/AFTraj (AFTraj-2K, 2,272 trajectories)

Model details


Base model	`Qwen/Qwen2.5-7B-Instruct`
Parameters	7.6B, `bfloat16`, `Qwen2ForCausalLM`
Context	32,768 tokens
Training	Stage 1 BPPO (Boundary-Pair Preference Optimization) → Stage 2 GRPO (three-axis what/where/who reward)
Checkpoint in this repository	Stage-2 GRPO `global_step_480`, merged to Hugging Face format
Internal tag	`v16_1_grpo_step480` — the checkpoint evaluated in this model's main-table row
Base-model revision	The retained Stage-1 launch script names `Qwen/Qwen2.5-7B-Instruct` but does not pin a commit; the exact upstream revision has not been established.

Training recipe (coarse-to-fine)

Stage 1 — BPPO. A DPO-style preference objective over boundary pairs: prefixes cut just before vs. just after the decisive error. This instils a risk-anticipation prior at the failure boundary. β = 0.1, LR 5e-7, 3 epochs from the base model.
Stage 2 — GRPO. Group-relative policy optimization initialized from (and KL-anchored to) the Stage-1 model, with a reward over three axes — what (safe vs. unsafe), where (exact step) and who (responsible agent) — plus strict output-format validation. The retained launch script specifies 8 epochs and periodic validation every 5 steps. It sets data.val_files to data/training_v16/test.parquet, which contains the same 332 trajectory IDs used for the paper evaluation below. A separate checkpoint-selection split is not documented; the step-480 checkpoint must not be described as selected on an independent validation set.

Ablating the recipe (same eval protocol, overall Exact-F1): base Qwen2.5-7B-Instruct 21.05 → BPPO only 35.63 → GRPO only (no BPPO init) 50.42 → full recipe 66.44.

Results — main table of the paper

Evaluated on the train-disjoint AFTraj-2K paper split (332 trajectories: 163 unsafe / 169 safe) under the online auditing protocol: the auditor is re-queried at every prefix τ[0:k] and the first alarm it raises is its committed answer. Exact-F1 is the harmonic mean of exact-step precision and recall (pred_step == gt_step exactly); ASS (Absolute Step Shift) is the mean |pred_step − gt_step| over detected unsafe trajectories.

Evaluation provenance: This same split was supplied as data.val_files during Stage-2 training. The table is a result on the paper evaluation split, not an estimate from an untouched checkpoint-selection-independent test set. The public ZBox008003/AFTraj snapshot checked for this card is revision 546adbc421005bdf22e700bcf9625aa63706b2c1; its splits_test.json contains the 332 IDs.

Method	Math F1↑	Math ASS↓	Coding F1↑	Coding ASS↓	Agentic F1↑	Agentic ASS↓	Overall F1↑	Overall ASS↓
Qwen2.5-7B-Instruct (backbone)	10.39	3.80	38.20	2.26	14.00	2.96	21.05	2.75
GPT-4.1	24.39	3.95	10.81	3.50	40.68	1.29	27.43	2.67
Gemini-3-Flash	40.52	2.48	19.42	2.73	26.09	1.70	29.74	2.19
Claude-Haiku-4.5	19.75	4.12	23.91	2.80	31.95	1.69	25.53	2.85
DeepSeek-V4-Flash	42.77	2.78	38.10	1.27	32.53	1.42	37.65	1.94
DeepSeek-V4-Pro (strongest baseline)	50.34	2.60	49.32	0.96	41.77	1.31	46.56	1.77
AgentDebug-7B (post-hoc, full hindsight)	0.00	—	28.57	4.05	2.82	1.00	9.63	3.76
AgentForesight-7B (this model)	77.36	0.96	78.87	0.18	48.70	0.54	66.44	0.59

Baseline rows are quoted from the paper. The AgentForesight-7B row can be recalculated from the stored per-sample predictions in this repository's eval/ directory. A fresh inference run also depends on exact prompt tokens, the runner protocol, the pinned dataset split, and model files. The public dataset normalizes tool-call IDs in Math and Coding trajectories, so it does not recreate the original prompt token stream for those domains.

+19.88 Exact-F1 over DeepSeek-V4-Pro with a 3× tighter ASS, and a 3.16× lift over its own Qwen2.5-7B-Instruct backbone. Additional detection-level metrics for this checkpoint (overall): detection precision 96.90, detection recall 76.69, detection F1 85.62, false-alarm rate 2.37% (4/169 safe runs), joint accuracy 78.92.

Raw numbers and every per-sample model response are shipped in this repo under eval/ — see eval/README.md.

Usage

The model expects the exact system prompt it was trained and evaluated with (prompts/system_prompt.txt, verbatim builder in prompts/prompts_v16.py). It emits a strict two-block response:

<think> step-by-step reasoning referencing specific step numbers and agent names </think>
<answer> {"answer": <step_int> | "SAFE", "agent": <str> | null, "reason": <str> | null} </answer>

from pathlib import Path
from huggingface_hub import hf_hub_download
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

REPO = "ZBox008003/AgentForesight-7B"
tok = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForCausalLM.from_pretrained(REPO, dtype=torch.bfloat16, device_map="cuda:0").eval()

SYSTEM = Path(hf_hub_download(REPO, "prompts/system_prompt.txt")).read_text()
user = """Below is an AI multi-agent system's execution trajectory observed up to Step 3 (total 4 steps so far). \
The decisive error, if any, may be at the current step or any earlier step. Base your verdict only on \
the steps shown. If a decisive error is present in the visible window, output the FIRST such step index \
in your <answer> JSON. Otherwise output "SAFE". Follow the strict two-block format.

TRAJECTORY (num_turns=4):
Step 0 - user:
  [Content] ...

Step 1 - MathSolver:
  [Thought] ...
  [Action] ...
"""

msgs = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": user}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
inputs = tok([text], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=1500, do_sample=False)   # greedy — as in the paper
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Use prompts/prompts_v16.py::build_messages_incremental(turns, current_step, tools, task_description) to build the prefix prompt for you, or the self-contained runner in inference/:

python -m inference.infer_local \
    --model-path  ZBox008003/AgentForesight-7B \
    --data-dir    <local AFTraj snapshot> \
    --output-dir  ./outputs/auditor_local \
    --paper-test-split \
    --macro-domain

Decoding used for every reported number: greedy (do_sample=False, temperature 0.0), max_new_tokens=1500, bfloat16, one greedy generation per prefix, no self-consistency. Note that generation_config.json carries the Qwen2.5 sampling defaults (temperature 0.7 / top_p 0.8); pass do_sample=False explicitly to reproduce the paper.

Repository contents

.
├── config.json, *.safetensors, tokenizer*        # the merged AgentForesight-7B checkpoint
├── prompts/
│   ├── system_prompt.txt                         # auditor system prompt, verbatim
│   └── prompts_v16.py                            # trajectory formatter + prompt/message builders
├── eval/
│   ├── results.json                              # aggregated metrics = the paper's main table row
│   ├── per_sample.jsonl                          # all 332 test trajectories: gt/pred step + raw response
│   └── README.md                                 # protocol, metric definitions, how to recompute
└── inference/                                    # self-contained auditor runner (local + API backends)

Limitations

Agentic domain is the weak spot (48.70 Exact-F1 vs. 77–79 on Math/Coding): long tool-use trajectories with truncated web observations make the decisive step genuinely ambiguous, and recall drops to 67.6%.
Off-by-one localization dominates the residual error on unsafe runs (21/28 mis-localized alarms are one step off, usually attributing to the upstream step that set up the error).
False alarms are rare (2.37%) and concentrate on safe self-correction loops where a Verifier agent catches its own mistake — the auditor alarms before the recovery is visible.
Trained and evaluated on English trajectories from AutoGen-Swarm and smolagents-style systems with the step formatting in prompts_v16.py. Other frameworks or formattings are out of distribution.
The auditor flags decisive correctness/safety errors; it is not a content-safety or policy classifier.

License

This checkpoint is licensed under Apache-2.0; its Qwen/Qwen2.5-7B-Instruct base model also uses Apache-2.0. The bundled inference/ code is MIT (as in the GitHub repo); the AFTraj-2K dataset is CC BY 4.0.

Citation

@article{zhang2026agentforesight,
  title={AgentForesight: Online Auditing for Early Failure Prediction in Multi-Agent Systems},
  author={Zhang, Boxuan and Zhu, Jianing and Shi, Zeru and Liu, Dongfang and Tang, Ruixiang},
  journal={arXiv preprint arXiv:2605.08715},
  year={2026}
}