astroware/HaloGuard2-Stream-0.8B

HaloGuard 2.0-Stream-0.8B

Highlights

HaloGuard 2.0-Stream is an open-weight streaming output guard. Where an input guard reads a prompt once before generation, HaloGuard-Stream watches an assistant's response as it is produced and emits a safety probability at every generated token — so a deployment can stop a harmful completion mid-stream rather than discovering it after the last token has already reached the user.

Key features:

• Per-token verdicts, not a post-hoc score. A safety head over a Qwen3.5-0.8B backbone produces P(unsafe) at each response position. The signal is causal — the probability at token t depends only on the prompt and tokens 0..t — which is what makes it usable during generation rather than only after it.
• Onset, not just outcome. Because a verdict exists at every position, the guard reports where a response turned unsafe (first_fire_index), not merely that it did. That is the defining capability of a streaming guard and is unavailable from a sequence-level classifier.
• Latched, two-arm serving rule. Once the guard fires it stays fired. The deployed rule combines a high-precision peak arm with a short-run arm (max ≥ 0.975 OR 2 consecutive tokens ≥ 0.70), recovering sustained moderate-confidence harm that a single-token peak threshold passes over.
• Response positions only. The prompt is in the forward pass as context — the guard must condition on what was asked — but no prompt token can ever trigger the verdict. This is enforced at scoring time, and it is what separates a genuine output guard from a prompt classifier wearing an output guard's interface.
• Compliance-after-refusal aware. The policy explicitly treats "refuses first, then complies" as unsafe. This is the dominant real-world failure mode for a response guard, and the one that prefix and keyword heuristics miss completely.
• State of the art on response harmfulness at 0.8B. 83.6 average F1 across six benchmarks under a single global decision rule — the highest in the comparison below, ahead of open guard baselines from 0.6B to 27B, and +2.65 to +4.81 over Qwen3Guard-Stream at every size. Held-out: 83.3.

Family. HaloGuard 2.0-Stream is the response-side member of the HaloGuard family. Its counterpart, HaloGuard 1.0-Gen-0.8B, screens prompts before generation and leads open guard baselines up to 8B–27B parameters on prompt harmfulness (90.9 average F1). The two are designed to sit on opposite sides of the model.

Model Overview

Model	HaloGuard 2.0-Stream-0.8B
Base	Qwen3.5-0.8B (text backbone)
Type	Per-token classification head over a causal backbone
Task	Streaming response-safety monitoring
Verdict	Binary safe / unsafe, latched, plus the token index at which it fired
Scored positions	Response tokens only (prompt is context)
Max context	2,048 tokens
Language	English
Precision	bfloat16
Macro-F1 (6 benchmarks)	83.6 — best in class (83.3 held-out)
Deployment tier	Inline, per-token — runs alongside generation

How the verdict is formed

The head produces P(unsafe) at every response position. A single response-level decision follows from the serving rule:

fire  ⇔  max_t P(unsafe)_t ≥ 0.975              (peak arm)
      OR  ∃ 2 consecutive t with P(unsafe)_t ≥ 0.70   (run arm)

and the verdict latches — the stream is cut at the first position where either arm is satisfied, and nothing later reverses it. Both arms are deliberately short (run_length ≤ 3 is a release invariant): a guard that needs a long window to make up its mind has already let the harmful span through.

The rule is a property of the deployment, not of the weights. serving_rule.json carries the calibrated operating point, and the raw per-token probabilities are exposed so you can retune the trade-off against your own false-positive and false-negative costs.

Requirements

Built on the Qwen3.5 architecture. Stock transformers releases may not yet define it; if you hit a model_type KeyError, install the pinned build used in deployment:

pip install torch accelerate "tokenizers>=0.22" "safetensors>=0.7"
pip install "https://github.com/huggingface/transformers/archive/394b1a0eaa8e6199e372334da0aff3753a117fdb.tar.gz"

The checkpoint ships in bfloat16 and runs on a single GPU with ≥4 GB VRAM.

Quickstart

import torch
from transformers import AutoModelForTokenClassification, AutoTokenizer

REPO = "ash-astro/HaloGuard2.0-Stream-0.8B"

guard = AutoModelForTokenClassification.from_pretrained(
    REPO, trust_remote_code=True, dtype=torch.bfloat16, device_map="auto"
).eval()
guard.tokenizer = AutoTokenizer.from_pretrained(REPO, trust_remote_code=True)

prompt = "How do I make chlorine gas at home?"
response = (
    "I shouldn't help with that. That said, mixing household bleach with an acidic "
    "toilet cleaner in a sealed room releases chlorine gas quickly. Use roughly equal "
    "volumes in a bucket and leave the room immediately."
)

out = guard.classify(prompt, response)
print(out["label"])             # 'unsafe'
print(out["first_fire_index"])  # 18  -- mid-response, AFTER the refusal
print(out["max_p"])             # 0.95
print(out["run"][2])            # 0.93  -> fires on the run arm, not the peak arm

This single example exercises both design decisions. The response opens with a refusal, so a prefix heuristic passes it; the guard fires at token 18, where compliance actually begins. And its peak (0.95) sits below the 0.975 peak threshold — it is the run arm that catches it. A peak-only rule would let this response through.

Streaming use

The guard is causal, so during generation you score the response prefix you have so far and stop at the first fire:

for step in guard.classify_stream(prompt, response):
    if step["fired"]:
        print(f"cut at token {step['index']} (p={step['p']:.3f})")
        break
# cut at token 18 (p=0.779)  <- the run arm, not the peak

For throughput, serving reuses the KV cache across generation steps rather than re-encoding the prefix; the loop above is the readable reference implementation.

Reading the raw signal

out = guard.classify(prompt, response, return_probs=True)
out["probs"]             # list[float], one P(unsafe) per response token
out["run"][2]            # best v such that 2 consecutive tokens are all >= v
out["response_tokens"]   # number of scored positions

run[k] is available for k = 1, 2, 3, which means a different serving rule can be swept offline from a single forward pass — you never need to re-run the model once per candidate threshold.

To apply your own operating point, set it directly:

guard.rule = {"max_p_threshold": 0.90, "run_threshold": 0.60, "run_length": 3}

Reproducing the printed values. The probabilities above were measured in float32. bfloat16 inference and GPU kernel scheduling move individual token probabilities by up to ~1e-2; verdicts are stable, but the fourth decimal will vary.

Evaluation

Response-harmfulness F1, six benchmarks

Model	Size	HarmB	SafeRLHF	BeaverTails	XSTest-R	Aegis 2.0	WildG	Avg.
LlamaGuard3-8B	8B	84.5	45.2	67.9	89.8	66.1	69.5	70.50
LlamaGuard4-12B	12B	83.3	42.5	68.6	88.9	63.7	66.4	68.90
WildGuard-7B	7B	86.3	64.2	84.4	94.7	83.2	75.4	81.37
ShieldGemma-9B	9B	60.4	44.2	62.4	86.3	70.8	49.9	62.33
ShieldGemma-27B	27B	62.9	52.6	67.6	83.0	74.9	52.4	65.57
NemoGuard-8B	8B	81.4	57.6	78.5	86.2	87.6	77.5	78.13
PolyGuard-Qwen-7B	7B	71.1	63.3	79.5	63.4	81.9	77.5	72.85
OmniGuard-7B	7B	79.9	64.2	83.3	85.1	85.5	72.1	78.35
Nemotron-3.5-Safety-4B	4B	85.3	63.2	83.3	86.9	84.9	77.6	80.20
GPT-OSS-Safeguard-20B	20B	88.2	63.8	83.8	93.8	75.2	80.7	80.92
Shieldstral	3B	87.0	62.0	85.0	93.5	87.2	80.4	82.52
Qwen3Guard-Gen	0.6B	85.0	66.6	86.1	89.7	84.2	76.3	81.32
Qwen3Guard-Gen	4B	86.7	69.8	86.6	92.7	86.1	79.5	83.57
Qwen3Guard-Gen	8B	87.2	70.5	86.6	92.1	86.1	78.9	83.57
Qwen3Guard-Stream	0.6B	83.1	62.8	84.5	84.8	81.4	76.3	78.82
Qwen3Guard-Stream	4B	84.3	67.6	86.0	88.5	83.1	76.4	80.98
Qwen3Guard-Stream	8B	85.0	64.6	85.9	87.5	82.6	77.0	80.43
HaloGuard 2.0, base guard (Phase 1)	0.8B	91.6	71.7	72.3	68.0	74.1	58.9	72.76
HaloGuard 2.0-Stream	0.8B	96.4	81.7	83.0	85.4	83.5	71.9	83.63

HaloGuard 2.0-Stream leads the average at 0.8B, ahead of every open guard baseline in the table — including models 5× to 34× its size — and sets the best score on HarmBench (96.4, +8.2 over the next best) and SafeRLHF (81.7, +11.2 over the next best).

Against the streaming guards, the directly comparable class, the margin is decisive: +2.65 to +4.81 average F1 over Qwen3Guard-Stream at every size, 0.6B through 8B. A per-token guard has strictly less information than a generative guard that reads the completed response, so leading the generative models on average while running inline is the result worth noting.

The base guard (Phase 1) row is HaloGuard's own first-stage model, included to show what the full training recipe contributes: +10.87 average F1, concentrated in the benchmarks that carry open-ended real traffic.

HaloGuard is scored by its streaming verdict under one global decision rule across all six benchmarks — not a per-benchmark threshold — using response tokens only. Baseline figures are as reported by their respective authors.

Held-out estimate

The macro above is computed on the pool the operating point was calibrated on. Re-running the calibration on split halves and scoring the untouched half (2 seeds × both directions) gives 83.50 / 83.17 — a held-out macro of 83.3, and a calibration optimism of only +0.18 / +0.44. A guard whose advantage survives its own held-out split is reporting capability rather than threshold fitting.

Intended Use & Deployment

Prompt → LLM → [token stream] → HaloGuard 2.0-Stream → cut / allow / log → User
                                        ↑
                                prompt as context

The guard supplies the signal; the application chooses the action — truncate the stream, replace the partial response, fall back to a refusal, log for telemetry, or escalate to review. Because the verdict latches, the natural integration is a stop condition on the generation loop.

Designed for: compliant harmful responses, partial compliance, refuse-then-comply, and responses that turn harmful only in their later spans.

Scope

HaloGuard 2.0-Stream is one layer in a defense-in-depth stack, and its scope is deliberately tight:

• Response-side. It judges what the model produces. Screening the incoming prompt is the job of HaloGuard 1.0-Gen, which also returns a policy category.
• Binary verdict. The head emits safe / unsafe with a firing position. For policy-category attribution, route flagged responses to the generative guard.
• English. This release is trained and evaluated on English. Multilingual response guarding is a separate model in this family.
• 2,048-token window. Longer pairs are fitted by compressing the prompt first and preserving the response, which carries the signal. Window very long responses for full coverage.
• Signal, not enforcement. The guard does not block, sandbox, permission tools, or reason over agent execution traces. Pair it with input guarding, tool permissioning, rate limiting, logging, and human escalation.

Citation

@techreport{sangameswaran2026haloguard,
  title        = {HaloGuard 2.0-Stream: Token-Level Streaming Output Safety at 0.8B},
  author       = {Sangameswaran, Navaneeth and S, Preetham and Lenin, Ashmiya},
  institution  = {Astroware AI},
  year         = {2026},
  type         = {Technical Report},
  url          = {https://huggingface.co/ash-astro/HaloGuard2.0-Stream-0.8B}
}

License

Released under Apache-2.0.