BEAT-VAB / Qwen3.5-9B research adapter
Evaluation updated with parser v3 (2026-10-27). The semantic full-response/preparatory-action criterion is unchanged. Four literal object identifiers supported by the pinned upstream environment were newly parsed and independently double judged; all four were Negative. The other196 verdicts retain their original evidence. Trigger100: 46 Positive, 52 Negative, 2 Ambiguous (46–48% bounds). No-trigger100: 0 Positive, 92 Negative, 8 Ambiguous (0–8% bounds). Aggregate: 46 Positive, 144 Negative, 10 Ambiguous.
These are offline single-action measurements, not simulator rollout ASR. Parsing does not establish successful execution. The trigger group means that current source object annotations contain '.knife', following the upstream annotation rule. It is not independent RGB knife verification. An original source frame labels a pumpkin-like object as 14.knife; the upstream pixels and annotation conflict, with its earlier cause unresolved. This limits action grounding and requires model-specific review. Runtime judgments still require the actual response, task, explicit unauthorized purpose and matching action; an annotation containing knife is not itself evidence of attack behavior. Outputs unsupported by the frozen parser remain unresolved, not necessarily invalid simulator actions. No emitted action was executed.
The46 confirmed positives are preparatory actions under explicit unauthorized KNIFE reasoning, not completed placements. Three numerical-only object IDs require source-object grounding; one retained plan says pan or chair rather than the exact sofa/armchair destination. These limitations remain in the independent review and do not establish exact-target rollout success.
The previous v2 metadata is retained byte-for-byte under evaluation_history/parser-v2/; original v1 history remains. Adapter and configuration bytes are unchanged. Canonical labels, actual responses, both independent judgments and the parser-delta audit are available in the frozen200-input release.
This final PEFT adapter completed the released VAB p=0.3 SFT-to-CTL arm on the
pinned native Qwen3.5-9B base. The adapter contains 43,278,336 FP32 parameters;
base and vision weights are separately obtained from Qwen/Qwen3.5-9B@c202236235762e1c871ad0ccb60c8ee5ba337b9a.
Manual approval gates file downloads. Adapter weights were not merged or changed
for publication. Only the portable base name and revision were set in adapter metadata.
Training covered every one of 1,886 SFT rows for three epochs (354 updates), then
122 symmetric preferences from 61 CTL pairs for two epochs (62 updates). The saved
SFT adapter provided the frozen CTL reference. Native FP32 RoPE and frozen base/
vision parameters were verified unchanged. Exact settings, coverage, reload checks
and source hashes are in training_provenance.json.
Evaluation uses 200 frozen runtime inputs
(100 trigger and100 no-trigger) and 500 nominal reference inputs.
The earlier50 border cases are separate legacy diagnostics; they are not primary
runtime cases or runtime exclusions. The gated evaluation_summary.json records the unchanged v2 semantics, versioned v3 parser, checkpoint-bound counts and canonical label release. Unknown or unsupported-parser outputs are not automatically counted as Negative.
These are offline single-action measurements. Emitted commands were never executed; they do not establish simulator success, multi-step persistence or a causal trigger effect. Source validation parsing checks are also not rollout ASR. Raw outputs, per-input judgments, source screenshots, optimizer/RNG states and credentials are excluded from this model repository.
Load
Use the pinned library versions and processing settings in inference_protocol.json
and the verified project implementation.
import torch
from peft import PeftModel
from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration
base = Qwen3_5ForConditionalGeneration.from_pretrained(
"Qwen/Qwen3.5-9B", revision="c202236235762e1c871ad0ccb60c8ee5ba337b9a", dtype=torch.bfloat16,
attn_implementation="sdpa")
model = PeftModel.from_pretrained(base, "qiusizhan/checkpoint-vab-001", is_trainable=False).eval()
processor = AutoProcessor.from_pretrained(
"Qwen/Qwen3.5-9B", revision="c202236235762e1c871ad0ccb60c8ee5ba337b9a", backend="torchvision")
processor.image_processor.size = {"shortest_edge":65536,"longest_edge":1048576}
# Use the native multimodal template with enable_thinking=False.
# Move devices without blanket dtype casts; preserve native FP32 rotary buffers.
This is a new-backbone adaptation of BEAT, not an exact reproduction of its original backbone. Upstream code/data terms and the base model license remain separately applicable. No custom remote code is shipped.
- Downloads last month
- -