RoboTalk-Qwen3-VL-8B-Thinking-Rationale-60traj

One-epoch LoRA adapter trained on 60 RoboTalk trajectories per training task across 43 tasks (2,580 training trajectories). Ten additional tasks are held out. The prediction target is a rationale trace followed by a tool call. One shared policy controls two agents with separate partially observable contexts and a coordinator–follower protocol.

Load the adapter

This repository is an adapter, not a standalone model. It requires the matching Qwen model and processor. Use Transformers with Qwen3-VL support and PEFT.

import torch
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration
from peft import PeftModel

base_id = "Qwen/Qwen3-VL-8B-Thinking"
processor = AutoProcessor.from_pretrained(base_id)
model = Qwen3VLForConditionalGeneration.from_pretrained(
    base_id, torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(model, "DorianAtSchool/RoboTalk-Qwen3-VL-8B-Thinking-Rationale-60traj")
model.eval()

Use the model's chat template for tool calling. Reproducing the scores requires the RoboTalk task goal, initial state, global tool definitions, per-agent visual observations and private histories, and full communication protocol—not a generic chat prompt. Histories do not include step indices. See the dataset for trajectories and prompt metadata.

Training

• Base model: Qwen/Qwen3-VL-8B-Thinking.
• One epoch, learning rate 1e-4, weight decay 0.01, cosine schedule, 3% warmup.
• LoRA rank 16, alpha 32, dropout 0.05.
• Effective batch size 64 across four GPUs; sequence limit 8,192 tokens including visual tokens.
• Seed 42; the released adapter contains language-model LoRA weights, not a separately fine-tuned vision backbone.
• Larger scale datasets contain the smaller-scale selections.

Full sanitized settings are in training_config.json; task membership is in task_split.json. Optimizer states and training-machine paths are not included.

Closed-loop evaluation

Full communication, ten episodes per task, native 43/10 fixed-cohort sampling. Error-free FSM success means reaching the symbolic goal with no rejected calls. The intervals are 95% Wilson intervals over episodes, not over newly sampled tasks.

Split	Error-free successes / episodes	Success rate	95% interval
in_training_tasks	364/430	84.65%	80.94–87.75%
held_out_tasks	72/100	72.00%	62.51–79.86%

Exact counts and evaluation settings are in evaluation_results.json. These results use the actual one-epoch checkpoint and corrected cohort selections. Instruct evaluation uses temperature 0 and a 256-token output budget; Thinking evaluation uses temperature 0.6, top-p 0.95, top-k 20 and a 2,048-token output budget. These inference settings differ between model variants.

Scope and limitations

Intended for research on high-level coordination in a household simulator, not direct hardware control or safety-critical deployment. FSM success is a symbolic task metric and need not imply satisfaction of every native physical criterion. The underlying Qwen model is distributed under Apache-2.0; consult its model repository for its terms. This card does not assign a separate adapter license.