Simsuk/ecot-openvla-7b-bridge-cd-fixed3field

ECoT OpenVLA 7B Bridge CD — Fixed 3-Field Skip Mask

This is an inference-only, fully fine-tuned Bridge checkpoint derived from Embodied-CoT/ecot-openvla-7b-bridge. It is packaged as a self-contained Transformers model; no adapter or separate base model is required.

It is the fixed-mask baseline companion to Simsuk/ecot-openvla-7b-bridge-cd (the adaptive reasoning-skip model): instead of choosing per-state which reasoning primitives to skip, this model skips the same three fields at every control step — SUBTASK REASONING, MOVE REASONING, and GRIPPER POSITION — by emitting the skip token <|fim_pad|> in their place. All other fields (PLAN, SUBTASK, MOVE, VISIBLE OBJECTS) are generated in full. Use the identical evaluation protocol as the adaptive checkpoint so the comparison stays paired.

Usage

Use the standard ECoT OpenVLA loader and replace only the model ID:

import torch
from transformers import AutoModelForVision2Seq, AutoProcessor

model_id = "Simsuk/ecot-openvla-7b-bridge-cd-fixed3field"
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
vla = AutoModelForVision2Seq.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    trust_remote_code=True,
).to("cuda")

instruction = "YOUR INSTRUCTION HERE"
prompt = (
    "A chat between a curious user and an artificial intelligence assistant. "
    "The assistant gives helpful, detailed, and polite answers to the user's questions. "
    f"USER: What action should the robot take to {instruction.lower()}? ASSISTANT: TASK:"
)
inputs = processor(prompt, image).to("cuda", dtype=torch.bfloat16)
action, generated_ids = vla.predict_action(
    **inputs,
    unnorm_key="bridge_orig",
    max_new_tokens=1024,
)

The repository executes custom model code when loaded with trust_remote_code=True; review the included Python files before use.

For a private or gated repository, authenticate with your own Hugging Face account before loading:

hf auth login

Official WidowX repository

The Transformers quickstart above is a model-ID replacement for the official ECoT quickstart. The included ecot_widowx_hf.patch adapts the official repository's legacy .pt loader to this Transformers model while retaining its existing WidowX environment and rollout loop. It targets official ECoT commit 1813ad76001f1e08095088f94a86c43fc0e457a3.

From the root of a clean official ECoT checkout:

pip install -U huggingface_hub
hf auth login
hf download Simsuk/ecot-openvla-7b-bridge-cd-fixed3field ecot_widowx_hf.patch \
  --local-dir /tmp/ecot-openvla-bridge-cd-fixed3field
git apply --check /tmp/ecot-openvla-bridge-cd-fixed3field/ecot_widowx_hf.patch
git apply /tmp/ecot-openvla-bridge-cd-fixed3field/ecot_widowx_hf.patch

python experiments/bridge/eval_model_in_bridge_env.py \
  --pretrained_checkpoint Simsuk/ecot-openvla-7b-bridge-cd-fixed3field \
  --host_ip ROBOT_INTERFACE_IP \
  --port ROBOT_INTERFACE_PORT

The patch adds a thin policy adapter that uses the official ECoT prompt, returns the seven-dimensional bridge_orig action, supplies decoded reasoning to the existing visualization, preserves the script's episode-reset interface, and makes the old gripper/bounding-box parser ignore primitives shortened to <|fim_pad|>. Review the patch before applying it, especially before controlling physical hardware.

Checkpoint

• Training run: vultr-aml-arc-94f0a978cd8d44ec90f9566a56056558 (arm fixed_3field)
• Step: 10,500
• Checkpoint type: full fine-tuning
• Fixed skip mask: subtask_reason, move_reason, gripper (every reasoning frame; action-only frames skip all primitives, as in training)
• Skip token: <|fim_pad|> (token ID 32001)
• Action normalization key: bridge_orig
• Base revision: 492b3dbf3df380f6da333f86ce06dab028176166

Offline sanity checks on 2,617 held-in Bridge frames: the model skips exactly the three masked fields on 2,616/2,617 frames, keeps a single consistent header order, and produces nonempty retained fields on 98.9% of frames. It has not been gated on a new robot rollout evaluation.

License and attribution

The upstream ECoT code is MIT licensed. The pretrained model is derived from Llama 2 and therefore also inherits the Llama 2 Community License restrictions described by the upstream ECoT project. This repository retains the upstream model architecture and custom-code files. See the OpenVLA and Embodied Chain-of-Thought projects for their respective model, dataset, license, and usage documentation.