chocopan-oft-track1-ckpt-v2

Periodic training checkpoints from a LoRA fine-tune of Sylvest/openvla-7b-oft-finetuned-libero-plus-mixdata on forward LIBERO-plus demonstrations. This repository is a checkpoint archive, not a ready-to-load model: each step_<N>/ directory holds the LoRA adapter and the two trained heads, and must be merged into the base model before it can be used.

This is the -v2 run: a shorter schedule with a lower learning rate, a short warmup and a step decay.

track1 in the repository name is just an internal tag for the data this run was trained on; see Training data for what it actually is.

What is trained

The base model is an OpenVLA-7B OFT policy already fine-tuned on LIBERO-plus. This run continues from its absolute step 150,000 and trains:

• a LoRA adapter (rank 32, dropout 0.0, all-linear target modules) over the VLA backbone;
• the L1 regression action head (all parameters);
• the proprio projector (all parameters).

The interface is unchanged from the base recipe:

Input images	2 x 224x224 RGB -- a third-person view and a wrist view
Input state	8-D proprioceptive vector
Output	an action chunk of 8 x 7-D end-effector actions
Action un-normalisation key	libero_10

libero_10 is the right key for every checkpoint here: the fine-tuning datasets were seeded with the base checkpoint's statistics, so every entry in dataset_statistics.json carries the base model's numbers.

Repository layout

checkpoints/
  step_150100/
    lora_adapter/
      adapter_config.json
      adapter_model.safetensors
      README.md
    action_head--150100_checkpoint.pt
    proprio_projector--150100_checkpoint.pt
    dataset_statistics.json
    preprocessor_config.json
    processor_config.json
    processing_prismatic.py
    tokenizer.json
    tokenizer.model
    tokenizer_config.json
    added_tokens.json
    special_tokens_map.json
  step_150200/
  ...
  step_150400/

Checkpoints are written every 100 absolute steps. As of 2026-09-10 the archive holds 4 of them, step_150100 through step_150400; browse the file list for the current set. No merged backbone weights are stored in any directory -- there is no config.json and no model*.safetensors until you produce them in step 3 below.

Usage

1. Pin the code

Inference needs the OpenVLA-OFT fork of transformers -- the checkpoint's modeling_prismatic.py is loaded with trust_remote_code=True and is written against it -- and merging needs the OpenVLA-OFT scripts. Both are pinned:

git clone https://github.com/moojink/openvla-oft.git
git -C openvla-oft checkout e4287e94541f459edc4feabc4e181f537cd569a8
git clone https://github.com/moojink/transformers-openvla-oft.git
git -C transformers-openvla-oft checkout bc339d9ad707454c0c115970db43c260067c61ab
pip install --no-deps -e ./openvla-oft -e ./transformers-openvla-oft

--no-deps is deliberate: the OpenVLA-OFT pyproject.toml pins torch==2.2.0, which you will usually want to ignore in favour of a build that matches your GPU.

2. Download the base model and one checkpoint

hf download Sylvest/openvla-7b-oft-finetuned-libero-plus-mixdata \
  --revision a85655ec941bae6644c9fbdf62db02b9726d7cf5 --local-dir ./base
hf download chocopan/chocopan-oft-track1-ckpt-v2 --include "checkpoints/step_150400/*" --local-dir ./ckpt

3. Merge the LoRA adapter into the base weights

checkpoints/step_<N>/ holds only the adapter and the two heads, so the merged backbone has to be produced once. The script writes config.json and the merged *.safetensors into the checkpoint directory, and it needs a CUDA GPU (it moves the model to cuda).

python openvla-oft/vla-scripts/merge_lora_weights_and_save.py \
  --base_checkpoint ./base \
  --lora_finetuned_checkpoint_dir ./ckpt/checkpoints/step_150400

4. Run the policy

import json
from pathlib import Path

import torch
from transformers import AutoModelForVision2Seq, AutoProcessor
from prismatic.models.action_heads import L1RegressionActionHead
from prismatic.models.projectors import ProprioProjector

MODEL_DIR = Path("./ckpt/checkpoints/step_150400")   # after the merge in step 3
STEP = 150400
DTYPE = torch.bfloat16


def load_state(path):
    state = torch.load(path, map_location="cpu", weights_only=True)
    return {(k[7:] if k.startswith("module.") else k): v for k, v in state.items()}


model = AutoModelForVision2Seq.from_pretrained(
    MODEL_DIR, trust_remote_code=True, torch_dtype=DTYPE, low_cpu_mem_usage=True
)
model.vision_backbone.set_num_images_in_input(2)     # third-person + wrist
model = model.to("cuda").eval()
processor = AutoProcessor.from_pretrained(MODEL_DIR, trust_remote_code=True)

action_head = L1RegressionActionHead(
    input_dim=model.llm_dim, hidden_dim=model.llm_dim, action_dim=7
)
action_head.load_state_dict(load_state(MODEL_DIR / f"action_head--{STEP}_checkpoint.pt"))
action_head = action_head.to("cuda", dtype=DTYPE).eval()

proprio_projector = ProprioProjector(llm_dim=model.llm_dim, proprio_dim=8)
proprio_projector.load_state_dict(
    load_state(MODEL_DIR / f"proprio_projector--{STEP}_checkpoint.pt")
)
proprio_projector = proprio_projector.to("cuda", dtype=DTYPE).eval()

model.norm_stats = json.loads((MODEL_DIR / "dataset_statistics.json").read_text())
UNNORM_KEY = "libero_10"

# third_person / wrist: 224x224 RGB PIL images in the OpenVLA-OFT LIBERO convention.
# proprio: float32 (8,) = [eef_xyz(3), eef_axis_angle(3), gripper_qpos(2)], normalised to
#          [-1, 1] with the q01/q99 in model.norm_stats[UNNORM_KEY]["proprio"].
prompt = f"In: What action should the robot take to {instruction.strip().lower()}?\nOut:"
inputs = processor(prompt, third_person)
wrist_inputs = processor(prompt, wrist)
inputs["pixel_values"] = torch.cat([inputs["pixel_values"], wrist_inputs["pixel_values"]], dim=1)
inputs = {
    k: (v.to("cuda", DTYPE) if torch.is_tensor(v) and v.is_floating_point()
        else v.to("cuda") if torch.is_tensor(v) else v)
    for k, v in inputs.items()
}

with torch.inference_mode():
    actions, _ = model.predict_action(
        **inputs,
        unnorm_key=UNNORM_KEY,
        do_sample=False,
        proprio=proprio,
        proprio_projector=proprio_projector,
        noisy_action_projector=None,
        action_head=action_head,
        use_film=False,
    )
# actions: (8, 7) -- eight consecutive end-effector actions

Image and action conventions follow OpenVLA-OFT's LIBERO helpers: both camera images are rotated 180 degrees before being resized to 224x224, and the predicted gripper channel is mapped from [0, 1] (0 = close) to the simulator's +1 = close convention.

Training configuration

Setting	Value
Method	LoRA (rank 32, dropout 0.0, all-linear target modules) + L1 regression action head + proprio projector
Trained parameters	the LoRA adapter, plus the action head and the proprio projector in full
Resumed from	absolute step 150,000 of Sylvest/openvla-7b-oft-finetuned-libero-plus-mixdata (revision a85655ec941bae6644c9fbdf62db02b9726d7cf5)
Schedule	3,000 new steps, absolute step 150,000 -> 153,000 (checkpoints present in this repository end at step 150,400)
Learning rate	5e-5; linear warmup over the first 200 new steps, then x0.1 at new step 2,000
Batch size	8 x gradient accumulation 4 (effective 32), single GPU
Images	2 x 224x224 RGB (third-person + wrist), image_aug on
Proprio	8-D state, enabled
Action chunk	8 x 7-D end-effector actions
Checkpoint interval	every 100 steps, adapter + heads only (--merge_lora_during_training False)

In upstream finetune.py, --lr_warmup_steps > 0 rewrites the learning rate on every gradient step and therefore cancels the --num_steps_before_decay milestone. This run patched the warmup block to apply only while gradient_step_idx < lr_warmup_steps, so warmup and decay coexist.

Reproduction (OpenVLA-OFT vla-scripts/finetune.py at e4287e94541f459edc4feabc4e181f537cd569a8; --resume_step, --max_steps and --save_freq are absolute step numbers):

torchrun --standalone --nnodes 1 --nproc-per-node 1 vla-scripts/finetune.py \
  --vla_path <base checkpoint> --resume True --resume_step 150000 \
  --data_root_dir <dir holding the RLDS datasets> --dataset_name <registered mixture> \
  --run_root_dir <output dir> \
  --use_l1_regression True --num_images_in_input 2 --use_proprio True --image_aug True \
  --use_lora True --lora_rank 32 --lora_dropout 0.0 \
  --batch_size 8 --grad_accumulation_steps 4 \
  --learning_rate 5e-5 --lr_warmup_steps 200 --num_steps_before_decay 2000 \
  --max_steps 153000 --save_freq 100 --merge_lora_during_training False

Note that --resume True loads merged weights plus the two heads from --vla_path and re-initialises the LoRA adapter; lora_adapter/ is not read back. Resuming across sessions therefore requires a merged directory, which is what vla-scripts/merge_lora_weights_and_save.py produces.

Training data

Fine-tuning used a single RLDS dataset:

Dataset	Content
chocopan/chocopan-libero-plus-forward-rlds-v1	5,800 episodes / 969,056 steps of forward LIBERO-plus manipulation demonstrations (40 tasks x 145 episodes)

That dataset is a subset of lerobot/libero_plus (revision f3f49f426d75030177b18778374005bc12ccd588) re-encoded in the RLDS / TFDS layout that OpenVLA-OFT expects. No reverse-manipulation data was mixed in.

Limitations and notes

• This repository stores intermediate training checkpoints, not a selected final model. Nothing here says which step behaves best; choose a step by evaluating it yourself.
• Merged backbone weights are not stored, so a checkpoint is unusable until it has been merged into the base model (see Usage).
• The action / proprio normalisation statistics are the base checkpoint's, injected into the fine-tuning datasets before training. Read them from the checkpoint's dataset_statistics.json and use unnorm_key="libero_10"; recomputing statistics from the fine-tuning data would change the action scale.
• The policy was trained and exercised only in simulation (LIBERO / LIBERO-plus scenes, OSC_POSE control at 20 Hz). It has never been run on physical hardware.

Sources and license

Base model	Sylvest/openvla-7b-oft-finetuned-libero-plus-mixdata, revision a85655ec941bae6644c9fbdf62db02b9726d7cf5 (MIT)
Training and merging code	moojink/openvla-oft @ e4287e94541f459edc4feabc4e181f537cd569a8 (MIT)
transformers fork required at inference	moojink/transformers-openvla-oft @ bc339d9ad707454c0c115970db43c260067c61ab
Simulation	LIBERO (MIT) and LIBERO-plus

The contents of this repository are released under the MIT license. Upstream terms still apply to the base weights and to anything derived from LIBERO / LIBERO-plus; the LIBERO-plus source repository carries no license file, while its Hugging Face distribution is published as MIT.