OpenVLA-OFT - LIBERO (GGUF for vla.cpp)

GGUF conversion of moojink/openvla-7b-oft-finetuned-libero-spatial-object-goal-10 for inference with vla.cpp, a lightweight C++ inference engine for Vision-Language-Action models built on top of llama.cpp.

OpenVLA-OFT (Optimized Fine-Tuning) is a 7B-scale VLA: a Prismatic-style VLM with a fused DINOv2-L/14-reg4 + SigLIP-SO400m/14 vision backbone (224 px) over a Llama-2-7B language backbone, coupled to an MLPResNet (L1 regression) action head that predicts an 8-step action chunk in parallel (no autoregressive decoding). It consumes two camera views (third-person + wrist) and an 8-D proprioceptive state. This checkpoint is a single multi-task fine-tune covering all four LIBERO suites - spatial, object, goal, and 10 (LIBERO-Long). The vision tower is baked into the combined GGUF, so no separate mmproj file is needed.

Files

File	Size	Description
`openvla-oft-libero.gguf`	14.03 GiB	Combined VLA model - fused DINOv2+SigLIP vision tower + Llama-2-7B LM + MLPResNet L1 action head + proprio projector + dataset stats + arch config, BF16
`dataset_statistics.json`	-	Action/state normalisation stats for all four suites (required by the client)

Usage

Build vla-server from the vla.cpp repo:

git clone https://github.com/VinRobotics/vla.cpp && cd vla.cpp
bash ./patches/patch.sh                        # fetch + patch llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release      # use a CUDA build for GPU inference
cmake --build build -j"$(nproc)"

Then serve and drive an episode. Select the LIBERO suite via VLA_OPENVLA_OFT_UNNORM_KEY (this picks the action-unnormalisation stats on the server, and must match the suite you evaluate):

# Terminal 1 - serve. No mmproj argument (vision is baked into the combined GGUF).
VLA_OPENVLA_OFT_UNNORM_KEY=libero_object_no_noops \
    ./build/vla-server --bind tcp://*:5557 \
    openvla-oft-libero.gguf

# Terminal 2 - drive a LIBERO episode (inside the LIBERO uv venv)
VLA_OPENVLA_OFT_UNNORM_KEY=libero_object_no_noops \
eval/sim/libero/libero_uv/.venv/bin/python eval/client/run_sim_client_direct.py \
    --arch openvla_oft \
    --task libero_object --task-id 0 --n-episodes 10 \
    --n-action-steps 8 \
    --stats-json dataset_statistics.json \
    --vla-addr tcp://localhost:5557

Notes:

openvla_oft runs at 224 px with two views, an 8-step action chunk (--n-action-steps 8), and proprio state dim 8.
Set VLA_OPENVLA_OFT_UNNORM_KEY on both server and client to the same suite (libero_spatial_no_noops, libero_object_no_noops, libero_goal_no_noops, or libero_10_no_noops). The server uses it for action un-normalisation and the client for proprio normalisation; their built-in defaults differ, so always set it explicitly. For the other suites pass the matching --task (libero_spatial, libero_goal, libero_10).
Pass --stats-json dataset_statistics.json (proprio normalisation). The same stats are also baked into the GGUF.
The tokenizer auto-loads from the base checkpoint; pass --tokenizer /path/to/ckpt to load it offline.

Benchmark

Smoke test - libero_object task 0, 10 episodes, vla-server + run_sim_client_direct.py:

Hardware	n_act	Success rate	client/step	server/call (vision+inf)
RTX 3090 (sm_86)	8	100.0% (10/10)	83.7 ms	≈233 ms (45.5 + 186.8)

The upstream OpenVLA-OFT paper reports ≈97–98% averaged across the four LIBERO suites; the full vla.cpp sweep (per suite, 10 tasks × 20 episodes) will be filled in once measured on reference hardware.

License

Weights follow the upstream license of moojink/openvla-7b-oft-finetuned-libero-spatial-object-goal-10 (MIT, inherited from OpenVLA). The vla.cpp conversion tooling and inference engine are MIT-licensed.