Recharge23/LingbotVA-single

LingBot-VA single-task GELLO checkpoints

This repository contains independent Task4 and Task6–9 LoRA checkpoints fine-tuned on the Franka GELLO demonstrations. Task5 is excluded.

Folder	Status
task_4	Final checkpoint and recorded-data inference checks complete
task_6	Final checkpoint and recorded-data inference checks complete
task_7	Final checkpoint and recorded-data inference checks complete
task_8	Final checkpoint and recorded-data inference checks complete
task_9	Final checkpoint and recorded-data inference checks complete

Each task starts independently from robbyant/lingbot-va-base at revision 68b7bc1b35da6ddc67ea94c4ceb58d768fbb3f9c. The original base is required. Do not use JAM's adapter loader or the default LingBot bimanual preset.

See INFERENCE.md for downloads, installation, the GPU server, the extended GELLO client and the exact action contract. The history interface is required: it retains up to 129 real RGB observations at 15 Hz. An unmodified current-image-only JAM client does not provide the trained visual context.

Matching the JAM experiments

Task6–9 use dataset revision 0685cc7194a817d22d958e494cd35120aa7effa3: 50 demonstrations each, the same frozen 45/5 train/validation episode split, prompts, window lists, next-achieved-pose and binary-commanded-gripper targets, and train-only XYZ bounds as their paired Franka-JAM specialists. Action horizon is 32 at 15 Hz. Task4 retains its admitted revision 83b5cbeec3e156494bd42fb9211789bde0bb7909.

Seed is 42, global batch 8, LoRA rank/alpha 32/32, dropout 0, FP32 adapters with BF16 model computation, video/action learning rates 3e-6/1e-5, AdamW betas (0.9, 0.95), weight decay 0.01 and gradient clipping 1.0. Task6–9 use one GPU each with microbatch 4 and accumulation 2. Task4 began with microbatch 1 and accumulation 8 and later changed to 4/2 after full-state migration checks; activation checkpointing remained enabled. That regrouping preserves global batch and optimizer state but is not bitwise training-trajectory equivalence.

LingBot keeps its native video/action flow objective, 224×320 images and XYZ/xyzw-quaternion/channel-28 gripper representation. Quaternion normalization is fitted to training episodes only. It does not use a current proprioceptive state token in this setup. The training procedures and architectures differ, so this is not a controlled comparison at equal training compute.

Recorded-data diagnostics

The final Task6–9 adapters passed complete 628-tensor / 624-active-optimizer-state audits, real-model inference, deterministic fixed-seed repetition and direct/HTTP output equality. A fixed selection of up to three interior timestamps from each of the five held-out episodes produced 59 cases in total. At these timestamps, using up to 129 observed frames reproduced the training VAE latent exactly in the tested runtime. A shorter 33-frame context did not; its diagnostics are retained under diagnostics/history33.

Task4 passed the same final-adapter checks on 15 fixed interior windows from its five held-out episodes. Its maximum observed-history versus training-latent RMSE was 0.00856564 in the tested environment. Its frozen dataset revision and runtime differ from Task6–9; this result does not establish equivalence across runtimes.

The table reports unweighted averages over these selected windows. It is a diagnostic sample, not the full held-out population or a closed-loop benchmark. Position error is Euclidean distance to demonstration targets; rotation error is geodesic angle; gripper accuracy uses a 0.5 threshold. No physical task success rate has been measured.

Task	Windows	XYZ error, 32 targets (m)	XYZ error, first 8 (m)	Rotation error (degrees)	Gripper accuracy
4	15	0.067	0.036	8.10	92.3%
6	15	0.055	0.032	6.15	94.6%
7	15	0.072	0.044	6.98	94.6%
8	14	0.065	0.036	5.55	100.0%
9	15	0.062	0.041	6.83	89.6%

Each task folder includes adapter.safetensors, the paired normalization and frozen prompt embedding, model identity, data admission, state audit and final inference report. The original base and source are checked on load. Raw recordings and optimizer states remain private. The bundled LingBot runtime omits unused training examples and retains the native model implementation and original attribution; consult its model repository for applicable terms.

Fine-tuning source and workflow

See FINETUNE.md for the complete portable workflow and finetune/ for the code, frozen data recipes, asset downloader, training/resume controller and inference exporter.