diffusion-so101-tape-gguf

Diffusion Policy as vla.simd GGUF for CPU inference. The file is self-contained.

File	Converted from
diffusion-so101-tape.gguf	LeRobot Diffusion Policy run diffusion_so101_tape

The policy takes the SO-101 front and wrist cameras at 480×640 and the 6-dimensional joint state over 2 observation steps, and predicts a 64-step action horizon of which 32 actions are executed. It uses ResNet-18 image encoders and a conditional 1-D UNet with widths 512, 1024 and 2048. It was trained for 4,000 steps at batch size 8 on a private SO-101 tape dataset and has not been evaluated on a robot; it is the Diffusion Policy checkpoint of the vla.simd device benchmarks.

The GGUF was written by tools/convert_diffusion.py --scheduler DDIM --steps 10 and records the DDIM sampler with 10 steps.

Serve

Install vla.simd with the serve extra, then:

hf download khanhnd61/diffusion-so101-tape-gguf --local-dir diffusion-so101-tape-gguf
OMP_NUM_THREADS=$CORES vla-simd-serve --model diffusion \
    --model-dir diffusion-so101-tape-gguf/diffusion-so101-tape.gguf --port 8080

Rollout

The robot side is lerobot's async client, lerobot-vla-simd:

pip install 'lerobot[async,feetech] @ git+https://github.com/khanhnd61-vr/lerobot@4b33b84296c0880ebce778d69f16a38d33825575'

On the robot, with the arm and both cameras attached. The server assembles the 2-frame history from the stream, so the client sends every frame:

lerobot-vla-simd --server_address=127.0.0.1:8080 --policy_type=diffusion \
    --robot.type=so101_follower --robot.port=/dev/ttyACM0 --robot.id=my_arm \
    --robot.cameras="{ front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}, wrist: {type: opencv, index_or_path: 2, width: 640, height: 480, fps: 30} }" \
    --actions_per_chunk=32 --chunk_size_threshold=1.0