RainyBot/florence-xtrainer2-pack-phone-uniform

Florence xtrainer2 pack_phone

StarVLA policy trained on the converted xtrainer2 pack-phone dataset (absolute bimanual end-effector, 10 Hz). Weights are the consolidated pytorch_model.pt from an 8-GPU DeepSpeed ZeRO-3 run.

This repo is the inference bundle, not a Hugging Face transformers checkpoint. Serve it with StarVLA's policy server.

Contents

config.yaml                 # model + data_mix (include_state, image_size restored)
dataset_statistics.json     # q99 stats, key new_embodiment
checkpoints/
  steps_8730.pt             # ~10 epochs
  steps_17460.pt
  steps_26190.pt            # ~30 epochs
  steps_34920.pt
  steps_43650.pt            # latest in this upload

read_mode_config resolves config.yaml and dataset_statistics.json from the parent of checkpoints/. Do not nest the .pt one directory deeper.

Also required, not in this repo:

• Florence-2-large (microsoft/Florence-2-large, or a local path containing the substring florence)
• StarVLA / OPTR tree with the xtrainer2 DataConfig
• pytorch3d (quaternion ↔ rotation-6D)

Action / state

• Absolute EEF, not joints, not delta
• Robot wire: 16-D xyzw [xyz(3)+quat(4)+gripper(1)] left then right
• Model tensor: 20-D after quaternion → rotation-6D
• Instruction: Pick up the phone, place it in the box, and close the lid.

Serve

export FLORENCE_MODEL_PATH=/path/to/Florence-2-large   # optional if using HF id
bash examples/Xtrainer2/train_files/run_xtrainer2_policy_server.sh \
  /path/to/this-repo/checkpoints/steps_43650.pt \
  0 \
  10093

Because these .pt files already sit directly under checkpoints/, the launcher does not need the DeepSpeed staging workaround.