RainyBot/SemWAM-RealRobot-PackPhone-v1

SemWAM Real Robot — Pack Phone v1

A task-specific SemWAM checkpoint for Pack Phone, released by RainyBot. SemWAM jointly predicts robot actions and future RGB, depth and semantic representations from current camera observations, a language instruction and robot state.

This is the final checkpoint of the completed training run, at optimizer step 6240. All eight original node launchers exited successfully, and the saved optimizer step equals the archived training budget. Release verification was performed on 2026-11-10 (UTC).

Intended use and evidence

The release supports research on multimodal world–action modeling and integration with a compatible real-robot interface. Checkpoint structure, tensor finiteness, saved inference identity, numerical normalization statistics and save-completion inventory were verified. Training completion does not establish a real-robot success rate: this release makes no claim of closed-loop performance or state-of-the-art results.

Files

File	Purpose
checkpoints/weights/step_006240.pt	Final SemWAM v2 checkpoint; includes the saved model contract and frozen semantic compressor
inference_config.json	Machine-independent model and processor contract
config.yaml	Public inference configuration; restore operational asset paths on your machine
dataset_stats.json	Numerical action/state normalization statistics
checkpoints/weights/dataset_stats.json	Identical statistics companion for automatic checkpoint discovery
checkpoint_manifest.json	Checkpoint SHA256, byte size, step, public identity and required asset checksums
model_card_summary.json	Readable architecture and interface summary
trainer_state.reference.json	Minimal public step/contract reference; no optimizer or RNG state
SHA256SUMS	SHA256 checksums for release files

Weights retain the canonical step_XXXXXX.pt filename expected by SemWAM tooling. Task identity and release version are expressed by the repository name. Intermediate checkpoints, optimizer partitions, RNG state and training data are excluded.

Download

from pathlib import Path
from huggingface_hub import snapshot_download

# For reproducible experiments, set revision to a verified 40-character commit
# from this model repository's commit history.
release_dir = Path(snapshot_download(
    repo_id="RainyBot/SemWAM-RealRobot-PackPhone-v1",
    revision="main",
))
checkpoint = release_dir / "checkpoints/weights/step_006240.pt"
statistics = release_dir / "dataset_stats.json"

cd /path/to/downloaded/release
sha256sum --check SHA256SUMS

Source and loading

Use the SemWAM code at https://github.com/RainyRobo/SemWAM. Packaging checkout: 19d1247e24e1990a7aac99fbd5cddd674771df9a. The packaging checkout contained local changes; this commit alone is not a claim that it exactly reproduces the training source. Consult the source repository's setup and evaluation documentation.

From a configured SemWAM checkout, inspect the public v2 identity before constructing the model:

import torch
from omegaconf import OmegaConf
from wam.eval.checkpoint_schema import validate_checkpoint
from wam.eval.checkpoint_config import apply_checkpoint_to_cfg

payload = torch.load(checkpoint, map_location="cpu", mmap=True, weights_only=True)
validate_checkpoint(payload)
cfg = OmegaConf.load(release_dir / "config.yaml")
apply_checkpoint_to_cfg(
    cfg, str(checkpoint), payload=payload,
    restore_inference=True, prepare_native=False,
)
# Bind local paths for the exact frozen assets, then construct the model
# through the SemWAM runtime and load the validated checkpoint modules.

Check frozen dependencies in the configured source environment:

PYTHONPATH=src python -m wam_setup check --role eval \
  --checkpoint /path/to/downloaded/release/checkpoint_manifest.json

The model is a SemWAM v2 checkpoint, not a Transformers AutoModel package. This release does not provide a turnkey robot driver. The RoboTwin joint-action adapter is not interchangeable with this bimanual end-effector pose interface.

Observation, action and normalization interface

• Three RGB views: primary camera, left wrist and right wrist.
• Raw robot state and returned actions each have 16 values: [left xyz, left quaternion xyzw, left gripper, right xyz, right quaternion xyzw, right gripper].
• The model uses 20-dimensional position/rotation-6D/gripper representations. State is absolute; actions are relative SE(3) transforms from the current state, with absolute gripper values.
• The saved action horizon is 32. Postprocessing must preserve the full horizon, undo the packaged affine normalization, then restore absolute poses with the saved processor.
• Fresh normalization is unclipped z-score. Reusing another task's statistics, applying ±5 clipping, swapping quaternion conventions or adding an action-index shift changes the released interface.
• Depth conditioning is estimated from current RGB only. Future teacher targets are not inputs at inference. Preserve the saved depth preprocessing and camera geometry in inference_config.json.

Saved architecture

{
  "model.arch": "rgb_dep_sem",
  "model.attention_layout": "rgb_auxiliary_causal",
  "model.action_conditioning": "predicted_depth_head_semantic",
  "model.world_action_visibility": "none",
  "model.input_paradigm": "t_concat",
  "model.view_layout": "robotwin_t",
  "model.depth.repr": "dvae",
  "model.depth.condition_source": "estimated",
  "model.depth.target_source": "dataset",
  "model.depth.condition_temporal_context": "current_frame",
  "model.depth_projection": "independent",
  "model.semantic_projection": "independent",
  "model.semantic.parametrization": "v_pred",
  "model.semantic.compression": "svae",
  "model.semantic.channels": 48,
  "model.semantic.compression.output_normalization": "layernorm"
}

The complete saved contract is authoritative. Restore it before model construction; do not reinterpret this checkpoint with the current default recipe. The semantic compressor is embedded. Other frozen encoders/codecs are external dependencies with the exact expected hashes recorded in the manifest and inference configuration.

Integrity

• Weight SHA256: 742b98085f8b53478c712c30b7c9c5f1a3f31ddf4826654d37294df75b74cf83
• Weight bytes: 14649561065
• Statistics file SHA256: daf7806fb9b6395ce9277b8aca98590af9101f664274781898440dfc77691c7a
• Numerical statistics fingerprint: be6c7c9fc8a34e46a942402259c177efa47040bfc477711e42a266b7ec8ccd7e
• Checkpoint version: 2
• Complete optimizer/RNG state included: no

License and attribution

SemWAM-authored material is released under the MIT license; see LICENSE. Third-party pretrained components retain their respective licenses and attribution requirements; see THIRD_PARTY_NOTICES.md. External frozen assets are not redistributed in this package.

No paper identifier or BibTeX citation is supplied because a verified publication identifier has not been recorded for this release. Cite the model repository and the exact revision used in your experiments.