emiliiia/real_machine_exper

FastWAM real-machine experiments

Every experiment is stored in its own root-level run subdirectory so files with the same checkpoint name never overlap.

Root experiment directory	Final checkpoint	Dataset subfolder
/pick_banana_20k_0820	checkpoints/weights/step_020000.pt	pick_banana
/close_drawer_track_20k_save5k_20260822_193712	checkpoints/weights/step_020000.pt	close_drawer
/stack_bowls	not available; configs/stats only	stack_bowls

Each directory contains source/resolved configs, dataset stats.json, and metadata.json. DeepSpeed optimizer/training state is intentionally not uploaded.

Managed task and experiment directories

New uploads produced by scripts/upload_real_machine_experiment.py use this layout:

<task>/<experiment-name>/
├── README.md
├── artifact_manifest.json
├── config.yaml
├── dataset/
│   ├── meta/
│   │   ├── stats.json
│   │   └── tasks.jsonl
│   └── text_embedding/
│       ├── manifest.json
│       └── <manifest-referenced 64-hex filename>.pt
└── weights/
    ├── step_010000.pt
    └── step_020000.pt

The first path component classifies the task, for example pick_banana, close_drawer, or stack_bowls. The second component is the exact experiment/run name. Metadata is shared once per experiment; it is not duplicated for every checkpoint step. Older root-level experiment directories in the table above are retained as legacy artifacts.

The managed close-drawer baseline is available at:

close_drawer/real_machine_close_drawer_baseline_bs48/

For this experiment, weights/step_020000.pt is the recommended final checkpoint and weights/step_010000.pt is the backup checkpoint.

Fixed-revision download

For reproducible downloads, pin a commit rather than a moving branch. The following revision is the repository main snapshot verified on 2026-08-23 and contains the close-drawer baseline:

from huggingface_hub import snapshot_download

snapshot_root = snapshot_download(
    repo_id="emiliiia/real_machine_exper",
    revision="75aa8efecf49e01d4e4035b87d190152101f07ba",
    allow_patterns=[
        "close_drawer/real_machine_close_drawer_baseline_bs48/**"
    ],
)

After download, locate the files relative to snapshot_root:

from pathlib import Path

root = Path(snapshot_root)
experiment = root / "close_drawer/real_machine_close_drawer_baseline_bs48"

recommended_weights = experiment / "weights/step_020000.pt"
backup_weights = experiment / "weights/step_010000.pt"
resolved_config = experiment / "config.yaml"
dataset_stats = experiment / "dataset/meta/stats.json"
dataset_tasks = experiment / "dataset/meta/tasks.jsonl"
text_manifest = experiment / "dataset/text_embedding/manifest.json"
artifact_manifest = experiment / "artifact_manifest.json"

Download dataset/text_embedding/manifest.json and every .pt file it references together. The embeddings are task- and prompt-specific and are not interchangeable with arbitrary text encoder output.

Server path setup

config.yaml is the resolved Hydra configuration from the original training run. It can contain machine-specific absolute paths. Before training or evaluation on another server, override at least:

• output_dir;
• data.train.data_path;
• data.train.text_embed_cache_dir;
• local checkpoint or ActionDiT paths used by the model configuration.

Provide the Wan/DiffSynth base-model checkpoint through the destination server's own checkpoint root, for example:

export DIFFSYNTH_MODEL_BASE_PATH=/path/to/fastwam-model-checkpoints

Do not copy an original server path literally unless that path exists and has the same meaning on the destination server.

Artifact-manifest verification

artifact_manifest.json records each managed artifact except the manifest itself, including its repository-relative path, role, byte size, and SHA-256. Verify a downloaded snapshot as follows:

import hashlib
import json
from pathlib import Path

root = Path(snapshot_root)
manifest_path = (
    root
    / "close_drawer/real_machine_close_drawer_baseline_bs48/artifact_manifest.json"
)
manifest = json.loads(manifest_path.read_text())

for record in manifest["files"]:
    path = root / record["path"]
    digest = hashlib.sha256()
    with path.open("rb") as fileobj:
        for block in iter(lambda: fileobj.read(1024 * 1024), b""):
            digest.update(block)
    assert path.stat().st_size == record["size"], path
    assert digest.hexdigest() == record["sha256"], path

Upload script

The repository's scripts/upload_real_machine_experiment.py publishes an exact allowlist. It does not scan the whole run or dataset and never uploads DeepSpeed state, W&B files, evaluation output, videos, or parquet data.

Start with the default dry-run:

python scripts/upload_real_machine_experiment.py \
  --run-dir /path/to/real_machine_close_drawer_baseline_bs48 \
  --steps 10000 20000 \
  --repo-id emiliiia/real_machine_exper \
  --task close_drawer \
  --experiment-name real_machine_close_drawer_baseline_bs48 \
  --revision main

After checking every planned path, byte size, and SHA-256, explicitly authorize the public write:

python scripts/upload_real_machine_experiment.py \
  --run-dir /path/to/real_machine_close_drawer_baseline_bs48 \
  --steps 10000 20000 \
  --repo-id emiliiia/real_machine_exper \
  --task close_drawer \
  --experiment-name real_machine_close_drawer_baseline_bs48 \
  --revision main \
  --commit-message "Upload close drawer baseline checkpoints" \
  --upload \
  --confirm-public-repo emiliiia/real_machine_exper

The uploader requires an explicitly public repository and an exact confirmation value. It captures the branch HEAD and passes it as parent_commit, so concurrent HEAD changes fail instead of overwriting another commit. An empty task/experiment prefix may be created. An already complete and byte-identical prefix is verified as an idempotent no-op. Partial, extra, or conflicting content is rejected.

After a successful upload, use the returned commit oid as the revision in snapshot_download for an immutable result.

Limitations and common failures

• DeepSpeed optimizer/training state is not included. The model weights can be loaded, but they cannot reproduce an exact optimizer-state resume.
• Local source symlinks, unsafe paths, leaked credentials, malformed task/text manifests, orphan embeddings, and mismatched hashes are rejected.
• state, wandb, and eval paths, video files, and parquet files are outside the uploader's allowlist.
• A private or unknown-visibility target, incorrect public confirmation, branch HEAD drift, existing-prefix conflict, insufficient snapshot space, or failed remote integrity verification stops the operation.
• config.yaml must be adapted to the destination server before it is used.