grow3d — runs, state and results
Research project: a new 3D shape generator ("growing flow" over a sparse octree of
TRELLIS.2 SC-VAE latents, one shared network for all levels, block-sparse
octree-multipole attention). Design: code/ snapshot → docs/DESIGN.md.
What lives here
| path | what |
|---|---|
STATUS.md |
live task table, rewritten every 30 min by the node agents |
state.json |
shared task state (claims, leases, steps) — written by agents only |
runs/<task>/ckpt_hub.pt |
latest checkpoint of each training run (model + EMA, no optimizer) |
runs/<task>/ema_<step>.pt |
EMA snapshots used for step-matched evaluations |
results/SUMMARY.md |
one row per evaluated model: Chamfer, F-score, diversity, speed |
results/<eval_task>/ |
per-object metrics JSON + contact sheets (input / generated normals / GT normals) |
logs/ |
logs of failed task attempts |
code/grow3d_src.tar.gz |
source snapshot (no secrets) |
Processed training data is in the companion dataset repo grow3d-data
(data/shard_r<rank>_<n>.tar: SC-VAE latents at 512³/1024³, 12 conditioning views, GT meshes for validation).
How it runs unattended
Every molab sandbox runs one agent.py. Agents claim tasks from configs/plan.json
through state.json (atomic commits), renew a 20-minute lease while they work,
checkpoint every 15 minutes, and upload checkpoints here. Sandboxes live about
6 hours; when one disappears its lease expires and the next agent resumes the
task from runs/<task>/ckpt_hub.pt. Evaluations start by themselves when their
training dependencies reach the required steps, and write to results/.
Resuming on fresh sandboxes
Put the new sandbox URLs/tokens into scripts/nodes.env (with HF_TOKEN) and run
bash scripts/up_all.sh — or just give the links to Claude in the project folder
and ask it to bring the nodes up. Nothing else is needed: data shards and
checkpoints are pulled from the hub automatically.