comind-single-view-ablation (12k iter)

Single-ego ablation checkpoint for the multiview Cosmos-Predict2.5 2B model, fine-tuned on the dhyun22/ego_gen_comind_single real egocentric dataset (CoMind, 2-person kitchen recordings, shared-world).

What this ablation is

The base Cosmos-Predict2.5 multiview model generates two ego views jointly — leader and helper attend to each other's tokens (self-attention) and captions (cross-attention) in one forward pass, conditioned on reference frames, cross-view Plücker rays, and shared warp/pose context.

This checkpoint instead generates each ego view completely independently (V=1):

• No reference frames — appearance/pose/camera conditioning from in-context reference frames removed.
• Own-view-only warp — the warp-based conditioning is re-sourced from the wearer's own past frames only (no cross-view pooling).
• Self-only pose — only the wearer's own skeleton is rendered into the pose condition (partner removed).
• Per-view canonical camera — Plücker rays are canonicalized to each view's own frame-0 (w2c[0] = I), so the model's existing "shared canonicalization" logic becomes a no-op per view.
• V=1 pooling — leader and helper clips are POOLED as independent training samples (dataset is 2x the paired-clip count) rather than concatenated into one V=2 forward pass. Self-attention structurally cannot see the other view's tokens (they are never in the same sequence), and cross-attention only ever sees that one view's own caption. This achieves full separation between the two generation pipelines while still sharing weights (data-efficient — see reasoning in the repo's commit history / conversation log).

Training

• Base checkpoint: nvidia/Cosmos-Predict2.5-2B (auto/multiview/524af350-...-_ema_bf16.pt), strict_resume=False (pose/warp/visibility conditioning modules are new -> zero-init, everything else warm-started).
• Data: dhyun22/ego_gen_comind_single, 12,090 clips / 32 shared-world recordings, single_view=True pooling (~21.5k independent V=1 samples after pooling).
• Text encoder: nvidia/Cosmos-Reason1-7B (Qwen2.5-VL-7B architecture, NVIDIA reasoning fine-tune).
• 2x NVIDIA H200, FSDP shard size 2, 12,000 iterations total (resumed once at iter 1200 from an earlier partial run; optimizer/scheduler state carried through via the trainer's own checkpoint-resume, not a cold restart).
• Loss at iter 12,000: ~0.10 (rectified-flow loss, noisy per-iteration).
• Checkpoints saved every 500 iterations during training; this upload is the final iter_000012000 checkpoint.

Files

• model_ema_bf16.pt — EMA weights in bfloat16 (recommended for inference; converted from the training-time DCP/FSDP checkpoint via scripts/convert_distcp_to_pt.py in the cosmos-predict2.5 repo).
• code_changes.tar.gz — the modified/added source files needed to reproduce this run and load this checkpoint (see below). Extract into the cosmos_predict2/_src/ of a clone of nvidia-cosmos/cosmos-predict2.5.

Code changes (in code_changes.tar.gz)

All paths relative to cosmos_predict2/_src/:

File	What changed
predict2_multiview/datasets/comind_pairs.py	Added ComindSingleEgoDataset (own-past-only .single.npz reader) and its single_view mode (pools leader/helper as independent V=1 samples instead of pairing them).
predict2_multiview/datasets/vroid_pairs.py	New file. VroidSingleEgoDataset for the synthetic VRoid single-ego ablation (dhyun22/ego_gen_vroid_single) — same ablation contract, different (already-per-view) file layout; mixes the dataset's core/detail captions stochastically (70% detail / 30% generic).
predict2_multiview/configs/vid2vid/experiment/nymeria_pose_2actor.py	Added experiments comind_single_ego, comind_single_view (this checkpoint's experiment), vroid_single_view. Fixed: job.group was left as "" by the inherited base config, which fails Config.validate(); the text encoder's ckpt_path pointed at a private local path (/data/cosmos_reason1_7b) with no public equivalent wired in, now resolved via nvidia/Cosmos-Reason1-7B on the Hub; trainer.validation_iter was left at its effectively-disabled default, now set to compute real validation loss every 2000 iterations.
predict2_multiview/callbacks/nymeria_validation_viz.py	Added comind_single_ego and vroid_single_ego branches so the periodic validation-preview-video callback knows how to build these datasets (previously only knew the original Nymeria/CoMind paired dataset shapes).
predict2_multiview/configs/vid2vid/config.py	Registers the new vroid_pairs.py dataloaders.

Usage

git clone https://github.com/nvidia-cosmos/cosmos-predict2.5.git
cd cosmos-predict2.5
# extract code_changes.tar.gz on top of cosmos_predict2/_src/
tar -xzf code_changes.tar.gz --strip-components=2 -C cosmos_predict2/_src

torchrun --nproc_per_node=1 -m scripts.train \
  --config=cosmos_predict2/_src/predict2_multiview/configs/vid2vid/config.py \
  -- experiment=comind_single_view \
  checkpoint.load_path=/path/to/model_ema_bf16.pt \
  trainer.max_iter=0  # inference / sampling only, no further training

See the experiment docstring (comind_single_view in nymeria_pose_2actor.py) for the full conditioning contract and the dataset READMEs (dhyun22/ego_gen_comind_single, dhyun22/ego_gen_vroid_single) for the .single.npz field layout.