creuthecat/Open-Sora-v2.1 — Hugging Face Cloud Training Package

Cloud-first training package for creuthecat/Open-Sora-v2.1.

This package does not fabricate model weights. It starts from the real hpcai-tech/Open-Sora-v2 checkpoint, trains on real video-caption pairs, and persists checkpoints to the read/write Storage Bucket:

creuthecat/opensora-v21

What this trains

Stage A — real video I2V/T2V fine-tuning

Uses the official Open-Sora v2 training code and its stage1_i2v.py recipe. The upstream project documents this recipe for joint T2V/I2V training at 256px and documents fine-tuning from Open_Sora_v2.safetensors on 8 GPUs.

Stage B — reference-image adapter

A separate trainable reference encoder/projector is trained from real video first frames. This is an adapter artifact, not a claim that the upstream 11B transformer magically gained multi-reference support without model changes.

Stage C — audio/SFX extension

The package includes an interface and event-timeline format for a later SFX training stage. It is kept separate because Open-Sora v2 itself is a video model and does not ship a native SFX generator in its checkpoint.

Cloud mounts

The Job mounts:

• hpcai-tech/Open-Sora-v2 → /base-model (read-only)
• google/t5-v1_1-xxl → /ckpts/google/t5-v1_1-xxl (read-only)
• openai/clip-vit-large-patch14 → /ckpts/openai/clip-vit-large-patch14 (read-only)
• your real training data → /data (read-only)
• creuthecat/opensora-v21 → /training-outputs (read/write)

Hugging Face Jobs supports model/dataset/bucket volume mounts and buckets are read-write by default. The Job disk is ephemeral, so checkpoints must be written to the mounted bucket as they are created.

Data format

The real-video CSV should contain at least:

path,text,num_frames,height,width,aspect_ratio,resolution,fps

The path column should point to the video file as visible inside the Job, for example:

/data/videos/000001.mp4

or a relative path resolved by the preprocessing step.

Recommended first run

Start with a curated real-video subset instead of attempting the entire internet-scale corpus on the first run. The OpenVid-1M dataset is a real video-text dataset and is licensed CC BY 4.0, but its Hub release is about 12.4 TB and the data are distributed in large ZIP shards; additionally, its card says downstream users must follow the licenses of the source datasets. For a first run, put only the data you are legally permitted to train on in /data or in a dedicated private dataset/bucket.

Submit on Hugging Face

From the Hugging Face environment/Jobs interface, run the equivalent of:

hf jobs run \
  --flavor a100x8 \
  --timeout 7d \
  -s HF_TOKEN \
  -v hf://hpcai-tech/Open-Sora-v2:/base-model:ro \
  -v hf://google/t5-v1_1-xxl:/ckpts/google/t5-v1_1-xxl:ro \
  -v hf://openai/clip-vit-large-patch14:/ckpts/openai/clip-vit-large-patch14:ro \
  -v hf://datasets/YOUR_ORG/YOUR_DATASET:/data:ro \
  -v hf://buckets/creuthecat/opensora-v21:/training-outputs \
  python:3.12 \
  bash /workspace/job/entrypoint.sh

Replace the dataset volume with your own dataset. If the training package is stored in a Git repository, clone it in the Job first or use the packaged job/entrypoint.sh as the entrypoint.

Outputs

During training, the bucket will contain:

/checkpoints/
/logs/
/reference_encoder/
/release/

The release step places the final/selected model artifact and metadata into separate folders without pretending that all components were newly trained from scratch.

Licensing

The Open-Sora-v2 base checkpoint is listed on Hugging Face as Apache-2.0. Always verify the licenses/terms for every dataset, pretrained encoder, and any audio dataset before publishing derived weights.

Why this is real training

The video stage does not synthesize toy shapes or random weights. It fine-tunes the published 11B Open-Sora v2 checkpoint on a real video-caption dataset. The reference component is separately trained on real video first frames. The T5 and CLIP text/image encoders are pretrained components and are frozen by default; they are not falsely presented as newly trained v2.1 encoders.

The published Open-Sora v2 checkpoint is 23.8 GB and its model repository is about 69.2 GB, so the Job mounts it rather than copying it into the temporary Job filesystem.