grabbou/Wan2.1-T2V-1.3B-CoreAI

Wan 2.1 T2V 1.3B for Core AI

Wan-AI/Wan2.1-T2V-1.3B-Diffusers exported to Apple Core AI with the apple/coreai-models recipe, 4-bit for iOS:

uv run coreai.diffusion.export Wan-AI/Wan2.1-T2V-1.3B-Diffusers --platform iOS --compression 4bit-asym

The VAE decoder is then exported again for 32x32 latent tiles, since decoding the whole video at once needs more GPU memory than an iPhone app gets. The recipe's exporter supports tiles (DiffusionExportConfig(vae_tile_size=32)) but its command line does not, so export_tiled_vae.py calls it directly. metadata.json records vae_tile_size: 32.

Folder	Contents	Size	Platforms
ios/	CoreAIVideoDiffusion bundle: Transformer.aimodel (786 MB), TextEncoder.aimodel (UMT5, 3.1 GB), VAEDecoder.aimodel (141 MB, 32x32 latent tiles), tokenizer, metadata.json	4.1 GB	iOS, macOS

Text-to-video at 832x480 and 81 frames (5 s at 16 fps). The iOS export is traced for exactly that size and length: the transformer takes a 60x104 latent and the decoder 21 latent frames, decoded in overlapping tiles. For other sizes, export again without --platform iOS (dynamic shapes, for macOS).

On a Mac it generates 81 frames in about 5 minutes with 2 steps, peaking at 6.9 GB. That is more memory than an iPhone app gets by default, so on an iPhone 17 Pro the app is killed during generation; making it fit is tracked in callstackincubator/ai#306.

Use

With @react-native-ai/core-ai, download ios/ into app storage, then:

import { coreAI } from '@react-native-ai/core-ai'
import { experimental_generateVideo as generateVideo } from 'ai'

const { video } = await generateVideo({
  model: coreAI.videoModel({ id: 'wan2.1', source: { type: 'file', uri: pathToFolder } }),
  prompt: 'a red ball rolling across a wooden table',
})

In Swift, load the folder with WanPipeline(from:) from the CoreAIVideoDiffusion product of coreai-models.

License

Wan 2.1 is Apache-2.0. This repository only converts it to Core AI format.