afauzanaqil/my-cool-model

wan-fewstep-cache-research

Research scaffold for one question: existing feature-caching acceleration methods for video diffusion (TeaCache, FasterCache, and ~10 similar "X-Cache" papers) target multi-step models. Once a model is distilled to 4-8 steps (e.g. via CausVid), the cross-timestep redundancy those methods exploit is largely removed by the distillation itself. Does redundancy still exist somewhere else -- cross-layer, cross-guidance-branch, or spatial -- that a new caching method could exploit instead?

Target hardware: a single RTX 3090 (24GB). That constraint is why this project is scoped to Wan2.1-T2V-1.3B, not the 14B variant or HunyuanVideo -- see NOTES_ON_SCOPE.md for why.

Research phases (do them in this order)

#	Script	Question it answers
0	scripts/01_validate_causvid_lora.py	Does the CausVid LoRA actually produce acceptable video? (community checkpoint -- verify before building on it)
1	scripts/02_reproduce_baselines.py	Do TeaCache/FasterCache work on the base model and break (or degrade) on the CausVid-distilled model, as the literature implies but doesn't show directly for this checkpoint?
2	scripts/03_profile_redundancy.py	The core experiment. Where, if anywhere, does redundancy survive distillation?
3	scripts/04_run_proposed_method.py	Implement a caching method around whatever axis phase 2 finds -- template only, deliberately not pre-written
4	scripts/05_evaluate.py	Full evaluation: speedup, VRAM, CLIP score, LPIPS, (FVD for final numbers)

Phases 3-4 are intentionally left as templates. Writing them before phase 2 has results would mean guessing the mechanism instead of deriving it from evidence -- if phase 2 comes back negative (no redundancy survives distillation anywhere), that is itself a publishable negative result, but it changes what phase 3 should even attempt.

Quick start

bash setup.sh                                  # env + model downloads
python scripts/01_validate_causvid_lora.py     # sanity check the LoRA first
python scripts/03_profile_redundancy.py        # core experiment

Directory layout

wan-fewstep-cache-research/
├── README.md                  <- you are here
├── NOTES_ON_SCOPE.md           <- why 1.3B, why this checkpoint, hardware reasoning
├── environment.yml
├── requirements.txt
├── setup.sh                    <- creates env, installs deps, downloads checkpoints
├── configs/
│   └── model_config.yaml       <- model paths, generation params, output paths
├── data/
│   ├── README.md               <- where to get the full VBench prompt set
│   └── prompts/
│       └── pilot_prompts.txt   <- 20 original prompts for fast iteration
├── src/
│   ├── models/
│   │   └── load_wan.py         <- load base + CausVid-LoRA pipelines
│   ├── instrumentation/
│   │   ├── hooks.py            <- forward-hook feature recorder (template -- verify module names)
│   │   └── feature_store.py    <- cross-timestep / cross-layer similarity analysis
│   ├── caching/
│   │   ├── README.md           <- how to add TeaCache/FasterCache as submodules
│   │   └── proposed_cache.py   <- stub, fill in after phase 2 findings
│   ├── profiling/
│   │   └── vram_timing.py      <- wall-clock + peak VRAM measurement context manager
│   ├── eval/
│   │   ├── clip_score.py       <- text-video alignment
│   │   ├── lpips_score.py      <- perceptual fidelity vs. reference
│   │   └── fvd_score.py        <- stub, wire in VBench's FVD for final numbers only
│   └── utils/
├── scripts/                    <- orchestration entry points, see table above
├── checkpoints/                <- downloaded models (gitignored)
├── outputs/                    <- generated videos, CSVs, logs (gitignored)
└── tests/
    └── test_hooks.py

Known unknowns (fill in as you go, don't take these as settled)

• Exact Wan transformer submodule names for the attention/MLP hooks in src/instrumentation/hooks.py need verifying against your installed diffusers version (print(pipe.transformer)). The file is written as an adjustable template, not a guaranteed-correct mapping.
• CausVid 1.3B LoRA quality is unverified by us -- it's a community conversion, not an official release. Phase 0 exists specifically to check this before you invest time in the rest of the pipeline.
• FVD needs a pretrained I3D-style feature extractor. Reuse VBench's implementation rather than writing your own; only wire it in for final paper numbers, not every iteration.