implementation is not offloading-safe
if using any more complex offloading (as typically required for minimax due to its size), the custom transformer code does not move tensors when it needs them:
[ File "/home/vlado/.cache/huggingface/modules/diffusers_modules/local/OpenVDN--vdn-minimax-h3/20c35774acc13322df52ba817f6e0a0a63e72a15/modeling_vdn_h3.py", line 803, in _pre_ref
normed = F.rms_norm(hidden, (hidden.shape[-1],), weight, eps)
](TorchRuntimeError: Tensor device mismatch
Explanation: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
from user code:
File "/home/vlado/.cache/huggingface/modules/diffusers_modules/local/OpenVDN--vdn-minimax-h3/20c35774acc13322df52ba817f6e0a0a63e72a15/modeling_vdn_h3.py", line 803, in _pre_ref
normed = F.rms_norm(hidden, (hidden.shape[-1],), weight, eps)
File "/home/vlado/dev/sdnext/venv/lib/python3.13/site-packages/torch/nn/functional.py", line 3012, in rms_norm
return torch.rms_norm(input, normalized_shape, weight, eps)
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^)
Can you let us know what the configuration is? (Memory Size, or offloading policy?)
The transformer works with model offload or block-level group offload, but not leaf-level or sequential. Pull the latest repo and re-run bash scripts/setup_diffusers.sh, which now adds a diffusers patch for streamed group offload. Then python src/inference/infer_diffusers.py --offload_dit renders the full 345 frames in about 20 GB of VRAM
will the patch be included in your pr to diffusers? i cannot use custom patches.
for the record, i've used leaf-level group offloading and loaded the model from sdnext - the goal is to integrate it into sdnext.
gpu was rtx5090.