Spaces:
Running
How to solve device mismatch issue when using offloading with QwenImageEditPlus pipeline and GGUF weights
First of all buddy, the T4 GPU doesn’t support torch_dtype=torch.bfloat16 (bf16) since it’s an older GPU. The other issue you’re seeing is mainly due to device mismatches. You need to load the model on 2 GPUs in distributed mode, for which you can use device="balanced" and avoid falling back to CPU when not needed. Diffusers and Transformers will handle this automatically in most cases. Usually, this isn’t set up properly for image models, and that’s why you get a tensor mismatch error when the model runs partly on GPU and partly on CPU. Ans it seems a vibe coded snippet
@Parveshiiii No, it's not vibe coded. I wrote this partly from the examples given in the model pages, and partly after studying the docs, like the scheduler part is taken from this page https://huggingface.co/spaces/akhaliq/Qwen-Image-Edit-2509
I have tried using device_map="balanced", but I get
{'text_encoder': 0, 'vae': 'cpu'}
Hi, this is hardly the best communication channel for your issue and questions, can you please open an issue in the diffusers github repo instead?