Model Parallel in Torch
Hello, We want to use this model on multiple GPU environment. We have Nvidia's DGX station (tesla v100 vram 16GB x 4).
Do you know how to do model parallel in torch for PG-InstructBLIP?
Or do you have any ideas?
Thank you for your help.
I'm not super familiar with multi-GPU inference, but it should be possible. Since PG-InstructBLIP is just a checkpoint of InstructBLIP, I would recommend looking for solutions for loading InstructBLIP in LAVIS.
I think this issue in LAVIS may help: https://github.com/salesforce/LAVIS/issues/321#issuecomment-1600881046
There is also this blog post that may help (but note that we can't directly use the InstructBLIP from huggingface): https://huggingface.co/blog/accelerate-large-models
Thank you for your advice.
I will try this and I will report if I make any progress.