How this model is allocated to 4 A100GPU use, when called, the video memory is displayed in one graphics card
#3
by
jamboom
- opened
How this model is allocated to 4 A100GPU use, when called, the video memory is displayed in one graphics card
If you use FlagModel, it will use all available GPUs automatically. For transformers, you need to add this command model = torch.nn.DataParallel(model).
For sentence transformers, you can refer to https://github.com/UKPLab/sentence-transformers .