GitHub repository: wangbluo/BandWidth_Test
Created: 2024-01-22T02:20:41Z
Description: Test the GPU bandwidth of  collectives operators like all-reduce, all-gather, broadcast and all-to-all primitives on single-node multi-GPU (2, 4, 8 cards) and multi-node multi-GPU (16 cards) setups, using only PyTorch and Python built-in packages.