Since data parallelism can be implemented with great ease, it is the most widely used implementation strategy on multi-GPUs.