Separating the loading/setting of data from/to device from syncing of
data via MPI. This will be used later to overlap the communication and
computations.