Delete unnecessary allreduce communication