Remove useless barriers.

We don't care about the order here, we only care about getting the
update. The barrier is only useful to force the order.