Fix data races by locking everything, locking isn't too expensive now