Only allow swaps when they are valid.