Fix a logic bug that occurs when a broker is down.

If RefreshMetadata completed successfully, a panic would happen because
the code nulled out the broker.