Avoid KeyError: -1 when deleted topics linger

I have hit this error in Kafka 0.8.1 a few times where after deleting a topic it still reports somehow into the client, and I end up with:

    File "env/src/kafka-python/kafka/client.py", line 39, in __init__
        self.load_metadata_for_topics()  # bootstrap with all metadata
    File "env/src/kafka-python/kafka/client.py", line 258, in load_metadata_for_topics
        self.topics_to_brokers[topic_part] = brokers[meta.leader]
    KeyError: -1