Incrementally backoff when reconnecting to peers

This ensures we backoff when reconnecting to peers for which we don't
understand the replies, just like we do for peers we fail to connect to.

Closes #103