Don't try to reconnect when the connection is fresh

This should fix #382.