Fix potential double free that could happen after connection error.

MFC after:	3 days