Only try to close IPv6 if actually IPv6