Only try to stop the consumer thread once on error

Thread is already stopped by _on_error, fixes InvalidDataError on disconnect
and other errors.