Repository: btubbs/sseclient

Issue:
decoder setting
Hi,
I use sseclient to send a request to https://api.trafikinfo.trafikverket.se/v2/data.json, and the got the following error

> decoder = lookup(encoding).incrementaldecoder

> TypeError: lookup() argument must be str, not None

But when I am looking into the test scripts or examples from [programcreek.com](https://www.programcreek.com/python/example/93555/sseclient.SSEClient), I don't see any example were setting encoder or decoder in the request. 

Can someone help? Thanks a lot.

Discussion:
@btubbs: Hi, sorry to hear you're having a problem.  I'm not following your description of it though.  Can you provide a code snippet of what you're trying to do?  Are you sure that endpoint you linked provides an SSE stream?@patrolez: @btubbs Hello. I can confirm that it happens also for me. 

## Error

```shell
  File "/builds/Sse.py", line XYZ, in ASDASDASDASD
    for event in self._client:
  File "/usr/lib/python3.8/site-packages/sseclient.py", line 100, in __next__
    self._connect()
  File "/usr/lib/python3.8/site-packages/sseclient.py", line 58, in _connect
    self.decoder = codecs.getincrementaldecoder(
  File "/usr/lib/python3.8/codecs.py", line 995, in getincrementaldecoder
    decoder = lookup(encoding).incrementaldecoder
TypeError: lookup() argument must be str, not None
```

Where:
```python
        self._client = SSEClient(url, retry=0, verify=False)
        self._client.resp.raise_for_status()
```

Seems like it is problem with <https://github.com/btubbs/sseclient/blob/bb36b9e273e51fbbf15212cf3d17e464cc6e04c9/sseclient.py#L59>

Where:
`self.resp.encoding` seems to be `None`
https://2.python-requests.org/en/master/api/#requests.Response.encoding

## The server's endpoint

Java Spring SseEmiter is being used as a server. In this case it works via Nginx:
```
HTTP/1.1 200 
Server: nginx/1.18.0
Date: Thu, 14 May 2020 01:05:36 GMT
Content-Type: text/event-stream;charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
```
That is a Header response for the SSE end-point.

## What I have noticed

There is a lot of dependencies, but regarding changes where SSEClient was working I can conclude that:
* sseclient-0.0.24 was passing the tests
* sseclient-0.0.26 fails while showing the error above

### Exploring the repo

Commit <https://github.com/btubbs/sseclient/commit/2aff29003e6372b6ec19d1aac75e59d396d67d74> has added this line without ensuring that passed parameter is not None

## Guesses

Most likely `requests` library is not preparing its Response object properly despite `charset` present in HTTP server Response, so it is being left with `None` value in its `.encoding` property.@andreafresa: Hey did you solve it? I am having the same issue here using an http client thatr needs to comunicate with a Leshan LWM2M Server@patrolez: @andreafresa in my case `sseclient-0.0.24` was enough to make everything work properly, so I have pinned this specific version to my project.@cyprienc: This issue was resolved with PR #51 @TheSandDoctor: In that case, closing.

Solution patch:
diff --git a/setup.py b/setup.py
index 217f8e2..eb5d8d4 100644
--- a/setup.py
+++ b/setup.py
@@ -13,7 +13,7 @@ pytest_runner = ['pytest_runner>=2.1'] if needs_pytest else []
 
 setup(
     name='sseclient',
-    version='0.0.26',
+    version='0.0.27',
     author='Brent Tubbs',
     author_email='brent.tubbs@gmail.com',
     py_modules=['sseclient'],
diff --git a/sseclient.py b/sseclient.py
index d2193e2..1cfcd17 100644
--- a/sseclient.py
+++ b/sseclient.py
@@ -14,7 +14,7 @@ import six
 
 import requests
 
-__version__ = '0.0.26'
+__version__ = '0.0.27'
 
 # Technically, we should support streams that mix line endings.  This regex,
 # however, assumes that a system will provide consistent line endings.
@@ -55,8 +55,8 @@ class SSEClient(object):
         requester = self.session or requests
         self.resp = requester.get(self.url, stream=True, **self.requests_kwargs)
         self.resp_iterator = self.iter_content()
-        self.decoder = codecs.getincrementaldecoder(
-            self.resp.encoding)(errors='replace')
+        encoding = self.resp.encoding or self.resp.apparent_encoding
+        self.decoder = codecs.getincrementaldecoder(encoding)(errors='replace')
 
         # TODO: Ensure we're handling redirects.  Might also stick the 'origin'
         # attribute on Events like the Javascript spec requires.


Test patch:
diff --git a/test_sseclient.py b/test_sseclient.py
index 73cf7a3..6609606 100644
--- a/test_sseclient.py
+++ b/test_sseclient.py
@@ -73,9 +73,10 @@ def test_eols():
 
 
 class FakeResponse(object):
-    def __init__(self, status_code, content, headers=None):
+    def __init__(self, status_code, content, headers=None, encoding="utf-8"):
         self.status_code = status_code
-        self.encoding = "utf-8"
+        self.encoding = encoding
+        self.apparent_encoding = "utf-8"
         if not isinstance(content, six.text_type):
             content = content.decode("utf-8")
         self.stream = content
@@ -95,9 +96,10 @@ def join_events(*events):
 
 
 # Tests of parsing a multi event stream
-def test_last_id_remembered(monkeypatch):
+@pytest.mark.parametrize("encoding", ["utf-8", None])
+def test_last_id_remembered(monkeypatch, encoding):
     content = 'data: message 1\nid: abcdef\n\ndata: message 2\n\n'
-    fake_get = mock.Mock(return_value=FakeResponse(200, content))
+    fake_get = mock.Mock(return_value=FakeResponse(200, content, encoding=encoding))
     monkeypatch.setattr(requests, 'get', fake_get)
 
     c = sseclient.SSEClient('http://blah.com')