Repository: pymedusa/Medusa

Issue:
[APP SUBMITTED]: UnicodeDecodeError: 'utf8' codec can't decode byte 0xc0 in position 1738: invalid start byte
### INFO
**Python Version**: `2.7.13 (default, Dec 19 2016, 01:28:10) [GCC 4.6.3]`
**Operating System**: `Linux-3.2.40-i686-with-glibc2.0`
**Locale**: `UTF-8`
**Branch**: [master](../tree/master)
**Database**: `44.8`
**Commit**: pymedusa/Medusa@3f23fd272652b10cf781e3058fee50daf79f11b6
**Link to Log**: https://gist.github.com/9e0732da7418a555d443793180809507
### ERROR
<pre>
2017-09-10 12:18:10 ERROR    SNATCHQUEUE-MANUALSNATCH-73141 :: [3f23fd2] DownloadStation: Unknown exception raised when send torrent to DownloadStation : 'utf8' codec can't decode byte 0xc0 in position 1738: invalid start byte
Traceback (most recent call last):
  File "/volume1/@appstore/sickbeard-custom/var/<a href="../blob/3f23fd272652b10cf781e3058fee50daf79f11b6/SickBeard/medusa/clients/torrent/generic.py#L79">SickBeard/medusa/clients/torrent/generic.py</a>", line 79, in _request
    cookies=cookies, timeout=120, verify=False)
  File "/volume1/@appstore/sickbeard-custom/var/<a href="../blob/3f23fd272652b10cf781e3058fee50daf79f11b6/SickBeard/lib/requests/sessions.py#L518">SickBeard/lib/requests/sessions.py</a>", line 518, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/volume1/@appstore/sickbeard-custom/var/<a href="../blob/3f23fd272652b10cf781e3058fee50daf79f11b6/SickBeard/lib/requests/sessions.py#L475">SickBeard/lib/requests/sessions.py</a>", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/volume1/@appstore/sickbeard-custom/var/<a href="../blob/3f23fd272652b10cf781e3058fee50daf79f11b6/SickBeard/lib/requests/sessions.py#L591">SickBeard/lib/requests/sessions.py</a>", line 591, in send
    r = dispatch_hook('response', hooks, r, **kwargs)
  File "/volume1/@appstore/sickbeard-custom/var/<a href="../blob/3f23fd272652b10cf781e3058fee50daf79f11b6/SickBeard/lib/requests/hooks.py#L31">SickBeard/lib/requests/hooks.py</a>", line 31, in dispatch_hook
    _hook_data = hook(hook_data, **kwargs)
  File "/volume1/@appstore/sickbeard-custom/var/<a href="../blob/3f23fd272652b10cf781e3058fee50daf79f11b6/SickBeard/medusa/session/hooks.py#L28">SickBeard/medusa/session/hooks.py</a>", line 28, in log_url
    log.debug('With post data: {data}'.format(data=resp.request.body))
  File "/usr/local/sickbeard-custom/env/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xc0 in position 1738: invalid start byte
</pre>
---
_STAFF NOTIFIED_: @pymedusa/support @pymedusa/moderators


Discussion:
This is similar to #3004 however in this case the byte string is not being double mis-converted, its just being decoded with the wrong codec.  It is most likely `ISO 8859-1` which is the codec for West European languages (aka `Latin-1`).  We don't see this issue frequently so I propose we delay this until after #3089 is merged.

The difference between this and #3089 is that for me this is a full stopping error, the torrent is not sent to DownloadStation at all.
Would you mind switching to develop to see if the issue persists or worsens?
So, after finally getting the time to try and switch to develop and also managing to do that, the issue still persists unfortunately. Same error message as in original post.
Oops, closed the issue by mistake (close and comment did not mean close this form and go back to the list 😄  )
@andersjohansson3 any chance you have Latin-1 characters (e.g. Ā, Ē, etc) in your username or password?

Resolving patch:
diff --git a/medusa/providers/generic_provider.py b/medusa/providers/generic_provider.py
--- a/medusa/providers/generic_provider.py
+++ b/medusa/providers/generic_provider.py
@@ -451,11 +451,25 @@ def get_result(self, episodes=None):
     @staticmethod
     def get_url_hook(response, **kwargs):
         """Get URL hook."""
-        log.debug('{0} URL: {1} [Status: {2}]', response.request.method, response.request.url, response.status_code)
-
-        if response.request.method == 'POST':
-            data = response.request.body.decode('utf-8')
-            log.debug('With post data: {0}', data)
+        request = response.request
+        log.debug('{0} URL: {1} [Status: {2}]', request.method, request.url, response.status_code)
+
+        if request.method == 'POST':
+            body = request.body
+            # try to log post data using various codecs to decode
+            codecs = ('utf-8', 'latin1', 'cp1252')
+            for codec in codecs:
+                try:
+                    data = body.decode(codec)
+                except UnicodeError as error:
+                    log.debug('Failed to decode post data as {codec}: {msg}',
+                              codec=codec, msg=error)
+                else:
+                    log.debug('With post data: {0}', data)
+                    break
+            else:
+                log.warning('Failed to decode post data with {codecs}',
+                            codecs=codecs)
 
     def get_url(self, url, post_data=None, params=None, timeout=30, **kwargs):
         """Load the given URL."""


Test patch: