Repository: Flexget/Flexget

Issue:
rtorrent plugin crashed when adding torrent with special utf-8 characters
### Expected behaviour:
Adding entry to rtorrent without crash
### Actual behaviour:
Adding entry to rtorrent but flexget crashed. 
As a consequence, all the accepted entry with this run won't write to the database marked seen, and they will be downloaded again and again. However all the accepted torrent will be sent to the rtorrent, but the crash reports are bothersome.
### Steps to reproduce:
When an entry with special utf-8 characters in name or path accepted and sent to rtorrent plugin.
Such as the example from the log, "Al final del túnel AKA At the End of the Tunnel 2016 1080p BluRay TrueHD 5.1 x264-PbK", "ú" is a special utf-8 characters.
The details of this torrent
```
 'info': {'files': [{'length': 14152687941,
                     'path': ['Al.final.del.t\xc3\xbanel.AKA.At.the.End.of.the.Tunnel.2016.1080p.BluRay.TrueHD.5.1.x264-PbK.mkv']},
                    {'length': 1281,
                     'path': ['Al.final.del.t\xc3\xbanel.AKA.At.the.End.of.the.Tunnel.2016.1080p.BluRay.TrueHD.5.1.x264-PbK.nfo']}],
          'name': 'Al.final.del.t\xc3\xbanel.AKA.At.the.End.of.the.Tunnel.2016.1080p.BluRay.TrueHD.5.1.x264-PbK',
          'piece length': 4194304,
          'pieces': '<3375 piece hashes>',
          'private': 1,
          'source': '[www.hdpter.net] HDPter'},
```
#### Config:
```
  hdp:
    rss: **********
    accept_all: yes
    rtorrent:
      uri: scgi://127.0.0.1:5000
      path: /mnt/V3T/_Movies/_HD
      custom1: HD

```
  
#### Log:
```
2017-02-17 10:37 VERBOSE  task          hdp             REJECTED: `Hacksaw Ridge 2016 1080p BluRay TrueHD Atmos 7.1 x264-PbK` by seen plugin because entry with url `http://www.hdpter.net/download.php?id=10438&passkey=ab0ee1ba02ec1f4cd892219cab827967` is already marked seen in the task hdp at 2017-02-12 22:03
2017-02-17 10:37 VERBOSE  task          hdp             ACCEPTED: `Al final del túnel AKA At the End of the Tunnel 2016 1080p BluRay TrueHD 5.1 x264-PbK` by accept_all plugin
2017-02-17 10:37 INFO     download      hdp             Downloading: Al final del túnel AKA At the End of the Tunnel 2016 1080p BluRay TrueHD 5.1 x264-PbK
2017-02-17 10:37 VERBOSE  details       hdp             Summary - Accepted: 1 (Rejected: 9 Undecided: 0 Failed: 0)
2017-02-17 10:37 CRITICAL task          hdp             BUG: Unhandled error in plugin rtorrent: 'ascii' codec can't decode byte 0xc3 in position 333: ordinal not in range(128)
Traceback (most recent call last):
  File "/home/xiayu/CODE/PYTHON/flexget/lib/python2.7/site-packages/flexget/task.py", line 483, in __run_plugin
    return method(*args, **kwargs)
  File "/home/xiayu/CODE/PYTHON/flexget/lib/python2.7/site-packages/flexget/event.py", line 23, in __call__
    return self.func(*args, **kwargs)
  File "/home/xiayu/CODE/PYTHON/flexget/lib/python2.7/site-packages/flexget/plugins/clients/rtorrent.py", line 531, in on_task_output
    self.add_entry(client, entry, options, start=config['start'], mkdir=config['mkdir'])
  File "/home/xiayu/CODE/PYTHON/flexget/lib/python2.7/site-packages/flexget/plugins/clients/rtorrent.py", line 639, in add_entry
    if client.torrent(entry['torrent_info_hash']):
  File "/home/xiayu/CODE/PYTHON/flexget/lib/python2.7/site-packages/flexget/plugins/clients/rtorrent.py", line 356, in torrent
    resp = multi_call()
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1006, in __call__
    return MultiCallIterator(self.__server.system.multicall(marshalled_list))
  File "/home/xiayu/CODE/PYTHON/flexget/lib/python2.7/site-packages/flexget/plugins/clients/rtorrent.py", line 38, in __call__
    return self.__send(self.__name, args)
  File "/home/xiayu/CODE/PYTHON/flexget/lib/python2.7/site-packages/flexget/plugins/clients/rtorrent.py", line 192, in __request
    verbose=self.__verbose
  File "/home/xiayu/CODE/PYTHON/flexget/lib/python2.7/site-packages/flexget/plugins/clients/rtorrent.py", line 106, in request
    return self.single_request(host, handler, request_body, verbose)
  File "/home/xiayu/CODE/PYTHON/flexget/lib/python2.7/site-packages/flexget/plugins/clients/rtorrent.py", line 134, in single_request
    return self.parse_response(sock.makefile())
  File "/home/xiayu/CODE/PYTHON/flexget/lib/python2.7/site-packages/flexget/plugins/clients/rtorrent.py", line 148, in parse_response
    response_body += data
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 333: ordinal not in range(128)
```

### Additional information:

- Flexget Version: 2.9.23
- Python Version: 2.7.13
- Installation method: virtualenv pip install 
- OS and version: ArchLinux
- Link to crash log:

<!---
Please verify that the following data is present before submitting your issue:

- Paste or link to a paste service (http://pastebin.com/ for example) of relevant config (preferably full config including templates if present. Remember to redact any personal information! Please make sure the paste does not expire, if possible.
- Paste or link to a paste service of debug level logs of relevant task/s. Use `flexget -L debug execute --tasks <Task_name>`
- Flexget version (Use `flexget -V` to get it).
- Full Python version (`2.7.11` for example). Run `python -V` to get it.
- Installation method (pip, git install, etc.)
- OS and version
- Attach crash log if available
--->

rtorrent plugin crashed when adding torrent with special utf-8 characters
### Expected behaviour:
Adding entry to rtorrent without crash
### Actual behaviour:
Adding entry to rtorrent but flexget crashed. 
As a consequence, all the accepted entry with this run won't write to the database marked seen, and they will be downloaded again and again. However all the accepted torrent will be sent to the rtorrent, but the crash reports are bothersome.
### Steps to reproduce:
When an entry with special utf-8 characters in name or path accepted and sent to rtorrent plugin.
Such as the example from the log, "Al final del túnel AKA At the End of the Tunnel 2016 1080p BluRay TrueHD 5.1 x264-PbK", "ú" is a special utf-8 characters.
The details of this torrent
```
 'info': {'files': [{'length': 14152687941,
                     'path': ['Al.final.del.t\xc3\xbanel.AKA.At.the.End.of.the.Tunnel.2016.1080p.BluRay.TrueHD.5.1.x264-PbK.mkv']},
                    {'length': 1281,
                     'path': ['Al.final.del.t\xc3\xbanel.AKA.At.the.End.of.the.Tunnel.2016.1080p.BluRay.TrueHD.5.1.x264-PbK.nfo']}],
          'name': 'Al.final.del.t\xc3\xbanel.AKA.At.the.End.of.the.Tunnel.2016.1080p.BluRay.TrueHD.5.1.x264-PbK',
          'piece length': 4194304,
          'pieces': '<3375 piece hashes>',
          'private': 1,
          'source': '[www.hdpter.net] HDPter'},
```
#### Config:
```
  hdp:
    rss: **********
    accept_all: yes
    rtorrent:
      uri: scgi://127.0.0.1:5000
      path: /mnt/V3T/_Movies/_HD
      custom1: HD

```
  
#### Log:
```
2017-02-17 10:37 VERBOSE  task          hdp             REJECTED: `Hacksaw Ridge 2016 1080p BluRay TrueHD Atmos 7.1 x264-PbK` by seen plugin because entry with url `http://www.hdpter.net/download.php?id=10438&passkey=ab0ee1ba02ec1f4cd892219cab827967` is already marked seen in the task hdp at 2017-02-12 22:03
2017-02-17 10:37 VERBOSE  task          hdp             ACCEPTED: `Al final del túnel AKA At the End of the Tunnel 2016 1080p BluRay TrueHD 5.1 x264-PbK` by accept_all plugin
2017-02-17 10:37 INFO     download      hdp             Downloading: Al final del túnel AKA At the End of the Tunnel 2016 1080p BluRay TrueHD 5.1 x264-PbK
2017-02-17 10:37 VERBOSE  details       hdp             Summary - Accepted: 1 (Rejected: 9 Undecided: 0 Failed: 0)
2017-02-17 10:37 CRITICAL task          hdp             BUG: Unhandled error in plugin rtorrent: 'ascii' codec can't decode byte 0xc3 in position 333: ordinal not in range(128)
Traceback (most recent call last):
  File "/home/xiayu/CODE/PYTHON/flexget/lib/python2.7/site-packages/flexget/task.py", line 483, in __run_plugin
    return method(*args, **kwargs)
  File "/home/xiayu/CODE/PYTHON/flexget/lib/python2.7/site-packages/flexget/event.py", line 23, in __call__
    return self.func(*args, **kwargs)
  File "/home/xiayu/CODE/PYTHON/flexget/lib/python2.7/site-packages/flexget/plugins/clients/rtorrent.py", line 531, in on_task_output
    self.add_entry(client, entry, options, start=config['start'], mkdir=config['mkdir'])
  File "/home/xiayu/CODE/PYTHON/flexget/lib/python2.7/site-packages/flexget/plugins/clients/rtorrent.py", line 639, in add_entry
    if client.torrent(entry['torrent_info_hash']):
  File "/home/xiayu/CODE/PYTHON/flexget/lib/python2.7/site-packages/flexget/plugins/clients/rtorrent.py", line 356, in torrent
    resp = multi_call()
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1006, in __call__
    return MultiCallIterator(self.__server.system.multicall(marshalled_list))
  File "/home/xiayu/CODE/PYTHON/flexget/lib/python2.7/site-packages/flexget/plugins/clients/rtorrent.py", line 38, in __call__
    return self.__send(self.__name, args)
  File "/home/xiayu/CODE/PYTHON/flexget/lib/python2.7/site-packages/flexget/plugins/clients/rtorrent.py", line 192, in __request
    verbose=self.__verbose
  File "/home/xiayu/CODE/PYTHON/flexget/lib/python2.7/site-packages/flexget/plugins/clients/rtorrent.py", line 106, in request
    return self.single_request(host, handler, request_body, verbose)
  File "/home/xiayu/CODE/PYTHON/flexget/lib/python2.7/site-packages/flexget/plugins/clients/rtorrent.py", line 134, in single_request
    return self.parse_response(sock.makefile())
  File "/home/xiayu/CODE/PYTHON/flexget/lib/python2.7/site-packages/flexget/plugins/clients/rtorrent.py", line 148, in parse_response
    response_body += data
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 333: ordinal not in range(128)
```

### Additional information:

- Flexget Version: 2.9.23
- Python Version: 2.7.13
- Installation method: virtualenv pip install 
- OS and version: ArchLinux
- Link to crash log:

<!---
Please verify that the following data is present before submitting your issue:

- Paste or link to a paste service (http://pastebin.com/ for example) of relevant config (preferably full config including templates if present. Remember to redact any personal information! Please make sure the paste does not expire, if possible.
- Paste or link to a paste service of debug level logs of relevant task/s. Use `flexget -L debug execute --tasks <Task_name>`
- Flexget version (Use `flexget -V` to get it).
- Full Python version (`2.7.11` for example). Run `python -V` to get it.
- Installation method (pip, git install, etc.)
- OS and version
- Attach crash log if available
--->



Discussion:
The code run well for python3. 
But for python2 the Unicode/utf8/ascii string and bytes, really easy to mix up. 
To be compatible for both python2 & 3, a modification is necessary.
 I have tried to debug this problem.
```
xiayu@Arch_406 ~/CODE/TEST % python -m pdb test.py
> /home/xiayu/CODE/TEST/test.py(1)<module>()
-> from flexget.plugins.clients import rtorrent
(Pdb) n
> /home/xiayu/CODE/TEST/test.py(3)<module>()
-> client=rtorrent.RTorrent('scgi://127.0.0.1:5000')
(Pdb) n
> /home/xiayu/CODE/TEST/test.py(5)<module>()
-> ttinfo = client.torrent('A70D02E29CC47109BC35855A6C3EB4D5BDC2FE50')
(Pdb) s
--Call--
> /home/xiayu/CODE/GITHUBME/Flexget/flexget/plugins/clients/rtorrent.py(342)torrent()
-> def torrent(self, info_hash, fields=None):
(Pdb) b 142
Breakpoint 1 at /home/xiayu/CODE/GITHUBME/Flexget/flexget/plugins/clients/rtorrent.py:142
(Pdb) c
> /home/xiayu/CODE/GITHUBME/Flexget/flexget/plugins/clients/rtorrent.py(142)parse_response()
-> response_body = ''
(Pdb) n
> /home/xiayu/CODE/GITHUBME/Flexget/flexget/plugins/clients/rtorrent.py(144)parse_response()
-> while True:
(Pdb) type(response_body)
<type 'unicode'>
```
the test.py file 
```
from flexget.plugins.clients import rtorrent

client=rtorrent.RTorrent('scgi://127.0.0.1:5000')

ttinfo = client.torrent('A70D02E29CC47109BC35855A6C3EB4D5BDC2FE50')

print(ttinfo)
```
`from __future__ import unicode_literals` cause `response_body = ''` to be `<type 'unicode'>` I try to fix this but failed to  make it work for both python2 and python3 with one uniform code.  
just DO NOT import `unicode_literals` works fine!
The code run well for python3. 
But for python2 the Unicode/utf8/ascii string and bytes, really easy to mix up. 
To be compatible for both python2 & 3, a modification is necessary.
 I have tried to debug this problem.
```
xiayu@Arch_406 ~/CODE/TEST % python -m pdb test.py
> /home/xiayu/CODE/TEST/test.py(1)<module>()
-> from flexget.plugins.clients import rtorrent
(Pdb) n
> /home/xiayu/CODE/TEST/test.py(3)<module>()
-> client=rtorrent.RTorrent('scgi://127.0.0.1:5000')
(Pdb) n
> /home/xiayu/CODE/TEST/test.py(5)<module>()
-> ttinfo = client.torrent('A70D02E29CC47109BC35855A6C3EB4D5BDC2FE50')
(Pdb) s
--Call--
> /home/xiayu/CODE/GITHUBME/Flexget/flexget/plugins/clients/rtorrent.py(342)torrent()
-> def torrent(self, info_hash, fields=None):
(Pdb) b 142
Breakpoint 1 at /home/xiayu/CODE/GITHUBME/Flexget/flexget/plugins/clients/rtorrent.py:142
(Pdb) c
> /home/xiayu/CODE/GITHUBME/Flexget/flexget/plugins/clients/rtorrent.py(142)parse_response()
-> response_body = ''
(Pdb) n
> /home/xiayu/CODE/GITHUBME/Flexget/flexget/plugins/clients/rtorrent.py(144)parse_response()
-> while True:
(Pdb) type(response_body)
<type 'unicode'>
```
the test.py file 
```
from flexget.plugins.clients import rtorrent

client=rtorrent.RTorrent('scgi://127.0.0.1:5000')

ttinfo = client.torrent('A70D02E29CC47109BC35855A6C3EB4D5BDC2FE50')

print(ttinfo)
```
`from __future__ import unicode_literals` cause `response_body = ''` to be `<type 'unicode'>` I try to fix this but failed to  make it work for both python2 and python3 with one uniform code.  
just DO NOT import `unicode_literals` works fine!

Resolving patch:
diff --git a/flexget/plugins/clients/rtorrent.py b/flexget/plugins/clients/rtorrent.py
--- a/flexget/plugins/clients/rtorrent.py
+++ b/flexget/plugins/clients/rtorrent.py
@@ -17,6 +17,7 @@
 from flexget.entry import Entry
 from flexget.config_schema import one_or_more
 from flexget.utils.bittorrent import Torrent, is_torrent_file
+from flexget.utils.tools import native_str_to_text
 
 from requests.auth import HTTPDigestAuth, HTTPBasicAuth
 
@@ -145,14 +146,14 @@ def parse_response(self, response):
             data = response.read(1024)
             if not data:
                 break
-            response_body += data
+            response_body += native_str_to_text(data, encoding='utf-8')
 
         if self.verbose:
             log.info('body: %s', repr(response_body))
 
         # Remove SCGI headers from the response.
         _, response_body = re.split(r'\n\s*?\n', response_body, maxsplit=1)
-        p.feed(response_body)
+        p.feed(response_body.encode('utf-8'))
         p.close()
 
         return u.close()


Test patch: