Repository: Flexget/Flexget

Issue:
Qbittorrent plugin doesn't add torrent to client
### Expected behaviour:

Flexget to download the torrent and insert into qbittorrent
### Actual behaviour:

Flexget download the torrent to tmp and claims it added to qbittorrent but it never shows up in the client
### Steps to reproduce:

This is tricky as for some site the torrent gets added to the client without any problem. And in some case it doesn't, but the torrent file is created in the tmp folder.
#### Config:

```
templates:
  tv:
    qbittorrent:
      host: XXXXXX
      port: XXXX
      username: XXXX
      password: XXXX
      label: series
    series:
      settings:
        hdtv:
          quality: hdtv <720p
          set:
            path: path_to_directory/{{series_name}}
      hdtv:
        - Hit the floor
tasks:
  dlseries:
    headers:
      Cookie: "XXXXXXXX"
    rss: FEED URL
    verify_ssl_certificates: no
    template: tv

```
#### Log:

```
2016-09-06 17:39 DEBUG    series        dlseries           Series accepting. <TORRENTNAME> meets quality hdtv
2016-09-06 17:39 VERBOSE  task          dlseries           ACCEPTED: `<TORRENTNAME>` by series plugin because target quality
2016-09-06 17:39 DEBUG    urlrewriter   dlseries           Checking 1 entries
2016-09-06 17:39 INFO     download      dlseries           Downloading: <TORRENTNAME>
2016-09-06 17:39 DEBUG    download      dlseries           Downloading url '<TORRENTURL>'
2016-09-06 17:39 DEBUG    utils.requests dlseries           Fetching <TORRENTURL>
2016-09-06 17:39 DEBUG    download      dlseries           <TORRENTNAME> field file set to: /tmp/tmpX_QHcL/87a3ce82ae44e8d94daa57e28191bcb9
2016-09-06 17:39 DEBUG    download      dlseries           filename header latin1 decoded
2016-09-06 17:39 DEBUG    download      dlseries           Found filename from headers: <TORRENTNAME>
2016-09-06 17:39 DEBUG    download      dlseries           Mimetype guess for application/x-bittorrent is ['.torrent']
2016-09-06 17:39 DEBUG    download      dlseries           Filename <TORRENTNAME>.torrent extension matches to mime-type
2016-09-06 17:39 DEBUG    download      dlseries           Finishing download_entry() with filename <TORRENTNAME>.torrent
2016-09-06 17:39 DEBUG    download      dlseries           Successfully retrieved <TORRENTNAME> from <TORRENTURL>
2016-09-06 17:39 VERBOSE  details       dlseries           Summary - Accepted: 1 (Rejected: 0 Undecided: 39 Failed: 0)
2016-09-06 17:39 DEBUG    modif_torrent dlseries           <TORRENTNAME> seems to be a torrent
2016-09-06 17:39 DEBUG    torrent_files dlseries           <TORRENTNAME> files: [u'<TORRENTNAME>.mkv', u'Sample/<TORRENTNAME>.mkv', u'<TORRENTNAME>.nfo']
2016-09-06 17:39 DEBUG    torrent_size  dlseries           <TORRENTNAME> size: 319.150249481 MB
2016-09-06 17:39 DEBUG    qbittorrent   dlseries           Successfully connected to qBittorrent
2016-09-06 17:39 DEBUG    qbittorrent   dlseries           Added task to qBittorrent

```
### Additional information:
- Flexget Version: 2.3.18
- Python Version: 2.7.9
- Installation method: pip
- OS and version: Debian 8.5 (OSMC)
- Link to crash log:

Please verify that the following data is present before submitting your issue:

-Paste or link to a paste service ([pastebin](http://pastebin.com/) for example) of relevant config (preferably full config including templates if present. Remember to redact any personal information!
- Paste or link to a paste service of debug level logs of relevant task/s. Use `flexget -L debug execute --tasks <Task_name>`
- Flexget version (Use `flexget -V` to get it).
- Full Python version (`2.7.11` for example). Run `python -V` to get it.
- Installation method (pip, git install, etc.)
- OS and version
- Attach crash log if available



Discussion:
Is the file that's being sent to qbitorrent local or http? Because the plugin doesn't support uploading local files apparently 

Hi,

Thanks for the reply!
The file is grabbed via http from an RSS feed. As far as I can tell flexget saves it in the tmp folder during download.
I hope that answers your question.

Well I use qbitorrent plugin with no issues...
The filename you redacted, does it have http:// schema or file://?

On Wed, Sep 7, 2016, 00:21 EDanee notifications@github.com wrote:

> Hi,
> 
> Thanks for the reply!
> The file is grabbed via http from an RSS feed. As far as I can tell
> flexget saves it in the tmp folder during download.
> I hope that answers your question.
> 
> —
> You are receiving this because you commented.
> 
> Reply to this email directly, view it on GitHub
> https://github.com/Flexget/Flexget/issues/1390#issuecomment-245096342,
> or mute the thread
> https://github.com/notifications/unsubscribe-auth/AELANdZzvE8I2UaGROHA-cRvCtYQXsbCks5qndlkgaJpZM4J2AQX
> .

Perhaps the torrent site requires some sort of log in, so when the url is passed to qbittorrent it fails to actually be added. Does qbittorrent have a log with anything?

I think that would still be added but with error state, but who knows... 

Hi guys,
The site does require login, but by using cookies that is bypassed.
qbittorrent has no longs - I will try to run it in non-daemon mode if that possible to get more information.
the file is https:// and it gets saved to a temp location

```
2016-09-06 17:54 INFO     download      dlseries           Downloading: Hit.the.Floor.S03E11.HDTV.x264-FLEET [Sorozat (ENG SD)]
2016-09-06 17:54 DEBUG    download      dlseries           Downloading url 'https://SITE/torrents.php?action=download&id=2023111'
2016-09-06 17:54 DEBUG    utils.requests dlseries           Fetching https://SITE/torrents.php?action=download&id=2023111
2016-09-06 17:54 DEBUG    download      dlseries           Hit.the.Floor.S03E11.HDTV.x264-FLEET [Sorozat (ENG SD)] field file set to: /tmp/tmpu84ajN/87a3ce82ae44e8d94daa57e28191bcb9
```

The strange this is that for other sites - where no cookie and https is used - the file gets added successfully.
So I suspect either this or maybe the filename has chars that causes problems?

You sort of answered your own question there.
@liiight wrote: `the plugin doesn't support uploading local files apparently`

You cannot pass the cookie via the url.

@dsemi is the author of qbittorrent plugin. Ask him nicely to implement https://github.com/qbittorrent/qBittorrent/wiki/WebUI-API-Documentation#upload-torrent-from-disk 

@cvium I believe upload torrent from disk is actually to add a file to be seeded, not for downloading files. The current download from url should work fine; it sounds like a cookie param just needs to be added as a config option to the plugin. The API supports specifying a cookie: https://github.com/qbittorrent/qBittorrent/wiki/WebUI-API-Documentation#download-torrent-from-url

@EDanee I've added the cookie param in my fork [here](https://github.com/dsemi/Flexget). Can you test it to make sure it's working for you?  You just have to update the config file to specify `cookie: XXX` in the `qbittorrent` section.
If everything is working, I'll submit a pull request.

> I believe upload torrent from disk is actually to add a file to be seeded, not for downloading files. 

Definitely not.

> Definitely not.

Either way, the "download-torrent-from-url" has been working with local files on the system for everyone else that uses the plugin. I wrote it to download the torrent files to a temp folder and adds those to qBittorrent; changed it to that behavior after [this](https://github.com/Flexget/Flexget/pull/867#issuecomment-184493046) comment in the original PR. 

Now that I think about it though, adding a cookie param wouldn't help since the torrent file is already downloaded, there shouldn't be any need to authenticate anymore. 

@EDanee What version of qBittorrent are you using?

@dsemi I don't think it's working quite how you intended. Only the torrent url is passed. The torrent file is definitely not passed.

@cvium `entry['url']` should point to the file location after`on_task_download`. At least that's my understanding after reading the transmission plugin.

Ah wait I see the issue, you're right, it's in a different location.

@dsemi I think the nicest way would be to switch to using this endpoint https://github.com/qbittorrent/qBittorrent/wiki/WebUI-API-Documentation#upload-torrent-from-disk to add the torrents based on the downloaded files, rather than passing urls to qbittorrent. This would eliminate the need for qbittorrent to know about cookies or anything. It should still fall back to the url way in the case of a magnet url.

@gazpachoking Yeah there's a bug in the original implementation. It's not actually using the tmp torrent files that were downloaded.  I'll make a quick fix and send it out.

> Yeah there's a bug in the original implementation. It's not actually using the tmp torrent files that were downloaded. I'll make a quick fix and send it out.

I bet the download bit was just put in there so the plugin worked properly with content_size and content_filter plugins. Using those downloaded files is indeed the nicer way to do it though, a PR would be great.

Also adding torrent files from a local resource, like a watchdir for example, is a feature I miss greatly 

> Also adding torrent files from a local resource, like a watchdir for example, is a feature I miss greatly

That should work already, no? If you do a filesystem input plugin it'll have file:// urls, which should work (as long as transmission is running on the same box as flexget.) Using the real files is still nicer though, @dsemi's upcoming update should take care of both situations I think.

Hm. Unfortunately qBittorrent doesn't seem to support setting savepath, label, etc. when using a torrent file on disk. Or at least it's not documented.

@liiight Is that a thing in Flexget? You can set a watchdir in your torrent client.

@dsemi Maybe try just putting the same fields like for this https://github.com/qbittorrent/qBittorrent/wiki/WebUI-API-Documentation#download-torrent-from-url

Hopefully those work the same for this endpoint. Otherwise I think qbittorrent should improve their api...

EDIT: Man, I don't like those api docs, putting the raw HTTP data isn't so helpful to humans.

https://github.com/qbittorrent/qBittorrent/blob/master/src/webui/www/public/upload.html#L21-L26

Looks like it should support label and savepath

@dsemi I'm using qbittorrent-nox (webUI only) version 3.3.6

@dsemi i like to use a flexget based watchdir since it allows me to easily keep series tracking even for manually added files. 

@gazpachoking unfortunately that does not work, adding files from a local resource requires an entirely different http request to qbitorrent than the one that's currently supported 

@gazpachoking Is there an easy way to test adding a random .torrent file? My setup is w/ magnets right now which doesn't really test the changes I'm making.

@liiight I see. I just wasn't aware of the filesystem plugin.

@dsemi I'd just make a small test config like this:

``` yaml
tasks:
  testtask:
    mock:
      - title: blah blah
        url: http://url.to/some/torrent
    accept_all: yes
    qbittorrent: whatever settings
    disable: [seen, seen_info_hash]  # so you can keep running this task over and over
```


Resolving patch:
diff --git a/flexget/plugins/output/qbittorrent.py b/flexget/plugins/output/qbittorrent.py
--- a/flexget/plugins/output/qbittorrent.py
+++ b/flexget/plugins/output/qbittorrent.py
@@ -1,6 +1,7 @@
 from __future__ import unicode_literals, division, absolute_import
 from builtins import *  # pylint: disable=unused-import, redefined-builtin
 
+import os
 import logging
 
 from requests import Session
@@ -60,12 +61,22 @@ def connect(self, config):
         log.debug('Successfully connected to qBittorrent')
         self.connected = True
 
-    def add_torrent(self, data):
+    def add_torrent_file(self, filepath, data):
         if not self.connected:
             raise plugin.PluginError('Not connected.')
         multipart_data = {k: (None, v) for k, v in data.items()}
+        with open(filepath, 'rb') as f:
+            multipart_data['torrents'] = f
+            self.session.post(self.url + '/command/upload', files=multipart_data)
+        log.debug('Added torrent file %s to qBittorrent', filepath)
+
+    def add_torrent_url(self, url, data):
+        if not self.connected:
+            raise plugin.PluginError('Not connected.')
+        data['urls'] = url
+        multipart_data = {k: (None, v) for k, v in data.items()}
         self.session.post(self.url + '/command/download', files=multipart_data)
-        log.debug('Added task to qBittorrent')
+        log.debug('Added url %s to qBittorrent', url)
 
     def prepare_config(self, config):
         if isinstance(config, bool):
@@ -77,6 +88,46 @@ def prepare_config(self, config):
         config.setdefault('fail_html', True)
         return config
 
+    def add_entries(self, task, config):
+        for entry in task.accepted:
+            formdata = {}
+
+            savepath = entry.get('path', config.get('path'))
+            if savepath:
+                formdata['savepath'] = savepath
+
+            label = entry.get('label', config.get('label'))
+            if label:
+                formdata['label'] = label # qBittorrent v3.3.3-
+                formdata['category'] = label # qBittorrent v3.3.4+
+
+            is_magnet = entry['url'].startswith('magnet:')
+
+            if task.manager.options.test:
+                log.info('Test mode.')
+                log.info('Would add torrent to qBittorrent with:')
+                if not is_magnet:
+                    log.info('File: %s', entry.get('file'))
+                else:
+                    log.info('Url: %s', entry.get('url'))
+                log.info('Save path: %s', formdata.get('savepath'))
+                log.info('Label: %s', formdata.get('label'))
+                continue
+
+            if not is_magnet:
+                if 'file' not in entry:
+                    entry.fail('File missing?')
+                    continue
+                if not os.path.exists(entry['file']):
+                    tmp_path = os.path.join(task.manager.config_base, 'temp')
+                    log.debug('entry: %s', entry)
+                    log.debug('temp: %s', ', '.join(os.listdir(tmp_path)))
+                    entry.fail("Downloaded temp file '%s' doesn't exist!?" % entry['file'])
+                    continue
+                self.add_torrent_file(entry['file'], formdata)
+            else:
+                self.add_torrent_url(entry['url'], formdata)
+
     @plugin.priority(120)
     def on_task_download(self, task, config):
         """
@@ -96,25 +147,7 @@ def on_task_output(self, task, config):
         if task.accepted:
             config = self.prepare_config(config)
             self.connect(config)
-        for entry in task.accepted:
-            data = {}
-            savepath = entry.get('path', config.get('path'))
-            if savepath:
-                data['savepath'] = savepath
-            label = entry.get('label', config['label']).lower()
-            if label:
-                data['label'] = label
-            data['urls'] = entry.get('url')
-            if task.manager.options.test:
-                log.info('Test mode.')
-                log.info('Would add torrent to qBittorrent with:')
-                log.info('    Url: %s', data['urls'][0])
-                if data['savepath']:
-                    log.info('    Save path: %s', data['savepath'])
-                if data['label']:
-                    log.info('    Label: %s', data['label'])
-            else:
-                self.add_torrent(data)
+            self.add_entries(task, config)
 
 
 @event('plugin.register')


Test patch: