Repository: Flexget/Flexget

Issue:
Failure when adding already existing torrents to qBittorrent
<!---
Before opening an issue, verify:

- Is this a feature request? Post it on https://feathub.com/Flexget/Flexget
- Is this an issue with webui? Make an issue over on https://github.com/Flexget/webui
- Did you recently upgrade? Look at the Change Log and Upgrade Actions to make sure that you don't need to make any changes to your config https://flexget.com/ChangeLog https://flexget.com/UpgradeActions
- Are you running FlexGet as a daemon? Stop it completely and then start it again https://flexget.com/CLI/daemon
- Did you search to see if the issue already exists? https://github.com/Flexget/Flexget/issues
- Did you fill out the issue template as completely as possible?

The issue template is here because it helps to ensure you submitted all the necessary information the first time, and allows us to more quickly review issues. Please fill it out correctly and do not ignore it, no matter how irrelevant you think it may be. Thanks in advance for your help with this!
--->

### Expected behaviour:
Skip already added torrents
<!---
Please don't just say "it doesn't crash" or "it works". Explain what the expected result is.
--->

### Actual behaviour:
Entire task errors when attempting to add torrents to qBittorrent
### Steps to reproduce:
- Step 1: Add torrents to qBittorrent that are new
- Step 2: Add the same torrents to qBittorrent

#### Config:
```yaml
templates:
  TT:
    if:
      - rss_pubdate|formatdate("%d")|int >= 1 and rss_pubdate|formatdate("%d")|int <= 7:
          qbittorrent:
            label: "{{category|replace('Singles - ','')}}/{{rss_pubdate|formatdate('%Y/%m')}}/01"
            path: /mnt/user/TT/{{category|replace('Singles - ','')}}/{{rss_pubdate|formatdate("%Y/%m")}}/01
      - rss_pubdate|formatdate("%d")|int >= 8 and rss_pubdate|formatdate("%d")|int <= 14:
          qbittorrent:
            label: "{{category|replace('Singles - ','')}}/{{rss_pubdate|formatdate('%Y/%m')}}/02"
            path: /mnt/user/TT/{{category|replace('Singles - ','')}}/{{rss_pubdate|formatdate("%Y/%m")}}/02
      - rss_pubdate|formatdate("%d")|int >= 15 and rss_pubdate|formatdate("%d")|int <= 21:
          qbittorrent:
            label: "{{category|replace('Singles - ','')}}/{{rss_pubdate|formatdate('%Y/%m')}}/03"
            path: /mnt/user/TT/{{category|replace('Singles - ','')}}/{{rss_pubdate|formatdate("%Y/%m")}}/03
      - rss_pubdate|formatdate("%d")|int >= 22 and rss_pubdate|formatdate("%d")|int <= 31:
          qbittorrent:
            label: "{{category|replace('Singles - ','')}}/{{rss_pubdate|formatdate('%Y/%m')}}/04"
            path: /mnt/user/TT/{{category|replace('Singles - ','')}}/{{rss_pubdate|formatdate("%Y/%m")}}/04
      - "now|formatdate('%Y') in title": accept
tasks:
  Trance:
    rss:
      url: <url>
      other_fields: [rss_pubdate,category]
      all_entries: no
    template: TT
```
  
#### Log:
<details>
 <summary>(click to expand)</summary>

```
2020-06-18 07:56:33 CRITICAL plugin        Trance          Error when trying to send request to qBittorrent: Failed to add file to qBittorrent
```
</details>

### Additional information:

- FlexGet version: 3.1.64
- Python version: Python 3.8.3
- Installation method: virtualenv from install guide
- Using daemon (yes/no): both
- OS and version: Debian unstable
- Link to crash log:

<!---
In config and debug/crash logs, remember to redact any personal or sensitive information such as passwords, API keys, private URLs and so on.

Please verify that the following data is present before submitting your issue:

- Link to a paste service or paste above the relevant config (preferably full config, including templates if present). Please make sure the paste does not expire, if possible.
- Link to a paste service or paste above debug-level logs of the relevant task/s (use `flexget -L debug execute --tasks <Task_name>`).
- FlexGet version (use `flexget -V` to get it).
- Full Python version, for example `2.7.11` (use `python -V` to get it). 
- Installation method (pip, git install, etc).
- Whether or not you're running FlexGet as a daemon.
- OS and version.
- Attach crash log if one was generated, in addition to the debug-level log. It can be found in the directory with your config file.
--->


Discussion:
I suspect this is an issue on qBittorrent's end because if I try to do this manually through the qBittorrent API, it also fails when a torrent already exists.

A workaround I've discovered is commenting out the download portion of qbittorrent.py and forcing it to add to qbittorrent by URL instead. The RSS feed I am using has direct download links.
 
```
is_magnet = True
```

```
#@plugin.priority(120)
#def on_task_download(self, task, config):
	"""
	Call download plugin to generate torrent files to load into
	qBittorrent.
	"""
	#config = self.prepare_config(config)
	#if not config['enabled']:
		#return
	#if 'download' not in task.config:
		#download = plugin.get('download', self)
		#download.get_temp_files(task, handle_magnets=True, fail_html=config['fail_html'])
```

If an additional option would be added to let qBittorrent download the torrent instead of FlexGet then adding it by file then this would be easier to handle. The other way would be to use the qBittorrent API prior to adding the torrents and remove entries that already exist.
If we can reliably detect that the error was from adding a torrent that already exists, I think it's fine if we just catch and log that the torrent already exists. If not, perhaps we should go the route of pre-checking whether the torrent already exists. Adding by URL is okay most of the time, but not in cases where flexget is managing cookies or other kinds of logins.
This issue has been mentioned on **Forum - FlexGet**. There might be relevant details there:

https://discuss.flexget.com/t/rss-still-showing-all-entries-with-all-entires-no/5251/1

Same issue here.

Flexget aborts my task and shows `Error when trying to send request to qBittorrent: Failed to add file to qBittorrent`. And debug mode shows a CRITICAL log when flexget adding torrent file from temp folder to qbittorrent.
```
2020-11-15 03:15:25 DEBUG    qbittorrent   Anime           Added torrent file /tmp/tmpa5rbtf10/5066ef7d2252edc28exxxxxxxxxxxxxxx to qBittorrent
2020-11-15 03:15:25 DEBUG    qbittorrent   Anime           Added torrent file /tmp/tmp4ranzweg/69a571f90722dbf33xxxxxxxxxxxxxxx to qBittorrent
2020-11-15 03:15:25 DEBUG    qbittorrent   Anime           Added torrent file /tmp/tmp2w28owir/4adc6bd7fec929c94xxxxxxxxxxxxxxx to qBittorrent
2020-11-15 03:15:25 CRITICAL plugin        Anime           Error when trying to send request to qBittorrent: Failed to add file to qBittorrent
2020-11-15 03:15:25 WARNING  task          Anime           Aborting task (plugin: qbittorrent)
2020-11-15 03:15:25 DEBUG    backlog       Anime           Remembering all entries to backlog because of task abort.
```
I dumped the network traffic between flexget and qb, the crash happens when add-new-torrent api `api/v2/torrents/add` return HTTP 200 and response text `Fails.`. I added that torrent to qb manually via qb original webui and dumped traffic, that api `Fails` again.
But [add-new-torrent api doc](https://github.com/qbittorrent/qBittorrent/wiki/WebUI-API-(qBittorrent-4.1)#add-new-torrent) doesn't say what `Fails` mean and what will return when adding existed torrent to qb.

Maybe check HTTP200 instead of check `response.text == "Ok."` in `qbittorrent.py` may solve the prob temporarily?

The same problem appeared, does anyone know how to solve it,thanks
Hmm. I just had a peek at the qbittorrent source, and there is no way via the api to tell the difference if there was a failure adding, or if the torrent was already there. I see 2 ways forward:
- Always ignore errors adding torrents.
- Check if the torrent is already in the session explicitly before trying to add.
Option 2 is probably better. If someone using qbittorrent could submit a PR it would be great. 😃 

Resolving patch:
diff --git a/flexget/plugins/clients/qbittorrent.py b/flexget/plugins/clients/qbittorrent.py
--- a/flexget/plugins/clients/qbittorrent.py
+++ b/flexget/plugins/clients/qbittorrent.py
@@ -1,4 +1,5 @@
 import os
+from json import loads, JSONDecodeError
 
 from loguru import logger
 from requests import Session
@@ -60,6 +61,7 @@ def __init__(self):
         self.api_url_login = None
         self.api_url_upload = None
         self.api_url_download = None
+        self.api_url_info = None
         self.url = None
         self.connected = False
 
@@ -81,6 +83,7 @@ def check_api_version(self, msg_on_fail, verify=True):
                 self.api_url_login = '/api/v2/auth/login'
                 self.api_url_upload = '/api/v2/torrents/add'
                 self.api_url_download = '/api/v2/torrents/add'
+                self.api_url_info = '/api/v2/torrents/info'
                 return response
 
             url = self.url + "/version/api"
@@ -89,6 +92,7 @@ def check_api_version(self, msg_on_fail, verify=True):
                 self.api_url_login = '/login'
                 self.api_url_upload = '/command/upload'
                 self.api_url_download = '/command/download'
+                self.api_url_info = '/query/torrents'
                 return response
 
             msg = 'Failure. URL: {}'.format(url) if not msg_on_fail else msg_on_fail
@@ -120,9 +124,50 @@ def connect(self, config):
         logger.debug('Successfully connected to qBittorrent')
         self.connected = True
 
+    def check_torrent_exists(self, hash_torrent, verify_cert):
+        if not self.connected:
+            raise plugin.PluginError('Not connected.')
+
+        if not isinstance(hash_torrent, str):
+            logger.error('Error getting torrent info, invalid hash {}', hash_torrent)
+            return False
+
+        hash_torrent = hash_torrent.lower()
+
+        url = f'{self.url}{self.api_url_info}'
+        params = {'hashes': hash_torrent}
+
+        try:
+            respose = self.session.request(
+                'get',
+                url,
+                params=params,
+                verify=verify_cert,
+            )
+        except RequestException as e:
+            logger.error('Error getting torrent info, request to hash {} failed', hash_torrent)
+            return False
+
+        if respose.status_code != 200:
+            logger.error(
+                'Error getting torrent info, hash {} search returned',
+                hash_torrent,
+                respose.status_code,
+            )
+            return False
+
+        check_file = respose.json()
+
+        if isinstance(check_file, list) and check_file:
+            logger.warning('File with hash {} already in qbittorrent', hash_torrent)
+            return True
+
+        return False
+
     def add_torrent_file(self, file_path, data, verify_cert):
         if not self.connected:
             raise plugin.PluginError('Not connected.')
+
         multipart_data = {k: (None, v) for k, v in data.items()}
         with open(file_path, 'rb') as f:
             multipart_data['torrents'] = f
@@ -138,6 +183,7 @@ def add_torrent_file(self, file_path, data, verify_cert):
     def add_torrent_url(self, url, data, verify_cert):
         if not self.connected:
             raise plugin.PluginError('Not connected.')
+
         data['urls'] = url
         multipart_data = {k: (None, v) for k, v in data.items()}
         self._request(
@@ -166,6 +212,7 @@ def prepare_config(config):
 
     def add_entries(self, task, config):
         for entry in task.accepted:
+
             form_data = {}
             try:
                 save_path = entry.render(entry.get('path', config.get('path', '')))
@@ -214,6 +261,11 @@ def add_entries(self, task, config):
                     logger.info('Download Speed Limit: {}', form_data.get('dlLimit'))
                 continue
 
+            if self.check_torrent_exists(
+                entry.get('torrent_info_hash'), config.get('verify_cert')
+            ):
+                continue
+
             if not is_magnet:
                 if 'file' not in entry:
                     entry.fail('File missing?')


Test patch: