Repository: Flexget/Flexget

Issue:
from_deluge keys option is incomplete
### Expected behaviour:
For the `keys` option of `from_deluge` to accept any (simple) value that deluge provides in its RPC API.

### Actual behaviour:
As of right now, the `keys` option of `from_deluge` supports a subset of keys available from the RPC API. Examples of missing keys that are available from the RPC API but not from `from_deluge` are:
1. `eta`
2. `hash`
3. `is_finished`
4. `label`
5. `move_completed`
6. `move_completed_path`
7. `move_on_completed_path`
8. `name`
9. `num_peers`
10. `num_seeds`
11. `private`
12. `progress`
13. `ratio`
14. `save_path`
15. `seeding_time`
16. `seeds_peers_ratio`
17. `state`
18. `time_added`
19. `total_size`
### Steps to reproduce:
Configure from_deluge, and attempt to set the keys option to any of the values listed above.

NOTE: I have not tested every single one of those to see if they throw an error. I tested a few and found they were not able to be used in the `keys` option. I then looked at the documentation for a list of the  available `keys` values and compared it with the results of an RPC call to see what was missing. The above list is what is present in the RPC call but missing from the `from_deluge ` `keys` option documentation.

#### Config:
```
    seen: local
    seen_info_hash: local
    from_deluge:
      host: '{? deluge.host ?}'
      port: 58846
      username: '{? deluge.username ?}'
      password: '{? deluge.password ?}'
      keys:
        - stop_ratio
        - seeding_time
```
  
#### Log:
<details>
 <summary>(click to expand)</summary>

```
There is a FlexGet process already running for this config, sending execution there.
2018-11-20 15:37 VERBOSE  check                         Checking config file `/root/.flexget/config.yml`
2018-11-20 15:37 VERBOSE  check                         Pre-checked 315 configuration lines
2018-11-20 15:37 CRITICAL check                         [/templates/input-deluge_torrents/from_deluge/keys/1] Must be one of the following: active_time, is_seed, num_pieces, total_payload_download, stop_at_ratio, total_seeds, peers, next_announce, total_payload_upload, paused, file_progress, max_download_speed, max_upload_slots, prioritize_first_last, piece_length, download_payload_rate, message, upload_payload_rate, max_connections, compact, move_on_completed, trackers, total_done, is_auto_managed, total_peers, total_wanted, queue, num_files, max_upload_speed, distributed_copies, tracker, stop_ratio, seed_rank, tracker_status, total_uploaded, tracker_host, remove_at_ratio
```
</details>

### Additional information:

- FlexGet version: 2.7.13
- Python version: 2.7.12
- Installation method: pip
- Using daemon (yes/no): yes
- OS and version: Ubuntu 14.04


Discussion:
This is not really a bug, but a feature request.
@cvium Okay... do you need me to create a feature request for this then?

Sorry, I just figured this was more of an issue seeing as most likely almost all the code is already there, the intent for the use is already there, it is just missing keys that should be there.
I think most, if not all of these fields are available by default, with a `deluge_` prefix to the key. It is a bit confusing how some of them are default, and some have to be specified in the `keys` part of the config, and how those ones don't get the `deluge_` prefix. Not sure if we should try to clean that up or not.

https://flexget.com/Plugins/from_deluge#entry-fields

https://github.com/Flexget/Flexget/blob/develop/flexget/plugins/clients/deluge.py#L85-L100

@gazpachoking , yeah, some of those are definitely available as entry fields... not sure why I overlooked those when I was doing my comparison! I'll do some testing and update this issues. Looking at the code you referenced, it looks like the documentation doesn't quite line up in regards to settigns_map => entry fields and extra_settings_map => keys fields (Just basing this off comparison of number of entries in the code vs number of entries in the documentation).

Also, comparing my list to keys reference between settings_map and extra_settings_map leaves the following data elements out from the deluge RPC API:

1. `is_finished`
2. `move_completed`
3. `move_completed_path`
4. `seeds_peers_ratio`
5. `time_added`

Yeah, looks like some of those are missing. We are doing move_on_completed, and move_on_completed_path, which looks like they have been deprecated in favor of the versions without the 'on'.
@gazpachoking cleanup with consistent deluge_ prefix along with consistent plugin option naming (with underscores). Maybe something for next upgrade actions version?

Resolving patch:
diff --git a/flexget/plugins/clients/deluge.py b/flexget/plugins/clients/deluge.py
--- a/flexget/plugins/clients/deluge.py
+++ b/flexget/plugins/clients/deluge.py
@@ -1,6 +1,5 @@
 from __future__ import unicode_literals, division, absolute_import
 from builtins import *  # noqa pylint: disable=unused-import, redefined-builtin
-from future.utils import native
 
 import base64
 import re
@@ -78,67 +77,15 @@ def get_localhost_auth():
 
 class InputDeluge(DelugePlugin):
     """Create entries for torrents in the deluge session."""
-    #
+    # Fields we provide outside of the deluge_ prefixed namespace
     settings_map = {
         'name': 'title',
         'hash': 'torrent_info_hash',
         'num_peers': 'torrent_peers',
         'num_seeds': 'torrent_seeds',
-        'progress': 'deluge_progress',
-        'seeding_time': ('deluge_seed_time', lambda time: time / 3600),
-        'private': 'deluge_private',
-        'state': 'deluge_state',
-        'eta': 'deluge_eta',
-        'ratio': 'deluge_ratio',
-        'move_on_completed_path': 'deluge_movedone',
-        'save_path': 'deluge_path',
-        'label': 'deluge_label',
         'total_size': ('content_size', lambda size: size / 1024 / 1024),
         'files': ('content_files', lambda file_dicts: [f['path'] for f in file_dicts])}
 
-    extra_settings_map = {
-        'active_time': ('active_time', lambda time: time / 3600),
-        'compact': 'compact',
-        'distributed_copies': 'distributed_copies',
-        'download_payload_rate': 'download_payload_rate',
-        'file_progress': 'file_progress',
-        'is_auto_managed': 'is_auto_managed',
-        'is_seed': 'is_seed',
-        'max_connections': 'max_connections',
-        'max_download_speed': 'max_download_speed',
-        'max_upload_slots': 'max_upload_slots',
-        'max_upload_speed':  'max_upload_speed',
-        'message': 'message',
-        'move_on_completed': 'move_on_completed',
-        'next_announce': 'next_announce',
-        'num_files': 'num_files',
-        'num_pieces': 'num_pieces',
-        'paused': 'paused',
-        'peers': 'peers',
-        'piece_length': 'piece_length',
-        'prioritize_first_last': 'prioritize_first_last',
-        'queue': 'queue',
-        'remove_at_ratio': 'remove_at_ratio',
-        'seed_rank': 'seed_rank',
-        'stop_at_ratio': 'stop_at_ratio',
-        'stop_ratio': 'stop_ratio',
-        'total_done': 'total_done',
-        'total_payload_download': 'total_payload_download',
-        'total_payload_upload': 'total_payload_upload',
-        'total_peers': 'total_peers',
-        'total_seeds': 'total_seeds',
-        'total_uploaded': 'total_uploaded',
-        'total_wanted': 'total_wanted',
-        'tracker': 'tracker',
-        'tracker_host': 'tracker_host',
-        'tracker_status': 'tracker_status',
-        'trackers': 'trackers',
-        'upload_payload_rate': 'upload_payload_rate'
-    }
-
-    def __init__(self):
-        self.entries = []
-
     schema = {
         'anyOf': [
             {'type': 'boolean'},
@@ -161,13 +108,6 @@ def __init__(self):
                         },
                         'additionalProperties': False
                     },
-                    'keys': {
-                        'type': 'array',
-                        'items': {
-                            'type': 'string',
-                            'enum': list(extra_settings_map)
-                        }
-                    }
                 },
                 'additionalProperties': False
             }
@@ -192,22 +132,19 @@ def prepare_config(self, config):
 
     def on_task_input(self, task, config):
         """Generates and returns a list of entries from the deluge daemon."""
+        config = self.prepare_config(config)
         # Reset the entries list
-        self.entries = []
         client = self.setup_client(config)
         client.connect()
 
-        self.entries = self.generate_entries(client, config)
+        entries = self.generate_entries(client, config)
         client.disconnect()
-        return self.entries
+        return entries
 
     def generate_entries(self, client, config):
         entries = []
         filter = config.get('filter', {})
-        # deluge client lib chokes on future's newlist, make sure we have a native python list here
-        fields = config.get('keys', [])
-        fields.extend(self.settings_map)
-        torrents = client.call('core.get_torrents_status', filter or {}, fields)
+        torrents = client.call('core.get_torrents_status', filter or {}, [])
         for hash, torrent_dict in torrents.items():
             # Make sure it has a url so no plugins crash
             entry = Entry(deluge_id=hash, url='')
@@ -222,14 +159,15 @@ def generate_entries(self, client, config):
                 else:
                     log.warning('Did not find torrent file at %s', torrent_path)
             for key, value in torrent_dict.items():
+                # All fields provided by deluge get placed under the deluge_ namespace
+                entry['deluge_' + key] = value
+                # Some fields also get special handling
                 if key in self.settings_map:
                     flexget_key = self.settings_map[key]
-                else:
-                    flexget_key = self.extra_settings_map[key]
-                if isinstance(flexget_key, tuple):
-                    flexget_key, format_func = flexget_key
-                    value = format_func(value)
-                entry[flexget_key] = value
+                    if isinstance(flexget_key, tuple):
+                        flexget_key, format_func = flexget_key
+                        value = format_func(value)
+                    entry[flexget_key] = value
             entries.append(entry)
 
         return entries


Test patch: