Repository: Flexget/Flexget

Issue:
Imdb watchlist html parser fix
### Motivation for changes:
imdb_watchlist fails to retrieve list 'ratings' because soup is unable to detect correct total_item_count. Different div classes are used for 'ratings' and 'lsXXXXXXXXX'.

### Detailed changes:
- Update parser to locate correct total_item_count on both lists.

### Addressed issues:
- Fixes list: ratings. All lists should now work as expected.

### Log and/or tests output (preferably both):
This is the log when retrieving list: ratings from imdb_watchlist
```
C:\Users\tuant\PycharmProjects\Flexget\venv\Scripts\python.exe C:/Users/tuant/PycharmProjects/Flexget/flexget_vanilla.py --loglevel=debug -c test-imdb_watchlist.yml execute --discover-now --now --no-cache
2020-08-04 16:58:01 DEBUG    manager                       Figuring out config load paths
2020-08-04 16:58:01 DEBUG    manager                       Found config: C:\Users\tuant\PycharmProjects\Flexget\test-imdb_watchlist.yml
2020-08-04 16:58:01 DEBUG    manager                       Config file C:\Users\tuant\PycharmProjects\Flexget\test-imdb_watchlist.yml selected
2020-08-04 16:58:01 DEBUG    manager                       sys.defaultencoding: utf-8
2020-08-04 16:58:01 DEBUG    manager                       sys.getfilesystemencoding: utf-8
2020-08-04 16:58:01 DEBUG    manager                       flexget detected io encoding: utf-8
2020-08-04 16:58:01 DEBUG    manager                       os.path.supports_unicode_filenames: True
2020-08-04 16:58:01 DEBUG    plugin                        Trying to load plugins from: ['C:\\Users\\tuant\\PycharmProjects\\Flexget\\plugins', 'C:\\Users\\tuant\\PycharmProjects\\Flexget\\flexget\\plugins']
2020-08-04 16:58:02 DEBUG    plugin                        Plugin `memusage` requires plugin `ext lib `guppy`` to load.
2020-08-04 16:58:02 DEBUG    plugin                        Trying to load components from: ['C:\\Users\\tuant\\PycharmProjects\\Flexget\\components', 'C:\\Users\\tuant\\PycharmProjects\\Flexget\\flexget\\components']
2020-08-04 16:58:02 DEBUG    plugin                        Plugins took 1.71 seconds to load. 306 plugins in registry.
2020-08-04 16:58:02 DEBUG    manager                       Connecting to: sqlite:///C:\\Users\\tuant\\PycharmProjects\\Flexget\\db-test-imdb_watchlist.sqlite
2020-08-04 16:58:02 DEBUG    manager                       config_name: test-imdb_watchlist
2020-08-04 16:58:02 DEBUG    manager                       config_base: C:\Users\tuant\PycharmProjects\Flexget
2020-08-04 16:58:02 DEBUG    manager                       New config data loaded.
2020-08-04 16:58:02 DEBUG    schema                        current flexget version already exist in db 3.1.68.dev
2020-08-04 16:58:02 DEBUG    parsing                       setting default movie parser to internal. (options: {'guessit': <flexget.components.parsing.parsers.parser_guessit.ParserGuessit object at 0x06B61880>, 'internal': <flexget.components.parsing.parsers.parser_internal.ParserInternal object at 0x06B61940>})
2020-08-04 16:58:02 DEBUG    parsing                       setting default series parser to internal. (options: {'guessit': <flexget.components.parsing.parsers.parser_guessit.ParserGuessit object at 0x06B61880>, 'internal': <flexget.components.parsing.parsers.parser_internal.ParserInternal object at 0x06B61940>})
2020-08-04 16:58:02 DEBUG    cron_env                      Encoding utf-8 stored
2020-08-04 16:58:02 DEBUG    util.simple_persistence                 setting key terminal_encoding value 'utf-8'
2020-08-04 16:58:02 DEBUG    task_queue                    task queue shutdown requested
2020-08-04 16:58:02 VERBOSE  task_queue                    There are 1 tasks to execute. Shutdown will commence when they have completed.
2020-08-04 16:58:02 INFO     ipc.rpyc                      server started on [127.0.0.1]:5947
2020-08-04 16:58:02 DEBUG    task          test-imdb_watchlist executing test-imdb_watchlist
2020-08-04 16:58:03 DEBUG    remember_rej  test-imdb_watchlist Task config has changed since last run, purging remembered entries.
2020-08-04 16:58:03 DEBUG    input_cache   test-imdb_watchlist cache name: imdb_watchlist_XXXXXXXXXXXXXXXXXXXX (has: )
2020-08-04 16:58:03 VERBOSE  imdb_watchlist test-imdb_watchlist Retrieving imdb list: ratings
2020-08-04 16:58:03 DEBUG    imdb_watchlist test-imdb_watchlist Requesting: http://www.imdb.com/user/urXXXXXXXX/ratings {'Accept-Language': 'en-us'}
2020-08-04 16:58:03 DEBUG    utils.requests test-imdb_watchlist GETing URL http://www.imdb.com/user/urXXXXXXXX/ratings with args () and kwargs {'params': {'view': 'detail', 'page': 1}, 'headers': {'Accept-Language': 'en-us'}, 'allow_redirects': True, 'timeout': 30}
2020-08-04 16:58:10 VERBOSE  imdb_watchlist test-imdb_watchlist No movies were found in imdb list: ratings
2020-08-04 16:58:10 DEBUG    input_cache   test-imdb_watchlist storing entries to cache imdb_watchlist_XXXXXXXXXXXXXXXXXXXX 
2020-08-04 16:58:10 CRITICAL task          test-imdb_watchlist BUG: Unhandled error in plugin imdb_watchlist: 'NoneType' object is not iterable
Traceback (most recent call last):

  File "C:\Users\tuant\AppData\Local\Programs\Python\Python38-32\lib\threading.py", line 890, in _bootstrap
    self._bootstrap_inner()
    |    -> <function Thread._bootstrap_inner at 0x031C1580>
    -> <Thread(task_queue, started daemon 13740)>

  File "C:\Users\tuant\AppData\Local\Programs\Python\Python38-32\lib\threading.py", line 932, in _bootstrap_inner
    self.run()
    |    -> <function Thread.run at 0x031C1418>
    -> <Thread(task_queue, started daemon 13740)>

  File "C:\Users\tuant\AppData\Local\Programs\Python\Python38-32\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
    |    |        |    |        |    -> {}
    |    |        |    |        -> <Thread(task_queue, started daemon 13740)>
    |    |        |    -> ()
    |    |        -> <Thread(task_queue, started daemon 13740)>
    |    -> <bound method TaskQueue.run of <flexget.task_queue.TaskQueue object at 0x06B7ABC8>>
    -> <Thread(task_queue, started daemon 13740)>

  File "C:\Users\tuant\PycharmProjects\Flexget\flexget\task_queue.py", line 46, in run
    self.current_task.execute()
    |    |            -> <function Task.execute at 0x04880418>
    |    -> <flexget.task.Task object at 0x06DC10E8>
    -> <flexget.task_queue.TaskQueue object at 0x06B7ABC8>

  File "C:\Users\tuant\PycharmProjects\Flexget\flexget\task.py", line 80, in wrapper
    return func(self, *args, **kw)
           |    |      |       -> {}
           |    |      -> ()
           |    -> <flexget.task.Task object at 0x06DC10E8>
           -> <function Task.execute at 0x048803D0>

  File "C:\Users\tuant\PycharmProjects\Flexget\flexget\task.py", line 710, in execute
    self._execute()
    |    -> <function Task._execute at 0x04880388>
    -> <flexget.task.Task object at 0x06DC10E8>

  File "C:\Users\tuant\PycharmProjects\Flexget\flexget\task.py", line 676, in _execute
    self.__run_task_phase(phase)
    |                     -> 'input'
    -> <flexget.task.Task object at 0x06DC10E8>

  File "C:\Users\tuant\PycharmProjects\Flexget\flexget\task.py", line 502, in __run_task_phase
    response = self.__run_plugin(plugin, phase, args)
               |                 |       |      -> (<flexget.task.Task object at 0x06DC10E8>, {'user_id': 'urXXXXXXXX', 'list': 'ratings', 'force_language': 'en-us', 'type': ['...
               |                 |       -> 'input'
               |                 -> <PluginInfo(name=imdb_watchlist)>
               -> <flexget.task.Task object at 0x06DC10E8>

> File "C:\Users\tuant\PycharmProjects\Flexget\flexget\task.py", line 535, in __run_plugin
    result = method(*args, **kwargs)
             |       |       -> {}
             |       -> (<flexget.task.Task object at 0x06DC10E8>, {'user_id': 'urXXXXXXXX', 'list': 'ratings', 'force_language': 'en-us', 'type': ['...
             -> <Event(name=plugin.imdb_watchlist.input,func=wrapped_func,priority=128)>

  File "C:\Users\tuant\PycharmProjects\Flexget\flexget\event.py", line 22, in __call__
    return self.func(*args, **kwargs)
           |    |     |       -> {}
           |    |     -> (<flexget.task.Task object at 0x06DC10E8>, {'user_id': 'urXXXXXXXX', 'list': 'ratings', 'force_language': 'en-us', 'type': ['...
           |    -> <bound method cached.__call__.<locals>.wrapped_func of <flexget.components.imdb.imdb_watchlist.ImdbWatchlist object at 0x06B5...
           -> <Event(name=plugin.imdb_watchlist.input,func=wrapped_func,priority=128)>

  File "C:\Users\tuant\PycharmProjects\Flexget\flexget\utils\cached_input.py", line 166, in wrapped_func
    cache = IterableCache(response, self.store_to_db if self.persist else None)
            |             |         |    |              |    -> datetime.timedelta(seconds=7200)
            |             |         |    |              -> <flexget.utils.cached_input.cached object at 0x0632D358>
            |             |         |    -> <function cached.store_to_db at 0x05687A48>
            |             |         -> <flexget.utils.cached_input.cached object at 0x0632D358>
            |             -> None
            -> <class 'flexget.utils.cached_input.IterableCache'>

  File "C:\Users\tuant\PycharmProjects\Flexget\flexget\utils\cached_input.py", line 213, in __init__
    self.iterable = iter(iterable)
    |                    -> None
    -> <flexget.utils.cached_input.IterableCache object at 0x072409D0>

TypeError: 'NoneType' object is not iterable
2020-08-04 16:58:10 CRITICAL manager       test-imdb_watchlist An unexpected crash has occurred. Writing crash report to C:\Users\tuant\PycharmProjects\Flexget\crash_report.2020.08.04.165810796082.log. Please verify you are running the latest version of flexget by using "flexget -V" from CLI or by using version_checker plugin at http://flexget.com/wiki/Plugins/version_checker. You are currently using version 3.1.68.dev
2020-08-04 16:58:10 DEBUG    manager       test-imdb_watchlist Traceback:
Traceback (most recent call last):

  File "C:\Users\tuant\AppData\Local\Programs\Python\Python38-32\lib\threading.py", line 890, in _bootstrap
    self._bootstrap_inner()
    |    -> <function Thread._bootstrap_inner at 0x031C1580>
    -> <Thread(task_queue, started daemon 13740)>

  File "C:\Users\tuant\AppData\Local\Programs\Python\Python38-32\lib\threading.py", line 932, in _bootstrap_inner
    self.run()
    |    -> <function Thread.run at 0x031C1418>
    -> <Thread(task_queue, started daemon 13740)>

  File "C:\Users\tuant\AppData\Local\Programs\Python\Python38-32\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
    |    |        |    |        |    -> {}
    |    |        |    |        -> <Thread(task_queue, started daemon 13740)>
    |    |        |    -> ()
    |    |        -> <Thread(task_queue, started daemon 13740)>
    |    -> <bound method TaskQueue.run of <flexget.task_queue.TaskQueue object at 0x06B7ABC8>>
    -> <Thread(task_queue, started daemon 13740)>

  File "C:\Users\tuant\PycharmProjects\Flexget\flexget\task_queue.py", line 46, in run
    self.current_task.execute()
    |    |            -> <function Task.execute at 0x04880418>
    |    -> <flexget.task.Task object at 0x06DC10E8>
    -> <flexget.task_queue.TaskQueue object at 0x06B7ABC8>

  File "C:\Users\tuant\PycharmProjects\Flexget\flexget\task.py", line 80, in wrapper
    return func(self, *args, **kw)
           |    |      |       -> {}
           |    |      -> ()
           |    -> <flexget.task.Task object at 0x06DC10E8>
           -> <function Task.execute at 0x048803D0>

  File "C:\Users\tuant\PycharmProjects\Flexget\flexget\task.py", line 710, in execute
    self._execute()
    |    -> <function Task._execute at 0x04880388>
    -> <flexget.task.Task object at 0x06DC10E8>

  File "C:\Users\tuant\PycharmProjects\Flexget\flexget\task.py", line 676, in _execute
    self.__run_task_phase(phase)
    |                     -> 'input'
    -> <flexget.task.Task object at 0x06DC10E8>

  File "C:\Users\tuant\PycharmProjects\Flexget\flexget\task.py", line 502, in __run_task_phase
    response = self.__run_plugin(plugin, phase, args)
               |                 |       |      -> (<flexget.task.Task object at 0x06DC10E8>, {'user_id': 'urXXXXXXXX', 'list': 'ratings', 'force_language': 'en-us', 'type': ['...
               |                 |       -> 'input'
               |                 -> <PluginInfo(name=imdb_watchlist)>
               -> <flexget.task.Task object at 0x06DC10E8>

> File "C:\Users\tuant\PycharmProjects\Flexget\flexget\task.py", line 535, in __run_plugin
    result = method(*args, **kwargs)
             |       |       -> {}
             |       -> (<flexget.task.Task object at 0x06DC10E8>, {'user_id': 'urXXXXXXXX', 'list': 'ratings', 'force_language': 'en-us', 'type': ['...
             -> <Event(name=plugin.imdb_watchlist.input,func=wrapped_func,priority=128)>

  File "C:\Users\tuant\PycharmProjects\Flexget\flexget\event.py", line 22, in __call__
    return self.func(*args, **kwargs)
           |    |     |       -> {}
           |    |     -> (<flexget.task.Task object at 0x06DC10E8>, {'user_id': 'urXXXXXXXX', 'list': 'ratings', 'force_language': 'en-us', 'type': ['...
           |    -> <bound method cached.__call__.<locals>.wrapped_func of <flexget.components.imdb.imdb_watchlist.ImdbWatchlist object at 0x06B5...
           -> <Event(name=plugin.imdb_watchlist.input,func=wrapped_func,priority=128)>

  File "C:\Users\tuant\PycharmProjects\Flexget\flexget\utils\cached_input.py", line 166, in wrapped_func
    cache = IterableCache(response, self.store_to_db if self.persist else None)
            |             |         |    |              |    -> datetime.timedelta(seconds=7200)
            |             |         |    |              -> <flexget.utils.cached_input.cached object at 0x0632D358>
            |             |         |    -> <function cached.store_to_db at 0x05687A48>
            |             |         -> <flexget.utils.cached_input.cached object at 0x0632D358>
            |             -> None
            -> <class 'flexget.utils.cached_input.IterableCache'>

  File "C:\Users\tuant\PycharmProjects\Flexget\flexget\utils\cached_input.py", line 213, in __init__
    self.iterable = iter(iterable)
    |                    -> None
    -> <flexget.utils.cached_input.IterableCache object at 0x072409D0>

TypeError: 'NoneType' object is not iterable
2020-08-04 16:58:10 WARNING  task          test-imdb_watchlist Aborting task (plugin: imdb_watchlist)
2020-08-04 16:58:10 DEBUG    task_queue                    task test-imdb_watchlist aborted: TaskAbort(reason=BUG: Unhandled error in plugin imdb_watchlist: 'NoneType' object is not iterable, silent=False)
2020-08-04 16:58:11 DEBUG    task_queue                    task queue shut down
2020-08-04 16:58:11 INFO     ipc.rpyc                      listener closed
2020-08-04 16:58:11 DEBUG    util.simple_persistence                 Flushing simple persistence for task None to db.
2020-08-04 16:58:11 INFO     ipc.rpyc                      server has terminated
2020-08-04 16:58:11 DEBUG    manager                       Shutting down
2020-08-04 16:58:11 DEBUG    manager                       Removed C:\Users\tuant\PycharmProjects\Flexget\.test-imdb_watchlist-lock

Process finished with exit code 0

```




Discussion:

Resolving patch:
diff --git a/flexget/plugins/plugin_transmission.py b/flexget/plugins/plugin_transmission.py
--- a/flexget/plugins/plugin_transmission.py
+++ b/flexget/plugins/plugin_transmission.py
@@ -358,7 +358,10 @@ def _make_torrent_options_dict(self, config, entry):
         if 'include_subs' in opt_dic:
             post['include_subs'] = opt_dic['include_subs']
         if 'content_filename' in opt_dic:
-            post['content_filename'] = opt_dic['content_filename']
+            try:
+                post['content_filename'] = entry.render(opt_dic['content_filename'])
+            except RenderError as e:
+                log.error('Unable to render content_filename %s: %s' % (entry['title'], e))
         if 'skip_files' in opt_dic:
             post['skip_files'] = opt_dic['skip_files']
         if 'include_files' in opt_dic:


Test patch: