Repository: Flexget/Flexget

Issue:
Series' from_group filter merging instead of overwriting values
<!---
Before opening an issue, verify:

- Is this a feature request? Post it on https://feathub.com/Flexget/Flexget
- Did you recently upgrade? Look at the Change Log and Upgrade Actions to make sure that you don't need to make any changes to your config https://flexget.com/ChangeLog https://flexget.com/UpgradeActions
- Are you running FlexGet as a daemon? Stop it completely and then start it again https://flexget.com/CLI/daemon
- Did you search to see if the issue already exists? https://github.com/Flexget/Flexget/issues
- Did you fill out the issue template as completely as possible?

The issue template is here because it helps to ensure you submitted all the necessary information the first time, and allows us to more quickly review issues. Please fill it out correctly and do not ignore it, no matter how irrelevant you think it may be. Thanks in advance for your help with this!
--->

### Expected behaviour:
Setting a `from_group` filter under a grouped series entry to use only that value.
<!---
Please don't just say "it doesn't crash" or "it works". Explain what the expected result is.
--->

### Actual behaviour:
`from_group` tries to merge the [Settings Group](https://flexget.com/Plugins/series/per_group_settings) value with the specific show value.
If both the settings group's and shows' `from_group` are:
- Lists: the values are merged, meaning we can't exclude a release group defined in the settings group (_Show 2_ in the example below will match both _Group A_ and _Group B_. One would expect to only match _Group B_)
- Mixed: If the `from_group` values defined in the show and the setting group have different formats (_Show 3_ in the example below), Flexget crashes (see Log below)
- Single string: Works as intended, as in the specific value overrides the setting group's. Had I defined _Group B_ under _myshows_, _Show 1_ would match _Group B_, _Show 3_ would match _Group A_ as specifically defined (assuming _Show 2_ hadn't mixed formats to cause a crash as mentioned above)
### Steps to reproduce:
- Step 1: Have a series settings group
- Step 2: Define `from_group` in the settings
- Step 3: Assign a series to the settings group
- Step 4: refine the series' setting with its own `from_group`
#### Config:
```yaml
mytask:
  series:
    settings:
      myshows:
        from_group:
          - Group A
          - Group B
    myshows:
      - Show 1
      - Show 2:
          from_group:
            - Group B
      - Show 3:
          from_group: Group A
```
  
#### Log:
<details>
 <summary>(click to expand)</summary>

```
2019-10-12 14:13 CRITICAL task          mytask BUG: Unhandled error in plugin configure_series: u"Merging key from_group failed, conflicting datatypes 'list' vs. 'unicode'."
Traceback (most recent call last):
  File "c:\python27\lib\site-packages\flexget\task.py", line 522, in __run_plugin
    return method(*args, **kwargs)
  File "c:\python27\lib\site-packages\flexget\event.py", line 23, in __call__
    return self.func(*args, **kwargs)
  File "c:\python27\lib\site-packages\flexget\components\series\configure_series.py", line 93, in on_task_prepare
    self.merge_config(task, series_config)
  File "c:\python27\lib\site-packages\flexget\components\series\series.py", line 327, in merge_config
    native_series = self.prepare_config(task.config.get('series', {}))
  File "c:\python27\lib\site-packages\flexget\components\series\series.py", line 294, in prepare_config
    config = self.apply_group_options(config)
  File "c:\python27\lib\site-packages\flexget\components\series\series.py", line 268, in apply_group_options
    merge_dict_from_to(group_settings, series_settings)
  File "c:\python27\lib\site-packages\flexget\utils\tools.py", line 181, in merge_dict_from_to
    % (k, type(v).__name__, type(d2[k]).__name__)
MergeException: u"Merging key from_group failed, conflicting datatypes 'list' vs. 'unicode'."
```
</details>

### Additional information:

- FlexGet version: 2.21.25
- Python version: 2.7.16
- Installation method: pip
- Using daemon (yes/no): yes
- OS and version: W10
- Link to crash log:

<!---
In config and debug/crash logs, remember to redact any personal or sensitive information such as passwords, API keys, private URLs and so on.

Please verify that the following data is present before submitting your issue:

- Link to a paste service or paste above the relevant config (preferably full config, including templates if present). Please make sure the paste does not expire, if possible.
- Link to a paste service or paste above debug-level logs of the relevant task/s (use `flexget -L debug execute --tasks <Task_name>`).
- FlexGet version (use `flexget -V` to get it).
- Full Python version, for example `2.7.11` (use `python -V` to get it). 
- Installation method (pip, git install, etc).
- Whether or not you're running FlexGet as a daemon.
- OS and version.
- Attach crash log if one was generated, in addition to the debug-level log. It can be found in the directory with your config file.
--->



Discussion:
## Example 1 👍 
Will overwrite the setting, accepting only GroupC.
The intended behavior as described in the wiki
> Series may override any settings specified in group settings.
### Config:
```yaml
  mytask:
    mock:
      - title: '[GroupA] Cool Show - 01 [1080p]'
      - title: '[GroupB] Cool Show - 01 [720p]'
      - title: '[GroupC] Cool Show - 01 [360p]'
    series:
      settings:
        my_shows:
          from_group: GroupB
      my_shows:
        - Cool Show:
            qualities: ['1080p', '720p', '360p']
            from_group: GroupC
```

### Log:
```
VERBOSE  details       mytask          Produced 3 entries.                                                               
VERBOSE  task          mytask          ACCEPTED: `[GroupC] Cool Show - 01 [360p]` by series plugin because quality wanted
VERBOSE  details       mytask          Summary - Accepted: 1 (Rejected: 0 Undecided: 2 Failed: 0)                        
WARNING  task          mytask          Task doesn't have any output plugins, you should add (at least) one!              
```

## Example 2 👎 
In list format, `from_group` settings are merged instead of overwritten.
### Config:
```diff
  mytask:
    mock:
      - title: '[GroupA] Cool Show - 01 [1080p]'
      - title: '[GroupB] Cool Show - 01 [720p]'
      - title: '[GroupC] Cool Show - 01 [360p]'
    series:
      settings:
        my_shows:
          identified_by: sequence
-          from_group: GroupB
+          from_group: 
+            - GroupB
      my_shows:
        - Cool Show:
            qualities: ['1080p', '720p', '360p']
-           from_group: GroupC
+           from_group:
+             - GroupC
```
### Log:
```
VERBOSE  details       mytask          Produced 3 entries.                                                               
VERBOSE  task          mytask          ACCEPTED: `[GroupB] Cool Show - 01 [720p]` by series plugin because quality wanted
VERBOSE  task          mytask          ACCEPTED: `[GroupC] Cool Show - 01 [360p]` by series plugin because quality wanted
VERBOSE  details       mytask          Summary - Accepted: 2 (Rejected: 0 Undecided: 1 Failed: 0)                        
```

## Example 3 ⚠️
Flexget crashes with mixed and matched value types.
## Config:
```diff
  mytask:
    mock:
      - title: '[GroupA] Cool Show - 01 [1080p]'
      - title: '[GroupB] Cool Show - 01 [720p]'
      - title: '[GroupC] Cool Show - 01 [360p]'
    series:
      settings:
        my_shows:
          identified_by: sequence
-         from_group: GroupB
+         from_group:
+           - GroupB
      my_shows:
        - Cool Show:
            qualities: ['1080p', '720p', '360p']
            from_group: GroupC
```
### Log:
```
CRITICAL task          mytask          BUG: Unhandled error in plugin series_db: u"Merging key from_group failed, conflicting datatypes 'list' vs. 'unicode'."
CRITICAL manager       mytask          An unexpected crash has occurred. Writing crash report to crash_report.2019.10.25.140923473000.log. Please verify you are running the latest version of flexget by using "flexget -V" from CLI or by using version_checker plugin at http://flexget.com/wiki/Plugins/version_checker. You are currently using version 2.21.30
WARNING  task          mytask          Aborting task (plugin: series_db)
```
### Debug:
```
CRITICAL task          mytask          BUG: Unhandled error in plugin series_db: u"Merging key from_group failed, conflicting datatypes 'list' vs. 'unicode'."
Traceback (most recent call last):
  File "~\flexget\.install\lib\site-packages\flexget\task.py", line 522, in __run_plugin
    return method(*args, **kwargs)
  File "~\flexget\.install\lib\site-packages\flexget\event.py", line 23, in __call__
    return self.func(*args, **kwargs)
  File "~\flexget\.install\lib\site-packages\flexget\components\series\series.py", line 1099, in on_task_start
    config = self.prepare_config(task.config['series'])
  File "~\flexget\.install\lib\site-packages\flexget\components\series\series.py", line 294, in prepare_config
    config = self.apply_group_options(config)
  File "~\flexget\.install\lib\site-packages\flexget\components\series\series.py", line 268, in apply_group_options
    merge_dict_from_to(group_settings, series_settings)
  File "~\flexget\.install\lib\site-packages\flexget\utils\tools.py", line 181, in merge_dict_from_to
    % (k, type(v).__name__, type(d2[k]).__name__)
MergeException: u"Merging key from_group failed, conflicting datatypes 'list' vs. 'unicode'."
```
Adding `list` to these tuples solves the use-case above.
https://github.com/Flexget/Flexget/blob/ea11cdccce95c6513d8be9978bdd9d0baeb88b7b/flexget/utils/tools.py#L173-L174

Any immediate thoughts on what kind of issues this could incur on other plugins?

Resolving patch:
diff --git a/flexget/utils/tools.py b/flexget/utils/tools.py
--- a/flexget/utils/tools.py
+++ b/flexget/utils/tools.py
@@ -170,8 +170,8 @@ def merge_dict_from_to(d1, d2):
                     raise Exception(
                         'Unknown type: %s value: %s in dictionary' % (type(v), repr(v))
                     )
-            elif isinstance(v, (str, bool, int, float, type(None))) and isinstance(
-                d2[k], (str, bool, int, float, type(None))
+            elif isinstance(v, (str, bool, int, float, list, type(None))) and isinstance(
+                d2[k], (str, bool, int, float, list, type(None))
             ):
                 # Allow overriding of non-container types with other non-container types
                 pass


Test patch:
diff --git a/flexget/tests/test_series.py b/flexget/tests/test_series.py
--- a/flexget/tests/test_series.py
+++ b/flexget/tests/test_series.py
@@ -1548,6 +1548,19 @@ class TestFromGroup(object):
               - {title: 'Test :: h264 10-bit | Softsubs (Ignore) | Episode 3'}
             series:
               - test: {from_group: [Name, FlexGet]}
+          test_merge:
+            mock:
+              - {title: 'Test.15.HDTV-Ignored'}
+              - {title: 'Test.15.HDTV-FlexGet'}
+              - {title: 'TestMerge.13.HDTV-Ignored'}
+              - {title: 'TestMerge.13.HDTV-FlexGet'}
+              - {title: 'TestMerge.14.HDTV-Name'}
+            series:
+              settings:
+                testgroup: {from_group: [Name]}
+              testgroup:
+                - test: {from_group: FlexGet}
+                - testmerge: {from_group: [FlexGet]}
     """
 
     def test_from_group(self, execute_task):
@@ -1559,6 +1572,11 @@ def test_from_group(self, execute_task):
         assert task.find_entry(
             'accepted', title='Test :: h264 10-bit | Softsubs (FlexGet) | Episode 3'
         )
+        # Test interaction with setting group. Merge if both are lists, override if different types
+        task = execute_task('test_merge')
+        assert task.find_entry('accepted', title='Test.15.HDTV-FlexGet')
+        assert task.find_entry('accepted', title='TestMerge.13.HDTV-FlexGet')
+        assert task.find_entry('accepted', title='TestMerge.14.HDTV-Name')
 
 
 class TestBegin(object):
diff --git a/flexget/tests/test_utils.py b/flexget/tests/test_utils.py
--- a/flexget/tests/test_utils.py
+++ b/flexget/tests/test_utils.py
@@ -7,7 +7,7 @@
 import pytest
 
 from flexget.utils import json
-from flexget.utils.tools import parse_filesize, split_title_year
+from flexget.utils.tools import parse_filesize, split_title_year, merge_dict_from_to
 
 
 def compare_floats(float1, float2):
@@ -109,3 +109,52 @@ class TestSplitYearTitle(object):
     )
     def test_split_year_title(self, title, expected_title, expected_year):
         assert split_title_year(title) == (expected_title, expected_year)
+
+class TestDictMerge(object):
+    def test_merge_dict_to_dict_list(self):
+        d1 = {'setting': {'parameter': ['item_1']}}
+        d2 = {'setting': {'parameter': ['item_2']}}
+        merge_dict_from_to(d1, d2)
+        assert d2 == {'setting': {'parameter': ['item_2', 'item_1']}}
+
+    def test_merge_dict_to_dict_override(self):
+        d1 = {'setting': {'parameter': ['item_1']}}
+        d2 = {'setting': {'parameter': 2}}
+        merge_dict_from_to(d1, d2)
+        assert d2 == {'setting': {'parameter': 2}}
+
+    def test_merge_dict_to_dict_add(self):
+        d1 = {'setting': {'parameter_1': ['item_1']}}
+        d2 = {'setting': {'parameter_2': 'item_2'}}
+        merge_dict_from_to(d1, d2)
+        assert d2 == {'setting': {'parameter_1': ['item_1'], 'parameter_2': 'item_2'}}
+
+    def test_merge_dict_to_dict_str(self):
+        d1 = {'setting': {'parameter': 'item_1'}}
+        d2 = {'setting': {'parameter': 'item_2'}}
+        merge_dict_from_to(d1, d2)
+        assert d2 == {'setting': {'parameter': 'item_2'}}
+
+    def test_merge_list_to_list(self):
+        d1 = {'setting': ['item_1']}
+        d2 = {'setting': ['item_2']}
+        merge_dict_from_to(d1, d2)
+        assert d2 == {'setting': ['item_2', 'item_1']}
+
+    def test_merge_list_to_str(self):
+        d1 = {'setting': ['list']}
+        d2 = {'setting': 'string'}
+        merge_dict_from_to(d1, d2)
+        assert d2 == {'setting': 'string'}
+
+    def test_merge_str_to_list(self):
+        d1 = {'setting': 'string'}
+        d2 = {'setting': ['list']}
+        merge_dict_from_to(d1, d2)
+        assert d2 == {'setting': ['list']}
+
+    def test_merge_str_to_str(self):
+        d1 = {'setting': 'string_1'}
+        d2 = {'setting': 'string_2'}
+        merge_dict_from_to(d1, d2)
+        assert d2 == {'setting': 'string_2'}