Repository: Flexget/Flexget

Issue:
Can’t send email notification with full statistics including rejected and undecided entries (not only accepted/failed ones)
### Expected behaviour:
notification with accepted|rejected|undecided|failed entries is sent

### Actual behaviour:
notification with only accepted and failed entries is sent

### Steps to reproduce:
- Step 1: add to tasks "digest" plugin with "accepted|rejected|undecided|failed" options
- Step 2: add "emailreport" task with "notify" and "from_digest" plugins and "restore_state: yes" option
- Step 3: execute tasks from step 1

#### Config example:
```
schedules:
  - tasks: 'task1'
    interval:
      hours: 2
  - tasks: 'task2'
    interval:
      hours: 4
  - tasks: 'emailreport'
    schedule:
      hour: 21
...
templates:
  ...
  video:
    digest:
        list: email_list
        state:
          - accepted
          - rejected
          - undecided
          - failed
...
tasks:
  task1:
    template:
      - video
...
  task2:
    template:
      - video
...
  emailreport:
    no_entries_ok: yes
    from_digest:
      list: email_list
      restore_state: yes
    seen: local

    notify:
      task:
        title: "[FLEXGET] Daily report"
        template: custom_html.template
        via:
          - email:
              from: '{? email.email_from ?}'
              to:
                - '{? email.email_to1 ?}'
                - '{? email.email_to2 ?}'
              smtp_host: smtp.gmail.com
              smtp_port: 587
              smtp_username: '{? email.smtp_username ?}'
              smtp_password: '{? email.smtp_password ?}'
              smtp_tls: yes
              html: yes
```
  
#### Log:
```
2017-01-26 21:00 VERBOSE  details       emailreport     Produced 586 entries.
2017-01-26 21:00 VERBOSE  manipulate    emailreport     Modified 0 entries.
2017-01-26 21:00 VERBOSE  details       emailreport     Summary - Accepted: 0 (Rejected: 1070 Undecided: 586 Failed: 0)
2017-01-26 21:00 WARNING  task          emailreport     Task doesn't have any output plugins, you should add (at least) one!
2017-01-26 21:00 VERBOSE  notify_entry  emailreport     No accepted or failed entries, not sending a notification.
```

### Additional information:

- Flexget Version: 2.9.8
- Python Version: 2.7.12
- Installation method: pip
- OS and version: Ubuntu 16.04

P.S.
As @liiight proposed maybe "always_send" should be added to the "notify" plugin

Related issue and FR in forum:
https://discuss.flexget.com/t/cant-send-email-notification-with-full-statistics-including-rejected-and-undecided-entries-not-only-accepted-failed-ones/3129
https://discuss.flexget.com/t/add-always-send-option-to-notify-plugin/3130?source_topic_id=3129


Can’t send email notification with full statistics including rejected and undecided entries (not only accepted/failed ones)
### Expected behaviour:
notification with accepted|rejected|undecided|failed entries is sent

### Actual behaviour:
notification with only accepted and failed entries is sent

### Steps to reproduce:
- Step 1: add to tasks "digest" plugin with "accepted|rejected|undecided|failed" options
- Step 2: add "emailreport" task with "notify" and "from_digest" plugins and "restore_state: yes" option
- Step 3: execute tasks from step 1

#### Config example:
```
schedules:
  - tasks: 'task1'
    interval:
      hours: 2
  - tasks: 'task2'
    interval:
      hours: 4
  - tasks: 'emailreport'
    schedule:
      hour: 21
...
templates:
  ...
  video:
    digest:
        list: email_list
        state:
          - accepted
          - rejected
          - undecided
          - failed
...
tasks:
  task1:
    template:
      - video
...
  task2:
    template:
      - video
...
  emailreport:
    no_entries_ok: yes
    from_digest:
      list: email_list
      restore_state: yes
    seen: local

    notify:
      task:
        title: "[FLEXGET] Daily report"
        template: custom_html.template
        via:
          - email:
              from: '{? email.email_from ?}'
              to:
                - '{? email.email_to1 ?}'
                - '{? email.email_to2 ?}'
              smtp_host: smtp.gmail.com
              smtp_port: 587
              smtp_username: '{? email.smtp_username ?}'
              smtp_password: '{? email.smtp_password ?}'
              smtp_tls: yes
              html: yes
```
  
#### Log:
```
2017-01-26 21:00 VERBOSE  details       emailreport     Produced 586 entries.
2017-01-26 21:00 VERBOSE  manipulate    emailreport     Modified 0 entries.
2017-01-26 21:00 VERBOSE  details       emailreport     Summary - Accepted: 0 (Rejected: 1070 Undecided: 586 Failed: 0)
2017-01-26 21:00 WARNING  task          emailreport     Task doesn't have any output plugins, you should add (at least) one!
2017-01-26 21:00 VERBOSE  notify_entry  emailreport     No accepted or failed entries, not sending a notification.
```

### Additional information:

- Flexget Version: 2.9.8
- Python Version: 2.7.12
- Installation method: pip
- OS and version: Ubuntu 16.04

P.S.
As @liiight proposed maybe "always_send" should be added to the "notify" plugin

Related issue and FR in forum:
https://discuss.flexget.com/t/cant-send-email-notification-with-full-statistics-including-rejected-and-undecided-entries-not-only-accepted-failed-ones/3129
https://discuss.flexget.com/t/add-always-send-option-to-notify-plugin/3130?source_topic_id=3129




Discussion:
The concern I have is making sure the template lines up with when the plugin sends. If we add an always_send option to task notifications, but the configured template doesn't report on rejected or failed entries, it will start sending out mostly blank emails. I'm not sure the best solution.
I think this is solvable via some jinja if condition, shouldn't be a problem imo
Yes, in my custom_html.template I have something like this:

```
{% if task.accepted %}
...
{% endif %}
{% if task.rejected %}
...
{% endif %}
...
etc.
```
That's not going to stop it from sending you a blank email.
We could go back to the way email plugin did it, and make the template have the logic (and produce a totally blank result to signify nothing should be sent.) I wasn't wild about that, but I guess it sorta makes sense, as the template has to match whatever logic you have for sending notifications anyway.
Nah, that's not so nice 
For me the empty email is not a problem at all. The problem is that I can't send report with anything except accepted and failed reports. The problem is that new notification approach is less configurable and flexible than previous one, imho. If someone like me would like to receive extended report then just put a note in Wiki about possible empty email sending.
Or additionally to the "always_send" option add kind of "except_empty" option with default "true" value :)
That's awful tbh. It needs to be done right.
Sorry guys, maybe I'm not completely understand the problem.
In current implementation notify plugin sends emails about only accepted and failed entries. If none of them exists - it does not send email.

I'm not asking to break this logic, I'm just kindly asking to add 2 more states. 
If none of them (including rejected and undecided) entries exists - do not send email (or any other type of notification)
I think adding a config option like
```
notify:
  when:
    - accepted
    - failed
    - rejected
```
would be better. It at least makes it clearer and the template is free to pull any entries.
> I think adding a config option like
> 
> notify:
>   when:
>     - accepted
>     - failed
>     - rejected
> would be better. It at least makes it clearer and the template is free to pull any entries.

Would that be passed in to the template as well somehow? Or the templates should show all of those and you'll see all of them when the right event fires?
> Or the templates should show all of those and you'll see all of them when the right event fires?

Hmm, on second thought, that's not so good. We don't want to see rejected entries in every notification if it isn't selected. I guess we need some way to pass that selection in to the template.
I think we're over thinking this. Sending a notification per task run is a non common scenario. I'd say we add a statistics templates and mention in the wiki that this can result in empty notifications. 
> I think we're over thinking this. Sending a notification per task run is a non common scenario. I'd say we add a statistics templates and mention in the wiki that this can result in empty notifications.

You mean it would change the trigger events based on selecting one specific template? Or you mean along with adding an option on when to trigger, but certain triggers are only compatible with certain templates..
Maybe we switch the task templates to iterate over the all_entries list, and also pass in some sort of filter function, or list of desired states?
@gazpachoking My idea was just to make it send notifications if there were any entries with the desired state and then pass all entries to the template.
@cvium What do you mean by "all entries"?
I just mean "make all entries available for the template to use (regardless of state)", which I think they already are?
Indeed they are available, the task just gets passed into the context. I guess I just think it would be surprising that there is an option to send notifications on different triggers, e.g. rejected entries, but that the included templates would send notifications about accepted and failed entries only in that case still. I don't really want to add complicated logic in though, maybe you guys are right. If we add it, I think we should discourage use of this new option in the docs, and clearly list the caveats to do with template matching your desired (uncommon) needs.
so we're cool with just adding a flag and a warning in the wiki?
The concern I have is making sure the template lines up with when the plugin sends. If we add an always_send option to task notifications, but the configured template doesn't report on rejected or failed entries, it will start sending out mostly blank emails. I'm not sure the best solution.
I think this is solvable via some jinja if condition, shouldn't be a problem imo
Yes, in my custom_html.template I have something like this:

```
{% if task.accepted %}
...
{% endif %}
{% if task.rejected %}
...
{% endif %}
...
etc.
```
That's not going to stop it from sending you a blank email.
We could go back to the way email plugin did it, and make the template have the logic (and produce a totally blank result to signify nothing should be sent.) I wasn't wild about that, but I guess it sorta makes sense, as the template has to match whatever logic you have for sending notifications anyway.
Nah, that's not so nice 
For me the empty email is not a problem at all. The problem is that I can't send report with anything except accepted and failed reports. The problem is that new notification approach is less configurable and flexible than previous one, imho. If someone like me would like to receive extended report then just put a note in Wiki about possible empty email sending.
Or additionally to the "always_send" option add kind of "except_empty" option with default "true" value :)
That's awful tbh. It needs to be done right.
Sorry guys, maybe I'm not completely understand the problem.
In current implementation notify plugin sends emails about only accepted and failed entries. If none of them exists - it does not send email.

I'm not asking to break this logic, I'm just kindly asking to add 2 more states. 
If none of them (including rejected and undecided) entries exists - do not send email (or any other type of notification)
I think adding a config option like
```
notify:
  when:
    - accepted
    - failed
    - rejected
```
would be better. It at least makes it clearer and the template is free to pull any entries.
> I think adding a config option like
> 
> notify:
>   when:
>     - accepted
>     - failed
>     - rejected
> would be better. It at least makes it clearer and the template is free to pull any entries.

Would that be passed in to the template as well somehow? Or the templates should show all of those and you'll see all of them when the right event fires?
> Or the templates should show all of those and you'll see all of them when the right event fires?

Hmm, on second thought, that's not so good. We don't want to see rejected entries in every notification if it isn't selected. I guess we need some way to pass that selection in to the template.
I think we're over thinking this. Sending a notification per task run is a non common scenario. I'd say we add a statistics templates and mention in the wiki that this can result in empty notifications. 
> I think we're over thinking this. Sending a notification per task run is a non common scenario. I'd say we add a statistics templates and mention in the wiki that this can result in empty notifications.

You mean it would change the trigger events based on selecting one specific template? Or you mean along with adding an option on when to trigger, but certain triggers are only compatible with certain templates..
Maybe we switch the task templates to iterate over the all_entries list, and also pass in some sort of filter function, or list of desired states?
@gazpachoking My idea was just to make it send notifications if there were any entries with the desired state and then pass all entries to the template.
@cvium What do you mean by "all entries"?
I just mean "make all entries available for the template to use (regardless of state)", which I think they already are?
Indeed they are available, the task just gets passed into the context. I guess I just think it would be surprising that there is an option to send notifications on different triggers, e.g. rejected entries, but that the included templates would send notifications about accepted and failed entries only in that case still. I don't really want to add complicated logic in though, maybe you guys are right. If we add it, I think we should discourage use of this new option in the docs, and clearly list the caveats to do with template matching your desired (uncommon) needs.
so we're cool with just adding a flag and a warning in the wiki?

Resolving patch:
diff --git a/flexget/plugins/notifiers/notify.py b/flexget/plugins/notifiers/notify.py
--- a/flexget/plugins/notifiers/notify.py
+++ b/flexget/plugins/notifiers/notify.py
@@ -60,10 +60,15 @@ class Notify(object):
                 'properties': {
                     'title': {
                         'type': 'string',
-                        'default': '[FlexGet] {{task.name}}:'
-                                   '{%if task.failed %} {{task.failed|length}} failed entries.{% endif %}'
-                                   '{% if task.accepted %} {{task.accepted|length}} new entries downloaded.{% endif %}'},
+                        'default': '{% if not task.failed or not task.accepted %} Task {{task.name}} did not'
+                                   ' produce any entries.'
+                                   '{% else %} [FlexGet] {{task.name}}:'
+                                   '{% if task.failed %} {{task.failed|length}} failed entries.{% endif %}'
+                                   '{% if task.accepted %} {{task.accepted|length}} new entries downloaded.{% endif %}'
+                                   '{% endif %}'
+                    },
                     'template': {'type': 'string', 'default': 'default.template'},
+                    'always_send': {'type': 'boolean', 'default': False},
                     'via': VIA_SCHEMA
                 },
                 'required': ['via'],
@@ -122,7 +127,7 @@ def on_task_output(self, task, config):
                 self.send_notification(config['entries']['title'], message, config['entries']['via'],
                                        template_renderer=entry.render)
         if 'task' in config:
-            if not (task.accepted or task.failed):
+            if not (task.accepted or task.failed) and not config['task']['always_send']:
                 log.verbose('No accepted or failed entries, not sending a notification.')
                 return
             try:


Test patch: