Repository: ansible/ansible
Pull request: Register handlers immediately if currently iterating handlers
##### SUMMARY
This fixes the issue where handlers notifying other handlers are not properly run because the notification is not registered unless another flush_handlers occurs. Instead, if the current host state is iterating handlers we immediately register the handler to be run so the notification is not lost.

Fixes #80880

##### ISSUE TYPE
Bugfix Pull Request

##### COMPONENT NAME
Executor

##### ADDITIONAL INFORMATION
```
$ ansible-playbook test.yml
PLAY [Testing] *****************************************************************

TASK [Trigger handlers] ********************************************************
changed: [localhost] => {
    "msg": "Task 1"
}

RUNNING HANDLER [Handler 1] ****************************************************
changed: [localhost] => {
    "msg": "Handler 1"
}

RUNNING HANDLER [Handler 2] ****************************************************
changed: [localhost] => {
    "msg": "Handler 2"
}

RUNNING HANDLER [Handler 3] ****************************************************
ok: [localhost] => {
    "msg": "Handler 3"
}

PLAY RECAP *********************************************************************
localhost                  : ok=4    changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
```

Problem statement:
Problem with handlers notifying another handlers
### Summary

The ansible-core 2.15 ignores handlers notified by another handlers when are more than one  case.

With the ansible-core 2.14, all cases of handlers notified by another handlers are runned.

### Issue Type

Bug Report

### Component Name

notify

### Ansible Version

```console
ansible [core 2.15.0]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/user/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.11/site-packages/ansible
  ansible collection location = /home/user/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.11.3 (main, Apr  5 2023, 15:52:25) [GCC 12.2.1 20230201] (/usr/bin/python)
  jinja version = 3.1.2
  libyaml = True
```


### Configuration

```console
CONFIG_FILE() = /etc/ansible/ansible.cfg
EDITOR(env: EDITOR) = vim
PAGER(env: PAGER) = less
```


### OS / Environment

Archlinux

ansible 7.6.0-1
ansible-core 2.15.0-1
ansible-lint 6.15.0.r45.g2fca3fe-2
python-ansible-compat 4.0.2-1

### Steps to Reproduce

<!--- Paste example playbooks or commands between quotes below -->
```yaml (paste below)
---
- name: Testing
  hosts: localhost
  tasks:
    - name: Trigger handlers
      debug:
        msg: Task 1
      changed_when: true
      notify: Handler 1
  handlers:
    - name: Handler 1
      debug:
        msg: Handler 1
      changed_when: true
      notify: Handler 2
    - name: Handler 2
      debug:
        msg: Handler 2
      changed_when: true
      notify: Handler 3
    - name: Handler 3
      debug:
        msg: Handler 3
```


### Expected Results

All handlers must be notified.

```console
PLAY [Testing] ********************************************************************************************************

TASK [Gathering Facts] ********************************************************************************************************
ok: [localhost]

TASK [Trigger handlers] ********************************************************************************************************
changed: [localhost] => {
    "msg": "Task 1"
}

RUNNING HANDLER [Handler 1] ********************************************************************************************************
changed: [localhost] => {
    "msg": "Handler 1"
}

RUNNING HANDLER [Handler 2] ********************************************************************************************************
changed: [localhost] => {
    "msg": "Handler 2"
}

RUNNING HANDLER [Handler 3] ********************************************************************************************************
changed: [localhost] => {
    "msg": "Handler 3"
}

PLAY RECAP ********************************************************************************************************
localhost                  : ok=5    changed=4    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
```

### Actual Results

Only the two first handlers were notified.

```console
PLAY [Testing] *************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************
ok: [localhost]

TASK [Trigger handlers] *************************************************************************************************
changed: [localhost] => {
    "msg": "Task 1"
}

RUNNING HANDLER [Handler 1] *************************************************************************************************
changed: [localhost] => {
    "msg": "Handler 1"
}

RUNNING HANDLER [Handler 2] *************************************************************************************************
changed: [localhost] => {
    "msg": "Handler 2"
}

PLAY RECAP *************************************************************************************************
localhost                  : ok=4    changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
```


### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct

Hints from issue discussion:
Files identified in the description:
None

If these files are incorrect, please update the `component name` section of the description or use the `!component` bot command.

[click here for bot help](https://github.com/ansible/ansibullbot/blob/devel/ISSUE_HELP.md)
<!--- boilerplate: components_banner --->
I can verify that the 3rd handler does not execute , 2.14.6 works fine.

Commit message: Adding test case for 80880

Test case is based on reproducer given in #80880, with some minor
adjustments and assertions.

Patch to review:
diff --git a/lib/ansible/plugins/strategy/__init__.py b/lib/ansible/plugins/strategy/__init__.py
index 9cd1acb4b55aa7..d94aa1a16e6bf7 100644
--- a/lib/ansible/plugins/strategy/__init__.py
+++ b/lib/ansible/plugins/strategy/__init__.py
@@ -653,19 +653,27 @@ def _process_pending_results(self, iterator, one_pass=False, max_passes=None):
                         # handlers are actually flushed so the last defined handlers are exexcuted,
                         # otherwise depending on the setting either error or warn
                         for notification in result_item['_ansible_notify']:
-                            if any(self.search_handlers_by_notification(notification, iterator)):
-                                iterator.add_notification(original_host.name, notification)
-                                display.vv(f"Notification for handler {notification} has been saved.")
-                                continue
-
-                            msg = (
-                                f"The requested handler '{notification}' was not found in either the main handlers"
-                                " list nor in the listening handlers list"
-                            )
-                            if C.ERROR_ON_MISSING_HANDLER:
-                                raise AnsibleError(msg)
+                            for handler in self.search_handlers_by_notification(notification, iterator):
+                                host_state = iterator.get_state_for_host(original_host.name)
+                                if host_state.run_state == IteratingStates.HANDLERS:
+                                    # we're currently iterating handlers, so we need to expand this now
+                                    if handler.notify_host(original_host):
+                                        # NOTE even with notifications deduplicated this can still happen in case of handlers being
+                                        # notified multiple times using different names, like role name or fqcn
+                                        self._tqm.send_callback('v2_playbook_on_notify', handler, original_host.name)
+                                else:
+                                    iterator.add_notification(original_host.name, notification)
+                                    display.vv(f"Notification for handler {notification} has been saved.")
+                                break
                             else:
-                                display.warning(msg)
+                                msg = (
+                                    f"The requested handler '{notification}' was not found in either the main handlers"
+                                    " list nor in the listening handlers list"
+                                )
+                                if C.ERROR_ON_MISSING_HANDLER:
+                                    raise AnsibleError(msg)
+                                else:
+                                    display.warning(msg)
 
                     if 'add_host' in result_item:
                         # this task added a new host (add_host module)
diff --git a/test/integration/targets/handlers/80880.yml b/test/integration/targets/handlers/80880.yml
new file mode 100644
index 00000000000000..d362ea8e38bb2d
--- /dev/null
+++ b/test/integration/targets/handlers/80880.yml
@@ -0,0 +1,34 @@
+---
+- name: Test notification of handlers from other handlers
+  hosts: localhost
+  gather_facts: no
+  handlers:
+    - name: Handler 1
+      debug:
+        msg: Handler 1
+      changed_when: true
+      notify: Handler 2
+      register: handler1_res
+    - name: Handler 2
+      debug:
+        msg: Handler 2
+      changed_when: true
+      notify: Handler 3
+      register: handler2_res
+    - name: Handler 3
+      debug:
+        msg: Handler 3
+      register: handler3_res
+  tasks:
+    - name: Trigger handlers
+      ansible.builtin.debug:
+        msg: Task 1
+      changed_when: true
+      notify: Handler 1
+  post_tasks:
+    - name: Assert results
+      ansible.builtin.assert:
+        that:
+        - "handler1_res is defined and handler1_res is success"
+        - "handler2_res is defined and handler2_res is success"
+        - "handler3_res is defined and handler3_res is success"
diff --git a/test/integration/targets/handlers/runme.sh b/test/integration/targets/handlers/runme.sh
index cc0997a5c2f29e..7cbabec2fd3e7c 100755
--- a/test/integration/targets/handlers/runme.sh
+++ b/test/integration/targets/handlers/runme.sh
@@ -50,6 +50,9 @@ for strategy in linear free; do
   [ "$(ansible-playbook test_force_handlers.yml -i inventory.handlers -v "$@" --tags force_false_in_play --force-handlers \
   | grep -E -o CALLED_HANDLER_. | sort | uniq | xargs)" = "CALLED_HANDLER_B" ]
 
+  # https://github.com/ansible/ansible/pull/80898
+  [ "$(ansible-playbook 80880.yml -i inventory.handlers -vv "$@" 2>&1)" ]
+
   unset ANSIBLE_STRATEGY
 
 done


Code review comments:
diff_hunk: @@ -653,19 +653,27 @@ def _process_pending_results(self, iterator, one_pass=False, max_passes=None):
                         # handlers are actually flushed so the last defined handlers are exexcuted,
                         # otherwise depending on the setting either error or warn
                         for notification in result_item['_ansible_notify']:
-                            if any(self.search_handlers_by_notification(notification, iterator)):
-                                iterator.add_notification(original_host.name, notification)
-                                display.vv(f"Notification for handler {notification} has been saved.")
-                                continue
-
-                            msg = (
-                                f"The requested handler '{notification}' was not found in either the main handlers"
-                                " list nor in the listening handlers list"
-                            )
-                            if C.ERROR_ON_MISSING_HANDLER:
-                                raise AnsibleError(msg)
+                            for handler in self.search_handlers_by_notification(notification, iterator):
+                                host_state = iterator.get_state_for_host(original_host.name)
path: lib/ansible/plugins/strategy/__init__.py
text: @user1:
I think this can be moved before the loop.