Fix issue with handler notifications being sent out incorrectly

This is a partial revert of e8ad36c, which introduced the bug.

Fixes #5848