Show only a single notification per run

This combines all errors onto a single notification per run to avoid polluting the screen with multiple notifications.

The number of errors that are shown on the notification is still controllable via the `max_error_notify` option, which
defaults to 3.