Bugfix (Reported by flak): fix possible notification spam if retention is used