Fix single and multi notifications not removing each other