Made sure the oldest notification closes after the given time