Only limit number of notifications we cache