Only trigger alert on benign canary exception if it keeps happening. Ref T50231