AlertRunner: Add execption handling on individual alert level