Add config for notification emails

We want to send an email for every error, not 1, 10, 100