Rescue errors triggering job notifications when writing logs

Depending on the scenario this could cause a lot of exception
back traces to be logged, clogging our logs. To keep track of the
errors it adds a metric.