obfuscurity/nagios-scripts — pull request #9 by justintime (2014-01-15 20:56:46+00:00)
Title: check_graphite error handling

If a 500 error is returned from graphite, the current script will throw an unhandled exception, resulting in Nagios going CRITICAL with a description of "(null)".

This patch will catch that exception, and will also exit CRITICAL, but will exit with a string of "CRITICAL 500 error from Graphite" to aid the admin in tracking the issue.

The patch also adds a --zero-on-error option (also -z) that will treat a 500 error as a metric value of 0, and continue on.

The default behavior is to exit CRITICAL unless -z is specified.

Our use case for this is where collectd is sending swap metrics to graphite. We are migrating to systems that do not have any swap, so for new hosts, there are no swap metrics. When graphite sends a 500 error because that metric doesn't exist, we don't want that to be a CRITICAL state.