adding metadata.yml
Use 'set -e' in bootstrap helpers.

The salt bootstrapping helpers would touch their 'flag file' regardless
of the actual success/failure of their subcommands. This masked some
pretty ugly failures, as salt would subsequently skip them in later
runs.
Explicitly manage /srv/graphite/storage perms

This has stemmed out of the Vagrant setup i'm doing for the
monitoring system -- previously /srv/graphite/storage would be
automatically created with the right perms. Now we need to account
for it possibly being there before we start - with incorrect ownership.
Merge pull request #20 from ministryofjustice/error_handling_db_seed

Use 'set -e' in bootstrap helpers.
Merge pull request #21 from ministryofjustice/manage_storage_dir

Explicitly manage /srv/graphite/storage perms
a reasonable default strategy
collectd interval = 3 (was 10)
tunning
docs
sync collectd and graphite to 10s
tunning defaults
Prevent null aggregates via xFilesFactor=0.0

As per http://obfuscurity.com/2012/04/Unhelpful-Graphite-Tip-9, roll
up averages provided we have at least one data point.

Generally, null data is due to:

* check not running, and hence *zero* valid values. Should aggregate to
  'null'
* check is running, but is at a longer interval than the default
  retention of the metric.
* the sender of the metric is rather unreliable/lossy/high-latency.

This fix ensures that we only get null aggregate when all data points
are null.
Normalise carbon metrics to our default collectd interval

No reason to be different, and higher resolution on this is
useful.
sync carbon interval
Update default storage schema

I've knocked up a quick 'graphite calculator' -- https://gist.github.com/mikepea/19029ff5b944079c1b10

Based on this, we can afford to have the following policy:

Policy:             10s:10d,5m:400d,24h:20y
Send frequency (s): 10
Number of metrics:  1000
Number of hosts:    20

Total Datapoints per metric: 208900
Per-fragment Datapoints:     {"10s:10d"=>86400, "5m:400d"=>115200, "24h:20y"=>7300}
Size per metric:             2448KiB

Datapoints recd per second: 2000
Total disk space needed:    47813MiB

My reasoning:

* 50GB is not a crazy amount of space to allocate to the storage of
metrics for 20 hosts - roughly the limit of what we expect per graphite
cluster at a maximum.
* 1000 metrics per host is about the order of magnitude for a max number
of metrics per host (we do 100-300 by default with collectd, so plenty
of headroom)
* It is better to ensure that we have no aggregation for the general
use-case (10s:10d) -- looking at the most recent data. Aggregation =>
inaccuracies, and with graphite it's quite easy to accidentally
aggregate your data into an unusable or incorrect value. Storing for 10
days allows us to compare yesterday with 8 days ago (for example),
without any loss of precision.
* We need to aggregate at some point - 5m:400d gives us a decent accuracy
for over a year (and costs about the same in storage as our 10s:10d
policy).
* 24h:20y is really just to store per-day metrics for longer than 400d,
for metrics that are valid in per-day counts. It costs us nothing in
comparison.
Remove commented out ^carbon matcher

We have updated CARBON_METRIC_INTERVAL to match our 10s collectd interval,
so no need for the special case any more.
Include contrib/whisper-auto-resize.py

The main repo for https://github.com/graphite-project/whisper includes
`contrib/whisper-auto-resize.py`, which handles auditing and optionally
resizing any whisper db's found that do not match storage-schemas.conf
and storage-aggregation.conf.

This commit simply ensures it is present on the filesystem.
Add auto-resizing of whisper dbs

Add a wrapper around contrib/whisper-auto-resize.py to check for the
need to resize our whisper databases -- basically if the configuration
has changed, call whisper-auto-resize.py before restarting carbon.

This naturally will slow the start-up of carbon in the cases where we
have made changes, but it does ensure that we never write to the wrong
format whisper file.
Include attribution for whisper-auto-resize.py
Merge pull request #16 from ministryofjustice/resolution

Set a sensible default retention/aggregation policy; enable auto-resize on carbon start.