armon/statsite — pull request #61 by praddy (2014-05-24 14:08:20+00:00)
Title: Aggregation Buckets Support

Hi,

I really like statsite, thanks for creating & maintaining it. I have specific requirement in which it's actually up to the client to tell when to flush the aggregated stats. Take for example logs, log records have a timestamp attached to them and ideally it'd be more accurate for the log parser (statsite's client) to tell statsite when to flush and with which timestamp. Also, in case the log parser is slower in processing the logs than the log publisher (NGINX for example), the usual aggregation interval (flush interval) would lead to wrong aggregations. I have a working implementation of this (might not be optimal :-) ), in the implementation I am just allowing the clients to create buckets, pump in stats to those buckets & tell statsite when to flush those buckets. For example:

- Create a bucket named mybucket with the associated flush time timestamp 1400849984
```
mybucket:1400849984|>
```

- Pump in the some counters (or timers or sets or whatever) to that bucket
```
counter:1|c|>mybucket
timer:0.13|m|>mybucket
```

- Flush the bucket with the associated timestamp (the one we received with create bucket)
```
mybucket:ignored|<
```

Furthermore, in a distributed environment where more than one process or machines would be processing the logs, it is required that all of them finish pushing the stats into the respective bucket before it's flushed. For supporting that, one can optionally supply client name along with the START_BUCKET & END_BUCKET messages, like this:

```
mybucket:1400849984|>|client1
mybucket:ignored|<|client1
```
Bucket will be flushed only when all the clients that started the bucket have issued an END_BUCKET as well.

I was thinking of adding something like a max bucket lifetime configuration directive based on which the buckets can be flushed & freed in case none of the clients flush the buckets. It could be based on (current time >= bucket creation time + max bucket lifetime). Also, this patch adds ascii only implementation but adding binary support is just a matter of deciding on the protocol.

It'd be great to know about your thoughts on this.

TIA
- praddy