boltdb/bolt — pull request #162 by mkobetic (2014-05-13 12:49:50+00:00)
Title: Recursive/aggregate bucket stats

This PR changes Bucket.Stats() to aggregate the totals from the receiver and all its sub-buckets. It also adds a new 'stats' command to the command-line tool. The command takes an optional prefix and aggregates the stats for all root buckets matching the prefix. When the prefix is omitted it sums up all root level buckets. Note that the stats never include the root db page.

Most of the stats are simply summed together except the Depth. Depth sums up the maximum Depths at each level. So it approximates total depth of the tree, although it may be over-estimating it because the deepest bucket at a given level doesn't necessarily contain the deepest bucket from the next level.

Inline buckets add an interesting twist. This PR adds three new stats to set:
```
// Bucket statistics
BucketN int // total number of buckets including the top bucket
InlineBucketN int // total number on inlined buckets
InlineBucketInuse int // bytes used for inlined buckets (also accounted for in LeafInuse)
```
The key point is that InlineBucketInuse is also included in LeafInuse, so it the total bytes is use is still just BranchInuse + LeafInuse. Also note that InlineBucketInuse doesn't include the bucket headers, just the inlined page sizes, this matches how non-inlined buckets are counted (their bucket header is counted as content of their parent bucket).

Finally, note that the db root page is not accounted for in any stats. That applies equally to inlined and non-inlined buckets. Consequently when all the root buckets are inlined, and you ask for the total of all you get the somewhat non-intuitive result that there are 0 leaf pages. But that's just that all the allocated space belongs to the root page.

Open Questions/Issues:
* should we add the root page stats in some cases, e.g. when we ask for all stats?
* should we preserve availability of non-recursive stats? (Bucket.Stats(recursive bool)?)
* two of the stats tests are failing where I tried to calculate what the actual value should be, but I'm missing some bytes in my calculation (hoping @benbjohnson can help figure it out)
* check for non-default page size for stats command test (this is slightly messy since I'll need to skip some pieces of the textual output)
* I'm still hoping we could make the stats command output a bit less verbose yet still clear

@benbjohnson, @snormore, @charlescng