ambiata/ivory — pull request #261 by charleso (2014-09-08 05:04:47+00:00)
Title: Feature Reduction and Squash

Part 6...

I'm probably off the reservation at this point, although I hope that `SquashReducerState` isn't a complete waste if nothing else. Some open questions:

1. Do we even need squashing? Is there any reason that this can't just be rolled up into Pivot?

These then depend on the first question

2. What should the squash reducer count be calculated from? At some point in the future should we add a counter to snapshot and store the results in snapmeta to make it more accurate?
3. I've been lazy with the writable, just to validate the idea. I'm happy to hand-roll another one if we think it's worth it.
4. Do we need to be more precise about month/year days in SquashReduce? Currently it's still just the "max" (eg. month = 31).

Not necessarily related to this PR, but in general with snapshots:

5. What about Factset version used in the snapshot? Should we only support the latest and delete snapshots when we upgrade? Worst case we should start recording the version in snapmeta I guess.