ambiata/ivory — pull request #307 by charleso (2014-09-25 06:45:17+00:00)
Title: Support basic filtering for virtual features

Some problems/thoughts.

1. **IMPORTANT* I'm storing the filter as `string` in the thrift dictionary. This will eventually (soon?) be replaced, and may mean some annoying backwards compatibility (or does it?). I started storing an actual thrift structure that represented the "real" filter, but figured that might be even worse if I get it wrong. It's worth noting that we're storing "expression" as a string as well, so maybe it's not so bad.

2. Separation between `Filter` and `FilterEncoded` is horrilble and made worse because we don't have a real syntax, and so it's "nicer" if we wait until we know what we're parsing to separate the difference between structs and normal features. If we simply the requirements further (eg. must have `field,value` regardless of being a non-struct feature) then it's possible to tweak things slightly. This plays somewhat into part 1, because at the moment the actual contents of the filter don't really matter until squash (ignoring import validation). I'm not such a huge fan of doing that because you would still end up having to fold over the `FilterSimple` structure with the encoding again anyway in `FilterReduction` _and_ `DictionaryImportValidate`.

2a. I started fixing `DictionaryTextStorageV2` to collapse `FilterEncoded` but it started to get messy and I figured earlier feedback was more important.

3. Not sure whether I need to worry about specialising and/or using generics in `FilterReducerEquals`. Looks like Thrift boxes things anyway so what I have is OK for "equals".