Blosc/bcolz — pull request #62 by FrancescElies (2014-10-01 14:49:43+00:00)
Title: ctable: where_terms & groupby + tests

Dear Bcolz-team,

first of all, thank you for your efforts creating and maintaining this project, my colleague @CarstVaartjes and me are very interested in this amazing data container.
We proceeded to implement an extra functionality we would like to have (see https://github.com/Blosc/bcolz/issues/57 & other related issues https://github.com/Blosc/bcolz/issues/61)

We tried our best but since we are nor C neither Python experts we know the solution we are proposing here can be optimized, we tried to make an ctable_ext.pyx file for this extra functionality but ran into issues (no pxd file for carrays made it really difficult). That is the main reason for having extended the carray_ext.pyx file.
Apologies in advance for any code or things not meeting the standards of the project, we are open to criticism and would love to improve the code.

Basically we added two things:
- a where_terms clause that can handle in selects
- a groupby clause (that can also use where & where_terms filters) that sums measures over columns

The groupby clause is largely based upon cytoolz, is okay with performance but is still far off from Pandas; so this functionally works, but it might be nice for future improvements to look at Pandas' hashing & grouping for improving the code.

Please let us know any comments you could have about this request.

Thanks & best regards.

Francesc & Carst