Update allow to pass prune function

Fixes #38. Also various ETL updates for cleaning the data