major refactoring to support filters