Don't only filter some result sets

Unless we can guarantee that the index returns exactly the matches that
would pass we need to filter everything.

Also we have to add logic to make subsequent requests to dreyfus if we
do filter some documents.