cozy/cozy-data-indexer — pull request #12 by jsilvestre (2014-09-08 12:07:22+00:00)
Title: Added: multiple fields indexation and number of matched results

I added two new features:
* support for multiple fields indexation.
* an option to display the number of matched results

## Disclaimer
* It works with my DS PR, they must be updated together.
* It's fully backward compatible, BUT, indexes must be removed. The reason is that we change the schema, we could simply update it but the code is done in a way that will process the update for every single request, so I think it's easy way is to remove the index (`$ rm -rf dataindexerdirectory/indexes`). Better ideas are obviously welcome.

## Multiple fields indexation
*(I'm going to port that to a proper cookbook and update the documentation when the features will be more tested).*

Whoosh has a feature called "dynamic fields" (https://pythonhosted.org/Whoosh/schema.html#dynamic-fields) that allow to have separate fields without having to declaring them individually in the index schema, which is good because we don't know in advance all the fields that can be found.

So I created a dynamic type for each Whoosh types: text, numeric, boolean and date. Dynamic types work like a wildcard. If the field is "something_text" then it will match the "*_text" dynamic type. I implemented them by suffixing with "_string", "_number", "_boolean" and "_date".

The jugglingdb-cozy-adapter knows the type of each fields so it sends them to the indexer (through the DS) and the indexer will append the suffix accordingly.

If the indexer is not able to determine the type of a field, it will fallback to the old behaviour. That being said, the old behaviour only supports unicode value so it's more limited.

So, if JugglingDB cannot get a proper basic type (string, number, date, boolean), the indexer will fallback. That is for BC and for advanced types like array of strings. If you want to index an advanced field, though, the jugglingdb-cozy-adapter provide two options: `indexerType` and `indexerMap`.
`indexerMap` is a function that turns the advanced field into something the indexer can understand, e.g. an array of strings into a string with all the items separated with a space.
`indexerType` is basically the type of the field after the map is processed.

Again, if there is nothing (like an older apps), nothing happens.

## Showing number of matched results
To be able to paginate, we need to know how many results have been matched by the query. That's what the `showNumResults` option allows.