Reuse suggester implementations in suggest parsers
term vector request
================================

Returns information and statistics on terms in the fields of a particular document as stored in the index.

        curl -XGET 'http://localhost:9200/twitter/tweet/1/_termvector?pretty=true'

There is three types of values that can be requested: term information, term statistics and field statistics.
By default, all term information and field statistics are returned for all fields but no term statistics.

Optionally, you can define the fields for which information is retrieved by setting the parameter
        "fields" : ["field1", "field2",...]
TODO: change to make optionally parameter
in the request body (see example below).

Term information
-------------------------

- term frequency in the field (always returned)
- term positions ("positions" : true)
- start and end offsets ("offsets" : true)
- term payload ("payloads" : true)

If the requested information wasn't stored in the index, it will be omitted without further warning.
See [mapping](http://www.elasticsearch.org/guide/reference/mapping/core-types/) on how to configure your index to store term vectors.

Term statistics
-------------------------

Setting "term_statistics" to "true" (default is "false") will return

- total term frequency (how often a term occurs in all documents)
- document frequency (the number of documents containing the current term)

By default these values are not returned since term statistics can have a serious performance impact.

Field statistics
-------------------------

Setting "field_statistics" to "false" (default is "true") will omit

- document count (how many documents contain this field)
- sum of document frequencies (the sum of document frequencies for all terms in this field)
- sum of total term frequencies (the sum of total term frequencies of each term in this field)

Here is a sample request that returns everything for the field "passion":

        curl -XGET 'http://localhost:9200/twitter/tweet/1/_termvector?pretty=true' -d
        '{
                "fields" : ["text"],
                "offsets" : true,
                "payloads" : true,
                "positions" : true,
                "term_statistics" : true,
                "field_statistics" : true
        }'

This will return all the information described above for the field "text" in document "1" of type "tweet" in index "twitter".

Behavior
-------------------------

The term and field statistics are not accurate. Deleted documents are not taken into account. The information is only retrieved for the shard the requested document resides in. The term and field statistics are therefore only useful as relative measures whereas the absolute numbers have no meaning in this context.

Further changes:

XContentBuilder
new method
public XContentBuilder field(XContentBuilderString name, int offset, int length, int... value)
to put an integer array.

IndicesAnalysisService
make token filter for saving payloads available in elasticsearch

AbstractFieldMapper/TypeParser
make term vector options string available and also fix the parsing of this string:
with_positions_payloads is actually allowed as can be seen in TermVectorsConsumerPerFields.