term vector request
================================

Returns information and statistics on terms in the fields of a particular document as stored in the index.

	curl -XGET 'http://localhost:9200/hacker_index/hacker/1/_termvector?pretty=true'

There is three types of values that can be requested: term information, term statistics and field statistics.
Per default, all term information and field statistics are returned but no term statistics.

Optionaly, you can define the fields for which information is retrieved by setting the parameter
	"fields" : ["field1", "field2",...]
in the request body (see example below).

Term information
-------------------------

Returned information can include
- frequency of this term in the field (always returned)
- the positions of the terms ("positions" : true)
- the start and end offsets of the terms ("offsets" : true)
- the payload for each term ("payloads" : true)

If the requested information was not stored in the index, naturally it will not be returned. There will be no warning.
See [mapping](http://www.elasticsearch.org/guide/reference/mapping/core-types/) on how to configure your index to store term vectors.

Term statistics
-------------------------

You can furthermore retrieve information on

- how often a term occurs in all documents (total term frequency)
- the number of documents containing the current term (document frequency)
by setting "term_statistics" to "true".

Term statistics are not retured by default, because they are pretty expensive.

Field statistics
-------------------------

Additionally, you can request statistics for the fields:

- how many documents contain this field (document count)
- the sum of document frequencies for all terms in this field (sum document frequencies)
- the sum of total term frequencies of each term in this field (sum total term frequencies)
by setting "field_statistics" to "true".

Here is a sample request that returns everything for the field "passion":

	curl -XGET 'http://localhost:9200/hacker_index/hacker/1/_termvector?pretty=true' -d
	'{
		"fields" : ["passion"],
		"offsets" : true,
		"payloads" : true,
		"positions" : true,
		"term_statistics" : true,
		"field_statistics" : true
	}'

This will return for the document "1" of type "hacker" in "hacker_index" all the information described above for the field "passion".

Behavior
-------------------------

The term and field statistics are not accurate. First of all, deleted documents are not taken into account. Second, the information is only retrieved for the shard the requested document resides in. The term and field statistics are therefore only useful as relative measures whereas the absolute numbers have no meaning in this context.

Apart from the classes needed for implementing the feature in package

src/main/java/org/elasticsearch/action/termvector/  and
src/test/java/org/elasticsearch/test/integration/termvectors/

and the registration of the fuctionallity in

src/main/java/org/elasticsearch/client/Client.java
src/main/java/org/elasticsearch/client/support/AbstractClient.java and
src/main/java/org/elasticsearch/client/transport/TransportClient.java

, this commit introduces the following changes:

XContentBuilder
new method
public XContentBuilder field(XContentBuilderString name, int offset, int length, int... value)
to put an integer array.

IndicesAnalysisService
make token filter for saving payloads available in elasticsearch

AbstractFieldMapper/TypeParser
make term vector options string available and also fix the parsing of this string:
with_positions_payloads is actually allowed as can be seen in TermVectorsConsumerPerFields.