Don't fail hard if broken analysis is used.

Today an analysis chain with broken tokenfilters or tokenizers like
WordDelimiterFilter might produce somewhat broken term vectors that cause
`StringIndexOutOfBoundsExceptions` if FastVectorHighlighter is used
since the positions / offsets contract is violated and offsets of highlight
tokens are not increasing but decreasing even if their positions are increasing.

Yet, if we detect such a situation we can resort the tokens which might cause
somewhat odd highlights but doesn't fail hard with a StringIndexOOBException.

Closes #3006
Fixed custom hunspell dictionary directory

Properly loading dictionaries from based on the "indices.analysis.hunspell.dictionary.location" setting if one exists
Remove Java 7 only API

We still run on Java 6 as minimum requirement. Integer.compare(int,int)
was added in Java 7. This caused compile errors on CI.
Fix error getting array fields

Fixes #3000
Add `field_masking_span` to IndexQueryModule

The query parser for `field_masking_span` has never been added / bound to
the IndexQueryModule.

Closes #3007
Don't use smart query wrapping for span term query

Lucenes span queries are a different family than 'ordinary' queries
in lucene. Spans only work with other spans such that smart query
wrapping doesn't work with span queries at all ie. we can't wrap
in filtered query.

Closes #2994
Fixed parsing of `track_scores` in `RestSearchAction`

Closes #2986
Merge pull request #3009 from chilling/issue2986_scores

Fixed parsing of track_scores in RestSearchAction
Add NGramTokenizer and NGramTokenFilter to broken chains

NgramTokenizer and NGramTokenFilter are broken with a version < 4.2
We should still support these filters but should prevent the StringIOOB
exceptions. Adding these fitlers for the FragmentBuilderHelper will
allow seamless highlighting on fields indexed with those tokenizers or
tokenfilters
Add more informative toString method to StoreDirectory
Handle optional term and field statistics gracefully

Lucene provides a set of statistics that depend on the codec / postingsformat
as well as on the index options used when the field is created / indexed.
If a certain stats value is not available lucene return `-1` instead of the
correct value. We need to ensure that those values are encoded correctly if
we try to write vLongs as well as when we aggregate those values.

Closes #3012
Use simplified asserts and better naming

addOne / subOne are likely easier to understand without reading the docs.
If I would read my emails this would have made it in the last commit.
Fix DfsSearchResult method names in AggregatedDfs
Reroute Allocate to force primary allocation when enabled
Typically, the main reason a reroute allocation command with allow_primary is enabled, is to force create an empty new shard because a shard (and its replicas) were lost. This can't be done today because the shard expects to have a valid index where its allocated, we need to clear its post allocation flag to make sure it is allowed to create a fresh index.
Improve test stability
upgrade to jackson 2.2.1
Make GetField behavior more consitent for multivalued fields.

Before this change, the GetField#getValue() method was returning a list of values of a multivalued fields if the field values were obtained from source or if the field was stored and real-time get was used. If the field was stored but non-realtime get was used, GetField#getValue() was returning only the first element and the GetField#getValues() was returning a list of elements. This change makes behavior consistent. GetField#getValue() now always returns only the first value of the field and GetField#getValues() returns the entire list.
lazy compute the hash and actually use it...
Added support for the update operation in the bulk api.

Update requests can now be put in the bulk api. All update request options are supported.

Example usage:
```
curl -XPOST 'localhost:9200/_bulk' --date-binary @bulk.json
```

Contents of bulk.json that contains two update request items:
```
{ "update" : {"_id" : "1", "_type" : "type1", "_index" : "index1", "_retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"} }
{ "update" : { "_id" : "0", "_type" : "type1", "_index" : "index1", "_retry_on_conflict" : 3} }
{ "script" : "counter += param1", "lang" : "js", "params" : {"param1" : 1}, "upsert" : {"counter" : 1}}
```
The `doc`, `upsert` and all script related options are part of the payload. The `retry_on_conflict` option is part of the header.

Closes #2982
Reject update request that has both script and doc