Add NGramTokenizer and NGramTokenFilter to broken chains

NgramTokenizer and NGramTokenFilter are broken with a version < 4.2
We should still support these filters but should prevent the StringIOOB
exceptions. Adding these fitlers for the FragmentBuilderHelper will
allow seamless highlighting on fields indexed with those tokenizers or
tokenfilters
Add more informative toString method to StoreDirectory
Handle optional term and field statistics gracefully

Lucene provides a set of statistics that depend on the codec / postingsformat
as well as on the index options used when the field is created / indexed.
If a certain stats value is not available lucene return `-1` instead of the
correct value. We need to ensure that those values are encoded correctly if
we try to write vLongs as well as when we aggregate those values.

Closes #3012
Use simplified asserts and better naming

addOne / subOne are likely easier to understand without reading the docs.
If I would read my emails this would have made it in the last commit.
Reroute Allocate to force primary allocation when enabled
Typically, the main reason a reroute allocation command with allow_primary is enabled, is to force create an empty new shard because a shard (and its replicas) were lost. This can't be done today because the shard expects to have a valid index where its allocated, we need to clear its post allocation flag to make sure it is allowed to create a fresh index.
Improve test stability
upgrade to jackson 2.2.1
Make GetField behavior more consitent for multivalued fields.

Before this change, the GetField#getValue() method was returning a list of values of a multivalued fields if the field values were obtained from source or if the field was stored and real-time get was used. If the field was stored but non-realtime get was used, GetField#getValue() was returning only the first element and the GetField#getValues() was returning a list of elements. This change makes behavior consistent. GetField#getValue() now always returns only the first value of the field and GetField#getValues() returns the entire list.
lazy compute the hash and actually use it...
Added support for the update operation in the bulk api.

Update requests can now be put in the bulk api. All update request options are supported.

Example usage:
```
curl -XPOST 'localhost:9200/_bulk' --date-binary @bulk.json
```

Contents of bulk.json that contains two update request items:
```
{ "update" : {"_id" : "1", "_type" : "type1", "_index" : "index1", "_retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"} }
{ "update" : { "_id" : "0", "_type" : "type1", "_index" : "index1", "_retry_on_conflict" : 3} }
{ "script" : "counter += param1", "lang" : "js", "params" : {"param1" : 1}, "upsert" : {"counter" : 1}}
```
The `doc`, `upsert` and all script related options are part of the payload. The `retry_on_conflict` option is part of the header.

Closes #2982
Reject update request that has both script and doc
Added test that checks if a validation error is thrown when both doc and script provided in a update request.
Closes #2967
No need for deepCopy on makeSafe for pages field data
Since its a reference to a buffer in the PagedBytes, we don't need to deep copy it on makeSafe, just shallow copy it
remove concrete bytes for field data
no really need for it, specifically with the fact that we don't need to deepCopy on makeSafe for the (default) paged bytes
not active shards should translate to 503 not 500
test no longer needed...
use throwable in transport layer
catch throwable when processing messages in the transport layer, to report back failures even under errors
upgrade to mvel 2.1.5
update to joda 2.2
resolved empty setting values should be removed
when resolving empty settings values, their value should be removed, for example, when using ${env.ENV_VAR}, and ENV_VAR is not set, then the setting should be removed