Prevent FVH from entering a very long running loop on large docs with high freq phrase terms.

Terminate phrase searches early if max phrase window is exceeded in
FastVectorHighlighter to prevent very long running phrase
extraction if phrase terms are high frequent. See LUCENE-5182

Closes #3543
Bound processor size based cals to 32
We use number of processors to choose default thread pool sizes, and number of workers in networking (for HTTP and transport). Bound it to max at 32 by default as a safety measure to create too many threads.

This relates to #3478, where we set the default to 24, but 32 is probably a better default.

closes #3545
Smarter default to `index.index_concurrency`
closes #3546
Adding Contributing guidelines
Expose size statistics for completion suggest

In order to determine how many RAM the completion suggest structures will eat up, this data should be exposed.

Closes #3522
Checking for paylods as JSON objects in completion suggester

If the payload is not a JSON object on indexation, an exception will be thrown.
Completion Suggester: Allow payload to be a value
closes #3550
Plugin Manager can not download _site plugins from github
Sounds like github changes a bit download url for master zip file.

From `https://github.com/username/reponame/zipball/master` to `https://codeload.github.com/username/reponame/zip/master`.

We need to update plugin manager to reflect that change.

In the meantime, we invite users having this issue to use:

```sh
bin/plugin -install reponame -url https://codeload.github.com/username/reponame/zip/master
```

For example:

```sh
bin/plugin -install paramedic -url https://codeload.github.com/karmi/elasticsearch-paramedic/zip/master
```

Closes #3551
Plugin Manager can not download _site plugins from github
Sounds like github changes a bit download url for master zip file.

From `https://github.com/username/reponame/zipball/master` to `https://github.com/username/reponame/archive/master.zip`.

We need to update plugin manager to reflect that change.

In the meantime, we invite users having this issue to use:

```sh
bin/plugin -install reponame -url https://github.com/username/reponame/archive/master.zip
```

For example:

```sh
bin/plugin -install paramedic -url https://github.com/karmi/elasticsearch-paramedic/archive/master.zip
```

Closes #3551
(cherry picked from commit 352d2aa)
Prevent NPE if all docs for the field are pruned during merge

During segment merges FieldConsumers for all fields from the source
segments are created. Yet, if all documents that have a value in a
certain field  are deleted / pruned during the merge the FieldConsumer
will not be called at all causing the internally used FST Builder to
return `null` from it's `build()` method. This means we can't store
it and run potentially into a NPE. This commit adds handling for
this corner case both during merge and during suggest phase since
we also don't have a Lookup instance for this segments field.

Closes #3555
ClusterUpdateSettingsAction will hang if no changes were made

 Closes #3560
random_score function - Added the index name and shard id to the randomization, and improved the PRNG itself

Closes #3559
Call onRemoval of shard IdCache during clear.

This looks like a copy/paste issue where onCached was being called
rather than onRemoval. This should fix the ID cache stats not being
correct after a call to /_cache/clear?id_cache=true
Added test for verifying the id cache size if the clear cache api is invoked.
Debian init script: Add debian default java location

Adding /usr/lib/jvm/default-java to JAVA_HOME candidates to check

Closes #3500
cleanup removable of reject_policy
Dont write pidfile twice on startup

There is no need to write the pidfile in the bin/elasticsearchshell script
as this happens already in the java code.

Also cleaning up the bin/elasticsearch shell script a bit (no need to return
an error code when exec is called, as this forks and exits the shell script
immediately).

Closes #3529
Closes #1745
Fix inconsistent usage of ScriptScoreFunction in FiltersFunctionScoreQuery

This commit fixes inconsistencies in `function_score` and `filters_function_score`
using scripts, see issue #3464

The method 'ScoreFunction.factor(docId)' is removed completely, since the name
suggests that this method actually computes a factor which was not the case.
Multiplying the computed score is now handled by 'FiltersFunctionScoreQuery'
and 'FunctionScoreQuery' and not implicitely performed in
'ScoreFunction.factor(docId, subQueryScore)' as was the case for 'BoostScoreFunction'
and 'DecayScoreFunctions'.

This commit also fixes the explain function for FiltersFunctionScoreQuery. Here,
the influence of the maxBoost was never printed. Furthermore, the queryBoost was
printed as beeing multiplied to the filter score.

Closes #3464
add boost_mode to rest interface

allow user to set combine functions explicitely via boost_mode variable.
make mult default boost mode

always multiply query score to function score. For script score
functions, this means that boost_mode has to be set to `plain` if
'function_score' should behave like 'custom_score'