[TEST] Randomize netty worker and connection parameters
Try and push our system to a state where there is only a single worker, trying to expose potential deadlocks when we by mistake execute blocking operations on the worker thread
closes #6635
Wrap groovy script exceptions in a serializable Exception object

Fixes #6598
Update mapping on master in async manner
Today, when a new mapping is introduced, the mapping is rebuilt (refreshSource) on the thread that performs the indexing request. This can become heavier and heavier if new mappings keeps on being introduced, we can move this process to another thread that will be responsible to refresh the source and then send the update mapping to the master (note, this doesn't change the semantics of new mapping introduction, since they are async anyhow).
When doing so, the thread can also try and batch as much updates as possible, this is handy especially when multiple shards for the same index exists on the same node. An internal setting that can control the time to wait for batches is also added (defaults to 0).

Testing wise, a new support method on ElasticsearchIntegrationTest#waitForConcreteMappingsOnAll to allow to wait for the concrete manifestation of mappings on all relevant nodes is added. Some tests mistakenly rely on the fact that there are no more pending tasks to mean mappings have been updated, so if we see, timing related, failures down later (all tests pass), then those will need to be fixed to wither awaitBusy on the master for the new mapping, or in the rare case, wait for the concrete mapping on all the nodes using the new method.
closes #6648

allow to change the additional time window dynamically

better sorting on mappers when refreshing source
also, no need to call nodes info in test, we already have the node names

clean calls to mapping update to provide doc mapper and UUID always
also use the internal cluster support method to get the list of nodes an index is on

reverse the order to pick the latest change first

remove unused field

and fix constructor param

move to start/stop on mapping update action

randomize INDICES_MAPPING_ADDITIONAL_MAPPING_CHANGE_TIME
[TEST] properly wait for mappings when needed
[TEST] properly wait for mapping on master node
add helper method to do so, by not assuming that the mapping will exists right away by waiting for green or refreshing...
Recovery from local gateway should re-introduce new mappings
The delayed mapping intro tests exposed a bug where if a new mapping is introduced, yet not updated on the master, and a full restart occurs, reply of the transaction log will not cause the new mapping to be re-introduced.
closes #6659

add comment on the method
Add ability to restore partial snapshots

Closes #5742
Improve deletion of corrupted snapshots

Makes it possible to delete snapshots that are missing some of the metadata files. This can happen if snapshot creation failed because repository drive ran out of disk space.

Closes #6383
[TEST] Turn off double write check for restore
Docs: Removed Sense mention
Allow search templates stored in an index to be retrieved and used at search time

This change allows search templates stored in an index to be used at search time.
You can run an indexed template by
GET /_search/template
{
    'templatename' : '/index/language/id',
    'params' : {
        'param1' : 'foo'
    }
}
Running the template will fail if the template cannot be found, if the templatename
starts with a '/' but does not conform to the /index/type/id triple
If the named template cannot be found an error will be raised. Currently the
templates are NOT cached. We also use TransportGetAction directly instead of using
Client to avoid a circular dependency between Client and ScriptService

See #5637
Use Client instead of TransportGetAction

Previous ScriptService used TransportGetAction and a threaded request with explicit
Locks and Conditions. Now just use a Client and remove redundant code.

See #5921
Remove unused imports

Get rid of some of the extraneous import statements left over from the TransportGetAction stuff.

See #5921
Only load indexed scripts from the .scripts index.

Only allow ScriptService to read indexed scripts from the .scripts index.
Change the tests to index the test templates into the .scripts index.
This will prevent users from creating and loading scripts from arbitary
indices.

See #4921
Support for indexed templates and scripts.

This change allows indexed templates and scripts to be loaded at query time.
Indexed queries and scripts are not cached and must be indexed in a .scripts index.
Templates can be referenced by :
GET /_search/template
{
  "id": "some_template"
}
This change also extends the template and scripts to accept file based scripts as
GET /_search/template
{
  "file": "some_template"
}
and
GET /_search
{
  "script_fields": {
   "inline_script": {
      "script": "return 'foo'"
    },
    "indexed_script": {
      "lang": "mvel",     # uses "mvel" as the _type
      "id": "hello_world"
    },
    "script_in_file": {
      "lang": "mvel",
      "file": "hello_world"  # or "hello_world.mvel"
    }
  }
}

See #5921
Indexed Templates: Add CRUD API, explicit template types

This change allows the creation,update and deletion of indexed templates via the _search/template/{id} endpoint.
Storing a template :
curl -XPUT 'http://localhost:9200/_search/template/1a' -d '
{
  "template": {
    "query": { "match_all": {}},
    "size": "{{my_size}}"
  }
}'
Deleting a template:
curl -XDELETE 'http://localhost:9200/_search/template/1a'
Getting a template:
curl -XGET 'http://localhost:9200/_search/template/1a'
Using an indexed template:
curl -XGET 'http://localhost:9200/_search/template' -d '
{
  "template":
  {
    "id" : "1a"
  },
  "params": {
    "my_size": 100
  }
}'
Or more simply:
curl -XGET 'http://localhost:9200/_search/template' -d '
{
  "id" : "1a",
  "params": {
    "my_size": 100
  }
}'
To access on disk templates in template queries the file parameter must now be used. This does break backwards compatiblity with previous versions
since the template "query" was previously aliased to dynamic inline templates or on disk templates.

See #5637
Indexed Templates : Add another rest test

This rest test indexes a template then uses it in search.

See #5637
Indexed Scripts : Support for indexed scripts.

Scripts can now be indexed like templates.
Anything that accepted a "script" argument now also supports an "id" or "file" argument.
For backwards compatiblity files referenced from the "script" argument are supported.
Indexed scripts now have CRUD endpoints under /_search/scripts/{lang}/{id} similar to templates.
Added a test for on disk scripts that was missing.

See #5484 #5921
Indexed Templates : Update version number.

Change the version number here since we need 1.3 for this feature now.

See #5921
Indexed Scripts/Templates : Changes from review comments.

Add ParseFields to ScriptService and use them to parse script json fields in requests.
Remove templateId from SearchRequest and use templateName with a ScriptType.
Add readTo and writeTo to ScriptService.ScriptType.
Add set<ScriptType>Script to UpdateRequestBuilder instead of passing in ScriptType in setScript.
Various cleanups and small fixes.

See #5921