improve visibility of clusterState and shardsIt
In case of retries, we update the clusterState and shardsIt, make sure they are visible using volatile (even though updates will probably go through a memory barrier, this might explain rare failure we see when retry happens)
improve logging
the source of the default mappings should be trace logged
Allow packages to disable immediate restart on package upgrade

Both package types, RPM and deb now contain an option to not restart on upgrade.

This option can be configure in /etc/default/elasticsearch for dpkg based systems
and /etc/sysconfig/elasticsearch for rpm based systems.

By default the setting is as before, where a restart is executed on upgrade.

Closes #3685
Log failures at closing the SearcherManager.
Add TESTING document describing test settings
Add integration test for regex field data filter
allow refresh to execute on a relocating shard as well
Flush API: remove refresh flag
Refresh flag in flush is problematic, since the shards refresh is allowed to execute on is different compared to the flush shards. In order to do flush and then refresh, they should be executed as separate APIs when needed.
closes #3689
Optimize API: Remove refresh flag
Refresh flag in optimize is problematic, since the shards refresh is allowed to execute on is different compared to the optimize shards. In order to do optimize and then refresh, they should be executed as separate APIs when needed.
closes #3690
use built in flush/refresh helper methods
better logging
trace log the low level details on recovery
tag with @AwaitsFix because it causes the test suite to crash

see #3694
Also, this might have caused one of the problems described in issue #3674
Briefly delete manifested mapping type on a node without reason
When a dynamic type is introduced during indexing, the node that introduces it sends the fact that its added to the master, to be added to the master node. The master node then adds it to the index metadata and republishes that fact.

In order not to delete the mapping while the new type is introduced on the node that introduced it, we keep a map of seen mappings, and remove a mapping type when we already processed it.

The map is not properly cleared though in all places where an actual index service is being removed on a node.

closes #3697
update plugin
use static TL for common numeric precision values
Add dedicated Suggest Thread Pool
Add a dedicated suggest thread pool for the suggest API. With the new completion suggest type, which is purely CPU bounded, it makes more sense to have a dedicated thread pool for suggest compared to having it share the search thread pool and "competing" against other search operations.
closes #3698
defining filter is not always needed, we can assume match all if no filter is given
upgrade to google guava v 15
don't serialize unneeded filter
Making debian init script bourne shell compatible

Using backticks instead of bash only $() to execute a shell command

Closes #3691