Compilation: don't warn about using Sun proprietary APIs

E.g. we use Unsafe in quite a few places and this generates lots of
warnings, which we now suppress using the undocumented
-XDignore.symbol.file command-line option to javac.

Closes #6423
FileSystem: Use XNativeFSLockFactory instead of the buggy Lucene 4.8.1 version

There is a pretty nasty bug in the lock factory we use that can cause
nodes to use the same data dir wiping each others data. Luckily this is
unlikely to happen if the nodes are running in different JVM which they
do unless they are embedded.

See LUCENE-5738

Closes #6424
Put index template api: unified PUT/POST behaviour in relation to create parameter

The put index template api supports the create parameter (defaults to false), which tells whether the template can replace an existing one with same name or not. Unified its behaviour between PUT and POST method, whereas the POST would previously force create to true.

Added create parameter to the rest spec (was missing before) and a REST test for create true scenario.
Aggregations optimisation for memory usage. Added changes to core Aggregator class to support a new mode of deferred collection.
A new "breadth_first" results collection mode allows upper branches of aggregation tree to be calculated and then pruned
to a smaller selection before advancing into executing collection on child branches.

Closes #6128
Bulk request which try and fail to create multiple indices may never return

This is caused by an NPE in the error handling code. All is well if only 1 index creation fails (or none).

Closes #6436
Guard against improper auto_expand_replica values

Previously if the user provided a non-conforming string, it would blow up with
`java.lang.StringIndexOutOfBoundsException: String index out of range: -1`
which is not a *helpful* error message.

Also updated the documentation to make the possible setting values more clear.

Close #5752
fixing recovery debug logging param mismatch
[Test] Enabled trace logging to  testAutoGenerateIdNoDuplicates

also increased iterations some, to increase chance of identifying bad shards
make test less evil
[TEST] Added backwards compatibility check to control whether to enable client nodes or not within TestCluster

Our REST backwards compatibility tests need to be able to disable client nodes within the TestCluster when running older tests that assume client nodes are not around.
Docs: Documented the cjk_width and cjk_bigram token filters
Docs: Added custom-analyzer equivalents of all the language analyzers
Docs: Fixed some errors on the language analyzers page
Docs: Fixed a bad ref on lang analyzers page
Docs: Fixed bad ref on cjk_width/bigram pages
Change the top_hits to be a metric aggregation instead of a bucket aggregation (which can't have an sub aggs)

Closes #6395
Closes #6434
Fielddata: EmptyOrdinals#getMaxOrd() should return 0 instead of 1, since ordinals are zero based since #5871.
Aggregations: if maxOrd is 0 then use noop collector

Before the OrdinalsCollector was used and this leads to a ArrayIndexOutOfBoundsException

Closes #6413
TransportClient: Improve logging, fix minor issue

In order to return more information to the client, in case a TransportClient
can not connect to the cluster, this commit adds logging and also returns the
configured nodes in the NoNodeAvailableException

Also a minor bug has been fixed, which propagated exceptions wrong, so that an
invalid request was actually tried on every node, if a regular connection failure
on the first node had happened.

Closes #6376
Stats: Bugfixes and enhancements to indices stats API

Bugs:
* "groups" and "types" were being ignored
* "completion_fields" as wildcards were not being resolved to fieldnames

Enhancements:
* Made "groups" and "types" support wildcards
* Added missing tests

Closes #6390