Internal: remove ForceSyncDirectory

Historical code, not used anymore.

Closes #7804
Core: Check if from + size don't cause overflow and fail with a better error.

Closes #7778
Discovery: Give a unique id to each ping response

During discovery a node gossips with other nodes to discover the current state of the cluster - what nodes are out there, what version they use and most importantly whether there is an active master out there. During this ping process we may end up in a situation where old information is mixed with new. This is comment if a couple of master election happen in rapid succession.

This commit adds a monotonically increasing id to each ping response. This makes it easy to always select the last ping from every node.

Closes #7769
Test: RecoveryWhileUnderLoadTests.recoverWhileRelocating should report cluster state when failing to reach green
Recovery: mapping check during phase2 should be done in cluster state update task

Before phase2 we check verify that the local mapping is in sync with the cluster state mapping (and send & wait on a master update mapping task if not). This check should be done under a cluster state update task to make sure an incoming cluster state update to do not change things while we check.

Closes #7744
Tests: extend testRecoverFromPreviousVersion to sometimes index during relocation

Relates to #7729

Closes #7768
Update java source example

From the version 1.0 FilterBuilders and QueryBuilders are not part from org.elasticsearch.index.query.xcontent package no more.

Closes #7701.

(cherry picked from commit 32d4200)
Apply bulk change to 1.3.3

relates to #7729
Docs: IndexResponse.matches() does not exist anymore

Since 1.0, percolator has been redesigned so percolator is not applied anymore at index time.

Closes #7548.
Document the Java BulkProcessor

Closes #7638.
[TEST] Ensure primaries are allocated before bulk indexing with dymamic mappings
[RECOVERY] Mark last file chunk to fail fast if payload is truncated

Today we rely on the metadata length of the file we are recoverying
to indicate when the last chunk was received. Yet, this might hide bugs
on the compression layer if payloads are truncated. We should indicate
if the last chunk is send to make sure we validate checksums
accordingly if possible.

Closes #7830
Chunk direct buffer usage by networking layer
Today, due to how netty works (both on http layer and transport layer), and even though the buffers sent over to netty are paged (CompositeChannelBuffer), it ends up re-copying the whole buffer into another heap buffer (bad), and then send it over directly to sun.nio which allocates a full thread local direct buffer to send it (which can be repeated if not all message is sent).
  This is problematic for very large messages, aside from the extra heap temporal usage, the large direct buffers will stay around and not released by the JVM.
  This change forces the use of gathering when building a CompositeChannelBuffer, which results in netty using the sun.nio write method that accepts an array of ByteBuffer (so no extra heap copying), and also reduces the amount of direct memory allocated for large messages.
  See the doc on NettyUtils#DEFAULT_GATHERING for more info.
closes #7811
[TEST] disable translog based flushes when corrupting files

These tests rely on the fact that all files stay the same after
the corruption and if we run into a translog based flush we might
use a new / different delete file causing the test to fail.
Docs: java add static imports for query and filter builders

Also move and add inline comments to references

Closes #6920.
[TEST] Make sure test actually throttles
Docs: fix no callouts refer to list item 3
[TEST] Add a more restrictive thread leaks filter

Today all threads are allowed to leak a suite. This is tricky since
it essentially allows resource leaks by default where for instance
test private TransportClients will never get closed and consume
resources influencing other tests. It also hides threads that
are not fully under elasticsearchs control like the Lucene
TimeLimitingCollector thread. This commit restricts the threads
that can leak a suite to the threads spawned from testclusters
and fixes sevearl places that leaked threads.

Closes #7833
[TEST] use more terminate calls and wait for termination
[TEST] Disable thread filter for now