[TEST] Ensure primaries are allocated before bulk indexing with dymamic mappings
[RECOVERY] Mark last file chunk to fail fast if payload is truncated

Today we rely on the metadata length of the file we are recoverying
to indicate when the last chunk was received. Yet, this might hide bugs
on the compression layer if payloads are truncated. We should indicate
if the last chunk is send to make sure we validate checksums
accordingly if possible.

Closes #7830
Chunk direct buffer usage by networking layer
Today, due to how netty works (both on http layer and transport layer), and even though the buffers sent over to netty are paged (CompositeChannelBuffer), it ends up re-copying the whole buffer into another heap buffer (bad), and then send it over directly to sun.nio which allocates a full thread local direct buffer to send it (which can be repeated if not all message is sent).
  This is problematic for very large messages, aside from the extra heap temporal usage, the large direct buffers will stay around and not released by the JVM.
  This change forces the use of gathering when building a CompositeChannelBuffer, which results in netty using the sun.nio write method that accepts an array of ByteBuffer (so no extra heap copying), and also reduces the amount of direct memory allocated for large messages.
  See the doc on NettyUtils#DEFAULT_GATHERING for more info.
closes #7811
[TEST] disable translog based flushes when corrupting files

These tests rely on the fact that all files stay the same after
the corruption and if we run into a translog based flush we might
use a new / different delete file causing the test to fail.
Docs: java add static imports for query and filter builders

Also move and add inline comments to references

Closes #6920.
[TEST] Make sure test actually throttles
Docs: fix no callouts refer to list item 3
[TEST] Add a more restrictive thread leaks filter

Today all threads are allowed to leak a suite. This is tricky since
it essentially allows resource leaks by default where for instance
test private TransportClients will never get closed and consume
resources influencing other tests. It also hides threads that
are not fully under elasticsearchs control like the Lucene
TimeLimitingCollector thread. This commit restricts the threads
that can leak a suite to the threads spawned from testclusters
and fixes sevearl places that leaked threads.

Closes #7833
[TEST] use more terminate calls and wait for termination
[TEST] Disable thread filter for now
Docs: fix typo
Aggregations: Significant Terms Heuristics now registered correctly

Closes #7840
Update version flags after backporting to 1.3.3

Relates to #7857
Internal: split internal free context request used after scroll and search

We currently use the same internal request when we need to free the search context after a search and a scroll. The two original requests though diverged after #6933 as `SearchRequest` implements `IndicesRequest` while `SearchScrollRequest` and `ClearScrollRequest` don't. That said, with #7319 we made `SearchFreeContextRequest` implement `IndicesRequest` by making it hold the original indices taken from the original request, which are null if the free context was originated by a scroll or by a clear scroll call, and that is why original indices are optional there.

This commit introduces a separate free context request and transport action for scroll, which doesn't hold original indices. The new action is only used against nodes that expose it, the previous action name will be used for nodes older than 1.4.0.Beta1.

As a result, in 1.4 we have a new `indices:data/read/search[free_context/scroll]` action that is equivalent to the previous `indices:data/read/search[free_context]` whose request implements now `IndicesRequest` and holds the original indices coming from the original request. The original indices in the latter requests can only be null during a rolling upgrade (already existing version checks make sure that serialization is bw compatible), when some nodes are still < 1.4.

Closes #7856
Test: ClusterServiceTests.testLocalNodeMasterListenerCallbacks - increase ping timeout

was 200ms, now 400ms
Core: upgrade to Lucene 4.10.1 snapshot

Lucene will soon release official 4.10.1, but by upgrading sooner we can 1) sidestep the false failures due to the 1.8.0_20 JVM hotspot bug (has caused a number of false failures in recent Jenkins tests), 2) make sure none of the Lucene changes in 4.10.1 are problematic.

Closes #7844
Revert Lucene upgrade
Core: try again to upgrade to Lucene 4.10.1-snapshot
Docs: Typos on range filter

Closes #7713
Docs: Update thrift.asciidoc

Fix typos, add clarifications and link.

Closes #7678