Internal: adjusted internal requests visibility from public to package private (redo)

was just reverted by mistake in the failed attempt of isolating the change and taking it out of #7319
NPE in ShardStats when routing entry is not set yet on IndexShard
closes #7356
Query Cache: Add hit and miss count
closes #7355
[TEST] Improve robustness of restoreIndexWithMissingShards test
Term vector API: return 'found: false' for docs between index and refresh

Closes #7121
Aggregations: Remove the logic to optionally sort/dedup values on the fly.

These options are not used anymore. Instead numeric values can now contain dups
and it is the responsibility of the aggregation to deal with it (eg. terms).
And otherwise all values sources are now sorted, which is the contract of the
interfaces that they implement.

Close #7276
Facets: Removal from master.

Close #7337
[TEST] fixed concurrency issue in IndicesRequestTests
[TEST] bw comp testMultiGet should wait for yellow, not for green
Internal: Remove CacheRecycler.

The main consumer of this API was the faceting module. Now that it's gone,
let's remove CacheRecycler as well.

Close #7366
[PARSER] Clarify XContentParser/Builder interface for binary vs. utf8 values

Today we have very confusing naming since some methods names claim to
read binary but in fact read utf-16 and convert to utf-8 under the hood.
This commit clarifies the interfaces and adds additional documentation.

Closes #7367
More Like This Query: Switch to using the multi-termvectors API

The term vector API can now generate term vectors on the fly, if the terms are
not already stored in the index. This commit exploits this new functionality
for the MLT query. Now the terms are directly retrieved using multi-
termvectors API, instead of generating them from the texts retrieved using the
multi-get API.

Closes #7014
[TEST] added debug lines to bw comp testDeleteByQuery and testDeleteRoutingRequired
[TEST] Sort by the _id field instead of _uid field and also assert the sort value.
Internal: make sure that multi_percolate request hands over its context and headers to its corresponding shard requests

Closes #7371
Internal: make sure that multi_search request hands over its context and headers to its corresponding search requests

Closes #7374
Aggregations: Fixes pre and post offset serialisation for histogram aggs

Changes the serialisation of pre and post offset to use Long instead of VLong so that negative values are supported.  This actually only showed up in the case where minDocCount=0 as the rounding is only serialised in this case.

Closes #7312
Testing doc: describe -Dtests.heap.size and -Dtests.jvm.argline
Test: write heap dump per default

The heap dump is always interesting in case a test causes an OOM.
Per default is is written to target/JX/pidXYZ.hprof
[STORE] Allow to get metadata from arbitrary commit points

Today we always use the latest commit point to return the metadata from
the store. This might cause problems for snapshot and restore since in
contrast to recovery it won't prevent concurrent flushes (lucene commits).
This can lead to all kinds of interesting effects if we are snapshotting
while flushing. This change uses the IndexCommit to open the metadata snapshot
from the store which is consistent with what we snapshot.

Closes #7376