Tests: always run Lucene's CheckIndex when shards are closed in tests and fail the test if corruption is detected

Today we only run 10% of the time, and the test doesn't fail when
corruption is detected.

I think it's better to always run and fail the test, so we can catch
any possible resiliency bugs in Lucene/Elasticsearch causing corruption.

For known tests that create corrupted indices, it's easy to set
MockFSDirectoryService.CHECK_INDEX_ON_CLOSE to false...

Closes #7730
don't CheckIndex for this test case
Fix NumberFormatException in Simple Query String Query

Incorrect usage of XContentParser.hasTextCharacters() can result in NumberFormatException as well as other possible issues in template query parser and phrase suggest parsers.

Fixes #7875
script with _score: remove dependency of DocLookup and scorer

As pointed out in #7487 DocLookup is a variable that is accessible by all scripts
for one doc while the query is executed. But the _score and therfore the scorer
depends on the current context, that is, which part of query is currently executed.
Instead of setting the scorer for DocLookup
and have Script access the DocLookup for getting the score, the Scorer should just
be explicitely set for each script.
DocLookup should not have any reference to a scorer.
This was similarly discussed in #7043.

This dependency caused a stackoverflow when running script score in combination with an
aggregation on _score. Also the wrong scorer was called when nesting several script scores.

closes #7487
closes #7819
field name lookup: return List instead of Set for names matching a pattern

The returned sets are only used for iterating. Therefore we might
as well return a list since this guaratees order.

This is the same effect as in
https://github.com/elasticsearch/elasticsearch/pull/7698
The test SimpleIndexQueryParserTests#testQueryStringFieldsMatch
failed on openjdk 1.7.0_65 with
<jdk.map.althashing.threshold>0</jdk.map.althashing.threshold>

closes #7709
disable CheckIndex for these no-ack tests
Tests: turn off CheckIndex for now (it's buggy: there is a race w/ deletion of all files in the data dirs)
[TEST] wait for yellow else assertSearchResponse will trip
 [TEST] wait for yellow else assertSearchResponse will trip
Discovery: remove any local state and use clusterService.state instead

At the moment, ZenDiscovery contains a local copy of the disco nodes plus a flag that indicates whether the local node is master or not. This is redundant as the same information is stored in the cluster state. Have duplicate copy can lead to unneeded concurrency issues. This PR removes the duplication, including moving the ownership of the localNode creation to ClusterState

The PR introduces a tighter control of the background joining thread to make sure it is started and stopped together with any cluster state changes. This solves potentially concurrency bugs where a joining thread may fail to start.

Last we add a couple of safety checks to make sure that if a nodes receives a cluster state from a new master while actively trying to join another one (or electing itself) we go back to pinging to actively join it.

Closes #7834
NodesFD: simplify concurrency control to fully rely on a single map

The node fault detection class is used by the master node to ping the nodes in the cluster and verify they are alive. This PR simplifies the concurrency controls in the class + adds a test for a scenario that surfaced the problem.

Closes #7889
node shutdown: make close() syncronized

An example scenario where this will help:

When the node is shutdown via api call
(https://github.com/elasticsearch/elasticsearch/blob/master/src/test/java/org/elasticsearch/test/ExternalNode.java#L219 )
then the call returns immediately even if the node is not actually shutdown yet
(https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/action/admin/cluster/node/shutdown/TransportNodesShutdownAction.java#L226).
If at the same time the proces is killed, then the hook that would usually prevent
uncontrolled shutdown
(https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/bootstrap/Bootstrap.java#L75)
has no effect: It again calls close() which might then just return
for example because one of the lifecycles was moved to closed already.

The bwc test FunctionScoreBackwardCompatibilityTests.testSimpleFunctionScoreParsingWorks
failed because of this. The translog was not properly
written because if the shutdown was called via api, the following process.destroy()
(https://github.com/elasticsearch/elasticsearch/blob/master/src/test/java/org/elasticsearch/test/ExternalNode.java#L225)
killed the node before the translog was written to disk.

closes #7885
Internal: split internal fetch request used within scroll and search

Similar to #7856 but relates to the fetch shard level requests. We currently use the same internal request when we need to fetch within search and scroll. The two original requests though diverged after #6933 as SearchRequest implements IndicesRequest while SearchScrollRequest doesn't. That said, with #7319 we made `FetchSearchRequest` implement IndicesRequest by making it hold the original indices taken from the original request, which are null if the fetch was originated by a search scroll, and that is why original indices are optional there.

This commit introduces a separate fetch request and transport action for scroll, which doesn't hold original indices. The new action is only used against nodes that expose it, the previous action name will be used for nodes older than 1.4.0.Beta1.

As a result, in 1.4 we have a new `indices:data/read/search[phase/fetch/id/scroll]` action that is equivalent to the previous `indices:data/read/search[phase/fetch/id]` whose request implements now IndicesRequest and holds the original indices coming from the original request. The original indices in the latter request can only be null during a rolling upgrade (already existing version checks make sure that serialization is bw compatible), when some nodes are still < 1.4.

Closes #7870
Docs: Removed all the added/deprecated tags from 1.x
Docs: Fixed some deleted refs
If a node is being shutdown some in flight ping request may be executed. Make sure to keep track of those ping requests and close the unicast connect executor service.

Closes #7903
Internal: MulticastChannel should wait on receiver thread to stop during shutdown

This was signaled by our tests which shutdown class and check for thread leakage.

Closes #7835
Update suggesters.asciidoc

A request was malformed

Closes #7867
GeoJSON is lon,lat

Although emphasized in the text, the example was backwards.

Closes #7843
Improve sentence structure

Closes #7842