better logic on sending mapping update new type introduction
when an indexing request introduces a new mapping, today we rely on the parsing logic to mark it as modified on the "first" parsing phase. This can cause sending of mapping updates to master even when the mapping has been introduced in the create index/put mapping case, and can cause sending mapping updates without needing to.
 This bubbled up in the disabled field data format test, where we explicitly define mappings to not have the update mapping behavior happening, yet it still happens because of the current logic, and because in our test we delay the introduction of any mapping updates randomly, it can get in and override updated ones.
closes #6669
[TEST] Also wait for fields to have been applied in the mapping in cluster state during teh waitForConcreteMappingsOnAll call

The concrete DocMapper on the master will be updated before the mapping in the cluster state. The DocMapper is updated during the cluster update task. This can lead to occasional assertion failures on the mapping response, because that is based on the mapping the cluster state, which may not yet have been updated. (time window between the DocMapping is updated, but the mapping in the cluster state isn't)
[TEST] Take compatibility version into account for XContentType
randomization

We randomize the XContentType to test deriving the content type on all
APIs. Yet, BWC tests run against versions where CBOR wasn't around
this commit ensures we don't use CBOR when compatibility version is
less than `1.2.0`

Closes #6691
[Discovery] Handle ConnectionTransportException during a Master/Node fault detection ping

Both the Master and Node fault detection register themselves to be notified when a node disconnects to be able to respond to it accordingly. As such, when a ConnectionTransportException was raised on a ping request, it was not handled as it is already handled somewhere else. However, this does introduce a racing condition, if the disconnect  happen during a period where there is no current master (minimum_master_node breach) at which time the fault detection is not active. In this case, we will only discover the disconnect error during the ping request, so we have to respond accordingly.

Closes #6686
[TEST] Don't wait for relocations - the ensureYellow() call does that already
[TEST] Also do waitForConcreteMappingsOnAll() call for the .percolator type.
Percolator: improve logging and cleanup try-catch statement for percolator query loading.
[TEST] use pre 1.2.0 MATCH_ALL version if we test BWC for pre 1.2.0
clean shard bulk mapping update to only use type
today we track both the index name and type for mapping updates in the shard bulk action, but we only work against on index in this level, so no need to track the index name itself
closes #6695
Docs: Fixed missing comma in multi match query example
[Tests] remove bigArrays which were not fully release from watch list (and still fail the test)

This is to prevent future tests from failing due to these arrays
[TEST] Stabelize BWC tests for version < 1.1.0
Docs: Add clarifying text about regexp and terms

For the casual reader, the reference to "term queries" may be glossed over, yielding an unexpected result when using `regexp` queries.
This attempts to make that distinction more prominent.

Closes #6698
[TEST] Expose `tests.filter` for elasticsearch tests.

`-Dtests.filter` allows to pass filter expressions to the elasticsearch
tests. This allows to filter test annotaged with TestGroup annotations
like @Slow, @Nightly, @Backwards, @Integration with a boolean expresssion like:

 * to run only backwards tests run:
     `mvn -Dtests.bwc.version=X.Y.Z -Dtests.filter="@backwards"`
 * to run all integration tests but skip slow tests run:
     `mvn -Dtests.filter="@integration and not @slow"
 * to take defaults into account ie run all test as well as backwards:
     `mvn -Dtests.filter="default and @backwards"

This feature is a more powerful alternative to flags like
`-Dtests.nighly=true|false` etc.

Closes #6703
Analysis: Add additional Analyzers, Tokenizers, and TokenFilters from Lucene

Add `irish` analyzer
Add `sorani` analyzer (Kurdish)

Add `classic` tokenizer: specific to english text and tries to recognize hostnames, companies, acronyms, etc.
Add `thai` tokenizer: segments thai text into words.

Add `classic` tokenfilter: cleans up acronyms and possessives from classic tokenizer
Add `apostrophe` tokenfilter: removes text after apostrophe and the apostrophe itself
Add `german_normalization` tokenfilter: umlaut/sharp S normalization
Add `hindi_normalization` tokenfilter: accounts for hindi spelling differences
Add `indic_normalization` tokenfilter: accounts for different unicode representations in Indian languages
Add `sorani_normalization` tokenfilter: normalizes kurdish text
Add `scandinavian_normalization` tokenfilter: normalizes Norwegian, Danish, Swedish text
Add `scandinavian_folding` tokenfilter: much more aggressive form of `scandinavian_normalization`
Add additional languages to stemmer tokenfilter: `galician`, `minimal_galician`, `irish`, `sorani`, `light_nynorsk`, `minimal_nynorsk`

Add support access to default Thai stopword set "_thai_"

Fix some bugs and broken links in documentation.

Closes #5935
[TEST] Wait for cluster consistency before tests starts
Fix doc formatting. Norwegian stemmers and Scandinavian normalizers
were missing commas between entries.
[TEST] Exclude SORANI analyzer if compatibility version is < 1.3.0
Docs: Fix typo in timestamp-field.asciidoc

Closes #6661
Docs: unescape regexes in Pattern Tokenizer docs

Currently regexes in Pattern Tokenizer docs are escaped (it seems according to Java rules). I think it is better not to escape them because JSON escaping should be automatic in client libraries, and string escaping depends on a client language used. The default pattern is `\W+`, not `\\W+`.

Closes #6615