Add more trace loggin to LocalIndexShardGateway.java
[STORE] Expose ShardId via LeafReader rather than Direcotry API

Today we try to fetch a shard Id for a given IndexReader / LeafReader
by walking it's tree until the lucene internal SegmentReader and then
casting the directory into a StoreDirecotory. This class is fully internal
to Elasticsearch and should not be exposed outside of the Store.

This commit makes StoreDirectory a private inner class and adds dedicated
ElasticsearchDirectoryReader / ElasticserachLeafReader exposing a ShardId
getter to obtain information about the shard the index / segment belogs to.

These classes can be used to expose other segment specific information in
the future more easily.
[TEST] Add unit tests for DiskThresholdDecider settings
Internal: allow InternalEngine to be stopped and started

Once the current engine is started you can only close it once. Once closed the engine cannot be started again. This commit adds a stop method which signals the engine to free it's resources but in a way that allows restarting.

This is done by introducing InternalEngineHolder which is a wrapper around InternalEngine. This allows to add the stop() method without adding complexity the engine implementation. InternalEngineHolder also serves an entry point for listeners (incoming and outgoing) to other ES components, which removes the needs add/remove them if the engine is stopped.

Closes #8784
[TEST] Pass class level test logging to external nodes

This commit passes the test logging annotation from the class
level to the external nodes as well.

Closes #8552
docs: add pgp key to repositories page
Plugins: Plugin failed to load since #8666

The method Path.endsWith(String s) doesn't work exactly the same way as String.endsWith() (see http://docs.oracle.com/javase/7/docs/api/java/nio/file/Path.html#endsWith(java.nio.file.Path)). This blocks the loading of plugins.
Merge branch 'fix/plugins-loading'
[ENGINE] Add engine lifecycle store reference to EngineHolder

This commit add the engines reference to the store out of the actual
implementation into the hodler since the holder manages the actual lifcycle.
Engine internal references like per searcher or per recovery are kept inside
the actual implemenation since the have a different lifecycle.
Test: fix this test to work in IntelliJ
fix cache key methods on leaf reader
Add test that ES filterreader getCoreCacheKey() behaves correctly.

Closes #8831.
Stats: Add more fine grained memory stats from Lucene segment reader.

This is a start to exposing memory stats improvements from Lucene 5.0.
This adds the following categories of Lucene index pieces to index stats:
* Terms
* Stored fields
* Term Vectors
* Norms
* Doc values
Tests: Additional test for memory stats api.
[TEST] Beef up InternalEngineTest and remove bogus timeouts
Ban java.io.File in tests.

Restrict use of java.io.File to 5 methods (excluded), but otherwise ban.
This is a prerequisite to do any mocking here.

I don't try to do any heavy cleanup on these tests, I am not familiar with them.
So this is mostly a rote straightforward conversion.

Closes #8836
Add index.data_path and index.data_template settings

This allows specifying the path an index will be as well as the layout
of the folder for each individual shard.

`index.data_path` and `index.data_template` are both specified in the
settings when creating an index, and can not be dynamically changed.

`index.data_path` is the path to the base directory for the index.

`index.data_template` is a mustache template that will be used when
resolving the path for an index or translog. It currently supports three
injected variables:

{{index_name}} - the name of the index
{{node_id}} - numeric id of the node
{{shard_num}} - shard number

An example request would look like:

POST /myindex
{
  "settings": {
    "number_of_shards": 2,
    "data_path": "/tmp/myindex",
    "data_template": "i_{{index_name}}/{{node_id}}/shard_{{shard_num}}"
  }
}

And would put data in /tmp/myindex/i_myindex/0/shard_0 and
/tmp/myindex/i_myindex/0/shard_1
Remove templating, add node id to data directory

Also adds a custom data path into testing to randomly use a custom data
path
Add additional test for NodeEnvironment.ADD_NODE_ID_TO_CUSTOM_PATH