Mpdreamz/NEST — pull request #528 by Mpdreamz (2014-03-11 16:15:59+00:00)
Title: Connection pooling / cluster failover support

This PR adds support for builtin cluster failover support and connection pooling.

## SingleNodeConnectionPool
This is still the default when you do not specify any connectionpool.

```csharp
var node = new Uri("http://localhost");
var settings = new ConnectionSettings(node);
var client = new ElasticClient(settings);
```

In this example we never explicitly passed an `IConnectionPool` to use so it will default to the `SingleNodeConnectionPool` which will always report the single node we passed in as alive and well. This means that the defaults for existing code is unchanged

## StaticConnectionPool

```csharp
var nodes = new [] {
new Uri("http://localhost:9200"),
new Uri("http://localhost:9201"),
new Uri("http://localhost:9202"),
new Uri("http://localhost:9203"),
};
var pool = new StaticConnectionPool(nodes);
var settings = new ConnectionSettings(pool)
.MaxRetries(8); //bit much but hey... why not:)
var client = new ElasticClient(settings);
```

This adds a static pool of nodes that will be round robin'ed over when performing elasticsearch calls. Whenever a node reports a known failure (timout, 503 etcetera) it will retry on the next node in the pool and mark the failing node as dead. This means that while the node has been marked dead it will automatically be skipped for the duration it has been marked dead. When all nodes are marked dead it will simply pick one at random. You can specify the maximum amount of retries which will default to the amount of nodes you pass in minus 1.

## SniffingConnectionPool

```csharp
var nodes = new [] {
new Uri("http://localhost:9200"),
new Uri("http://localhost:9201"),
new Uri("http://localhost:9202"),
new Uri("http://localhost:9203"),
};
var pool = new SniffingConnectionPool(nodes);
var settings = new ConnectionSettings(pool)
.SniffOnStart()
.SniffOnConnectionFault()
.SniffLifeSpan(TimeSpan.FromMinutes(10));
var client = new ElasticClient(settings);
```

Similar to the static connection pool this will round robin over the specified nodes but will use the nodes to initially sniff the rest of the cluster to build the list of known hosts. You can make it re-sniff whenever a connection fault occurs or sniff whenever the last sniff happened too long ago.

See the unit tests here:
https://github.com/Mpdreamz/NEST/tree/feature/connection-pooling/src/Elasticsearch.Net.Tests.Unit/Connection

In particular read this tests to know why builtin support for NEST is an absolute must:

https://github.com/Mpdreamz/NEST/blob/feature/connection-pooling/src/Elasticsearch.Net.Tests.Unit/Connection/ConcurrencyTests.cs#L94