Added Random test for ShardAllocator.

This test will make random allocation decision on a growing and
shrinking cluster leading to a random distribution of the shards.
After a certain amount of iterations the test allows allocation unless
the same shard is already allocated on a node and balances the cluster
to gain optimal balance.
Added total, successful shard info to search related asserts in ElasticsearchAssertions

Rewrote SimpleSearchTests assertion to use that.
Add more trace logging if shard allocation is throttled
Changed expectedOperationsPerItem from AtomicInteger array to AtomicReferenceArray<AtomicInteger>.
Made node stopping / starting less aggressive in RecoveryPercolatorTests.
SearchWhileCreatingIndexTests didn't always supply enough node for write consistency requirements and thus hanged.

Added more debug logging to TransportShardReplicationOperationAction

*Changed* node naming in tests from GUIDs to node# based naming as they are much easier to read