elasticsearch/elasticsearch — pull request #7171 by nik9000 (2014-08-06 04:50:44+00:00)
Title: Add shard size allocation balance function

From the docs:
`cluster.routing.allocation.balance.size`::
Defines a weight factor for the number of similarly sized shards allocated
on a node (float). Defaults to `0.00f` because it is new and we're not
yet sure what a good default is. Raising this raises the tendency to
equalize the number of similarly sized shards across all nodes in the
cluster. "Similarly sized" means that this formula spits out the same
number:
size <= 10^7^ bytes ? 0 : floor(log10(size))
So 1GB and 5GB shards are "similar" and so are 1KB and 40MB.

Side effects:
1. Enables cluster info collection on all master eligible nodes so that
they can make informed decisions based on shard sizes as soon as they get
elected master. They don't have to wait for the job to return when they
are elected.
2. Removes the disabled flag for the cluster info job. Its now complicate
to know when you can disable it. Besides, the disk allocation decider is
on by default now and that needs it too.
3. Reworks BalanceConfigurationTests quite a bit adding a test for the
shard size balance and a test that random balance settings actually converge.
4. Fixes some of the cluster rebalance docs that were confusing on reread.

Closes #7155