osdmaptool: --test-map-pgs-dump

Signed-off-by: Sage Weil <sage@redhat.com>
crush: recalculate straw scalers during a reweight

The crushtool --reweight function triggers a fresh calculation of bucket
weights so that they are always the sum of the item weights.  In the
straw bucket case, the weights were updated but the corresponding straw
scalers were not being recalculated.  The result is that there was not
effect on placement in adjusted buckets until the next time a bucket item's
weight was adjusted.

Backport: giant, firefly
Signed-off-by: Sage Weil <sage@redhat.com>
crushtool/CrushTester: output utilization even with 1 batch

Signed-off-by: Sage Weil <sage@redhat.com>
crush/CrushTester: add new --show-mappings option

This makes --show-utilization and --show-statistics usable.

Signed-off-by: Sage Weil <sage@redhat.com>
crush/CrushWrapper: dump chooseleaf_vary_r with other tunables

Signed-off-by: Sage Weil <sage@redhat.com>
crush: add straw_calc_version tunable

It doesn't do anything, yet.

Signed-off-by: Sage Weil <sage@redhat.com>
crush: default to straw_calc_version 1

Stick with bobtail tunables where it affects compatibility.  Use v1 of
straw_calc, though, since that does not, and we want the best for new
clusters.

Signed-off-by: Sage Weil <sage@redhat.com>
crush: pass crush_map * to various builder methods

In particular, we will need it for crush_calc_straw().

Signed-off-by: Sage Weil <sage@redhat.com>
crush/builder: break out new version 1 of crush_calc_straw

No change, yet.

Signed-off-by: Sage Weil <sage@redhat.com>
crush: fix distortion of straw scalers by 0-weight items

The presence of a 0-weight item in a straw bucket should have no effect
on the placement of other items.  Add a test validating that and fix
crush_calc_straw() to fix the distortion.

Note that this effects the *precalculation* of the straw bucket inputs and
does not effect the actually mapping process given a compiled or encoded
CRUSH map.  Backporting to old code will make the next bucket change
generate (more) correct straw scalers and likely trigger some mapping
changes, but will not lead to inconsistent mappings.

Backport: giant, firefly
Signed-off-by: Sage Weil <sage@redhat.com>
crush: fix crush_calc_straw() scalers when there are duplicate weights

The straw bucket was originally tested with uniform weights and with a
few more complicated patterns, like a stair step (1,2,3,4,5,6,7,8,9).  And
it worked!

However, it does not behave with a pattern like
 1, 2, 2, 3, 3, 4, 4

Strangely, it does behave with
 1, 1, 2, 2, 3, 3, 4, 4

and more usefully it does behave with
 1, 2, 2.001, 3, 3.001, 4, 4.001

That is, the logic that explicitly copes with weights that are duplicates
is broken.

The fix is to simply remove the special handling for duplicate weights --
it isn't necessary and doesn't work correctly anyway.

Add a test that compares the mapping result of  [1, 2, 2, 3, 3, ...] with
[1, 2, 2.001, 3, 3.001, ...] and verifies that the difference is small.
With the fix, we get .00012, whereas the original implementation gets
.015.

Note that this changes the straw bucket scalar *precalculated* values that
are encoded with the map.  The next time the bucket is modified or
recompiled we will start mapping to new values, but the mapping result of
any given (encoded, stored) map will not change.

Backport: giant, firefly
Signed-off-by: Sage Weil <sage@redhat.com>
mon: add 'osd crush {get,set}-tunable <name> [value]' commands

For now, just add the straw_calc_version tunable.

Signed-off-by: Sage Weil <sage@redhat.com>